Behave JSON result

There are 2 approaches to importing Behave JSON Result files using the API:

This Behave framework supports only Gherkin test cases. Test step details for other types will be excluded from the parsed results.

1. Import Test Result only

This approach allows you to quickly upload test results to AgileTest without needing to include extra metadata like test execution timestamps, environment details, or other specific execution parameters.

You can locate the endpoint at the POST /ds/test-executions/behave in the API document.

When you click on the 🔽 button, a dialog will appear. Here you can input your information for the test result file upload.

There are 2 main sections:

Parameters: fields that you define as key parameters required to make a successful API request. These parameters include:

Parameters	Type	Required	Description
projectKey	string	✅	Key of the project where the test results will be imported.
milestoneId	integer		Milestone id to link to Test execution.
testExecutionKey	string		Test execution key to import test report.
testPlanKeys	array[string]		Test plan key to import test report. If you specify a Test Plan, the tests will automatically be added to it if they are not already included.
testEnvironments	array[string]		A string containing a list of test environments separated by ",".
fixVersions	array[string]		A string containing a list of fixVersions separated by “,”.
revision	string		Revision to link to Test execution.

Request body: where you can input a JSON file, which is a Behave report that is going to be imported.

Test result example

CODE

[
{
  "elements": [
    {
      "keyword": "Scenario",
      "location": "features/shopping.feature:5",
      "name": "Add item to cart",
      "status": "passed",
      "steps": [
        {
          "keyword": "Given",
          "location": "features/shopping.feature:6",
          "match": {
            "arguments": [
              {
                "name": "product_name",
                "value": "Laptop"
              },
              {
                "name": "price",
                "value": "1000"
              }
            ],
            "location": "features/steps/shopping_steps.py:5"
          },
          "name": "I have a product named \"Laptop\" priced at 1000",
          "result": {
            "duration": 7.605552673339844e-05,
            "status": "passed"
          },
          "step_type": "given"
        },
        {
          "keyword": "When",
          "location": "features/shopping.feature:7",
          "match": {
            "arguments": [],
            "location": "features/steps/shopping_steps.py:10"
          },
          "name": "I add the product to my cart",
          "result": {
            "duration": 2.09808349609375e-05,
            "status": "passed"
          },
          "step_type": "when"
        },
        {
          "keyword": "Then",
          "location": "features/shopping.feature:8",
          "match": {
            "arguments": [
              {
                "name": "total",
                "value": "1000"
              }
            ],
            "location": "features/steps/shopping_steps.py:14"
          },
          "name": "my cart total should be 1000",
          "result": {
            "duration": 1.8835067749023438e-05,
            "status": "passed"
          },
          "step_type": "then"
        }
      ],
      "tags": [
        "ATP-1858"
      ],
      "type": "scenario"
    },
    {
      "keyword": "Scenario",
      "location": "features/shopping.feature:11",
      "name": "Checkout cart",
      "status": "passed",
      "steps": [
        {
          "keyword": "Given",
          "location": "features/shopping.feature:12",
          "match": {
            "arguments": [
              {
                "name": "total",
                "value": "1000"
              }
            ],
            "location": "features/steps/shopping_steps.py:18"
          },
          "name": "I have a cart with a total of 1000",
          "result": {
            "duration": 2.6941299438476562e-05,
            "status": "passed"
          },
          "step_type": "given"
        },
        {
          "keyword": "When",
          "location": "features/shopping.feature:13",
          "match": {
            "arguments": [],
            "location": "features/steps/shopping_steps.py:25"
          },
          "name": "I checkout",
          "result": {
            "duration": 1.811981201171875e-05,
            "status": "passed"
          },
          "step_type": "when"
        },
        {
          "keyword": "Then",
          "location": "features/shopping.feature:14",
          "match": {
            "arguments": [],
            "location": "features/steps/shopping_steps.py:29"
          },
          "name": "my cart should be empty",
          "result": {
            "duration": 1.6927719116210938e-05,
            "status": "passed"
          },
          "step_type": "then"
        }
      ],
      "tags": [
        "ATP-1857"
      ],
      "type": "scenario"
    },
    {
      "keyword": "Scenario",
      "location": "features/shopping.feature:17",
      "name": "Add multiple items to cart",
      "status": "passed",
      "steps": [
        {
          "keyword": "Given",
          "location": "features/shopping.feature:18",
          "match": {
            "arguments": [
              {
                "name": "product1",
                "value": "Laptop"
              },
              {
                "name": "price1",
                "value": "1000"
              },
              {
                "name": "product2",
                "value": "Phone"
              },
              {
                "name": "price2",
                "value": "500"
              }
            ],
            "location": "features/steps/shopping_steps.py:33"
          },
          "name": "I have products named \"Laptop\" priced at 1000 and \"Phone\" priced at 500",
          "result": {
            "duration": 3.695487976074219e-05,
            "status": "passed"
          },
          "step_type": "given"
        },
        {
          "keyword": "When",
          "location": "features/shopping.feature:19",
          "match": {
            "arguments": [],
            "location": "features/steps/shopping_steps.py:39"
          },
          "name": "I add both products to my cart",
          "result": {
            "duration": 1.7881393432617188e-05,
            "status": "passed"
          },
          "step_type": "when"
        },
        {
          "keyword": "Then",
          "location": "features/shopping.feature:20",
          "match": {
            "arguments": [
              {
                "name": "total",
                "value": "1500"
              }
            ],
            "location": "features/steps/shopping_steps.py:14"
          },
          "name": "my cart total should be 1500",
          "result": {
            "duration": 1.71661376953125e-05,
            "status": "passed"
          },
          "step_type": "then"
        }
      ],
      "tags": [
        "ATP-1856"
      ],
      "type": "scenario"
    },
    {
      "keyword": "Scenario",
      "location": "features/shopping.feature:23",
      "name": "Apply discount to cart",
      "status": "passed",
      "steps": [
        {
          "keyword": "Given",
          "location": "features/shopping.feature:24",
          "match": {
            "arguments": [
              {
                "name": "total",
                "value": "1000"
              }
            ],
            "location": "features/steps/shopping_steps.py:18"
          },
          "name": "I have a cart with a total of 1000",
          "result": {
            "duration": 2.5033950805664062e-05,
            "status": "passed"
          },
          "step_type": "given"
        },
        {
          "keyword": "When",
          "location": "features/shopping.feature:25",
          "match": {
            "arguments": [
              {
                "name": "amount",
                "value": "100"
              }
            ],
            "location": "features/steps/shopping_steps.py:44"
          },
          "name": "I apply a discount of 100",
          "result": {
            "duration": 1.7881393432617188e-05,
            "status": "passed"
          },
          "step_type": "when"
        },
        {
          "keyword": "Then",
          "location": "features/shopping.feature:26",
          "match": {
            "arguments": [
              {
                "name": "total",
                "value": "900"
              }
            ],
            "location": "features/steps/shopping_steps.py:14"
          },
          "name": "my cart total should be 900",
          "result": {
            "duration": 1.7881393432617188e-05,
            "status": "passed"
          },
          "step_type": "then"
        }
      ],
      "tags": [
        "ATP-1855"
      ],
      "type": "scenario"
    },
    {
      "keyword": "Scenario",
      "location": "features/shopping.feature:29",
      "name": "Attempt to checkout with insufficient funds",
      "status": "passed",
      "steps": [
        {
          "keyword": "Given",
          "location": "features/shopping.feature:30",
          "match": {
            "arguments": [
              {
                "name": "total",
                "value": "1000"
              }
            ],
            "location": "features/steps/shopping_steps.py:18"
          },
          "name": "I have a cart with a total of 1000",
          "result": {
            "duration": 2.47955322265625e-05,
            "status": "passed"
          },
          "step_type": "given"
        },
        {
          "keyword": "And",
          "location": "features/shopping.feature:31",
          "match": {
            "arguments": [
              {
                "name": "amount",
                "value": "500"
              }
            ],
            "location": "features/steps/shopping_steps.py:48"
          },
          "name": "I have only 500 in my account",
          "result": {
            "duration": 3.2901763916015625e-05,
            "status": "passed"
          },
          "step_type": "given"
        },
        {
          "keyword": "When",
          "location": "features/shopping.feature:32",
          "match": {
            "arguments": [],
            "location": "features/steps/shopping_steps.py:53"
          },
          "name": "I checkout multiple products",
          "result": {
            "duration": 2.5033950805664062e-05,
            "status": "passed"
          },
          "step_type": "when"
        },
        {
          "keyword": "Then",
          "location": "features/shopping.feature:33",
          "match": {
            "arguments": [
              {
                "name": "message",
                "value": "Insufficient funds"
              }
            ],
            "location": "features/steps/shopping_steps.py:60"
          },
          "name": "I should see an error message \"Insufficient funds\"",
          "result": {
            "duration": 1.8835067749023438e-05,
            "status": "passed"
          },
          "step_type": "then"
        }
      ],
      "tags": [
        "ATP-1854"
      ],
      "type": "scenario"
    }
  ],
  "keyword": "Feature",
  "location": "features/shopping.feature:2",
  "name": "Shopping Cart",
  "status": "passed",
  "tags": [
    "ATP-1856"
  ]
}
]

Learn more about the Behave testing framework.

Then, you will receive an output example

Output example

CODE

{
    "id": "321016",
    "key": "ATP-1856",
    "url": "https://company.atlassian.net/browse/ATP-1856"
}

In case you want to use the command line, file pipeline, and other tools instead of API docs, you can copy the curl and input the token to import test results.

CODE

curl -X 'POST' \
  'https://api.agiletest.app/ds/test-executions/behave?projectKey=ATP' \
  -H 'accept: application/json' \
  -H 'Authorization: JWT <token>' \
  -H 'Content-Type: application/json' \
  --data-binary '@my-report.json'

In that, <token> is the result from the Get Authenticated step.

2. Import Test Result with New Execution Details

This approach allows you to import Behave JSON test results while including additional execution details, such as timestamps, environment details, test execution keys, and other specific parameters related to the test executions.

You can locate the endpoint at the POST /ds/test-executions/behave/multipart in the API document.

For this request, it does not require any parameters. Instead, you need to submit two separate files:

A file containing the test results to import. This file resembles the test result files used in the Import Test Result only approach.
A file containing the test execution details. This file has some fields, and each of these fields serves a specific purpose in creating and organizing the test execution:
- The fields object contains standard issue creation information, like what project it belongs to and what type of issue it is. It has 3 compulsory fields summary, project.id, and issuetype.id. Additionally, you can include other optional Jira fields, such as labels, priority, or any custom fields that your Jira instance uses.
- The agileTestFields object contains Agile Test-specific information for linking the execution to milestones and test plans, both of which are optional.

Field	Group	Type	Required	Description
summary	fields	string	✅	Summary of Test execution issue that will be created.
project.id		string	✅	Project ID in which the Test execution will be created.
issuetype.id		string	✅	The identifier specifying the type of issue to be created.
…		…		…
milestoneId	agileTestFields	integer		The identifier of the milestone that this test execution should be associated with.
testPlanKeys		array of string		An array of issue keys of Test plan that this test execution is related to.
testEnvironments		array of string		A string containing a list of test environments separated by ",".
fixVersions		array of string		A string containing a list of fixVersions separated by “,”.
revision		string		Revision to link to Test execution.

Test Execution detail example

CODE

{
  "fields": {
    "summary": "Regression test",
    "project": {
      "id": "17110"
    },
    "issuetype": {
      "id": "16396"
    }
  },
  "agileTestFields": {
    "milestoneId": 5555,
    "testPlanKeys": ["ATP-364", "ATP-362"]
  }
}

Make sure all details provided are accurate and relevant; otherwise, the request will fail. For example, if the issuetype.id you specify a value that does not exist in the project with ID 17110, the system will return an error with the corresponding error code 400.

Then, you will receive an output example

Output example

CODE

{
  "id": "321348",
  "key": "ATP-1859",
  "url": "https://company.atlassian.net/browse/ATP-1859"
}

In case you want to use the command line, file pipeline, and other tools instead of API docs, you can copy the curl and input the token to import test results.

CODE

curl -X 'POST' \
  'https://api.agiletest.app/ds/test-executions/behave/multipart' \
  -H 'accept: application/json' \
  -H 'Authorization: JWT <token>' \
  -H 'Content-Type: multipart/form-data' \
  -F 'results=@my-report.json;type=application/json' \
  -F 'testExecution=@test-execution.json;type=application/json'

In that, <token> is the result from the Get Authenticated step.

Should you need any assistance or further AgileTest inquiries, contact our 24/7 Support Desk!