Cucumber & Behave JSON report

About Cucumber

Cucumber and Behave are testing frameworks that support Behavior-Driven Development (BDD), enabling users to write tests in plain language using Gherkin syntax. This approach makes tests easy to read and understand for all team members, regardless of their technical background.

Common structure

Since both frameworks are similar in many ways, their report also hold much resemblance in their structures. Both types of report contain basic blocks, namely Feature, Scenario, Given, When, and Then.

Cucumber report example

JSON

[
  {
    "uri": "features/sample.feature",
    "id": "sample-feature",
    "keyword": "Feature",
    "name": "Sample Feature",
    "description": "This is a sample feature file.",
    "line": 1,
    "elements": [
      {
        "id": "sample-feature;sample-scenario",
        "keyword": "Scenario",
        "name": "Sample Scenario",
        "description": "",
        "line": 3,
        "type": "scenario",
        "steps": [
          {
            "keyword": "Given ",
            "name": "I have a Cucumber test",
            "line": 4,
            "match": {
              "location": "sample_steps.rb:10"
            },
            "result": {
              "status": "passed",
              "duration": 1000000
            }
          },
          {
            "keyword": "When ",
            "name": "I run the test",
            "line": 5,
            "match": {
              "location": "sample_steps.rb:15"
            },
            "result": {
              "status": "passed",
              "duration": 2000000
            }
          },
          {
            "keyword": "Then ",
            "name": "I should see a report",
            "line": 6,
            "match": {
              "location": "sample_steps.rb:20"
            },
            "result": {
              "status": "failed",
              "error_message": "Expected report to be visible",
              "duration": 1500000
            }
          }
        ]
      }
    ]
  }
]

Click here to expand...

CODE

[
  {
    "keyword": "Feature",
    "name": "Sample Feature",
    "description": "This is a sample feature.",
    "status": "passed",
    "elements": [
      {
        "keyword": "Scenario",
        "name": "Sample Scenario",
        "description": "",
        "tags": ["@tag1"],
        "steps": [
          {
            "keyword": "Given ",
            "step_type": "given",
            "name": "I have a Behave test",
            "status": "passed",
            "duration": 0.001
          },
          {
            "keyword": "When ",
            "step_type": "when",
            "name": "I run the test",
            "status": "passed",
            "duration": 0.002
          },
          {
            "keyword": "Then ",
            "step_type": "then",
            "name": "I should see a report",
            "status": "failed",
            "error_message": "Expected to see a report but did not.",
            "duration": 0.001
          }
        ]
      }
    ]
  }
]

To visualize the structure more clearly, let’s take a look at the diagram below.

image (2)-20241211-190211.png — Cucumber & Behave structure diagram

How Agile Test would parse a Cucumber or Behave report

For Cucumber and Behave, users must include the issue key in the report before uploading it to AgileTest. Otherwise, the import will fail.

Since Agile Test doesn’t create new tests in case of importing Gherkin reports, only Test run result and attachments will be parsed and attached to pre-existing tests in your instance.

Supported status tags

Step status	Description
`pass`, `passed` or no tag	PASSED → Test case yields the expected result.
`failed`, `failure` , `fail`, or `error`	FAILED → Test case returns the actual result not as expected.
`blocked`	BLOCKED → Test case is considered blocked due to a specific reason.
`skipped`	SKIPPED → Test case is skipped in the test run.
`retest`	RETEST → Test case should be rerun again.
`query`	QUERY → The test case requires more details to have the final result.
`todo`, `to do`, or non-defined statuses	TODO → Test case is not yet run. It’s the default status of every Test case in Agile Test.

Test result with attachments

Element structure

JSON

 "embeddings": [
              {
                "data": "<BASE64_ENCODED_FILE>",
                "mime_type": "<mime type>"
              }
            ]

Example

JSON

[
  {
    "elements": [
      {
        ...
        "steps": [
          {
            ...
            "embeddings": [
              {
                "data": "SGVsbG8gd29ybGQ=",
                "mime_type": "text/plain"
              }
            ]
          }
        ]
      }
    ]
  }
]

Should you need any assistance or further AgileTest inquiries, please contact us here!