JUnit XML report
About JUnit
JUnit is a unit testing open-source framework for the Java programming language. Java Developers use this framework to write and execute automated tests. In Java, there are test cases that have to be re-executed every time a new code is added. Though this framework has been implemented originally for Java, its XML report formatted file could be generated by many test frameworks from different programming languages.
Common structure
In a basic JUnit report, you should find common tags depicting test suites and test cases.
To visualize the structure more clearly, let’s take a look at the diagram below.
-20241121-075433.jpeg?inst-v=e3835335-d0be-4f4c-a82c-c62b3d80caa9)
JUnit test report structure diagram
How AgileTest would parse a JUnit report
In AgileTest, all test cases imported using the automation feature are categorized as Generic Tests.
Imported test cases are distinguished by the classname and name attributes parsed from the XML file. The combination of these attributes is referred to as the Test Definition in AgileTest. Users can find the Test Definition located just below the Description area of the test case tickets. The Summary of each test case issue corresponds to the value of the name attribute.
Supported status tags
Test run status tag | Description |
|---|---|
| PASSED → Test case yields the expected result. |
| FAILED → Test case returns the actual result not as expected. |
| BLOCKED → Test case is considered blocked due to a specific reason. |
| SKIPPED → Test case is skipped in the test run. |
| RETEST → Test case should be rerun again. |
| QUERY → Test case requires more details to have the final result. |
| TODO → Test case is not yet run. It’s the default status of every Test case in AgileTest. |
The priority of status tags in AgileTest follows this order: failed > blocked > retest > skipped > query > passed (or no tag).
If multiple tags are added to a single <testcase> element, the tag with the highest priority will be parsed as the test status in AgileTest.
Test result without steps
Test suite is a group of Test cases. Many Test suites are grouped inside a <testsuites> element.
The context information will be reflected, each row shows result of each time the script rerun the test. Let’s take a look at the example below.
The result will be displayed in Context section.
Context name is constituted by “TestSuite” and name attribute of
<testsuite>element.Test run status will be parsed accordingly following the mentioned rule.
Duration is the duration value of
<testcase>element.Status is the status value of
<testcase>element.
JUnit report example
<testsuites id="" name="" tests="2" failures="1" skipped="0" errors="0" time="6.707531">
<testsuite name="check context" timestamp="2024-01-23T03:45:35.369Z" hostname="chromium" tests="2" failures="1" skipped="0" time="7.768" errors="0">
<testcase name="Login with common user's credentials" classname="example.spec.ts" time="6.161">
</testcase>
<testcase name="Add items to cart" classname="example.spec.ts" time="1.607">
<failure message="Assertion error message" type="AssertionError">
<!-- Call stack printed here -->
</failure>
</testcase>
</testsuite>
</testsuites>
Context information parsed from test report
Test result with steps
As you may know, the default JUnit report format does not support test steps. However, the community has made customizations using the <property> element to store test steps in the report. AgileTest also recognizes and supports this customization for better test result management.
Elements would be parsed similarly to Test result without steps, but with some enhancements.
Context name is constituted by “TestSuite” and name attribute of
<testsuite>element.Test run status will be parsed accordingly following the mentioned rule.
Duration is the duration value of
<testcase>element.Step status is based on the text inside the square brackets (case insensitive)
Test step status property | Description |
|---|---|
| PASSED → Test step returns the expected outcome. |
| FAILED → Test step yields wrong result. |
| BLOCKED → Test step is blocked due to a certain reason. |
| SKIPPED → Test step is skipped. |
| RETEST → Test step is due to rerun again. |
| QUERY → Test step requires more information. |
| TODO → Test step is marked to do, which is the default status of step status. |
Test result with attachments
Attachments are also part of the customization supported in AgileTest. However, unlike test steps, attachments are included in the test report as elements (or tags).
There are 3 ways to include attachments in your test report:
1. Default attachment structure of JUnit
Some frameworks generate attachments and store them in the test report with the structure as below.
Element structure
<system-out>
<![CDATA[
[[ATTACHMENT|file path or base64 content]]
]]>
</system-out>Example
<testcase>
<system-out>
<![CDATA[
[[ATTACHMENT|test-failed-1.png]]
]]>
</system-out>
</testcase>2. Attachment element with name attribute and content inside
Element structure
<attachment name="filename"> base64 content </attachment>Example
<testcase>
....
<attachments>
<attachment name="test.png">iVBORw0KGgoAAAANSUhEUgAAADIAAAAyCAYAAAAeP4ixAAAAQ0lEQVR42u3PQREAAAgDINc/9Mzg14MGZNrOAxERERERERERERERERERERERERERERERERERERERERERERERERERuVh9TJWdbWjzsQAAAABJRU5ErkJggg==</attachment>
</attachments>
</testcase>3. Attachment element with separate name and content elements inside
Element structure
<attachment>
<name>filename</name>
<rawcontent> base64 content </rawcontent>
</attachment>Example
<testcase>
....
<attachments>
<attachment>
<name>test.png</name>
<rawcontent>iVBORw0KGgoAAAANSUhEUgAAADIAAAAyCAYAAAAeP4ixAAAAQ0lEQVR42u3PQREAAAgDINc/9Mzg14MGZNrOAxERERERERERERERERERERERERERERERERERERERERERERERERERuVh9TJWdbWjzsQAAAABJRU5ErkJggg==</rawcontent>
</attachment>
</attachments>
</testcase>
Should you need any assistance or further AgileTest inquiries, please contact us here!