Robot framework XML report

About Robot framework

Robot is a testing framework inspired from JUnit and NUnit but introducing some new functionalities that make it more powerful and easier to use, such as annotations, flexible test configurations, support for data-driven testing, and so much more.

Common structure

In a basic TestNG report, you should find common tags depicting test suites and test cases.

Robot report example

XML

<?xml version="1.0" encoding="UTF-8"?>
<robot generator="Robot Framework 6.1.1" generated="20241210 10:00:00">
  <suite name="SampleTestSuite" source="SampleTestSuite.robot">
    <status status="FAIL" starttime="20241210 09:58:00.000" endtime="20241210 10:00:00.000">
      <message>Test suite failed due to test case failures.</message>
    </status>
    <test name="Test Case 1">
      <status status="PASS" starttime="20241210 09:58:10.000" endtime="20241210 09:58:20.000"/>
    </test>
    <test name="Test Case 2">
      <status status="FAIL" starttime="20241210 09:58:30.000" endtime="20241210 09:58:40.000">
        <message>Error: Expected value '42', but got '41'.</message>
      </status>
    </test>
    <test name="Test Case 3">
      <status status="SKIP" starttime="20241210 09:59:00.000" endtime="20241210 09:59:10.000">
        <message>Test skipped due to a precondition failure.</message>
      </status>
    </test>
  </suite>
  <statistics>
    <total>
      <stat pass="1" fail="1" skip="1" total="3"/>
    </total>
    <tag>
      <stat name="Critical" pass="1" fail="1" skip="1"/>
    </tag>
  </statistics>
  <errors>
    <msg timestamp="20241210 09:58:50.000">
      Some setup steps failed, causing subsequent tests to fail or skip.
    </msg>
  </errors>
</robot>

To visualize the structure more clearly, let’s take a look at the diagram below.

image (10)-20241211-034333.png — Robot test report structure diagram

How Agile Test would parse a Robot report

In AgileTest, all test cases imported using the automation feature are classified as Generic Tests.

Imported test cases are distinguished by the name attributes of the <suite> and <test> elements, following the format: name of <suite> element + '.' + name of <test> element.

This combination is referred to as the Test Definition in Agile Test and can be found just below the Description area of the test case ticket. The Summary of each test case issue is based on the value of the name attribute.

Supported status attribute

In Robot XML report, executed test cases are marked with <status> elements containing status attribute inside blocks of <kw>.

XML

<kw name="Open Browser" library="SeleniumLibrary">
  <arguments>
    <arg>https://example.com</arg>
    <arg>Chrome</arg>
  </arguments>
  <status status="PASS" starttime="20241220 10:01:05.000" endtime="20241220 10:01:10.000"/>
</kw>

Test case’s status will be calculated based on step statuses.

Test run status attribute	Description
`PASS`, `PASSED`	PASSED → Test case yields the expected result.
`FAIL`, `FAILED`, or `ERROR`	FAILED → Test case returns the actual result not as expected.
`BLOCKED`	BLOCKED → Test case is considered blocked due to a specific reason.
`SKIPPED`	SKIPPED → Test case is skipped in the test run.
`RETEST`	RETEST → Test case should be rerun again.
`QUERY`	QUERY → The test case requires more details to have the final result.
`TODO`, `TO DO`, non-defined statuses, or no status attribute	TODO → Test case is not yet run. It’s the default status of every Test case in Agile Test.

The priority of status tags in Agile Test is as follows:

failed > blocked > retest > skipped > query > passed (or no tag).

If multiple tags are added to a single <test> element, the tag with the highest priority will be used as the test status in Agile Test.

Test result with attachments

Agile Test supports 2 customized structures for integrating attachments into your test reports. Users can choose either option to upload test results along with evidence files.

1. Attachment element with name attribute and content inside

Element structure

CODE

<attachment name="filename"> base64 content </attachment>

Example

CODE

<test>
  ....
  <attachments>
    <attachment name="test.png">iVBORw0KGgoAAAANSUhEUgAAADIAAAAyCAYAAAAeP4ixAAAAQ0lEQVR42u3PQREAAAgDINc/9Mzg14MGZNrOAxERERERERERERERERERERERERERERERERERERERERERERERERERuVh9TJWdbWjzsQAAAABJRU5ErkJggg==</attachment>
  </attachments>
</test>

2. Attachment element with separate name and content elements inside

Element structure

CODE

<attachment> 
  <name>filename</name>
  <rawcontent> base64 content </rawcontent>
</attachment>

Example

CODE

<test>
  ....
  <attachments>
    <attachment> 
      <name>test.png</name>
      <rawcontent>iVBORw0KGgoAAAANSUhEUgAAADIAAAAyCAYAAAAeP4ixAAAAQ0lEQVR42u3PQREAAAgDINc/9Mzg14MGZNrOAxERERERERERERERERERERERERERERERERERERERERERERERERERuVh9TJWdbWjzsQAAAABJRU5ErkJggg==</rawcontent>
    </attachment>
  </attachments>
</test>

Should you need any assistance or further AgileTest inquiries, please contact us here!