Test Step Status Calculation

Table of contents:

The test step status provides insights into its current combined state of the test case’s execution.

AgileTest provides some native Test Step Statuses (cannot be modified no deleted):

To Do: The Test Step is in pending execution state (this is a non-final status)
Pass: The Test Step passed successfully
Fail: The Test Step failed

1. Manage Test Step Status

Jira Admin can modify, create and edit additional Test Steps Statuses in Jira Admin: Manage Test Step Statuses.

When creating/editing a Test Step Status, it is necessary to identify the Map Status (Steps Running Status) to which the Test Step Status should be mapped.

The "final" attribute in a Test Step's status is essential. Final statuses are given priority over non-final , ensuring that AgileTest consistently focuses on final outcomes (e.g., PASS, FAIL) during the calculation of a Test Step's status. If a Test Step is in a final status and a new Test Execution is initiated, this new execution will not affect the Test Step's overall status.

The rank of the final statuses: Pass < TO DO < FAIL

This approach is beneficial for focusing exclusively on the most recent definitive result and excluding intermediate Test Executions in statuses like TO DO.

2. Calculation of The Given Test Step Status

A Test Execution's status is usually auto-calculated based on individual test step statuses, but a specific status can be enforced for a Test Execution, potentially setting all steps to that status (e.g., marking a Test Execution as "PASS” sets all steps to "PASS").

The calculation follows these steps:

Map each test step status to its corresponding test status.
Compare all mapped test statuses.
1. If both status is PASS then the overall Test Execution will be that status
2. A FAIL status in any step, sets the Test Execution status to FAIL
3. If all steps are PASS but one step is not yet completed, the the overall status is TO DO
4. The most recently created status is presumably the highest-ranked status (except mapping with PASS and TO DO)
5. The highest-ranked status among these becomes the Test Execution status (except FAIL since FAIL ultimately determines the overall execution status )

The order of steps does not impact the overall status of the Test Execution.

2.1. Consequences:

Only PASS and TO DO status can force test step status.
Only to do step can be forced by FAIL run.
FAIL step will take over test run status.
If all step is the same status then the overall Test Execution status will be that status.

2.2. Example:

The table below presents examples based on the above Test Step Statuses Calculation.

Examples	Status of the steps name (Name) (The order is irrelevant)	Calculated value of the steps names (Map Status)	Explanation
1	PASS TO DO PASS	TO DO	The overall has at least test step to be completed yet so the run is marked To Do
2	PASS PASS PASS	PASS	All steps have the same value (marked as PASS) => Leading to an overall status of PASS.
3	PASS FAIL TO DO	FAIL	One of the step statuses holds a higher rank compared to the others (Pass < TO DO < FAIL)
4	SKIPPED [2] TO DO PASS BLOCKED QUERY	SKIPPED [2]	The most recently created status is presumably the highest-ranked status (Skipped)
5	BLOCKED RETEST SKIPPED [1]	RETEST	The highest rank determines the overall status (RETEST is the status that was created most recently compared to Blocked and Skipped)
6	PASS PASS QUERY BLOCKED FAIL RETEST	FAIL	The highest rank determines the overall status (Typically, Retest should be the highest rank but available Fail ultimately determines the run status)

Should you need any assistance or further AgileTest inquiries, contact here!