.png)
Test Script Calculation Scope
Test Script Calculation Scope refers to selecting test run items as the source for calculating a requirement’s status. When this scope is applied, AgileTest evaluates only the linked test run items and their test run item statuses to determine the requirement status.
In AgileTest, there are 6 commonly used Test Run Item statuses, each representing a different run outcome
FAILED : The test run item was executed and produced a result that did not match the expected outcome. An issue was identified and needs investigation or fixing.
PASS : The test run item was executed successfully, and the actual result matched the expected result.
QUERY : The test run item execution result is unclear or requires clarification. Additional information, discussion, or confirmation is needed before a final status can be assigned.
SKIPPED : The test run item was intentionally not executed during a test run, usually due to scope decisions, time constraints, or because it was not applicable.
TODO : The test run item has not been executed yet and is planned to be run.
BLOCKED : The test run item could not be executed due to external dependencies or issues, such as missing test data, environment problems, or unresolved defects.
Based on these Test Run Item statuses, AgileTest calculates the Requirement Status to reflect the overall testing state of each requirement.
Currently, there are 6 types of Requirement Status: NOK; NOT RUN; UNKNOWN; OK; UNCOVERED; COVEREDLet’s explore what each of these Requirement Status types represents and how they are calculated.
1. Apply Test Script Scope:
To apply Test Script Scope for the Requirement Status calculation, follow these steps:
Choose the Test Scripts tab in the Requirement Detail Screen.
Click the “Analysis & Scope” box.
Select the “Test Script” scope.
Click the “Apply” button to calculate the Requirement Status.
2. Requirement Status Explanation
2.1 NOK
NOK means that the requirement is not yet passable within the current testing scope due to one or more failed test run items.
Calculation mechanism:
A requirement has status = NOK
IF: at least 1 linked test run item with the Status = FAILED
Example: This requirement has 2 FAILED test run items, so it has the NOK status, regardless of other test run item statuses.
Other examples:
| Test Run Item 1 | Test Run Item 2 | Test Run Item 3 | Requirement Status |
|---|---|---|---|---|
Scenario 1 | Failed | Pass | Pass | NOK |
Scenario 2 | Skipped | Failed | Todo | NOK |
Scenario 3 | Query | Blocked | Failed | NOK |
2.2 NOT RUN
NOT RUN means that the requirement cannot yet be evaluated because its linked test run items have not been executed.
Calculation mechanism:
A requirement has a status = NOT RUN
IF: at least 1 linked test run item with the status = TODO
AND: no linked test run item with the status = FAILED
Example: This requirement has 1 TODO test run item and does not have any FAILED ones, so it has the NOT RUN status.
Other examples:
| Test Run Item 1 | Test Run Item 2 | Test Run Item 3 | Requirement Status |
|---|---|---|---|---|
Scenario 1 | Query | Pass | Todo | NOT RUN |
Scenario 2 | Skipped | Failed | Todo | NOK |
Scenario 3 | Pass | Todo | Blocked | NOT RUN |
2.3 UNKNOWN
UNKNOWN means that the requirement cannot yet be confirmed within the current testing scope because some linked test run items have neither passed nor failed, making the overall result inconclusive.
Calculation mechanism:
A requirement has a status = UNKNOWN
IF: at least 1 linked test run item with the status = RETEST/ BLOCKED or QUERY
AND: no linked test run item with the status = TODO or FAILED
Example: This requirement has no TODO and FAILED test run items, and it has some test run items with the QUERY and BLOCKED statuses, so it has the UNKNOWN status.
Other examples:
| Test Run Item 1 | Test Run Item 2 | Test Run Item 3 | Requirement Status |
|---|---|---|---|---|
Scenario 1 | Query | Blocked | Pass | UNKNOWN |
Scenario 2 | Pass | Failed | Todo | NOK |
Scenario 3 | Pass | Todo | Pass | NOT RUN |
2.4 OK
OK means that the requirement can be considered successfully validated within the current testing scope because all linked test run items have passed.
Calculation mechanism:
A requirement has a status = OK
IF: all linked test run items have the PASS or SKIPPED status
Example: This requirement has only PASS and SKIPPED test run items, so it has the OK status.
Other examples:
| Test Run Item 1 | Test Run Item 2 | Test Run Item 3 | Requirement Status |
|---|---|---|---|---|
Scenario 1 | Pass | Pass | Skipped | OK |
Scenario 2 | Skipped | Failed | Skipped | NOK |
Scenario 3 | Pass | Skipped | Query | UNKNOWN |
2.5 UNCOVERED
UNCOVERED means that the requirement cannot be evaluated because no test run items are linked to it within the current testing scope.
Calculation mechanism:
A requirement has the status = UNCOVERED
IF: no test run items and test cases are linked to it
AND: no test session items are linked to it
Example: This requirement has no linked test run items, test cases, or test session items, so it has the UNCOVERED status.
Other examples:
| Test Run Item 1 | Test Run Item 2 | Test Run Item 3 | Requirement Status |
|---|---|---|---|---|
Scenario 1 | - | - | - | UNCOVERED |
Scenario 2 | - | - | Todo | NOT RUN |
Scenario 3 | - | Skipped | - | OK |
2.6 COVERED
COVERED means that the requirement is only linked with Exploratory tests.
Calculation mechanism:
A requirement gets the status = COVERED
IF: at least 1 test session linked to it
AND: no test cases, test run items are linked to the requirement.
Example: This requirement is only linked with one test session, so it has the status COVERED
Should you need any assistance or further AgileTest inquiries, contact our 24/7 Support Desk!