Bitbucket is one of the widely known CI/CD tool. In this document, we will show you how to integrate AgileTest with Bitbucket via REST API.
However, in order to utilize AgileTest API, Client id and Client secret are needed. Please refer to this instruction to get them first Access API documentation.
For Data center version, you will need a Personal access token (PAT) instead. In Jira, select ⚙️ (top right corner) → System -> Administeringpersonalaccess tokens, and create yourself a token. Please refer to this article from Jira for more details.
Make sure your account has permission to create TestCase and TestExecution issues; otherwise, the import will fail.
Setup your Bitbucket repository
In your Bitbucket repository, go to Repository settings → Repository variables and create respective repository variables for acquired client id and client secret.
In this example, we will create 5 repository variables CLIENT_ID, CLIENT_SECRET, PROJECT_KEY, BASE_AUTH_URL, and BASE_URL.
Repository variables
Variables
Description
Is secured?
Example
BASE_AUTH_URL
Base URL used for requesting authenticating token
https://agiletest.atlas.devsamurai.com
BASE_URL
Base URL to submit your report to
https://api.agiletest.app
CLIENT_ID
The client id that you have requested earlier
******
CLIENT_SECRET
The client secret that you have requested earlier
******
PROJECT_KEY
Your project key
RKE
However, in case you are working with AgileTest Data Center version, you only need 3 variables as follows.
Variables
Description
Is secured?
Example
DC_TOKEN
Your Personal access token
BASE_URL
Base URL to submit your report to
PROJECT_KEY
Your project key
RKE
Upload your test result
Here is a sample NUnit report to play around with.
If you need more detailed examples, please refer to our public repository on AgileTest GitHub for additional sample projects.
Use API
Below is a sample bitbucket-pipelines.yml file.
AgileTest Cloud - YAML file
AgileTest Cloud - bitbucket-pipelines.yml
YAML
image: mcr.microsoft.com/dotnet/sdk
options:
docker: true
definitions:
steps:
- step: &linting
name: Linting
caches:
- dotnetcore
script:
- echo "linting"
pipelines:
pull-requests:
develop:
- step: *linting
master:
- step: *linting
# TODO: build docker image
branches:
nunit:
# - step: *linting
- step: &run-test
name: Test and upload report
script:
# Run test
- cd ./Nunit/PrimeService && dotnet restore
- cd .. && dotnet sln add PrimeService/PrimeService.csproj
- cd ./PrimeService.Tests && dotnet restore
- cd .. && dotnet sln add PrimeService.Tests/PrimeService.Tests.csproj
- dotnet test --logger:"nunit;LogFilePath=test-result.xml"
# Upload the report
- export token=$(curl "$BASE_AUTH_URL/api/apikeys/authenticate" -X POST -H 'Content-Type:application/json' --data '{"clientId":"'"$CLIENT_ID"'","clientSecret":"'"$CLIENT_SECRET"'"}' | tr -d '"')
- curl -X POST "$BASE_URL/ds/test-executions/nunit?projectKey=$PROJECT_KEY&testExecutionKey=$TEST_EXECUTION_KEY" -H "Content-Type:application/xml" -H "Authorization:JWT $token" --data-binary "@PrimeService.Tests/test-result.xml"
In this file, we use 2 endpoints
api/v1/apikeys/authenticate (for Cloud only) to get temporary token using acquired client_id and client_secret.
ds/test-executions/nunit to submit test report to Agile Test so that the application could create or update Test execution and Test cases accordingly.
To learn more about these endpoints, please refer to this document API document.
AgileTest Data center - YAML file
For Data center version, we only use 1 endpoint ds/test-executions/nunit with DC_TOKEN variable instead of the token that is requested using client id and client secret. Let’s take a look at the file below!
AgileTest Data center - bitbucket-pipelines.yml
YAML
image: mcr.microsoft.com/dotnet/sdk
options:
docker: true
definitions:
steps:
- step: &linting
name: Linting
caches:
- dotnetcore
script:
- echo "linting"
pipelines:
pull-requests:
develop:
- step: *linting
master:
- step: *linting
# TODO: build docker image
branches:
nunit:
# - step: *linting
- step: &run-test
name: Test and upload report
script:
# Run test
- cd ./Nunit/PrimeService && dotnet restore
- cd .. && dotnet sln add PrimeService/PrimeService.csproj
- cd ./PrimeService.Tests && dotnet restore
- cd .. && dotnet sln add PrimeService.Tests/PrimeService.Tests.csproj
- dotnet test --logger:"nunit;LogFilePath=test-result.xml"
# Upload the report
- curl -X POST "$BASE_URL/ds/test-executions/nunit?projectKey=$PROJECT_KEY&testExecutionKey=$TEST_EXECUTION_KEY" -H "Content-Type:application/xml" -H "Authorization:Bearer $DC_TOKEN" --data-binary "@PrimeService.Tests/test-result.xml"
The file sets up a two-step process: run-test and upload-report. These steps run with different container images – run-test uses a .NET image, and upload-report uses a specialized image for the Agile Test CLI.
In run-teststep, we run the automation test scripts and the process yields the report. The report is then pushed to pipeline artifact in order to be used for the later step. Next, in upload-report step, the report is uploaded to AgileTest by using AgileTest CLI. Below is the command used for the task.
Same as Cloud, in this file we also have 2 steps, run-test and upload-report. However, in run-test step, the command would look a little bit different 😉 .
For each test run, a new Test Execution will be created along with linked Test Cases. However, if the Test Cases already exist in your project, Agile Test will only generate a new Test Execution and link the matching Test Cases to it, including the test results
View Pipeline Status
AgileTest provides a simple way to track the status of your pipeline after it has been triggered. To enable this feature, just add the following directives to your workflow file.
For Cloud version, we use the endpoint test-executions/{test_execution_id}/pipeline/history
However, if you are working with AgileTest Data center version, please use rest/agiletest/1.0/test-executions/{test_execution_id}/pipeline/history instead.
Your file should look like this now.
YAML file that includes pipeline status update call
YAML
name: Agile Test Demo
# This workflow represents a set of basic End-to-End tests
on:
workflow_dispatch:
env:
BASE_URL: ${{ vars.BASE_URL }}
DC_TOKEN: ${{secrets.DC_TOKEN}}
PROJECT_KEY: ${{ vars.PROJECT_KEY }}
jobs:
test:
name: run test
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Set up JDK 17
uses: actions/setup-java@v4
with:
java-version: "17"
distribution: "temurin"
cache: maven
- name: Build with Maven
run: mvn -B package --file pom.xml
- name: Submit test result to Agile Test
run: |
docker run --rm \
-e AGILETEST_BASE_URL=$BASE_URL \
-e AGILETEST_DC_TOKEN=$DC_TOKEN \
-v ${{ github.workspace }}/target/surefire-reports:/reports \
ghcr.io/agiletestapp/agiletest-cli:latest \
--data-center \
test-execution import --framework-type junit --project-key $PROJECT_KEY \
/reports/TEST-calculateTest.xml
- name: After-Run Script
if: always()
run: |
after-script:
- if [ "$BITBUCKET_EXIT_CODE" -eq 0 ]; then
RESULT="success";
else
RESULT="failed";
fi
- export token=$(curl "$BASE_AUTH_URL/api/apikeys/authenticate" -X POST -H 'Content-Type:application/json' --data '{"clientId":"'"$CLIENT_ID"'","clientSecret":"'"$CLIENT_SECRET"'"}' | tr -d '"')
- |
curl "$BASE_AUTH_URL/ds/test-executions/$TEST_EXECUTION_KEY/pipeline/history?projectKey=$PROJECT_KEY" -X POST -H "Content-Type: application/json" -H "Authorization: JWT $token" --data '{ "buildNumber": '"$BITBUCKET_BUILD_NUMBER"', "tool": "'"bitbucket"'", "repoSlug": "'"$BITBUCKET_REPO_SLUG"'", "workspace": "'"$BITBUCKET_WORKSPACE"'", "result": "'"$RESULT"'" }'
If you include this part to your YAML file, on pipeline’s completion, a history log with pipeline status will be added on the right panel of the execution issue screen as below.
History log on pipeline status
Should you need any assistance or further AgileTest inquiries, contact here!
JavaScript errors detected
Please note, these errors can depend on your browser setup.
If this problem persists, please contact our support.
