TeamCity is a powerful Continuous Integration/Continuous Deployment (CI/CD) server developed by JetBrains. It automates software build, test, deployment, and reporting processes. In this document, we will show you how to integrate AgileTest with CircleCI.
Firstly, you need to acquire Client id and Client secret from AgileTest. Please refer to this document for more details Access API documentation .
Make sure your account has permission to create TestCase and TestExecution issues; otherwise, the import will fail.
Setup your TeamCity project Unlike GitHub, GitLab, or Bitbucket, TeamCity is solely a CI/CD platform. It doesn’t have source repositories or version control system. That’s why when using CI/CD platform, users need to connect it with other source repositories.
In this example below, we integrate a TeamCity pipeline with a GitHub repository.
Open the pipeline settings and add 3 variables BASE_AUTH_URL , BASE_URL , and PROJECT_KEY . Then expand Secrets section and continue to add CLIENT_ID and CLIENT_SECRET with respective Client id and Client secret you have acquired earlier.
Pipeline environment settings
Variables
Description
Is secured?
Example
BASE_AUTH_URL
Base URL used for requesting authenticating token
https://agiletest.atlas.devsamurai.com
BASE_URL
Base URL to submit your report to
https://api.agiletest.app
CLIENT_ID
The client id that you have requested earlier
******
CLIENT_SECRET
The client secret that you have requested earlier
******
PROJECT_KEY
Your project key
RKE
However, in case you are working with AgileTest Data Center version, you only need 3 variables as follows.
Variables
Description
Is secured?
Example
DC_TOKEN
Your Personal access token
BASE_URL
Base URL to submit your report to
PROJECT_KEY
Your project key
RKE
Next, create a job that could achieve your use case. There could be more than one way for you to setup a pipeline in TeamCity. However, in order to simplify the process we will configure the pipeline in 2 separate steps or jobs.
Upload test result Here is a sample JUnit report to play around with.
JUnit report example
CODE
<?xml version="1.0" encoding="UTF-8"?>
<testsuite xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="https://maven.apache.org/surefire/maven-surefire-plugin/xsd/surefire-test-report-3.0.xsd" version="3.0" name="calculateTest" time="0.005" tests="2" errors="0" skipped="0" failures="0">
<properties>
<property name="java.specification.version" value="17"/>
<property name="sun.jnu.encoding" value="UTF-8"/>
<property name="java.class.path" value="/Users/thachnguyen/Work/autotest/JunitAgileTestDemo/target/test-classes:/Users/thachnguyen/Work/autotest/JunitAgileTestDemo/target/classes:/Users/thachnguyen/.m2/repository/org/junit/jupiter/junit-jupiter-api/5.8.2/junit-jupiter-api-5.8.2.jar:/Users/thachnguyen/.m2/repository/org/opentest4j/opentest4j/1.2.0/opentest4j-1.2.0.jar:/Users/thachnguyen/.m2/repository/org/junit/platform/junit-platform-commons/1.8.2/junit-platform-commons-1.8.2.jar:/Users/thachnguyen/.m2/repository/org/apiguardian/apiguardian-api/1.1.2/apiguardian-api-1.1.2.jar:/Users/thachnguyen/.m2/repository/org/junit/jupiter/junit-jupiter-engine/5.8.2/junit-jupiter-engine-5.8.2.jar:/Users/thachnguyen/.m2/repository/org/junit/platform/junit-platform-engine/1.8.2/junit-platform-engine-1.8.2.jar:/Users/thachnguyen/.m2/repository/io/rest-assured/rest-assured/5.1.1/rest-assured-5.1.1.jar:/Users/thachnguyen/.m2/repository/org/apache/groovy/groovy/4.0.1/groovy-4.0.1.jar:/Users/thachnguyen/.m2/repository/org/apache/groovy/groovy-xml/4.0.1/groovy-xml-4.0.1.jar:/Users/thachnguyen/.m2/repository/org/apache/httpcomponents/httpclient/4.5.13/httpclient-4.5.13.jar:/Users/thachnguyen/.m2/repository/org/apache/httpcomponents/httpcore/4.4.13/httpcore-4.4.13.jar:/Users/thachnguyen/.m2/repository/commons-logging/commons-logging/1.2/commons-logging-1.2.jar:/Users/thachnguyen/.m2/repository/commons-codec/commons-codec/1.11/commons-codec-1.11.jar:/Users/thachnguyen/.m2/repository/org/apache/httpcomponents/httpmime/4.5.13/httpmime-4.5.13.jar:/Users/thachnguyen/.m2/repository/org/hamcrest/hamcrest/2.1/hamcrest-2.1.jar:/Users/thachnguyen/.m2/repository/org/ccil/cowan/tagsoup/tagsoup/1.2.1/tagsoup-1.2.1.jar:/Users/thachnguyen/.m2/repository/io/rest-assured/json-path/5.1.1/json-path-5.1.1.jar:/Users/thachnguyen/.m2/repository/org/apache/groovy/groovy-json/4.0.1/groovy-json-4.0.1.jar:/Users/thachnguyen/.m2/repository/io/rest-assured/rest-assured-common/5.1.1/rest-assured-common-5.1.1.jar:/Users/thachnguyen/.m2/repository/io/rest-assured/xml-path/5.1.1/xml-path-5.1.1.jar:/Users/thachnguyen/.m2/repository/org/apache/commons/commons-lang3/3.11/commons-lang3-3.11.jar:/Users/thachnguyen/.m2/repository/org/json/json/20210307/json-20210307.jar:/Users/thachnguyen/.m2/repository/org/junit/platform/junit-platform-reporting/1.9.1/junit-platform-reporting-1.9.1.jar:/Users/thachnguyen/.m2/repository/org/junit/platform/junit-platform-launcher/1.9.1/junit-platform-launcher-1.9.1.jar:"/>
<property name="java.vm.vendor" value="Eclipse Adoptium"/>
<property name="sun.arch.data.model" value="64"/>
<property name="java.vendor.url" value="https://adoptium.net/"/>
<property name="os.name" value="Mac OS X"/>
<property name="java.vm.specification.version" value="17"/>
<property name="sun.java.launcher" value="SUN_STANDARD"/>
<property name="user.country" value="VN"/>
<property name="sun.boot.library.path" value="/Library/Java/JavaVirtualMachines/temurin-17.jdk/Contents/Home/lib"/>
<property name="sun.java.command" value="/Users/thachnguyen/Work/autotest/JunitAgileTestDemo/target/surefire/surefirebooter-20250507171755717_3.jar /Users/thachnguyen/Work/autotest/JunitAgileTestDemo/target/surefire 2025-05-07T17-17-55_670-jvmRun1 surefire-20250507171755717_1tmp surefire_0-20250507171755717_2tmp"/>
<property name="http.nonProxyHosts" value="local|*.local|169.254/16|*.169.254/16"/>
<property name="jdk.debug" value="release"/>
<property name="surefire.test.class.path" value="/Users/thachnguyen/Work/autotest/JunitAgileTestDemo/target/test-classes:/Users/thachnguyen/Work/autotest/JunitAgileTestDemo/target/classes:/Users/thachnguyen/.m2/repository/org/junit/jupiter/junit-jupiter-api/5.8.2/junit-jupiter-api-5.8.2.jar:/Users/thachnguyen/.m2/repository/org/opentest4j/opentest4j/1.2.0/opentest4j-1.2.0.jar:/Users/thachnguyen/.m2/repository/org/junit/platform/junit-platform-commons/1.8.2/junit-platform-commons-1.8.2.jar:/Users/thachnguyen/.m2/repository/org/apiguardian/apiguardian-api/1.1.2/apiguardian-api-1.1.2.jar:/Users/thachnguyen/.m2/repository/org/junit/jupiter/junit-jupiter-engine/5.8.2/junit-jupiter-engine-5.8.2.jar:/Users/thachnguyen/.m2/repository/org/junit/platform/junit-platform-engine/1.8.2/junit-platform-engine-1.8.2.jar:/Users/thachnguyen/.m2/repository/io/rest-assured/rest-assured/5.1.1/rest-assured-5.1.1.jar:/Users/thachnguyen/.m2/repository/org/apache/groovy/groovy/4.0.1/groovy-4.0.1.jar:/Users/thachnguyen/.m2/repository/org/apache/groovy/groovy-xml/4.0.1/groovy-xml-4.0.1.jar:/Users/thachnguyen/.m2/repository/org/apache/httpcomponents/httpclient/4.5.13/httpclient-4.5.13.jar:/Users/thachnguyen/.m2/repository/org/apache/httpcomponents/httpcore/4.4.13/httpcore-4.4.13.jar:/Users/thachnguyen/.m2/repository/commons-logging/commons-logging/1.2/commons-logging-1.2.jar:/Users/thachnguyen/.m2/repository/commons-codec/commons-codec/1.11/commons-codec-1.11.jar:/Users/thachnguyen/.m2/repository/org/apache/httpcomponents/httpmime/4.5.13/httpmime-4.5.13.jar:/Users/thachnguyen/.m2/repository/org/hamcrest/hamcrest/2.1/hamcrest-2.1.jar:/Users/thachnguyen/.m2/repository/org/ccil/cowan/tagsoup/tagsoup/1.2.1/tagsoup-1.2.1.jar:/Users/thachnguyen/.m2/repository/io/rest-assured/json-path/5.1.1/json-path-5.1.1.jar:/Users/thachnguyen/.m2/repository/org/apache/groovy/groovy-json/4.0.1/groovy-json-4.0.1.jar:/Users/thachnguyen/.m2/repository/io/rest-assured/rest-assured-common/5.1.1/rest-assured-common-5.1.1.jar:/Users/thachnguyen/.m2/repository/io/rest-assured/xml-path/5.1.1/xml-path-5.1.1.jar:/Users/thachnguyen/.m2/repository/org/apache/commons/commons-lang3/3.11/commons-lang3-3.11.jar:/Users/thachnguyen/.m2/repository/org/json/json/20210307/json-20210307.jar:/Users/thachnguyen/.m2/repository/org/junit/platform/junit-platform-reporting/1.9.1/junit-platform-reporting-1.9.1.jar:/Users/thachnguyen/.m2/repository/org/junit/platform/junit-platform-launcher/1.9.1/junit-platform-launcher-1.9.1.jar:"/>
<property name="sun.cpu.endian" value="little"/>
<property name="user.home" value="/Users/thachnguyen"/>
<property name="user.language" value="en"/>
<property name="java.specification.vendor" value="Oracle Corporation"/>
<property name="java.version.date" value="2025-01-21"/>
<property name="java.home" value="/Library/Java/JavaVirtualMachines/temurin-17.jdk/Contents/Home"/>
<property name="file.separator" value="/"/>
<property name="basedir" value="/Users/thachnguyen/Work/autotest/JunitAgileTestDemo"/>
<property name="java.vm.compressedOopsMode" value="Zero based"/>
<property name="line.separator" value=" "/>
<property name="java.specification.name" value="Java Platform API Specification"/>
<property name="java.vm.specification.vendor" value="Oracle Corporation"/>
<property name="surefire.real.class.path" value="/Users/thachnguyen/Work/autotest/JunitAgileTestDemo/target/surefire/surefirebooter-20250507171755717_3.jar"/>
<property name="sun.management.compiler" value="HotSpot 64-Bit Tiered Compilers"/>
<property name="ftp.nonProxyHosts" value="local|*.local|169.254/16|*.169.254/16"/>
<property name="java.runtime.version" value="17.0.14+7"/>
<property name="user.name" value="thachnguyen"/>
<property name="path.separator" value=":"/>
<property name="os.version" value="15.2"/>
<property name="java.runtime.name" value="OpenJDK Runtime Environment"/>
<property name="file.encoding" value="UTF-8"/>
<property name="java.vm.name" value="OpenJDK 64-Bit Server VM"/>
<property name="java.vendor.version" value="Temurin-17.0.14+7"/>
<property name="localRepository" value="/Users/thachnguyen/.m2/repository"/>
<property name="java.vendor.url.bug" value="https://github.com/adoptium/adoptium-support/issues"/>
<property name="java.io.tmpdir" value="/var/folders/f0/hxzh9_fx1zb29pdn334l6nr40000gn/T/"/>
<property name="java.version" value="17.0.14"/>
<property name="user.dir" value="/Users/thachnguyen/Work/autotest/JunitAgileTestDemo"/>
<property name="os.arch" value="aarch64"/>
<property name="java.vm.specification.name" value="Java Virtual Machine Specification"/>
<property name="native.encoding" value="UTF-8"/>
<property name="java.library.path" value="/Users/thachnguyen/Library/Java/Extensions:/Library/Java/Extensions:/Network/Library/Java/Extensions:/System/Library/Java/Extensions:/usr/lib/java:."/>
<property name="java.vm.info" value="mixed mode, sharing"/>
<property name="java.vendor" value="Eclipse Adoptium"/>
<property name="java.vm.version" value="17.0.14+7"/>
<property name="java.specification.maintenance.version" value="1"/>
<property name="sun.io.unicode.encoding" value="UnicodeBig"/>
<property name="socksNonProxyHosts" value="local|*.local|169.254/16|*.169.254/16"/>
<property name="java.class.version" value="61.0"/>
</properties>
<testcase name="testAdd" classname="calculateTest" time="0.002"/>
<testcase name="testSubtract" classname="calculateTest" time="0.0"/>
</testsuite>
If you need more detailed examples, please refer to our public repository on AgileTest GitHub for additional sample projects.
Use API We will create a two-step job to help upload test result to Agile Test.
Step 1: Build and run test In step 1, the pipeline runs tests and yield the test reports. The reports are generated and stored in target/surefile-reports directory .
Add a step of Maven type and name it “Build & Test”.
Set Goal = test.
Step 1 configuration
Step 2: Upload test result to Agile Test via API In step 2, the pipeline uploads test reported (stored in Artifact in the last job) to Agile Test using curl to send API requests.
Add a step of Script type and name it “Upload test result”.
In Script content field, add the scrip below and save. All parameters encapsulated in % will be replaced with preconfigured secrets and parameters on pipeline run.
BASH
cd target/surefire-reports
export token=$(curl "%BASE_AUTH_URL%/api/apikeys/authenticate" \
-X POST \
-H 'Content-Type:application/json' \
--data '{"clientId":"'"%CLIENT_ID%"'","clientSecret":"'"%CLIENT_SECRET%"'"}' | tr -d '"')
curl -X POST \
-H "Content-Type:application/xml" \
-H "Authorization:JWT $token" \
--data-binary "@TEST-calculateTest.xml" "%BASE_URL%/ds/test-executions/junit?projectKey=%PROJECT_KEY%" Step 2 configuration
For AgileTest Data center , the curl requests should look like this.
BASH
cd target/surefire-report
curl -X POST \
-H "Content-Type:application/xml" \
-H "Authorization:Bearer %DC_TOKEN%" \
--data-binary "@TEST-calculateTest.xml" "%BASE_URL%/ds/test-executions/junit?projectKey=%PROJECT_KEY%" At this stage, your pipeline is ready.
Use AgileTest CLI Unlike API methods, this approach necessitates dividing the process into two discrete jobs.
Job 1: Build and run test In job 1, the pipeline runs tests and yield the test reports. Then it uploads test reports to Artifact registry.
Add a step of Maven type and name it “Build & Test”.
Set Goal = test.
Set Shared files = **/TEST-*.xml or name of the report files after the testing phase finishes.
Job 1 configuration
Job 2: Upload test result to Agile Test via API In job 2, the pipeline uploads test reported (stored in Artifact in the last job) to Agile Test using curl to send API requests.
Tick Job 1 checkbox in Dependencies section. Don’t forget to change Ignore artifacts to Use artifacts , otherwise all shared files from Job 1 will be ignored altogether 😉.
Job 2 configuration
Add a step of Script type and name it “Upload test result”.
In Script content field, add the command below and save. All parameters encapsulated in % will be replaced with preconfigured secrets and parameters on pipeline run.
BASH
cd target/surefire-reports
agiletest --client-id %CLIENT_ID% --client-secret %CLIENT_SECRET% \
--base-auth-url %BASE_AUTH_URL% --base-url %BASE_URL% \
test-execution import \
--framework-type junit --project-key %PROJECT_KEY% \
TEST-calculateTest.xml In case you are working with AgileTest Data center version, your Script content should look like this.
BASH
cd target/surefire-reports
agiletest --base-url %BASE_URL% \
--data-center-token %DC_TOKEN% \
--data-center \
test-execution import \
--framework-type junit --project-key %PROJECT_KEY% \
TEST-calculateTest.xml Switch Run in Docker on and select Dockerfile . Fill in Describe in command line the content below and save.
DOCKER
# Use official AgileTest CLI image as base
FROM ghcr.io/agiletestapp/agiletest-cli:latest
# Copy application code
COPY . .
At this stage, your step should look like this.
Upload test result step in Job 2
Now your pipeline is ready to use 😉
Should you need any assistance or further AgileTest inquiries, contact here !
