Skip to main content
Skip table of contents

TeamCity

TeamCity is a powerful Continuous Integration/Continuous Deployment (CI/CD) server developed by JetBrains. It automates software build, test, deployment, and reporting processes. In this document, we will show you how to integrate AgileTest with TeamCity.

Prerequisite:

Make sure your account has permission to create TestCase and TestExecution issues; otherwise, the import will fail.

As a Jira admin, you can update this permission in the Space setting → Permission → Create Issues.

  • For the Cloud version, Client id and Client secret are needed. Please refer to this instruction to get them: API integration

  • For Data Center version, you will need a Personal access token (PAT) instead. In Jira, select âš™ď¸Ź (top right corner) → System -> Administering personal access tokens, and create yourself a token. Please refer to this article from Jira for more details.

1. Setup your TeamCity project

Unlike GitHub, GitLab, or Bitbucket, TeamCity is solely a CI/CD platform. It doesn’t have source repositories or version control system. That’s why when using CI/CD platform, users need to connect it with other source repositories.

To learn more about how to setup a project or a pipeline in TeamCity, please refer to this document: Getting Started with TeamCity.

When you integrate AgileTest with TeamCity, you need to securely store certain authentication details, such as your Client ID and Client Secret. By doing it, you ensure that sensitive data is kept secure and is not exposed in your code or logs, reuse variables in different places, etc. These credentials are also required for AgileTest to authenticate and communicate with TeamCity in the pipeline.

1.1 For Cloud version

We recommend that you set up 5 repository variables:

Variables

Description

Is secured

Example

BASE_AUTH_URL

Base URL used for requesting authenticating token

 

https://agiletest.atlas.devsamurai.com

BASE_URL

Base URL to submit your report to

 

https://api.agiletest.app

CLIENT_ID

The client id that you have requested earlier

âś…

******

CLIENT_SECRET

The client secret that you have requested earlier

âś…

******

PROJECT_KEY

Your project key

 

RKE

1.2 For Data Center version

In case you are working with AgileTest Data Center version, you only need 3 variables as follows.

Variables

Description

Is secured

Example

DC_TOKEN

Your Personal access token

âś…

 

BASE_URL

Base URL to submit your report to

 

 

PROJECT_KEY

Your project key

 

RKE

1.3 Setup TeamCity variables

To set up these variables, you can open the pipeline settings and

  • Add 3 variables BASE_AUTH_URL, BASE_URL, and PROJECT_KEY in the Parameters tab.

  • Then expand Secrets section and continue to add CLIENT_ID and CLIENT_SECRET.

Teamcity.png

2. Upload your test result

After you have set up the TeamCity project, you can prepare the YAML file in TeamCity to connect AgileTest with this tool and upload the test results.

You can have your report in different types of test frameworks. Here is a sample JUnit report to play around with.

JUnit report example
CODE 
<?xml version="1.0" encoding="UTF-8"?>
<testsuite xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="https://maven.apache.org/surefire/maven-surefire-plugin/xsd/surefire-test-report-3.0.xsd" version="3.0" name="calculateTest" time="0.005" tests="2" errors="0" skipped="0" failures="0">
  <properties>
    <property name="java.specification.version" value="17"/>
    <property name="sun.jnu.encoding" value="UTF-8"/>
    <property name="java.class.path" value="/Users/thachnguyen/Work/autotest/JunitAgileTestDemo/target/test-classes:/Users/thachnguyen/Work/autotest/JunitAgileTestDemo/target/classes:/Users/thachnguyen/.m2/repository/org/junit/jupiter/junit-jupiter-api/5.8.2/junit-jupiter-api-5.8.2.jar:/Users/thachnguyen/.m2/repository/org/opentest4j/opentest4j/1.2.0/opentest4j-1.2.0.jar:/Users/thachnguyen/.m2/repository/org/junit/platform/junit-platform-commons/1.8.2/junit-platform-commons-1.8.2.jar:/Users/thachnguyen/.m2/repository/org/apiguardian/apiguardian-api/1.1.2/apiguardian-api-1.1.2.jar:/Users/thachnguyen/.m2/repository/org/junit/jupiter/junit-jupiter-engine/5.8.2/junit-jupiter-engine-5.8.2.jar:/Users/thachnguyen/.m2/repository/org/junit/platform/junit-platform-engine/1.8.2/junit-platform-engine-1.8.2.jar:/Users/thachnguyen/.m2/repository/io/rest-assured/rest-assured/5.1.1/rest-assured-5.1.1.jar:/Users/thachnguyen/.m2/repository/org/apache/groovy/groovy/4.0.1/groovy-4.0.1.jar:/Users/thachnguyen/.m2/repository/org/apache/groovy/groovy-xml/4.0.1/groovy-xml-4.0.1.jar:/Users/thachnguyen/.m2/repository/org/apache/httpcomponents/httpclient/4.5.13/httpclient-4.5.13.jar:/Users/thachnguyen/.m2/repository/org/apache/httpcomponents/httpcore/4.4.13/httpcore-4.4.13.jar:/Users/thachnguyen/.m2/repository/commons-logging/commons-logging/1.2/commons-logging-1.2.jar:/Users/thachnguyen/.m2/repository/commons-codec/commons-codec/1.11/commons-codec-1.11.jar:/Users/thachnguyen/.m2/repository/org/apache/httpcomponents/httpmime/4.5.13/httpmime-4.5.13.jar:/Users/thachnguyen/.m2/repository/org/hamcrest/hamcrest/2.1/hamcrest-2.1.jar:/Users/thachnguyen/.m2/repository/org/ccil/cowan/tagsoup/tagsoup/1.2.1/tagsoup-1.2.1.jar:/Users/thachnguyen/.m2/repository/io/rest-assured/json-path/5.1.1/json-path-5.1.1.jar:/Users/thachnguyen/.m2/repository/org/apache/groovy/groovy-json/4.0.1/groovy-json-4.0.1.jar:/Users/thachnguyen/.m2/repository/io/rest-assured/rest-assured-common/5.1.1/rest-assured-common-5.1.1.jar:/Users/thachnguyen/.m2/repository/io/rest-assured/xml-path/5.1.1/xml-path-5.1.1.jar:/Users/thachnguyen/.m2/repository/org/apache/commons/commons-lang3/3.11/commons-lang3-3.11.jar:/Users/thachnguyen/.m2/repository/org/json/json/20210307/json-20210307.jar:/Users/thachnguyen/.m2/repository/org/junit/platform/junit-platform-reporting/1.9.1/junit-platform-reporting-1.9.1.jar:/Users/thachnguyen/.m2/repository/org/junit/platform/junit-platform-launcher/1.9.1/junit-platform-launcher-1.9.1.jar:"/>
    <property name="java.vm.vendor" value="Eclipse Adoptium"/>
    <property name="sun.arch.data.model" value="64"/>
    <property name="java.vendor.url" value="https://adoptium.net/"/>
    <property name="os.name" value="Mac OS X"/>
    <property name="java.vm.specification.version" value="17"/>
    <property name="sun.java.launcher" value="SUN_STANDARD"/>
    <property name="user.country" value="VN"/>
    <property name="sun.boot.library.path" value="/Library/Java/JavaVirtualMachines/temurin-17.jdk/Contents/Home/lib"/>
    <property name="sun.java.command" value="/Users/thachnguyen/Work/autotest/JunitAgileTestDemo/target/surefire/surefirebooter-20250507171755717_3.jar /Users/thachnguyen/Work/autotest/JunitAgileTestDemo/target/surefire 2025-05-07T17-17-55_670-jvmRun1 surefire-20250507171755717_1tmp surefire_0-20250507171755717_2tmp"/>
    <property name="http.nonProxyHosts" value="local|*.local|169.254/16|*.169.254/16"/>
    <property name="jdk.debug" value="release"/>
    <property name="surefire.test.class.path" value="/Users/thachnguyen/Work/autotest/JunitAgileTestDemo/target/test-classes:/Users/thachnguyen/Work/autotest/JunitAgileTestDemo/target/classes:/Users/thachnguyen/.m2/repository/org/junit/jupiter/junit-jupiter-api/5.8.2/junit-jupiter-api-5.8.2.jar:/Users/thachnguyen/.m2/repository/org/opentest4j/opentest4j/1.2.0/opentest4j-1.2.0.jar:/Users/thachnguyen/.m2/repository/org/junit/platform/junit-platform-commons/1.8.2/junit-platform-commons-1.8.2.jar:/Users/thachnguyen/.m2/repository/org/apiguardian/apiguardian-api/1.1.2/apiguardian-api-1.1.2.jar:/Users/thachnguyen/.m2/repository/org/junit/jupiter/junit-jupiter-engine/5.8.2/junit-jupiter-engine-5.8.2.jar:/Users/thachnguyen/.m2/repository/org/junit/platform/junit-platform-engine/1.8.2/junit-platform-engine-1.8.2.jar:/Users/thachnguyen/.m2/repository/io/rest-assured/rest-assured/5.1.1/rest-assured-5.1.1.jar:/Users/thachnguyen/.m2/repository/org/apache/groovy/groovy/4.0.1/groovy-4.0.1.jar:/Users/thachnguyen/.m2/repository/org/apache/groovy/groovy-xml/4.0.1/groovy-xml-4.0.1.jar:/Users/thachnguyen/.m2/repository/org/apache/httpcomponents/httpclient/4.5.13/httpclient-4.5.13.jar:/Users/thachnguyen/.m2/repository/org/apache/httpcomponents/httpcore/4.4.13/httpcore-4.4.13.jar:/Users/thachnguyen/.m2/repository/commons-logging/commons-logging/1.2/commons-logging-1.2.jar:/Users/thachnguyen/.m2/repository/commons-codec/commons-codec/1.11/commons-codec-1.11.jar:/Users/thachnguyen/.m2/repository/org/apache/httpcomponents/httpmime/4.5.13/httpmime-4.5.13.jar:/Users/thachnguyen/.m2/repository/org/hamcrest/hamcrest/2.1/hamcrest-2.1.jar:/Users/thachnguyen/.m2/repository/org/ccil/cowan/tagsoup/tagsoup/1.2.1/tagsoup-1.2.1.jar:/Users/thachnguyen/.m2/repository/io/rest-assured/json-path/5.1.1/json-path-5.1.1.jar:/Users/thachnguyen/.m2/repository/org/apache/groovy/groovy-json/4.0.1/groovy-json-4.0.1.jar:/Users/thachnguyen/.m2/repository/io/rest-assured/rest-assured-common/5.1.1/rest-assured-common-5.1.1.jar:/Users/thachnguyen/.m2/repository/io/rest-assured/xml-path/5.1.1/xml-path-5.1.1.jar:/Users/thachnguyen/.m2/repository/org/apache/commons/commons-lang3/3.11/commons-lang3-3.11.jar:/Users/thachnguyen/.m2/repository/org/json/json/20210307/json-20210307.jar:/Users/thachnguyen/.m2/repository/org/junit/platform/junit-platform-reporting/1.9.1/junit-platform-reporting-1.9.1.jar:/Users/thachnguyen/.m2/repository/org/junit/platform/junit-platform-launcher/1.9.1/junit-platform-launcher-1.9.1.jar:"/>
    <property name="sun.cpu.endian" value="little"/>
    <property name="user.home" value="/Users/thachnguyen"/>
    <property name="user.language" value="en"/>
    <property name="java.specification.vendor" value="Oracle Corporation"/>
    <property name="java.version.date" value="2025-01-21"/>
    <property name="java.home" value="/Library/Java/JavaVirtualMachines/temurin-17.jdk/Contents/Home"/>
    <property name="file.separator" value="/"/>
    <property name="basedir" value="/Users/thachnguyen/Work/autotest/JunitAgileTestDemo"/>
    <property name="java.vm.compressedOopsMode" value="Zero based"/>
    <property name="line.separator" value="&#10;"/>
    <property name="java.specification.name" value="Java Platform API Specification"/>
    <property name="java.vm.specification.vendor" value="Oracle Corporation"/>
    <property name="surefire.real.class.path" value="/Users/thachnguyen/Work/autotest/JunitAgileTestDemo/target/surefire/surefirebooter-20250507171755717_3.jar"/>
    <property name="sun.management.compiler" value="HotSpot 64-Bit Tiered Compilers"/>
    <property name="ftp.nonProxyHosts" value="local|*.local|169.254/16|*.169.254/16"/>
    <property name="java.runtime.version" value="17.0.14+7"/>
    <property name="user.name" value="thachnguyen"/>
    <property name="path.separator" value=":"/>
    <property name="os.version" value="15.2"/>
    <property name="java.runtime.name" value="OpenJDK Runtime Environment"/>
    <property name="file.encoding" value="UTF-8"/>
    <property name="java.vm.name" value="OpenJDK 64-Bit Server VM"/>
    <property name="java.vendor.version" value="Temurin-17.0.14+7"/>
    <property name="localRepository" value="/Users/thachnguyen/.m2/repository"/>
    <property name="java.vendor.url.bug" value="https://github.com/adoptium/adoptium-support/issues"/>
    <property name="java.io.tmpdir" value="/var/folders/f0/hxzh9_fx1zb29pdn334l6nr40000gn/T/"/>
    <property name="java.version" value="17.0.14"/>
    <property name="user.dir" value="/Users/thachnguyen/Work/autotest/JunitAgileTestDemo"/>
    <property name="os.arch" value="aarch64"/>
    <property name="java.vm.specification.name" value="Java Virtual Machine Specification"/>
    <property name="native.encoding" value="UTF-8"/>
    <property name="java.library.path" value="/Users/thachnguyen/Library/Java/Extensions:/Library/Java/Extensions:/Network/Library/Java/Extensions:/System/Library/Java/Extensions:/usr/lib/java:."/>
    <property name="java.vm.info" value="mixed mode, sharing"/>
    <property name="java.vendor" value="Eclipse Adoptium"/>
    <property name="java.vm.version" value="17.0.14+7"/>
    <property name="java.specification.maintenance.version" value="1"/>
    <property name="sun.io.unicode.encoding" value="UnicodeBig"/>
    <property name="socksNonProxyHosts" value="local|*.local|169.254/16|*.169.254/16"/>
    <property name="java.class.version" value="61.0"/>
  </properties>
  <testcase name="testAdd" classname="calculateTest" time="0.002"/>
  <testcase name="testSubtract" classname="calculateTest" time="0.0"/>
</testsuite>

If you need more detailed examples, please refer to our public repository on AgileTest GitHub for additional sample projects.

There are 2 main approaches to uploading your test results:

2.1 Use API

2.1.1 For Cloud version

Step 1: Build and run test

In step 1, the pipeline runs tests and yield the test reports. The reports are generated and stored in target/surefile-reports directory.

  1. Add a step of Maven type and name it “Build & Test”.

  2. Set Goal = test.

step 1.png
Step 2: Upload test result to Agile Test via API

In step 2, the pipeline uploads test reported (stored in Artifact in the last job) to Agile Test using curl to send API requests.

  1. Add a step of Script type and name it “Upload test result”.

  2. In Script content field, add the scrip below and save. All parameters encapsulated in % will be replaced with preconfigured secrets and parameters on pipeline run.

CODE 
cd target/surefire-reports
export token=$(curl "%BASE_AUTH_URL%/api/apikeys/authenticate" \
-X POST \
-H 'Content-Type:application/json' \
--data '{"clientId":"'"%CLIENT_ID%"'","clientSecret":"'"%CLIENT_SECRET%"'"}' | tr -d '"')
curl -X POST \
-H "Content-Type:application/xml" \
-H "Authorization:JWT $token" \
--data-binary "@TEST-calculateTest.xml" "%BASE_URL%/ds/test-executions/junit?projectKey=%PROJECT_KEY%"
step 2.png

2.1.2 For Data Center version

For the Data Center version, the step 1 and 2 are the same as the Cloud version. You only need to change the script:

CODE 
cd target/surefire-report
curl -X POST \
-H "Content-Type:application/xml" \
-H "Authorization:Bearer %DC_TOKEN%" \
--data-binary "@TEST-calculateTest.xml" "%BASE_URL%/ds/test-executions/junit?projectKey=%PROJECT_KEY%"

2.2 Use AgileTest CLI

You could find the specs of our CLI here https://agiletestapp.github.io/agiletest-cli/ .

Unlike API methods, this approach necessitates dividing the process into two discrete jobs.

2.2.1 For Cloud version

Job 1: Build and run test

In job 1, the pipeline runs tests and yield the test reports. Then it uploads test reports to Artifact registry.

  1. Add a step of Maven type and name it “Build & Test”.

  2. Set Goal = test.

  3. Set Shared files = **/TEST-*.xml or name of the report files after the testing phase finishes.

job 1.png

Learn more about wildcards in TeamCity. Wildcard Support | TeamCity

Job 2: Upload test result to Agile Test via API

In job 2, the pipeline uploads test reported (stored in Artifact in the last job) to Agile Test using curl to send API requests.

  1. Tick Job 1 checkbox in Dependencies section. Don’t forget to change Ignore artifacts to Use artifacts, otherwise all shared files from Job 1 will be ignored altogether.

job 2.png

  1. Add a step of Script type and name it “Upload test result”.

  2. In Script content field, add the command below and save. All parameters encapsulated in % will be replaced with preconfigured secrets and parameters on pipeline run.

CODE 
cd target/surefire-reports
agiletest --client-id %CLIENT_ID% --client-secret %CLIENT_SECRET% \
    --base-auth-url %BASE_AUTH_URL% --base-url %BASE_URL% \
    test-execution import \
    --framework-type junit --project-key %PROJECT_KEY% \
    TEST-calculateTest.xml

  1. Switch Run in Docker on and select Dockerfile. Fill in Describe in command line the content below and save.

CODE 
# Use official AgileTest CLI image as base
FROM ghcr.io/agiletestapp/agiletest-cli:latest
# Copy application code
COPY . .

At this stage, your step should look like this.

test result in job 2.png

2.2.2 For Data Center version

All setup steps match the Cloud version. The difference lies in the script used in step 3 of Job 2.

CODE 
cd target/surefire-reports
agiletest --base-url %BASE_URL% \
    --data-center-token %DC_TOKEN% \
    --data-center \
    test-execution import \
    --framework-type junit --project-key %PROJECT_KEY% \
    TEST-calculateTest.xml

 

Should you need any assistance or further AgileTest inquiries, contact our 24/7 Support Desk!

JavaScript errors detected

Please note, these errors can depend on your browser setup.

If this problem persists, please contact our support.

Contact Support Close