Skip to main content
Skip table of contents

Jenkins

Jenkins is a popular open-source automation server that helps with Continuous Integration and Continuous Delivery (CI/CD) processes. It allows developers to automate the building, testing, and deployment of software applications.

In this document, you will learn about how to integrate AgileTest with Jenkins.

Firstly, you need to acquire Client id and Client secret from AgileTest. Please refer to this document for more details Access API documentation.

Make sure your account has permission to create TestCase and TestExecution issues; otherwise, the import will fail.

Setup your Jenkins instance

While this page covers the basics of integrating AgileTest with Jenkins, it will not cover the fundamentals of Docker, which you can refer to in the Docker Getting Started Guide and install Docker on your local machine before proceeding further.

In Jenkins, go to Dashboard Manage JenkinsSystemGlobal properties and tick Environment variables. Then, create 2 variables CLIENT_ID and CLIENT_SECRET and fill in your Client id and Client secret respectively.

Screen Shot 2025-01-13 at 16.26.07.png

Global properties

Upload test result

Here is a JUnit report example to experiment with.

JUnit report example
XML
<?xml version="1.0" encoding="UTF-8"?>
<testsuite xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="https://maven.apache.org/surefire/maven-surefire-plugin/xsd/surefire-test-report-3.0.xsd" version="3.0" name="calculateTest" time="0.005" tests="2" errors="0" skipped="0" failures="0">
  <properties>
    <property name="java.specification.version" value="17"/>
    <property name="sun.jnu.encoding" value="UTF-8"/>
    <property name="java.class.path" value="/Users/thachnguyen/Work/autotest/JunitAgileTestDemo/target/test-classes:/Users/thachnguyen/Work/autotest/JunitAgileTestDemo/target/classes:/Users/thachnguyen/.m2/repository/org/junit/jupiter/junit-jupiter-api/5.8.2/junit-jupiter-api-5.8.2.jar:/Users/thachnguyen/.m2/repository/org/opentest4j/opentest4j/1.2.0/opentest4j-1.2.0.jar:/Users/thachnguyen/.m2/repository/org/junit/platform/junit-platform-commons/1.8.2/junit-platform-commons-1.8.2.jar:/Users/thachnguyen/.m2/repository/org/apiguardian/apiguardian-api/1.1.2/apiguardian-api-1.1.2.jar:/Users/thachnguyen/.m2/repository/org/junit/jupiter/junit-jupiter-engine/5.8.2/junit-jupiter-engine-5.8.2.jar:/Users/thachnguyen/.m2/repository/org/junit/platform/junit-platform-engine/1.8.2/junit-platform-engine-1.8.2.jar:/Users/thachnguyen/.m2/repository/io/rest-assured/rest-assured/5.1.1/rest-assured-5.1.1.jar:/Users/thachnguyen/.m2/repository/org/apache/groovy/groovy/4.0.1/groovy-4.0.1.jar:/Users/thachnguyen/.m2/repository/org/apache/groovy/groovy-xml/4.0.1/groovy-xml-4.0.1.jar:/Users/thachnguyen/.m2/repository/org/apache/httpcomponents/httpclient/4.5.13/httpclient-4.5.13.jar:/Users/thachnguyen/.m2/repository/org/apache/httpcomponents/httpcore/4.4.13/httpcore-4.4.13.jar:/Users/thachnguyen/.m2/repository/commons-logging/commons-logging/1.2/commons-logging-1.2.jar:/Users/thachnguyen/.m2/repository/commons-codec/commons-codec/1.11/commons-codec-1.11.jar:/Users/thachnguyen/.m2/repository/org/apache/httpcomponents/httpmime/4.5.13/httpmime-4.5.13.jar:/Users/thachnguyen/.m2/repository/org/hamcrest/hamcrest/2.1/hamcrest-2.1.jar:/Users/thachnguyen/.m2/repository/org/ccil/cowan/tagsoup/tagsoup/1.2.1/tagsoup-1.2.1.jar:/Users/thachnguyen/.m2/repository/io/rest-assured/json-path/5.1.1/json-path-5.1.1.jar:/Users/thachnguyen/.m2/repository/org/apache/groovy/groovy-json/4.0.1/groovy-json-4.0.1.jar:/Users/thachnguyen/.m2/repository/io/rest-assured/rest-assured-common/5.1.1/rest-assured-common-5.1.1.jar:/Users/thachnguyen/.m2/repository/io/rest-assured/xml-path/5.1.1/xml-path-5.1.1.jar:/Users/thachnguyen/.m2/repository/org/apache/commons/commons-lang3/3.11/commons-lang3-3.11.jar:/Users/thachnguyen/.m2/repository/org/json/json/20210307/json-20210307.jar:/Users/thachnguyen/.m2/repository/org/junit/platform/junit-platform-reporting/1.9.1/junit-platform-reporting-1.9.1.jar:/Users/thachnguyen/.m2/repository/org/junit/platform/junit-platform-launcher/1.9.1/junit-platform-launcher-1.9.1.jar:"/>
    <property name="java.vm.vendor" value="Eclipse Adoptium"/>
    <property name="sun.arch.data.model" value="64"/>
    <property name="java.vendor.url" value="https://adoptium.net/"/>
    <property name="os.name" value="Mac OS X"/>
    <property name="java.vm.specification.version" value="17"/>
    <property name="sun.java.launcher" value="SUN_STANDARD"/>
    <property name="user.country" value="VN"/>
    <property name="sun.boot.library.path" value="/Library/Java/JavaVirtualMachines/temurin-17.jdk/Contents/Home/lib"/>
    <property name="sun.java.command" value="/Users/thachnguyen/Work/autotest/JunitAgileTestDemo/target/surefire/surefirebooter-20250507171755717_3.jar /Users/thachnguyen/Work/autotest/JunitAgileTestDemo/target/surefire 2025-05-07T17-17-55_670-jvmRun1 surefire-20250507171755717_1tmp surefire_0-20250507171755717_2tmp"/>
    <property name="http.nonProxyHosts" value="local|*.local|169.254/16|*.169.254/16"/>
    <property name="jdk.debug" value="release"/>
    <property name="surefire.test.class.path" value="/Users/thachnguyen/Work/autotest/JunitAgileTestDemo/target/test-classes:/Users/thachnguyen/Work/autotest/JunitAgileTestDemo/target/classes:/Users/thachnguyen/.m2/repository/org/junit/jupiter/junit-jupiter-api/5.8.2/junit-jupiter-api-5.8.2.jar:/Users/thachnguyen/.m2/repository/org/opentest4j/opentest4j/1.2.0/opentest4j-1.2.0.jar:/Users/thachnguyen/.m2/repository/org/junit/platform/junit-platform-commons/1.8.2/junit-platform-commons-1.8.2.jar:/Users/thachnguyen/.m2/repository/org/apiguardian/apiguardian-api/1.1.2/apiguardian-api-1.1.2.jar:/Users/thachnguyen/.m2/repository/org/junit/jupiter/junit-jupiter-engine/5.8.2/junit-jupiter-engine-5.8.2.jar:/Users/thachnguyen/.m2/repository/org/junit/platform/junit-platform-engine/1.8.2/junit-platform-engine-1.8.2.jar:/Users/thachnguyen/.m2/repository/io/rest-assured/rest-assured/5.1.1/rest-assured-5.1.1.jar:/Users/thachnguyen/.m2/repository/org/apache/groovy/groovy/4.0.1/groovy-4.0.1.jar:/Users/thachnguyen/.m2/repository/org/apache/groovy/groovy-xml/4.0.1/groovy-xml-4.0.1.jar:/Users/thachnguyen/.m2/repository/org/apache/httpcomponents/httpclient/4.5.13/httpclient-4.5.13.jar:/Users/thachnguyen/.m2/repository/org/apache/httpcomponents/httpcore/4.4.13/httpcore-4.4.13.jar:/Users/thachnguyen/.m2/repository/commons-logging/commons-logging/1.2/commons-logging-1.2.jar:/Users/thachnguyen/.m2/repository/commons-codec/commons-codec/1.11/commons-codec-1.11.jar:/Users/thachnguyen/.m2/repository/org/apache/httpcomponents/httpmime/4.5.13/httpmime-4.5.13.jar:/Users/thachnguyen/.m2/repository/org/hamcrest/hamcrest/2.1/hamcrest-2.1.jar:/Users/thachnguyen/.m2/repository/org/ccil/cowan/tagsoup/tagsoup/1.2.1/tagsoup-1.2.1.jar:/Users/thachnguyen/.m2/repository/io/rest-assured/json-path/5.1.1/json-path-5.1.1.jar:/Users/thachnguyen/.m2/repository/org/apache/groovy/groovy-json/4.0.1/groovy-json-4.0.1.jar:/Users/thachnguyen/.m2/repository/io/rest-assured/rest-assured-common/5.1.1/rest-assured-common-5.1.1.jar:/Users/thachnguyen/.m2/repository/io/rest-assured/xml-path/5.1.1/xml-path-5.1.1.jar:/Users/thachnguyen/.m2/repository/org/apache/commons/commons-lang3/3.11/commons-lang3-3.11.jar:/Users/thachnguyen/.m2/repository/org/json/json/20210307/json-20210307.jar:/Users/thachnguyen/.m2/repository/org/junit/platform/junit-platform-reporting/1.9.1/junit-platform-reporting-1.9.1.jar:/Users/thachnguyen/.m2/repository/org/junit/platform/junit-platform-launcher/1.9.1/junit-platform-launcher-1.9.1.jar:"/>
    <property name="sun.cpu.endian" value="little"/>
    <property name="user.home" value="/Users/thachnguyen"/>
    <property name="user.language" value="en"/>
    <property name="java.specification.vendor" value="Oracle Corporation"/>
    <property name="java.version.date" value="2025-01-21"/>
    <property name="java.home" value="/Library/Java/JavaVirtualMachines/temurin-17.jdk/Contents/Home"/>
    <property name="file.separator" value="/"/>
    <property name="basedir" value="/Users/thachnguyen/Work/autotest/JunitAgileTestDemo"/>
    <property name="java.vm.compressedOopsMode" value="Zero based"/>
    <property name="line.separator" value="&#10;"/>
    <property name="java.specification.name" value="Java Platform API Specification"/>
    <property name="java.vm.specification.vendor" value="Oracle Corporation"/>
    <property name="surefire.real.class.path" value="/Users/thachnguyen/Work/autotest/JunitAgileTestDemo/target/surefire/surefirebooter-20250507171755717_3.jar"/>
    <property name="sun.management.compiler" value="HotSpot 64-Bit Tiered Compilers"/>
    <property name="ftp.nonProxyHosts" value="local|*.local|169.254/16|*.169.254/16"/>
    <property name="java.runtime.version" value="17.0.14+7"/>
    <property name="user.name" value="thachnguyen"/>
    <property name="path.separator" value=":"/>
    <property name="os.version" value="15.2"/>
    <property name="java.runtime.name" value="OpenJDK Runtime Environment"/>
    <property name="file.encoding" value="UTF-8"/>
    <property name="java.vm.name" value="OpenJDK 64-Bit Server VM"/>
    <property name="java.vendor.version" value="Temurin-17.0.14+7"/>
    <property name="localRepository" value="/Users/thachnguyen/.m2/repository"/>
    <property name="java.vendor.url.bug" value="https://github.com/adoptium/adoptium-support/issues"/>
    <property name="java.io.tmpdir" value="/var/folders/f0/hxzh9_fx1zb29pdn334l6nr40000gn/T/"/>
    <property name="java.version" value="17.0.14"/>
    <property name="user.dir" value="/Users/thachnguyen/Work/autotest/JunitAgileTestDemo"/>
    <property name="os.arch" value="aarch64"/>
    <property name="java.vm.specification.name" value="Java Virtual Machine Specification"/>
    <property name="native.encoding" value="UTF-8"/>
    <property name="java.library.path" value="/Users/thachnguyen/Library/Java/Extensions:/Library/Java/Extensions:/Network/Library/Java/Extensions:/System/Library/Java/Extensions:/usr/lib/java:."/>
    <property name="java.vm.info" value="mixed mode, sharing"/>
    <property name="java.vendor" value="Eclipse Adoptium"/>
    <property name="java.vm.version" value="17.0.14+7"/>
    <property name="java.specification.maintenance.version" value="1"/>
    <property name="sun.io.unicode.encoding" value="UnicodeBig"/>
    <property name="socksNonProxyHosts" value="local|*.local|169.254/16|*.169.254/16"/>
    <property name="java.class.version" value="61.0"/>
  </properties>
  <testcase name="testAdd" classname="calculateTest" time="0.002"/>
  <testcase name="testSubtract" classname="calculateTest" time="0.0"/>
</testsuite>

If you need more detailed examples, please refer to our public repository on AgileTest GitHub for other sample projects.

Use API

Next, go back to Dashboard and create a new item of scripted pipeline type.

Screen Shot 2025-01-13 at 16.27.27.png

Then link this pipeline to your project repository.

Below is an example of .jenkinsfile. In it, we add another variable named PROJECT_KEY and assign it with Jira project key, for example “ATP”.

GROOVY
pipeline {
    agent {
        docker {image 'maven:3.9.9-amazoncorretto-17-al2023'}
    }

    environment{
        PROJECT_KEY = 'ATP'
    }

    stages {
        stage('Run test') {
            steps {
                script{
                    sh'echo "Testing.."'
                    sh'mvn test'
                    token=sh(script: """curl 'https://agiletest.atlas.devsamurai.com/api/apikeys/authenticate' -X POST -H 'Content-Type:application/json' --data '{"clientId":"'"${env.CLIENT_ID}"'","clientSecret":"'"${env.CLIENT_SECRET}"'"}'""", returnStdout:true).trim()
                    echo "${token}"
                    sh"""curl "https://api.agiletest.app/ds/test-executions/junit?projectKey=${PROJECT_KEY}" -X POST -H 'Content-Type:application/xml' -H "Authorization:JWT ${token}" --data "@target/surefire-reports/TEST-calculateTest.xml" """
                }
            }
        }
    }
}

In this file, we use 2 endpoints

  • api/v1/apikeys/authenticate to get temporary token using acquired client_id and client_secret.

  • ds/test-executions/nunit to submit test report to Agile Test so that the application could create or update Test execution and Test cases accordingly.

To learn more about these endpoints, please refer to this document API documentation.

Use AgileTest CLI

Agile Test also offers you a simpler way to upload your test result via CLI.

You could find the specs of our CLI here: https://agiletestapp.github.io/agiletest-cli/ .

Below is a sample .jenkinsfile

YAML
pipeline {
    agent none
    environment{
        PROJECT_KEY = 'ATP'
    }
    stages {
        stage('Run test') {
            agent {
                docker {
                    image 'maven:3.9.9-amazoncorretto-17-al2023'
                }
            }
            steps {
                script {
                    sh'echo "Testing.."'
                    sh'mvn test'
                    }
                stash includes: '**/target/surefire-reports/*.xml', name: 'reports'
            }
        }
        stage('Upload report') {
            agent {
                docker {
                    image 'ghcr.io/agiletestapp/agiletest-cli:latest'
                    args '--entrypoint=""'
                }
            }
            steps {
                echo "test"
                sh 'ls'
                script {
                    sh'''agiletest --client-id ${CLIENT_ID} --client-secret ${CLIENT_SECRET} \
                        test-execution import \
                        --framework-type junit --project-key ${PROJECT_KEY} \
                        target/surefire-reports/TEST-calculateTest.xml'''
                }
            }
        }
    }
}

This file sets up two stages: Run Test and Upload Report. Each stage runs in a separate container using a different image. The Run Test stage uses a Maven image to execute the tests, while the Upload Report stage uses a Docker image designed specifically for the Agile Test CLI to upload the test results.

In Run test stage, we run the automation test scripts and the process yields the reports. Then, in Upload report stage, the report is uploaded to Agile Test by using AgileTest CLI. Below is the command used for the task.

BASH
agiletest --client-id ${CLIENT_ID} --client-secret ${CLIENT_SECRET} \
              test-execution import \
              --framework-type junit --project-key ${PROJECT_KEY} \
              target/surefire-reports/TEST-calculateTest.xml

For each test run, a new Test Execution will be created along with linked Test Cases. However, if the Test Cases already exist in your project, Agile Test will only generate a new Test Execution and link the matching Test Cases to it, including the test results.

View Pipeline Status

AgileTest provides a simple way to track the status of your pipeline after it has been triggered. To enable this feature, just add the following scripts to your Jenkins file:

CODE
pipeline {
    stages {...}

    // Add a history status after pipeline finished
    post {
        success {
            script {
                // This will run if the build is success
                def token = getApiToken()
                sendBuildStatus(token, "success")
            }
        }
        failure {
            script {
                // This will run if the build is failed
                def token = getApiToken()
                sendBuildStatus(token, "failed")
            }
        }
    }
}


def getApiToken() {
    return sh(script: """
        curl -s 'https://agiletest.atlas.devsamurai.com/api/apikeys/authenticate' -X POST -H 'Content-Type:application/json' \
        --data '{"clientId":"'"$env.CLIENT_ID"'", "clientSecret":"'"$env.CLIENT_SECRET"'"}' \
        | tr -d '"'
    """, returnStdout: true).trim()
}

def sendBuildStatus(token, status) {
    def response = sh(script: """
        curl -s -H "Content-Type:application/json" -H "Authorization:JWT $token" \
        --data '{ "buildURL": "'"$env.BUILD_URL"'", "tool":"jenkins", "result":"${status}" }' \
        "https://agiletest.atlas.devsamurai.com/ds/test-executions/${TEST_EXECUTION_KEY}/pipeline/history?projectKey=${PROJECT_KEY}"
    """, returnStdout: true).trim()

    echo "API Response for ${status} build: ${response}"
}

Once set up, your finished pipeline will automatically display Success or Failed status in the Recent Runs history on AgileTest.


Should you need any assistance or further AgileTest inquiries, contact here!

JavaScript errors detected

Please note, these errors can depend on your browser setup.

If this problem persists, please contact our support.