Skip to main content
Skip table of contents

CircleCI

CircleCI is a cloud-based continuous integration and continuous delivery (CI/CD) platform designed to automate software development workflows. It enables developers to ship code faster, more reliably and with confidence. In this document, we will show you how to integrate Agile Test with CircleCI.

Firstly, you need to acquire Client id and Client secret from AgileTest. Please refer to this document for more details Access API documentation.

Make sure your account has permission to create TestCase and TestExecution issues; otherwise, the import will fail.

Setup with CircleCI

Unlike GitHub, GitLab, or Bitbucket, CircleCI is solely a CI/CD platform. It doesn’t have source repositories or version control system. That’s why when using CI/CD platform, users need to connect it with other source repositories.

Before proceeding further, make sure you have at least one project set up in your Circle CI instance → Create a project in CircleCI.

In this example, we will integrate a CircleCI project with a GitHub repository. Once all the setup steps are completed, a config.yml file should be present in the .circleci directory within your repository.

image (69).png

Github repository

Below is a config example file.

YAML
# This config was automatically generated from your source code
# Stacks detected: cicd:github-actions:.github/workflows,cicd:jenkins:.,deps:java:.
version: 2.1
jobs:
  test-java:
    docker:
      - image: cimg/openjdk:17.0
    steps:
      - checkout
      - run:
          name: Calculate cache key
          command: |-
            find . -name 'pom.xml' -o -name 'gradlew*' -o -name '*.gradle*' | \
                    sort | xargs cat > /tmp/CIRCLECI_CACHE_KEY
      - restore_cache:
          key: cache-{{ checksum "/tmp/CIRCLECI_CACHE_KEY" }}
      - run:
          command: mvn verify
      - store_test_results:
          path: target/surefire-reports
      - save_cache:
          key: cache-{{ checksum "/tmp/CIRCLECI_CACHE_KEY" }}
          paths:
            - ~/.m2/repository
  deploy:
    # This is an example deploy job, not actually used by the workflow
    docker:
      - image: cimg/base:stable
    steps:
      # Replace this with steps to deploy to users
      - run:
          name: deploy
          command: "#e.g. ./deploy.sh"
      - run:
          name: found jenkins config
          command: ":"
      - run:
          name: found github actions config
          command: ":"
workflows:
  build-and-test:
    jobs:
      - test-java
    # - deploy:
    #     requires:
    #       - test-java

Next, we need to add the required parameters. Go to Project Settings → Environment Variables and add CLIENT_ID, CLIENT_SECRET, and PROJECT_KEY.

image-20250117-083624.png

Project environment variable setting

Upload test result

Here's a JUnit report example to experiment with.

JUnit report example
CODE
<?xml version="1.0" encoding="UTF-8" ?>
<testsuite tests="15" failures="0" name="ut.com.xpandit.raven.service.impl.IssueDataSetTest" time="0.163" errors="0" skipped="0">
  <properties>
    ...
  </properties>
  <testcase classname="ut.com.xpandit.raven.service.impl.IssueDataSetTest" name="testApplyOptions_withValidIssueAndValidLimitOverflowOption_returnsExpectedSubset" time="0.114"/>
  <testcase classname="ut.com.xpandit.raven.service.impl.IssueDataSetTest" name="testApplyOptions_withNullOptionsAndValidIssue_throwsIllegalArgumentException" time="0.002"/>
  <testcase classname="ut.com.xpandit.raven.service.impl.IssueDataSetTest" name="testApplyOptions_withValidIssueAndValidEmptyOptions_returnsAllIssues" time="0.002"/>
  <testcase classname="ut.com.xpandit.raven.service.impl.IssueDataSetTest" name="testApplyOptions_withValidIssueAndValidGlobalSearchOptions_returnsExpectedTests" time="0.016"/>
  <testcase classname="ut.com.xpandit.raven.service.impl.IssueDataSetTest" name="testApplyOptions_withValidIssueAndInvalidColumnSearchOption_returnsAllTests" time="0.007"/>
  <testcase classname="ut.com.xpandit.raven.service.impl.IssueDataSetTest" name="testApplyOptions_withValidIssueAndValidLimitUnderOption_returnsExpectedSubset" time="0.001"/>
  <testcase classname="ut.com.xpandit.raven.service.impl.IssueDataSetTest" name="testApplyOptions_withValidIssueAndValidGlobalSearchOptionThatMachesIssueKey_returnsExpectedTestWithMatchedKey" time="0.006"/>
  <testcase classname="ut.com.xpandit.raven.service.impl.IssueDataSetTest" name="testApplyOptions_withValidIssueAndValidSummaryColumnAscSortOption_returnsExpectedIssuesInAscOrder" time="0.006"/>
  <testcase classname="ut.com.xpandit.raven.service.impl.IssueDataSetTest" name="testApplyOptions_withValidIssueAndValidSummaryColumnDescSortOption_returnsExpectedIssuesInDescOrder" time="0.002"/>
  <testcase classname="ut.com.xpandit.raven.service.impl.IssueDataSetTest" name="testApplyOptions_withValidIssueAndValidGlobalSearchOptionThatMatchesAllElements_returnsAllTests" time="0.001"/>
  <testcase classname="ut.com.xpandit.raven.service.impl.IssueDataSetTest" name="testApplyOptions_withValidIssueAndValidColumnSearchOptionThatMatchesOneElement_returnsOneTest" time="0.002"/>
  <testcase classname="ut.com.xpandit.raven.service.impl.IssueDataSetTest" name="testApplyOptions_withValidIssueAndValidColumnSearchOptionThatMatchesNoIssue_returnsEmptyList" time="0.001"/>
  <testcase classname="ut.com.xpandit.raven.service.impl.IssueDataSetTest" name="testApplyOptions_withValidIssueAndValidGlobalSearchOptionThatMachesNoIssue_returnsEmptyList" time="0.001"/>
  <testcase classname="ut.com.xpandit.raven.service.impl.IssueDataSetTest" name="testApplyOptions_withValidIssueAndValidKeyColumnDescSortOption_returnsExpectedIssuesInDescOrder" time="0.001"/>
  <testcase classname="ut.com.xpandit.raven.service.impl.IssueDataSetTest" name="testApplyOptions_withValidIssueAndValidKeyColumnAscSortOption_returnsExpectedIssuesInAscOrder" time="0.001"/>
</testsuite>

If you need more thorough examples, please refer to our public repository for more sample projects Agile Test GitHub.

Use API

To use API to upload reports, you need to add an additional step.

YAML
# steps:
- run:
    command: |
      cd target/surefire-reports
      export token=$(curl 'https://agiletest.atlas.devsamurai.com/api/apikeys/authenticate' -X POST -H 'Content-Type:application/json' --data '{"clientId":"'"$CLIENT_ID"'","clientSecret":"'"$CLIENT_SECRET"'"}' | tr -d '"')
      curl -X POST -H "Content-Type:application/xml" -H "Authorization:JWT $token" --data-binary "@TEST-calculateTest.xml" "https://api.agiletest.app/ds/test-executions/junit?projectKey=$PROJECT_KEY"

In this step, we use 2 endpoints.

  • api/v1/apikeys/authenticate to get temporary token using acquired client_id and client_secret.

  • ds/test-executions/junit to submit test report to Agile Test so that the application could create or Test execution and Test cases accordingly.

To learn more about these endpoints, please refer to this document API document.

JavaScript errors detected

Please note, these errors can depend on your browser setup.

If this problem persists, please contact our support.