How to integrate CITS with Azure DevOps Release Pipeline

This document talks about how you can seamlessly integrate CITS into Azure DevOps and run your automated test cases as part of CI — CD.

We would walk through the 2 main techniques by which we can run CITS test cases in Azure DevOps.

1. Azure Build Pipeline
2. Azure Release Pipeline

In this article we will focus on Azure Release Pipeline. For Azure Build Pipeline refer to this article

Before we get started, few things to know about Microsoft Azure DevOps :

Now let’s get started…

Chapter 2: Azure Release Pipeline

For Chapter 1: Azure Build Pipeline, refer this article.

1. Configure an Azure Release Pipeline

a. Log in to your Azure — Organization.
b. Select Pipelines >> Releases

2. Inside the Release Pipeline add the Artifacts and the Stages

In this particular demo I have added the following Artifacts and Stages :

- Artifacts : I have chosen the Git Repository where my CITS scripts are present. You can always chose Azure Repo or the Artifacts generated after a particular Build has run

- Stages : I have added 3 stages (DEV — TEST — ACCEPTANCE). But you can start off with one at first and follow the following steps to configure it to run CITS scripts -

a. Select the stage template — we would go for Empty job

b. Add Tasks to the Stage to be performed on the Agent

We will use 2 tasks :

i. Set up the Command Line task with the following parameters :

Display name : Choose your own Display Name. I chose “Execute CITS”

Script :

Run.bat -run -project_location $(ProjectLocation) -release $(ReleaseName) -testset $(TestSet)

I chose the Agent as Windows. Hence I am using Run.bat in the script above. If your Agent is Ubuntu or MacOS, you need to use Run.command in the script. Refer to Chapter 1 for details.

Working Directory : This is a file location type input field. You can choose by clicking on the ellipses . For me this is $(System.DefaultWorkingDirectory)/CITS

So the final set up is :

ii. Set up the Publish Test Results task

Display name : Choose your own Display Name. I chose “Publish Test Results”
Test result format : Junit
Test results files : junit.xml
Search folder :

$(System.DefaultWorkingDirectory)/CITS/$(ProjectLocation)/Results/TestExecution/$(ReleaseName)/$(TestSet)/Latest/

Test run title : Choose your own Test run title.

So the final set up is :

a. Use Variables

In the above setups we have defined 3 variables :

$(ProjectLocation)
$(ReleaseName)
$(TestSet)

You have the option of using Pipeline Variables or Variable Groups. I have defined a Variable Group here and set the scope as “Release”.

Please note you can add as many variables as you want. The above 3 are just the 3 basic variables that are needed to execute scripts in CITS.

Now you are all set to trigger your Release Pipeline. All you need is to create a “Release” as shown below :

Your execution will begin :

The final state is as follows :

If you traverse into a Stage, and view Tests we should see the full Test Report

Some Handy Links :

1. Azure DevOps Project : https://dev.azure.com/401532/CITS/
2. GitHub Repository : https://github.com/ghoshasish99/CITS

References :

1. Azure DevOps Documentation :
https://docs.microsoft.com/en-us/azure/devops/?view=azure-devops

2. CITS GitHub Repository :
https://github.com/CognizantQAHub/Cognizant-Intelligent-Test-Scripter

3. YAML syntax documentation :
https://docs.ansible.com/ansible/latest/reference_appendices/YAMLSyntax.html