Integrate your test automation framework with Azure Test Plan

Ever wondered how to make your automated tests written in Java/JavaScript/Python, integrate with Azure Test Plan 🤔? If yes, then look no further!! In this article we will look into a step by step process of how we can tweak our existing test automation framework to interact with Azure Test Plan. Microsoft has always loved C# and Visual Studio, so in Azure DevOps, the integration of C# tests with Test Plan come out of the box. For everything else, we need to find our own ways..

In order to achieve this we will make use of the super cool Azure DevOps APIs which by the way are very nicely documented by Microsoft but are way too many and way too confusing 🤯. Finding the exact API we need, is like finding a needle in a haystack.

  1. Create a new “Run” in Azure DevOps when the automated tests run as part of Build or Release pipeline
  2. Pull the manual test cases, which correspond to the automated test cases, into this new run.
  3. Mark the outcome of the “executed” test cases in Test Plan as Passed or Failed, or leave them as Active (default) if they are not included for execution.
  4. If a testcase fails, automatically create a bug and associate it with the failing test case. This can be viewed in the “Run” result.
  5. If the testcase passes on a rerun or in the next iteration, close the defect and automatically mark the test case as Passed.
  1. Get Test Plan ID
Endpoint : 
https://dev.azure.com/yourOrganizationName/yourProjectName/_apis/test/plans?api-version=5.0
Method :
GET
JsonPath to get Test Plan ID :
$.value.[?(@.name == 'yourPlanName')].id

2. Get Test Suite ID

Endpoint : 
https://dev.azure.com/yourOrganizationName/yourProjectName/_apis/test/plans/planID/suites?api-version=5.0
Method :
GET
JsonPath to get Test Suite ID :
$.value.[?(@.name == 'yourSuiteName')].id

3. Get Test Case ID

Endpoint : 
https://dev.azure.com/yourOrganizationName/yourProjectName/_apis/test/plans/planID/suites/suiteID/points?api-version=5.0
Method :
GET
JsonPath to get Test Case ID :
$..[?(@.name == 'yourTestCaseName')].id

4. Get Test Point ID

Endpoint : 
https://dev.azure.com/yourOrganizationName/yourProjectName/_apis/test/plans/planID/suites/suiteID/points?testCaseId=tcID&api-version=5.0
Method :
GET
JsonPath to get Test Point ID :
$.value.[0].id

5. Create Test Run

Endpoint : 
https://dev.azure.com/yourOrganizationName/yourProjectName/_apis/test/runs?api-version=5.0
Method :
POST
Content-Type :
application/json
Sample Payload:
{"name":"runName","plan":{"id":planID},"pointIds":[pointID]}
JsonPath to get Test Run ID :
$.id

6. Get Test Result ID

Endpoint : 
https://dev.azure.com/yourOrganizationName/yourProjectName/_apis/test/runs/runID/results?api-version=6.0-preview.6
Method :
GET
JsonPath to get Test Result ID :
$.value.[0].id

7. Create Bug (Optional step)

Endpoint : 
https://dev.azure.com/yourOrganizationName/yourProjectName/_apis/wit/workitems/$bug?api-version=5.0
Method :
POST
Content-Type :
application/json-patch+json
Sample Payload:
[{"op": "add","path": "/fields/System.Title","from":null, "value":"titleOfBug"}]
JsonPath to get Bug ID :
$.id

8. Update Results in Test Run

Endpoint : 
https://dev.azure.com/yourOrganizationName/yourProjectName/_apis/test/runs/runID/results?api-version=6.0-preview.6
Method :
PATCH
Content-Type :
application/json
Sample Payload if Passed:
[{ "id": resultID , "outcome": "PASSED" ,"state": "Completed", "comment": "Execution Successful" }]
Sample Payload if Failed:
[{ "id": resultID , "outcome": "FAILED" ,"state": "Completed", "comment": "Execution Failed", "associatedBugs": [{"id":bugID}]}]

And voila!! There you go.. These are all you need to be able to report your automated test case status to Azure Test Plan.

When all test cases pass, your Test Plan looks like this :

And if any test case fails it looks like this :

I was able to successfully integrate this set up with Java-Cucumber tests and Robot-Python tests. Likewise this can be extended to any JS framework like WebDriverIO or Cypress.

To view a simple python implementation of this you can check my GitHub link:

To view the complete Microsoft Documentation on Azure DevOps check this link:

Test Automation Consultant @Cognizant. Tech enthusiast. Innovation champion.