Integrate your test automation framework with Azure Test Plan

Ever wondered how to make your automated tests written in Java/JavaScript/Python, integrate with Azure Test Plan 🤔? If yes, then look no further!! In this article we will look into a step by step process of how we can tweak our existing test automation framework to interact with Azure Test Plan. Microsoft has always loved C# and Visual Studio, so in Azure DevOps, the integration of C# tests with Test Plan come out of the box. For everything else, we need to find our own ways..

In order to achieve this we will make use of the super cool Azure DevOps APIs which by the way are very nicely documented by Microsoft but are way too many and way too confusing 🤯. Finding the exact API we need, is like finding a needle in a haystack.

The objective :

  1. Create a new “Run” in Azure DevOps when the automated tests run as part of Build or Release pipeline
  2. Pull the manual test cases, which correspond to the automated test cases, into this new run.
  3. Mark the outcome of the “executed” test cases in Test Plan as Passed or Failed, or leave them as Active (default) if they are not included for execution.
  4. If a testcase fails, automatically create a bug and associate it with the failing test case. This can be viewed in the “Run” result.
  5. If the testcase passes on a rerun or in the next iteration, close the defect and automatically mark the test case as Passed.

The API Solution :

  1. Get Test Plan ID

2. Get Test Suite ID

planID-is available from step1

3. Get Test Case ID

planID-is available from step1
suiteID-is available from step2

4. Get Test Point ID

planID-is available from step1
suiteID-is available from step2
tcID-is available from step3

5. Create Test Run

planID-is available from step1
pointID-is available from step4

6. Get Test Result ID

runID-is available from step5

7. Create Bug (Optional step)

8. Update Results in Test Run

runID-is available from step5
resultID-is available from step6
bugID-is available from step7

And voila!! There you go.. These are all you need to be able to report your automated test case status to Azure Test Plan.

When all test cases pass, your Test Plan looks like this :

And if any test case fails it looks like this :

I was able to successfully integrate this set up with Java-Cucumber tests and Robot-Python tests. Likewise this can be extended to any JS framework like WebDriverIO or Cypress.

One thing to note however is , the above setup shows the examples with 1 Test Plan, 1 Test Suite and 1 test case to illustrate how the process works. However based on the needs, this can be modified to handle multiple Plans, Suites and Test Cases.

To view a simple python implementation of this you can check my GitHub link:

To view the complete Microsoft Documentation on Azure DevOps check this link:

Test Automation Consultant @Cognizant. Tech enthusiast. Innovation champion.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store