Containerized Automated Tests in AWS

Published in

The Startup

6 min readNov 17, 2020

In this article, we will talk about how one can easily containerize and run automated test cases as part of CI — CD using Amazon Web Services ‍🚀

I won’t discuss too much on why we should containerize our automated tests. That’s a topic which I have covered extensively in one of my previous articles. You can read it here.

Here we will directly dig deep into the Amazon services that we can use and see how we can effectively use them.

What we need :

AWS CodeCommit — This is the version control system where you can check in your test automation code. [Details]
AWS CodeBuild—This is used for building/testing the code. [Details]
AWS ECR(Elastic Container Registry)—This is where we store, manage and deploy containers. [Details]
AWS S3(Simple Storage Service) — This is where we store data and artifacts. [Details]
AWS CodePipeline — As the name suggests, we can create delivery pipelines using this service. [Details]

The workflow to be discussed is as follows :

Step 1 : Create your automated tests and push the code to AWS CodeCommit

Log on to AWS console as IAM user → Go to Services → Search for CodeCommit → Click on [Create repository] → Fill details → Click [Create]

To push your code to CodeCommit, use regular git commands :

git add .
git commit -m "Your_Commit_Message"
git push

Your file structure should tentatively look like this, however you can make modifications according to your needs :

(root directory name)
    |-- buildspec.yml
    |-- Dockerfile
    |-- releasespec.yml

We will deal with the details of the buildspec.yml in Step 4 and releasespec.yml in Step 5.

Step 2: Create an S3 Bucket for storing build artifacts

Go to Services → Search for S3 → Click on [Create bucket] → Fill details → Click [Create]

Step 3: Create an ECR for storing the automated test images

Go to Services → Search for ECR → Click on [Create repository] → Fill details → Click [Create]

Step 4: Set up a CodeBuild for building docker image for automated test and push it to ECR

Before setting up the CodeBuild, we should take a step back and look at the buildspec.yml, I briefly mentioned above. The CodeBuild requires a yaml file which should contain the details of the activities that are expected to be performed during the ‘build’ process. Our buildspec should look somewhat like this :

version: 0.2phases:
  pre_build:
    commands:
      - echo Logging in to Amazon ECR...
      - aws ecr get-login-password --region $AWS_DEFAULT_REGION | docker login --username AWS --password-stdin $AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com  build:
    commands:
      - echo Build started on `date`
      - echo Building the Docker image...          
      - docker build -t $AWS_REPO:$IMAGE_TAG .
      - docker tag $AWS_REPO:$IMAGE_TAG $AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com/$AWS_REPO:$IMAGE_TAG 
     
  post_build:
    commands:
      - echo Build completed on `date`
      - echo Pushing the Docker image...
      - docker push $AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com/$AWS_REPO:$IMAGE_TAG
      - aws s3 cp "docker-compose.yml" s3://$S3_BUCKET_NAME/docker-compose.yml
      - aws s3 cp "releasespec.yml" s3://$S3_BUCKET_NAME/releasespec.yml

As obvious from the buildspec above, there are 3 important activities that we are performing :

In Pre-Build stage, we are logging in to the ECR, created in Step 3
In Build stage, we are building the test automation image and tagging it
In Post-Build stage, we are pushing the image to ECR and as a build artifact publishing the docker-compose.yml and releasespec.yml files to S3, created in Step 2

If we observe carefully in the buildspec.yml, we are using some environment variables like $AWS_DEFAULT_REGION, $AWS_ACCOUNT_ID etc. We will define these variables shortly.

So lets go ahead and create the CodeBuild.

Go to Services → Search for CodeBuild→ Click on [Create build project] → Fill details (Some important configurations are listed below)→ Click [Create build project] → To run the build , click [Start Build]

Project Name and Description
Source :
Source Provider : AWS CodeCommit
Repository : Name of the CodeCommit Repository
Branch : master (or the name of the branch where your latest code is)

3. Environment :
Service Role : Select your service role. Learn more about service roles here.
Environment Variables : Add the following variables -

4. Buildspec:
Usually if there is a buildspec.yml in the root directory of your repo, you don't need to specify anything here. If you have a spec file which has a different name, then specify it here.

If everything goes well, your CodeBuild will succeed and you will be able to see the following:

Step 5: Set up a CodePipeline for executing the tests in different environments — Sequence of CodeBuilds

For now, we will focus on a single CodeBuild that will execute the Tests in a particular environment (In this case, DEV environment)

To create this CodeBuild, follow the exact same steps as mentioned in Step 4 with only the following exceptions:

Source : Amazon S3 → Select the Bucket Name

2. Buildspec : Select the releasespec.yml as input

As explained above, it is the buildspec file that determines what activities need to be executed as part of the build. So let’s take a quick look at the releasespec.yml file

version: 0.2phases:
  pre_build:
    commands:
      - echo Logging in to Amazon ECR...
      - aws ecr get-login-password --region $AWS_DEFAULT_REGION | docker login --username AWS --password-stdin $AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com  build:
    commands:
      - echo Test Execution starting...
      - AWS_ACCOUNT_ID=$AWS_ACCOUNT_ID AWS_DEFAULT_REGION=$AWS_DEFAULT_REGION AWS_REPO=$AWS_REPO PROJECT_NAME=WebserviceDemoProject RELEASE_NAME=AWS TESTSET_NAME=Test docker-compose run automation 
    
  post_build:
    commands:
      - aws s3 cp Results s3://$S3_BUCKET_NAME/$ENV --recursive
      - docker-compose down

As obvious from the buildspec above, there are 3 important activities that we are performing :

In Pre-Build stage, we are logging in to the ECR
In Build stage, we are executing the tests by passing a set of environment variables to the docker-compose.yml
In Post-Build stage, we are pushing the Test Results to S3 and killing the test containers

A quick look at the docker-compose file :

version: "3"
services:
  automation:
    image: ${AWS_ACCOUNT_ID}.dkr.ecr.${AWS_DEFAULT_REGION}.amazonaws.com/${AWS_REPO}:latest
    container_name: automation-test
    environment:
      - PROJ=Projects/${PROJECT_NAME}
      - RELEASE=${RELEASE_NAME}
      - TESTSET=${TESTSET_NAME}    
    volumes:
      - ./Results/:/workspace/Projects/${PROJECT_NAME}/Results/TestExecution/${RELEASE_NAME}/${TESTSET_NAME}