by Bhanu Pratap Singh & Kingshuk Nandy
Introduction
Stable and reliable infrastructure is one of the most important factors for any test automation project. The ROI on a test automation project is only achievable when the automated suites are executed regularly on the controlled environment.
The second point to be considered is – how easily the infrastructure can be scaled to handle the increase in volume of automated tests and reduce in the total execution time.
The below proposed architecture describes one of the optimal solutions to address these challenges, with a list of benefits, the primary ones being the stability, reliability and cost benefits.
In this solution we are using Infrastructure as a code to handle dynamics of infrastructure creation as per our requirement at the run time, based on the selected configuration.
Here one can ask. Why Terraform?
In the current business scenario, it’s quite challenging to choose one cloud provider, when we have multiple options with their own set of benefits. Also, from the deployment point of view, multiple cloud deployment increases fault-tolerance of the system.
Terraform is cloud-agnostic, allows a single configuration to be used to manage multiple providers, and to even handle cross-cloud dependencies.
And in the present DevOps culture, automated tests are an integral part of our deployment pipelines. Therefore, terraform is an obvious choice to simplify the management and orchestration, helping operators build large-scale multi-cloud infrastructures.
Benefits of using cloud infrastructure with Terraform for running automated Test Suites
- Cloud-agnostic environment with Infrastructure as a code
- Cost Benefits
- Stability
- Scalability
- Reliability
- Good Quality Execution Video for Failure Analysis
- Better collaboration and visibility
Proposed Architecture Design
Component Details
CI tool: (Jenkins)
Jenkins is an open source, self-reliant automation server, initially developed for implementation of continuous integration (CI)– a development practice. From Jenkins 2 onwards, there has been an exceptional increase of plugins to support building, deploying and automating any project. It also has a Terraform plugin for Jenkins, which allows for the creation of test infrastructure before running the build and their deletion after the build is completed.
Infrastructure as a code approach: (Terraform)
Terraform is a cloud agnostic tool for building, changing, and versioning infrastructure, safely and efficiently.
Terraform can manage existing and popular cloud service providers like AWS, Azure, GCP etc. as well as custom in-house solutions. It helps in modeling complete cloud infrastructure as a code i.e Infrastructure is described using a high-level configuration syntax. This allows a blueprint of an Infrastructure to be versioned and treated as it would not need any other code. Additionally, infrastructure can be shared and re-used.
This way the time required to manage the resources is reduced. There is no need to individually create and configure the cloud resources and figure out the relevant interdependencies, Terraform handles all of that.
This Terraform template will launch an EC2 instance running a custom Amazon AMI in a Public Subnet within a VPC with Internet Gateway attached.
The route tables associated with the public subnet can be configured to be accessed only from the organization’s public IP as a security precaution. The role required for the instance to upload data to S3 bucket can also be defined within the template resources.
The custom Amazon AMI has all the required software installed and configured to run the test suite and initiate the test execution. This saved our setup time. Once the execution is complete the reports can be uploaded to an S3 bucket put request via AWS CLI.
Version Control: (GIT)
Git is an open source distributed version control system. The latest code for the Katalon test suites is maintained in git that can be pulled once the ec2 instance is up and registered as Jenkins node, then initiate the suite execution on the latest code.
Automation Tool: (Katalon Studio)
In our setup we used katalon studio to automate our test suites. Katalon Studio is a free automation testing solution. It is built on top of the open-source automation frameworks Selenium, Appium with a specialized IDE interface for API, web and mobile testing.
The test automation framework provided within Katalon Studio was developed with the keyword-driven approach as the primary test authoring method with data-driven functionality for test execution.
Cloud Storage Service (AWS S3)
After the execution of the automated test, the reports can be easily stored in any cloud storage service depending on the service provider; here we have choices: e.g. Amazon S3, Microsoft Azure Blob Storage, and google cloud storage etc.
In this case as we had selected AWS, hence, we used S3 for storing the reports.
Amazon S3 is an object storage service for storing large unstructured data with virtually unlimited scalability. With low cost, ease of data access and security of the buckets, S3 stand a strong candidate to store the date-based reports for the executions.
Once the execution of the whole suite is complete, the reports/recordings can be maintained under a date-based folder structure into a S3 bucket for any future references.
Lifecycle policies for the S3 bucket can be used to move objects from S3 buckets to cold/archival object storage i.e. Amazon Glacier to further reduce the cost.
Conclusion
We have designed a cloud-agnostic architecture for automated test execution which is stable, reliable, scalable and with cost benefits.
As we are building and terminating the test environment dynamically at runtime, therefore we pay as we use. Also, we are not tied up with one cloud service provider, it gives us the flexibility to evaluate and choose. Also, the migration is hassle free.
The goal that we had in mind was to design a solution that gives us a maximum ROI on any Test automation project and help us deliver a high-quality product with Agility.
References
https://aws.amazon.com/blogs/apn/terraform-beyond-the-basics-with-aws/