5 Situations When You should not Consider Test Automation

The overwhelming adoption of DevOps practices and the proliferation of agile as a standard development methodology has crunched software development cycles and dramatically accelerated releases.  This has made testing and testing fast a necessity. So much so, that it’s now impossible to think of software development without factoring in test automation. Owing to the immense benefits test automation brings to software development it’s not a surprise that the global automation testing market size is expected to grow from USD 8.52 billion in 2018 to USD 19.27 billion by 2023.

Clearly, good and comprehensive testing lies in the heart of software development today. However, while test automation is a powerful weapon that can help in driving software quality, not all test cases are meant for automation. Sometimes knowing when not to automate can be just as important!

The road to 100% test automation leads to testing failure

The surface area of testing is increasing at an incredible rate especially as software applications need to interact with one another through API’s. However, aiming for 100% automation is like chasing a mirage –destined for failure. The focus for organizations thus should not be on achieving 100% test automaton but rather on identifying methods to achieve the best possible testing outcomes. Identifying the candidates for test automation is as important as building the test automation itself. Features, Functionality or Scenarios, feature flagging or flighting that need faster feedback, increased test coverage, and repeated testing are great test automation choices. .  Tests that need an investigative approach and require the human touch to spot failures, like exploratory testing, user experience or usability testing and accessibility testing are best left for manual testing.

The roller coaster that is regression testing

As surprising as this may sound, there still are many situations in the software development lifecycle when future iterations do not touch the old code base significantly. Small and self-contained software projects or projects that have extremely clearly defined requirements are good for manual testing. The regression testing effort in these situations is reduced considerably as the development for each phase is completed and then accepted only after testing. The time and effort advantage that test automation offers is not significant in such environments, especially when we factor in the effort it takes to develop a testing suite.

When change is a constant – Frequent functional changes and a fluid GUI

In case the GUI is not frozen, or the development team is still battling ongoing dynamism in functional requirements, test automation can be a challenge to implement. Building a test automation suite, the test cases etc. takes significant effort. Once the suite is built, continuously tweaking it to adjust to the changes in the project scope is not easy and certainly not advisable. The testing team must be involved from the very beginning of the project to design, plan, and build an effective and robust automation testing suite. Failing that, the developed test automation suite will most likely be incomplete and, hence, ineffective. In such cases, test automation will be more of a burden than a boon.

The positive about Random Negative Testing

Most applications need random negative testing to see that invalid inputs or unexpected user behaviors are handled appropriately. The aim of negative testing is to ensure that invalid data in areas such as in populating required fields, correspondence between data and field types, data bounds and limits etc. does not lead to the application crashing. It helps in identifying the weak points of the application and hence helps in improving the application quality. We can easily automate random negative testing to gain greater test coverage and track errors. However, automating random negative testing can be an unnecessary overhead when all you need to do is test an interface, its user-friendliness or the navigation among web pages.

The A team – Does your testing team have the superheroes it needs?

This is the elephant in the testing room. Test automation is impossible without the right set of resources capable of driving successful implementation and outcomes. Test automation does not just need testers. For most test automation efforts to be successful, you need the developers to join that tribe. Remember that the test automation suite is also a product. It has to be supported and has maintenance needs just like any other software product. If an organization does not have access to trained resources who can not only build but also manage, support, and maintain the test automation suite, it is best not to consider test automation.

Along with this, there are times when tests are likely to be only run once or very few times on one iteration. In such situations, it makes sense to not expend time, effort, and money on automation initiatives -just go manual! When choosing between automation testing versus manual, start with answering a few questions. The answers to each of these will have a bearing on the ultimate effectiveness of test automation. This may help you take the right decision on whether to automate tests or to employ manual testing:

• Will there be a productivity impact? How will the productivity increase?
• Will there be increased test coverage and test accuracy?
• Do the tests need a large data input?
• How often will the functionality change?
• What will be the effort of script maintenance?
• Do you have an intensive GUI?

Our suggestion? As a thumb rule, it makes sense to automate functions that are 80% stable and unchanging. It is also important to account for script maintenance and to ensure that this can be done easily -functions can change and hence the scripts need to be reused and updated accordingly.

It’s time to take a strategic approach to testing. Finding that balance between test automation and manual testing is now the secret to achieving testing success and great products.