10 Ways to Increase Automation Test Coverage

By Haritha Hari, Lead QA Consultant at ThoughtWorks Technologies

A Quality Analyst (QA) like myself would probably have faced one or more of the hurdles that I have articulated below -

1. Long running projects with extreme delivery pressures, multiple services and huge data involvements
2. An application’s code whose initial functionalities have been developed by the client and have less test coverage than you are accustomed to
3. Legacy applications that are not unit testable and applications whose entire logic lies in database queries

To make life a little easier, listed below are 10 tips to increase automation test coverage. These ideas are easy to incorporate, will improve test automation coverage and will reduce manual testing effort. They should also enhance the efficiency of a continuous delivery practice.

1) Introduce a new ‘To Be Automated’ lane, after ‘In Testing’ on the iteration wall. This will ensure all story cards have automation tests written before they are moved to the ‘Done’ section.

2) A code coverage tool could be made part of the build pipeline. The latter can be made to fail if there is a drop below an agreed benchmark. This will ensure stakeholders are held responsible for coverage (and test automation). This approach will also identify automated areas and limit the scope of manual regression cycles.

3) Create an automation backlog board out of the coverage report. Analyze the report and link it to features that can then be converted to test cards. This will communicate what has and should be covered in automation tests, creating visibility on the pending automation tests.

4) Automating the above backlog is a continuous effort. And, testers should include developers either in between iterations or during the regression phase. A target of two test cards per iteration can be set for automation. Additionally, the client’s buy-in for an iteration timeline can help clear a good majority of automation backlog.

5) Ensuring product quality is a collective responsibility and the entire team should be aware of the test pyramid. Encourage developers to write unit tests without moving it up in the test pyramid, thereby increasing investment.

6) Monitor Git for every story code committed to understanding the test coverage, with respect to already written unit and API tests. This helps testers write tests only for the non-automated part of the check-in. Also, by comparing the release branch and master branch, testers optimize their effort only on the impacted areas for manual regression.

7) Capture test cases during story creation. Developers then, tend to adopt a tester’s perspective and ponder over what tests should go into which layer of the test pyramid. This is a good beginning for a shift-left test principle.

8) Introduce the acceptance criteria ‘write API tests’ to ensure positive functional tests are given importance even during the development phase. This will serve as a quick check to developers who may not have mastered the process. This eases testers’ efforts when catching up with tests for all the edge cases in a story.

9) Execute tests in Devbox or Volleyball to make certain that all functionalities are working fine and no tests are broken. This is also a platform for testers and developers to jointly validate if tests are being written in the correct test pyramid layers, and also if the data setup and assertions are per standard.

10) Include QAs in estimation sessions. This can influence the estimation points. If there is an additional effort in data setup or a change in the testing approach, stakeholders should consider this while estimating for a story. A simple code change might lead to a huge functional impact and thereby increase testing effort which means a 1 pointer story may not be a 1 pointer, after all.

The measures that we have discussed here,when introduced at the right time will help clear automation backlog on any project. When all stakeholders are motivated to follow processes, QAs can achieve the dream of an excellent automation code coverage and increased ROI.