Agile: Identifying when and what to automate

Identifying when and what to automate is critical to success of any automation endeavor. Before we can identify these when and what, it is important to identify the challenges that come across in automation:

• The initial builds coming out in a sprint\iteration, tend to have defects, mostly blockers not allowing automation script development to proceed.

• In the case of sprints\continuous integration builds do not have fully developed features initially. There maybe\are significant changes with subsequent builds, including naming conventions for UI elements which results in a lot of rework for the automation team.

• Testing of screen\UI layout is nearly impossible\very costly with automation and requires at least one round of manual testing.

• Generally manual test cases are very granular and detailed. Manual cases include a lot of positive and negative tests so that the feature is thoroughly tested. Automating all of them as they are provides very little business value. Automation testing requires automation scenarios that are mostly business\user scenarios. Automation scenarios can be developed from the granular manual test cases. The granular manual test cases should be covered in automation only for regression testing.

• Automation should be implemented selectively and the shelf life of the product should be an important factor in making this selection. It is important to understand that automation is not required everywhere and every time.

The summary of the above points for test automation is as follows:

• Automation should begin only when the application is ‘automation ready’. The efficiency of automation effort is directly proportional to the stability of the application\functionality.

• For majority of the applications, 100% automation is not possible to achieve. Even if it is doable, it does not provide ROI/business value.

The proposal is as follows:

Test cases should be executed manually in current sprint\iteration. During the current sprint the test scenarios which are suitable for automation should be queued for automation in the automation backlog. The exercise of selection of the automation candidates/scenarios should involve automation team members. The items in the backlog should be prioritized and then picked up for automation.