Let's break the ice - To Automate or Not to Automate?

Test automation could be a key for success in software development. However an incorrect analysis of automation feasibility could result in unnecessary efforts by the Test Engineer.

Anyway, an unstable product will cause the test to not be able to be run. The test will not be able to be run manually either. So this case, completely rejects the possibility.

Feasibility will be determined according to the project schedule, technological impediments, and business value that the automation suite provides. If our feasibility analysis gives us good results, the next step should be to focalize in “Designing” a maintainable framework following international standards and patterns.

Key factors

Would you automate a product that takes 2-3 months to be developed?

Project schedule is a key for ROI. The number of times the tests will be run will be crucial in relation to the time invested in creating them.

Would you automate a product being developed in an obsolete technology?
Developments not within technological trends will conclude our analysis with negative results. Obsolete technologies don’t have the support of the community and it will be annoying to solve issues yourself.  

Would you automate any test cases that only you (the tester) will run?

Business Value refers to the benefit that a client (outsourced scenario) or the team receives of an automated bunch of test cases. In many cases, the automated test suite is part of the deliverable product and the client could use it in the future.

Would you automate the validation of the content of an image, or an exported PDF file that includes the information you need? Many times we face limitations that technology can not resolve. On the other hand, Human beings get bored running manual test cases but an automated test is not going to find new bugs. Human beings have creativity and some test must be done by a person. Is the application failing because we found bugs/regressions or because our framework is failing? An incorrect framework design will lead to false negatives, we should design a framework taking care in reporting and the right validations. From a QA perspective, if we get a low automation feasibility, it will make sense to focalize the QA effort in manual testing. Otherwise, the procedure to apply automation from scratch to a project will be:

1. Automation Feasibility validation. Run checklist.
2. Customize your development process to provide support to automated testing.
3. Tool choice. Use an existing one or create your own (language and libraries - i.e: Webdriver)
4. Isolated environments creation (Data and cleanup process).
5. Develop your test cases and run them.
6. Identify test cases candidates for automation.
7. Develop your automated scripts.
8. Execute your scripts against 1 environment.
9. Refact your scripts to give support to other environments.
10. Run test suite crossbrowsing (i.e Selenium grid)
11. Configure CI server
12. Report results.

This blog we will describe each point listed above.