The Issue

After developing an understanding of the technologies powering automation, you can start to think about how best to implement and optimize your scripts. Writing, running, and maintaining automation utilizes resources, and attempting to blindly automate everything is not an optimal approach. Instead, having a high-level understanding of when to/when not to automate will allow you to streamline your testing efforts, and eliminate wasted time in script development.

When To Automate

Let’s start with a list of test characteristics that define good automation candidates:

• Repetitive tests that run for multiple builds
• Tests that tend to cause human error
• Tests that require multiple data sets
• Tests that take a lot of effort and time when manual testing
  

The extent of automation is dependent on a number of different factors including (but not limited to) budgets, project timelines, resource count, skill set, existing toolsets, geographic distribution - all of which need to be factored in when deciding when, where and how much to automate.

Ultimately, tests should be reliable. You can spend years automating your tests but if the tests are not of a good quality then they are effectively useless, as they are unable to fulfill their purpose and provide accurate feedback.

When Not To Automate

Now that we have an idea of when we should be automating, let's look at when NOT to automate.

Some scenarios are simply not feasible to automate; they are either actually impossible due to technological limitation and the complexity of the scenario, or the resource cost of automating it greatly outweighs the cost of a simple manual test. Developing a way to automate this scenario is not worth it when compared to manually testing it with your device. Here are essential questions to consider before automating:

• Is the application at the end of its lifecycle and about to be discarded? Investing in automation might not give you the desired ROI.
• Are the manual tests are extremely simple? For example, if all it takes to run the test is to reload the browser then the cost of automation wouldn’t be worth it.
• Is the code is already extremely simple and easy to follow? If so, then there should already be very few opportunities for bugs, and automation could be an unnecessary use of time (e.g. Static Websites).
• Does the functionality of the software see a lot of changes and experiments? If so, then it might not be worth it to have to continually change the automation code as the functionality changes.

The Takeaway

Knowing when you have enough test automation coverage will more often than not be a challenging subject to address. In developing your automation suite, you want to ensure that you are not wasting resources on attempting to automate cases that are not feasible to automate. At the same time, a test case that IS feasible to automate may not be worth the effort. Understanding why you are asking yourself these questions is the first step to developing your understanding of automation practices.