Exploratory Telescope

As DevOps continues to expand, understanding how each team’s function contributes to your end goal is imperative. When looking at QA, it is tempting to push for your testing to be 100% automation.  After all, if automation means you’re hitting your test case completion goal faster, who’s complaining? The problem lies in the assumption that there is no longer value in manual testing. The truth is the exact opposite; automation has freed up more resources to be designated to testing manually, specifically exploratory, which offers insight and procedure that cannot be automated.

Automate Everything?

We love automation. After all, automation enables your testing to be continuous, scaleable, and efficient. Our main product, Awetest, is a platform entirely based around building, managing, and executing your automated test scripts. However, there is a time and place for everything, and automation is no exception to this.

We’ve previously discussed challenges with automation implementation, and specifically when and when not to automate. For some complex scenarios, the effort expended in building an automation script for your test case will expend more resources than running it manually over a period of time. If one of your automated test cases fail, and you find the reason is due a new defect, how will your preexisting automation scripts allow you to gain more insight? This is where manual testing (and more specifically exploratory testing) comes in.

Exploratory Testing

In our previous post on exploratory testing, we discussed the many advantages to be gleaned from incorporating exploratory testing into your test plan. These included manually finding defects by testing various scenarios around your functionality, and subsequently expanding your test coverage through test case design. In order to have the resources to do this, you must have some sort of automation framework in place. At the bare minimum, ensure that you are automating your core, repeatable functionality.

The results of these automated tests will provide context for exploratory tests; an area of your application with noticeably more functionality errors is something worth investigating. Focusing resources on exploratory testing around the area can lead to a breakthrough and solve the issue faster than re-running your automation test cases.

While improvisation and creativity do play their part in exploratory testing, there are benefits to having structure. Designing scenarios specifically for exploratory testing, scenarios which aren’t covered by your foundation of automation, will let you cover flows from a user’s perspective, giving you the window to catch something not explicitly defined in your predefined test cases.

Blending Manual With Automated

The value of both automation and manual shine when they are combined. With your primary test cases automated, you will free up more manual resources to focus on exploratory testing in parallel. This creates a positive feedback loop: automated tests are executing while you simultaneously design new test cases through exploratory testing, allowing you to write and execute more automated tests, etc.

When your automation is geared towards minimizing execution time, you will be creating more time for exploratory tests, expanding your test coverage. As you increase your test coverage, removing redundant and outdated cases based on your current application build (and your newer test cases) is necessary to avoid wasting testing resources. This has the added bonus of providing future testers a thorough test suite incorporating your exploratory test flows.

The key to success in both manual and automated testing is proper planning. Automation is not the replacement to manual, it is the complement. Understanding that they should be incorporated, side-by-side, will allow you to leverage your testing resources to drastically increase your test coverage, and subsequently maximize the “Quality” of your DevOps initiative.