The structure and frequency of testing automation is often debated. Should you automate your tests? How should you automate? When and which tests should you automate? The debate can go on for days due to varying test results and reliability. To further complicate matters, conducting tests through an agile environment seems to make test automation a necessity rather than an alternative, even though it’s more complex to execute.
Test automation includes a combination of white-box (code-based application) and black-box (requirements and functionality) tests. White-box testing can be a bit easier in this context because the tests tend to be less fragile to the testing environment, which makes them less costly. Because black-box tests tend to be UI (user interface) and UX (user experience) based, they are more expensive to both develop and maintain.
Agile test automation adds to the traditional automation tests by requiring shorter, quicker iterations that make the timing of the tests absolutely crucial. The feedback from the tests now has to be instant in order for the testing to be effective in the new agile environment. With such a daunting task, there are a few things you will want to consider if you are interested in automating tests in an agile environment:
- Organize and re-organize your tests. Possibly the most crucial piece before you start testing, manually or automated, is making sure you have your tests categorized and organized in the right place. Most often tests are categorized by Core/Critical Path, Release and Regression.
- Don’t try to automate everything. While the need to be faster and more efficient is important in an agile testing environment, it is actually more effective to run some tests manually in addition to the automated tests. For example usability tests and acceptance tests are best suited for manual testing since they rely on the user experience rather than technical program capabilities. On the other hand, load tests (also known as “stress tests”) can be more automated (or a combination of automated and manual) since the tests would be looking at the applications ability to handle network pressures.
- Keep your tests flexible. Lots of things are going on in agile testing situations so the need to be nimble is even greater than traditional testing. As you are determining how best to automate or manually execute your tests, it’s important to be flexible enough to make adjustments on the fly. For example, when certain defects are fixed and deployed during the initial phases of your testing, you may have to manually retest the fixes to ensure the changes won’t be missed by the automation. Or perhaps you had planned to manually test for functionality and automating the test would be faster and produce better results. In any case, staying flexible from the very beginning ensures that your tests produce the best results possible.
- Maintain documentation.Ensuring your application requirements are well defined and documented will reduce any confusion when it comes to your validation. It is often the case that a new tester isn’t clear on what the expected result for an interaction within an app should be – they may be unable to reach the correct co-worker for clarification, and end up wasting time trying to figure out something that should be readily available common knowledge. Having a central document that can address these issues will save time in test development and execution.
- Continuous Integration/Delivery. Having the proper tools in place to ensure your delivery/integration is continuous, along with proper version control/management, is key. This will allow you to respond efficiently and push through an update or fix to your app quickly (especially important if this was stemming from a customer reported issue). Tools such as Jenkins are commonly used across agile teams to maintain a CD/CI process.
Agile test automation in agile development environments is the future of software testing, in general, and testing automation. But it doesn’t have to throw a wrench in your current procedures. As you tweak your team’s responsibilities and consider tools make sure you are doing it the right away from the very beginning. Start by organizing your tests into categories so that the leader and team know where each process happens and how. Continue all of your traditional manual tests and your automated tests in traditional environments, even while making the shift to more agile environments. And whatever you do, make sure your tests are flexible throughout to address any needs for changing. That really goes for any type of software testing, but can be even more helpful in agile environments.
