The last 6 years have been a challenging, yet rewarding journey for us at 3Qi Labs. We have conducted millions of tests using a variety of open source technologies for numerous applications, in companies and projects of varying shapes, sizes and geographies. In creating, executing, and maintaining these tests, we’ve come to realize something very important regarding writing automated test scripts:
HOW you build your automation (The design and architecture of the scripts and supporting framework) is vital, and even more important than WHAT you are building it to do (Functional coverage, number of tests, complexity of logic, depth of testing, etc.).
Lately, we’ve been running a lot of BDD (Behavior-Driven Development) testing using our favorite workhorses: Cucumber, Ruby, & Selenium WebDriver. In testing through these (and in addressing best practices for our growing team), we’ve come up with a simple set of frameworks and rules to focus our processes and methodology. A big one for us is the 30-30-30 rule:
- A Feature File should have less than 30 scenarios
- A Scenario should have less than 30 steps
- A Step should have less than 30 lines of code
The above list refers to components of a Cucumber Feature File, and a corresponding Step Definition in Ruby. As a refresher, a Cucumber Script is called a Feature File, containing a Feature Definition, Scenario, and Step. Our Feature Definition describes what will be tested (in this case, the Zoho CRM Leads and Accounts Happy Path Tests). The Scenarios break up the Feature into individual, executable tests for specific functionalities (such as Creating & Deleting a Lead or Account). Within the scenario, the Step is the individual command that will be running for the given line (Creating a new account, confirming deletion, etc.). The example for the described feature file is as shown below:
Breaking down a Cucumber Feature File.
The Step component directly corresponds to a Step Definition file. The Step Definition contains the Ruby code that powers the individual Steps. We have highlighted the Step Definition for a Step (Then I create new lead for firstname and lastname and email and company) in the below example. For more detail on the concepts regarding Feature Files and Step Definitions, take a look at our post on Cucumber.
The corresponding Step Definition file, powered by Ruby code.
In developing the 30-30-30 rule, we focused on a few key automation concepts, and observed how applying the rule created a higher quality automation experience:
- Maintainability: Maintenance accounts for the bulk of the ongoing cost in any automation effort. As with any other coding effort, less is more, and automation is no different. Fewer lines of code for each method/step makes it easier to maintain, debug & subsequently enhance.
- Readability: The more steps a test has, the longer it is, and the harder it is to follow. This is especially the case for someone who is unfamiliar with how a Feature File reads. Minimizing the amount of lines you use not only increases comprehension of the test, but allows you to incorporate concepts like structured scripting (grouping common tasks in linear scripts into single line functions), making your tests more dynamic.
- Transfer of Ownership: We are constantly building automation and handing it off to customer teams; the success of the transfer almost entirely depends on the complexity of the automation code. If the code is complex and cluttered, it will be unnecessarily hard to understand. Maintenance and enhancement on that code will be just as difficult, especially for someone who did not write it.
Now, we know nothing is set in stone, and we realize that the 30-30-30 rule has to be broken from time to time. Depending on the complexity of the applications and scenarios involved, you may have to add more steps/code/scenarios to support your testing. These cases vary, and we ensure that our analysts have a concise reason for needing to break this rule.
Take time to think about this rule, and how it can pertain to writing automation test scripts. If you find that your code is more often than not cluttered, hard to read, and seems like a chore to debug/enhance, try applying the 30-30-30 rule to your scripts. Feel free to let us know if there are similar rules that you enforce in your automation approach.
