“Frameworks” is an overused, yet necessary, part of any successful automation effort. Automation has been around for a long time (Mercury Systems which is part of HP since 2005 was founded in 1980!) and many of the approaches are the same as they were back then, but the underlying technologies and tools have evolved. So today, we have frameworks to aid in simplifying and scaling your test automation efforts. The 5 primary test automation frameworks we’ll be exploring are: Linear, Structured, Data-Driven, Keyword-Driven, and Hybrid.

Linear Scripting

This is the simplest of all frameworks, and very similar to the Record-Playback mechanism – where every single interaction is captured and detailed out in the script/feature file. Each step would be detailed along with checkpoints and validation steps. Linear Scripting is a good way to get started with some very basic automation but is far from scalable and cannot be used to build out any meaningful automation coverage. Yet, it is crucial to understand because it forms the basis for all other frameworks discussed later in this chapter

There are a few characteristics that stand out in a Linear Scripted test- First off, there is a separate line/step for each interaction, which makes the script very verbose. Next, each and every item is hard-coded, which is impractical for the test being reused.

As the name suggests, Linear Scripting, is an incremental automation approach where every new interaction would have to be added to your automation tests – which while easy to do, will result in a rapid increase in volume of scripts/feature files as you expand your coverage.

Pros Cons
  • Fastest way to generate script
  • Little reuse of scripts
  • Automation expertise is not required
  • Test data is hard coded into the script
  • Easiest way to learn the features of the Testing Tool
  • Maintenance nightmare

 

Structured Scripting

Structured scripting is a framework which organizes a code base into reusable routines, called custom steps within the context of the Cucumber language.  Custom steps are functions that can be called throughout the script to build efficiencies in code reusability and maintenance.

An approach to create a structured script would be to first create a linear script and refactor common routines into custom steps.  Once the custom steps have been created, other instances of the same routines that have been defined can be replaced with the custom step.  By declaring one custom step to represent numerous lines of code builds reusability.  The fact that if this function breaks, the user only needs to address the issue in one place, as a result building efficiency in maintenance.   As the script grows and becomes more complex, the need for creating reusable code becomes more apparent.

Below you can see how tasks for a simple ‘Login & Create a lead’ script on Zoho.com have been grouped into functions creating a clean looking script.  In the below example each of the underlying steps that are being rolled up into a function/custom steps have been commented (#) out.

Cucumber's feature files provide a clean, simple language front end to the Ruby code powering it.

The steps in the above feature file have been condensed into a few lines

So the resulting feature file looks as follows. Here is the testament of the power of frameworks – a test that (using Linear Scripting) was 40+ lines long is now down to 8 lines – That is an 80% reduction in the steps!!

From the front end, the steps will appear as such.

Here is what the finalized feature file will look like.

Pros Cons
  • Higher level of code reuse as opposed to Linear Scripting
  • Some technical expertise is necessary to write scripts
  • Scripts are less costly to develop due to higher code reuse
  • More time is needed to plan and prepare test scripts
  • Easier script maintenance
  • Test data is hard coded within the scripts

 

Data-Driven Testing

A Data-Driven testing framework refers to the repeating of the same test case scenario using different data sets – making the test case, data centric. As a result, a QA Engineer is able to test multiple scenarios using one single test case. A Data-Driven script is parameterized, replacing hardcoded values with variables, in which the data may be loaded from a number of sources, including: the Gherkin file, text files, Excel files, CSV files and databases. Upon execution, the test case will read a data row, send that data to the script, and will repeat until there is no more data to be read

Pros Cons
  • Changes to the scripts do not affect test data
  • More time is needed to plan and prepare both test scripts and test data
  • Test cases can be executed with multiple sets of data
  • Relative level if technical expertise is necessary to write these scripts
  • Multiple test scenarios can be executed by varying just the test data in the external file
 

 

Keyword-Driven Testing

A Keyword-Driven testing framework leverages an external file to reference the functionality of the script.  It is important to note that this is different from a Data-Driven testing framework, as one focuses on functionality and the other focuses on data.  Within a Keyword-Driven framework the QA Engineer is able to manage the script through an external file.  The file format of the external file is typically an Excel file or CSV file.

Keyword-Driven, also known as Table-Driven, scripting requires the development of keywords and data-tables, that are independent from the automation tool itself. Keyword-Driven scripting requires two steps:

Step 1: creating the data-tables to be used for running the test

Step 2: referencing the external data-file in the script to access the information to be run.

Pros Cons
  • Provides high code reusability
  • Initial investment is high
  • Centralized test data
  • High automation expertise required
 
  • Benefits of investment only seen if app is large

 

 

Hybrid Test Automation

A Hybrid Test Automation testing framework leverages all frameworks based on the tests that the QA Engineer is trying to implement.

1

The great ability of Hybrid Test Automation is it can be customized to your specific needs, for your requirements. You can pick and choose the capabilities of each framework that you want for your specific set of requirements, the goal of doing this is to pull on the strengths and mitigate the weaknesses of two or more frameworks. So in the end, your Hybrid Test Automation Framework could look completely different from another engineer’s framework since it is so customizable and there is no predefined structure specific to a Hybrid Test Automation Framework.