Automation Best Practices: Building From Scratch

by admin on April 18, 2012 in Best Practices, Blog Series, code examples, No Comments

Visit this post for an index for our

– Best Practices for Achieving Automated Regression Testing Within the Enterprise

This is perhaps the most critical aspect of a good test automation implementation. The decisions you make during the ‘Build’ phase of the implementation will impact you throughout your automation life-cycle. This means the initial building out phase of a proper test automation implementation requires a number of things which we will be covering in this section of the our blog series: Best Practices for Achieving Automated Regression Testing Within the Enterprise

Before the build out phase of good test automation can begin you’ll need a starting plan for testing the application. This means defining initial coverage goals for the automation and setting  some preliminary milestones for effort comparisons. These should be small at first; we might set our first milestone as simply as scripting a  variable driven login method to gain access to the application for our tests in the future. It’s important to let milestones build on themselves and to add as much value down the road as possible and to build simple test cases that can be moved into a central utility file and refactored as one line function calls in upcoming test cases.

Next, there must be an exploratory phase of testing the application and inspecting the elements to evaluate the effort needed to build out automated test case. Lastly, environmental details need to be agreed upon so that test cases are built to the same standards and specifications as the application. The good news here is that Awetest is flexible enough to allow for sudden changes in your automation (and underlying Development) process but “design” and “architecture” decisions made during the Build phase will impact the time, effort & performance of your automation infrastructure in the future.

Bottom Line: Automation is a development exercise and should be treated as such.
Based on our experience here are some of the key learning and findings for Building your automation from the ground up:

2.1       Plan Your Automation

Before diving head first into automation, be sure you understand the monetary, time, and resource investments needed to properly implement test automation in the development / QA cycle of your company.  To help you get started, the Awetest team provides a range of professional services that include Quickstart, Migration, and By-the-Hour Guidance packages to help companies build their test automation implementation from the ground or porting previously built test cases into the Awetest system to the eventual routine maintenance always needed for keeping everything running smoothly.

Choose a hybrid testing framework that supports Data and Keyword-driven Testing as well as Modularity and Model-based Testing. With Awetest’s ‘Assets Tab’ spreadsheets and custom method libraries can be saved and called from within automation test cases.

2.2       Define Target Initial Coverage

Having a planning session of what needs to be automated before you start trying to automate every manual test you can think of with these five key principles of automation in mind:

• Time Savings – Time to perform the manual test vs. scheduling and viewing results of an automated test
• Repeatability – Ease of preforming the manual test with the exact parameters every time.
• Reusability – Does this happen over and over within a given work flow. Test that require different data sets to pass are definitely worth automating.
• High Risk – If there is any area of the application you need to be absolutely sure functions properly
• Configuration Dependencies – If you need to test on multiple operating system, browser, or device combinations.

A good place to start would be taking a look at your existing test cases in Quality Center or other Application Lifecycle Management systems being used and then identifying the ones that fit your automation criteria. This will allow you to get a good sense of the percentage of total test cases you will want to automate and a preliminary sense of the relative effort it will take to automate them. We recommend starting the build of your automated regression tests with a test case that is very well understood from both a development and business need standpoint like a login or create account test case.

2.3       Don’t Automate Everything

Sometimes the best automation is no automation – spending a week to script a difficult rare activity is not a good use of time – do that activity manually during test runs for now and come back to the problem later. Many times just taking a break from a particular problem point can clear the brain enough to take up the challenge again. Remember 99% of web interactions can be automated with endless time to build the test scripts but if that is holding back your overall automation efforts it is completely okay to shelve the problem temporarily, test the problem manually, and get actual results. In practice we find that is you are spending more than x6 the amount of time to automated a test verses test it manually then you might as well put it in the manual testing bin and move on.

2.4       Organize your tests

Organize your test cases into thoughtful categories, core functionality, problem areas, unit tests, and area’s currently under development. This makes testing areas of the application that the time of release more target-able and can prove a real time saver with agile development teams.

Awetest come with a test case management system for organizing your automation test scripts into meaningful categories that will help your testers build easy test cases at the drop of a hat. This system also comes with drag/drop/reorder capability to set the order of the test cases/scripts.  This gives you flexibility to reorder categories or test cases as required and is useful when you want to add additional test cases to your regression suite in the future. This can also be used to order the test cases to be executed in a regression job run. You can use this capability from the test cases page or schedule jobs page  and your changes will be saved throughout the application.

2.5       Don’t Reinvent the Wheel

Always use an extensible test automation framework with the best testing technologies available. Awetest is built from the ground up using variety of different open-source test automation frameworks with their own strengths and weaknesses. We aren’t trying to reinvent the wheel here, but we are trying to create the best quality tire on the market, and that means spending time improving where the rubber hits the road and not trying to reinvent rubber entirely. This approach means, that with Awetest, there is no real need for software test professionals who already use these popular technologies to relearn anything new and more importantly, this means test cases already built in using these frameworks will easily port over for implementation in Awetest.

When Creating Test Cases in Awetest, you have the option to select the script type for that particular test case.  Currently there are 4 options for regression test cases. These are Awetest Native DSL, and raw Selenium, Sikuli or Watir scripts allowing you to leverage any existing test cases you might have built in those testing languages directly inside of Awetest with the same maintenance capabilities and level of reporting as Awetest’s Native DSL.

Not only that, but you can mix and match your testing languages from within one script file. Awetest has the ability to call Sikuli scripts from within an Awetest script. This means you can utilize Sikuli for individual browser interactions without having to script the whole test case in the Sikuli language.

Figure 1 Sikuli script being called from within an Awetest Script.

Use this link for a quick reference index of all posts in this test automation blog series: Best Practices for Achieving Automated Regression Testing Within the Enterprise

ABOUT THE AUTHOR