Implementing WAVE Accessibility Testing with Firefox Toolbar & Cucumber

WAVE (Web Accessibility Versatile Evaluator) is a free tool that was developed by WebAIM, which evaluates the accessibility of online content. Within the context of test automation, WAVE can be implemented by loading its’ Firefox plug-in and running the compliance report on any site rendered by the browser.

This proof of concept leverages both the WAVE Firefox toolbar and the 3Qi Labs’ Awetest framework, to demonstrate the ease of combining both technologies to produce a compliance report outlined in Section 508 and WCAG 2.0 guidelines. This document will provide an overview of the WAVE Firefox toolbar, and the implementation of the tool using Awetest and Cucumber.

WAVE Firefox toolbar

The WAVE Firefox toolbar is a plug-in that allows the user to run a web accessibility compliance report by navigating to a site and executing the appropriate commands. This compliance tools is based upon the guidelines set forth by Section 8 of the Rehabilitation Act of 1973 (US) and by WCAG 2.0 of the World Wide Consortium (W3C) – the main international standards organization for the world wide web.

Different web accessibility elements to consider for people with either Visual, Auditory, Motor, or Cognitive disabilities can be found here.

The latest build of the WAVE toolbar plug-in is version 1.1.8 and can be installed here. Once the WAVE plug-in has been installed, a new toolbar will appear within the Firefox browser.

The WAVE plugin displays in your toolbar

There are two workflows to run a WAVE compliance report manually: From the Toolbar, and Using Shortcut Keys. See below:

WAVE Compliance Report Instructions – From the Toolbar:

Open Firefox browser
Navigate to the site which the report will be ran against
Click on the ‘Error, Features, and Alerts’ button in the WAVE toolbar
Verify compliance report
Close the browser

WAVE Compliance Report Instructions – Using Shortcut Keys:

Open Firefox browser
Navigate to the site which the report will be ran against
Click ‘Alt – T’
Click ‘WAVE’ option from the sub menu
Select ‘Error, Features, and Alerts’ option
Verify compliance report
Close the browser

Awetest Implementation

Awetest uses a combination of technologies in order to automate the WAVE compliance report workflow, these technologies include: the Cucumber script, the Ruby (Watir) script, and the WAVE Firefox plug-in. This will discuss in detail how each of these pieces fit together.

The Cucumber Script

The Cucumber script contains the high level scenario that contains the test’s workflow which is associated to its respective Ruby/Watir step. Within Awetest the Cucumber scripts exists within the Test Cases/Category/Scripts section of Awetest, and can either be uploaded or created within the application.

The cucumber scripts are highlighted in the above image

Below is the Cucumber script which implements the Using Shortcut Keys workflow mentioned above to verify Web Accessibility for the California DMV website as an example using the WAVE toolbar.

This script verifies web accessibility using the WAVE toolbar.

Within Awetest there are predefined steps that do not need to be recreated within the Ruby file, which correspond to the following:

When I go to the url “<url>” ::line 7
Then I click “<link>” ::line 11, 15, 19, 23
The I close the browser ::line27

The Ruby Script

The Ruby script contains the test steps which the Cucumber file executes in order to perform the outlined test case workflow. Within Awetest the Ruby file exists within the Assets/Step Definitions section of Awetest, and can either be uploaded or created within the application.

The step definitions contain the steps executed by the Cucumber feature file.

Below is the Ruby script for the workflow described above.

Here is the full step definition file, in Ruby

The step, “I launch a FireFox Browser instance” sets the FireFox browser profile to the WAVE plug-in (wavetoolbar1_1_8_en.xpi). This step is needed due to the fact that by default the Watir framework launches a browser with a clean profile, regardless if the user has installed any browser plug-ins using the default profile.

The step, “I run the WAVE plugin” executes the WAVE compliance tool using the shortcut keys, by explicitly sending the correct key sequence.

The step, “I verify with a screenshot if there are any errors” captures all the WAVE notification elements and loops through items and identifies if an “ERROR” tag exists. In the event that a WAVE notification element contains an “ERROR” tag, a flag is set to true. If the flag is true then a screenshot is taken and a failure is thrown with the message “At least one instance of a WAVE validation failure is present”, otherwise a screenshot is taken.

The WAVE Firefox Plug-in

In order to run the WAVE Firefox plug-in, the script needs to reference a copy of the WAVE toolbar xpi (wavetoolbar1_1_8_en.xpi) file which can be found here at WebAIM’s site. In order to reference this file in Awetest it needs to be uploaded to the project’s respective Assets/Variable Documents location.

Awetest Report

Upon running the script in Awetest a report is generated, and is located within the Reports section of Awetest.

The actual report indicates the status of each test step executed with a screenshot where “I verify with a screenshot if there are any errors” is explicitly called.

Below is a sample screenshot of the verification point in the script. These images are accessible by clicking on the camera icon on the same line of the verification step.