Sikuli Best Practices

If you have been struggling to build complex interactive UI application tests with DOM based technologies like Watir or Selenium.

You have come to the right blog, because project Sikuli is transforming the test case creation process by adding visual image verification to the arsenal of test automation.

We have been working with Sikuli for a good while now and have already blogged about some of our best practices for using Sikuli.  This post is a cohesive collection of all things Sikuli.

In the following two minute demo we will take you through a very simple Google Search Script. Although the demo seems simple it will touch upon most of the Sikuli Best Practices to follow.

A Sikuli test case might look as simple as this…

Sikuli Best Practices

Click vs Double Click

Remember to use double click when trying to access applications. Because Sikuli is moving the mouse to the portion of the screen it matches and then preforming an action on the portion of the screen, if you are trying to launch an application from the desktop, in this instance Google Chrome, you will need to double click the image to launch the application. So if some click in your script isn’t working try a double click.

Waiting for Page to be Fully Loaded

Like any testing framework Sikuli has to wait for loading of webpages and local applications. This means that if you launch your Google Chrome search page and try to execute your script before Google can retrieve all of your search results then your script could raise an exception because it has run to fast and is further down the script.

To get around this, Sikuli has included the wait() and waitVanish() methods. These allow you to specify whether to wait for a certain image to appear before moving on in the script or waiting for an image to vanish before continuing. These can be very powerful in tackling the page loading times but it not the only was to tackle the problem.

In fact, in the case of the above Sikuli Google Search Script, because of Google’s auto-complete javascript begin active, the second you start typing into the search bar the page changes to a results page and the results are updated live upon every keystroke. So if you were to try to validate your results without waiting your script would raise an exception. Also if you try to use Sikuli’s image wait() method to look for an image on Google’s landing page it will not work because the second you start typing you are taken to the results page.

In this case, it is important to remember that Sikuli is built using Python. Meaning, we can use most python native commands within Sikuli’s IDE. With this script we simply changed what Sikuli was waiting for from an image to an integer. Sikuli will then follow Python’s wait method and wait for the time set as an integer. This lets our script have the time it needs to get around Google’s auto-complete JS.

Improving your image matching

Similarity Index

If you are running into a wall with a certain image not getting matched up by Sikuli there are a few things you can try to improve your chances of getting a match. One thing you can try is increasing or decreasing the matching threshold, or “Similarity Index”, which will let you tune the efficacy of your images within your script.

Clean Desktop Background

Another thing you can try is eliminating complex desktop backgrounds that may be confusing Sikuli’s image recognition algorithms. We set our backgrounds to a plain grey background when working with Sikuli scripts locally or on a virtual machine. We also try to eliminate any desktop icon’s we don’t absolutely need to give Sikuli less to match against.

Target Offset

Target Offset can be used to dial in the place on the screen where Sikuli will actually be clicking. This can be very helpful when you need to click on a very specific part of an image to get the reaction your test case is expecting. Target Offset is designated using X and Y coordinates from Sikuli’s grid view.

Using Sikuli’s Hotkeys

This is something we found from our own research on Sikuli’s best practices. We had already been using them but hadn’t really considered that they could be considered in a best practice post. Thanks to Mario Maldonado for illuminating this one for us.

For a documentation on Sikuli’s hotkeys check out this page.

More Sikuli Documentation

Thanks Again Mario Maldonado!
Enhanced by Zemanta