As useful as Sikuli appeared at first glance, I quickly found it to be a real handful to implement and maintain. In both cases, implementation and maintenance, I’m not referring to difficulty in the actual scripting component itself, but in making sure that you are scripting with images that are usable by Sikuli for the purposes of running your script.

Through trial-and-error, I’ve found some of the less obvious features of Sikuli’s IDE to be indispensable for the purposes of creating scripts that are reliably functional.

Let’s say that you have a large image, but you specifically need to click only a small button within the image:

screenshot Sikuli IDE with screenshot of Google homepage

Click the thumbnail of your screenshot that appears within the Sikuli IDE:

screenshot of pop-up window that appears when you click thumbnails in Sikuli IDE

At the top of the window, you will see two buttons to the right of the File button. The button to the far right is Target Offset:

Target Offset button clicked in file pop-up window in Sikuli IDE

Clicking Target Offset will give you access to a grid view of your image and a set of crosshairs that you can use to target a specific point on the grid. This is ideal for clicking sensitive items on the screen that may be small or close to other clickable items. For fine tuning, you can zoom in and out of the image using your mouse.

The middle button is Matching Preview. It contains an adjustable Similarity Index that you can use to further tune the efficacy of your images within your script. It provides a heat mapping style color overlay to achieve this, the colors ranging from deep blue to bright red, depending how well the image on the screen matches the image file in your script. Once I discovered these two features of the IDE, I was quickly able to produce scripts that were a great deal more usable.

screenshot of Matching Preview in Sikuli IDE

I hope that you find this helpful as you use Sikuli, I’ll be sharing more tips as I learn the software better.

Enhanced by Zemanta