Sikuli Automation Tool with Coded UI

Sikuli:

Sikuli automates anything you see on the screen. It uses image recognition to identify and control GUI components. It is useful when there is no easy access to the GUI elements.

Advantages of using Sikuli :

1. Using Sikuli we can automate a desktop as well as a Web application
2. Easy and simple to use
3. Can be easily integrated with Java, C# and other popular programming languages
4. Can be used to automate Flash Objects / Flash Websites
5. Uses powerful “”Visual Match” mechanism to automate applications
6. Open source

There are certain situations in the automation where automation tool does not find appropriate elements.

Eg: Consider a desktop application which is completely developed in C# but HTML layer is also embedded in it. Now, if we are using CodedUI, AutoIt, TestStack,White or any other automation tool for desktop automation, it is very hard or sometimes impossible to locate elements efficiently. Sikuli can be very helpful in such scenarios.

My aim in this blog is to show how we can integrate Sikuli with Visual studio so that we can use Sikuli with C#.

Sikuli Integration with Visual Studio for Coded UI test cases:

Pre-requisites :

1. Visual Studio should be installed on the system; preferably Visual Studio 2012
2. Any screen-shot capturing tool like Greenshot etc.
3. Java 1.6 tested with jre-6u35-windows-i586.exe
4. Microsoft .Net framework 4.5
5. NuGet

Steps :

1. Create new project in Visual Studio 2012
2. In the “Solution Explorer” – right-click on the solution
3. Choose Manage NuGet Package for Solution

Search for “SikuliIntegrator” and install

5. After the installation “JSikuliModule.jar” should be part of your solution, together with some additional files
6. Right-Click on “JSikuliModule.jar” and choose “Properties”
7. Set “Copy to Output Directory” property to “Copy always”

After performing step 7, there can be issues in copying the jar into the results folder of your test case. We can write code to copy jar from source to destination.

Once all the 7 steps are performed successfully, it’s very easy to use Sikuli in your C# code. Let’s see some useful methods of Sikuli.

Add SikuliModule in the namespace in the following way so that we can use Sikuli related classes and methods

using SikuliModule

General Syntax :
SikuliAction.[ACTION](ARGUMENTS);

Example of some of the methods supported by SikuliIntegrator :

Click
SikuliAction.Click(pattern);

Double Click
SikuliAction.DoubleClick(pattern);

Right Click
SikuliAction.RightClick(pattern);

Hover
SikuliAction.Hover(pattern);

Wait
SikuliAction.Wait(pattern, timeout);

Drag and Drop
SikuliAction.DragAndDrop(fromPattern, toPattern);

Where pattern is path to image

I hope this blog will help in solving your element identification problems to a great extent.

References :

https://qaagent.wordpress.com/2013/07/17/how-to-use-sikuli-in-my-c-coded-tests/

http://www.sikuli.org/