- March 10, 2016
- Posted by: Mahesh Kulkarni
- Category: Blogs

Sikuli:
Sikuli automates anything you see on the screen. It uses image recognition to identify and control GUI components. It is useful when there is no easy access to the GUI elements.
Advantages of using Sikuli :
- Using Sikuli we can automate a desktop as well as a Web application
- Easy and simple to use
- Can be easily integrated with Java, C# and other popular programming languages
- Can be used to automate Flash Objects / Flash Websites
- Uses powerful “”Visual Match” mechanism to automate applications
- Open source
There are certain situations in the automation where automation tool does not find appropriate elements.
Eg: Consider a desktop application which is completely developed in C# but HTML layer is also embedded in it. Now, if we are using CodedUI, AutoIt, TestStack,White or any other automation tool for desktop automation, it is very hard or sometimes impossible to locate elements efficiently. Sikuli can be very helpful in such scenarios.
My aim in this blog is to show how we can integrate Sikuli with Visual studio so that we can use Sikuli with C#.
Sikuli Integration with Visual Studio for Coded UI test cases:
Pre-requisites :
- Visual Studio should be installed on the system; preferably Visual Studio 2012
- Any screen-shot capturing tool like Greenshot etc.
- Java 1.6 tested with jre-6u35-windows-i586.exe
- Microsoft .Net framework 4.5
- NuGet
Steps :
- Create new project in Visual Studio 2012
- In the “Solution Explorer” – right-click on the solution
- Choose Manage NuGet Package for Solution
Search for “SikuliIntegrator” and install
- After the installation “JSikuliModule.jar” should be part of your solution, together with some additional files
- Right-Click on “JSikuliModule.jar” and choose “Properties”
- Set “Copy to Output Directory” property to “Copy always”
After performing step 7, there can be issues in copying the jar into the results folder of your test case. We can write code to copy jar from source to destination.
Once all the 7 steps are performed successfully, it’s very easy to use Sikuli in your C# code. Let’s see some useful methods of Sikuli.
Add SikuliModule in the namespace in the following way so that we can use Sikuli related classes and methods
using SikuliModule
General Syntax : SikuliAction.[ACTION](ARGUMENTS);
Example of some of the methods supported by SikuliIntegrator :
Click SikuliAction.Click(pattern);
Double Click SikuliAction.DoubleClick(pattern);
Right Click SikuliAction.RightClick(pattern);
Hover SikuliAction.Hover(pattern);
Wait SikuliAction.Wait(pattern, timeout);
Drag and Drop SikuliAction.DragAndDrop(fromPattern, toPattern);
Where pattern is path to image
I hope this blog will help in solving your element identification problems to a great extent.
References :
https://qaagent.wordpress.com/2013/07/17/how-to-use-sikuli-in-my-c-coded-tests/
Worth for Automation framework
Visual Studio: 2017 Enterprise Edition
I am getting the below error, can anybody help me on this, please
Test method CodedUITestProject31.CodedUITest1.CodedUITestMethod1 threw exception:
System.Exception: Exception in thread “main” java.lang.UnsatisfiedLinkError: C:\Users\karth\AppData\Local\Temp\tmplib\Win32Util.dll: Can’t find dependent libraries
at java.lang.ClassLoader$NativeLibrary.load(Native Method)
at java.lang.ClassLoader.loadLibrary0(Unknown Source)
at java.lang.ClassLoader.loadLibrary(Unknown Source)
at java.lang.Runtime.load0(Unknown Source)
at java.lang.System.load(Unknown Source)
at com.wapmx.nativeutils.jniloader.NativeLoader.loadLibrary(NativeLoader.java:44)
at org.sikuli.script.Win32Util.(Win32Util.java:19)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Unknown Source)
at org.sikuli.script.Env.getOSUtil(Env.java:91)
at org.sikuli.script.ScreenHighlighter.init(ScreenHighlighter.java:180)
at org.sikuli.script.ScreenHighlighter.(ScreenHighlighter.java:293)
at org.sikuli.script.Screen.initBounds(Screen.java:105)
at org.sikuli.script.Screen.(Screen.java:117)
at com.sikuliintegrator.Operation.RightClick(Operation.java:116)
at com.sikuliintegrator.Pointer.main(Pointer.java:106)
How we can scroll down . I am using desktop application
glad to be one of several visitors on this awe inspiring site : D.
I visited a lot of website but I conceive this one has got something extra in it in it
Getting below error after performing all the steps
driver = new ChromeDriver(@”C:\\”);
Thread.Sleep(5000);
driver.Navigate().GoToUrl(“http://www.google.com”);
Screen screen = new Screen();
Pattern googletextbox = new Pattern(@”C:\Users\DWS\Desktop\genter.png”);
screen.Wait(googletextbox, 5);
screen.Type(googletextbox,”hello”);
Error is : Message: Test method DesignSystemTestPro.CodedUITest1.CodedUITestMethod1 threw exception:
System.Net.WebException: Unable to connect to the remote server —> System.Net.Sockets.SocketException: No connection could be made because the target machine actively refused it [::1]:8080