Top 5 Open Source Automation Tools for iOS and Android

Test Automation nowadays plays a very important aspect in software product testing. Hence more companies are focusing on providing Test automation services.  There are multiple positives to go for test automation. This includes minimizing test execution time, ensuring larger coverage in less time in critical release phases, reliable and repetitive runs during the product development phase to ensure no regression issues are introduced. Also, it minimizes the risk of human error or negligence while doing repetitive regression testing cycles.

There are a lot of tools available in the market, both open source and paid ones. Though there are pros and cons of using both paid and open-source tools, later categories of tools are widely used across the automation testing community. Choosing the right automation tool according to project requirements is the very tricky part, as it could get your test strategy past the winners to mark on the other side it could prove to be a disaster, decoupling testing and development planning in agile methodologies.

Below are 5 top open-source iOS and android automation tools that are most widely used and have wide community support.

1) Calabash (For Android and iOS)

What it is – calabash is an open-source acceptance testing framework for android and iOS automation. Calabash provides a separate library for android and iOS automation. It’s a cross-platform framework and supports cucumber which enables to an expression of the behaviour of the app in the natural English language implementing BDD (Behaviour Driven Development).

Calabash provides a bridge that allows cucumber tests to run and validate on iOS and Android. Cucumber tests are written as a list of statements that form the number of test scenarios. All the statements in cucumber are defined using Ruby language.

In Calabash, a cucumber statement must be defined only once but can be repeated for different scenarios of a cucumber script.

The actual tests are written in Gherkin, backed with Ruby code, and run in the context of the Cucumber Framework.

Pros

1. Large Community Support.
2. Simple and expressive English like test statements
3. Supports all actions on screen such as swipe, pinch, rotate, and tap.
4. Large and enthusiastic community.
5. Cross platform development support(same code works for android as well as iOS devices)

Cons

1. After the failure of the test steps, all subsequent steps will be skipped. This can cause more severe product issues to be missed out.
2. Takes time to run tests as it always installs App first by default. However, this setting can be overridden by a hook that we can configure in the code.
3. Requires calabash framework to be instrumented inside IPA file for iOS.
4. We must have the code of the App in the case of iOS.
5. Not friendly to languages other than ruby.

2) Appium (Android and iOS)

What it is – Appium is an open-source test automation framework by sauce labs for use with native, hybrid and mobile web apps. Appium library functions inside framework make calls to Appium server running in the background which operates the connected device. It uses JSONWireProtocol internally to interact with iOS and Android apps using Selenium’s WebDriver.

Unlike calabash which mainly supports development in ruby, Appium can be used in a framework with the language of your choice from Java, python, ruby and all others that the selenium-web driver supports.

Appium server gets hosted on a Node server. One can start the Appium server by firing a set of Node commands. If using Appium Standalone Application as a server (installable downloaded from the Appium website), the ‘Inspector’ tool provides the ability to locate/identify/operate on all locators of the Application configured.

Pros 

1. Supports multiple languages.
2. Does not require Source code access.
3. Cross-platform script development.
4. Large community support.
5. Supports script recording on Mac
6. Can extract identifiers using Appium Server application’s ‘Inspector’ tool.
7. In-built support for ‘Selendroid’ by Appium Server desktop application.
8. It also uses the vendor provided framework: UIAutomation for iOS, UIAutomator and Selendroid for android 4.2+ and 2.3+ respectively.
9. Supports physical devices as well as Emulators.
10. Supports native, hybrid and mobile web application automation.

Cons

1. Frequent unstable releases for Appium Server desktop application.
2. Script recording is not available for Application server desktop applications for Windows OS.

3) Robotium (Android)

What it is – Robotium is an open-source test framework for developing functional, system and acceptance test scenarios. It is very similar to selenium however only for android. It’s listed under Apache Licence 2.0.

Because of its simplicity and ability to create powerful and robust automation scenarios, it’s gaining popularity in the test automation community.

It uses run time binding to GUI components hence. It installs a test case suite as an application on an Android device or emulator and hence provides a real environment for the execution of tests.

Pros 

1. Easy to write test scripts in minimal time.
2. Automation of pre-installed apps is possible.
3. Automatically follows the current activity.
4. Faster and more robust test execution as compared to Appium due to run time binding with GUI components.
5. Can work without access to code or knowing the implementation of the app.
6. Support for Activities, Dialogs, Toasts, Menus, Context Menus, and other Android SDK controls.

Cons

1. Can’t handle flash or web components.
2. Supports development in Java.
3. Can be slow on older devices.
4. Does not support automation of iOS devices which can be a deal-breaker where the test strategy involves automation of both android and iOS devices automation.
5. No inbuilt support for record and playback. Paid tools are available for recording such as TestDroid and Robotium Recorder.

4) Frank (iOS)

What it is – Frank is iOS app automation framework and provides the ability to write tests scenarios in structured English sentences using cucumber.

Frank requires it to be compiled inside the application under test and hence changes are mandatory to your source code. It’s a tool written mainly by some fellow Thought workers that use a combination of Cucumber and JSON commands that are sent to a server running inside your native application and leverage UISpec to run the commands.

Pros 

1. Test scenarios are written in understandable English sentences with help of the cucumber framework.
2. Symbiote – Live inspection tool is included.
3. Proves efficiency when the team is having experience with web automation framework with selenium, and cucumber.
4. Active community support.
5. Continuously expanding library.

Cons

1. Limited support for gestures.
2. A bit difficult to run tests on the device.
3. Modification to the configuration file is required to run on real devices.
4. The recording feature is not available.

5) UIAutomator (Android)

What it is – UIAutomator is a test framework by Google that provides advanced UI testing of native android apps and games. It’s a java library containing API to create functional UI tests and also an execution engine to run the tests. This library comes in built with android SDK. Many tutorials are available for beginners to get started. The advantage is it gives special privileges to Junit test cases while running to access different processes. Even though it’s good and simpler for native app automation, it has very limited or almost no support for web view automation. It only supports devices with API level 16 and above, however, that should not be a big factor or deal-breaker as most of the apps these days support API level 19 and above.

Pros

1. Simple and easy to learn tutorials are available.
2. Library supported and maintained by google community.
3. The third party paid integration with cloud based test management is available.

Cons 

1. Supports only android 4.1 and above.
2. Script recording is not supported.
3. Support is focused on Java.
4. You can’t get current activity or Instrumentation.
5. ‘WebView’ is not currently supported and hence hybrid apps.
6. Library support is in Java only so it’s very hard if someone wants to blend it with cucumber which is in ruby. However, Java has its own BDD frameworks which can be used, but not much in practice though.

Cheat Sheet for Open Source Automation Tools