Effective Mobile Test Automation: A Hybrid Approach

Effective Mobile Test Automation: A Hybrid Approach

By: Eran Kinsbruner, Director of Product Marketing, Perfecto Mobile

Today’s mobile market is extremely dynamic, complex and fragmented. This year alone, a large number of devices will be introduced, including open source operating systems that will inevitably further complicate the environment. This level of complexity creates many challenges for businesses and introduces various risks to the application vendors (enterprises, telecommunication providers, insurance companies, game developers, etc.). The need for mobile application testing has never been more imperative – or more complex.

As users continue to adopt mobile applications, companies cannot risk launching a poor mobile app – the business repercussions are endless and the customer base is often unforgiving. Users expect a mobile application to function even better than company’s website. Consumers are looking for a seamless user experience, regardless of the platform.

To accomplish this, companies must utilize proper testing and take a hybrid approach, utilizing two important methods: native object analysis and visual object analysis. For example, a tremendous number of Android and iOS operating systems (Android 2.x, 3.x, 4.x, iOS 4.x, etc.) are running on a variety of devices that differ in screen size, hardware, keyboard characteristics, web browser versions and more. These differences translate into huge challenges – and various bugs.

When manually testing and automating complex applications, you’ll find that they may look different on each device, perform in different ways, or not install correctly. To test and automate these applications on mobile devices, it is important to have access to the application objects (buttons, lists, edit boxes etc.) in order to run the various functions through the application and develop relevant test cases and scenarios. A hybrid approach to testing, using both native and visual objects, will best catch any bugs or issues that arise. Each method covers different aspects.

There are two commonly used object analyses that are relevant to mobile test automation – both are extremely useful and frankly, they cannot be avoided. You must use both to properly test applications and get the best results. These methods include:

1. 1.       Optical Character Recognition (OCR)

Optical Character Recognition essentially uses a smart software engine that converts scanned images of handwritten, typewritten or printed text into machine-encoded text.

In the screenshot below you will see the text truncation on two different devices (the iPhone on the left, and Samsung Galaxy SIII on the right). Also note that on the iPhone the “Login” button looks perfectly fine, while on the Samsung it is truncated (Native recognition will not fail the detection of the Login button, while OCR will).

Figure 1: Representation of a text on an iPhone versus truncated text on Android device and difference between iOS and Android.

1. 1.       Object Level Analysis (Native Objects Recognition)

Object level analysis is used to extract the application object identifier with its properties from the actual native operating system source code, just like the developer used. This is an accurate and fast method to recognize buttons, lists and other objects used by the application.

The main differences between the 2 methods include:

Using a Hybrid Approach

The combination of OCR and Native approaches allows users to build a single script that will be portable across different devices. Essentially, it will give you a much larger scope of test coverage. You can’t avoid having support for both options. The ability to combine both capabilities within one script, across multiple devices makes your automation robust, efficient and allows the user to detect with confidence the relevant Native and GUI defects within the mobile application.

The importance of a hybrid approach is demonstrated in the following examples. In figure two, an HTML5 page shows a bug: only part of the text is visible within the textbox. When using Native object analysis, the result of “Edit.Set.Get” (retrieving the object content) will allow the entire sentence even though it’s not visible on the iPhone screen, as opposed to the OCR analysis (Figure 3), which will only show the actual, short text on the screen:

Figure 2: Native Object Analysis

Figure 3: Visual Object Analysis (OCR)

Preparing for the future of mobile

Clearly, mobile has entered the market and swept into businesses with unprecedented speed. The industry has evolved extremely quickly, and it will continue to present new challenges for enterprises as more developments are made. For this reason, it is important to combine both object recognition methods when implementing mobile test automation to gain flexibility and overcome the challenges of complex mobile applications. Implementing this method now is a good practice as mobile apps are only going to become more complex as additional operating systems enter the market. A hybrid approach helps to increase coverage and make scripts bulletproof.

Eran Kinsbruner is a Director at Perfecto Mobile, the leading cloud-based mobile testing, automation and monitoring company based in Woburn, MA. You can reach him at erank@perfectomobile.com.