Mobile Quality Begins with Agile

Featured Article by Eran Kinsbruner, Director, Perfecto Mobile

The mobility trend is forcing companies to look, think and feel agile. With a heavier focus on delivery than ever before, a need for continuous integration support, a focused level of testing has suddenly become increasingly important to agile organizations.

Planning – Me, My Space, My Market

Determining the key deliverables is the most essential first step in any plan. When establishing a mobile strategy in an agile environment, testing should be a major factor – the risks associated with releasing an untested app are tremendous. Once an app is released, it is out there for the world to use (and publicly rate).

Consider these questions when beginning an application strategy with agile:

– Which Operating Systems (OS) and versions should be supported?

– How do you choose the right mobile devices for your organization to test on?

– What testing tools should you use?

– What other factors, like location and voice, should you factor into your test plan?

– How will you break your test plan down? Into functional tests? Unit tests?

When creating a device cloud for application testing, organizations need to determine which devices and operating systems should be chosen for its specific market. No market is the same – end users vary according to segments. For example, while Android accounts for more than 80 percent of testing hours in the advertising segment, it reaches less than 20 percent within retail companies. According to Perfecto Mobile’s Device Statistics Report, this suggests that mobile advertisers may see the highest return on customers using Android, but retail organizations going mobile may see the opposite from their user base and choose to target iOS as a primary mobile platform. This kind of variance is experienced across all industries, so it’s important for organizations, like retailers, to research and choose the right mix of operating systems and devices for their market, their space, and for their end users.

An agile cycle (sprints e.g.) is expected to result in a shippable product. For mobile, this means an executable application that is able to be released to the Android and Apple application stores after it was proven to perform and run perfect on the supported platforms. As these release cycles are very short, it’s important to realize that Agile mandates automation testing. Having teams ready to execute as part of the CI process is crucial, and a set of predefined unit/regression and other functional/performance testing options key to meet such agile sprints on time and with appropriate quality.

Design for Testability

This is a crucial aspect in mobile application development: testing and future maintainability of your product. When beginning to design your mobile application, pre-development dev/agile teams need to think with the end-goal in mind – and enable the testing teams to be able to test the product. Having an application with even some functionality inaccessible for manual or automated testing (e.g. can be tested only via white-box and unit tests) will make the future level of this function support limited since it cannot be handed to testers to visibly test it. Furthermore – functional automation testing relies on easy access to the main features of your mobile application, so when designing your mobile app – think about the future testability of it (Manual and Automation), as well as portability across devices.

Execution

This phase is all about governance. There are so many versions of devices, for example, iPhone 3, 4, 4s, 5, and so on. It’s an ongoing cycle, constantly adding to the roster of devices organizations need to deal with – they need to test on each and every version. The entire process needs to be well governed and secure in order to be effective. This governance and the security of devices are major company assets and resources in the overall mobile testing strategy. From all of the planning and defining, this stage runs the tests and prevents users from getting detrimental bugs and a poor user experience.

It is also about the ability to store and record events that happened in the test cycle so developers can easily see what is happening in order to replicate events and take inventory of all the different bugs to restore them, etc. In the planning stage, these tools should be selected but during execution they are being used to shorten response times, prevent major problems and improve applications.

Test Plan Considerations

Not all test types (fig 1) are always required in the regression cycle for mobile apps or even in full cycles, but it is very important to go through these testing types to match them to your mobile products. Having as many of these types automated and executed as part of your regular cycles/sprints will reduce quality risks and provide sufficient code coverage of your mobile apps, which increases confidence in what is being released to the market.

Fig 1 – Mobile Testing Considerations (Perfecto Mobile)

Release and Sustenance

At the end of the day, mobile applications being released need to be continuously supported.  It’s essential to listen to the customers from various channels and gather feedback to improve the application at all times because inevitably, there are going to be defects from time to time and companies must be there as soon as it happens. Having an agile-ready quality environment which consists of CI process and relevant test plans reduces TTM, increases reaction to defects, and enhances existing products (e.g. adding new OS version support fast).

By proactively collecting data and feedback from analytics and monitoring tools and app stores, this stage aims to address these problems as quickly and efficiently as possible, before widespread users are experiencing any difficulty at all. This process needs to be well-managed and well-monitored throughout the entire application lifecycle.

Mobile is so dynamic; these phases are in place to help organizations release – and hold – a successful mobile application. The ability to execute these phases successfully relies on thorough planning coupled with agile practices and the proper tools for the team, which will help enterprises reach their goals at a much faster pace.

About Eran Kinsbruner:

Eran Kinsbruner is director of product marketing at Perfecto Mobile, the leading cloud-based mobile application testing and automation company. Formerly CTO for mobile testing and Texas Instruments and project manager at Matrix, Eran has been in testing since 1999 with experience that includes managing teams at Qulicke & Soffa, Sun Microsystems, General Electric, and NeuStar. The co-inventor of a test exclusion automated mechanism for mobile J2ME testing at Sun Microsystems, Eran has extensive experience in the mobile testing world. You can find Eran on Twitter @ek121268, LinkedIn, and his professional mobile testing blog at ek121268.wordpress.com. Eran also writes regularly for the Perfecto Mobile blog.