[Testing] Sanjay.zalavadia:
Testing iOS and Android software presents unique challenges that require unique test conditions.

Testing iOS and Android software presents unique challenges. As easily portable devices, smartphones and tablets are used in a variety of settings, and wireless connectivity may widely fluctuate and acutely affect the performance of any applications in use. Unlike with PCs that have wired connections, dev/test teams cannot assume relatively stable network conditions when crafting a mobile game, messaging client or news reader.

Moreover, despite enormous leaps forward in CPU and GPU design since the iPhone’s debut in 2007, mobile hardware still is considerably less capable than desktops or laptops, especially when it comes to device memory. In practice, this can means suboptimal performance when working with mostly interpreted languages such as JavaScript (in contrast, PCs often have RAM to spare and can accordingly overcome flaws in language design), as well as frequent crashes.

Still, the difficulties of have not discouraged shops from trying their luck. There are more than 1 million apps in both the Apple App Store and Google Play, and Microsoft has revealed that the count for the Windows Phone Store is now at 300,000. With the market moving toward mature software that takes advantage of increasingly powerful endpoints and addresses functionalities once reserved for PCs, will be instrumental for fostering collaboration and coordinating both manual and automated tests.

How can developers and QA engineers deal with so many mobile devices and platforms?

Creating software for mobile devices has never been simple or easy. In the early days, there were incredible constraints on hardware as well as relatively few APIs and toolkits for expediting development. Over the years, some of these challenges have been overtaken by new ones surrounding sustainable monetization and consistency across a wide range of platforms.

Teams have to address users who may delete an app if it crashes even once, but how can they do so when there are so many device/OS combinations to account for? Native vs. HTML5 development is a conversation for another time – let’s look at how an application developed with either of these methodologies might be tested. Many of the issues that apps face in the wild originate from overlooked mobile testing conditions, which if implemented may have produced a more polished product. Here are a few to keep in mind:

Too much manual testing: Manual tests aren’t bad – they’re critical to many QA workflows. But teams can easily become over-reliant on them, which doesn’t scale well given how fragmented the mobile ecosystem is. Android KitKat, for example, only runs on 20 percent of Android devices as of August 2014. Automated processes are needed.

Insufficient simulation of real-world conditions: As discussed earlier, smartphones and tablets don’t exist in a vacuum. They’re carried inside buildings with poor reception, or packed along into remote regions with only 2G or 3G coverage. Tests have to account for these realities as well as limitations on memory, screen resolution and battery life.

Low attention to region/language settings: This one flies under the radar since many developers target only specific sets of users. For apps with an international or multilingual audience, it is important to see if the platform in question has a translation option and whether app performance is affected by switching from setting to another.

Overall, mobile testing is about scalability for many devices and consistency despite constraints. A blend of automated and manual tests is usually the best way forward.

“Manual testing is a definite need, however, there are so many devices and combinations in the market today that it is necessary to use automated mobile tools as well,” stated a software engineer from CallFlow, in a post on LinkedIn. “User expects (sic) the application to stay on, connected, and perform at all times. To meet these expectations, the mobile testing strategy should include real device testing under various real world conditions. That includes various signal strengths, networks, speed and more.”

The stakes for mobile testing: Even big companies can miss bugs as apps scale

Facebook, with its 1 billion users, is obviously an outlier in the software world, but its recent battle with a bug in its iOS app illustrates how mobile testing requires tremendous time and effort as well as top-notch tools. The social network’s engineers were noticing an issue related to the Apple CoreData System, but due to the size and rapid evolution of the Facebook codebase, parsing the crash reports proved a monumental undertaking.

“[C]ertain fundamental programming challenges inevitably become more difficult with scale,” explained Slobodan Predolac and Nicolas Spiegelberg, engineers at Facebook. “Debugging, for example, can prove difficult even if you can reliably reproduce the problem – and this difficulty increases when debugging a highly visible but nondeterministic issue in a rapidly changing codebase.”

Ultimately, the Facebook team was able to identify the issue through close collaboration and a focus on programming fundamentals. The fix may have reduced the app’s iOS crash rate by 50 percent.

Users often have little patience for app crashes, so this is an important development. While most other shops won’t operate at Facebook’s scale, they’ll still have to deal with similar performance issues that could manifest due to adverse real-world conditions and/or other flaws in the code. A test management solution enables developers and software testers to scale their workflows and find defects early and often at low cost.