Are Inadequate Mobile App Testing and QA Hindering Your Success?

Let’s face it: the fun part about designing or developing a mobile app is, well, designing and developing the mobile app.

Concepting is a blast, because that’s when the light goes off, inspiration hits you and you’re caught up in the infatuation of a new idea. Designing is when you get to see that idea on paper (or a computer screen) for the first time, and prototyping brings that idea to life. Then you get to development, where a gorgeous prototype turns into a fully functional app, and the problem-solving element brings out the Sherlock Holmes in every mobile app developer.

But then there’s quality assurance and mobile app testing. With the exception of the rare QA geek who is passionate about perfecting every detail, almost nobody gets excited about mobile app testing. Usually you don’t go to sleep at night, thrilled at the prospect of catching bugs and proofreading copy the next morning.

Unfortunately, mobile app testing and QA are also one of the processes most likely to get shortchanged over the course of your project. This may sound familiar to you: your client, investors or crowdfunding backers set a deadline that you find challenging, but doable.

However, over time, things get pushed back. There are many ways this can happen: maybe an idea that sounded brilliant in concepting suddenly seems less practical or effective. Maybe your design team has conflicting ideas about what the app should look like, and goes through multiple rounds just to produce initial assets. Maybe you’ve received client or backer feedback that you need to incorporate, but which slows things down — especially if they decide they want added functionality.

And then there are the fire drills you don’t see coming. You can’t use a particular asset due to copyright, so now you have to swap out every instance of a certain photo or illustration with something brand new. Maybe you’re working with a particular brand or product that has made a sudden change, either in their look and feel or in their product and service offerings, but for whatever reason you’re not winning the battle to push back the deadline.

Well, the app still needs to be made, right? Something, somewhere needs to get cut. Because QA and mobile app testing are close to the end of the design and development process, and because the importance of mobile app testing is slightly less visible than, say, actually building the product, you might give your QA team less time to comb through the app.

That’s not just a great way to make your proofreaders and mobile app testers lose morale, but it can also result in disaster. Let’s look at a few use cases that prove the point.

Here’s the truth: no matter how beautiful and usable your app is, no matter what important need it fills for your potential users, nobody will use it if it crashes all the time, or if the experience is broken. That’s the ROI of mobile app testing: you won’t get ROI if you don’t do it (unless you’re super lucky, or your developer is a flawless robot incapable of erring).

Polls show that only 16% of users will give a buggy, failing mobile app a second chance. That means that unless you have a captive audience (and unless you’re reading this from a Google, Apple, Facebook or Twitter office, this is unlikely to apply to you), you really need to get it right the first time. The following three apps rushed out the door with some serious issues, and only two survived the onslaught of customer complaints that met them.

Forecast was a heartbreaking example of a mostly well-loved app that was brought down by bugs, crashes and other QA problems. Taking Foursquare and Facebook functionality to the next level to help users plan and broadcast social events and check-ins, iPhone users truly loved the app. But while Forecast’s iOS reviews were a chorus of praise, Android reviews were a pandemonium of disdain.

Of course, 2012 was a different time for Android, and OS fragmentation was much more pronounced in the era of Jelly Bean. That said, had Forecast invested more time into QA and mobile app testing, they might have avoided many of the complaints their Android users lobbed — for example, that the app crashed constantly. In addition, Forecast didn’t heed their customers’ requests to schedule events more than two weeks in advance. A big aspect of mobile app testing is listening to beta user feedback, and while the user isn’t always right, Forecast’s call in this case didn’t do them any favors. The app eventually shut down, much to the chagrin of its loyal iOS users.

Facebook’s mobile woes are so famous now it almost seems pointless to mention them, but the app is still worlds better than it was in 2012, when Kevin C. Tofel of Gigaom wrote about simply going to Facebook’s mobile site in his smartphone browser and bypassing the broken app altogether to get his social fix.

The problem? In the past, one major culprit was Facebook’s use of hybrid apps, which Zuckerberg says relied too heavily on HTML5. The other issue at play is that Facebook is an ever-changing service, always adding new features and functionality. Every time that happens, though, some bugs are guaranteed to happen.

What are your options at that point? You either take the time to thoroughly test and QA each iteration of the app — and listening to customers complain that their mobile app doesn’t live up to the web experience — or you roll out the features without smoothing out all of the bugs, and listen to complaints about the app crashing or not working the way it’s supposed to.

Facebook seems to take the latter approach very frequently, and luckily it has a huge brand and over a billion users to fall back on. However, most startups don’t have that kind of safety net, and a poor user experience could sink your app fast. Remember: having fewer features is still better than having a million features that don’t work.

Mobile app designers and developers tend to be a dreamy bunch, with plenty of hopes and aspirations for their ideas. Maybe you want to become “the Uber of” your particular field or niche. Maybe you want your app to end up in some celebrity guru’s Instagram. Maybe you’d love to be featured on a top 10 list.

Chances are, though, you don’t want to end up on The Daily Beast’s list of 10 Worst iPhone Apps.

Okay, this one is a bit meta. How did an app as integral to, well, the rest of Apple’s app ecosystem end up on a list of Apple’s worst apps? Says The Daily Beast, “the iPad App Store makes browsing slow and tedious.” A big component of mobile app testing is ensuring that your app loads and behaves smoothly and quickly on a multitude of devices (and, in some cases, operating systems). This wasn’t the App Store’s only faux paus, though — in addition, design and usability suffer due to the inability to see more than 10 apps at a time.

Of course, the fact that you sort of need App Store for iPad to access apps on your iPad makes iPad users a captive audience, and it’s doubtful Tim Cook is losing much sleep over this. However, a bit of mobile app testing and listening to feedback from beta users could have prevented this app from landing on a “10 Worst” list. Remember: what barely makes a dent in Apple’s reputation could be a PR catastrophe for a smaller business.

As necessary as thorough QA and mobile app testing are, there are plenty of obstacles out there. If these hurdles are giving you migraines and haunting your nightmares, try fine-tuning your process:

Android developers have been dealing with this problem for a while now: unlike Apple, which manufactures its own phones and tablets, Android devices seem basically countless at this point. Samsung, HTC, Motorola and other electronics companies all have their own idea of what a mobile app should look and feel like, and their specs can vary wildly. With the iPhone 6 Plus, though, iOS developers can get in on the mobile-app-testing-for-different-screen-size fun.

Of course, testing for different specs is more than a matter of, “Will this design look just as good on a 5.5 inch screen?” There are several other variables to take into account, including screen density and resolution, CPU, RAM and even battery life (will this app suck the life out of a certain kind of phone? If so, you might want to consider options for adjusting the power leach). If you develop both for Android and Apple, this becomes doubly as difficult.

Of course, there are plenty of emulation tools that can make the job easier on you, especially when it comes to screen size and resolution. However, the gold standard is to test your app on a variety of different devices with different specs, since you can’t faithfully reproduce every spec that’s important to mobile app testing. A combination of a good emulation software tool and a network of colleagues with different smartphones is really what you want to shoot for when putting your app through the QA process.

Android, we’re lookin’ at you. Sure, an app may run like a dream on Lollipop, but what about folks with KitKat? Or, heavens forbid, Jelly Bean or Ice Cream Sandwich? (Sometimes, it seems like developing for Android can raise your insulin levels.)

iOS fragmentation isn’t so bad, of course, because it’s more centrally managed. When Apple wants you to update your OS, you update your OS. When Android comes out with an OS update, it’s up to one of those many device manufacturers to deliver that update. However, any time an OS comes out with a new update, whether it’s iOS or Android, there’s the potential that your app experience might break in some way due to the new OS specs.

What’s a mobile app testing team to do? Stay on top of OS updates, and test on multiple versions if possible. Even if your app is already on the market, pay attention to user feedback and see if a change in OS version results in a change in user satisfaction, and update accordingly.

There’s no law that says that you have to design both for iOS and Android, but if you can, it gives you a powerful edge over OS-specific apps. Even if you’re an Android fanatic, Apple has sold a whopping billion iOS devices, and Android has a billion users. You don’t need a degree in business or mathematics to see the potential there.

However, designing and developing a mobile app for two different operating systems comes with twice the QA and mobile app testing headaches. Remember what happened to Forecast — happy iOS users, but dissatisfied Android patrons?

Let that be a lesson to you: if you do take the leap into cross-platform compatibility, you need to make sure you’ve invested enough in your QA and mobile app testing process to handle the various quirks of two different operating systems.

“Development hell” isn’t just a Hollywood phenomenon: it can kill a brilliant mobile app idea, as well. If you take too long to design and develop a truly disruptive app, what was once a unique, niche idea might become a crowded market before your app goes live. Indeed, if you’re waiting until your app is perfect before it goes to market, it will never see the light of day: there isn’t a single mobile app out there that can’t be improved upon.

Every editor knows this. Think about every book you’ve read by a major publisher in which a typo of some sort has managed to squeak by. Think of every major newspaper that has ever issued a redaction (hint: that’s every major newspaper). Again, your team is, in all likelihood, made up of humans, not robots.

That means that you need to be careful about your project management. Rush your app out the door, and it might be a stunning failure, a mere shadow of what it could be. Wait too long, and another app has taken its spot (and your market share).

At a certain point, you need to make sure that the mobile app testing is done and thorough, but you also need to take a deep breath and realize that if problems arise, you might need to issue a patch or an update. Release the very best mobile app you can, but don’t be held back by the quest for perfection.

If you’re the designated QA buff at your design firm or mobile app startup, feel free to send this post to anyone who has ever made you debug way too much code in way too little time, or who has suggested that automated tools can completely replace the work of an experienced human (they help, but they won’t tell you when a color just looks off, or when the content just gets boring). Mobile app testing may not be the most exciting part of making an app, but it’s still one of the most important.

For anyone who isn’t responsible for QA, take time today to thank your mobile app testing team. They’ve likely saved you from a few close calls!