The mobile app development field is incredibly crowded — and that’s unlikely to change anytime soon. Over 99.99% of consumer apps are never considered commercially successful by their developers, and 75% of new users never return to an app after their first day.
In this hyper-competitive environment, you’d think user testing would be a no-brainer, but in fact it’s severely under-utilized. Only 29% of mobile app developers use exploratory testing, and most mobile teams fail to make quality assurance a high priority.
User testing can give your company an edge over at least some of your competitors. Your app will have a better UX because you’ve put in the time to make sure your users actually find it fun and intuitive. You’ll get in-demand features and upgrades out to market sooner because you’ve collected feedback. You’ll be able to avoid the cost of rushing through patches for major bugs (or the user churn and negative reviews that comes with that messy process).
In short, you’ll have a better shot — consistently — of being in that elite 0.01% of apps that become serious commercial successes.
User Testing Never Ends
Okay, maybe we’re exaggerating here, but only a little bit. Too many companies approach user testing like it’s the last hill at the end of a marathon. They go through most of the project, from initial research through development completely internally.
And finally, when they have a (more or less) finished app, they schedule an outside contractor to perform some perfunctory user acceptance testing and smooth out any remaining rough edges.
One problem with leaving mobile app testing to the later stages of development is it limits just how much you can do with the information you collect. At that point in the project, you’ve budgeted a very limited amount of time for changes. If you spot any major problems, there’s a big incentive to make the minimum changes necessary and save most of the work for future updates. Either you deliver an inferior product to market or you release late (and probably over-budget).
But an even more serious problem is that your users don’t really inform your product in any serious way. You can’t get user experience feedback to test your ideas for new features — or design and layout innovation. You can’t set benchmarks to make your app more accessible, popular or user-friendly. At best, you can superficially confirm your intuitions, but not in any systematic way. That means if your team makes a bad design decision early on, it’s likely going to make it all the way to the market.
By making mobile app user testing part of every stage of the process, you can make a better product, and in many cases, even save time. In particular, starting your QA process early in the mobile app prototyping phase can help you nail down what users want early on. That means you’ll spend less time debating and pursuing flawed UX design concepts or features that won’t hold much appeal to your users. In many cases, you should start even earlier, studying how users interact with your competitors’ apps to help you spot gaps in the market.
Phases of User Testing
The way you structure your user testing depends on a lot of considerations, including your application, software development methodology, budget, goals, and competition. However, there are some basic development guidelines you can use to structure your mobile app testing program, and ensure you get the right type of input at each waypoint.
UserTesting, a company focused on UX across a wide range of platforms and industries, divides the process into six phases: Research and Planning, Design, Prototyping, Development + Staging, Release, and Maintenance + Optimization.
In our experience, rapid prototyping can change this structure significantly. Mobile app developers often build prototypes starting with the design phase, or even in planning as a way to work out the concepts or attract investors. They might then iterate over the initial stages again once they have investor capital to do deeper research and refine the design and prototype.
Keeping that in mind, here’s our vision of how to use mobile app user testing at each stage of the development process.
1. Research
In the research phase, your focus is on understanding the market and how your app fits in. You’ll look at competitors, profile your ideal user and investigate market opportunities. Later on, research extends to technical issues like platforms, requirements, and road-mapping the product.
Your research needs to target issues where you face uncertainty or doubt. If you’re new to a market or sector, you’ll want to look for patterns in user demand and demographics. Does the industry leader have younger or older users than their top competitor? Do the major apps target different industries, income levels or geographic regions?
Once you have this information, you can start drawing connections between user demographics and the design and features of each app. This research can help you tease out a niche where you can target users who aren’t well served by competitors.
On a more granular level, you may want to work with individual users and record how they use competitors’ apps to accomplish various tasks. You can gather a wealth of information this way.
Recording user actions will let you explore technical issues, like determining how long each app takes to complete basic tasks. Conducting follow up interviews will allow you to dig into user preferences as well. You’ll be able to quiz users on how they feel about the design and flow of each app, what features they’d like to see in a competing app, which apps they’re likely to stick with, and other topics.
2. Design
Testing in the design phase can help you critically examine your team’s ideas and revamp them to better fit expectations. This can be as simple as enlisting a few friends to look at design sketches or as complex as a formal, scientific study of user reactions.
You can also use more active user testing techniques to investigate things like user flow and organization. For example, with card sorting you can see what data groupings make sense to users by having them manually arrange cards representing different types of data and functionality in your app.
3. Prototyping
Mobile app prototyping is a chance to go beyond sketches, and actually test and review your design ideas. With a fully functional prototype, you can iron out glitches and inconsistencies earlier while refining your design concept in a way that isn’t possible with wireframes.
With an iterative approach, the prototyping phase can be a fantastic opportunity to set and reach benchmarks. If you’re having issues with usability, you might want to gauge missed clicks, or time to complete a task.
If you’re more interested in refining your market position, you can gauge positive vs. negative opinions, net promoter scores, or approval compared to market leaders. Then, track these scores against demographic data to see how well you’re appealing to your core users.
4. Development
In general, we believe that everything that can be handled in prototyping should, but there are certain things you can’t test until you’re actually in development. Most obviously, you need to make sure the thing works the way you anticipate on various platforms. Mobile app testing needs to spot problems with the UI, poor or inconsistent performance, stability issues, and the like.
But there are also some aspects you can’t fully model in a prototype. You can mock up integrations with backend databases or other apps, but you may not be able to simulate every use case or circumstance in the prototyping phase.
Finally, your user acceptance testing has to model real-world conditions. If your users will be using voice commands on a crowded city street, test out voice commands in that environment. If you’re offering support for disabled users (and you should), hire disabled testers. The more thorough you are in this phase, the less likely you’ll encounter bugs after your app launches.
5. Release
If you front load your user testing program, you won’t have a lot of last-minute changes to worry about, but they can still creep in. If they do, you need to document them and prioritize future updates to address shortcomings or new feature ideas you discovered late in the process.
Release is also a great opportunity to hand testing off to your marketing team, so they can monitor how your app performs in the real world. Are users sharing your app with friends? What kind of social media traction and reviews is it getting?
How are downloads stacking up against expectations, and just as importantly, what are your users saying about your app? Are there any missing features you could push out? Any design aspects that users particularly enjoy? The quicker you learn, the better you can measure your app throughout its lifecycle.
6. Maintenance and Future Releases
As we said in the beginning, user testing never ends. You need to use remote testing to gather ongoing user reactions, plan new features and execute new releases.
This testing phase not only helps you keep the product aligned to the market, but it also prevents complacency in your team. It’s easy to breathe a sigh of relief when the project is “done,” but as long as your app is successful, it’s never really done. By looking at it as an ongoing project with upcoming deadlines and goals, you can avoid delays and maximize ROI.
User Testing: Time-Consuming but Necessary
If you want to send a successful app to market, testing has to be part of your DNA. If user testing is missing from any step of your production process, you could potentially be missing major issues with your app. And no, we aren’t just talking to neophyte developers here: even grizzled veterans need to remember the importance of testing.
We get it: testing is difficult, and without a robust solution to prototype your app (and record your users’ behavior), it can seem like an unnecessary chore. But virtually every app you’ve ever fallen in love with has been through a rigorous testing process — and if you want to build an app that someone else loves, you need to go through the exact same process.
Proto.io lets anyone build mobile app prototypes that feel real. No coding or design skills required. Bring your ideas to life quickly! Sign up for a free 15-day trial of Proto.io today and get started on your next mobile app design.
Got any user testing tips we missed? Let us know by tweeting us @Protoio!
