Augmented reality is decades younger than virtual reality, but it has already surpassed VR in many ways. While the fully immersive virtual or mixed reality of science fiction movies is still a few years away, augmented reality is already empowering users to overlay games, objects, and useful information on top of the world around them.
As 3D displays and other technologies continue to mature, augmented reality designers will profoundly change the way we use apps, providing even deeper integration of our tech into our daily lives. Here’s what AR designers need to know.
Augmented Reality Isn’t as New as You Think
Pokémon Go was the first app to catapult augmented reality apps into the popular consciousness. First released in July 2016, the game became a worldwide hit almost instantly, with over 500 million downloads by the end of the year. Although the first release had some serious technical issues, for early adopters it marked the start of a new and thrilling chapter in gaming, turning the whole world into a game arena.
But Pokémon Go wasn’t the first augmented reality app — not by a longshot. In fact, AR was more than 25 years old when Go came out. The term was originally coined by Tom Caudwell, a Boeing researcher who was working on a system that used a digital display to aid aircraft electricians.
Early AR experiments were a lot different than modern efforts. Computing hardware and graphics were primitive by modern standards, meaning that inventors had to use interesting workarounds to get everything to work.
For example, some innovators used visual displays with electromechanical elements to make the technology more immersive. However, even before these experiments, researchers had already developed (at least, on a basic level) the predecessor of what would eventually become one of the most useful tools for augmented reality designers and developers: GPS navigation.
While the technology wasn’t as polished in the early 90s — and GPS navigation couldn’t quite create a visual overlay from the users’ perspective — the basic idea was already there. Decades before the iPhone, users could already use a device that would react to their surroundings and give them helpful information so they could perform real-world tasks more effectively.
Since then, augmented reality app designers have built an endless stream of useful apps. AR software has done everything from overlaying information on top of football fields during live broadcasts, to aiding industrial designers, to putting kitten ears on your head during a live video chat.
Augmented reality designers should understand they’re not peering into new, undiscovered territory — they’re treading a well-beaten path. There are some amazingly innovative apps, such as Froggipedia, which teaches you about the frog’s life cycle and organs, and even allows you to dissect a virtual frog with an Apple pencil. And there are powerful tools, like Google’s Maps API, which lets independent designers and developers build apps that actually recognize and interact with the landscape around the user — something that wasn’t really possible without a massive budget before.
As a designer, it’s great to get excited about AR and explore new possibilities, but there’s no need to reinvent the wheel. You should approach augmented reality like any other app, learning from others’ successes (and failures) and harnessing the latest and greatest tools available.
Augmented Reality Designers Should Look Beyond Mobile End Users
The companies that create augmented reality apps for consumers get the most buzz, but AR is growing rapidly across the board. According to a study by Mind Commerce, mobile augmented reality is growing rapidly and will reach 54% of the AR market by 2023. However, growth in head-mounted displays (or HMDs) is even more impressive and expected to achieve 69% CAGR from 2018 to 2023. And while there’s a lot of opportunity for consumer-grade designers, the real world-changing AR applications in the near-term may be in business — particularly industrial areas.
For example, remember that early Boeing research we talked about earlier? It may not have transformed the aircraft industry at the time, but Boeing never gave up and the tech eventually caught up with the use case. Now, Boeing is testing an AR app that makes it easier for technicians to wire aircrafts. As users walk through the fuselage, the viewing goggles show them a 3D model of the wiring, replacing the extremely complex, twenty-foot-long drawings they traditionally use to guide them with a hands-free display.
Although the app is still in testing, the initial results are extremely impressive. The new tool decreases wiring time by 30% and improves first-time quality by an astonishing 90%!
Another really fascinating example is Unearth, a construction app we’ve written about before. They integrate drone and ground surveying, up to the moment weather reporting, and other data to revamp the entire construction process for the 21st century — from blueprinting, through logistics and materials management, to marketing your product and handing it off to operations and maintenance.
This isn’t necessarily AR by the strictest definition. The goal isn’t to overlay virtual images on what you’re actually seeing at the site. But it isn’t that far from AR either, and it’s not hard to imagine how augmented reality app design will be used to enhance apps like Unearth in the future. We expect many other industry-specific applications to follow, integrating real world geo-located data with project management tools.
For augmented reality designers and aspiring AR designers, that means there’s a lot more out there than flashy consumer apps. If you’re an expert in UI design, or have industry-specific technical knowledge, project management experience or other supplementary skills to complement your design background, it’s time to stretch your imagination. There’s a lot of room for you to follow your vision and find your own niche in the augmented reality app design world.
Augmented Reality Will Dramatically Alter UX Design
User experience design has a lot of well-worn UX design conventions. If an app is well-designed, a user can pick it up and intuitively grasp how to switch screens, navigate menus and find the information and functions they’re looking for within a few minutes of loading it for the first time — in most cases, without reading any instructions.
Augmented reality is going to break a lot of those conventions, and profoundly transform UX design. On a basic level, designers have a lot more space to play with. While a conventional flat screen interface is limited by the form factor of the screen, augmented reality designers can work with 360 degrees, along with depth. How dramatically that changes the UX depends on the interface. If you’re using a hands-free 3D viewer, you can radically alter the interface, filling the space around users with objects, controls, and information. Over time, this will change the way designers balance functionality and ease of use — in three dimensions, you can throw a lot more at your user without making the app cumbersome.
But even with 2D screen-based augmented reality, designers have a lot of room to create novel user interfaces and experiences. You can tie virtual objects to different physical locations or orientations with respect to the user, for example by letting users place reminders or virtual tools in different places. You can create apps that interact with changing world conditions, providing rich contextual input. You can even incorporate interactions with other users more naturally.
But as the technology progresses to allow more seamless use of hands-free 3D devices, the possibilities get even richer, incorporating gesture and gaze control to allow designers to move into mixed reality territory. From the user’s perspective, objects become more real, as they’re anchored to the world around the user. With persistent digital worlds that users can interact with, using hands, voice controls, and gaze, the distinctions between the augmented and physical world will continue to blur, and may eventually dissolve completely.
As VentureBeat points out, these new opportunities come with new challenges:
“One of the greatest challenges presented by these new interfaces will be to define new interaction standards. The development of the personal computer and the smartphone has introduced interaction standards that have become second nature over time. These include clicking through different windows, keying in text input, and zooming in with a finger pinch gesture. These standards were designed for their corresponding hardware peripherals like the mouse or touchscreen, but what’s the mouse or touchscreen of AR?”
That question probably doesn’t have a simple answer. Conventions will change radically and may fragment for a while. But over time, innovative designers will develop new conventions, with the right balance of voice, kinesthetic, gesture, and peripheral controls to make augmented reality UX as intuitive as any 2D app.
AR Interfaces Will Make Common Tasks Easier For Users
The Nielsen Norman Group identifies three ways AR can improve UX: decreasing interaction cost, reducing cognitive load, and decreasing attention switching.
- Interaction Cost: Every interaction requires something from a user, whether it’s just pressing a button, saying a command or navigating a complex menu. These interaction costs can profoundly affect UX — that’s one reason why user flow is so important.
If a function takes a lot of extra work to access or isn’t organized in an intuitive way, it harms productivity, and makes the app less useful and enjoyable. This is particularly significant if you’re using an app as a tool for a real-world task. You have to look down and find the information you need before you can continue your task.
Augmented reality apps can minimize or even eliminate interaction cost. For example, let’s say you’re assembling a chair using an augmented reality app. When you open the box, the app automatically highlights the first parts you need in your visual field. The seat, one of the legs, and a screw glow brightly. As you pick them up, the app shows you an animation, indicating how to insert the leg and tighten the screw. When you finish, the app repeats the process, highlighting the next part in your visual field, and providing another animation as soon as you pick it up.
You never have to look at the instruction book or compare parts with schematics — the app just shows you what you need. You end up with a completely assembled chair, all without once having to issue a command or search for information.
- Cognitive Load: You may have noticed another effect of our hypothetical furniture app: you don’t have to keep information in your short-term memory. There’s no need to remember what step you’re on, or which way the washer faces, or whether the long screw goes with the arm or the leg.
That’s a nice perk for assembling furniture, but for more complex tasks it can be transformative. Imagine being a drug researcher working on animal testing. There are different groups of animals, receiving various substances according to a very specific set of rules.
Dosing, observation, data recording, and other tasks have to be performed correctly every time, or you could invalidate the research. Augmented reality could greatly reduce the risks of breaking protocols, missing steps or incorrectly recording data. - Attention Switching: Using a conventional app to guide you through real-world tasks, you have to switch your attention from the screen to the task, and back. You may need to dig through multiple screens to retrieve different types of information, then use both information sets in your task.
For example, if you’re a facilities maintenance technician repairing an industrial machine, you might have to simultaneously look at the technical manual and the maintenance records and work orders to find out when it had been repaired and serviced, what readings the gauges typically reported, what maintenance was due, and what problems (if any) the machine was currently having. At the same time, they’d need detailed servicing instructions, and the ability to record gauge readings, update the work order, and add any necessary notes.
An augmented reality app designer could dramatically simplify the process by integrating information from multiple sources in the technician’s visual field. The technician wouldn’t have to look back and forth between multiple screens and could dictate gauge readings or even enter them directly into the app automatically.
With Proto.io, Diving into Virtual and Augmented Reality is Easy
Augmented reality isn’t new — but that doesn’t make it any less revolutionary. As more app developers (and hardware manufacturers) adopt and create augmented and alternate reality solutions, the future of UX design is guaranteed to change.
Regardless of if you’re just dipping your feet into the water, or you’re an advanced UX designer who has been playing in AR and VR for decades, developing a prototype can help you completely flesh out your ideas — and help prepare them for the market.
Want to get started? Proto.io allows users to develop their own virtual reality experience, all without writing a single line of code.
Proto.io lets anyone build mobile app prototypes that feel real. No coding or design skills required. Bring your ideas to life quickly! Sign up for a free 15-day trial of Proto.io today and get started on your next mobile app design.
Have any great tips for augmented reality designers? Let us know by tweeting us @Protoio!
