Prototyping an augmented reality experience with paper, scissors, and a little Figma fakery: A Case Study
By: Richard L Lam
Tags: Augmented Reality, UX Design, Figma, Tourism, Prototyping

Simulating an AR trigger and virtual tour guide with paper cut-outs

Augmented Reality (AR) is steadily trickling into every industry, from the filtered Snapchats you send your friends, to the Ikea furniture you can virtually place in your apartment.

So what about tourism? If you’re on vacation, would an augmented reality walking tour interest you? What kind of characters and stories would you want to see? How long could you hold your phone up for? What would keep you engaged?

And from a practical freelance UX designer standpoint — assuming you don’t have 3D modellers and software engineers on hand — how do you prototype and test an AR tour experience with your standard set of pen-and-paper, Sketch-and-Figma?

Short answer: You have to get creative and fake it.

Using Figma to Smart Animate on top of recorded video GIFs

Scope

Our team of three was hired by WayBak, a company that is developing AR historical walking tours in New York City. We were brought on for a two-and-a-half-week sprint to build the functional base design of their iOS app and flesh out their tour structure, based on our research of how people interact with AR and what they want to experience on a tour.

Research

To wrap our heads around the relatively-unexplored world of AR tourism, we interviewed:

People who wrote, designed, and led walking tours, to get an idea of how they accommodate their audience and guide them through an experience,

AR professionals, to get an idea of what kinds of features and functionality we could realistically build in, and of course,

Travellers who participate in walking tours, to learn what they want out of the experience.

These informed what our primary user would be:

Primary User Traits

We also created a Competitive and Comparative Landscape Matrix, to see how other tour options landed in terms of linearity, and how much physical activity the traveller would be undertaking. This ranged from sedentary hop-on-hop-off bus tours where you had some choice in where you wanted to stop, to borough-sprawling bicycle tours from Point A to Point B, and so on.

Also of note are comparators such as museum audio guides and AR apps, that enhance your experience in a set location, as well as Pokemon Go, a free-range open-world experience that encourages you to walk all over your city:

What’s your preferred tour style?

Combining all of this research, we knew we had to design something that:

Was constantly engaging and surprising in its content, interactivity, and immersiveness

Would help them learn and understand the new AR technology

Would allow some flexibility and freedom to explore

Could help them navigate

Did not tire them out

So what did we come up with?

A test user following our camera movements, despite it being pre-recorded (!)

Design

Onboarding

“This is a new experience for a lot of people. My mom doesn’t know anything about this kind of stuff. How would you explain this to her?” — Test User

Our primary user mentioned needing some help with new technologies. With AR being an unfamiliar thing for many people, we downloaded a number of AR apps to see how they onboard and teach their users how to use the app and its features.

Some are quick and simple, like HotStepper, which barely explains what it’s going to do before a goofy cartoon character shows up in front of you to help you navigate to a destination:

Others are very thorough, like BBC’s Civilisations app and its ten-minute orientation, getting you to place, rotate, and x-ray view a historical object floating in your living room:

We settled somewhere in the middle, as per our users’ needs. Upon opening the app, three screens of context and instruction are provided:

With that in the back of their minds, the user then begins their first tour, where they receive additional progressive onboarding that advises them to put on headphones, teaches them how to trigger the tour to start, learn to take photos, and engage with the content:

This onboarding method proved to be highly successful, and the process highly learnable. Once the users successfully lined up their first image, they quickly and intuitively proceeded to do the same once they reached the second stop. Compared to spatial anchoring (another method of triggering AR content), the dotted-lined image recognition approach proved to be much easier for our test users.

Touring

“I think the dotted lines and arrows are really cool. It feels really fun to be able to see your own alternate reality pathfinder through city streets. Pretty sci-fi.” — Test User

As for the tour itself, the key finding from our initial interviews was that when people go on walking tours, they want immersion and exploration. They want all their senses engaged, and have opportunities for choice and self-discovery. Can a phone app replicate that?

We tested out a few approaches for this: For immersion, we wanted to engage as many senses as possible. We would obviously want a visual component, with virtual ‘guides’ that could tell you stories and directions plotted on floor. But we also wanted to fully utilize audio, with spoken narration and ambient sounds. Lastly, we wanted to use the phone’s haptic feedback to vibrate and alert the user when important things were happening.

As for exploration, we had a challenge: Our clients had a strict, ‘guided’ narrative in mind for each tour, telling the history of a neighbourhood and its historical figures as you go from stop to stop. Essentially, a linear, steady drip-feed of information.

However, many of our testers saw this experience as a self-guided tour, one that they could pick-and-choose landmarks and places that interested them, and skip information if they wanted to. So we tested something in the middle:

We included a recommended route that users could take for the ‘intended experience’. However, we didn’t want to prevent someone from jumping ahead to, say, the horse monument if that was of more interest.

About half of our test users went with the recommended route. But the other half chose to skip ahead. This made space for a valuable conversation with the client to start thinking about more self-contained narratives at each stop, and allow for something that closer resembled a collection of inter-related short stories, rather than a discrete novel.

Engaging Content

“It’s like a scavenger hunt. I’m a completionist and would want to find them all.” — Test User

Our early interviews showed that the most memorable part of walking tours for people were the little surprises along the way. Travellers may have signed up to see the key landmarks, but what stuck in their heads years later was the local pub recommendation, or the discolored brick stuck in the building wall with a neat story.

So to keep users engaged and surprised throughout our experience, we tried breaking up content and delivering it up in different ways. Rather than just having stories narrated to you in a continuous stream, we added context-specific hotspots that users could interact with for optional content. We saw these as supplementary and gamifiable content ‘snacks’ that could to deepen the experience along the way, but not be essential to the core narrative our clients were trying to tell.

This ended up being the most popular part of our tour experience. One user wanted to see ten of these hotspots, and said it was where she would spend the most of her time.

We also added our own version of a ‘surprise’ along the way, where a user was prompted to “look up” partway through their walk to see a simulation of a historic local emptying their chamberpot out the window at them, which was a delightful — if only a little disgusting — suggestion from our client. That also proved to be thoroughly appreciated by our users.

Lastly, if you wanted to see a full user walkthrough, here’s a video:

Future Considerations

“I don’t know about walking three blocks in Manhattan with an AR thing out…There are periods when I want to put the phone down and not stare at it.” — Test User

Extended-arm-fatigue is a real problem in the AR space, as being users are asked to hold their phones out for extended periods of time. Part of the reason we emphasized breaking up the content and relying more on audio and haptic features was so that users had more opportunities to put their phone down and take breaks.

Safety and Accessibility also came up a number of times in our testing. We tried considering content that didn’t require having your phone out while trying to cross the street safely, or navigating around crowded tourist spots. We wanted to accommodate people who had hearing, vision, or mobility challenges, by providing subtitles, transcriptions, audio read-outs, and spoken instruction.

We also want to figure out how this tour experience can be made more social. How does a pair of friends — or a family of four — take this tour together, when it all happens on a small screen with a pair of headphones?

These questions aren’t unique to the AR space, but are especially important in considering a traveller’s walking tour. We have some ideas, but they would need more testing to determine their validity. WayBak is taking our designs to software engineers to be be developed now, with an aim to attract investors in the new year, so hopefully we’ll be able to do that soon.

Until then, keep faking it ’til you make it (literally)!