UX In Augmented Reality Apps & Games.
By: Josip Vlah
Tags: UI, AR, Augmented, Reality, UX

Introduction

Mobile & Web technologies are progressing on daily basis. New applications are rising on the market, games are being published, solutions are being created.

Be aware: This is from my own perspective on the matter. I have been creating and playing around with Augmented Reality for almost 2 years now and my background is development, not design.

This will be tested on Swift with ARKit but the logic applies to other systems.

Augmented Reality has proven to be one of the core technologies which will rise in the next decade. Niantic and their Pokemon GO sky-rocketed AR into amazing new heights and showcased the true power of it. Furthermore with the 2020 Apple Showcase of the new Lidar Sensor on their iPhone 12 as well as Ipad Pro AR is pushing is soon to become the normal standard of immersive interaction.

Tabular View with Foreground or not?

The first thing that separates Augmented Reality from the “usual” user experience design is the camera. First thing when you are doing the mockups you have to think about the placement of elements and which kind of interaction will you have with the elements presented in the AR Scene.

Let’s take a look at some of the “mainstream” applications and solutions and their design thinking.

All Credits go to ©IKEA

First, we are going to take a look at the IKEA Place application which is available for download on iOS.

This application has been developed using the ARKit Framework for Apple devices.

When we are looking at the user experience design of the application it is probably the best way to go if you don't have a lot of object interaction within the application.

Moreover, as you can see the camera covers the whole screen, it ignores the safe area of the screen. This is a good approach because it allows the user to see more real-time camera on the screen while using the application.

Another way to view this is with casual and widely accepted Tab Views.

TabView Example

Tab Views are widely accepted and you can find them in most applications today, this design enables the user to view the application in a familiar environment. One of the drawbacks of the design is minimizing the camera view on the screen of the phone especially if the user has a smaller screen size.

3D Menus & Plane Tracking

Furthermore, if you are interacting with objects in the AR and enabling the user to choose more options you don’t want your app to be crowded. People are visual creatures, they like to get to one place as fast as they can, especially when interacting with immersive software such as AR.

© Rovio Entertainment

Rovio’s Angry Birds is the prime example of good user experience when it comes to more complex AR applications (features wise). For example as seen above on the third screen from the left side they have created a AR menu which tracks your camera based on your position. I have personally tested the game and when it comes to it’s usage on the Ipad Pro with the Lidar Sensor the experience beyond satistiying.

Futhermore, when testing the game using the Iphone XS Max the experience was similar and they have managed to completely work they way around the unnecessary Tab items and menus by creating a “Plane” which holds the buttons and interacts with the users as they move the camera around the surface.

One more example tested on the Iphone XS Max © Rovio Entertainment

When it comes to AR video games as well as other more complex software such as Engineering based AR Applications this is by far the best way to go. Moreover, it provides a more in depth experience to the user and gives it more entertainment value.

Visual, Audio and Text Pointers

It is always important to remember your targeted audience. First things first to remember. Not everybody is still familiar with the AR. This is where Visual, Audio and Text pointers come in handy. For example as shown on the example below, you can create pointers by adding a simple text which in one sentence summarizes the interaction to point the user in the right direction.

This can also be achieve using visuals which represent familiar interactions. As stated above you can place those visuals as a menu of some sort of live on the camera view or need the object. For me I think that it is better to place them near the object and let them be dymanic. Which is summary means that you they will change according to the view angle of the camera.

Audio is also another way to go and look at the AR. Expecially when you are creating and working with Immersive Storytelling Experiences. You can inform the user about the usage of an audio in your application and they can follow the Audio Instructions according to the needs of the application.

© The Verge

Moving to Interaction with Objects & Ergonomics

One other crucial thing to consider when creating and working with Augmented Reality is the interaction with objects and animated elements.

In some specific applications such as IKEA Place mentioned above, uses are able to place the objects onto real-time surface tracked by the camera. They can pinch, rotate, tap, swipe the object and with that directly interact with the object. This is the best principle to look at object interaction. People are familiar with that set of environment and there is no need to create additional elements (e.g buttons) for object interaction. Of course you have specific examples where that is necessary but in most cases this is the way to go.

Here we need to consider the ergonomics for example, if you are targeting the specific audience (e.g You are making in-house application for a specific company which will not be available for wide audience) then you have less things to worry about, especially if the company will provide the hardware such as Ipad Pro or Microsoft Hololens (MR)

Ergonomics are important, we have established that users will interact with the object but hand gestures and multitouch, but think about this. You have an 12.9 Inch Ipad, if you are enabling the interaction only targeting the specific area where the object is placed users will have a hard time interacting with it since they cannot cross over to the other side of the screen with only one hand. It is much easier to enable the users either to chose their preferred model of interaction or by enabling them to interact with it by tapping on the camera view screen.

Conclusion

To conclude the whole story, the main point of Augmented Reality is to make it more user friendly for the wider Audience. First you need to look who will use your application the most and who is your targeted audience. Moreover, try to minimize the usage of Menus as well as Items in Augmented Reality, rather use the hand gestures and point the users on the right direction.