Tips on designing for AR/VR (with examples)
By: Fabin Rasheed
Tags: Design, Augmented Reality, Virtual Reality, Technology, UX

A few tips on designing for AR /VR with examples from my experience.

Over the last few years I have designed and prototyped multiple AR /VR experiences. Having worked with 3D for some time and being interested in immersive digital experiences, AR/VR was an obvious choice for me. I started off creating simple experiences and eventually moved into creating full scale products. Although not an all-inclusive list, following are some design tips from my experience with examples from my projects:

Tip 1: Remind the user regularly where they are in an experience

One of my earlier projects on the field of *R was a VR project — a guided walkthrough of historical sites for students in rural schools. The Google Cardboard had just released during that period. I used the Google camera app to capture 360 photos of a historical site and labelled them with the local Telugu language. These photos were used to create a step by step walkthrough of the site along with audio narration, again in Telugu.

Presence in terms of location, status, remaining content etc. was found to be very important to a VR experience. One of the key learnings from the intial stages of this project was that the students had trouble learning the relative locations in the walkthrough, ie. they were not able to memorise what exactly comes after a particular doorway.

I solved this by using an interactive map inspired from popular games. The introduction of the map helped establish relation between locations much better and made it easier t for the students to remember relative locations. Thus, reminding the user where they are and what are nearby or reachable or what’s next is very important in any immersive experience.

Tip 2 — Increase immersion by including other human senses

One way to increase immersion is by including other human senses in the experience. I did a project called fiSeeyo marrying Haptics with VR. It was a remote physiotherapy solution for upper limb motor function enhancement. A doctor could remotely use a 3D mouse to give physiotherapy exercises for the upper limbs of a patient. The patient on the other side of the system uses VR and a Haptic pen to perform gamified tasks prescribed by the doctor.

Haptics and VR seemed a perfect match since manipulation of objects with different weights and placement based on touch feedback — like putting a box in a hole - was much more intuitive with the Haptic pen and the cardboard headset. Adding the extra sense of touch made it not only beneficial but aided in the manipulation of 3D objects. Thus it helped in increasing the involvement of the user in the experience.

Tip 3– Minimize user movement

An interesting problem that I had to tackle was how to rotate an object in 3D. During my time in Adobe I did a project on sculpting in Augmented Reality. (It went viral as one of the first sculpting experiences in AR) . The object to be sculpted was a sphere. Since it was designed for AR the user could go around the sphere and sculpt it. This seemed well and good in the beginning, but then it started weighing down on the users’ experience as fatigue set in. I realised that moving around the object can be used minimally for something like visualisation. If the user has to work on the object continually, we should give an option to rotate the object without the user having to move.

This is when I introduced something called an “orbiter”. This is a circle at the bottom right corner, which on-tap will centre the sphere. On dragging the orbiter, the object in focus will rotate. Calibrating some of the control variables I got a zen setting which made it smooth to rotate and sculpt the object. The users loved it! I found that this could also be used in 3D applications to manipulate 3D views as well.

Tip 4 — Use intuitive gestures

I absolutely love gestures. They are some of the most expressive ways of interacting with technology (maybe after voice). But they should be used minimally and should be made as intuitive as possible. Ameyt World was a project for assembling 3D blocks together and giving them interactivity in mobile AR. It was more like legos with actions (yes, Minecraft AR stole my thunder). I started off by using gestures for manipulating the view in the creation mode.

This included two-finger pinch-zoom, one finger drag to rotate and two-finger movement for panning. But the problem was that users were accustomed to panning using one finger (eg. Google Maps). Rotation became a new interaction. So I decided to follow the popular model and gave one finger drag to move and introduced two finger to rotate.

This led to another problem. Building a new block was done by tapping on an existing block. Thus a single finger swipe became a tap first (which built a block) and then drag. On top of this, there was another problem. I let users rotate a block by swiping it:

So, tap drag erroneously built or rotated a block. This led to a very exciting design-engineering challenge to find sweet spots in dragging gesture time, building gesture time and block rotating gesture time. Since I was designing and developing at the same time, I could go through multiple options with users and find the sweet spot. Adding a UI element could have solved this problem easily, but I did not want to clutter the UI and overload users (esp. beginners). The final solution which had lesser but intuitive gestures worked well and was intuitive even for beginners:

Thus, at many places where gestures are implemented, its about finding a right balance between the number and type of gestures and UI elements.

Tip 5- When AR is limited, think of VR

For pre-visualisation and planning of movie shots, I did a project which helped movie directors plan a shot in AR at a location. A director could arrive to a location and using a mobile device place props, lights, camera angles on the location or even leave audio or written notes in space in Augmented Reality. This helped them plan a shot and pass it on to his/her colleagues as instructions.

The problem came when we realised that the location is not available for a long time during planning stages. This is when we introduced a VR mode in the AR application where the 360 view of the location could be captured and the user could later plan shots at the comfort of their homes.

This solution could also be adopted for other applications like furniture placement, interior designing etc.

Bonus Tip - Fly high in the sky

A fish swims in the water, a lion roams the jungles and a bird flies high in the sky. Design for context. Try to leave those 3D models with animations in their actual environments. Yes a bird would look cool flying inside your office room, but it would look even better and natural flying high up in the skies. Also use elements creatively to accomplish a purpose.

The clip above is from a project called Aquila. The 3D model of an eagle (rigged and animated) responded to voice commands like “have food”, “fly away” etc. similar to some of the existing pet apps. But unlike other pet apps, this app was an interface for flying a Drone. When the command “be my eyes” is uttered to the eagle, the feed on the mobile device was changed to the Drone’s feed. Hence the metaphor of the eagle flying and being the eyes in the sky fit well and helped accomplish the task of initiating the drone feed creatively with AR. And of-course the AR eagle flying in the sky looked more natural.