# 3D User Interface Design and Usability for Immersive VR
## 2 Literature Review
### 3D User Interactions Modes  
Gesture-based interaction coupled with VR are the two technologies that truly complement each other. As an HMD-based VR completely blocks out user's hands or body, a gesture-based interaction using hand tracking allows the user to visualize the virtual hands in the VR environment. Additionally, gestures can be easily employed in the VR to perform the tasks that would otherwise require additional interaction elements. Therefore, gestures are an obvious solution to the interaction issues that arise in VR and provides a completely device-free and touch-free solution to many interaction problems.  
[PERSON] et al. [7] explored using slide-based presentations in a virtual environment (VE). They used an infra-red camera for hand gesture recognition, along with a 3D human body tracking system. Additionally, they also employed voice recognition in their system. [PERSON] et al. [8] explored a system for hand gesture recognition and interaction in a real-time vision-based recognition framework. Their work allowed the controlling of the cursor in the desktop environment and they analyzed different tasks that are required to control the cursor.  
Figure 1: A typical Virtual Reality (VR) system process. The user’s movement is tracked and translated into the input for the application. The output of the application is a rendered image sent to the Head Mounted Display (HMD).  
Figure 2: Typical immersive VR system components.  
Recently, the size of the display became so large that the existing methods have more difficultly controlling the application. [PERSON] et al. [9] presented a 3D interaction mechanism using Kinect that allows to control the interface from a distance using the joint tracking information from Kinect to implement the gestures.  
A hybrid system comprising of air gestures and touch gestures is presented by [PERSON] et al. [10]. These hybrid interactions allow small developer group meetings using Kinect. In addition, [PERSON] et al. [11] also explored the applications of gestures in various medical systems for interaction tasks.  
A VE allows for different types of interactions, the variety of which should be addressed intuitively. [PERSON] and [PERSON] [12] developed a gesture-based approach using the Leap Motion controller to manipulate virtual objects in a VR environment. In addition, also using the Leap Motion, [PERSON] [13] presented a 3D user interface for manipulating different contents in a 3D scene. [PERSON] et al. [14] presented a hand gesture-based interaction system that allows the direct manipulation of digital models.  
[PERSON] et al. [15] presented two criteria for categorizing pointing approaches in 3D VEs in the domain of mobile devices. The classifications that are based on problems and factors affecting pointing performance help improve the current techniques and design new ones. [PERSON] et al. [16] presented in-air gestures to complement the touch-based input and study the impact of a touch-free user interface in the presence of a touchscreen on mobile devices. [PERSON] et al. [17] presented approaches for interpreting speech and pen-based gestural input in an efficient manner. They also explored multimodal input that incorporates both gestures and speech.  
Gaze detection, using eye tracking has been employed in a number of interaction mechanisms. The idea of combing gaze with the hand gestures for remote target selection in 2D interaction was presented by [18] and [19]. [PERSON] et al. [19] proposed a hybrid gaze and hand gesture interaction mechanism for large scale displays using Kinect. They created a photo sorting app that was tested with two mechanisms: gestures, and gaze with gestures. Their study showed that adding gaze with gestures significantly improves the usability of the interaction mechanism. In general, users preferred the gaze mechanism, but it introduced some errors that required further study.