# A Systematic Review of Virtual Reality Interfaces for Controlling and Interacting with Robots
## 3 Results and Discussion
### Usability  
The Usability category highlights user experiences of VR interfaces. At the moment, research in this area is primarily focused on comparing traditional interfaces for robots to a VR interface. Traditional interfaces include two types. The most common one utilizes a monitor to view data and either a keyboard and mouse or a gaming controller to interact with the robot. The second one uses direct manipulation where users can physically grab and move the robot around and \"program\" the robot to complete a task, this is also frequently called learning from demonstration. Ref. [39] compares four different interfaces for a robotic manipulator: direct manipulation, computer (keyboard, mouse, and monitor), a partial VR interface that only uses positional hand tracking and uses a monitor instead of a VR headset, and a full VR interface with positional hand tracking and a VR headset. In an 18 person user study using a robotic manipulator in a cup stacking task, they found that their full VR interface was significantly better compared to the keyboard and monitor interface with a 66% improvement in task competition time, lower workload, higher usability, and higher likability score. Additionally, 5 of their 18 users were never able to complete the task with the keyboard and monitor interface, but they were all able to complete the task with the full VR interface. However, the full VR interface was slower and higher workload compared to direct manipulation, but it did have marginally higher usability and likability scores. Furthermore, when comparing the full VR interface to the partial VR interface, they found that task completion times were not significantly faster for the full VR interface, but it had higher usability and was marginally more likable than the partial VR interface. A similar 11 person user study was done in [51], where they compare their VR programming interface with a direct manipulation interface and a keyboard, mouse, and monitor interface. Their VR interface has one major difference from most others in the area, in that they use gesture recognition to teleoperate the robot rather than VR controllers. Their results show that the direct manipulation approach on average took the shortest amount of time and caused the smallest number of collisions. However, the VR approach was considered more natural among users and still performed better on average in performance time and number of collisions compared to the keyboard and monitor interface. Ref. [52] evaluates the use of VR for teleoperation and telemanipulation tasks using mobile robots equipped with a robotic arm and compares displaying camera streams on a monitor and a VR headset to display streams from stereo cameras to provide stereo-vision for an operator. In their 16 person user study, they found that in driving and observations tasks that there was not substantial difference in task completion times. However, for manipulation tasks there was a 20-25% increase in completion time when using the traditional interface over the VR one. They attribute these results to the stereo-vision the VR headset provides over a monitor, and that for manipulation tasks the addition of depth perception increases people's capability to estimate distances, but this benefit is not significant for driving or observation tasks. Ref. [53] aims to see if VR interfaces lead to improvement in workload, situational awareness and performance for operators of multiple robots. They conduct an 8 person user study comparing a traditional keyboard, mouse, and monitor interface to a VR interface in both an indoor and outdoor scenario with three types of robots: aerial, ground, and manipulator. In general, they show that results are better with the VR interface over the traditional one in terms of operator performance, workload, and situational awareness. However, similar to [52], they found that performance when driving the mobile robot was better with the traditional interface. Ref.