# Augmented Reality-Based Interface for Bimanual Robot Teleoperation
## 3 Proposed Augmented Reality-Based User Interface  
In order to overcome the aforementioned problems of the conventional PC-based interface, this work proposes the use of AR technology to improve the user ergonomics and task performance. In particular, the conventional PC-based interface used in the previous setup (see Figure 1a) is replaced by an AR headset in the new setup--see Figure 4--allowing the user to see the relevant information in the form of holograms while still seeing at all times the real elements involved in the task: robots, workpiece, tool, etc. Note that the remaining elements of the new setup (see Figure 4) are the same as in the previous setup (see Figure 1a): an STR with an F/T sensor and a cylinder-shaped tool with a piece of cloth; a WR with a flat workpiece of methacrylate attached to the end-effector using a self-made piece; and a gamepad to command both robots. Figure 4: New setup used for the real experimentation. Figure 5 shows the methodology considered in this work to develop and validate the proposed AR-based interface. Although this methodology is used below to design the AR interface for the specific bimanual robot teleoperation task at hand, it is generic and, in general, it can be applied to design AR interfaces for other types of applications. Firstly, the requirements of the applications were established based on the opinions of several users who previously tested the conventional PC-based interface. These requirements are summarized in Table 1.  
A mockup design was developed taking into account this information. The designed AR-based interface has, from a functionality perspective, two kinds of virtual objects: firstly, \begin{table}
\begin{tabular}{l} \hline The user should have the option to see the full boundaries when required \\ The part of the boundary activated should be indicated (e.g., visually, sound, etc.) \\ STR tool reference direction should be indicated \\ WR rotation reference direction should be indicated \\ The new interface should use a similar interaction device to that of the previous PC-based \\ interface (i.e., gamepad, joystick, or similar) \\ Alarm sounds should be used to indicate boundary activation \\ The user should have the option to remove all holograms \\ Holograms should not disturb the user visibility during the task \\ The user should have the option to configure, activate, and deactivate the alarm sounds \\ \hline \end{tabular}
\end{table} 
Table 1: Application requirements. Figure 5: Flowchart of the methodology proposed in this work for designing the AR-based interface. those representing the STR and WR reference indicators; and, secondly, those corresponding to the boundaries information. In order to develop both kinds of virtual objects, several tools and strategies related to the mockup design were used. These preliminary designs were validated by some users before their implementation. Once the preliminary design was finished, the following step was to study the best option of AR headset to be used for the application at hand. Several considerations were taken into account, especially the following: first, the capability of the device to be used in industrial environments; second, the stability of the holograms, which is important when working in this kind of application; third, the computational power of the device; fourth, the sound capabilities; and fifth, the communication capability (i.e., Bluetooth and WiFi). Note that most AR headsets in the market accomplish the aforementioned requirements. However, among all of them, Microsoft _HoloLens glasses_[76] were chosen because the second generation of this device offers several services that could be added to the final version of the interface according to the company needs [77]. Once the AR headset was selected, the interface was developed. Using a PC workstation, the proposed virtual objects were created and assembled in a virtual space using Blender 2.7 [78] and _Unity_[79], respectively.