# Augmented Reality-Based Interface for Bimanual Robot Teleoperation
## 1 Introduction
### Previous Research
#### 1.2.2 Assisted Robot Teleoperation  
The remote control or _teleoperation_ of robots by users has been studied for many years [36] and still represents a relevant research field in robotics. Robot teleoperation is required for a wide variety of reasons: when the working environment is dangerous to humans (e.g., in space [37], radioactive zones [29; 38], aerial zones [18; 39], or underwater areas [35; 40]); when performing rescue operations [41], and when precision surgeries need to be performed [42; 43; 44; 45], among others. Nowadays, there are sophisticated artificial intelligence (AI) techniques that allow the automation of complex tasks that not so long ago had to be performed by means of human teleoperation. However, despite current advances in AI, there are still many tasks that cannot be fully automated due to their complexity or subjectivity. However, these tasks can be partially automated, allowing the cooperation between human and robot, introducing shared-control architectures [46]. Hence, many recent contributions have focused on human-robot interaction and, more specifically, on advanced robot teleoperation [16; 30; 31; 47; 48; 49; 50; 51], which is also the case of this paper. Telepresence [36] allows the user to perform the robot teleoperation task by means of an interface, achieving a result less dependent on their skills. Telepresence is currently a trending research topic thanks to the introduction of new technologies, such as augmented and virtual reality [51], visual interfaces [42], haptic devices [52], or a combination of them [16; 30; 43], to perform direct control teleoperation. For instance, the authors in [53] proposed a low-cost telerobotic system based on virtual reality technology and the homunculus model of mind. In this case, the user was able to move both robotic arms according to the dynamic mapping between the user and the robot developed. In addition, the user was able to see the real workspace in the virtual environment using feedback from a camera. Similarly, the authors in [54] proposed a virtual reality interface based on the three-dimensional coordinates of the shoulder, elbow, wrist, and hand captured by a Kinect camera to model the geometry of the human arms and perform the mapping with the robot arms. As in [53], the user receives visual feedback from a camera placed on the robot. In both cases, robot manipulation tasks were performed. However, for more complex tasks (e.g., surface treatment tasks), interfaces developed with virtual reality techniques can increase the time of completion of the task and worsen the quality of the surface finishing, compared to that obtained by the human operator using direct teleoperation. This is due to two facts: on the one hand, when using virtual reality, it is difficult to incorporate all the necessary information of the task in the virtual world and in real time, and, on the other hand, the user already has a real notion of the robotic system and, hence, is able to guide it naturally and intuitively using direct teleoperation. For this reason, in order to obtain the best of both worlds (i.e., direct teleoperation and teleoperation based on virtual reality), the present work proposes to use interfaces based on augmented reality to provide a solution to a greater number of industrial tasks carried out with bimanual robots. Other approaches try to ease the teleoperation of bimanual robotic systems, such as in [48], where the authors developed a bimanual robot application in which a robot arm is teleoperated to grasp the workpiece, whilst the other robotic arm is automatically controlled using visual servoing in order to keep the workpiece visible for the camera. Since the performance of robot teleoperation may rely on the user's skills, some approaches are focused on incorporating restrictions that prevent the user from commanding the robot into failure situations.