## 3 User Interface Overall Description  
In this section, an overall description of the user interface is presented, including the submodules that enable control of the robot team in a human-supervisory manner. As a matter of fact, the system is intended to be used by untrained robotic operators, which might be experts on the scientific scenario where robotic intervention is going to be performed. Figure 9 provides an overall description of the multimodality features of the user interface, which can be summarised as follows:  
* _Cooperative Mission Plan_: This module has been designed in order to provide the operator with a graphical tool (i.e., mission plan) that shows the steps of the planned mission, their corresponding behaviour's state, as well as the possibility to activate the current state of the mission in a graphical manner. The mission is adapted to the current operation in advance, creating the behaviours by using scripts (see Listing 1) which implement cooperative synchronisation of the robots by processing feedback and control signals appropriately, as presented in more detail in the next section. It is important to explain that the script module enables the possibility to implement more sophisticated modes of interaction, such as implementing specific object recognition modules for certain objects as well as their grasping determination procedures. * _Unilateral Input Devices_: These are the devices that let the operator control the system without providing feedback. * _Bilateral Input Devices_: These devices are specifically designed for manual teleoperation modes, allowing control of the system by using joints and world coordinate commands and giving force feedback to the operator, according to the force sensor integrated into the robot's arms. * _Bilateral Output Devices_: These are devices that, for the moment, are used for representation of the information to the user. In the context of the TWINBOT project, the virtual and augmented reality components already enable input and output control signals. Also, haptic devices are attached to the operator's fingers to provide force feedback. Audio is also a very important feedback source to the operator. * _Vision and Grasping_: This module is implemented as a vision library that runs on the server side and can also be invoked via scripts from the client's side. It enables the user to interact with the robots in a higher-level way (i.e., objects and regions of interest). The module is able to track and detect trained metallic objects such as pipes in a robust manner. Also, it enables the calculation of distances to targets using on-hand monocular cameras (e.g., laparoscopic). The grasping determination module, which allows for calculation of potential grasping points for a given contour, can be activated by the user during the operation. This grasping functionality is demonstrated in one of the experiments presented in this article, facing in simulation the problem of grasping a big object (e.g., pipe) with two robots, taking into account that, once the object has been approached, it is partially perceived by the cameras. The experiment needs the user's supervision to select the required grasp in each robot, apart from monitoring the cooperative grasp mission to avoid unexpected situations such as failures in perception. For more details about the system multimodality, for example, the way the controls are used to interact with the robots and user interface, please refer to [2]. The HRI adapts to the current configuration of the robot functionality that has been extended for multi-agent operations. To accomplish this function, the user is required to select the current robot team configuration, which can be applied to both the real robotic team as well as the training simulator. This is especially useful during the preparation time of the intervention. In fact, specific tools and cooperative behaviours scripts are implemented during this phase. Once the user interface establishes a connection to the sensors and robots, the control view comes up.