## 4 Experiments
### Experiment II: Validation in Underwater Simulator  
The same behaviour configurations used in the previous experiment have also been utilised and adapted accordingly in order to be able to pick up and transport in a cooperative manner a pipe from the seafloor in the simulation training tool for the TWINBOT project (see Figure 16). The intervention video can be found in Section 4.5. As a matter of fact, in this experiment, apart from the leader-follower behaviours, the mission plan allows for the activation of a closed-loop approach to the target behaviour by using the depth sensor. Also, recovery transport to the surface is performed using a similar procedure. In Figure 17, the mission plan behaviour's state selection and the effect on the user interface are shown. The usability of the HRI cooperative behaviours has been validated by performing first an intervention on the training simulator using two operators: the first one controlling the auxiliary robot and the leader and the second operator controlling the second mobile manipulator. It was necessary to perform the intervention by having an operator leader that decided when a synchronised operation should be done. As it can be seen in Figure 18, the intervention was performed five times and it failed \begin{table}
\begin{tabular}{c c} \hline \hline  & **Time-Based Efficiency** \\ \hline Without cooperative behaviours & 0.12 goals/min \\ With cooperative behaviours & 0.37 goals/min \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Time-based efficiency of the transport task using the two CERNBots. Figure 14: Task sequence carried out by two cooperative robots: (**1**) approaching the target, (**2**) grasping, (**3**) lifting, and (**4**) transporting. three times, especially in the transport of the pipe (step 5), with some difficulties in the coordinated grasping (step 4). Moreover, the experiment was performed using one operator and the cooperative behaviours, according to the mission plan (see Figure 19). The first attempt failed due to excessive pressure while grasping the object. The following attempts were successful and very smooth when approaching the seafloor and when transporting the pipe to the surface. Table 2 shows the time-based efficiency of task execution. In this case, the calculation of the metric has been slightly modified. In its definition, the time-based efficiency is computed as the sum of \(\frac{n_{i}}{t_{i}}\), where \(n_{i}\) is 1 if the task was accomplished and 0 if the task was not completed and where \(t_{i}\) is the execution time of the task. This means that, in the context of this experiment, a failed task out of the  
Figure 15: Mission plan sequence: (**1**) approaching the target, (**2**) grasping, (**3**) lifting, and (**4**) transporting. five in the entire operation would impact only the execution time of that specific task. However, it is safe to assume that a failed task compromises the entire operation. For this reason, for calculation of the time-based efficiency, if a task was not completed, the entire efficiency was considered 0. Considering the values of Table 2, it is possible to use the expert-relative efficiency metric to compute the percentage gain in terms of efficiency by using the scripting behaviours and a single operator. In this case, the ideal completion time for the task of an expert user has been replaced by the completion time of the single operator using scripting behaviours. The resulting value is \(E_{e}=31.8\%\), which is the relative efficiency of two operators in the completion of an operation with respect to the single operator using behavioural scripts to accomplish the same tasks. It appears that the single operator is over three times more efficient in this context. \begin{table}
\begin{tabular}{c c c} \hline \hline
**Time-Based Efficiency** & **Average** & **Standard Deviation** \\ \hline
2 operators & 0.37 & 0.08 \\
1 operator & 1.38 & 0.20 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Time-based efficiency of the transport task in the simulated scenario.