## 5 Applications in Human-Robot Interaction
### Human-Robot Collaboration  
The creation of XR systems enabling robots to perform collaborative tasks with humans has been widely explored. In this context, ref. [85] presented a set of techniques for ad hoc HRC (i.e., collaborative activities where the robot works without pre-planned actions) using digital tabletops and mixed reality. However, restrictions presented by the COVID-19 pandemic constrained the experimental validation of the system in a VR simulation. A solution that links AR and eye-tracking technologies to enable 3D object detection in a collaborative way was presented in [87]. A key novelty that the authors of [87] presented is the use of gaze information to help detect and segment unknown objects. For this, they used the HoloLens 2 AR device, which has a built-in eye tracker over Unity, and for which information is connected with ROS using ROS#. The authors claimed to be the first to use AR in an HRC setting to segment unknown 3D objects without deep learning techniques, and their system was able deal with cases where existing deep neural learning techniques have failed. Other related works using Hololens and ROS (connected by ROS#) to facilitate HRC activities have been described in [84; 91]. A novel approach using Hololens for empowering human-centric assembly (HCA) by combining a digital twin [92] developed in a Java-based 3D environment with AR methods (executed in Unity) and event-driven function blocks [93] has recently been proposed by [88]. Validation of their system was \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline \multicolumn{3}{c}{**Article**} & \multicolumn{3}{c}{**Hardware Integrated**} & \multicolumn{2}{c}{**Software Tools Integrated**} \\ \hline
**Year** & **Reference** & **XR Type** & **Robots** & **Devices** & **Communication** & **Interaction** \\ \hline \multirow{3}{*}{2018} & \multirow{3}{*}{[83]} & \multirow{3}{*}{AR} & \multirow{3}{*}{Turtlebot} & \multirow{3}{*}{Tablet} & ROS (no details) & \multirow{3}{*}{Vorria SDK} \\  & & & & & & on how Unity and ROS are connected) & \\ \hline \multirow{3}{*}{2020} & \multirow{3}{*}{[46]} & \multirow{3}{*}{AR} & COMAU & \multirow{3}{*}{Transwatch, COMAU Racer5} & Smartwatch, & ROS Bridge and & Microsoft \\  & & & & \multirow{3}{*}{Holdens} & Projector, Camera & & \\  & & & & HoloLens & & & \\ \hline \multirow{3}{*}{2020} & \multirow{3}{*}{[84]} & \multirow{3}{*}{AR} & KUKA KR6 & \multirow{3}{*}{HoloLens} & ROS and ROS bridge \#, & \multirow{3}{*}{Vorria SDK} \\  & & & & & & bridge \#, & \\ \hline \multirow{3}{*}{2021} & \multirow{3}{*}{[85]} & \multirow{3}{*}{VR} & Franka Emika & N.A & ROS, ROS\# & N.A. \\ \hline \multirow{3}{*}{2021} & \multirow{3}{*}{[86]} & \multirow{3}{*}{AR} & Turtlebot2 & \multirow{3}{*}{Tablet} & ROS, ROS & N.A. \\  & & & & & & \\ \cline{1-1} \cline{5-5}  & & & Turtlebot3, & & & & \\ \cline{1-1} \cline{5-5}  & & & & & & Bridge, TCP/IP, Photon & \\ \hline \multirow{3}{*}{2022} & \multirow{3}{*}{[87]} & \multirow{3}{*}{VR} & \multicolumn{3}{c}{Sotos G5 mobile robot} & HoloLens 2, & \multirow{3}{*}{ROS, ROS\#} & Mixed Reality Toolkit \\ \cline{1-1} \cline{5-5}  & & & & & & \\ \cline{1-1} \cline{5-5}  & & & & & & \\ \hline \multirow{3}{*}{2022} & \multirow{3}{*}{[88]} & \multirow{3}{*}{MR/AR} & ABB industrial robot & HoloLens & FB network & N.A \\ \cline{1-1} \cline{5-5}  & & & & & & \\ \cline{1-1} \cline{5-5}  & & & & & & \\ \cline{1-1} \cline{5-5}  & & & & & & \\ \hline \multirow{3}{*}{2022} & \multirow{3}{*}{[89]} & \multirow{3}{*}{VR} & Universal Robot & Xsens motion, & \multirow{3}{*}{HTC VIVE} & ROS, ROS\# & N.A \\ \cline{1-1} \cline{5-5}  & & & & & & \\ \cline{1-1} \cline{5-5}  & & & & & & \\ \cline{1-1} \cline{5-5}  & & & & & & \\ \cline{1-1} \cline{5-5}  & & & & & & \\ \cline{1-1} \cline{5-5}  & & & & & & \\ \cline{1-1} \cline{5-5}  & & & & & & \\ \hline \hline \end{tabular}
\end{table} 
Table 8: Articles presenting XR solutions supporting Human–Robot collaboration tasks. centered on performance evaluation in an engine assembly case.