## 5 Applications in Human-Robot Interaction
### End User Programming of Industrial Robots  
Robot programming is an emergent area where game engines and XR technologies are implemented for creating HRI systems. In these applications, the reduction of training effort available when using intuitive, simple, and usable interfaces is fundamental. In this context, ref. [73] have presented GhostAR. This time-space editor combines the Oculus Rift and the ZED Dual Cameras to enable users to program and edit the actions of robots through Programming by Demonstration (PbD). This system can display \"ghosts\", images representing a timeline of captured human motions. These \"ghosts\" are used as time-space references, which can be used to edit HRC tasks using a virtual avatar. PbD solutions using the HoloLens have been explored using AR [58] and MR [74] methods, providing promising results compared to conventional programming through teaching pendant and kinesthetic programming. In this context, ref. [75] have proposed a markerless solution that records and translates the operator's hand movements into robot motions and commands using Hololens 2. However, HoloLens is considered an expensive device [76; 77; 78]. In contrast, smartphone-based AR applications such as [55] can represent a more cost-effective solution. The authors of [75] claim that their proposed approach \"is computationally lightweight compared to other Learning from Demonstration techniques, such as Machine Learning approaches\". However, they claim that this solution may not be suitable for applications that require high accuracy in robot motions. A projection-augmented tabletop interface, denoted as PATI, was proposed in [48] to overcome the perception limits (visual tracking and object referencing) of PbD. In PATI, the task-level specifications defined in the TUI are translated to robot commands and motion plans. For this, the PATI system can detect touch gestures and track objects in the working environment. The authors of PATI claim that their  
\begin{table}
\begin{tabular}{c c c c c c c} \hline \hline  & \multicolumn{2}{c}{**Article**} & \multicolumn{2}{c}{**Hardware Integrated**} & \multicolumn{2}{c}{**Software Tools Integrated**} \\ \hline
**Year** & **Reference** & **XR Type** & **Robots** & **Devices** & **Communication** & **Interaction** \\ \hline
2019 & [48] & AR & UR5 & Kinect 2, projector & ROS, ROS\# & TouchScript, Point Cloud \\ \hline
2019 & [73] & AR & \
\begin{tabular}{c} GripperBot, \\ Cambot, \\ bot \\ \end{tabular} &
\begin{tabular}{c} Oculus Rift \\ Sensors, \\ stereo camera \\ \end{tabular} & ROS, ROS\# & N.A \\ \hline
2020 & [74] & MR &
\begin{tabular}{c} KUKA iiwa, \\ UR10e \\ \end{tabular} & HoloLens & ROS (no details and ROS are connected) & MoveIt, Mixed Reality Toolkit \\ \hline
2020 & [55] & MR/AR &
\begin{tabular}{c} OpenManipulator \\ X \\ IRB 2600 \\ \end{tabular} & Smartphone & Bluetooth & Google’s AR-Core & \\ \hline
2021 & [75] & AR &
\begin{tabular}{c} UR5 and ABB \\ IRB 2600 \\ \end{tabular} & HoloLens 2 & ROS, ROS\# & N.A. \\ \hline
2021 & [58] & AR & UR10 & HoloLens & ROS, ROS\# &
\begin{tabular}{c} ZXing, \\ Reality Toolkit, \\ MoveIt \\ \end{tabular} & Mixed Reality Toolkit \\ \hline \hline \end{tabular}
\end{table}
Table 6: Articles presenting XR solutions for programming robots using game engines.  
system can be learned in a shorter period of time than state-of-the-art PbD systems, and can enable users to be more efficient, producing less physical and mental workload.