# Closed-Loop Robotic Arm Manipulation Based on Mixed Reality
## 5 Case Study  
For the validation of the developed robotic arm navigation tool, a set of experiments has been set up in a laboratory-based machine shop. More specifically, a UR10 collaborative robot has been used which is installed in the machine shop. It is stressed that there has also been developed an additional functionality in order to enable users to create different configurations of the robotic arm and its surroundings which has facilitated experimentation with different configurations. Initially, the testing was focused on the 3D scanning of the robotic arm surroundings and, more importantly, on the 3D regeneration of the 3D map by the framework. Upon completion of this step, either the engineer or the shop-floor technician is able to navigate the robotic arm remotely from the provided AR-based GUI. In the experimental tests of the framework, five engineers and shop-floor technicians participated. Each of the participants had to perform a set of actions in two different situations, i.e., the current situation, involving the hardwired robotic controller and the developed wireless application. In each of the experiments, the number of errors was measured, as was the user's awareness, the ease of use, and the time needed for completion of the assigned tasks. An error is defined as an action that leads to conflict between the human operator and the robot and also as a robot motion that leads to conflict between the robot and any other object in its surroundings. It is stressed that the experiments were  
Figure 5: AR visualization of the robot safety zones and the next robot position.  
conducted with the use of the UR10 robotic arm, which is considered to be collaborative, and upon collision the robot motion was automatically halted. Thus, no health risk was induced during the experiments. In order to record user awareness and ease of use, a short interview with the participants was performed after the end of the experiments. Finally, the whole process was recorded for each individual, including the number of errors and the total time needed for each experiment to reach completion.  
Six metrics have been used in evaluating HMI: (1) Task effectiveness, (2) Neglect Tolerance, (3) Robot Attention Demand, (4) Free Time, (5) Fan Out, and (6) Interaction Effort, on a scale of 1 to 10. A short description of each metric is presented in Table 2[38]:  
As far as the experiment scenarios are concerned, two different scenarios have been tested. In the first scenario, the operator had to work in collaboration with the robotic arm in order to assemble a mechanism. The assembly process involved the collection of the assembly components by the robotic arm and their placement on the assembly, while the operator had to secure the assembled components with screws. It is stressed that during the execution of this scenario the participants selected a set of predefined movement sequences from the GUI of the developed application. Such movement sequences were pre-created by the engineering department and uploaded on a Cloud database which served as a repository for such assembly scenarios. However, through the GUI of the application, the shop-floor engineer could alter the parameters of the movement sequence so that the collaboration between the human and the robotic arm was further facilitated, considering the safety limitations. For the second scenario, the operator had to transfer high-volume objects with the use of the robotic arm under low visibility circumstances. In Figure 6, the assembly sequence, in the form of steps, is presented.