## 5 Experimental Results and Implementation into the Assembly Process
The principle of parts detection can be described in these steps:  
1. Get images from the mixed reality device and collaborative robot hand;
2. Send images to the front of data;
3. Get images by NVIDIA Xavier embedded device;
4. Inference images by selected CNN model and acquire part position data;
5. Send bounding box (contour) and classification value by TCP communication to both devices (mixed reality and collaborative robot). The implementation principle of parts recognition into the collaborative work cell is shown in Figure 16. \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Testing** & **Platform Type** & **Platform Parameters** & **Delay [ms]** \\ \hline NVIDIA APU & Xavier AGX & Maximum power 30W & 690.71 \\ Conda GPU & GPU & Nvidia RTX 2060 & 166.00 \\ Conda CPU & CPU & Intel i9-10900 KF 3,7 Ghz & 701.12 \\ WSL2 CPU & CPU & i7-9750H 2,7 Ghz & 1522.88 \\ WSL2 GPU & AGX & NVIDIA GTX 1660 Ti DirectML & 5807.61 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: The acquired inference results for the Mask CNN model. Figure 15: SMART vision system of the collaborative work cell: (**a**) CNN processing unit NVIDIA Xavier AGX; (**b**) Mixed reality device camera; (**c**) Collaborative robot ABB Yumi Vision system integrated into the right hand. Images captured by all vision systems (static dual 4K e-con cameras connected to NVIDIA Xavier AGX and JetPack 4.4, integrated Cognex 7200 camera in ABB Yumi right hand and Microsoft Hololens 2 internal head camera) are shown in Figure 17.  
The application for the mixed reality device Hololens 2 is coded in the same software (Unreal Engine) as a sample generation software but does not use Python programming language and is coded only by Blueprint programming language with UXTool library. The designed calibration principle of all used vision systems to one Cartesian coordinate is shown in Figure 18 and their synchronization is realized in these steps:  
* NVIDIA Xavier AGX system with e-con dual 4K camera is static and default zero position is set as fixed X, Y offset in pixels;
* ABB Yumi collaborative robot position is realized by offset from home point to the axis of the left-hand vision system in Rapid programming language;
* The mixed reality device Microsoft Hololens 2 is synchronized by QR code placed on the lower-left corner of the assembly table and rotation measured by integrated MEMS sensors. * The information about part position adjusted by QR code is shared between all devices as Part identification, where position is X, Y; size W, H, and contour of the detected part is represented by array of points C.  
Figure 16: The principle of an experimental assisted assembly work cell with CNN processing unit, mixed reality device, and collaborative robot. Figure 17: Images captured in the assisted assembly work cell by: **(a)** Nvidia Xavier AGX e-con camera; **(b)** Mixed reality Microsoft Hololens 2 head camera; **(c)** ABB Yumi Cognex Vision system. Current deep learning frameworks provide only image augmentation, which only reduces the number of images that are needed to be prepared. It means, the most monotonous works in deep learning implementation to real application still exist. This is the main reason why is not profitable to use deep learning in small series assembly tasks. On the other hand, an automated generation of training samples from CAD models, which are available before production starts, can help to implement more assisted assembly solutions into practice. The research field in an automated generation of training samples for CNN models from 3D virtual models has a big potential to expand. Current progress in GPU with real-time raytracing can provide new rendering possibilities to reach cinematic quality in object visualization and fast preparation of virtual samples. An interesting project is Kaolin from NVIDIA, a modular differentiable rendering for applications like high-resolution simulation environments, though it is still only as a library under research.