# A Voice-Enabled ROS2 Framework for Human-Robot Collaborative Inspection
## 3 System Implementation
### Digital Computation and Visualization Layer
#### 3.2.4 Quality Inspection Server  
The quality inspection server basically refers to the training of the model proposed in Section 2, as well as its integration in the production in order to be utilized in collaboration with different modules. For the implementation of the model's architecture, PyTorch's (version 2.1.0) framework was utilized. PyTorch is an optimized Deep Learning tensor library based on Python that provides tensor computation with strong GPU acceleration support for automatic differentiation for creating and training neural networks. For the visualization and the collection of the data, dedicated Python (version 3.8.10) libraries like OpenCV and PIL were used. For the integration of the model and the communication with the different modules, an action server based on the ROS2 framework was utilized. Figure 8: Digital scene including the object to be inspected and the robot manipulator. Dataset Generation and Augmentation  
For an object detection process, raw images, as well as the bounding boxes of the desired objects among these images, are the necessary information for the training procedure to begin. However, the different models implemented for this object detection task do not employ the same method of data processing. That means that different formats of data are available. For instance, one popular format used for object detection is the COCO (Common Objects in Context) [63] dataset, which provides labelled images with corresponding bounding boxes for various object categories. In the current implementation, the dataset consists of 1200 images, which were collected using an rc_visard65 camera by ROBOCEPITION GmbH (sourced from Munich, Germany) [64] and python-based scripts for acquiring and storing data through the ROS2 network. The dataset includes images in which the objects are exhibited in different lighting conditions, as well as backgrounds for helping the model generalize and avoid overfitting. Specifically, for this work, a set of six categories was established to denote whether objects on a motor, which is the component of interest for the inspection, were correctly or incorrectly positioned. The corresponding quantity of images for each class is presented in Table 3.  
The task of labeling the images is accomplished manually by employing a graphical user interface (GUI) developed in Python. For each image of the dataset, there is a corresponding label file that consists of all the information regarding the objects of interest. A label file is basically a text file with every row dedicated to a different object on the corresponding image. Each row has five different values; the first one is an integer associated with the class of the object, and the other four describe the bounding's box center coordinates (x, y) as well as its width and height. An indication of the corresponding labeled data using an automotive engine is visualized in Figure 9. \begin{table}
\begin{tabular}{c c c} \hline \hline
**Classes** & **Training Images** & **Validation Images** \\ \hline Correct Screw & 661 & 72 \\ Faulty Screw & 743 & 82 \\ Correct Connector & 383 & 42 \\ Faulty Connector & 469 & 52 \\ Correct Valve & 259 & 28 \\ Incorrect Valve & 289 & 32 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Training and validation data. Figure 9: Labeled data of an automotive engine part. Moreover, within this scenario, data augmentation methods were employed to apply diverse transformations to the original dataset, thereby generating extra images. These transformations are implemented through Python-based functions, and involve rotations, flips, crops, and more sophisticated techniques such as mosaic and mix-up. Inspection Model Training  
In the present work, the training process of the implemented model spans over 300 epochs, employing a linear learning rate that gradually decreases. This ensures that the network can adapt its weights effectively in later epochs in order to learn new features and perform higher accuracy in the object detection task.