# A Voice-Enabled ROS2 Framework for Human-Robot Collaborative Inspection
## 5 Results  
The results of the first set of experiments in terms of timing are analyzed in Table 6 and Figure 13.  
Comparing the cycle time results of the proposed framework with the current state of the production, there is a significant reduction from the existing 60 s that are devoted for inspection. The developed framework provides effective collaborative inspection functionalities between the human and the robot, and manages to complete the inspection tasks in 39.6 s, on average, which is 20.4 s less, or a reduction of 33.4%, compared to the current state (Figure 14). Additionally, compared to previous iterations of similar quality inspection modules [70,71], the proposed solution results in a significant reduction in the average time allocated to robot movements by 19-7 s or by 24.2-11.1% (Table 7), respectively. Finally, the developed framework expands on previous architectures [71] by providing more flexibility in the control of the human in the process of quality inspection. The second set of experiments contains the calculation of the accuracy of the model over training in a validation dataset, as well as its performance in real-time operation. The results for the training accuracy are displayed in Figure 15. The final trained network scored 0.98 for mAP and 0.88 for mAP0.5:0.95, meaning that the model correctly detected and classified 98% of the objects of the validation dataset when the IoU threshold was constant and equal to 0.5, and 88% of the objects while the IoU threshold was progressing. As for the system's evaluation in real time, the findings for the 18 conducted tests are summarized in Table 8. \begin{table}
\begin{tabular}{c c c c} \hline \hline \multirow{2}{*}{**Framework**} & **Avg. Robot Action** & **Avg. Robot Action** & **Human Control over** \\  & **Time (s)** & **Time (\%)** & **the Process** \\ \hline
[71] & 251.00 & 70.72 & NO \\
[70] & 203.00 & 82.00 & NO \\ Proposed solution & 25.00 & 63.1 & YES \\ \hline \hline \end{tabular}
\end{table} 
Table 7: Comparison of existing frameworks. Figure 14: Cycle time comparison. Based on these experiments, it is observed that the proposed quality inspection module obtained the following results: (a) screws 94% TP--0.2% FP--6% FN, (b) connectors 90% TP--3% FP--8% FN, and (c) valve/pipes 76% TP--0% FP--24% FN. This is also depicted graphically in Figure 16.  
Figure 16: Conducted experiments on the quality inspection module. Figure 15: Object detection module evaluation metrics. \begin{table}
\begin{tabular}{c c c c c c c c c c c c c c c c c c c} \hline \hline \multirow{2}{*}{**ID**} & \multicolumn{5}{c}{**Screws**} & \multicolumn{5}{c}{**Conn  
To validate these findings thoroughly, an additional statistical analysis was conducted. Specifically, to assess the model's performance across the three classes of interest and the final selected confidence threshold value (0.92), the \(F1\) score was computed. This decision was based on the simplicity of the metric and the desire to evaluate the system's performance at the designated confidence threshold that is used during the system's operation. The \(F1\) score can be calculated from the precision and recall by the following equation:  
\[F1=\frac{2\cdot Precision\cdot Recall}{Precision+Recall} \tag{5}\]  
Using this formula, respective \(F1\) score values for each test are measured, as illustrated in Table 8. The mean of these scores per class indicates that the model achieved an accuracy of 0.96 for faulty screws, 0.94 for faulty connectors and 0.85 for faulty valves. The lower score for the valves compared to the other components is attributed to the similarity between faulty and correct classes for this specific part, resulting in some errors. Nevertheless, overall accuracy remains high, ensuring the model's robustness in an industrial setting. To evaluate the system's performance across all relevant classes, one could calculate the average \(F1\) scores across them, resulting in a final value of 91% of correct predictions for the system using the \(F1\) metric.