## 5 Results
### Real-World Experimental Results  
Figure 14 displays results from live inference in a local environment. The results showcase that both models performed well enough for general detection but showed a tendency towards poor generalization. Figure 14,d show great examples of bad generalization. This was the main barrier to training models on synthetic data. However, during the procedure testing, we found that the xlarge model performed exceptionally well with this in mind.  
While neither of the models managed to detect the fences with decent confidence, as illustrated in Figure 14, they exceeded expectations based on the low confidence from training, as shown in Table 1. The detections were occasional, with the smaller model performing better than the xlarge model. During inference in the real world, the fences had zero false positives, unlike inference on synthetic data where fences had frequent false positives.  
Despite the models' confidence measurements within real-world object detection, stability was not appropriately showcased. The remaining frames could have been detection failures and misleading representations, as only a few images were included in the result.  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Class** & **Precision** & **Recall** & **[EMAIL_ADDRESS]** & **mAP** \\ \hline All & \(-0.81\%\) & \(-0.21\%\) & \(-0.41\%\) & \(-1.40\%\) \\ Forklift & \(0.10\%\) & \(0.00\%\) & \(0.00\%\) & \(-0.71\%\) \\ Pallet & \(-0.41\%\) & \(0.10\%\) & \(0.61\%\) & \(6.70\%\) \\ Container & \(0.50\%\) & \(0.70\%\) & \(0.00\%\) & \(-0.30\%\) \\ Barrel & \(0.71\%\) & \(0.00\%\) & \(0.10\%\) & \(-0.65\%\) \\ Human & \(-0.30\%\) & \(-0.70\%\) & \(-0.10\%\) & \(-0.65\%\) \\ Paperbox & \(0.10\%\) & \(0.71\%\) & \(0.10\%\) & \(-1.22\%\) \\ Crate & \(-0.40\%\) & \(0.00\%\) & \(0.20\%\) & \(-0.51\%\) \\ Fence & \(-6.65\%\) & \(-2.93\%\) & \(-4.70\%\) & \(-8.43\%\) \\ \hline \hline \end{tabular}
\end{table}
Table 1: Results from the small model displayed as the difference in percentage from the xlarge model.  
Figure 14: Live inference in a local environment: (**c**,**e**) small model; (**a**,**b**,**d**,**f**) xlarge model.  
The confidence of both models on the same object was plotted to show how the stability can vary and influence the average confidence.  
The first object was the forklift trained on one asset and evaluated on both models over 309 frames, as illustrated in Figure 15. The confidence of the small model in Figure 15a was lower and more unstable than the xlarge model in Figure 15b. Table 2 shows the statistics of the graphs. The small model varied more with less overall detection, unlike the xlarge model, which demonstrated higher stability and precision.  
The second object was a paper box, as presented in Figure 16. Similar to the inference on the forklift, the xlarge model performed significantly better than the smaller one in every instance, as shown in Table 3. It is worth noting the small model's performance on the paper box in contrast to the forklift, implying that the number of assets significantly affects the resulting generalization.  
\begin{table}
\begin{tabular}{c c c c c c c} \hline \hline \multirow{2}{*}{**Model**} & **Average** & **Confidence** & **Confidence** & **Confidence** & **False Positives** & **False Negatives** \\  & **Confidence** & **0.4–0.6** & **0.6–0.8** & **0.8–1.0** & & **False Positives** \\ \hline Small & 0.2779 & 112/432 frames & 44/432 frames & 15/432 frames & 149/432 frames & 261/432 frames \\ xlarge & 0.5754 & 98/230 frames & 102/230 frames & 7/230 frames & 0/230 frames & 23/230 frames \\ Small in \% & 27.79\% & 25.92\% & 10.19\% & 3.47\% & 34.49\% & 60.41\% \\ xlarge in \% & 57.54\% & 42.61\% & 44.35\% & 3.04\% & 0.00\% & 10.00\% \\ \hline \hline \end{tabular}
\end{table}
Table 2: Stabilization results from forklift detection over 10 s with small and xlarge model.  
Figure 15: Stabilization graph of a forklift over 10 s: (**a**) small model; (**b**) xlarge model.