## 4 Results and Discussion
### Automated Detection
Table 4 contains the label- and example-based metrics of the improvements proposed: the baselines of previous published works (Baseline [10] and CycleGAN [30]), the updated Detectron2 baseline trained on the new simulated dataset, the fine-tuned versions with CycleGAN and CUT converted images, and the remaining improvements upon CUT, including pseudo labeling (CUT + PL), the conversion of the CBIR databases (CUT + PL + CC), the simulation of a fixed camera setup (CUT + PL + CC + FCS), and post processing (CUT + PL + CC + FCS + PP). For label-based precision, recall, F1-score and Matthews Correlation Coefficient (MCC) we report both macro and micro averages. For every metric higher is better, except for the Hamming loss. As stated in [PERSON] et al. [30], our primary metric of interest is the example-based F1-score because we prefer high detection rates for each vehicle in detriment of high detection rates for each category (i.e., the label-based alternative) and, more importantly, the F1-score not only takes into account both precision and recall, but does not consider true negatives. This is paramount in our context, since one vehicle does not include all possible parts. As such, one model might have higher detection rates than the other. \begin{table}
\begin{tabular}{c c c} \hline \hline  & **FID w/ CycleGAN** & **FID w/ CUT** \\ \hline Simulated vs. Adapted & 35.40 & 58.17 \\ Real vs. Adapted & 60.04 & 56.87 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Fréchet Inception Distances (FIDs) between the simulated/real domains and the domain adapted versions of the simulated images by CycleGAN and CUT. Lower FIDs mean closer distributions, so CUT achieves better approximation to the real domain. Figure 6: Examples of domain adapted images with CycleGAN and CUT, and their comparison to real images collected on the factory floor. We also include a comparison between the old and new versions of the simulated datasets on the first two columns. accuracies by increasing the number of true negatives at the expense of true positives, which is less reflected in the F1-score. Thus, the last two columns of Table 4 include the F1-score improvements upon the baselines, in percentage and percentage points (pp). Finally, we present the multi-label multi-class metrics for the final 150 CBIR codes and for the 53 Detectron2 macro classes. The new baseline clearly outperforms its older version, achieving 0.735 (macro) and 0.455 (CBIR) example-based F1-scores compared to the previous 0.528 and 0.388. It even performs better than the previously domain adapted version (CycleGAN [30]). This allows us to conclude that the updated dataset with its improved vehicle colors and textures, although 17 times smaller than the original, is clearly more adequate for our task. Fine-tuning with CUT converted images leads to a 5.8 pp improvement over the new baseline compared to the 2.8 pp improvement obtained with CycleGAN, which is consistent with previous observations. Pseudo labeling further boosts performance, resulting in a 9 pp improvement, and converting the CBIR databases with CUT improves the F1-score from 0.537 to 0.565, an overall improvement of 11 pp. Detection examples can be found in Figure 7. After fine-tuning with CycleGAN and CUT converted data there is a slight increase in the number of detected instances and a reinforcement of the confidence scores. Pseudo labeling greatly improves the number of detected instances. However, there are still some failure cases, mainly switches between left and right (note that we consider the left side the driver's side) and in particular the image on the 4 th row. The detector identifies some objects as belonging to the back of the car, (e.g., door handles identified as plate lights, B.PLIi, on columns 2 and 3). After pseudo labeling they are identified as door handles, but still from the back of the car (B.Hn instead of SFL.Hn). Moreover, there seems to be an excessive amount of detections inside the window.