## 3 Results
### Object Detection Results  
We trained four different models for object detection in our robot. These models are YOLOv8N [29], YOLOv8M [29], YOLOv8X [29], and RT-DETR-L [7]. All models were pre-trained on the COCO dataset [30]. The objective was to evaluate the capabilities of various models, each with a different number of parameters, to find the optimal balance between accuracy and runtime, making the application suitable for the experiment. The models were solely trained on the synthetic dataset (SGFD) and tested on the real dataset (RGFD). The training utilized default hyperparameters, including various augmentation techniques such as image rotation, cropping, and color jitter. We conducted the training and evaluation on an NVIDIA RTX 3080 Ti, with all models undergoing 100 training epochs.  
We used the standard metric mean average precision (mAP) to evaluate the trained detector models' accuracy. The mAP evaluates object detection models by considering both precision (the model's accuracy in identifying only relevant objects) and recall (the model's ability to identify all relevant objects). It is calculated across various IoU (intersection over union) thresholds. IoU is a measure used to quantify the overlap between the predicted and ground truth bounding boxes.  
Table 2 presents the results of two popular mAP variants, mAP50 and mAP50-95, demonstrating that all models could reliably identify objects in the real dataset, even without being trained on real-world data. This success was anticipated, considering the toy-like setup of our experiment does not significantly challenge modern object detection models, thus simplifying the task. Among the models, YOLOv8N emerged as the most suitable choice because of its optimal balance between runtime efficiency and accuracy. Notably, our approach did not rely on the object detection models for color recognition; instead, we employed a straightforward color distance method. Specifically, we calculated the average pixel value within the detected object's bounding box and compared it to a reference value. This strategy effectively utilized the clear differentiation of colors in the RGB spectrum to accurately identify color ranges. An intriguing observation was that the transformer-based model, RT-DETR, was the only one that underperformed, seemingly unable to generalize effectively between the synthetic and the real dataset.  
Figure 5 displays a snapshot taken by the camera alongside the outcomes following the object detection process.  
\begin{table}
\begin{tabular}{l c c c c c} \hline \hline
**Model** & **Class** & **Instances** & **mAP50** & **mAP50-95** & **GFLOPs** \\ \hline \multirow{4}{*}{YOLOv8N} & all & 400 & 0.994 & 0.834 & \multirow{4}{*}{8.1} \\  & cube & 161 & 0.993 & 0.836 & \\ \cline{1-1}  & cylinder & 89 & 0.995 & 0.882 & \\ \cline{1-1}  & star & 150 & 0.995 & 0.783 & \\ \hline \multirow{4}{*}{YOLOv8M} & all & 400 & 0.983 & 0.818 & \multirow{4}{*}{78.7} \\  & cube & 161 & 0.970 & 0.777 & \\ \cline{1-1}  & cylinder & 89 & 0.995 & 0.881 & \\ \cline{1-1}  & star & 150 & 0.983 & 0.795 & \\ \hline \hline \end{tabular}
\end{table}
Table 2: Performance evaluation of object detection models on RGDF. The table displays the count of instances identified within the 100 images that comprise the dataset.