# Nano Aerial Vehicles for Tree Pollination
## 4 Results
### Evaluation
As we can see in Table 11, when the confidence ratio was greater than zero, there was a decrease in accuracy (increase in FP) and an increase in recall (decrease in FN). As the Flower Detection Dataset was only trained for one flower type, the values of the metrics decreased. By assuming the confidence threshold value that maximises the F1 score, the values of the metrics stabilised at around the same values. Precision was the best-performing metric for the three models when the confidence threshold maximised the F1 score, which means that there were more FNs than FPs. To better understand the results obtained with the Flower Detection Dataset, we decided to train the three YOLO models with the TensorFlow Flower Detection Dataset. As before, the training was performed for each model individually for 300 epochs with a batch size of 64 images and an input resolution of 320 \(\times\) 320 pixels using an NVIDIA GeForce 3090 Graphics Processing Unit (GPU) with 32 GigaBytes (GB) of available memory. It was also chosen so we could use the pre-trained weights and configuration provided by the YOLO developers. The results of the validation set were generated using an exact input resolution of 320 \(\times\) 320 with a batch size of 64 and an IoU threshold of 50%. Table 12 shows the confidence threshold values that maximised the F1 score for each model in the validation set. This value represented the best balance between precision and recall, which maximised the number of true positives and minimised the number of false positives and false negatives. All three models had similar confidence threshold values and similar confidence in their predictions. Table 13 shows the results of the test set. The inference was performed for a 0% confidence threshold and the confidence threshold value that maximised the F1 scores in the validation set. The inference was performed with a batch size of 8, as the processing capacity was smaller, and an IoU threshold of 50%. \begin{table}
\begin{tabular}{l c c c c c c c} \hline \hline \begin{table}
\begin{tabular}{l c} \hline \hline
**Model** & **Confidence Threshold** & **F1 Score** \\ \hline YOLOv5 & 56\% & 80\% \\ YOLOv7 & 75\% & 80\% \\ YOLOR & 73\% & 82\% \\ \hline \hline \end{tabular}
\end{table} 
Table 12: Confidence threshold values that optimised the F1 score obtained with the TensorFlow Flower Detection Dataset. Similar to the Flower Detection Dataset, the results indicate that this approach is only applicable for mAP comparison and not for practical use, as the results for a confidence threshold greater than 0% are not meaningful. A recall value higher than or equal to 95% indicates that quite a few flowers were detected and correctly classified according to the bunch conditions. However, a precision value lower than 25% indicates that there were quite a few incorrect detections of flowers, which were also expected to have a low F1 score value. Assuming the confidence threshold value that maximises the F1 score, there was a considerable increase in precision and F1 score, albeit at the cost of a slight decrease in recall and mAP. The precision values were above 80%, indicating that the models sometimes made incorrect detections. However, a higher number of flowers were missed. An F1 score above 80% demonstrates that the balance between precision and recall was much higher, whereas the mAP value barely changed with the threshold change. Overall, the results for the three models are promising and similar. YOLOR achieved the best performance in most metrics. To better understand the outcomes of some of the metrics it was necessary to observe the model detections on the test set images. Figure 8 presents the results of the ability of each YOLO version to detect grape bunches in a test set image. The models had good responses despite the complexity of the image, i.e., flowers of different sizes, overlapping, and with lower resolution. Figure 8: Detection of flowers in samples from the test set of the TensorFlow Flower Detection Dataset. Orange bounding boxes present ground truth.