## 3 Materials and Methods
### Evaluation Metrics  
Although the DL models were trained with four classes, besides the classification problem, they were also evaluated for their ability to detect, i.e., locate the fruits in the image regardless of their ripeness stage. The HSV colour space model was only evaluated for the classification problem. A \"correct detection\" is commonly established through the Intersection over Union (IoU) metric, measuring the overlapping area between the predicted bounding box (\(B_{\text{p}}\)) and the groundtruth bounding box (\(B_{\text{gt}}\)) divided by the union area (1). In this case, a correct detection was considered if \(IoU>=50\%\). \[\text{IoU}=\frac{\text{Area}(B_{\text{p}}\cap B_{\text{gt}})}{\text{Area}(B_{ \text{p}}\cup B_{\text{gt}})} \tag{1}\]  
To better benchmark the two DL models, the metrics used by the Pascal VOC challenge [65] (Precision \(\times\) Recall curve and Mean Average Precision) were chosen, with the addition of the following metrics:  
* Recall (2)--ability of the model to detect all the relevant objects (i.e., all groundtruth bounding boxes);
* Precision (3)--ability to identify only the relevant objects;
* F1-Score (4)--first harmonic mean between Recall and Precision. The number of groundtruths (relevant objects) can be computed by the sum of the True Positives and False Negatives (TP + FN) and the number of detections is the sum of the TP's and False Positives (TP + FP). TP's are the correct detections of the groundtruths, FP's are improperly detected objects and FN are undetected. \[\text{Recall}=\frac{\text{TP}}{\text{TP}+\text{FN}}=\frac{\text{TP}}{\text{ All groundtruths}} \tag{2}\]  
Figure 6: Workflow of the performed methods to reach the HSV colour space model. \[\text{Precision}=\frac{\text{TP}}{\text{TP}+\text{FP}}=\frac{\text{TP}}{\text{ All detections}} \tag{3}\]  
\[\text{F1}-\text{Score}=2\times\frac{\text{Precision}\times\text{Recall}}{\text{ Precision}+\text{Recall}} \tag{4}\]  
All the detections performed by a DL model have a confidence rate associated with them. Graphically representing the ratio between Precision and Recall (Precision \(\times\) Recall curve) can be seen as a trade-off between Precision and Recall for different confidence values associated with the bounding boxes generated by a detector. A great object detector keeps its Precision rates high, while its Recall increase. Thus, a high Area Under the Curve (AUC) tends to indicate both high Precision and Recall. However, it is difficult to accurately measure the AUC, as the Precision \(\times\) Recall curve is often a zigzag-like curve. To overcome this problem was calculated the Average Precision (AP) metric by the all-point interpolation approach. In this case, the AP (5) is obtained by interpolating the Precision at each level, taking the maximum Precision (\(P_{\text{interp}}(R)\)) call value is greater or equal than \(R_{\text{n+1}}\). \[\text{AP}=\sum_{\text{n}}(R_{\text{n+1}}-R_{\text{n}})P_{\text{interp}}(R_{ \text{n+1}}) \tag{5}\]  
If there is more than one class to detect, the Mean Average Precision (mAP) metric is used, simply the average AP over all classes (6). \(\text{AP}_{\text{i}}\) represents the AP of class i and NC is the number of classes evaluated. \[\text{mAP}=\frac{\sum_{\text{i}=1}^{\text{NC}}\text{AP}_{\text{i}}}{\text{NC}} \tag{6}\]  
The final step of the inference was to optimise the confidence score, using the cross-validation technique: validation set augmentations were removed, and the F1-Score was computed for all the confidence thresholds from 0% to 100%, into steps of 1%. The confidence threshold that optimises the F1-Score was selected for the model's normal operation. The test set was used to evaluate both models, and the whole inference process occurred on the Google Colab server, using a Tesla T4 GPU. To assess the classification ability, the evaluation metrics chosen were the Precision and Recall for each class that will act as building blocks for the Macro F1-Score and Balanced Accuracy metrics.