# Integration of Deep Learning Vision Systems in Collaborative Robotics for Real-Time Applications
## 3 Materials and Methods
### Performance Metrics Used  
Evaluating the performance of object detection algorithms is essential to guarantee their accuracy and reliability in real-life scenarios. There are several metrics for evaluating the effectiveness of these algorithms, each offering a unique view of their performance and providing quantitative evaluations that help test and compare detection models. The following steps explore their importance and methodologies for evaluating the effectiveness of detection systems. Intersection over Union (IoU) is a popular metric used to assess a model's ability to accurately distinguish between the boundaries of the object and the background. This is achieved by comparing the area of the manually labelled box in the dataset with the area of the bounding box predicted by the model. The closer this comparison is to a value of one, the more accurate and precise the model's performance. Figure 6 shows an example of this metric, with the true bounding boxes and the predicted bounding boxes. However, many other parameters must be taken into account, such as accuracy, precision, average precision, mean average precision, recall, F1-score, and inference time [33]. Another fundamental metric for assessing the correct process of detected objects is confusion matrices. These statistically evaluate the results obtained during the classification phase. The simplest representation in ML is a binary classification, a \(2\times 2\) matrix, capable of correctly predicting a positive (true positive or TP), correctly predicting a negative (true  
Figure 5: Architecture of the YOLOv10 model (adapted from [39]). negative or TN), incorrectly predicting a positive (false positive or FP), and incorrectly predicting a negative (false negative or FN). The matrix tells you which classes perform best and worst. Refining the model makes the matrix evolve into a diagonal matrix, since there are fewer wrong predictions, strongly densifying the main diagonal of the matrix [33]. * Accuracy is the overall performance parameter of the model in one class. In a multi-class problem, this metric is the ratio between the number of correct predictions and the total number of predictions made. \[Accuracy=\frac{TP+TN}{TP+TN+FP+FN}\]
* Precision is the fraction of true positives in relation to the total number of positives. In a multi-class problem, it can be understood as the fraction of the number of correct predictions of a given class in relation to the sum of this value and the number of instances in which the model wrongly labelled that class in an object belonging to a different class. \[Precision=\frac{TP}{TP+FP}\]
* Average precision is used to quantify the precision of the detections made, and a different value will be obtained for each available class. \(AP_{c}\) corresponds to the sum for each class of the precision values recorded in each instance. In resume, AP measures performance by class. * Mean average precision is the benchmark metric to analyze object detection algorithms. It comprises the average accuracy of all the classes and, in itself, gives a fairly accurate description of the capabilities of a given model. mAP measures the average performance while considering all classes. * Recall is the ratio between the number of correctly predicted positives detected and the sum of the number of correctly predicted positives and the number of incorrectly predicted negatives. In a multi-class problem, it can be understood as the ratio between the number of correct predictions for a given class and the sum of the previous number and the number of times the model failed to detect an instance of that class. \[Recall=\frac{TP}{TP+FN}\]  
Figure 6: Example representation of the bounding boxes used for the IoU calculation using the bonbon packaging line. * The F1-score is an evaluation of the precision and recovery of the model of a class, which provides a metric of its balance. In a multi-class problem, this graph will consist of N curves, where N is the number of classes in the dataset.