# Object and Event Detection Pipeline for Rink Hockey Games
## 4 Experiments and Evaluation  
### Object Detection  
In our experiments, we trained a YOLOv7-based model for object detection using footage acquired, essentially, from professional rink hockey games. We performed both qualitative and quantitative evaluations to assess the performance of this model. The evaluation process consisted of applying YOLOv7 to detect objects of interest in multiple video frames from rink hockey games [40; 41] and comparing the results to manually annotated ground truth labels. We used precision (\(p\)), recall (\(r\)), and F1 score (\(F_{1}\)), the harmonic mean of precision and recall, as the three quantitative metrics to evaluate the performance of the system, according to the following equations:  
\[p=\frac{TP}{TP+FP} \tag{1}\]\[r=\frac{TP}{TP+FN} \tag{2}\]  
\[F_{1}=2\times\frac{p\times r}{p+r}. \tag{3}\]  
The model was trained using 100 epochs and a batch size of 32, with the system being implemented on a Windows desktop with a Zotac Gaming GeForce RTX 3090 Trinity OC 24 GB GDDR6X GPU and 64 GB of RAM, using the Darknet framework in Python 3.7.0. When testing on frames not seen in the training phase (present in the test subset), the overall performance was quite good (around 80%). The results of the evaluation are presented in the following figures. Figure 5 presents a multi-frame inferred plot of the test subset, showing the labels that the trained model detected. Figure 6 represents the trade-off between precision and recall in the classification model. As can be seen from the graph, the precision-recall values are higher for the player, goalkeeper, and goal classes, as we had more annotated images for these classes. However, the precision-recall for the ball is lower, as we have fewer annotated images for this class. Figure 5: Multi-frame (\(4\times 4=16\) frames) inferred plot of the test subset. The upper-left frame is at the coordinate (row, col) = (1, 1). Figure 6: Precision–recall curves. It is important to note that, in our evaluation, we only considered detections with an objective confidence score above 0.3. This threshold was chosen to balance the trade-off between detecting more objects while maintaining a slightly higher precision. Also, as seen in Figure 6, our trained model got a score of 0.764 [EMAIL_ADDRESS] and, in this context, 0.764 represents the mAP score, i.e., the mean average precision (the average of AP over all detected classes, calculated for each class separately) ranging between 0 and 1. The parameter 0.5 represents the IoU (intersection over union) selected threshold; in other words, the model is able to detect objects with an overlap of at least 50% with the ground truth. Having said that, in the context of this work, a mAP score of 0.764 is a fairly good score for object detection models. In follow-up to what was previously said, we can also use the \(F_{1}\) score to evaluate the model performance. The \(F_{1}\) score is a metric that is calculated as the harmonic mean of precision and recall. This metric also has a range of 0 to 1, with 1 representing perfect precision and recall and 0 representing the worst possible value. Beyond that, precision reflects the number of samples that were predicted as positive and are actually positive (ground truth), and recall measures the number of actual positive samples that were predicted correctly as positive. As shown in the Figure 7, the model has a maximum \(F_{1}\) score of 0.79 at 0.240. Analyzing this graph reveals that there is a large plateau with \(F_{1}>0.75\) approximately parallel to the confidence axis, which can be considered a very good indicator of the model behavior. The metrics \(F_{1}\) score, precision, and recall are all derived from the values of the confusion matrix, which is another useful tool for understanding the performance of a classification model, since it shows where the model is making mistakes and how it performs on different classes. In this case, as can be seen in the normalized confusion matrix obtained in Figure 8, when evaluating the test subset.