# Using Deep Learning for Collecting Data about Museum Visitor Behavior
## 4 Experimental Results  
### Performance Analysis  
In order to assess the performance of the proposed system, we employed the detection evaluation metrics used in the most popular competitions, such as the COCO Detection Challenge ([[https://competitions.codalab.org/competitions/20794](https://competitions.codalab.org/competitions/20794)]([https://competitions.codalab.org/competitions/20794](https://competitions.codalab.org/competitions/20794)) (accessed on 13 October 2021)). Before illustrating these metrics, however, it is necessary to introduce some fundamental concepts. The goal of an object detector is to predict the position of objects of a certain class in an image or a video with a high degree of confidence. For this purpose, object detectors place bounding boxes in the image to identify the positions of the detected objects. A detection can, therefore, be represented by three features: The class of the object, the bounding box that contains it, and the confidence score. The Confidence Score is defined as the probability that a bounding box contains an object. It is, hence, usually a value between 0 and 1 that expresses how confident the model is about the prediction [53]. Another fundamental concept is that of Intersection over Union (IoU), which is defined as the ratio between the area of the intersection between a predicted bounding box \((B_{p})\) and a ground-truth bounding box \((B_{gt})\) and the area of their union:  
\[IoU=\frac{area(B_{p}\cap B_{gt})}{area(B_{p}\cup B_{gt})}. \tag{3}\]  
We have a perfect match when \(IoU=1\), while if the bounding boxes do not overlap at all, we have \(IoU=0\). Therefore, IoU values near to 1 are significantly better. Confidence Score and Intersection over Union are used to evaluate a detection. Specifically, there is a True Positive (TP) when:  
1. The confidence score is higher than a given threshold value;
2. The predicted class is the same as that of the ground-truth;
3. The predicted bounding box has an \(IoU\) higher than a threshold value (e.g., 0.75). On the other hand, there is a False Positive (FP) if one of the last two conditions is not valid. In the event that multiple predictions match the same ground-truth, the one with the highest confidence score is considered a TP, whilst all the others are considered as false positives. We have a False Negative (FN) when the Confidence Score of a detection of a supposed ground-truth is lower than the threshold value, whilst we have a true negative (TN) when the Confidence Score of a detection of anything is lower than the threshold value. True negatives, however, are usually not taken into account in evaluating object detection algorithms. Based on the previous definitions, it is possible to define the _Precision_ as follows:\[Precision=\frac{TP}{TP+FP} \tag{4}\]  
and _Recall_ as follows:  
\[Recall=\frac{TP}{TP+FN}. \tag{5}\]  
By setting the threshold for the Confidence Score at different values, we can obtain different pairs of Precision-Recall, which can be plotted on a graph in the form of _Precision-Recall curves_. It is possible to summarize the shape of these curves through a single numerical value, known as _Average Precision (AP)_[54]. This value is defined as the Precision averaged over a set of eleven Recall values equally spaced \([0,0.1,0.2,\ldots,1]\):  
\[AP=\frac{1}{11}\sum_{r\in\{0,0.1,\ldots,1\}}p_{interp}(r). \tag{6}\]  
The Precision value for each Recall level is interpolated considering the maximum Precision calculated for a system for which the corresponding Recall exceeds \(r\):  
\[p_{interp}(r)=\max_{\tilde{r}\
eq r\geq r}p(\tilde{r}) \tag{7}\]  
where \(p(\tilde{r})\) is the Precision measured at Recall \(\tilde{r}\). The purpose of the interpolation is to reduce the impact of wiggles in the Precision-Recall curves due to small variations in the classification of the retrieved objects. For a system to obtain a high value of Average Precision, it must therefore have a high Precision value at all levels of Recall.