# Automatic Handgun Detection with Deep Learning in Video Surveillance Images
## 2 Related Works
### Performance Metrics  
In order to compare the results obtained by the different object detection models included in this study, it is essential to establish a standardized framework that provides the performance metrics on which the comparisons are based. The main way to promote the development of these standardized comparison frameworks has been to conduct competitions that establish common rules to solve a particular problem and measure the quality of the final results achieved with a unique test dataset. The most popular competitions in image-based object detection are:  
* The PASCAL VOC Challenge [14];
* The COCO Object Detection Challenge [15];
* The Open Images Challenge [16]. Those competitions used the _mean average precision_ (mAP) as the main metric, considering this as the mean--for all the classes considered in the problem--of the estimated area under the _precision \(\times\) recall curve_ (PxK curve). To consider the detection of an object as correct (_true positive_ or TP), incorrect (_false positive_ or FP), or undetected (_false negative_ or FN), two values related to the bbox area obtained for each detected object (\(B_{p}\)) are considered:  
* _Confidence score_ of the detection: This is the value in the range \([0,1]\) obtained by the algorithm, which represents the certainty value of the object's membership within the box with the indicated class;
* _Intresection over union_ (IoU): This takes into account the area of the object bbox in the ground truth (\(B_{\text{gt}}\)) and that of the bbox obtained by the detection algorithm (\(B_{p}\)) when both areas overlap. It is calculated as the ratio between the values of the intersection of the areas by the junction of both areas (see Equation (1) and Figure 3). By its own definition, it is a value in the range \([0,1]\). \[\text{IoU}=\frac{\left(B_{\text{p}}\cap B_{\text{gt}}\right)}{\left(B_{\text {p}}\cup B_{\text{gt}}\right)} \tag{1}\]  
The IoU and confidence score values are used to determine if each detected object is considered a true/false positive (TP/FP). In general, for a detected object to be considered the correct detection (TP), three conditions must be met:  
1. The confidence score for \(B_{\text{p}}\) is greater than a threshold value;
2. The class that is predicted for the detected object matches the class included in the ground truth (GT) for that object;
3. the IoU value for the detected object exceeds a threshold (usually \(\geq\)0.5). If any of the above criteria is not met, the object is considered an FP (incorrect detection). Some additional rules for determining the TP and FP counts are included in the case  
Figure 2: Difference between classification (**a**) and detection (**b**) problems. Classification must indicate the type of object in the image, while detection localizes the desired objects in an image indicating the degree of certainty of the detection. of the PASCAL VOC Challenge [14; 17]. For example, in the case of multiple detections that correspond to the same object in the GT, this is considered a single TP that corresponds to the \(B_{\text{p}}\) with the highest confidence score value, and the rest are considered FPs. With the total number of TPs and FPs, it is possible to calculate the _precision_ and _recall_ values. They correspond respectively to the proportion of correct detections, or the _positive predictive value_ (PPV), and the ability to correctly detect the positives, or the _true positive rate_ (TPR). It is important to note that the FN calculation is performed indirectly because as GT_P is the number of positives included in the ground truth, then:  
\[\text{GT\_P}=(\text{TP}+\text{FN})\Rightarrow\text{FN}=(\text{GT\_P}-\text{ TP})\]  
In general, precision and recall vary in opposite directions when the confidence scores change, in such a way that trying to reduce the FP by increasing the precision (i.e., by increasing the confidence score) causes an increment in the number of FNs.