# Automatic Handgun Detection with Deep Learning in Video Surveillance Images
## 4 Results  
Several metrics were computed to evaluate each model after the eight scheduled experiments with a test subset of 183 images with a total of 255 guns in them (i.e., GT_P = 255). The values of the metrics for each model are summarized in Table 1. This table shows the number of TPs, FPs, and FNs obtained for a confidence value of 0.5 with the correspondent values of the precision, recall, and F1 score. Finally, the AP value was obtained as the area under the PxR curve as stated by the toolkit developed by [PERSON] et al. [17]. As mentioned in the previous section, the experimental models were trained in two rounds: (1) with the original dataset; and (2) with the modified dataset by blending the pose skeletons obtained with OpenPose for every input image. The purpose of this procedure was to look for differences in performance, training each model with the two  
Figure 7: PxR curve for the YOLOv3 model (with the pose, i.e., Experiment 8) showing the AP calculation as the area under the curve (see Equations (2) and (3)), as stated by the toolkit provided by [PERSON] et al. [17]. aforementioned datasets. Moreover, two additional experiments were run on the model RetinaNet to analyze the effect of model fine-tuning on the (un)frozen backbone network. The results in Table 1 show that the AP values obtained by the models trained on our dataset without the pose (exp. 1, 3, 5, and 7) were similar to those obtained by the object detection algorithms that constitute the state-of-the-art. In these experiments, the overall best performing model--with the highest AP value--was RetinaNet fine-tuned with the unfrozen backbone (exp. 5, AP = 96.36%). In contrast, the Faster R-CNN exhibited the lowest AP (exp. 1 and 2). Although YOLOv3 (exp. 7 and 8) produced intermediate AP values (88.49\(\sim\)90.09%), it was the model that offered the highest precision (94.79\(\sim\)96.23%) and F1 score (91.74\(\sim\)93.36%) values. Figure 8 displays the test results of the RetinaNet model for two randomly chosen images. These results revealed the superior detection capability of the model trained with the unfrozen backbone (Figure 8b) because this model was able to detect an FN, previously undetected handgun (see top row) and discard an FP--previously incorrect detection--(see the bottom row). The other main objective of the study was to analyze the influence of the inclusion of pose information in the dataset. To accomplish this, a second modified dataset was built, from the original, blending in each image the pose skeletons obtained by OpenPose applied to the input images. This modified dataset allowed training the experimental models on images with pose information. The experimental result showed that the explicit inclusion \begin{table}
\begin{tabular}{c l l l l l l l} \hline \hline
**Exp.** & **Model** & **\#TP** & **\#FP** & **\#FN** & **Precision (\%)** & **Recall (\%)** & **F1 (\%)** & **AP (\%)** \\ \hline
1 & Faster R-CNN & 194 & 88 & 31 & 68.79 & 86.22 & 76.53 & 81.43 \\
2 & Faster R-CNN (with pose) & 190 & 71 & 35 & 72.80 & 84.44 & 78.19 & 80.79 \\
3 & RetinaNet\({}^{\text{frz}}\) & 211 & 25 & 16 & 89.41 & 92.89 & 91.54 & 93.12 \\
4 & RetinaNet\({}^{\text{frz}}\) (with pose) & 203 & 29 & 22 & 87.50 & 90.22 & 88.84 & 89.71 \\
5 & RetinaNet\({}^{\text{trz}}\) & 219 & 35 & 6 & 86.22 & 97.23 & 91.44 & 96.36 \\
6 & RetinaNet\({}^{\text{trz}}\) (with pose) & 210 & 25 & 15 & 89.36 & 93.33 & 91.30 & 92.82 \\
7 & YOLOv3 & 200 & 11 & 25 & 94.79 & 88.89 & 91.74 & 88.49 \\
8 & YOLOv3 (with pose) & 204 & 8 & 21 & 96.23 & 90.67 & 93.36 & 90.09 \\
[44] & [PERSON]’s work (with pose) & 158 & 2 & 39 & 98.75 & 80.20 & 88.51 & 83.6 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Assessment metrics obtained in the experiments with a confidence score of 0.5 (except AP). Figure 8: Results for the test images obtained by RetinaNet\({}^{\text{frz}}\) (with a _frozen backbone_) and RetinaNet\({}^{\text{trz}}\) (with an _unfrozen backbone_).