# Automatic Handgun Detection with Deep Learning in Video Surveillance Images
## 5 Conclusions  
This work presented a study of three CNN-based object detection models (Faster R-CNN, RetinaNet, and YOLOv3)--pretrained on the MS COCO dataset--applied to handgun detection in video surveillance images. The three main objectives of the study were to:  
1. Compare the performance of the three models;
2. Analyze the influence of fine-tuning with an unfrozen/frozen backbone network for the RetinaNet model;
3. Analyze the improvement of the detection quality by model training on the dataset with pose information--associated with held handguns--including by a simple method of blending the skeleton poses in the input images. Using transfer learning by pretraining on the MS COCO dataset, it was possible to obtain the initial values for the experimental models' parameters, avoiding starting from scratch and overcoming the scarcity of training data. To set the network parameters for the specific detection problem, a dataset composed of 1220 images--with \"handgun\" as the only target class--was chosen following the selection criterion adapted to the problem. The assessment of the results in the eight experiments carried out on the 183 test images--unseen during training--was accomplished by comparing, for every model, the standardized metrics (shown in Table 1): precision, recall, F1 score, and average precision (AP) or area under the PxR curve. The results of the experiments conducted showed that:  
1. RetinaNet trained by the unfrozen backbone on images without the pose information (exp. 5) obtained the best results in terms of the average precision (96.36%) and recall (97.23%);
2. YOLOv3--in Experiments 7 and 8--obtained the best precision (94.79\(\sim\)96.23%) and F1 score values (91.74\(\sim\)93.36%);
3. The training on images with pose-related information by blending the pose skeletons--generated by OpenPose--in the input images obtained worse detection results for the Faster R-CNN and RetinaNet models (exp. 2, 4, and 6). However, in Experiment 8, YOLOv3 consistently improved every assessment metric by training on images incorporating the explicit pose information (precision \(\uparrow\) 1.44, recall \(\uparrow\) 1.78, F1 \(\uparrow\) 1.62, and AP \(\uparrow\) 1.60). This promising result encouraged us to further our studies on the ability to improve the way pose information is incorporated into the models;
4. When the models were trained on the dataset including the pose information, our method of blending the pose skeletons obtained better results than the previous alternative methods. RetinaNet and YOLOv3 (exp. 5 and 8) achieved respectively the highest recall (97.23%) and precision values (96.23%). Therefore, it would be desirable in future works to bring together in a single model the positive characteristics of these two architectures. Finally, our results also compared favorably with an alternative method that also considered the pose information. Considering the specific results from the tests with YOLOv3, some of the false positives detected were found to derive from the inability to distinguish classes of objects similar in size to handguns and held similarly to a handgun (e.g., smartphone, wallet, book, etc.). Future work should be focused on removing these types of false positive training models to recognize such objects, increasing the size and quality of the dataset. Our work represents the first case in which pose information has been combined with handgun appearance on this problem (as far as we are aware). In future work, we plan to extend this to consider the variation of the pose in time, which may in fact provide more information. For that, we will consider LSTM (long short-term memory) [50], as well as other methods (several have been proposed for the problem of action recognition). Particular care will be taken in that case regarding the computational time required. All the authors contributed equally to this work. All authors read and agreed to the published version of the manuscript.