# Development and Optimization of Deep Learning Models for Weapon Detection in Surveillance Videos
## 4 Results and Discussion
### Experimentation
All the results described above and graphs for the best performing models on RTX 2060 machine are listed and shown in Table 3 and Figure 8.  
It can be observed from the above that the model performed best on RTX 2060 when having a subdivision of 32 and an image resolution of 608 \(\times\) 608. We further moved toward changing the image resolution, subdivision, and other parameters and found that RTX 2060 GPU was not able to load the model if we changed the network to do more complex work; therefore, we switched to RTX 2080 TI GPU to further change the settings to get better results. We changed some hyperparameters as follows:  
* IOU loss removed,
* IOU normalization 0.05 to 0.07. In the first run, it gave us adequate results with mAP increasing to 90.4 and loss reducing to 4.57. We further tuned the parameters to get better results with mAP becoming 90.9 and loss hitting the lowest of 1.24 utilizing the following settings, i.e.,  
* IOU loss removed,
* IOU normalize change from 0.05 to 0.07,
* Removed new coordinates,
* Changed logistic to linear loss function in YOLO layers. All the results described above and graphs for the best performing models on RTX 2080 TI machine are listed and shown in Table 4 and Figures 9 and 10. \begin{table}
\begin{tabular}{l l l l l} \hline \hline \multirow{2}{*}{**Subdivision**} & **Parameters** & \multirow{2}{*}{**Resolution**} & \multirow{2}{*}{**mAP**} & \multirow{2}{*}{**Improvement**} \\  & **Tuned** & & & \\ \hline
64 & No & \(512\times 512\) & 85.5 & No \\ \hline
32 & No & \(512\times 512\) & 91.2 & No \\ \hline \multirow{4}{*}{32} & Object & & & \\  & smoothness = 0 & & & \\  & Scale x y = 1.05 & & & \\  & Removed IOU & \(512\times 512\) & 90.4 &
\begin{tabular}{l} No but average \\ loss reduced \\ normalization \\ \end{tabular} \\ \hline
32 &
\begin{tabular}{l} Max batches = \\ 10,000 \\ \end{tabular} & \(608\times 608\) & 90.9 & No \\ \hline
64 & No & \(640\times 640\) & 83.3 & No \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Model 1: Scaled-YOLOv4 Optimization Configuration, and Parameter Tuning using GPU RTX 2060. Figure 8: RTX 2060 trained the best performing models: mAP vs. avg. loss. (**a**) Model with subdivision of 32 and image resolution of 512 \(\times\) 512 with default hyperparameters settings showing mAP of 91.2 and avg. loss of 16.2. (**b**) Model with subdivision of 32 and image resolution of 608 \(\times\) 608 with default hyperparameters settings showing mAP of 91.2 with reduced avg. loss of 11.2. \begin{table}
\begin{tabular}{l l l l l} \hline \hline
**Subdivision** & **Parameters Tuned** & **Resolution** & **mAP** & **Improvement** \\ \hline \multirow{2}{*}{32} & IOU loss removed & \multirow{2}{*}{512 \(\times\) 512} & \multirow{2}{*}{90.4} & \multirow{2}{*}{No} \\  & IOU normalization changed from 0.05 to 0.07 & & & \\ \hline \multirow{4}{*}{32} & IOU loss removed & \multirow{2}{*}{512 \(\times\) 512} & \multirow{2}{*}{90.9} & Average loss \\  & IOU normalization removed & & & reduced \\ \cline{1-1} \cline{3-3} \cline{5-6} \cline{5-6} \cline  
Finally, after many different parameters tweaking, we changed the following hyperparameters to get the highest mAP of 92.1, beating the previous model mAP. Some of the settings are as follow:  
* Object smoothness = 0,
* Scale x y = 1.05,
* Subdivision = 8 with image resolution 512 \(\times\) 512. To go to an even lower subdivision made us move to a high-end GPU because RTX 2060 and RTX 2080 TI GPUs were not capable of running the complex calculation. We ran the final training on Tesla T4, which gave us the expected results with parameter setting and model inference outcome as listed and shown in Table 5 and Figure 11.  
Figure 9: RTX 2080 TI trained the best performing models: mAP vs. avg. loss: Model with IoU loss removed and IoU normalization change from 0.05 to 0.07 with default hyperparameters settings at an image resolution of 512 \(\times\) 512 giving reduced mAP of 90.1 and better avg loss of 4.57. Figure 10: RTX 2080 TI trained the best performing models: mAP vs. avg.