# Evaluating GAN-Based Image Augmentation for Threat Detection in Large-Scale X-ray Security Images
## 4 Image Augmentation
### Experiment Results
Nevertheless, the results show that using both image synthesis and image transformation still gives the best performance in this metric. Thus, making the model more reliable in terms of detecting threat objects when they are present in the image. Next, we looked at the localization accuracy of the threat detection model trained on the augmented dataset compared to the previous approaches, shown in Tables 9-11 for the SIXray10, SIXray100, and SIXray1000 subsets, respectively. The baseline approach, which is a class activation map (CAM) [40] attached to a classification model, and the proposed approach in [17] were both significantly underperforming when compared to the Faster-RCNN trained on both original and augmented dataset. This is because Faster-RCNN is explicitly tuned to the object detection task, while the first two approaches derived the bounding box predictions indirectly by tracing the gradients of the most discriminant features in the image. This metric only evaluates the detection rate of the model for each instance in the positive sample. Thus, the test set for the SIXray10 and SIXray100 subsets are the same. Since object detection models are only trained on the positive samples, which are also the same images for both SIXray10 and SIXray100 training sets, we observe the same performance from the models in both subsets. In contrast, classification models are heavily affected by the imbalance in both training and testing, as reflected by their performance on this metric, which progressively decreases as the level of imbalance increases. Nevertheless, using both augmentation techniques still proved to give the best performance in this metric on all levels of imbalance. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Approach** & **Gun** & **Knife** & **Wrench** & **Pliers** & **Scissors** & **Mean** \\ \hline Base & 31 & 68 & 71 & 61 & 75 & 61.2 \\ Base + Transf & 28 & 57 & 68 & 53 & 67 & 54.6 \\ Transl & 23 & 59 & 71 & 58 & 66 & 55.4 \\ Gen & 23 & 55 & 71 & 59 & 72 & 56.0 \\ Gen + Transl & 21 & 55 & 72 & 56 & 69 & 54.6 \\ Transl + Transf & 19 & 47 & 68 & 54 & 59 & 49.4 \\ Gen + Transf & 19 & 45 & 64 & 53 & 63 & 48.8 \\ Gen + Transf + Transf & 15 & 49 & 64 & 53 & 62 & **48.6** \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Log Average Miss Rate (%) for SIXray10. \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**Approach** & **Gun** & **Knife** & **Wrench** & **Pliers** & **Scissors** & **Mean** \\ \hline Base & 50 & 11 & 9 & 10 & 4 & 16.8 \\ Base + Transf & 67 & 23 & 14 & 15 & 10 & 25.8 \\ Transl & 67 & 31 & 13 & 14 & 10 & 27.0 \\ Gen & 72 & 40 & 14 & 12 & 9 & 29.4 \\ Gen + Transl & 70 & 31 & 12 & 12 & 11 & 27.2 \\ Transl + Transf & 76 & 36 & 16 & 14 & 17 & 31.8 \\ Gen + Transf & 77 & 44 & 17 & 15 & 17 & **34.0** \\ Gen + Transf + Transf & 79 & 42 & 16 & 15 & 14 & 33.2 \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Mean Average Precision for SIXray1000. \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**Approach** & **Gun** & **Knife** & **Wrench** & **Pliers** & **Scissors** & **Mean** \\ \hline Base & 49 & 84 & 85 & 80 & 84 & 76.4 \\ Base + Transf & 41 & 76 & 83 & 77 & 81 & 71.6 \\ Transl & 34 & 77 & 85 & 80 & 84 & 72.0 \\ Gen & 34 & 73 & 85 & 80 & 87 & 71.8 \\ Gen + Transl & 34 & 75 & 84 & 79 & 85 & 71.4 \\ Transl + Transf & 27 & 71 & 85 & 77 & 82 & 68.4 \\ Gen + Transf & 24 & 65 & 83 & 77 & 84 & 66.6 \\ Gen + Transl + Transf & 22 & 67 & 82 & 77 & 82 & **66.0** \\ \hline \hline \end{tabular}
\end{table} 
Table 7: Log Average Miss Rate (%) for SIXray100. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Approach** & **Gun** & **Knife** & **Wrench** & **Pliers** & **Scissors** & **Mean** \\ \hline ResNet50 [8] & 64 & 57 & 50 & 69 & 17 & 51.4 \\ ResNet50+CHR [17] & 69 & 59 & 54 & 77 & 16 & 54.9 \\ Faster-RCNN [24] & 86 & 72 & 68 & 77 & 79 & 76.4 \\ Faster-RCNN+AUG 1  & 86 & 72 & 68 & 77 & 79 & 76.4 \\ Faster-RCNN+AUG 1  & 86 & 74 & 73 & 79 & 78 & **78.0** \\ \hline \hline \end{tabular}
\end{table} 
Table 10: Localization Accuracy (%) for SIXray100.