# Evaluating GAN-Based Image Augmentation for Threat Detection in Large-Scale X-ray Security Images
## 4 Image Augmentation
### Experiment Results
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Approach** & **Gun** & **Knife** & **Wrench** & **Pliers** & **Scissors** & **Mean** \\ \hline Base & 64 & 89 & 87 & 91 & 84 & 83.0 \\ Base + Transf & 49 & 84 & 87 & 88 & 83 & 78.2 \\ Transl & 51 & 82 & 88 & 90 & 85 & 79.2 \\ Gen & 44 & 76 & 89 & 91 & 76 & 75.2 \\ Gen + Transl & 46 & 83 & 88 & 90 & 86 & 78.6 \\ Transl + Transf & 40 & 79 & 87 & 89 & 83 & 75.6 \\ Gen + Transf & 34 & 74 & 86 & 89 & 81 & **72.8** \\ Gen + Transl + Transf & 34 & 74 & 88 & 88 & 84 & 73.6 \\ \hline \hline \end{tabular}
\end{table} 
Table 8: Log Average Miss Rate (%) for SIXray100. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Approach** & **Gun** & **Knife** & **Wrench** & **Pliers** & **Scissors** & **Mean** \\ \hline ResNet50 [8] & 48 & 53 & 28 & 40 & 2 & 34.1 \\ ResNet50+CHR [17] & 58 & 49 & 41 & 50 & 15 & 42.7 \\ Faster-RCNN [24] & 86 & 72 & 68 & 77 & 79 & 76.4 \\ Faster-RCNN [24] & 86 & 72 & 68 & 77 & 79 & 76.4 \\ Faster-RCNN+AUG 1  & 86 & 72 & 68 & 77 & 79 & 78.0 \\ \hline \hline \end{tabular}
\end{table} 
Table 11: Localization Accuracy (%) for SIXray100. \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**Approach** & **Gun** & **Knife** & **Wrench** & **Pliers** & **Scissors** & **Mean** \\ \hline Base & 49 & 84 & 85 & 80 & 84 & 76.4 \\ Base + Transf & 41 & 76 & 83 & 77 & 81 & 71.6 \\ Transl & 34 & 77 & 85 & 80 & 84 & 72.0 \\ Gen & 34 & 73 & 85 & 80 & 87 & 71.8 \\ Gen + Transl & 34 & 75 & 84 & 79 & 85 & 71.4 \\ Transl + Transf & 27 & 71 & 85 & 77 & 82 & 68.4 \\ Gen + Transf & 24 & 65 & 83 & 77 & 84 & 66.6 \\ Gen + Transl + Transf & 22 & 67 & 82 & 77 & 82 & **66.0** \\ \hline \hline \end{tabular}
\end{table} 
Table 7: Log Average Miss Rate (%) for SIXray100. \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**Approach** & **Gun** & **Knife** & **Wrench** & **Pliers** & **Scissors** & **Mean** \\ \hline ResNet50 [8] & 44 & 53 & 28 & 40 & 2 & 34.1 \\ ResNet50+CHR [17] & 58 & 49 & 41 & 50 & 15 & 42.7 \\ Faster-RCNN [24] & 86 & 72 & 68 & 77 & 79 & 76.4 \\ Faster-RCNN [24] & 86 & 72 & 68 & 77 & 79 & 76.4 \\ Faster-RCNN+AUG 1  & 86 & 72 & 68 & 77 & 79 & 78 \\ Faster-RCNN+AUG 1  & 72 & 68 & 77 & 79 & 78 \\ Faster-RCNN+AUG 1  & 86 & 74 & 73 & 79 & 78 & **78.0** \\ \hline \hline \end{tabular}
\end{table} 
Table 9: Localization Accuracy (%) for SIXray10. Finally, the image-level performance of the detection model is summarized in Tables 12 and 13. As an automation tool in X-ray security systems, the best approach is the one in which the TPR is high, and the FPR is low. These metrics give us more insight into the individual impact of image transformation and image synthesis in improving the performance of the detection model. Since we cannot calculate the FPR in an object detection task due to not knowing the exact quantity of true negatives, we instead calculate the FPR and TPR at the image level. The TPR is positively influenced by image transformation because this approach focuses on strengthening the model's understanding of the threats that are present in the training set by creating new images of different perspectives that these threats might appear. Since the threats and the context under which they appear, which are learned from the training set, are expected to appear in the test set, the model is better equipped at detecting them causing a performance boost. On the other hand, image synthesis approaches are shown to have little to no effect on the TPR because the information provided by these approaches does not have a strong tie to the threats that are present in the test set. Image synthesis approaches provide information about threat objects that are not present in the test set since the generated and translated threat objects are completely new. Not only are the threat objects different, but the context in which they appear, i.e., the type of baggage and other objects in the background, are also completely new and does not appear in the test set as well.