# Evaluating GAN-Based Image Augmentation for Threat Detection in Large-Scale X-ray Security Images
## 4 Image Augmentation
### Experiment Results  
In this section, we present the results of the experiments, along with our interpretations. At test time, we used all the images in the test set of all SIXray subsets, which includes both positive and negative samples. Tables 3-5 show the mean AP of the detection model for the SIXray10, SIXray100, and SIXray1000 subsets, respectively. The gun class has relatively higher AP compared to the other classes, likely because these threat objects tend to have more district features compared to the other classes. For instance, a knife or a wrench can easily be mistaken for a harmless metal rod that makes up the framework of the luggage. Additionally, guns make up a large part of the class distribution of threats objects. Consequently, scissors are the least represented class among all the threat objects, and it has the least AP across all subsets. The overall model performance indicates that using either image transformation or any of the image synthesis approaches brought almost the same performance gain across all subsets. Thus, image transformation and image synthesis seem provide about the same quality of additional information to the threat detection model. However, it can also be observed that combining image synthesis and image transformation provides the most performance gain, which implies that the additional information from either approach boosts the performance of the detection model in different way. Furthermore, it appears that the specific approach to image synthesis does not heavily impact performance improvement. The kind of information provided by each approach and how these different information combines to further enhance the overall performance of the threat detection model is revealed by analyzing the results from other metrics. Meanwhile, as the level of imbalance increases, we also achieve increased performance gain with all the approaches, which indicates that augmentation is an effective method that can be used with other methods to combat the class-imbalance problem. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Approach** & **Gun** & **Knife** & **Wrench** & **Pliers** & **Scissors** & **Mean** \\ \hline Base & 67 & 21 & 22 & 31 & 9 & 30.0 \\ Base + Transf & 74 & 36 & 24 & 35 & 16 & 37.0 \\ Transl & 78 & 38 & 22 & 31 & 16 & 37.0 \\ Gen & 77 & 44 & 22 & 32 & 13 & 37.6 \\ Gen + Transl & 78 & 41 & 22 & 33 & 15 & 37.8 \\ Transl + Transf & 81 & 47 & 27 & 39 & 23 & 43.4 \\ Gen + Transf & 82 & 53 & 29 & 39 & 21 & **44.8** \\ Gen + Transl + Transf & 84 & 49 & 28 & 39 & 21 & 44.2 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Mean Average Precision for SIXray100. \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**Approach** & **Gun** & **Knife** & **Wrench** & **Pliers** & **Scissors** & **Mean** \\ \hline Base & 81 & 48 & 48 & 60 & 40 & 55.4 \\ Base + Transf & 83 & 55 & 53 & 66 & 48 & 61.0 \\ Transl & 83 & 52 & 49 & 62 & 59 & 61.0 \\ Gen & 83 & 61 & 49 & 62 & 47 & 60.4 \\ Gen + Transl & 84 & 60 & 50 & 64 & 49 & 61.4 \\ Transl + Transf & 83 & 64 & 52 & 66 & 60 & 65.0 \\ Gen + Transf & 84 & 65 & 57 & 67 & 57 & 66.0 \\ Gen + Transl + Transf & 88 & 65 & 57 & 67 & 59 & **67.2** \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Mean Average Precision for SIXray10. Tables 6-8 show the LAMR of the detection model for the SIXray10, SIXray100, and SIXray1000 subsets, respectively. For this metric, lower values indicate better performance. We obtain similar observations as we had with the mean AP metric. However, with LAMR, the performance gain we obtain as the level of imbalance increases is different. There is a performance drop from 12.5% to 10% from SIXray10 to SIXray100 and no further improvement from SIXray100 to SIXray1000. The reason for this is that LAMR focuses more on the detection of threat objects in the positive samples. Unlike AP, it is not heavily affected by the imbalance in the dataset caused by the negative samples.