# Evaluating GAN-Based Image Augmentation for Threat Detection in Large-Scale X-ray Security Images
## 4 Image Augmentation
### Experiment Results
In some cases, image synthesis might even degrade the TPR by overwhelming the detector with information that does not appear in the test set. On the contrary, the strength of image synthesis approaches can be largely attributed to the improvement in the FPR, where image synthesis approaches significantly outperform the image transformation approach. It is likely that the baseline model tries to fit previously unseen objects in the background to any of the threat object classes it has learned. Threat detection models are only trained on images that contains threats. Therefore, they are more likely to associate normal objects from negative samples as threats. Since image transformation only uses the information from the positive samples, it did not prevent the model from assuming previously unseen normal objects are threat objects causing the \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Approach** & **SIXray10** & **SIXray100** & **SIXray1000** \\ \hline Base & 26.3 & 25.8 & 29.8 \\ Base + Transf & 26.9 & 26.6 & 21.0 \\ Transl & 12.9 & 12.6 & 9.7 \\ Gen & 10.2 & 10.5 & **5.7** \\ Gen + Transl & 12.5 & 12.5 & 10.0 \\ Transl + Transf & 6.6 & 6.6 & 7.8 \\ Gen + Transf & **5.9** & **5.9** & 7.7 \\ Gen + Transl + Transf & 7.8 & 7.7 & 6.8 \\ \hline \hline \end{tabular}
\end{table} 
Table 13: False Positive Rate (%) across subsets. \begin{table}
\begin{tabular}{c c c} \hline \hline
**Approach** & **SIXray10/100** & **SIXray1000** \\ \hline Base & 94.5 & 91.2 \\ Base + Transf & **96.6** & 89.3 \\ Transl & 94.6 & 90.7 \\ Gen & 93.7 & 86.9 \\ Gen + Transl & 95.0 & 91.1 \\ Transl + Transf & 92.8 & **92.2** \\ Gen + Transf & 92.8 & 91.0 \\ Gen + Transl + Transf & 94.4 & 90.9 \\ \hline \hline \end{tabular}
\end{table} 
Table 12: True Positive Rate (%) across subsets. FPR to not improve or even worsen. When we used images from negative samples as the background and blended them with the generated or translated X-ray images of threat objects, the model was able to better differentiate the threat objects against several examples of normal objects that usually only appears in the images found from the negative samples causing significant improvement in the FPR. By observing the TPR and FPR, we can further infer that the information provided by any of the image synthesis approaches is the same since all of them brings about the same improvement. Hence, one can choose to use any of the image synthesis approaches and expect to get the same results. Some variations in the results shown for the SIXray1000 dataset can be attributed to its relatively smaller training set, which is in relation to its unrealistically extreme case of imbalance. For instance, Table 12 demonstrates that the TPR is substantially improved by using image transformation in SIXray10 and SIXray100 subsets. However, in the SIXray1000 subset, image transformation slightly degraded the TPR. The two smaller subsets have about 7500 positive samples in the training set, whereas there are only about 3500 positive samples in the training set for the SIXray1000 subset, which may not have been able to visually represent the types of threats present in the test set. Thus, when image transformation is added during training, the model was limited to detect the kinds of threats that are learnt from the training set but were not represented in the test set, causing the slight degradation in the TPR performance. Moreover, the improvement of the FPR in the SIXray1000, as shown in Table 13, can also be explained by the smaller training set, wherein the model was less confident in associating normal objects to threat objects given its limited knowledge. Regardless, we still observe that image augmentation approaches generally have the same effect to this largest subset. Image transformation improves the model performance by ensuring the detection of threat objects when they are present in the image, but it does not help the detector to avoid frequently misclassifying normal objects as threats.