# Pixel-Level Analysis for Enhancing Threat Detection in Large-Scale X-ray Security Images
## 6 Experiment Results and Discussion
### Experiment Results  
Tables 4-6 show the quantitative results of the experiments on the predefined test datasets. We compared our approach to the baseline classification method, CHR [3]; our previous classification method, GBAD [16]; and an end-to-end instance segmentation method, Mask-RCNN [43]. The removal of the mislabeled negative samples caused a substantial performance boost in the reevaluation of the previous works showing that their true strength was obscured by the quality of the dataset. Still, results revealed that image-level approaches drastically fell behind object-level and pixel-level approaches. Moreover, it is especially concerning how image-level approaches cannot cope with increasing ratios of imbalance. In contrast, both Mask R-CNN and our proposed approach showed more robustness to the imbalance, proving that localized methods such as object-level and pixel-level approaches can drastically enhance the performance of threat detection models, which is of utmost importance in security applications. Furthermore, our approach enjoys an even larger boost in performance compared to Mask R-CNN thanks to the uncoupling of the detection and segmentation task, which allowed for the use of the entire training set. Figure 7: Full detection pipeline. The multi-label detection confidence of two randomly sampled images from the positive and negative samples is shown in Table 7. Detection confidence of 100% conveys that the model is certain that the particular threat is present in the image, while detection confidence of 0% conveys that the model is certain that the image does not contain the particular threat. Analysis of the detection confidence will enable us to gain more insight into the overall m_AP_ performance of each approach. m_AP_ relies on the ranking of the detection confidence on each target class. Thus, a false positive with high detection confidence and a false negative with low detection confidence causes significant degradation in the performance. Meanwhile, a true positive with high detection confidence and a true negative with low detection confidence is desired for optimum performance. For the positive sample, i.e., Figure 8a, CHR [3] correctly detected the knife with relatively high confidence, but the rest of the threat classes also had considerably high detection confidence, especially the gun class. On the other hand, GBAD [16] also correctly detected the knife with 100% certainty and had extremely low detection confidence for the wrench, pliers, and scissors classes, but, unfortunately, it also had 100% certainty for the presence of a gun. From this sample alone, we might expect that GBAD performs worse on the gun and better on the rest of the target classes when compared to CHR. Indeed, we see this reflected in Table 5. The filtering mechanism of Mask R-CNN [43] and our proposed method allowed for the complete suppression of potential false positives to 0%. Both segmentation approaches correctly detected the knife and did not make any other false detections. Still, our approach boasts higher detection confidence for the same detected object. For the negative sample, i.e., Figure 8b, both CHR and GBAD incorrectly detected a gun object in the image with high confidence and, for the rest of the classes, the latter method continued to have extremely low detection confidence. It can be observed that GBAD had a tendency to have extreme detection confidence for any case, which may have been the reason for its marginal improvement over CHR. Meanwhile, Mask R-CNN was not able to suppress the incorrect detection of scissors, unlike our approach, which was able to correctly suppress any potentially wrong detection. It can be observed that