# Pixel-Level Analysis for Enhancing Threat Detection in Large-Scale X-ray Security Images
## 1 Introduction
However, the second part of object separation requires additional information about the material properties of each object in the image, which is currently not available in the public domain. As a response to the growing interest in computer vision research on X-ray security images, [PERSON] et al. [3] recently published SIXray, which is the largest database of X-ray security images that closely mirrors the practical scenario. The dataset demonstrates the visual complexities of X-ray security images, including overlapping of objects, but more importantly, the dataset places emphasis on the issue of imbalance as well. In practice, the data distribution of X-ray security images is highly skewed towards the majority class or  
Figure 1: Instance segmentation. (**a**) Input image. (**b**) Binary masks for each target object. (**c**) Isolated objects using the binary masks (for visualization). Figure 2: Object separation. (**a**) Input image. (**b**) Binary masks for each target object. (**c**) Isolated objects using the binary masks (for visualization). negative samples, i.e., images that do not contain threats, since threat objects are rarely encountered during security screening compared to normal objects. Due to the overrepresentation of the majority class, conventional models that are trained with imbalance datasets are extremely biased at always predicting the majority class. However, this becomes a major problem because the misclassification cost of the minority class or positive samples, i.e., images containing at least one threat, is significantly higher [9]. A straightforward approach in balancing an imbalanced dataset would be to either oversample the minority class or under-sample the majority class. Oversampling the minority class is not only inefficient when the majority class is magnitudes of times larger but has also been consistently shown to be inferior to other approaches. Consequently, under-sampling the majority class leads to a loss of a considerable amount of information depending on the scale of imbalance, which results in a surge of false positives as the conventional model tries to fit objects from unseen negative samples to any of the designated threat objects. Yet, it is of utmost importance to keep the false positives low if the detection algorithm is to be used as an automation tool in security systems to aid human inspectors [10]. In this paper, we investigated the impact of pixel-level analysis when it is integrated into the threat detection pipeline to address the first part of the object separation task in a large-scale and imbalanced X-ray security image dataset, also referred to as a practical X-ray dataset. We trained a Faster-RCNN [11] to localize possible threat regions on a balanced subset of the dataset to maximize the detection of the true positives. We then trained a DeepLabV3+ [12] to classify each pixel in the possible threat regions such that the regions without any pixel-level predictions are discarded as false positives. The two models were combined in one detection pipeline to produce the final predictions. Both models were selected after an exhaustive evaluation of the state-of-the-art object detection and semantic segmentation models, respectively. The main contributions of this paper are as follows:  
1. Reintroducing object separation as a unique task domain for X-ray security images;
2. Using segmentation as a mechanism to address class imbalance problem in practical X-ray security dataset;
3. Exhaustive evaluation of detection and segmentation deep learning models on the SIXray dataset; and
4. Development of a two-stage threat detection model that decouples detection and segmentation in order to maximize the use of available annotation.