# Pixel-Level Analysis for Enhancing Threat Detection in Large-Scale X-ray Security Images
## 1 Introduction  
X-ray imaging is widely used for securing public spaces [1]. Developing algorithms that provide aid to human inspectors to accomplish the monotonous and nontrivial task of detecting threats in X-ray security images is of utmost importance. Recently, deep learning has become the most dominant method used in the field of automatic threat detection in X-ray security images [2]. The most common approach is to adopt deep learning models trained on natural images and apply these models to X-ray security images. However, X-ray security images do not have as many rich features as natural images. For instance, X-ray security images have limited color range, lower contrast, and poor texture. The most prominent distinguishing feature of X-ray security images is the visibility of overlap between objects, which is a challenge when adopting deep learning models because object overlap aggravates intra-class variations [3]. The pixels in X-ray security images provide insight into the state of overlap between objects since each pixel corresponds to the radiation intensity that is attenuated by all overlapping objects [4]. Darker regions in X-ray security images signify higher attenuation, which could be caused by several overlapping objects or non-overlapping objects made up of higher density materials. Similarly, lighter regions in the image mean lower attenuation. Thus, an intuitive approach to handle overlapping objects is to analyze the X-ray security images in a pixel-wise manner. Pixel-level deep learning method continues to be a popular approach used in the related field of medical X-ray imaging [5]. Pixel-level analysis can also bring similar improvements to efficiency and reliability not only limited to X-ray security applications but also encompassing other related X-ray imaging fields, such as structural materials inspection [6]. In natural images, image segmentation is the task domain concerned with the pixel-wise analysis of images, which is further broken down into tasks, namely, semantic segmentation and instance segmentation [7]. Semantic segmentation aims to classify each pixel in the image into either of the available classes of objects. Instance segmentation combines object localization and pixel-wise classification such that it results in the delineation of each object instance in the image. With regards to threat detection in X-ray security images, instance segmentation would seem to be the appropriate task domain that allows pixel-level localization of each potential threat. However, this task cannot be directly adopted to X-ray security images because it restricts that each pixel is classified into only one object instance. Figure 1 shows instance segmentation in natural images, which do not exhibit overlap. Instead, we observe occlusion, wherein the objects in front fully cover parts of the objects behind. Yet, for X-ray security images with overlapping objects, a pixel can belong to as many object instances that are overlapping, as shown in Figure 2. Thus, a more specific task domain, called object separation, is required to address the issue of overlap. Object separation can be thought of as a multi-class and multi-label version of instance segmentation. This task domain was first introduced in [8] as a method of separating potentially overlapping objects in X-ray security images, then assigning the correct pixel values according to the estimated atomic number of each object. The first part of the object separation task can detect prohibited items with non-organic material properties, such as weapons, through their shape, texture, and other visual features. The second part can detect organic prohibited items, despite the lack of visual features, such as explosives and illegal drugs, by estimating their material composition. Thus, the complete object separation task domain can be used as a general solution to detecting prohibited items in X-ray security images. Deep learning can be leveraged in the first part of the object separation task since annotations can be readily attainable.