# On Using XMC R-CNN Model for Contraband Detection within X-Ray Baggage Security Images

Authors: Yong Zhang; Weiwu Kong; Dong Li; Xudong Liu
Journal: Mathematical Problems in Engineering
 (2020)

---

## Abstract

We present an X-ray material classifier region-based convolutional neural network (XMC R-CNN) model for detecting the typical guns and the typical knives in X-ray baggage images. The XMC R-CNN model is used to solve the problem of contraband detection in overlapped X-ray baggage images by the X-ray material classifier algorithm and the organic stripping and inorganic stripping algorithm, and better detection rate and the miss rate are achieved. The detection rates of guns and knives are 96.5% and 95.8%, and the miss rates of guns and knives are 2.2% and 4.2%. The contraband detection technology based on the XMC R-CNN model is applied to X-ray baggage images of security inspection. According to user needs, the safe X-ray baggage images can be automatically filtered in some specific fields, which reduces the number of X-ray baggage images that security inspectors need to screen. The efficiency of security inspection is improved, and the labor intensity of security inspection is reduced. In addition, the security inspector can screen X-ray baggage images according to the boxes of automatic detection, which can improve the effect of security inspection.


---

## Body

## 1. Introduction

In recent years, with the increasing seriousness of terrorist activities, the safety of air transport has been paid more and more attention by all countries in the world. At present, there are three pain points of the security inspection, which need to be solved urgently: first, how to improve the effect of security inspection; second, how to improve the efficiency of security inspection; and third, how to reduce labor intensity of security inspection. The contraband detection technology is not used in the traditional technical solution, because the baggage contents are complex and highly varying. How to accurately identify the contraband in the X-ray baggage image is the most important and most difficult challenge for human operators. In addition, during peak periods, the human operators have limited time to screen images.As the most popular machine learning method, deep learning has achieved excellent results in object classification and detection. For the task of X-ray image classification, the previous work proposed the traditional machine learning method. In this paper, the XMC R-CNN method based on deep learning will be used to detect the contraband within X-ray baggage security images.In the ImageNet Large-Scale Visual Recognition Challenge 2012 (ILSVRC12), Hinton's team won the championship with the AlexNet model constructed by convolutional neural networks (CNNs), which ignited the enthusiasm of academic and industrial for deep learning. In order to use the deep learning method to detect the contraband in X-ray baggage security images, the framework of deep learning, the backbone model of deep learning, and the detection model of deep learning should be understood, as shown in Figure1.Figure 1
Deep learning map.With the upsurge of deep learning research, various open-source frameworks of deep learning emerge in endlessly. The main framework is introduced in [1–4], such as TensorFlow, Caffe, MXNet, Keras, CNTK, Torch, and Theano. Different deep learning frameworks usually have different model design, interface, deployment, performance, and architecture design, so their advantages and disadvantages are different.The classic backbone models mainly include LeNet [5], AlexNet [6], ZFNet [7], VGGNet [8], GoogleNet [9], ResNet [10], and DetNet [11]. Although the algorithms of object detection are different, the convolutional neural networks are usually used to process the input image, generate the feature map, and then use various algorithms to complete the region generation and loss calculation. The convolutional neural networks are the backbone of the whole detection algorithm. The basic components of the backbone include convolution layer, activation function layer, pooling layer, dropout layer, batch normalization layer, and full connection layer.The main backbone models are introduced above, which are usually used for object classification models. We study that the contraband detection belongs to classification and detection, and only classification is not enough. There is much contraband in X-ray baggage security images, so it is not only necessary to classify different contraband but also to determine the locations and sizes of the contraband. The classic classification and detection models include R-CNN [12], Fast R-CNN [13], Faster R-CNN [14], Mask-CNN [15], SSD [16], YOLO [17], and R-FCN [18]. These models have developed from two stages to one stage, from bottom-up only to top down, from the single-scale network to the feature pyramid network, and many algorithms have achieved excellent results on the ImageNet dataset.Computer-aided screening (CAS) [19] has also been widely used in the automatic detection of the contraband in X-ray baggage images; however, this largely remains an unsolved problem. The contraband detection based on multiview X-ray images is carried out in each X-ray image, and then, the constraint between multiview images is used to improve the detection accuracy in [20–27]. The contraband detection in computed tomography (CT) images is to extend the detection method of the single X-ray 2D image to 3D image in [28–35]. As mentioned above, both the contraband detections in multiview images and CT images are traditional technologies. The contraband detection by deep learning is proposed in [19, 36–38], and the accuracy of contraband detection is improved through deep learning methods. At present, there are still some problems in using these techniques to detect the contraband in overlapped X-ray baggage images. For example, a knife or a gun is covered by other objects in the baggage, and the outline of the knife or the gun is not clear in the original X-ray image, and its color is shown as orange (it is not metal color), so it cannot be correctly detected by the prior detection algorithm. Also, an explosive is hidden under several steel plates in the baggage, and it cannot be seen at all in the original image. It is difficult for the security inspector to determine that there is a dangerous object in this area. The XMC R-CNN model is used to solve the problem of contraband detection in overlapped X-ray baggage images by the X-ray material classifier algorithm and the organic stripping and inorganic stripping algorithm.In the next section, we review related work on the dual-energy X-ray material classifier and the contraband detection based on dual-energy X-ray data. In Section3, we discuss the contraband detection technology based on the XMC R-CNN model, including the detection model design and the algorithm implementation. In Section 4, we introduce the training and test results of the model. We summarize the work of this paper in Section 5.


## 2. Related Work

We now summarize the related work on the dual-energy X-ray material classifier and the contraband detection based on dual-energy X-ray data.


### 2.1. X-Ray Material Classifier

The dual-energy method of material classifier has been widely used in the X-ray security inspection systems by measuring the difference of attenuation coefficient of different materials for high- and low-energy X-ray in [39–46]. The physical principle of the dual-energy X-ray radiography is based on the exponential law of photon radiation attenuation. When an X-ray beam passes through an object, the detector signal can be described by the Beer–Lambert law:(1)I=∫I0Ee−μE,ZτdE,where E is the photon energy, I0E is the initial energy intensity of photons emitted from the source, μE,Z is the attenuation coefficient of a material with atomic number Z for impinging photons with energy E, and τ is the material thickness.In previous work for the material classifier in the X-ray baggage images, the log-ratioR is defined as the vertical axis, and 1/Hi is defined as the horizontal axis by Ogorodnikov and Petrunin in [47]:(2)R=logHilogLo,where Hi is the high-energy transparency and Lo is the low-energy transparency. By formulas (1) and (2), for a given material with atomic number Z, R is the unique value of the material and does not depend on the thickness. Therefore, R can be used to discriminate materials.Theα-curve method computes the features of the X-ray material in [48–50]:(3)α1=−logHi,α2=−logHi+logLo,where α2 is defined as the vertical axis and α1 is defined as the horizontal axis. The two materials can be easily differentiated if their α-curves are largely separated. On the contrary, the materials cannot be easily differentiated if their α-curves are too close.


### 2.2. Contraband Detection

Most contraband detection algorithms based on deep learning use the RGB images and do not really use the X-ray high- and low-energy data. Therefore, the material information is not fully used by the detection algorithm. The contraband detection within X-ray baggage imagers has not been well explored by machine vision community due to the lack of publicly available X-ray image datasets. Since the contraband detection within X-ray baggage imagers is a challenging problem, the detection models that use the X-ray high- and low-energy data are significantly more limited in the literature.The dual-energy images which provide material information about the objects are used for the contraband detection in [21, 51].The authors show that using multiple local color and texture features improves classification and detection performance. They first present an extensive evaluation of standard local features for object detection on a large X-ray image dataset in a structured learning framework. Then, they propose two dense sampling methods as a key-point detector for textureless objects and extend the SPIN (the image generation process can be visualized as a sheet spinning about the normal of a point) color descriptor to utilize the material information. Finally, they propose a multiview branch-and-bound search algorithm for multiview object detection.The two-channel dual-energy networks and the four-channel dual-energy networks are described in [37]. For each input, the high- and low-energy images are transformed into the feature space of a given method. The final convolutional layer feeds into the fully connected (FC) layers. The networks used in this work consist of 16 convolutional layers and 3 fully connected layers. The author experimented with the four-channel dual-energy networks, and the four-channel dual-energy networks with −logH,Δ,−logL,Σ perform the best performance metrics, where Σ=H+L and Δ=H−L are used.


## 2.1. X-Ray Material Classifier

The dual-energy method of material classifier has been widely used in the X-ray security inspection systems by measuring the difference of attenuation coefficient of different materials for high- and low-energy X-ray in [39–46]. The physical principle of the dual-energy X-ray radiography is based on the exponential law of photon radiation attenuation. When an X-ray beam passes through an object, the detector signal can be described by the Beer–Lambert law:(1)I=∫I0Ee−μE,ZτdE,where E is the photon energy, I0E is the initial energy intensity of photons emitted from the source, μE,Z is the attenuation coefficient of a material with atomic number Z for impinging photons with energy E, and τ is the material thickness.In previous work for the material classifier in the X-ray baggage images, the log-ratioR is defined as the vertical axis, and 1/Hi is defined as the horizontal axis by Ogorodnikov and Petrunin in [47]:(2)R=logHilogLo,where Hi is the high-energy transparency and Lo is the low-energy transparency. By formulas (1) and (2), for a given material with atomic number Z, R is the unique value of the material and does not depend on the thickness. Therefore, R can be used to discriminate materials.Theα-curve method computes the features of the X-ray material in [48–50]:(3)α1=−logHi,α2=−logHi+logLo,where α2 is defined as the vertical axis and α1 is defined as the horizontal axis. The two materials can be easily differentiated if their α-curves are largely separated. On the contrary, the materials cannot be easily differentiated if their α-curves are too close.


## 2.2. Contraband Detection

Most contraband detection algorithms based on deep learning use the RGB images and do not really use the X-ray high- and low-energy data. Therefore, the material information is not fully used by the detection algorithm. The contraband detection within X-ray baggage imagers has not been well explored by machine vision community due to the lack of publicly available X-ray image datasets. Since the contraband detection within X-ray baggage imagers is a challenging problem, the detection models that use the X-ray high- and low-energy data are significantly more limited in the literature.The dual-energy images which provide material information about the objects are used for the contraband detection in [21, 51].The authors show that using multiple local color and texture features improves classification and detection performance. They first present an extensive evaluation of standard local features for object detection on a large X-ray image dataset in a structured learning framework. Then, they propose two dense sampling methods as a key-point detector for textureless objects and extend the SPIN (the image generation process can be visualized as a sheet spinning about the normal of a point) color descriptor to utilize the material information. Finally, they propose a multiview branch-and-bound search algorithm for multiview object detection.The two-channel dual-energy networks and the four-channel dual-energy networks are described in [37]. For each input, the high- and low-energy images are transformed into the feature space of a given method. The final convolutional layer feeds into the fully connected (FC) layers. The networks used in this work consist of 16 convolutional layers and 3 fully connected layers. The author experimented with the four-channel dual-energy networks, and the four-channel dual-energy networks with −logH,Δ,−logL,Σ perform the best performance metrics, where Σ=H+L and Δ=H−L are used.


## 3. Detection Model Design

In this paper, the Faster R-CNN model is combined with the characteristics of the X-ray baggage image, and the XMC R-CNN model is designed for the automatic detection of the typical guns and the typical knives in the X-ray baggage image. The framework of the system is Caffe, and the backbone model of the system is VGG-16 [8].


### 3.1. XMC R-CNN Detection Model

The XMC R-CNN model is composed of two modules. The first module is X-ray material classifier (XMC) that strips organic and inorganic, and the second module is the Faster R-CNN detector that uses the proposed regions. The XMC R-CNN algorithm model mainly includes the following steps, as shown in Figure2.Figure 2
XMC R-CNN model.


#### 3.1.1. X-Ray Material Classifier

High-energy data and low-energy data are input, and different material values of organic, mixture, and inorganic are output through the material classification algorithm.


#### 3.1.2. Organic Stripping and Inorganic Stripping

High-energy data, low-energy data, and material values are input, and the material value of the organic, the gray value of the organic, the material value of the inorganic, and the gray value of the inorganic are output through the organic stripping and inorganic stripping algorithm.


#### 3.1.3. Convolutional Layer Features

A convolutional layer features adopts the Simonyan and Zisserman model [8] (VGG-16), which has 13 shareable convolutional layers. The organic, mixture, and inorganic images are input, which sizes are different. The feature maps of the input image are output, which will be used for the region proposal network (RPN) layers and the full connection layers.


#### 3.1.4. Region Proposal Networks

A region proposal network [14] is mainly used to generate region proposals. Firstly, anchor boxes of multiple scales and aspect ratios are designed. By the SoftMax layer, it can be determined that anchor boxes belong to foreground or background. Then, the anchor boxes are modified by binding box regression and a more accurate region proposal is obtained.


#### 3.1.5. Region of Interest Pooling

The RoI (Region of Interest) pooling layer in Faster R-CNN is adopted. The feature map from the last layer of VGG-16 and the region proposals from RPN are input, and the fixed size 7 × 7 proposal feature map is output.


#### 3.1.6. Contraband Detection

The proposal feature maps are input. Through the full convolutional layer and the SoftMax layer, we can get the classification of region proposal. At the same time, we can get the locations of the detection boxes by the binding box region. Finally, the classifications and the locations of the typical guns and the typical knives are output.The detailed implementation of Step (3), (4), (5), and (6) can be seen in the Faster R-CNN [14] and VGG-16 [8]. The X-ray material classifier algorithm of Step (1) and the organic stripping and inorganic stripping algorithm of Step (2) will be described in the following sections.


### 3.2. X-Ray Material Classifier

In this paper, the organic glass (chemical formula: C5H8O2) is selected to represent the organic material in Table 1, the aluminum (model: 2A12) is selected to represent the mixture material in Table 2, and the carbon steel (model: 45#) is selected to represent the inorganic material in Table 3.Table 1
Element content data of organic glass.
ElementAtomic numberAtomic weightPercentageC612.0159.98H11.0088.05O816.0031.97Table 2
Element content data of aluminum.
ElementAtomic numberAtomic weightPercentageCu2963.554.35Mg1224.311.5Mn2554.940.6Al1326.9893.55Table 3
Element content data of carbon steel.
ElementAtomic numberAtomic weightPercentage (%)C612.010.46Si1428.090.27Mn2554.940.65P1530.970.04S1632.070.40Cr2452.000.25Ni2858.690.25Fe2655.8597.68Three kinds of materials are selected from different thickness ranges, and data are collected; then, the curves are drawn as shown in Figure3. The horizontal axis is the high-energy value (Hi) of the dual-energy X-ray, and the vertical axis is the material value (Mat) of the dual-energy X-ray. The log-ratio R is defined as the material value. 122 curves are evenly inserted between the three curves, which, respectively, represent the different materials of organic, mixture, and inorganic. Through these 125 curves, the material values of any point in the space are calculated, and the material table of the dual-energy X-ray is generated.Figure 3
X-ray material classifier model.The whole material space is divided into five regions: low-gray unrecognizable region where the image shows red, high-gray unrecognizable region where the image shows gray, organic region where the image shows orange, mixture region where the image shows green, and inorganic region where the image shows blue, as shown in Figure3.The material value of the low-gray unrecognizable region is defined as 125, the material value of the high-gray unrecognizable region is defined as 0, the material value of the organic region is defined as 1, the material value of the inorganic region is defined as 124, and the material value of the mixture region needs to be calculated, which is the transition region from the inorganic region to the organic region. Three material curves in the mixture region have been obtained. According to the X-ray material classifier model in Figure3, the low- and high-energy data of other 122 curves with different thickness are calculated. Through the materials of 125 curves and 100 data sampling points of different material thickness, the mixture region can be divided into 12276 grid cells. As shown in Figure 3, each grid cell is composed of four points (A, B, C, and D). Using the linear interpolation algorithm, we can calculate the material value of any point in the quadrilateral (A, B, C, and D). Finally, the material values of all points in the whole material space can be calculated and exported to the material table.


### 3.3. Organic Stripping and Inorganic Stripping

By means of physical model designing, experimental data sampling, and the using of the grid cell method and linear interpolation algorithm, the organic or the inorganic can be screened separately from the overlapped X-ray baggage image. The organic glass (chemical formula: C5H8O2) is selected to represent the organic material, the carbon steel (model: 45#) is selected to represent the inorganic material, and the overlapped simulants of the organic glass and the carbon steel are selected to represent the mixture material, whose thickness is different.The effective atomic number [52] is given by(4)Zeff=∑ai∗Zi3.5∑ai1/3.5,(5)ai=mi∗ZiAi,where Zeff is the effective atomic number, mi is the mass percentage of the element, Zi is the atomic number, and Ai is the atomic weight. According to formulas (1) and (2), the effective atomic number Zfe of the carbon steel is equal to 25.91, and the effective atomic number Zgl of the organic glass is equal to 6.56. The material of the effective atomic number Zn (7∼25) can be obtained by overlapping the organic glass and the carbon steel with different thicknesses.As shown in Figure4, the horizontal axis is the high-energy value (Hi) of the dual-energy X-ray and the vertical axis is the material value (Mat) of the dual-energy X-ray. The log-ratio R is defined as the material value. The orange curve is the organic glass curve, which represents the organic material. The blue curve is the carbon steel curve, which represents the inorganic material. The curves between the orange curve and the blue curve are the different thicknesses overlapped curves of the organic glass and the carbon steel, which represents the mixture material.Figure 4
Organic tripping and inorganic stripping model.By overlaying the organic glass (Zgl=6.5) and the carbon steel (Zfe=26), the effective atomic number Zn (7∼25) is obtained. For example, if we want to calculate the material with Zn=7, take Zeff=7 into formulas (1) and (2), so we can get the mass ratio of the organic glass and the carbon steel (mgl:mfe = 0.997572383:0.002427617). The length and width of the organic glass and the carbon steel models are the same in the test process, we know that the density of the carbon steel is 7.82 g/cm3, and the density of the organic glass is 1.18 g/cm3, so that the volume ratio of the organic glass and the carbon steel can be calculated and then converted to the thickness ratio (Tgl:Tfe = 0.999632928:0.000367072). Similarly, we can calculate the material with Zn = 8-25, and the corresponding thickness of the organic glass and the carbon steel is shown in Table 4.Table 4
Effective atomic number ratio data.
Effective atomic numberOrganic glass thickness percentageCarbon steel thickness percentage6.51070.9996329280.00036707280.9985481860.00145181490.9970477530.002952247100.9950345260.004965474.........220.8960181670.103981833230.8635136260.136486374240.8178312770.182168723250.7499596510.2500403492601According to Table4, the material of the effective atomic number Zn = 7∼25 can be obtained by overlapping the organic glass and the carbon steel with different thicknesses. 9 sampling points are designed for each material, and 171 sampling points are obtained in total. Six groups of data were collected at each sampling point, and a total of 1026 sampling data points were obtained.For example, the thickness of the organic glass is 32.7 mm, and the thickness of the carbon steel is 0.012 mm. We know that the density of the carbon steel is 7.82 g/cm3, and the density of the organic glass is 1.18 g/cm3. In this way, we can calculate the mass ratio of the organic glass and the carbon steel (mgl:mfe = 38.568:0.09384), and bring the mgl and mfe into formulas (1) and (2), so that we can get Zeff = 7.00039831652385≈7 (i.e., lines 7-6 in Table 5). Similarly, it can be calculated that each point in Table 5 meets the corresponding effective atomic number. See Table 5 for the experimental data of the design sampling point, where Z-n is the effective atomic number and the serial number of the sampling point, Tgl is the thickness of the organic glass, Tfe is the thickness of the carbon steel, Gm is the high- and low-energy value of the mixture material, Ggl is the high- and low-energy value of the organic glass, and Gfe is the high- and low-energy value of the carbon steel.Table 5
Sampling experimental data.
Z-nTglTfeGmGglGfeHiLoHiLoHiLo7-110.90.0042622230826872485396037327-213.60.0052009165720811839391335757-316.30.0061727137418331590388034787-421.80.008127095113491116382333427-527.20.0111378261220993379732567-632.70.012837579907708375431517-743.60.016635415698526365129527-849.00.018313184358247353227127-954.50.021638619412633872440...........................10-111.770.0631422850321330733996377310-219.960.126312255272425243932360210-335.790.1818631447199817593801326010-444.950.2215231129167414323738312610-558.140.31129773127010413613286610-679.930.47084408276383460256910-799.960.54582645534033335236110-8119.90.583001613752613202215610-9159.60.781447117811529781848...........................25-10.77126051435402640072658146825-21.632.12177373139513916182875325-32.082.70145853739233881152056025-42.662.47124542138893834130544225-54.003.927131973796372976420925-66.376.3830170361335223347825-79.3412.1219044352034172134825-810.7914.011226338432541343125-911.7215.25916322330786917An organic stripping and inorganic stripping table is generated from the data in Table5. As shown in Figure 4, there are four points: A, B, C, and D. Through Table 5, we can get A (Gm, Ggl, Gfe), B (Gm, Ggl, Gfe), C (Gm, Ggl, Gfe), and D (Gm, Ggl, Gfe). Through the bilinear interpolation algorithm, we can calculate the (Gm, Ggl, Gfe) value of any point in the quadrilateral (A, B, C, and D), so that we can calculate the (Gm, Ggl, Gfe) value of any point in the coordinate system. Finally, we can realize the organic stripping and inorganic stripping of dual-energy X-ray security images.In Figure5(a), a knife is covered with some objects. In the original image, the outline of the knife is not clear, and its color is shown as orange, so it cannot be judged as a dangerous metal. In Figure 5(b), the knife is clearly visible, and it is shown as a metal color (blue), because the organic stripping algorithm is used. The outline of this knife is clear in the organic stripping image, and it is easy to be detected by the XMC R-CNN model.Figure 5
(a) Original X-ray image; (b) organic stripping image.
(a)(b)In Figure6(a), an explosive is hidden under several steel plates. In the original image, it is difficult for the security inspector to determine that there is a dangerous object in this area. In Figure 6(b), the explosive is clearly visible, and it is shown as an explosive color (orange), because the inorganic stripping algorithm is used. This is a good basis for detecting more kinds of contraband in X-ray baggage images in the future.Figure 6
(a) Original X-ray image; (b) inorganic stripping image.
(a)(b)


## 3.1. XMC R-CNN Detection Model

The XMC R-CNN model is composed of two modules. The first module is X-ray material classifier (XMC) that strips organic and inorganic, and the second module is the Faster R-CNN detector that uses the proposed regions. The XMC R-CNN algorithm model mainly includes the following steps, as shown in Figure2.Figure 2
XMC R-CNN model.


### 3.1.1. X-Ray Material Classifier

High-energy data and low-energy data are input, and different material values of organic, mixture, and inorganic are output through the material classification algorithm.


### 3.1.2. Organic Stripping and Inorganic Stripping

High-energy data, low-energy data, and material values are input, and the material value of the organic, the gray value of the organic, the material value of the inorganic, and the gray value of the inorganic are output through the organic stripping and inorganic stripping algorithm.


### 3.1.3. Convolutional Layer Features

A convolutional layer features adopts the Simonyan and Zisserman model [8] (VGG-16), which has 13 shareable convolutional layers. The organic, mixture, and inorganic images are input, which sizes are different. The feature maps of the input image are output, which will be used for the region proposal network (RPN) layers and the full connection layers.


### 3.1.4. Region Proposal Networks

A region proposal network [14] is mainly used to generate region proposals. Firstly, anchor boxes of multiple scales and aspect ratios are designed. By the SoftMax layer, it can be determined that anchor boxes belong to foreground or background. Then, the anchor boxes are modified by binding box regression and a more accurate region proposal is obtained.


### 3.1.5. Region of Interest Pooling

The RoI (Region of Interest) pooling layer in Faster R-CNN is adopted. The feature map from the last layer of VGG-16 and the region proposals from RPN are input, and the fixed size 7 × 7 proposal feature map is output.


### 3.1.6. Contraband Detection

The proposal feature maps are input. Through the full convolutional layer and the SoftMax layer, we can get the classification of region proposal. At the same time, we can get the locations of the detection boxes by the binding box region. Finally, the classifications and the locations of the typical guns and the typical knives are output.The detailed implementation of Step (3), (4), (5), and (6) can be seen in the Faster R-CNN [14] and VGG-16 [8]. The X-ray material classifier algorithm of Step (1) and the organic stripping and inorganic stripping algorithm of Step (2) will be described in the following sections.


## 3.1.1. X-Ray Material Classifier

High-energy data and low-energy data are input, and different material values of organic, mixture, and inorganic are output through the material classification algorithm.


## 3.1.2. Organic Stripping and Inorganic Stripping

High-energy data, low-energy data, and material values are input, and the material value of the organic, the gray value of the organic, the material value of the inorganic, and the gray value of the inorganic are output through the organic stripping and inorganic stripping algorithm.


## 3.1.3. Convolutional Layer Features

A convolutional layer features adopts the Simonyan and Zisserman model [8] (VGG-16), which has 13 shareable convolutional layers. The organic, mixture, and inorganic images are input, which sizes are different. The feature maps of the input image are output, which will be used for the region proposal network (RPN) layers and the full connection layers.


## 3.1.4. Region Proposal Networks

A region proposal network [14] is mainly used to generate region proposals. Firstly, anchor boxes of multiple scales and aspect ratios are designed. By the SoftMax layer, it can be determined that anchor boxes belong to foreground or background. Then, the anchor boxes are modified by binding box regression and a more accurate region proposal is obtained.


## 3.1.5. Region of Interest Pooling

The RoI (Region of Interest) pooling layer in Faster R-CNN is adopted. The feature map from the last layer of VGG-16 and the region proposals from RPN are input, and the fixed size 7 × 7 proposal feature map is output.


## 3.1.6. Contraband Detection

The proposal feature maps are input. Through the full convolutional layer and the SoftMax layer, we can get the classification of region proposal. At the same time, we can get the locations of the detection boxes by the binding box region. Finally, the classifications and the locations of the typical guns and the typical knives are output.The detailed implementation of Step (3), (4), (5), and (6) can be seen in the Faster R-CNN [14] and VGG-16 [8]. The X-ray material classifier algorithm of Step (1) and the organic stripping and inorganic stripping algorithm of Step (2) will be described in the following sections.


## 3.2. X-Ray Material Classifier

In this paper, the organic glass (chemical formula: C5H8O2) is selected to represent the organic material in Table 1, the aluminum (model: 2A12) is selected to represent the mixture material in Table 2, and the carbon steel (model: 45#) is selected to represent the inorganic material in Table 3.Table 1
Element content data of organic glass.
ElementAtomic numberAtomic weightPercentageC612.0159.98H11.0088.05O816.0031.97Table 2
Element content data of aluminum.
ElementAtomic numberAtomic weightPercentageCu2963.554.35Mg1224.311.5Mn2554.940.6Al1326.9893.55Table 3
Element content data of carbon steel.
ElementAtomic numberAtomic weightPercentage (%)C612.010.46Si1428.090.27Mn2554.940.65P1530.970.04S1632.070.40Cr2452.000.25Ni2858.690.25Fe2655.8597.68Three kinds of materials are selected from different thickness ranges, and data are collected; then, the curves are drawn as shown in Figure3. The horizontal axis is the high-energy value (Hi) of the dual-energy X-ray, and the vertical axis is the material value (Mat) of the dual-energy X-ray. The log-ratio R is defined as the material value. 122 curves are evenly inserted between the three curves, which, respectively, represent the different materials of organic, mixture, and inorganic. Through these 125 curves, the material values of any point in the space are calculated, and the material table of the dual-energy X-ray is generated.Figure 3
X-ray material classifier model.The whole material space is divided into five regions: low-gray unrecognizable region where the image shows red, high-gray unrecognizable region where the image shows gray, organic region where the image shows orange, mixture region where the image shows green, and inorganic region where the image shows blue, as shown in Figure3.The material value of the low-gray unrecognizable region is defined as 125, the material value of the high-gray unrecognizable region is defined as 0, the material value of the organic region is defined as 1, the material value of the inorganic region is defined as 124, and the material value of the mixture region needs to be calculated, which is the transition region from the inorganic region to the organic region. Three material curves in the mixture region have been obtained. According to the X-ray material classifier model in Figure3, the low- and high-energy data of other 122 curves with different thickness are calculated. Through the materials of 125 curves and 100 data sampling points of different material thickness, the mixture region can be divided into 12276 grid cells. As shown in Figure 3, each grid cell is composed of four points (A, B, C, and D). Using the linear interpolation algorithm, we can calculate the material value of any point in the quadrilateral (A, B, C, and D). Finally, the material values of all points in the whole material space can be calculated and exported to the material table.


## 3.3. Organic Stripping and Inorganic Stripping

By means of physical model designing, experimental data sampling, and the using of the grid cell method and linear interpolation algorithm, the organic or the inorganic can be screened separately from the overlapped X-ray baggage image. The organic glass (chemical formula: C5H8O2) is selected to represent the organic material, the carbon steel (model: 45#) is selected to represent the inorganic material, and the overlapped simulants of the organic glass and the carbon steel are selected to represent the mixture material, whose thickness is different.The effective atomic number [52] is given by(4)Zeff=∑ai∗Zi3.5∑ai1/3.5,(5)ai=mi∗ZiAi,where Zeff is the effective atomic number, mi is the mass percentage of the element, Zi is the atomic number, and Ai is the atomic weight. According to formulas (1) and (2), the effective atomic number Zfe of the carbon steel is equal to 25.91, and the effective atomic number Zgl of the organic glass is equal to 6.56. The material of the effective atomic number Zn (7∼25) can be obtained by overlapping the organic glass and the carbon steel with different thicknesses.As shown in Figure4, the horizontal axis is the high-energy value (Hi) of the dual-energy X-ray and the vertical axis is the material value (Mat) of the dual-energy X-ray. The log-ratio R is defined as the material value. The orange curve is the organic glass curve, which represents the organic material. The blue curve is the carbon steel curve, which represents the inorganic material. The curves between the orange curve and the blue curve are the different thicknesses overlapped curves of the organic glass and the carbon steel, which represents the mixture material.Figure 4
Organic tripping and inorganic stripping model.By overlaying the organic glass (Zgl=6.5) and the carbon steel (Zfe=26), the effective atomic number Zn (7∼25) is obtained. For example, if we want to calculate the material with Zn=7, take Zeff=7 into formulas (1) and (2), so we can get the mass ratio of the organic glass and the carbon steel (mgl:mfe = 0.997572383:0.002427617). The length and width of the organic glass and the carbon steel models are the same in the test process, we know that the density of the carbon steel is 7.82 g/cm3, and the density of the organic glass is 1.18 g/cm3, so that the volume ratio of the organic glass and the carbon steel can be calculated and then converted to the thickness ratio (Tgl:Tfe = 0.999632928:0.000367072). Similarly, we can calculate the material with Zn = 8-25, and the corresponding thickness of the organic glass and the carbon steel is shown in Table 4.Table 4
Effective atomic number ratio data.
Effective atomic numberOrganic glass thickness percentageCarbon steel thickness percentage6.51070.9996329280.00036707280.9985481860.00145181490.9970477530.002952247100.9950345260.004965474.........220.8960181670.103981833230.8635136260.136486374240.8178312770.182168723250.7499596510.2500403492601According to Table4, the material of the effective atomic number Zn = 7∼25 can be obtained by overlapping the organic glass and the carbon steel with different thicknesses. 9 sampling points are designed for each material, and 171 sampling points are obtained in total. Six groups of data were collected at each sampling point, and a total of 1026 sampling data points were obtained.For example, the thickness of the organic glass is 32.7 mm, and the thickness of the carbon steel is 0.012 mm. We know that the density of the carbon steel is 7.82 g/cm3, and the density of the organic glass is 1.18 g/cm3. In this way, we can calculate the mass ratio of the organic glass and the carbon steel (mgl:mfe = 38.568:0.09384), and bring the mgl and mfe into formulas (1) and (2), so that we can get Zeff = 7.00039831652385≈7 (i.e., lines 7-6 in Table 5). Similarly, it can be calculated that each point in Table 5 meets the corresponding effective atomic number. See Table 5 for the experimental data of the design sampling point, where Z-n is the effective atomic number and the serial number of the sampling point, Tgl is the thickness of the organic glass, Tfe is the thickness of the carbon steel, Gm is the high- and low-energy value of the mixture material, Ggl is the high- and low-energy value of the organic glass, and Gfe is the high- and low-energy value of the carbon steel.Table 5
Sampling experimental data.
Z-nTglTfeGmGglGfeHiLoHiLoHiLo7-110.90.0042622230826872485396037327-213.60.0052009165720811839391335757-316.30.0061727137418331590388034787-421.80.008127095113491116382333427-527.20.0111378261220993379732567-632.70.012837579907708375431517-743.60.016635415698526365129527-849.00.018313184358247353227127-954.50.021638619412633872440...........................10-111.770.0631422850321330733996377310-219.960.126312255272425243932360210-335.790.1818631447199817593801326010-444.950.2215231129167414323738312610-558.140.31129773127010413613286610-679.930.47084408276383460256910-799.960.54582645534033335236110-8119.90.583001613752613202215610-9159.60.781447117811529781848...........................25-10.77126051435402640072658146825-21.632.12177373139513916182875325-32.082.70145853739233881152056025-42.662.47124542138893834130544225-54.003.927131973796372976420925-66.376.3830170361335223347825-79.3412.1219044352034172134825-810.7914.011226338432541343125-911.7215.25916322330786917An organic stripping and inorganic stripping table is generated from the data in Table5. As shown in Figure 4, there are four points: A, B, C, and D. Through Table 5, we can get A (Gm, Ggl, Gfe), B (Gm, Ggl, Gfe), C (Gm, Ggl, Gfe), and D (Gm, Ggl, Gfe). Through the bilinear interpolation algorithm, we can calculate the (Gm, Ggl, Gfe) value of any point in the quadrilateral (A, B, C, and D), so that we can calculate the (Gm, Ggl, Gfe) value of any point in the coordinate system. Finally, we can realize the organic stripping and inorganic stripping of dual-energy X-ray security images.In Figure5(a), a knife is covered with some objects. In the original image, the outline of the knife is not clear, and its color is shown as orange, so it cannot be judged as a dangerous metal. In Figure 5(b), the knife is clearly visible, and it is shown as a metal color (blue), because the organic stripping algorithm is used. The outline of this knife is clear in the organic stripping image, and it is easy to be detected by the XMC R-CNN model.Figure 5
(a) Original X-ray image; (b) organic stripping image.
(a)(b)In Figure6(a), an explosive is hidden under several steel plates. In the original image, it is difficult for the security inspector to determine that there is a dangerous object in this area. In Figure 6(b), the explosive is clearly visible, and it is shown as an explosive color (orange), because the inorganic stripping algorithm is used. This is a good basis for detecting more kinds of contraband in X-ray baggage images in the future.Figure 6
(a) Original X-ray image; (b) inorganic stripping image.
(a)(b)


## 4. Evaluation

Training and testing are performed via the use of Caffe [53], a deep learning tool designed and developed by the Berkley Vision and Learning Center. The framework of the system is Caffe, the backbone model of the system is VGG-16, and the algorithm model of the system is XMC R-CNN.


### 4.1. Dataset Design

The experimental samples of different material guns are shown in Table6, and the experimental samples of different type knives are shown in Table 7.Table 6
Experimental samples of the gun.
No.PhotoNameMaterialG-1PistolMetalG-2PistolPlasticG-3PistolWoodG-4PistolMetalG-5RevolverMetalG-6RevolverPlasticG-7RevolverWoodG-8RevolverMetalG-9RifleMetalG-10RiflePlasticTable 7
Experimental samples of the knife.
No.PhotoNameMaterialK-1DaggerMetalK-2DaggerMetal and plasticK-3DaggerMetalK-4Double edged knifeMetalK-5Double edged knifeMetalK-6JackknifeMetalK-7JackknifeMetalK-8Keychain knifeMetalK-9Watermelon knifeMetalK-10ToolsMetalK-11Fruit knifeMetalK-12Fruit knifeMetalK-13Utility knifeMetalK-14Utility knifeMetal and plasticK-15BladeMetalK-16Kitchen knifeMetalK-17ScissorsMetal and plasticK-18ScissorsMetal and plasticK-19SwordMetalK-20SwordMetal


### 4.2. Training and Testing

The thirty pieces of baggage of different sizes and materials are filled with different objects, whose six pieces of baggage are complex, eighteen pieces of baggage are medium complex, and six pieces of baggage are simple. Each baggage is divided into three layers with the top facing up, and each layer is divided into nine areas. As shown in Figure7, 1–9, 11–19, and 21–29 are used to represent the nine areas of each layer, and the baggage internal space is divided into 27 areas for placing contraband samples.Figure 7
Interior space division of the baggage.Four directions of each luggage are as shown in Figure8. The top of the luggage is upward to enter the X-ray equipment channel, the top of the luggage is upward to enter the X-ray equipment channel at an angle of about 45 degrees, the bottom of the luggage is upward to enter the X-ray equipment channel, and the bottom of luggage is upward to enter the X-ray equipment channel at an angle of about 45 degrees.Figure 8
Sketch map of luggage direction.The gun samples in Table6 and the knife samples in Table 7 are placed in 27 space areas within each luggage, and the sample data are collected in four different directions as shown in Figure 8. The total number of collected samples is (10 + 20) × 30 × 27 × 4 = 97200. As shown in Figure 9, the 80000 samples are selected as the training dataset, and the 17200 samples are selected as the test dataset.Figure 9
Contraband images.


### 4.3. Test Results

At present, the accuracy and speed of various classical models are tested through the ILSVRC (ImageNet Large-Scale Visual Recognition Challenge) dataset or the PASCAL VOC (Pattern Analysis, Statistical Modelling, and Computational Learning, and Visual Object Classes) dataset. Because the characteristics of the images in these datasets are different from the characteristics of the X-ray baggage images, the performance and indicators of different models on the X-ray baggage image datasets are different from the ILSVRC dataset and the PASCAL VOC dataset. According to the test results of various classical models on the ILSVRC dataset and the PASCAL VOC dataset, then two classic models are selected for comparative testing on the X-ray baggage images dataset.By comparing the speed and accuracy of various classical models in [12–18], the basic conclusion is that the accuracy of the Faster R-CNN model is higher, while the processing speed of the SSD model and the YOLO model are faster. Therefore, the Faster R-CNN model and the SSD model are selected for comparative testing on the X-ray baggage image dataset. The experimental environment is built as follows. The configuration of the high-performance graphics workstation is shown in Table 8. The operating system is Windows 10 × 64. The CUDA (Compute Unified Device Architecture) version is CUDA 10.0, and the CUDNN (CUDA Deep Neural Network) version is CUDNN 10.1, and the Python version is Python 3.7.Table 8
High-performance graphic workstation configuration.
SpecificationsProcessorIntel Xeon® Processor E5-2623 v3 (10M Cache, 3.00 GHz)Memory16 GB (2 × 8GB) DDR4 2133 MHz with ECC REGGPU cardNvidia Tesla Titan XSystem disk120 GB Intel® SSD Pro 2500 Series 2.5 SATA 6.0 Gb/s Solid State DriveStorage4 TB Seagate Enterprise Class 3.5 SATA 6.0 Gb/sThe Faster R-CNN model and the SSD model are tested on the X-ray baggage image dataset. The Caffe deep learning framework is used in the Faster R-CNN model, and the Torch deep learning framework is used in the SSD model, and both network models are the VGG-16 model. IoU (intersection-over-union) is a concept used in the target detection, which is the overlap rate between the generated candidate bound and the ground truth bound. Here, IoU is equal to 0.5, which is used to test the X-ray baggage image dataset.The performance of the proposed method and the prior work is evaluated by comparing the following indicators: true Positive (TP%), a positive sample is predicted to be positive, and it can be called true accuracy. True negative (TN%), a negative sample is predicted to be negative, and it can be called false accuracy. False positive (FP%), a negative sample is predicted to be positive, and it can be called false alarm rate. False negative (FN%), a positive sample is predicted to be negative, and it can be called miss rate. Precision (PRE = TP/(TP + FP)), which is the proportion of true positive samples in the predicted positive examples. Recall (REC = TP/(TP + FN)), which is the proportion of true positive samples in the actual positive examples. Accuracy (ACC = (TP + TN)/(TP + FN + FP + TN)), which is the proportion of true positive samples and true negative samples in the all samples. The PR curve is a precision-recall curve, and AP is the area under the PR curve; mAP (mean average precision) is the average value of each category of AP. AP measures the quality of the model in each category, and mAP measures the quality of the model in all categories.The Faster R-CNN model is that merging RPN and Fast R-CNN into a single network by sharing their convolutional features. The framework of the system is Caffe, and the backbone model of the system is VGG-16. The SSD model is a one-stage object detection algorithm. The framework of the system is Torch, and the backbone model of the system is VGG-16. Table9 shows the test results of the PASCAL VOC dataset and the X-ray baggage image dataset. The mAP (VOC) and the FPS (VOC) are obtained on the PASCAL VOC dataset, and the mAP (X-ray) and the FPS (X-ray) are obtained on the X-ray baggage image dataset. The accuracy of the Caffe + VGG-16 + Faster R-CNN model is lower on the PASCAL VOC dataset, but the accuracy of the Caffe + VGG-16 + Faster R-CNN model is higher on the X-ray baggage images dataset. The processing speed of the Torch + VGG-16 + SSD512 model is faster on the X-ray baggage image dataset and the PASCAL VOC dataset. The processing speed of the Caffe + VGG-16 + Faster R-CNN model and the Torch + VGG-16 + SSD512 model is faster on the PASCAL VOC dataset than the X-ray baggage images dataset, because the image size of the PASCAL VOC dataset is 500 × 375, and the image size of the X-ray baggage images dataset is 1024 × 700.Table 9
Test results of the X-ray baggage image dataset.
ModelmAP (VOC)mAP (X-ray)FPS (VOC)FPS (X-ray)Faster R-CNN73.262.574SSD51276.852.11910Considering the accuracy of the Faster R-CNN model is higher than the SSD model and the processing speed of the Faster R-CNN model can also meet the current application needs, which it takes 2 seconds to collect an X-ray image. Therefore, the XMC R-CNN model was designed based on the Faster R-CNN model combined with the characteristics of the X-ray baggage images for the automatic detection of the typical guns and the typical knives in the X-ray baggage images.Table10 shows the test results of the XMC R-CNN model. The detection rates (TP%) of guns and knives are 96.5% and 95.8%, the miss rates (FN%) of guns and knives are 2.2% and 4.2%, and the accuracies (ACC) of guns and knives are 97.1% and 93.1%. It can be seen from the data that the X-ray image features of the guns are obvious, and all types of guns include barrels, butts, triggers, and other components, and these features are obviously different from the features of the common objects in the passenger baggage, so the detection rate of the gun is relatively high and the miss alarm rate of the gun is relatively low. On the contrary, the X-ray image features of the knives are weak, and the features of different types of knives are significantly different. In addition, these features are similar to the features of objects in the passenger baggage, such as baggage handles, baggage locks, umbrellas, and electronic equipment. Therefore, the detection rate of the knife is relatively low and the miss alarm rate of the knife is relatively high. At present, the maximum processing speed of the XMC R-CNN model is 250 milliseconds per image, and the processing speed can meet the requirement of collecting an X-ray image for 2 seconds. In Figure 10, the knives in these images are covered with some objects. The knives are not detected in the X-ray baggage image dataset without the material classifier, and these knives are detected in the XMC R-CNN model with the material classifier.Table 10
Test results of contraband detection.
TP (%)TN (%)FP (%)FN (%)PRERECACCGuns97.896.93.12.291.397.897.1Knives95.891.38.84.287.995.893.1Figure 10
Test images.
(a)(b)(c)(d)


## 4.1. Dataset Design

The experimental samples of different material guns are shown in Table6, and the experimental samples of different type knives are shown in Table 7.Table 6
Experimental samples of the gun.
No.PhotoNameMaterialG-1PistolMetalG-2PistolPlasticG-3PistolWoodG-4PistolMetalG-5RevolverMetalG-6RevolverPlasticG-7RevolverWoodG-8RevolverMetalG-9RifleMetalG-10RiflePlasticTable 7
Experimental samples of the knife.
No.PhotoNameMaterialK-1DaggerMetalK-2DaggerMetal and plasticK-3DaggerMetalK-4Double edged knifeMetalK-5Double edged knifeMetalK-6JackknifeMetalK-7JackknifeMetalK-8Keychain knifeMetalK-9Watermelon knifeMetalK-10ToolsMetalK-11Fruit knifeMetalK-12Fruit knifeMetalK-13Utility knifeMetalK-14Utility knifeMetal and plasticK-15BladeMetalK-16Kitchen knifeMetalK-17ScissorsMetal and plasticK-18ScissorsMetal and plasticK-19SwordMetalK-20SwordMetal


## 4.2. Training and Testing

The thirty pieces of baggage of different sizes and materials are filled with different objects, whose six pieces of baggage are complex, eighteen pieces of baggage are medium complex, and six pieces of baggage are simple. Each baggage is divided into three layers with the top facing up, and each layer is divided into nine areas. As shown in Figure7, 1–9, 11–19, and 21–29 are used to represent the nine areas of each layer, and the baggage internal space is divided into 27 areas for placing contraband samples.Figure 7
Interior space division of the baggage.Four directions of each luggage are as shown in Figure8. The top of the luggage is upward to enter the X-ray equipment channel, the top of the luggage is upward to enter the X-ray equipment channel at an angle of about 45 degrees, the bottom of the luggage is upward to enter the X-ray equipment channel, and the bottom of luggage is upward to enter the X-ray equipment channel at an angle of about 45 degrees.Figure 8
Sketch map of luggage direction.The gun samples in Table6 and the knife samples in Table 7 are placed in 27 space areas within each luggage, and the sample data are collected in four different directions as shown in Figure 8. The total number of collected samples is (10 + 20) × 30 × 27 × 4 = 97200. As shown in Figure 9, the 80000 samples are selected as the training dataset, and the 17200 samples are selected as the test dataset.Figure 9
Contraband images.


## 4.3. Test Results

At present, the accuracy and speed of various classical models are tested through the ILSVRC (ImageNet Large-Scale Visual Recognition Challenge) dataset or the PASCAL VOC (Pattern Analysis, Statistical Modelling, and Computational Learning, and Visual Object Classes) dataset. Because the characteristics of the images in these datasets are different from the characteristics of the X-ray baggage images, the performance and indicators of different models on the X-ray baggage image datasets are different from the ILSVRC dataset and the PASCAL VOC dataset. According to the test results of various classical models on the ILSVRC dataset and the PASCAL VOC dataset, then two classic models are selected for comparative testing on the X-ray baggage images dataset.By comparing the speed and accuracy of various classical models in [12–18], the basic conclusion is that the accuracy of the Faster R-CNN model is higher, while the processing speed of the SSD model and the YOLO model are faster. Therefore, the Faster R-CNN model and the SSD model are selected for comparative testing on the X-ray baggage image dataset. The experimental environment is built as follows. The configuration of the high-performance graphics workstation is shown in Table 8. The operating system is Windows 10 × 64. The CUDA (Compute Unified Device Architecture) version is CUDA 10.0, and the CUDNN (CUDA Deep Neural Network) version is CUDNN 10.1, and the Python version is Python 3.7.Table 8
High-performance graphic workstation configuration.
SpecificationsProcessorIntel Xeon® Processor E5-2623 v3 (10M Cache, 3.00 GHz)Memory16 GB (2 × 8GB) DDR4 2133 MHz with ECC REGGPU cardNvidia Tesla Titan XSystem disk120 GB Intel® SSD Pro 2500 Series 2.5 SATA 6.0 Gb/s Solid State DriveStorage4 TB Seagate Enterprise Class 3.5 SATA 6.0 Gb/sThe Faster R-CNN model and the SSD model are tested on the X-ray baggage image dataset. The Caffe deep learning framework is used in the Faster R-CNN model, and the Torch deep learning framework is used in the SSD model, and both network models are the VGG-16 model. IoU (intersection-over-union) is a concept used in the target detection, which is the overlap rate between the generated candidate bound and the ground truth bound. Here, IoU is equal to 0.5, which is used to test the X-ray baggage image dataset.The performance of the proposed method and the prior work is evaluated by comparing the following indicators: true Positive (TP%), a positive sample is predicted to be positive, and it can be called true accuracy. True negative (TN%), a negative sample is predicted to be negative, and it can be called false accuracy. False positive (FP%), a negative sample is predicted to be positive, and it can be called false alarm rate. False negative (FN%), a positive sample is predicted to be negative, and it can be called miss rate. Precision (PRE = TP/(TP + FP)), which is the proportion of true positive samples in the predicted positive examples. Recall (REC = TP/(TP + FN)), which is the proportion of true positive samples in the actual positive examples. Accuracy (ACC = (TP + TN)/(TP + FN + FP + TN)), which is the proportion of true positive samples and true negative samples in the all samples. The PR curve is a precision-recall curve, and AP is the area under the PR curve; mAP (mean average precision) is the average value of each category of AP. AP measures the quality of the model in each category, and mAP measures the quality of the model in all categories.The Faster R-CNN model is that merging RPN and Fast R-CNN into a single network by sharing their convolutional features. The framework of the system is Caffe, and the backbone model of the system is VGG-16. The SSD model is a one-stage object detection algorithm. The framework of the system is Torch, and the backbone model of the system is VGG-16. Table9 shows the test results of the PASCAL VOC dataset and the X-ray baggage image dataset. The mAP (VOC) and the FPS (VOC) are obtained on the PASCAL VOC dataset, and the mAP (X-ray) and the FPS (X-ray) are obtained on the X-ray baggage image dataset. The accuracy of the Caffe + VGG-16 + Faster R-CNN model is lower on the PASCAL VOC dataset, but the accuracy of the Caffe + VGG-16 + Faster R-CNN model is higher on the X-ray baggage images dataset. The processing speed of the Torch + VGG-16 + SSD512 model is faster on the X-ray baggage image dataset and the PASCAL VOC dataset. The processing speed of the Caffe + VGG-16 + Faster R-CNN model and the Torch + VGG-16 + SSD512 model is faster on the PASCAL VOC dataset than the X-ray baggage images dataset, because the image size of the PASCAL VOC dataset is 500 × 375, and the image size of the X-ray baggage images dataset is 1024 × 700.Table 9
Test results of the X-ray baggage image dataset.
ModelmAP (VOC)mAP (X-ray)FPS (VOC)FPS (X-ray)Faster R-CNN73.262.574SSD51276.852.11910Considering the accuracy of the Faster R-CNN model is higher than the SSD model and the processing speed of the Faster R-CNN model can also meet the current application needs, which it takes 2 seconds to collect an X-ray image. Therefore, the XMC R-CNN model was designed based on the Faster R-CNN model combined with the characteristics of the X-ray baggage images for the automatic detection of the typical guns and the typical knives in the X-ray baggage images.Table10 shows the test results of the XMC R-CNN model. The detection rates (TP%) of guns and knives are 96.5% and 95.8%, the miss rates (FN%) of guns and knives are 2.2% and 4.2%, and the accuracies (ACC) of guns and knives are 97.1% and 93.1%. It can be seen from the data that the X-ray image features of the guns are obvious, and all types of guns include barrels, butts, triggers, and other components, and these features are obviously different from the features of the common objects in the passenger baggage, so the detection rate of the gun is relatively high and the miss alarm rate of the gun is relatively low. On the contrary, the X-ray image features of the knives are weak, and the features of different types of knives are significantly different. In addition, these features are similar to the features of objects in the passenger baggage, such as baggage handles, baggage locks, umbrellas, and electronic equipment. Therefore, the detection rate of the knife is relatively low and the miss alarm rate of the knife is relatively high. At present, the maximum processing speed of the XMC R-CNN model is 250 milliseconds per image, and the processing speed can meet the requirement of collecting an X-ray image for 2 seconds. In Figure 10, the knives in these images are covered with some objects. The knives are not detected in the X-ray baggage image dataset without the material classifier, and these knives are detected in the XMC R-CNN model with the material classifier.Table 10
Test results of contraband detection.
TP (%)TN (%)FP (%)FN (%)PRERECACCGuns97.896.93.12.291.397.897.1Knives95.891.38.84.287.995.893.1Figure 10
Test images.
(a)(b)(c)(d)


## 5. Conclusion and Future Work

In this work, the XMC R-CNN model is explored in the tasks of classification and detection within X-ray baggage images. The main contribution of the XMC R-CNN model is used to solve the problem of contraband detection in overlapped X-ray baggage images by the X-ray material classifier algorithm and organic stripping and inorganic stripping algorithm, and the detection rate and the miss rate that meet the requirements of screening on-site are achieved by the deep learning method. The detection rate is greater than 95%, and the miss rate is less than 5%. In some applications, it has exceeded the level of security inspectors.The automatic detection technology of the contraband based on the XMC R-CNN model is applied to the X-ray baggage security image. According to user needs, the safe X-ray baggage images can be automatically filtered in some specific fields, which reduces the number of X-ray baggage images that security inspectors need to screen. The efficiency of security inspection is improved and the labor intensity of security inspection is reduced. In addition, the security inspector can screen the X-ray baggage image according to the box of automatic detection, which can improve the effect of security inspection.Future work will consider exploring the threat image projection (TIP) research in order to increase the training dataset, which can improve the accuracy of the automatic detection algorithm. On the other hand, future work will consider developing the XMC R-CNN model based on the safe objects to reduce the false alarm rate. These safe objects have the marked features and are common in passenger baggage, such as baggage handles, baggage locks, umbrellas, watches, power banks, and electronic equipment. The detection rate of the safe object is improved, and the false alarm rate of the contraband is reduced.



---

*Source: 1823034-2020-09-16.xml*