EAOD‐Net: Effective anomaly object detection networks for X‐ray images

IET Image ProcessingVolume 16, Issue 10 p. 2638-2651 ORIGINAL RESEARCHOpen Access EAOD-Net: Effective anomaly object detection networks for X-ray images Chunjie Ma, Chunjie Ma orcid.org/0000-0002-6348-671X Beijing Key Laboratory of Computational Intelligence and Intelligent System, Beijing University of Technology, 100 Pingleyuan, Beijing, ChinaSearch for more papers by this authorLi Zhuo, Corresponding Author Li Zhuo [email protected] Beijing Key Laboratory of Computational Intelligence and Intelligent System, Beijing University of Technology, 100 Pingleyuan, Beijing, China Correspondence Li Zhuo, Beijing Key Laboratory of Computational Intelligence and Intelligent System, Beijing University of Technology, 100 Pingleyuan, Beijing 100124, China. Email: [email protected]Search for more papers by this authorJiafeng Li, Jiafeng Li Beijing Key Laboratory of Computational Intelligence and Intelligent System, Beijing University of Technology, 100 Pingleyuan, Beijing, ChinaSearch for more papers by this authorYutong Zhang, Yutong Zhang Beijing Key Laboratory of Computational Intelligence and Intelligent System, Beijing University of Technology, 100 Pingleyuan, Beijing, ChinaSearch for more papers by this authorJing Zhang, Jing Zhang orcid.org/0000-0003-1290-0738 Beijing Key Laboratory of Computational Intelligence and Intelligent System, Beijing University of Technology, 100 Pingleyuan, Beijing, ChinaSearch for more papers by this author Chunjie Ma, Chunjie Ma orcid.org/0000-0002-6348-671X Beijing Key Laboratory of Computational Intelligence and Intelligent System, Beijing University of Technology, 100 Pingleyuan, Beijing, ChinaSearch for more papers by this authorLi Zhuo, Corresponding Author Li Zhuo [email protected] Beijing Key Laboratory of Computational Intelligence and Intelligent System, Beijing University of Technology, 100 Pingleyuan, Beijing, China Correspondence Li Zhuo, Beijing Key Laboratory of Computational Intelligence and Intelligent System, Beijing University of Technology, 100 Pingleyuan, Beijing 100124, China. Email: [email protected]Search for more papers by this authorJiafeng Li, Jiafeng Li Beijing Key Laboratory of Computational Intelligence and Intelligent System, Beijing University of Technology, 100 Pingleyuan, Beijing, ChinaSearch for more papers by this authorYutong Zhang, Yutong Zhang Beijing Key Laboratory of Computational Intelligence and Intelligent System, Beijing University of Technology, 100 Pingleyuan, Beijing, ChinaSearch for more papers by this authorJing Zhang, Jing Zhang orcid.org/0000-0003-1290-0738 Beijing Key Laboratory of Computational Intelligence and Intelligent System, Beijing University of Technology, 100 Pingleyuan, Beijing, ChinaSearch for more papers by this author First published: 22 April 2022 https://doi.org/10.1049/ipr2.12514AboutSectionsPDF ToolsRequest permissionExport citationAdd to favoritesTrack citation ShareShare Give accessShare full text accessShare full-text accessPlease review our Terms and Conditions of Use and check box below to share full-text version of article.I have read and accept the Wiley Online Library Terms and Conditions of UseShareable LinkUse the link below to share a full-text version of this article with your friends and colleagues. Learn more.Copy URL Share a linkShare onFacebookTwitterLinkedInRedditWechat Abstract Anomaly object detection is the core technology in the application for X-ray images. However, the accuracy of current X-ray anomaly object detection method still needs to be improved. In this paper, an effective anomaly object detection network is proposed to improve the detection accuracy of anomaly object for X-ray images. Firstly, learnable Gabor convolution layer, deformable convolution, and spatial attention mechanism are introduced to enhance the representative ability of features in ResNeXt. Then, dense local regression is applied to predict the offset of multiple dense boxes in region proposal to locate the object accurately. At last, bigger discriminative RoI pooling is proposed to classify the candidate boxes to improve the accuracy of object classification. Experimental results on the SIXray and OPIXray datasets show that compared with the state-of-the-art methods, the proposed EAOD-Net can achieve the competitive detection performance. 1 INTRODUCTION Ensuring the personal safety of passengers is the primary task for the transportation departments [1]. There may be dangerous goods in passengers' luggage, such as controlled knives, explosives, guns, and radioactive substances [2]. Anomaly object detection is the core technology in the application for X-ray images, in this application, high-precision detection performance is necessary. In practical applications, X-ray has unique advantages in many aspects, such as industrial components defect detection [3, 4], COVID-19 cases detection [5, 6], skeletal fluorosis diagnosis [7], especially in security inspection. At present, the contraband of X-ray images is mainly detected in a manual way, which leads to the accuracy of security inspection easily being affected by the individual status and experience of the security inspectors. The missing and wrong inspection of anomaly objects often occurs, resulting in great security risks. In recent years, researchers began to apply various technical means to increase the detection accuracy of anomaly objects for X-ray images in practical applications. However, the anomaly object detection for X-ray images enfaces great challenges, due to the following reasons: (1) the objects in the luggage are often placed at any angle; (2) the multiple objects are overlapped; (3) the variations of the object scales are dramatic. Recent years have witnessed the unprecedented success of convolutional neural networks (CNNs) in object detection [8-10]. These methods were originally designed for natural images. Although they could achieve high accuracy, the performance is not satisfactory when applied to the X-ray image object detection task, due to the fact that the X-ray images have different characteristics from the natural images, such as the lack of texture and colour information [11]. Therefore, in order to improve the accuracy of object detection for X-ray images, it is essential to design the anomaly object detection method for X-ray images specifically in accordance with the characteristics of the X-ray images. This article presents an anomaly object detection method for X-ray images with multiple convolutions and bigger discriminative region of interest (RoI) pooling. The main contributions of this study are as follows: (1) According to the fact that the object edge in the X-ray images is obvious, but the texture is scarce, ResNeXt-50 is used as the backbone network and a learnable Gabor convolution layer is introduced into the lower layer to capture the edge of the object. In addition, deformable convolution is incorporated into the middle layer to solve the multi-scale deformation problem of the objects. And the spatial attention mechanism is further performed in the high layer to improve the semantic representative ability of the deep features; (2) dense local regression (DLR) is used to locate the objects finely; (3) bigger discriminative RoI pooling (BDRP) is used to classify the objects accurately. In the rest of the paper, after reviewing related work in Section 2, the proposed method is described in details in Section 3. Section 4 reports the experimental results and analysis. Finally, the conclusions are drawn in Section 5. 2 RELATED WORK In recent years, object detection based on deep learning has made great advances, and various object detection methods have emerged [12-15]. The researchers proposed some anomaly object detection methods for X-ray images [16-18], which refer to the framework of object detection in natural images. Akcay et al. [19] has pioneered the research work on using deep learning to classify the anomaly objects for X-ray images. In their work, they created a dataset for firearm detection, and employed GoogleNet and AlexNet to classify the anomaly objects. The experimental results clearly demonstrate the applicability of CNN within X-ray baggage imagery. In ref. [20], they further studied the classification and location of anomaly objects for X-ray images. The features extracted by using bag-of-visual-word (BoVW) model were compared with those from CNN. The experimental results show that the features extracted by CNN have a stronger representative ability than the traditional BoVW features. At the same time, it was proved that the region based object detection network model is more suitable for anomaly object detection in X-ray images. The samples are hard to collect in practical applications. Thus, the amount of the samples is usually not sufficient to train the CNN models. For this reason, Zhu et al. [21] used the improved self-attention generative adversarial network (SAGAN) to increase the number of the X-ray images. And then, the cycle generative adversarial networks (CGAN) is utilised to transform the shape and posture of the contraband. The transformed image is combined with the background to generate a synthesis image, which is input into the single shot multibox detector (SSD) for anomaly object detection. The experimental results show that the detection performance has been improved. Dumagpi et al. [22] adopted deep convolutional generative adversarial networks (DCGAN) to transform the contraband objects. The transformed ones are then fused with the normal X-ray images to generate the synthesis image. The enhanced dataset is used to train Faster R-CNN. The experimental results show that the method can solve the problem of sample imbalance to a certain extent. Similar to the object detection task for the natural images, there is a huge difference in the number of foreground and background in anomaly object detection of X-ray images. To solve this problem, Jaccard et al. [23] proposed a framework, namely cascaded structure tensor (CST), which uses a cascaded structure to extract contour-based proposals at first, and then the proposals are input to the feedforward neural network for detection. Experimental results show that CST can effectively solve the problem of the imbalance between the number of the foreground and background in the task of anomaly object detection for X-ray images. In order to further improve the detection accuracy, some researchers began to study the multi-view anomaly object detection for X-ray images. Liang et al. [12] used multi-view images to detect anomaly objects for X-ray images. In the training stage, all the viewing angles of the same object can be obtained after training a scanner to score the performance according to the index. In the detection stage, if one image is justified as abnormal, the images in the same group are regarded as containing anomaly objects. In ref. [24], multi-view images are fused by CNN, and then the fused features are used as the input for subsequent detection. In ref. [25], multi-view images are input into the same CNN for feature extraction firstly, then the extracted features are input into multi-view pooling for fusion. Finally, the fused 3D features are input into 3D region proposal network (RPN) and 3D RoI pooling for subsequent classification and regression. In general, the X-ray image object detection methods based on deep learning generally adopted the basic idea and framework of object detection for natural images. However, because the X-ray images have unique properties different from the natural images, the current detection accuracy is still unable to meet the requirements of practical applications. 3 PROPOSED EAOD-NET SCHEME According to the characteristics of the X-ray images, such as sharp edges, variable scales and different angles of the objects, this paper uses ResNeXt as the backbone network and proposes an anomaly object detection method for X-ray images. The overall network architecture and flow diagram are shown in Figures 1 and 2. It adopts a two-stage framework. At the first stage, RPN is used to determine the candidate regions of the objects, in which a CNN network model is designed to extract the deep features. At the second stage, DLR module is applied to predict the locations finely and BDRP module to classify the objects accurately. FIGURE 1Open in figure viewerPowerPoint The overall network architecture of the proposed method FIGURE 2Open in figure viewerPowerPoint Flow diagram of the EAOD-Net Next, this paper will introduce the three key components of the framework in details, i.e. feature extraction, DLR and BDRP[26] are introduced. Feature extraction ResNeXt [26] is a network architecture proposed by Xie et al. in 2016. The architecture is much simpler than all Inception models, and requires considerably fewer hyper-parameters to be set by hand. It adopts the idea of group convolution, and each branch uses the same topological structure. Thereby, more diverse feature representation can be learned. In recent years, ResNeXt has been increasingly applied to various industrial applications [27-30], and achieved excellent performance. However, ResNeXt is designed for the natural images. When applied to the X-ray image processing, it is necessary to consider the specific characteristics of the X-ray images to design the corresponding mechanism. This paper uses ResNeXt-50 as the backbone network. Considering that the edge of the objects for X-ray images is very clear, a learnable Gabor convolutional layer is added to the lower layer to enhance the network's ability to capture the edge information of the object. Due to the characteristics of X-ray images with variable scales and different angles, deformable convolution is used at Stage 3–Stage 5 of ResNeXt-50 to further improve the ability to characterise the multi-scale deformation. A spatial attention module is added to the higher layer to improve the semantic representative ability of the deep features. The detailed implementation of the learnable Gabor convolutional layer, deformable convolution and spatial attention mechanism is discussed below. 3.1.1 Learnable Gabor convolutional layer Gabor filter is a kind of linear filter, which can analyse whether there is a specific frequency component in an image [31-33]. The two-dimensional Gabor filter is modulated by a harmonic function and a Gaussian function. Some studies have shown that the Gabor filter is particularly similar to the partial response of human visual cortex and is extremely sensitive to the image edge, so it is widely used in low-level visual feature extraction. In this paper, a learnable Gabor convolution layer is introduced into ResNeXt-50 to enhance the ability to capture the edge information of the object. The two-dimensional Gabor filter is defined as: g ( x , y , λ , θ , φ , σ , γ ) = exp − x ′ 2 + γ 2 y ′ 2 2 σ 2 exp i 2 π x ′ λ + φ \begin{eqnarray} g(x, y , \lambda , \theta , \varphi , \sigma , \gamma )\!=\!\exp \! {\left(\!-\frac{x^{\prime 2}+\gamma ^{2} y^{\prime 2}}{2 \sigma ^{2}}\!\right)} \!\exp \!{\left(i{\left(2 \pi \frac{x^{\prime }}{\lambda }\!+\!\varphi \right)}\right)}\hskip-6pt\nonumber\\[-4pt] \end{eqnarray} (1) Real: g ( x , y , λ , θ , φ , σ , γ ) = exp − x ′ 2 + γ 2 y ′ 2 2 σ 2 cos 2 π x ′ λ + φ \begin{eqnarray} g(x, y , \lambda , \theta , \varphi , \sigma , \gamma )=\exp {\left(-\frac{x^{\prime 2}+\gamma ^{2} y^{\prime 2}}{2 \sigma ^{2}}\right)} \cos {\left(2 \pi \frac{x^{\prime }}{\lambda }+\varphi \right)}\hskip-6pt\nonumber\\[-4pt] \end{eqnarray} (2) Imaginary: g ( x , y ; λ , θ , φ , σ , γ ) = exp − x ′ 2 + γ 2 y ′ 2 2 σ 2 sin 2 π x ′ λ + φ \begin{eqnarray} g(x, y ; \lambda , \theta , \varphi , \sigma , \gamma )=\exp {\left(-\frac{x^{\prime 2}+\gamma ^{2} y^{\prime 2}}{2 \sigma ^{2}}\right)} \sin {\left(2 \pi \frac{x^{\prime }}{\lambda }+\varphi \right)}\hskip-6pt\nonumber\\[-4pt] \end{eqnarray} (3) And x ′ = x cos θ + y sin θ y ′ = y cos θ − x sin θ \begin{equation} \begin{aligned} &x^{\prime }=x \cos \theta +y \sin \theta \\ &y^{\prime }=y \cos \theta -x \sin \theta \end{aligned} \end{equation} (4)where λ is the wavelength of harmonic function, θ is the direction of parallel fringe between normal and Gabor kernel function, φ is the phase shift, σ is the standard deviation of Gaussian factor, and γ is the spatial aspect ratio. Experimental results show that the real component can smooth the image, while the imaginary component is more sensitive to the edge of the two-dimensional Gabor filter. Therefore, the imaginary component is used as the kernel function. Figure 3 is the learnable Gabor convolution layer structure proposed in this paper, where F k $Fk$ , k ∈ ( 1 $k\in (1$ , 2 … N ) $2\ldots N)$ represents the output features after the previous convolution layer, k denotes k th $\text{th}$ channel, N is the total number of the channels, F k j ′ ${Fk}_j^\prime$ , k ∈ ( 1 $k\in (1$ , 2 … N ) , j ∈ ( 1 , 2 , 3 , 4 ) $2\ldots N),j\in (1,2,3,4)$ are the output feature maps after F k $Fk$ is convoluted with four directions Gabor filter, j represents the j th $\text{th}$ direction. And F k ′ ${Fk}^\prime$ , k ∈ ( 1 , 2 … N ) $k\in (1,2\ldots N)$ represents the fused feature maps of F k j ′ ${Fk}_j^\prime$ . FIGURE 3Open in figure viewerPowerPoint Learnable Gabor convolution layer structure The parameters of the kernel function in the Gabor convolutional layer are learnable, in which the optimal parameters can be automatically learned according to the characteristics of the anomaly object in the X-ray image. At the same time, in order to fully exploit the edge-sensitive characteristics of the Gabor filter, this paper uses the Gabor filters with the angles of 0 ° , 45 ° , 90 ° , and 135 ° to perform convolution operations on the input features. To ensure that the layer is easy to implement, we sets the size of the Gabor filter to 3 × 3, stride = 1 and padding = 1. Through this setting, the layer can be inserted into any position of CNN. In this paper, the Gabor convolutional layer is added to the second convolution layer of ResNeXt-50, which is labelled as conv2 in Table 1. After the input image went through the first 3 × 3 convolution in the stage2 of ResNeXt-50, its features were enhanced through the Gabor convolution layer, and the enhanced features were processed through 1 × 1 convolution for feature fusion, and then the fused features were used for subsequent processing. TABLE 1. Position and parameter configuration of the learnable Gabor convolutional layer in the ResNeXt-50 Stage Output ResNeXt-50(32 × 4d) conv1 112 × 112 7 × 7, 64, stride 2 3 × 3 max pool, stride 2 3 × 3 max pool, stride 2 conv2 56 × 56 1 × 1 , 128 3 × 3 , 128 , C = 32 Gabor 3 × 3 , 128 , 1 × 1 , 256 × 3 $\left[ \def\eqcellsep{&}\begin{array}{cc} 1\times 1,\ 128\eqcellsep\ \\ 3\times 3,\ 128,\eqcellsep\quad C=32\\ \def\eqcellsep{&}\begin{array}{c} \text{Gabor}\ 3\times 3,\ 128,\\ 1\times 1,\ 256 \end{array} \eqcellsep \def\eqcellsep{&}\begin{array}{c} \ \\ \ \\ \end{array} \end{array} \right]\times 3$ 3.1.2 Deformable convolution In the X-ray images, the shape and size of the anomaly objects vary dramatically. Even if the same object is placed in different positions, it will render huge differences. Because the conventional convolution is limited to the fixed geometric structure, it is difficult to effectively extract the features of anomaly objects from different aspects. Therefore, this paper introduces deformable convolution [34] into ResNeXt to solve the complex deformation problem of the anomaly object. The structure and flow diagram of deformable convolution are shown in Figures 4 and 5. FIGURE 4Open in figure viewerPowerPoint Schematic diagram of deformable convolution FIGURE 5Open in figure viewerPowerPoint Flow diagram of the deformable convolution The previous feature maps are input into the convolution layer. After deformable convolution, the offset field is obtained, including 3N channels. Among them, 2N channels represent the learned offsets, including both x and y directions, and the remaining N channels represent the learned offset's weights. Their function is to assign a weight to each point in the feature map to ensure that the network can learn more important features. Then the input feature map is modulated with offsets and offset weights to obtain the output feature map. The specific modulation process is expressed by the following mathematical formula: y ( ρ ) = ∑ k = 1 K x ( ρ + Δ ρ k ) · w k · Δ m k , \begin{equation} y(\rho )=\sum _{k=1}^{K}x(\rho +\Delta \rho _{k})\cdot w_k \cdot \Delta m_k,\end{equation} (5)where x and y are the input feature map and the output feature map, respectively. p represents the p th $\text{th}$ point in the feature map. k is the k th $\text{th}$ point in the convolution kernel, and the range of its value is [1, K] . For example, when the size of the convolution kernel is 3 × 3 , K = 9 $3\times 3, K=9$ . w k $w_k$ is the weight corresponding to the k th $\text{th}$ point in the convolution kernel, and Δ m k $\Delta m_k$ is the offset weight learned by the convolution layer, and the range of its value is [0, 1] . Figure 6 shows the comparison between the conventional convolution and deformable convolution. It can be seen that when the shape of anomaly objects in the X-ray image changes due to their different positions, the conventional convolution can only sample the area with a fixed size, while the deformable convolution can change the sampling area with the change of the object shape. The deformable convolution can effectively enhance the adaptability of features to the geometric deformation and scale change of the object, which greatly increases the proportion of the foreground object in the receptive field. Obviously, this ability is particularly important for the feature extraction of anomaly objects for X-ray images. FIGURE 6Open in figure viewerPowerPoint The comparison diagram of sampling points between deformable convolution and conventional convolution. The first column represents the X-ray images of the same pair of scissors under different viewing angles, the second column and the third column represent the sampling point distribution when using conventional convolution and deformable convolution to extract features of the scissors This paper replaces conventional convolution with deformable convolution in Stage 3–Stage 5 of ResNeXt-50, and the specific position is shown in Table 2. TABLE 2. The position of spatial attention module in ResNeXt-50 Stage Output ResNeXt-50(32 × 4d) conv3 28 × 28 DCNv 2 1 × 1 , 256 3 × 3 , 256 , C = 32 1 × 1 , 512 × 4 $\text{DCNv}2\left[ \def\eqcellsep{&}\begin{array}{cc} 1\times 1,\ 256\eqcellsep \ \\ 3\times 3,\ 256,\eqcellsep C=32\\ 1\times 1,\ 512\eqcellsep \ \end{array} \right]\times 4$ conv4 14 × 14 DCNv 2 1 × 1 , 512 3 × 3 , 512 , C = 32 1 × 1 , 1024 SA × 6 $\left[ \def\eqcellsep{&}\begin{array}{c} \text{DCNv}2\left[ \def\eqcellsep{&}\begin{array}{cc} 1\times 1,\ 512\eqcellsep \ \\ 3\times 3,\ 512,\eqcellsep C=32\\ 1\times 1,\ 1024\eqcellsep \ \end{array} \right]\\ \text{SA} \end{array} \right]\times 6$ conv5 7 × 7 DCNv 2 1 × 1 , 1024 3 × 3 , 1024 , C = 32 1 × 1 , 2048 SA × 3 $\left[ \def\eqcellsep{&}\begin{array}{c} \text{DCNv}2\left[ \def\eqcellsep{&}\begin{array}{cc} 1\times 1,\ 1024\eqcellsep\ \\ 3\times 3,\ 1024,\eqcellsep C=32\\ 1\times 1,\ 2048\eqcellsep \ \end{array} \right]\\ \text{SA} \end{array} \right]\times 3$ FPN 3.1.3 Spatial attention mechanism The attention mechanism is unique for the human brain. The human brain quickly scans an image to obtain the object area that needs to be focused on, and then enhances it while suppressing other useless areas. The ability of human beings is generated through long-term evolution, greatly improving the efficiency and accuracy of human visual information processing. Based on human perception, many attention mechanisms have been proposed, such as spatial attention [35], channel attention [36], spatial–temporal attention [37], and so on. Spatial attention (SA) can improve the representative ability of the features for the important areas, especially anomaly objects. Therefore, this paper chooses SA to enhance the representative ability of the features. The spatial attention module is shown in Figure 7. FIGURE 7Open in figure viewerPowerPoint Spatial attention module In Figure 7, F k $Fk$ , k ∈ ( 1 $k\in (1$ , 2, … N ) $\ldots N)$ represents the input feature maps, and k represents the k th $\text{th}$ channel. Perform MaxPooling and AvgPooling operations on the corresponding points of each channel of the feature map to obtain two feature matrices M and A . After adding the two matrices, M A $MA$ is obtained, and then the probability matrix P is calculated by using the Sigmoid activation function. After P is multiplied by the input feature map, the output feature map O k $Ok$ , k ∈ ( 1 $k\in (1$ , 2, … N ) $\ldots N)$ is obtained. The mathematical formula can be expressed as follows: M i j = max ∑ k = 1 C F k i , j , i ∈ ( 1 , 2 , … , H ) , j ∈ ( 1 , 2 … , W ) , \begin{equation} M_{ij}\!=\!\text{max}{\left(\sum _{k=1}^{C}Fk{\left(i,j\right)}\!\right)},i\!\in \!(1,2,\ldots,H),\ j\!\in \!(1,2\ldots,W), \end{equation} (6) A i j = 1 H × W ∑ k = 1 C F k ( i , j ) , i ∈ ( 1 , 2 , … , H ) , j ∈ ( 1 , 2 … , W ) , \begin{equation} A_{ij}\!=\!\frac{1}{H\times W}\sum _{k=1}^{C}{Fk(i,j)},\ i\!\in \!(1,2,\ldots,H),\ j\!\in \!(1,2\ldots,W), \end{equation} (7) O k = F k · 1 1 + e − ( M i j + A i j ) , i ∈ ( 1 , 2 , … , H ) , j ∈ ( 1 , 2 … , W ) , \begin{equation} O_k\!=\!Fk \cdot \frac{1}{1+e^{-(M_{ij}+A_{ij})}},i\!\in \!(1,2,\ldots,H),\ j\!\in \!(1,2\ldots,W), \end{equation} (8)where A represents the feature matrix after AvgPooling, H and W represents the height and width of the input feature map respectively, k represents the k th $\text{th}$ channel of the feature map, C represents the total number of channels, F k $Fk$ represents the input feature map of the k th $\text{th}$ channel, M represents the feature matrix after MaxPooling, and O k $O_k$ represents the final output feature map. This paper adds the spatial attention module to the Stage 4 and Stage 5 of ResNeXt-50. The specific location is shown in Table 2. By using the spatial attention mechanism, the weight of anomaly object features can be enhanced, which greatly enhances the ability of the features to characterise the important objects. Bigger discriminative RoI pooling X-ray image is input into the feature extraction network to get the feature map, which is sent to the RPN to determine the candidate areas of the objects, and then the candidate areas are sent to the object classification and coordinate regression module to determine the refined location and category of the objects. This paper uses bigger discriminative RoI pooling, named as BDRP, to classify the category of the objects, as shown in Figure 1. In BDRP, the feature map X output by RPN is first processed by RoI pooling, and a reshaped feature map with the fixed size can be obtained. The reshaped feature map is passed through a fully connected layer to obtain the offset matrix Δ p i j $\Delta p_{ij}$ of the region, and the region of interest F after the offset is obtained by Δ p i j $\Delta p_{ij}$ . Next, F is passed through a CNN layer to obtain the weight matrix W ( F ) $W(F)$ of each pixel in F . This step refers to the basic idea of the attention mechanism, that is, assigning more important weights to the important points through learning. After that, this paper multiplies the weight matrix with the original region of interest F . This process starts from the upper left corner and uses each (2 × 2) region as a bin, then the four points corresponding to each bin in F and W ( F ) $W(F)$ are multiplied, added together and averaged. After the above operations, a new RoI O is finally obtained, the specific calculation can be expressed by a mathematical formula as follows: O i , j = ∑ ρ , f ∈ bin ( i , j ) X ( ρ 0 + ρ + Δ ρ i j ) · w ( f ) / n i j , \begin{equation} O{\left(i,j\right)}=\sum _{\rho ,f\in \text{bin}(i,j)} X(\rho _0 + \rho + \Delta \rho _{ij}) \cdot w(f)/n_{ij}, \end{equation} (9)where O represents the output feature map, X represents the input feature map, p represents the pixel in bin ( i , j ) $\text{bin}(i,j)$ , p0 represents the pixel in the upper left corner of the input feature map, Δ p i j $\Delta p_{ij}$ represents the learned offsets, f represents the learned weight position, w ( f ) $w(f)$ represents the learned weight corresponding to f position and n i j $n_{ij}$ represents the number of pixels in the ( i , j ) $(i,j)$ th $\text{th}$ bin. After the new area O is obtained, it is input into two fully connected layers for classification, and the final classification result can be obtained. Dense local regression Tradi