# Deep Feature Fusion Based Dual Branch Network for X-ray Security Inspection Image Classification
## 3 Method
### Feature Fusion Model  
While generating features in a CNN model, with the network level deepening continuously, the feature maps generated at different levels show various characteristics. Specifically, the low-level features have higher resolution. This better preserves the spatial detail information the image itself has, but the semantic information utilized for recognition is insufficient. The high-level features are processed by multiple convolutional layers so that their semantic information is comparatively rich. However, due to the lack of resolution in the convolutional calculation process, the loss of spatial information is  
Figure 1: Dual branch network architecture. As for separating it from up to down, the entire architecture can consider as a fusion strategy that adopts both early fusion (up) and late fusion (down) at the level of feature and loss, respectively. quite enormous. Consequently, the receptive ability for detail becomes poor. The task of deep feature fusion is to endow the two types of features with efficient fusion and complementary advantages, so as to improve the classification and detection performance of the network. In single-label image classification tasks, high-level features are adequate to make the model recognize and classify sample images. However, in multi-label object recognition tasks, sample images often contain multiple objects belonging to various classes. In this case, a CNN model is not capable of recognizing all objects when merely using semantic information from high-level features, especially objects of smaller sizes. Therefore, a network designed for single-label classification has poor performance for this task. Given this situation, this paper proposes a CNN model architecture that adds a deep feature fusion module, as shown in Figure 2.  
As for CNN, a prevalent variant such as ResNet [34] consists of an initial 7 \(\times\) 7 convolutional layer and four blocks. These blocks usually build by stacks of convolutional layers and pooling layers with similar structures. Sample image delivered to the CNN model will be operated by these components to produce features of various sizes. The feature produced by the first convolutional layer, whose semantic information is so lacking as too defy calculation, is out of consideration. Other features generated by blocks can all be utilized as features to be fused. Our deep feature fusion module precisely revolves around these four features. However, it is noteworthy that not all four features are necessary to attend feature fusion. The selection strategy involves a tradeoff with accuracy versus speed. Further analysis will be described herein in Section 4.4.2 through experiments. To enable feature fusion between high-level and low-level features and preserve spatial detail information of the low-level feature, we make the plane size of high-level features and low-level features identical using upsampling operation. There are three prevalent  
Figure 2: Proposed Deep Feature Fusion Model (DFFM) architecture. Components in leftmost dashed box belong to backbone model (e.g., ResNet34 in this figure). Components in the whole solid box comprise the model proposed in this paper. Numbers (H \(\times\) W \(\times\) C) in blocks indicate the output dimension. upsampling methods, i.e., deconvolution [35], reshape [36], and bilinear interpolation. Deconvolution brings learnable parameters, which improves the network performance but also increases the computational complexity. Reshape operation increases plane size at the expense of channels, which results in low computational complexity but poor performance. In contrast, bilinear interpolation increases plane size on the premise of keeping channels unchanged. It makes better tradeoff between performance and computational complexity, which is the most prevalent at present. The upsampling operation in this paper adopts this method and mainly acts it on high-level features. Followed by the upsampling of high-level features, dimension reduction (dim-reduce) operation is performed.