# Object Detection Network Based on Feature Fusion and Attention Mechanism
## 3 Methods
### Visual Attention Mechanism  
AF R-CNN uses the two parallel branches of the trunk and the attention branch to generate attention-aware features and then select regions on the feature map after adding an attention mask. Our attention modules were constructed by a mask branch. Given the feature map, \(T(x)\), as the input, the mask branch learns the mask, \(M(x)\) by using the bottom-up top-down structure [7; 33; 34; 35]. The mask, \(M(x)\), is used as control gates for \(T(x)\). The attention module \(H(x)\) is defined as:  
\[H_{i,c}(x)=M_{i,c}(x)\times T_{i,c}(x) \tag{2}\]  
where \(i\) ranges over all spatial positions and \(c\) represents channels.  
Figure 6 display the architecture of the mask branch. The mask branch contains two steps: Feed-forward sweep and top-down feedback. The two steps behave as a bottom-up top-down fully convolutional structure. The feed-forward sweep operation extracts global information quickly. Additionally, the top-down feedback operation feeds back the global information to the feature map. Firstly, max pooling was performed in the fusion feature map to improve the receptive field. The global information, which was expanded by the top-down structure, guides features when reaching the lowest resolution. The extended structure up samples global prior information by bilinear interpolation. To make the size of the output feature map the same as the input feature map, the output dimension of the bilinear interpolation was the same as the max pooling. Then, the attention module used the sigmoid layer to normalize the output results, with \(M(x)\). \(M(x)\) ranging from 0 to 1. This was the attention weight of the feature map at each location. This weight matrix, \(M(x)\), and the feature map matrix, \(T(x)\), were multiplied to obtain the desired attention-weighted feature map.  
The structure of this encoder-decoder (bottom-up top-down) is often used in image segmentation, such as FCN [7]. We applied a bottom-up top-down structure to the object detection net. The structure of bottom-up top-down first extracted high-level features and increased the receptive field of the model through a series of convolution and pooling. Pixels activated in the high-level feature map can reflect the area where attention is located. The size of the feature map was enlarged by the up sample to the same size as the original input, so that the area of the attention was mapped to each pixel of the input. We called this the attention map, (\(M(x)\)). Each pixel value in the attention map (\(M(x)\)) output by the soft mask branch was equivalent to the weight of each pixel value on the original feature map, which enhanced the meaningful features and suppressed meaningless information. This weight matrix, \(M(x)\), and the feature map matrix, \(T(x)\), were multiplied to obtain the desired attention-weighted feature map, (\(H(x)\)).  
Figure 5: Detailed process of feature fusion. We fused feature maps from the third, fourth, and fifth convolution layers(con_3, con_4, con_5).  
The object detection module shares the convolution parameters with the attention module to achieve an integrated end-to-end object detection network. The mask branch aims at improving features rather than solving complex problems directly. It worked as a feature selector, which enhances good features and suppress noises from features.