# BFE-Net: Bidirectional Multi-Scale Feature Enhancement for Small Object Detection
## 3 Method
### High-Level Guided Model  
In order to extract richer abstract features, the current general backbone networks such as ResNet [19] and VGG [31] have a deeper network structure; however, in the detection process, with the increase in depth, through continuous down sampling and feature extraction, the position information of the deep feature map is weakened, which is not conducive to target detection and localization.  
In the standard RetinaNet [11], in order to obtain more abstract features, P6 is obtained via a 3 \(\times\) 3 stride-2 conv on C5; P7 is computed by applying ReLU followed by a 3 \(\times\) 3 stride-2 conv on P6. This operation not only obtains rich semantic information but also loses a lot of location information. In order to make up for the loss of high-resolution information caused by too deep network layers in the standard RetinaNet, the High-level Guided Model was designed. Inspired by [30], this model uses the highest-level output feature of the backbone network as Query, and then uses the lower-level features as both Key and Value, respectively, and integrates the higher-resolution lower-level information into the highest-level features using the visual attributes of low-level pixels to render high-level concepts to achieve high-level and bottom-level feature fusion.  
High-level Guided Model (HGM) can be categorized as a low-level information enhancement transformer, which utilizes high-level features with rich category information for the weighted selection of low-level information. By selecting precise resolution details, the structure makes up for the high-resolution information lost in the propagation process and enriches the highest-level information.  
We denote the output of the convolutional layer in each scale as {C2, C3, C4, C5} according to the settings in [16]. Specifically, the highest-level C5 in the backbone network is defined as \(Q\), {C2, C3, C4} are used as \(K\) and \(V\) at the same time, respectively. We take C5 as \(Q\) and C4 as \(K\) and \(V\) as an example to describe the feature fusion process, as shown in Figure 6.  
Figure 5: Illustration of other structures of Dilated block: (**a**) series connection; (**b**) parallel connection. The dilated convolution of residual connections can be described as a recursive formulation.  
Figure 6: Illustration of High-level Guided Model (HGM).  
First, \(K\) uses global average pooling (GAP) to obtain the weighted value \(W\) and \(V\) uses a 3 \(\times\) 3 convolution kernel with step size \(S\) for down sampling to obtain \(V_{down}\), so that the size of a is the same as \(Q\). Second, \(Q\) uses the 1 \(\times\) 1 convolutional layer for dimension reduction according to the dimensions of \(K\). Then \(W\) weights \(Q\) to obtain \(Q_{att}\). Finally, \(Q_{att}\) and \(V\) perform element-wise addition to obtain M\({}_{4}\). In particular, we formulate this procedure as follows.  
\[\text{M}_{i}=Conv_{3\times 3}((Conv_{1\times 1}(Q)\otimes GAP(K))\otimes Conv _{3\times 3}(V),(i=2,3,4), \tag{5}\]  
where the symbol \(\otimes\) denotes feature fusion by matrix multiplication. \(Conv_{3\times 3}\) is a 3 \(\times\) 3 stride-2 conv to reduce the size of \(V\).  
We perform the above with C2, C3 as \(K\) and \(V\), and we obtain {M\({}_{2}\), M\({}_{3}\)}. Then, we perform the concatenate operation on {M\({}_{2}\), M\({}_{3}\), M\({}_{4}\)} to obtain M.  
\[\text{M}=\sum\
olimits_{i=2}^{4}F_{concat}(\text{M}_{i}), \tag{6}\]  
M and C5 are fused via the concatenate method, and then by 3 \(\times\) 3 stride-2 conv for down sampling, and finally, C6 is obtained after dimensionality reduction by a 1 \(\times\) 1 conv, as shown in Figure 3a.  
\[\text{C6}=Conv_{1\times 1}(Conv_{3\times 3}(F_{concat}(\text{M},\text{C5}))), \tag{7}\]