# DMA-Net: Decoupled Multi-Scale Attention for Few-Shot Object Detection
## 3 Proposed Methods
### Module Architecture
#### 3.2.1 Decoupled Multi-Scale Attention Module
Subsequently, we applied nearest-neighbor upsampling to upscale the features of the upper layers and merged these upsampled features with the channel dimension-aligned feature maps in the corresponding layers using elementwise addition. In this iteration, we obtained P5 after simply reducing the channel dimension, and P6 by performing a max-pooling on P5. We also used a \(3\times 3\) convolutional layer after each merged feature map to mitigate the effect of upsampling. The final set of features [P2, P3, P4, P5, P6] was obtained by the above operations. If we were to send this feature set into the RPN for proposal generation directly, the RPN would lose contact with support information. It is critical to establish a relationship with support information and to do so prior to the RPN. It is also convenient  
Figure 3: Decoupled Multi-scale Attention Module (DMAM). to use an attention operation. Constructing support features for use as weights and acting on the query features gives the positions related to objects high weight, and effectively transfers the attention of the RPN to a position related to the objects. In [28; 36], single-scale support features were used to transfer the attention, which is inappropriate in the case of multi-scale query features, and many problems regarding feature size must be considered. According to the above description, we applied an FPN after positive support features to extract multi-scale support features [S2, S3, S4, S5, S6]. At all levels of the FPN, shallow features were rich in space information and deep features were rich in semantic information; thus, we used both shallow features and deep features of the support images. We also considered spatial information and semantic information together, which can alleviate the problems surrounding spatial information in FSOD tasks. After extracting the features of the positive support images and query images, we performed an attention operation layer to layer to achieve effective fusion and to maximize the use of support information. In this paper, we used a simple attention module to construct the relation between the query and support branch prior to the use of the RPN. We added FPN to both the positive support branch and the query branch, and there were five layers of features with different scales in each branch. Therefore, we were obligated to add attention layer by layer. Constructing a relation scale to scale can also reduce the influence between different scale features. There were five attention modules in five layers, which represented the same feature fusion method but had different feature sizes. As shown in Figure 5, query features and positive support features were fed into the attention module in each layer. We denoted query features as \(Q\in t^{H\times W\times C}\) and positive support features as \(S\in t^{M\times M\times C}\). We fused them using a depth-wise convolution method, processed the positive support features as a convolution kernel, and then used this kernel to perform sliding computing on the query features. The final attention feature map was obtained after this operation. The depth-wise convolution can be defined as:  
\[A_{h,w,c}=\sum_{i,j}S_{i,j,c}.Q_{h+i-1,w+j-1,c},i,j\in\{1,\dots,M\}, \tag{2}\]  
where \(A\) is the final attention feature, \(S\) and \(Q\) represent features from the support branch and the query branch respectively. In this study, we used global average pooling on positive support features, which is defined as:  
\[S^{\prime}_{1,1,c}=\frac{1}{M\times M}\sum_{h,w}S_{h,w,c},h,w\in\{1,\dots,M\}, \tag{3}\]  
Figure 4: Multi-scale Feature Extractor. Therefore, the kernel size generated by \(S\in t^{S\times S\times C}\) was processed to \(1\times 1\times C\). Using \(1\times 1\) convolution can reduce the burden of the parameters, spatial clutter, and the consumption of query information itself. We used one \(3\times 3\) conv and two sibling \(1\times 1\) convs on the final attention feature to generate proposals for each corresponding layer.