# DMA-Net: Decoupled Multi-Scale Attention for Few-Shot Object Detection
## 3 Proposed Methods
### Module Architecture
#### 3.2.1 Decoupled Multi-Scale Attention Module
For {A2, A3, A4, A5, A6} at each layer, we assigned the anchors with areas of \(\left\{32^{2},64^{2},128^{2},256^{2},512^{2}\right\}\) pixels, and all used the multiple aspects ratios of \(\left\{1:2,1:1,2:1\right\}\), similarly to [19]. We also obtained 15 anchors in total. At the stage of filtering anchors, we reserved anchors as positive and negative proposals based on their intersection-over-union (IoU) ratios with a ground-truth bounding box. The strategy was similar to that used in [3]; we set the anchor with an IoU higher than 0.7 or with the highest IoU and the corresponding ground-truth bounding box as positive proposals and assigned the anchor with an IoU below 0.3 to all ground-truth bounding boxes to negative proposals. Based on the above operations, we were able to obtain an RPN with the capability to reserve more proposals related to the object than before. Decoupled Gradient Module  
In this paper, we conducted an analysis of the discrepancies between the proposed model and the original Faster R-CNN [3], and proposed decoupling between the support branch and the query branch to establish consistency in the type of gradient and in the updated parameters, thereby reducing the mutual interference between the two branches. The main contradictions of Faster R-CNN [3] are as follows. It consists of three primary modules for high-quality object detection tasks: RPN, RCNN, and a backbone shared by RPN and RCNN. The RPN module generated anchors through the features extracted by the backbone and filtered out proposals sent to the RCNN module for detection. Concurrently, the RPN screened out anchors as positive samples as well as negative samples that were used to calculate the loss to optimize the RPN. This process was class-agnostic, because the RPN filter proposals based on the foreground and background did not involve proposal categories. The RCNN module first mapped the proposals from the RPN to the features extracted by the backbone, and then produced a series of proposals of the same size for the detection module through the RoI alignment operation for further classification and bounding box regression of the objects. This process was class-specific, because the RCNN was obliged to perform classification for each proposal in order to find the categories corresponding to their class scores. According to the above details, the RPN and RoI alignment module used the features extracted by the backbone concurrently, but their operating process was different (class-agnostic and class-specific, respectively), as were their purposes: one told the network where to look, while the other told the network what to look at. However, Faster R-CNN [3] was an end-to end system that optimized all the modules jointly through the following objective function:  
\[\mathcal{L}_{\text{total}}\ =(\underbrace{\mathcal{L}_{\text{{rpn}}}^{cls}+ \mathcal{L}_{\text{{rpn}}}^{reg}}_{\text{{rpntask}}})+\eta\cdot(\underbrace{ \mathcal{L}_{\text{{rcnn}}}^{cls}+\mathcal{L}_{\text{{rcnn}}}^{reg}}_{\text{{ rnntask}}}), \tag{4}\]  
where \(\eta\) is a balanced hyperparameter for the _rpn_ and _rcnn_ tasks. Two detection tasks in the RCNN have contradictions in the joint optimization process (i.e., task classification requires  
Figure 5: Attention module. translation-invariant features, while task regression requires translation-covariant features). Therefore, there may be potential optimization problems if these contradictions are used in the scenarios of rare samples. To solve the contradictions of RPN, [46] used the decoupling module to decouple the RPN, RCNN, and backbone. An affine function was used during forward propagation so that the RPN and RCNN would not directly share features from the backbone. In the backward propagation stage, there was a decoupling coefficient multiplied by the gradient, which weighted the gradients from the RPN and RCNN.