# DMA-Net: Decoupled Multi-Scale Attention for Few-Shot Object Detection
## 3 Proposed Methods
### Module Architecture
#### 3.2.1 Decoupled Multi-Scale Attention Module
Typically, the decoupling coefficient of the RPN is set near 0, and the gradient of the RCNN is given a high weight so that the gradient of the RCNN leads the update of backbone parameters, while the gradient from the RPN loss is used to optimize the module itself. To resolve the contradictions between the classification and regression tasks in the detection module, the two tasks were separated to construct two branches, allowing the support branch to participate in the classification task. Thus, the root cause of the two primary contradictions directly shared the same features, leading to the potential optimization problem. However, the proposed method avoided the aforementioned potential problems, as the RPN and RCNN did not share the same features directly in our model. The features for the RPN were extracted by the feature extractor and then passed through the attention module, while the features for RoI were extracted by the original feature extractor. This operation avoids the contradictions between the RPN and RCNN, but also prevents the loss of information after the features pass through the attention module. We also did not directly use the same features for the classification and regression tasks. For the regression task, we directly used the features combined with positive support features, while the proposed method for the classification task involved both positive and negative support features. This will be described in Section 3.2.2. Therefore, no decoupling operations were necessary for the forward propagation process. Nevertheless, the support branch in our model intersected before the RPN and in RCNN. The RPN is a class-agnostic module that primarily detects object locations, while the RCNN is a class-specific module that performs object classification. The support branch in our model was also class-specific. Therefore, there may have been a similar contradiction of Faster R-CNN between our support and query branches if the type of the gradient of RPN and that of the support branch were inconsistent. To prevent the contradictions, we added a decoupled gradient module (DGM) between the support and query branches before the RPN, and their gradients were decoupled to reduce the possibility of unnecessary information exchange during the process of optimization, which may have caused the entire network to fail to reach the optimal solution. As shown in Figure 6, during the forward propagation process, the proposed module does not perform any operation because the proposed features are not directly shared during forward propagation. During backward propagation, we decoupled the gradient from the RPN. We set a decoupling coefficient \(\lambda\), and the module obtained the gradient, multiplied it by \(\lambda\), and propagated it to the support branch. The gradient descent step after applying DGM can be defined as:  
\[\theta_{s}\leftarrow\theta_{s}-\gamma\cdot\mathbb{F}\Bigg{(}\lambda\cdot \frac{\partial\mathcal{L}_{rpn}^{q}}{\partial\theta_{rpn}^{s}}\Bigg{)}-\gamma \cdot\mathbb{R}\Bigg{(}\frac{\partial\mathcal{L}_{rcm}^{q}}{\partial\theta_{ rcm}^{s}}\Bigg{)}\,, \tag{5}\]  
where \(\theta_{s}\), \(\theta_{rpn}^{s}\), and \(\theta_{rcm}^{s}\) are learnable parameters; \(\gamma\) is the learning rate; the two partial derivatives indicate the gradients from the RPN and RCNN; and \(\mathbb{F}\) and \(\mathbb{R}\) represent the gradient propagation of the FPN and remodeling parts in the support branch, respectively. During training, \(\lambda\) should be set to 0 to stop the gradient, and the gradient descent step becomes:  
\[\theta_{s}\leftarrow\theta_{s}-\gamma\cdot\mathbb{R}\Bigg{(}\frac{\partial \mathcal{L}_{rcm}^{q}}{\partial\theta_{rcm}^{s}}\Bigg{)}\,. \tag{6}\]When the parameters in the support branch are updated, the gradients are always class-specific, which avoids the unnecessary exchange between class-specific and class-agnostic information during the optimization process. We provide the pseudo code of DGM in Algorithm 1.