# Automatic Detection of Quasi-Periodic Emissions from Satellite Observations by Using DETR Method
## 3 Automatic Detection Method Development
### Improving Deformable Attention Mechanism  
This architecture demonstrates high accuracy in detecting overlapping targets during target detection. To further enhance the spatial localization, the deformable convolution technique is utilized [29]. Deformable convolution provides an effective means of emphasizing sparse spatial localization. In deformable DETR, the optimal sparse spatial sampling method of deformable convolution is combined with the relational modeling capability of the Transformer [30]. This combination not only accelerates the convergence of DETR but also reduces the complexity of the problem. In light of these advancements, we introduce the deformable attention mechanism into our model, aiming to improve the performance and efficiency of the target detection task. The original Transformer attention mechanism has a limitation in that it attends to all spatial locations within the feature map. In contrast, the deformable attention module addresses this limitation by selectively focusing on a small set of key sampling points near the reference point. This selection is independent of the spatial size of the feature map. In the deformable attention module, a set of query vectors is used to calculate K offsets. These offsets are then added to the reference point, resulting in K coordinates of the attention points. Notably, this approach avoids considering H (height) \(\times\) W (width) points across the entire feature map, thus significantly reducing the computational complexity (see Figure 7). Figure 7: Illustration of the proposed deformable attention module. For an input feature map \(x\in R\) (C \(\times\) H \(\times\) W), where C, H, and W represent the number of channels, height, and width, respectively, let q denote the index of a query element. \(Z_{q}\) represents the original feature on the feature map corresponding to the query element. The 2D reference point is denoted as \(p_{q}\). The computation of deformable attention features can be described by Equation (5). \[DeformAttn\big{(}Z_{q},p_{q},x\big{)}=\sum_{m=1}^{M}W_{m}[\sum_{k=1}^{K}A_{mpk }\cdot W_{m}^{\prime}x(p_{q}+\Delta p_{mpk})] \tag{5}\]  
\(M\) denotes the multi-head attention mechanism, representing the index of the attention head. The variable k denotes the sampling points, while \(K\) refers to the total number of sampling points (K \(<<\) HW). \(\Delta p_{mpk}\) represents the sampling offset of the kth sampling point in the \(m_{th}\) attention head, and \(A_{mpk}\) denotes the attention weight requiring normalization. Since \(p_{q}+\Delta p_{mpk}\) is a small value, bilinear interpolation is employed. The proposed model introduces the variable attention mechanism deformable attention to replace the global sampling utilized in the original encoder, aiming to address the slower and computationally expensive DETR detection challenge. Drawing inspiration from deformable models, this mechanism employs a sparse sampling strategy to streamline the computational requirements while capturing more intricate features. Simultaneously, we simplify the network structure by transitioning from the original multi-scale feature extraction method to extracting solely single-scale features, thereby lowering the computational costs without a notable decrease in accuracy. Furthermore, the implemented multi-layer attentional fusion mechanism in the encoder integrates residual connections at each layer to enhance the feature learning and improve the effectiveness of feature extraction. The encoder processes the input image by first resizing and normalizing it. The deformable attention module then selectively focuses on key sampling points near the reference points, and multi-layer fusion enhances the representational power of the encoder by integrating residual connections at each layer. In the decoder, the fused features are initially processed, refined through attention mechanisms, and further refined through multiple layers. The final stage generates the predicted bounding boxes and class labels for the objects in the image.