## 3 Method
### Parallel Attention
#### 3.2.2 Proposed Parallel Attention
The role of the 1 \(\times\) 1 convolution here is mainly to adjust the number of feature channels to match the subsequent depth-wise convolution (DWConv), which decomposes standard convolution into depthwise convolution and pointwise convolution, reducing the computational load and the number of parameters while maintaining the model's expressive power. Then, the features, after depthwise convolution, are reshaped into an appropriate form for subsequent attention calculations. The reshape operation here converts the two-dimensional feature map into a one-dimensional feature vector to match the query (Q), key (K), and value (V) vectors in the attention mechanism. After obtaining Q and K, they are fed into a dot product operation to calculate the similarity between them. The result of the dot product is passed through a softmax function for normalization to obtain the attention weight matrix. The attention weight matrix represents the correlation between different positions, where positions with higher weights have a greater influence on the current position. Finally, the attention weight matrix is dot-multiplied with the value vector V to obtain the final output features. Specifically, for the input feature X, the phase extraction module extracts the phase information \(\Phi_{q}\) and \(\Phi_{k}\). Then, \(\Phi_{q}\) and \(\Phi_{k}\) are processed through \(1\times 1\) convolution and depthwise convolution to obtain Q and K. Meanwhile, the input feature X is also processed to obtain V. The dot product result of Q and K is normalized by softmax and then multiplied with V to obtain the final output feature. The formulas are  
\[Q=\text{DWConv}(1\times 1\text{Conv}(\Phi_{q})), \tag{8}\]  
\[K=\text{DWConv}(1\times 1\text{Conv}(\Phi_{k})), \tag{9}\]  
\[V=\text{DWConv}(1\times 1\text{Conv}(X)), \tag{10}\]  
\[A=\text{softmax}\bigg{(}\frac{QK^{T}}{\sqrt{d_{k}}}\bigg{)}, \tag{11}\]  
Figure 3: The architecture of proposed phase-aware multi-head self-attention. \[\text{Output}=AV, \tag{12}\]  
where \(d_{k}\) is the dimension of Q and K, used to scale the dot product result to prevent numerical overflow and gradient vanishing. In this way, the phase-aware multi-head self-attention module can effectively capture the phase information in the input features and incorporate it into the attention calculation, thereby enhancing the model's sensitivity and processing capability for different frequency information. Additionally, this module also features a multi-head mechanism, where the input features are divided into multiple heads, and each head independently performs the aforementioned phase-aware attention calculation. The outputs of all heads are then concatenated and projected through a fully connected layer to obtain the final output features. The multi-head mechanism enhances the model's expressive power and its ability to model different feature subspaces. In summary, the phase-aware multi-head self-attention module significantly improves the model's ability to understand and process input features by introducing phase information and a multi-head mechanism, making it suitable for various tasks that require capturing complex feature relationships.