## 3 Methodology
### U-Former Deraining Subnetwork
#### 3.5.2 Stb  
As shown in Figure 7, each STB contains two submodules. The input \(F_{sf}\) asses through the first LayerNorm layer to acquire the normalized feature map \(F^{\prime}_{sf}\in R^{\frac{H}{2}\times\frac{W}{2}\times C}\), which then enters the STB for self-attention computation using W-MSA.  
\[\left\{\begin{array}{l}M_{i}=softmax(\frac{Q_{i}\cdot K^{T}_{i}}{\sqrt{A_{ k}}})\\ head_{i}=M_{i}\cdot V_{i}\end{array}\right. \tag{12}\]  
The feature map \(F^{\prime}_{sf}\) is linearly mapped to obtain \(Q\), \(K\), and \(V\), and based on the multi-head attention calculation described in Section 3.3.1, \(Q\), \(K\), and \(V\) are divided into \(h_{s}\) projection heads. Self-attention is computed for each projection space, as shown in Equation (12).  
The self-attention maps obtained from the \(s\) projection heads are concatenated using the Concate operation, followed by a LayerNorm layer and an MLP module. The features from different depths are fused via short connections to acquire the ultimate result of the first module, \(F^{\prime}_{out}\).  
The input to the second submodule of the STB is the output \(F^{\prime}_{out}\) from the first submodule. The processing steps in this submodule are similar to those in the first, but  
Figure 7: Structure of STB.  
when calculating window-based self-focus, putting emphasis on individual shifts in the window shifts the window to the lower-right to enable information exchange between different windows.  
The U-Former deraining subnetwork replaces the 3 \(\times\) 3 convolutional layers in the original UNet with STBs, offering several advantages:  
Enhanced feature representation: STBs outperform traditional 3 \(\times\) 3 convolutional layers in handling both local and global information. Traditional convolutional layers, with limited receptive fields, may fail to capture long-range dependencies, while STBs with window-based attention can more effectively integrate distant information. This improves feature extraction, capturing finer details and structural information in images, leading to clearer and more realistic image reconstruction results for deraining tasks.  
Improved long-range dependency and global information integration: rainy images are highly complex, requiring the model to handle long-range dependencies and integrate global information. Traditional 3 \(\times\) 3 convolutional layers may fail to capture distant correlations in the image. STBs, through self-attention and shifted window attention, better address long-range dependencies, effectively integrating global information, thus improving deraining performance.  
Reduced parameter count and computational complexity: although STBs are generally more complex than traditional convolutional layers, their attention-based nature allows for a decrease in the quantity of variables and computational complexity. This is crucial for efficiently processing large-scale image data in deraining tasks, as it balances high performance with improved processing speed.  
Modular design and scalability: STBs offer a strong modular design, making it easier to integrate and extend them within various network structures. By using STBs in the U-Former, we not only enhance feature extraction capabilities, but also make it easier to adjust and expand the model to address different image deraining tasks. In contrast, traditional convolutional layers, while simple, may not offer the flexibility required to adapt to complex network structures.  
In conclusion, applying the Swin Transformer module to the U-Former significantly improves the network's feature extraction capabilities while maintaining efficiency and flexibility, enabling the model to achieve higher performance in deraining tasks and better meet the demands of complex image processing challenges.