# Physics-Based Noise Modeling and Deep Learning for Denoising Permanently Shadowed Lunar Images
## 3 Methods
### REWin Transformer Block  
Due to the limitations of traditional Transformers in multi-scale information extraction and local information processing [37; 38; 39; 40], we propose the REWin Transformer module, which aims to reduce redundancy, refine multi-scale information, and further enhance the refined local features, thus achieving superior image denoising performance. The REWin Transformer module integrates the self-attention mechanism to capture global information, while utilizing a feedforward neural network to extract and learn key local contextual information. The module consists of three core components: SW-MSA, GFRN, and GCEN. Figure 7 illustrates the specific structure of this module. When the input feature map is passed through the REWin Transformer module, its computational process can be expressed as follows:  
\[ \begin{array}{l}X_{l}^{\prime}=SW\text{-}MSA(LN(X_{l}))+X_{l}\\ X_{l}^{\prime\prime}=GFRN\big{(}LN\big{(}X_{l}^{\prime}\big{)}\big{)}+X_{l}^{ \prime}\\ X_{l}^{\prime\prime\prime}=SW\text{-}MSA\big{(}LN\big{(}X_{l}^{\prime\prime }\big{)}\big{)}+X_{l}^{\prime\prime}\\ X_{l}^{\prime\prime\prime\prime}=GCLN\big{(}LN\big{(}X_{l}^{\prime\prime \prime}\big{)}\big{)}+X_{l}^{\prime\prime\prime}\end{array}  \tag{5}\]  
The three key modules, SW-MSA, GFRN, and GCEN, will be introduced separately below. Their specific structures are shown in Figures 8-10, respectively. Figure 8: The structure of the SW-MSA. Figure 10: The structure of the GCEN. Figure 7: The structure of the REWin block. Figure 9: The structure of the GW-MSA. Firstly, SW-MSA performs self-attention within non-overlapping local windows. Compared to global self-attention, SW-MSA significantly reduces computational cost. Given a two-dimensional input feature \(X\in R^{C\times H\times W}\), SW-MSA initially divides X into non-overlapping \(M\times M\) blocks. Subsequently, the features of each window are flattened and transposed, resulting in \(X^{i}\in R^{M^{2}\times C}\) for each window. Self-attention is then applied to the features within each window. The calculation procedure for the k-th head is described in Equation (6) [19]. \[ \begin{gathered} X=\big{\{}X^{1},X^{2},\cdots,X^{N}\big{\}}\\ N=\frac{HW}{M^{2}}\\ Y^{i}_{k}=Attention\Big{(}X^{i}W^{Q}_{k},X^{i}W^{K}_{k},X^{i}W^{V}_{k} \Big{)},i=1,\cdots,N\\ \hat{X}_{k}=\big{\{}Y^{1}_{k},Y^{2}_{k},\cdots,Y^{N}_{k}\big{\}} \end{gathered}  \tag{6}\]  
\(W^{Q}_{k},W^{K}_{k}\),and \(W^{V}_{k}\in\mathbb{R}^{C\times d_{k}}\) are the projection matrices for Q, K, and V. Subsequently, the outputs from all heads are merged and linearly projected to produce the ultimate output of SW-MSA. As per earlier research [41,42], relative position encoding is integrated into this module. Consequently, the self-attention calculation can be represented by Equation (7) [19]. \[Attention(Q,K,V)=Softmax\Big{(}\frac{QK^{T}}{\sqrt{d_{k}}}+B\Big{)}V \tag{7}\]  
Here, B represents the relative position bias, which comes with learnable parameters [41,42]. In the process of enhancing feature representation through the self-attention mechanism, conventional Feedforward neural networks (FFN) [38] play a key role in independently processing pixel position information. However, the direct use of multi-scale information may introduce redundancy, which can affect the effectiveness of the features. Therefore, designing an efficient FFN to enhance feature representation is crucial for improving image recovery performance. When using SW-MSA as the basic module for feature extraction, redundant information that interferes with noise removal often exists in the feature map. To address this issue, we introduce GFRN to simplify and optimize feature transformation. Specifically, GFRN is constructed by incorporating PConv [43] and DWConv [44], which are used to reduce redundant computations and optimize feature information. Additionally, a gating mechanism is introduced to further alleviate the processing burden of redundant information. GFRN can be expressed by the following equation:  
\[