# Boosting Noise Reduction Effect via Unsupervised Fine-Tuning Strategy
## 3 Methodology
### Supervised Preprocessing Stage  
Due to the remarkable capabilities of CNNs in learning generalized image priors from extensive data, CNN-based models have become prevalent in image restoration and related tasks. The UNet, a classic CNN model, is particularly renowned for its distinctive architecture--a U-shaped encoder-decoder structure with skip connections. This design facilitates efficient information flow across various resolution levels, enabling the model to capture both high-level semantic features and intricate details. The skip connections play a crucial role in fusing feature maps from the encoder and decoder pathways, contributing to the precise localization of objects in the image. Nevertheless, the convolution operation stands out as the core module, resulting in a relatively small receptive field for processing. Recently, another influential class of architectures, the Transformer, has demonstrated significant performance gains in both natural language and high-level vision tasks. In the supervised noise reduction stage, drawing inspiration from the U-shaped network in the DIP method [39; 52] and aiming to further enhance network performance, we opt to incorporate the Transformer block into the U-shaped network to preprocess the input noisy images. This strategic integration leverages the strengths of both UNet's architecture and Transformer's capabilities.  
Figure 2: The architecture of our two-stage denoising approach. It consists of two stages: supervised preprocessing and fine-tuned image generation. The objective of the first stage is to input a noisy image, \(y\), and generate a preprocessed image, \(x_{p}\), while the second stage is used to fine-tune this image and produce a better denoised image, \(x_{f}\), as the final output.  
In terms of specific implementation, our proposed image restoration approach involves the utilization of a U-shaped Transformer (U-Transformer) network, as illustrated in Figure 3. The U-shaped network architecture serves as the foundation of our model, where the convolution operation in each layer of the original UNet is substituted with a Transformer block. Within the Transformer block, improvements have been incorporated into both the multi-head self-attention (MSA) and feed-forward network (FFN) components. To enhance computational efficiency, our MSA module applies self-attention across channels rather than spatial dimensions. This approach computes feature cross-covariance within channels, generating a transposed attention map that implicitly encodes global context information. The chosen FFN promotes information flow across hierarchical stages, allowing each stage to focus on details that complement the MSA's capabilities. During the training phase, we employ the Adam optimizer and the L1 loss function for model optimization, defined as  
\[Loss_{1}=MAE=|x_{p}-x_{t}| \tag{6}\]  
where \(x_{p}\) represents the output image of the network and \(x_{t}\) represents the ground truth image.  
Figure 3: The architecture of the proposed U-Transformer. Our U-Transformer employs a U-shaped framework, with the Transformer block at its core. Each Transformer block comprises two sequentially executed multi-head self-attention (MSA) modules and feedforward neural network (FFN) modules. Within the MSA modules, deep convolutions are utilized to capture local context before computing cross-covariance across channels, thereby generating an attention map that implicitly encodes global context. The FFN modules consist of two branches: one is responsible for normal convolution feature extraction, while the other is tasked with generating gating weights. These weights are then element-wise multiplied with the features, and the resulting output is combined with a residual connection.