# Boosting Noise Reduction Effect via Unsupervised Fine-Tuning Strategy
###### Abstract
However, the denoising performance of unsupervised models still exhibits a considerable gap compared to supervised models, assuming the availability of a large training dataset. Given that supervised, unsupervised, and self-supervised models have their advantages and disadvantages in noise reduction, a natural idea is to merge them to enhance the robustness and generalization ability of the noise reduction model in order to achieve better noise reduction results. In this paper, a novel two-stage denoising model, which enhances denoising performance through an unsupervised fine-tuning strategy, is proposed. More specifically, in the first stage we propose replacing the convolutional layers with Transformer modules in the UNet network framework of the DIP method, resulting in a new supervised denoising network (named U-Transformer) with stronger denoising performance. After the given noisy image undergoes processing by the supervised U-Transformer denoising model, the quality of the resulting denoised image (referred to as preprocessed image) is significantly improved. However, there is still slight residual noise and artifacts due to data bias, resulting in a small deviation of the image intensity from the original image. Aiming to achieve better denoising results, we introduce an extremely simplified network comprising only one Transformer module for fine-tuning the preprocessed image under the unsupervised training method. Experimental results show that our method achieves excellent denoising results in both synthetic and real-world scenarios and outperforms the current leading methods. The major contributions of this paper are as follows:  
1. By combining supervised and unsupervised methods, this study presents a holistic approach to address the image denoising problem, aiming to leverage the benefits of labeled data (the prior knowledge embedded within it) in the supervised method and the adaptive nature (fine-tuning capability) of the unsupervised method. Therefore, our combined approach enhances the robustness, generalization ability, and adaptability of the denoising model. 2. Our noise reduction model achieves plug-and-play capability. While the fine-tuning stage we use relies on the noise reduction performance of the supervised denoising models, in theory it is a relatively independent process. In practice, during the preprocessing stage, any supervised model can be used as a processing method for the given noisy images. This demonstrates the modularity of our approach, enabling us to apply the proposed fine-tuning strategy and combine it with any future advanced supervised models that may arise. As a result, we have the opportunity to obtain a noise reduction model with better noise reduction performance. 3. The efficiency of the unsupervised noise reduction stage has been enhanced. Utilizing the preprocessing model yields high-quality outcomes (i.e., preprocessed images), resulting in a small difference between the preprocessed image and the final denoised image. This subtle difference significantly reduces the difficulty of non-linear mapping in the fine-tuning network. Consequently, with only a minimalist network comprising one Transformer module, we can accomplish the fine-tuning task. This network possesses a significant advantage in the efficiency of network parameter updates, resulting in much less execution time required during the fine-tuning phase compared to conventional unsupervised fine-tuning methods. The paper is structured as follows: Section 2 offers an overview of the background knowledge and the DIP denoising model pertinent to our study. In Section 3, we elaborate on our networks employed in the two stages, detailing their structure and hybrid loss function, respectively. Section 4 presents the experimental dataset, settings, and results,comparing them with other state-of-the-art (SOTA) denoising algorithms. Lastly, Section 5 provides a comprehensive summary of our work.