# A Masked-Pre-Training-Based Fast Deep Image Prior Denoising Model
## 1 Introduction
Recently, Transformer models have gained prominence in image denoising [17]. The Google research team utilized the self-attention mechanism of the Transformer architecture to address the limitations of CNNs in global information modeling, resulting in an enhanced performance. Based on this, [PERSON] et al. introduced the SwinIR (Image Restoration Using Swin Transformer) denoising network [18]. SwinIR effectively learns the mapping from noisy images to clean images through the integration of shallow and deep feature extraction and advanced image reconstruction techniques. This network significantly improves the restoration of image details and reduces noise interference. Although SwinIR demonstrates superior denoising capabilities, it requires extensive training, particularly for extremely low-quality images. Furthermore, [PERSON] et al. introduced the Restormer (Restoration Transformer) denoising model [19], which has garnered attention due to its efficiency. This network with a U-shaped architecture employs the Multi-Dconv Head Transposed Attention (MDTA) and the Gated-Dconv Feed-forward Network (GDFN) to establish a novel Transformer module, enhancing performance. Restormer demonstrates high efficiency and advanced denoising capabilities, although it still demands substantial computational resources. In addition, the Uformer model [20] employed an encoder-decoder architecture to enhance the network's ability to capture local information through localized strengthened window Transformer modules. [PERSON] et al. introduced the Efficient Wavelet Transformer (EWT) [21] model, which combines the advantages of CNNs and Transformers through a specially designed dual-stream feature extraction block. EWT achieves comprehensive processing of different levels of image information. While EWT demonstrates a desirable balance between model performance, size, execution time, and GPU memory consumption, it underperforms in preserving image texture fidelity compared to some mainstream denoising models like SwinIR, particularly when handling specific noise levels and datasets. Additionally, [PERSON] et al. introduced the HCFormer model [22], extending the application of Transformer-based denoising models to low-dose CT image denoising. HCFormer incorporates a Neighborhood Feature Enhancement (NEF) module, replacing conventional Multiple-Layer Perceptron (MLP) layers, to efficiently extract channel-level features, thereby demonstrating superior performance in low-dose CT image denoising tasks. Overall, while supervised deep-learning-based denoising models offer significant advantages, their training requires extensive noisy image-clean image pairs, which can be challenging to assemble [23]. Moreover, if the noise distribution in test images significantly deviates from that in the training set, the denoising performance may degrade due to data bias issues, highlighting a limitation in generalization. In recent years, several methodologies have been developed to address the challenge of inadequate training datasets for supervised denoising models by utilizing noisy images alone, without the need for clean reference images (Ground Truth) during training. The self-supervised denoising model called Noise2 Noise [24], introduced by [PERSON] et al. in 2018, represents a significant breakthrough in this field. Although Noise2 Noise still requires two paired noisy images from the same scene for training, experimental results have shown that effective denoising is achievable solely through training with noisy images. Subsequently, [PERSON] et al. introduced Noise2 Void [25], which employs a blind-spot technique that allows training without paired noisy images, significantly reducing the data requirements for training denoising models. Similar methodologies are also employed within the Noise2 Self [26] framework. Despite achieving commendable denoising results, these models necessitate training with noisy images that display notable correlation with the observed noisy data.