# A Triple Deep Image Prior Model for Image Denoising Based on Mixed Priors and Noise Learning
## 2 Related Work
### Deep Image Prior  
In general, inverse tasks such as image denoising can be expressed as energy minimization problems using the following formula:  
\[x^{*}=\underset{x}{min}\,E(x;x_{0})+R(x) \tag{6}\]  
where \(x_{0}\) the corrupted image, \(x\) the undistorted image, \(x^{*}\) the denoised image and \(R(x)\) a regularizer. The purpose of the \(E(x;x_{0})\) is to limit the search scope of the solution space. The regularizer \(R(x)\) encourages solutions to contain less noise and be more natural by using a generic prior. Unlike supervised models that learn priors from large amounts of data, [PERSON] et al. [34] suggested the prior can be sufficiently captured by a particular generator structure and interpreted the network as the following parametrization:  
\[x=f_{\theta}(z) \tag{7}\]  
where \(z\) is a tensor obeying a random distribution and the \(\theta\) denotes the network parameters. The implicit prior can supersede the regularizer \(R(x)\) in the following method:  
\[\theta^{*}=\underset{\theta}{argmin}\,E(f_{\theta}(z);x_{0}) \tag{8}\]  
Figure 1: The network architecture of Restormer.  
\[x^{*}=f_{\theta^{*}}(z) \tag{9}\]  
The specific implementation process is that the deep network model \(f_{\theta}\) continuously optimizes the network parameters \(\theta\) to minimize the objective function \(E\) of the generated image \(f_{\theta}(z)\) and the noisy image \(x_{0}\). Under a particular number of iterations, the generated image \(f_{\theta^{*}}(z)\) will fit most of the natural parts of the noisy image \(x_{0}\) before fitting the noisy parts. The best estimation of undistorted images is the output image \(x^{*}=f_{\theta^{*}}(z)\) produced from random noise \(z\) by ConvNets \(f\) with optimized parameters \(\theta^{*}\). Besides, DIP uses a U-net type encoder-decoder as network architecture, where the skip connection from the encoder layer to the corresponding decoder layer inhibits the loss of feature information in the network. Figure 2 depicts the network architecture of DIP.  
Regarding the original DIP denoising model, there are still some deficiencies such as single internal priors, insufficient constraints and a fixed number of stopped iterations, which make its overall denoising performance inferior to some classical denoising models.  
(1) Single internal priors: Although implicit priors are crucial in the detailed recovery of particular images, using only a single internal prior cannot learn the general information of most areas. Therefore, the overall denoising effect of DIP is inferior compared with the supervised model that learns from numerous image pairs. The single internal prior also contributes to the problem of image quality dependence. Because only internal priors are involved in the denoising process of the model, the denoising performance heavily depends on the quality of the target image.  
(2) Insufficient constraints: DIP uses only the MSE between the noisy image and the output image to guide the denoising process of the network, which may cause the network parameters to deviate from the correct solution space due to lack of constraints. Meanwhile, DIP ignores the constraints on noise learning. Constraints on the content learning alone are not enough to stably generate images with rich detail.  
(3) Fixed number of stopped iterations: The DIP demonstrates that an appropriate network architecture will converge faster for natural-looking images, while presenting significant inertia for noise. Therefore, the model's training requires a manual restriction of the number of iterations to prevent the model from overfitting noise. However, an exact and effective selection criterion was not provided in the original DIP.  
To highlight the strengths and novelty of the proposed method, we list the performance comparison of different DIP variants in Table 1.  
Figure 2: The network architecture of DIP.