# A Triple Deep Image Prior Model for Image Denoising Based on Mixed Priors and Noise Learning
## 3 Methodology
### Basic Idea
The other strategy first predicts the noise and then obtains the final clean image by removing the predicted noise from the noisy image:  
\[I_{Content}=I_{Noisy}-I_{Noise}=I_{Noisy}-f_{noise}(I_{Noisy}) \tag{11}\]  
where \(I_{Noise}\) denotes the learned noise, and \(f_{noise}\) indicates the noise generator such as DIP2 or DIP3. Both learning strategies have their own strengths, content learning demonstrate more robust noise suppression effectiveness [43], and noise learning is beneficial to prevent model degradation and preserve the structure [41]. Considering these theories, we propose a TripleDIP denoising strategy to learn content and noise separately. Note that the final output image is generated only by the DIP branch which learns the image content. Meanwhile, the other two DIP branches for learning noise independently are used to constrain the model's denoising direction and provide more exquisite image restoration details. Specifically, assuming that \(I_{Noise1}\) and \(I_{Noise2}\) denote the learned noise from DIP2 and DIP3, respectively, the theoretical representation of two-branch independent noise learning is as follows:  
\[I_{Noisy}\approx I_{Content}+I_{Noise1} \tag{12}\]  
Equation (12) demonstrates the sum of the generated content image \(I_{Content}\) and the learned noise \(I_{Noise1}\) should approximate the inputted noisy image \(I_{Noisy}\). \[I_{Remainder}=I_{Noise1}-I_{Noise2}\approx 0 \tag{13}\]  
\[I_{Extenal}\approx I_{Content}+I_{Remainder} \tag{14}\]  
where the \(I_{Remainder}\) indicates the difference between the learned noise from two branches of DIP, and the \(I_{Extenal}\) represents the external prior image generated by Restormer. There is a constraint such that the two independently-generated noises should be similar and, additionally, the sum of \(I_{Remainder}\) and the content image \(I_{Content}\) should approach the preprocessed image \(I_{Extenal}\). Thus, TripleDIP can exploit noise learning and external prior images to perform the denoising task. The added constraints also enrich the noise details that the model can learn. Besides, two-branch independent noise learning makes the generation process of the clear image more stable and efficient. Therefore, the TripleDIP's denoising performance can be significantly improved over the original DIP model. (3) Automatic stop criterion based on SSIM;  
In a standard DIP, the fixed number of stopped iterations makes the generated solution likely to deviate from the natural image and approach the target image, thus resulting in overfitting problem. To obtain the output results with higher image quality in model training, we propose an automatic stop criterion based on SSIM. Specifically, we select the SSIM value between the network output \(I_{Content}\) and the preprocessed image \(I_{Extenal}\) to evaluate the model's training quality. The method can reflect the model training effect more accurately because the preprocessed image contains more natural parts. We then set a search range (i.e., 50 steps) for each iteration. If the SSIM value of an iteration is the maximum value within the search range, it can be considered that the model's SSIM value will not increase in the subsequent training process. Therefore, the number of iterations with the maximum SSIM is selected to terminate the model's iterative process and generate the denoised image. To validate the automatic stop criterion, we perform experiments on Lena images with noise level of 25 on the Set12 dataset and record the value of loss function and PSNR for each iteration. For a more intuitive representation, we enlarge the loss function value by a factor of 1000 and plot them in Figure 3. As seen from Figure 3, by using automatic stopping criterion, the model stops training at 6819 iteration steps, at which point the PSNR value was 33.31 dB. During training, the maximum PSNR value is 33.46 dB, and the corresponding number of iterations is 8463 steps.