SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Dataset and Implementation Details We evaluate the proposed method for color images in the three benchmark datasets containing RGB natural images: Kodak dataset, CBSD68 [15] and CSet9. DIV2K [22] and CBSD500 dataset [3] are used as training datasets. The synthetic noise images for each noise model are generated and fixed through the training process. For the sake of fair comparison, we use the same modified U-Net [5] for all methods. When training, we randomly clip the training images to patches with the resolution of  $128 \times 128$ . AdamW optimizer [14] is used to train the network. We train each model for 5000 steps with the batch size of 32. To reduce memory, we utilize the tricks of cumulative gradient and mixed precision training. The learning rate is initialized to  $1 \times 10^{-4}$  for first 4000 steps and it is decreased to  $1 \times 10^{-5}$  for final 1000 steps. All the models are implemented in PyTorch [18] with NVidia V100. The pixel value range of all clean images is [0, 255] and the parameters of noise models are built on it. Noisy images will be scaled when fed into the network. When an iterative algorithm is needed to solve Eq. 7, we set the number of iterations as 10. The more details of implementation are described in Supplementary Material.
Baseline and Comparison Methods We use supervised learning with MSE loss as the baseline model. Noisier2Noise and Neighbor2Neighbor are used as comparison methods. Since our approach
Table 1: The application range of different methods including supervised learning (SL), Noisier2Noise (Nr2N), Neighbor2Neighbor (Nb2Nb), Noise2Score (N2S) and ours.  $\checkmark$  means applicable and  $\times$  means not applicable or incapable to perform. For Neighbor2Neighbor,  $\checkmark^{*}$  means that direct application is not feasible but indirect application is;  $\times^{*}$  means that the application is not feasible but model training is executable.
No.	Noise Model	SL	Nr2N	Nb2Nb	N2S	Ours
1	y=x+ε,ε~N(0,σ2I)	✓	✓	✓	✓	✓
2	y=x+ε,ε~N(0,σ2A^T A)	✓	✓	×*	✓	✓
3	y=x+ε,ε~N(0,diag(ax+b1)^2)	✓	✓	✓	×	✓
4	y=x+ε,ε~N(0,A^T diag(ax+b1)^2 A)	✓	✓	×*	×	✓
5	y=η⊙x,ηi~G(α,α)	✓	×	✓	✓	✓
6	y=Aη⊙x,ηi~G(α,α)	✓	×	✓*	✓	✓
7	y=1/λη,ηi~P(λxi)	✓	×	✓	✓	✓
8	y=1/λAη,ηi~P(λxi)	✓	×	✓*	✓	✓
9	y=(η+1)⊙x,ηi~R(σ)	✓	×	×*	×	✓
10	y=A(η+1)⊙x,ηi~R(σ)	✓	×	×*	×	✓
11	y=η⊙x+ε,ηi~G(α,α),ε~N(0,σ^2I)	✓	×	✓	×	✓
12	y=Aη⊙x+ε,ηi~G(α,α),ε~N(0,σ^2I)	✓	×	×*	×	✓
13	y=1/λη+ε,ηi~P(λxi),ε~N(0,σ^2I)	✓	×	✓	×	✓
14	y=1/λAη+ε,ηi~P(λxi),ε~N(0,σ^2I)	✓	×	×*	×	✓
15	y=(η+1)⊙x,ηi~R(σ),ε~N(0,σ^2I)	✓	×	×*	×	✓
16	y=A(η+1)⊙x,ηi~R(σ),ε~N(0,σ^2I)	✓	×	×*	×	✓
is identical to Noise2Score when the noise model follows exponential family distributions, we do not compare to it through metrics. Because Noisier2Noise can only be applied to additive noise, we do not train models by Noisier2Noise for other noise models. Though Neighbor2Neighbor is not suitable for some noise models from the perspective of theoretical analysis, we still train corresponding models and report its results. Table 1 shows the comparison of application range for different methods, including additive Gaussian noise, multiplicative noise and mixture noise. Based on it our experiments are conducted. Only supervised learning and our approach can handle all noise models listed in Table 1.
Parameters of Noise Models Here, we emphasize that for all noise models in our experiments,  $A$  is set as a  $3 \times 3$  convolution transform with the kernel of
$$
\left( \begin{array}{l l l} 0. 0 5 & 0. 1 & 0. 0 5 \\ 0. 1 & 0. 4 & 0. 1 \\ 0. 0 5 & 0. 1 & 0. 0 5 \end{array} \right) \tag {19}
$$
if it is used. The additive Gaussian noise in every mixture noise model is set as  $\mathcal{N}(\mathbf{0},100\mathbf{I})$ . Other parameters will be given later. Finally, all parameters are assumed to be known in our experiments.
Additive Gaussian Noise Using additive Gaussian noise, we consider four kinds of noise models with different  $\pmb{\Sigma}$  corresponding from No.1 to No.4 in Table 1. Our method is compared to supervised learning, Noisier2Noise and Neighbor2Neighbor as shown in Table 2. For the first two noise models  $\sigma$  is 25, and for the rest  $a = 0.98$  and  $b = 25$ . As expected, supervised learning performs best. In the cases without  $\mathbf{A}$  Neighbor2Neighbor is the best among three other unsupervised learning methods. However, in the cases with  $\mathbf{A}$  Neighbor2Neighbor performs very poorly. Our approach outperforms other unsupervised learning methods in the second noise model and is competitive in the whole.
Multiplicative Noise We consider the combination of three various multiplicative noise model (Gamma, Poisson and Rayleigh) and a convolution transform  $A$ . They are corresponding from No.5 to No.10 in Table 1. Our method is compared to supervised learning and Neighbor2Neighbor and the results are shown in Table 3. Because Noisier2Noise can not address such multiplicative noise models, we neglect it. Though the noise is not pixel-wise independent when the convolution transform exists,
Table 2: Quantitative comparison for various parameters of  $\Sigma$  in additive Gaussian noise using different methods in terms of PNSR (dB)/SSIM. Bold indicates the best result among three unsupervised methods, while underlined indicates the second-best result.
Method	No.1: Gaussian, σ = 25 w/o A	No.2: Gaussian, σ = 25 w/ A
Kodak	CSet9	CBSD68	Kodak	CSet9	CBSD68
SL	32.44/0.887	30.27/0.891	31.32/0.892	34.80/0.928	32.53/0.924	34.08/0.938
Nr2N	31.80/0.863	29.75/0.878	30.84/0.871	33.87/0.914	32.02/0.916	33.40/0.926
Nb2Nb	31.96/0.875	29.90/0.882	30.92/0.880	27.89/0.673	27.88/0.742	27.86/0.722
Ours	31.92/0.870	29.86/0.875	30.91/0.877	33.99/0.914	32.16/0.914	33.45/0.926
No.3: Gaussian, a = 0.98, b = 25 w/o A	No.4: Gaussian, a = 0.98, b = 25 w/ A
Method	Kodak	CSet9	CBSD68	Kodak	CSet9	CBSD68
SL	30.92/0.856	28.69/0.860	29.68/0.856	33.02/0.904	30.76/0.900	32.11/0.912
Nr2N	30.07/0.813	28.02/0.834	28.99/0.818	32.28/0.884	29.98/0.886	31.55/0.896
Nb2Nb	30.42/0.840	28.22/0.848	29.29/0.842	25.02/0.557	24.44/0.621	25.01/0.619
Ours	29.68/0.797	27.58/0.806	28.70/0.806	31.88/0.884	29.50/0.874	31.14/0.895
Table 3: Quantitative comparison for various multiplicative noise models using different methods in terms of PNSR (dB)/SSIM. For Neighbor2Neighbor (Nb2Nb), if the noise model is constructed with  $\mathbf{A}$ , it can be regarded as the one without  $\mathbf{A}$  through  $\mathbf{A}^{-1}\mathbf{y}$ . Thus we do not provide the metrics. Bold indicates the better result between two unsupervised methods.
Method	No.5: Gamma, α = 26 w/o A	No.6: Gamma, α = 26 w/ A
Kodak	CSet9	CBSD68	Kodak	CSet9	CBSD68
SL	33.51/0.916	30.72/0.898	32.44/0.922	33.02/0.916	30.41/0.897	32.15/0.921
Nb2Nb	32.98/0.908	30.33/0.890	31.97/0.913	-	-	-
Ours	32.61/0.894	29.89/0.870	31.51/0.898	31.90/0.877	29.26/0.858	30.63/0.878
No.7: Poisson, λ = 0.2 w/o A	No.8: Poisson, λ = 0.2 w/ A
Method	Kodak	CSet9	CBSD68	Kodak	CSet9	CBSD68
SL	32.90/0.902	30.56/0.894	31.87/0.908	32.55/0.902	30.27/0.894	31.64/0.907
Nb2Nb	32.50/0.893	30.17/0.886	31.48/0.899	-	-	-
Ours	32.38/0.886	29.98/0.874	31.31/0.891	31.84/0.872	29.48/0.864	30.56/0.873
No.9: Rayleigh, σ = 0.3 w/o A	No.10: Rayleigh, σ = 0.3 w/ A
Method	Kodak	CSet9	CBSD68	Kodak	CSet9	CBSD68
SL	35.29/0.939	32.44/0.922	34.39/0.947	34.63/0.939	31.97/0.920	33.94/0.946
Nb2Nb	16.55/0.865	15.16/0.844	16.74/0.862	-	-	-
Ours	34.25/0.915	31.45/0.892	33.34/0.923	32.87/0.894	30.30/0.869	31.85/0.901
we can execute its inverse transform on  $\pmb{y}$  so that the requirement of pixel-wise independence is satisfied for Neighbor2Neighbor. Therefore, tackling noise models with a convolution transform is equivalent to the situations without  $\pmb{A}$ . That is why we do not provide corresponding metrics result for Neighbor2Neighbor in Table 3. We set  $\alpha$  as 26 for the Gamma noise,  $\lambda$  as 0.2 for the Poisson noise, and  $\sigma$  as 0.3 for the Rayleigh noise. When the noise model is based on Gamma or Poisson noise, it is unbiased, i.e.  $\mathbb{E}[y\mid x] = x$ . In these cases, Neighbor2Neighbor is better than ours. However, when the noise model is based on Rayleigh noise which is biased our approach still has excellent performance while Neighbor2Neighbor is poor.
Mixture Noise We also consider the combination of three various multiplicative noise model (Gamma, Poisson and Rayleigh) and a convolution transform  $\mathbf{A}$ . For each one, additive Gaussian noise with  $\Sigma = 100I$  is added to construct mixture noise models. They are corresponding from No.11 to No.16 in Table 1. Because of the same reason discussed before, our method is compared to supervised learning and Neighbor2Neighbor, and Noisier2Noise is neglected. The experimental
Table 4: Quantitative comparison for various mixture noise models using different methods in terms of PNSR (dB)/SSIM. For each noise model, additive Gaussian noise with  $\Sigma = 100I$  is added. Bold indicates the better result between two unsupervised methods.
Method	No.11: Gamma, α = 26 w/o A	No.12: Gamma, α = 26 w/ A
Kodak	CSet9	CBSD68	Kodak	CSet9	CBSD68
SL	32.86/0.902	30.23/0.889	31.70/0.906	33.02/0.882	29.30/0.875	30.42/0.882
Nb2Nb	32.30/0.892	29.80/0.880	31.21/0.896	26.93/0.676	25.06/0.663	26.33/0.710
Ours	32.13/0.880	29.61/0.861	31.08/0.887	30.74/0.853	28.40/0.841	29.58/0.854
Method	No.13: Poisson, λ = 0.2 w/o A	No.14: Poisson, λ = 0.2 w/ A
Kodak	CSet9	CBSD68	Kodak	CSet9	CBSD68
SL	32.49/0.892	30.18/0.887	31.40/0.897	31.44/0.877	29.36/0.876	30.32/0.878
Nb2Nb	32.03/0.884	29.74/0.880	30.98/0.888	26.82/0.646	25.48/0.669	26.32/0.688
Ours	32.10/0.879	29.70/0.870	31.03/0.883	31.11/0.855	28.80/0.851	29.85/0.856
Method	No.15: Rayleigh, σ = 0.3 w/o A	No.16: Rayleigh, σ = 0.3 w/ A
Kodak	CSet9	CBSD68	Kodak	CSet9	CBSD68
SL	34.38/0.926	31.79/0.913	33.39/0.933	32.83/0.908	30.48/0.898	31.74/0.910
Nb2Nb	16.54/0.856	15.15/0.838	16.71/0.851	16.40/0.704	15.06/0.738	16.56/0.736
Ours	33.54/0.902	30.94/0.883	32.68/0.913	31.14/0.867	28.94/0.847	30.23/0.876
Figure 2: PSNR V.S. disturbance rate for No.14 and No.15 noise models.
results are shown in Table 4. Due to the additive Gaussian Noise, Neighbor2Neighbor are not able to handle the cases with  $\mathbf{A}$  through the inverse convolution transform. The correlation of noise hampers the performance of Neigh2bor2Neighbor. Except the first noise model in Table 4, our approach all outperforms Neighbor2Neighbor and achieve excellent performance near to supervised learning.
Robustness Evaluation To apply our approach, the parameters of noise models have to be known beforehand. Therefore their precision may impact on the performance. We choose No.14 and No.15 noise models in Table 1 as examples to show the robustness to parameters' precision. Suppose  $k$  is one of parameters, we disturb it by  $(a + 1)k$  where  $a \sim \mathcal{N}(0, r^2)$  and  $r$  is called the disturbance rate. The larger  $r$ , the less precise the noise model. As  $r$  is increasing, the PSNR of the denoised result is shown in Figure 2. When  $r = 0.1$ , the PSNR reduces about 1 dB, which displays the robustness.
In this paper, we propose a new approach for unsupervised image denoising. The key part is Proposition 3.1. Based on it, we construct an equation, Eq. 7 about the clean image  $x$  and the noisy image  $y$ . After the score function of  $y$  is estimated, the denoised result can be obtained by solving the equation. Our approach can be applied to many different noise models as long as Eq. 7 is solvable. The denoising performance is competitive for simple noise models and excellent for complicated ones. We hope that this work is helpful to address sophisticated image denoising problems in practice.
Figure 3: Qualitative Comparison using CBSD68 dataset (cropped to  $256 \times 256$ ). From the first row to the last: (1) Gaussian noise,  $\sigma = 25$  w/o  $A$ ; (2) Gaussian noise,  $\sigma = 25$  w/  $A$ ; (3) Rayleigh noise,  $\sigma = 0.3$  w/o  $A$ ; (4) Rayleigh noise,  $\sigma = 0.3$  w/  $A$ ; (5) Poisson noise,  $\lambda = 0.2$  w/  $A$  added by Gaussian Noise with  $\sigma = 10$ ; (6) Rayleigh noise,  $\sigma = 0.3$  w/o  $A$  added by Gaussian Noise with  $\sigma = 10$ . Noisy: noisy image, GT: ground-truth image, SL: supervised learning, Nb2Nb: Neighbor2Neighbor, Nr2N: Noisier2Noise.
[18] Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. Automatic differentiation in pytorch. 2017.
[19] Yuhui Quan, Mingqin Chen, Tongyao Pang, and Hui Ji. Self2self with dropout: Learning self-supervised denoising from single image. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 1890–1898, 2020.
[20] Sathish Ramani, Thierry Blu, and Michael Unser. Monte-carlo sure: A black-box optimization of regularization parameters for general denoising algorithms. IEEE Transactions on image processing, 17(9):1540-1554, 2008.
[21] H Robbins. An empirical bayes approach to statistics, proceedings of the third berkeley symposium on mathematical statistics and probability, 1956.
[22] Radu Timofte, Shuhang Gu, Jiqing Wu, and Luc Van Gool. Ntire 2018 challenge on single image super-resolution: Methods and results. In Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pages 852-863, 2018.
[23] Pascal Vincent. A connection between score matching and denoising autoencoders. Neural computation, 23(7):1661-1674, 2011.
[24] Xiaohe Wu, Ming Liu, Yue Cao, Dongwei Ren, and Wangmeng Zuo. Unpaired learning of deep image denoising. In European conference on computer vision, pages 352-368. Springer, 2020.
[25] Jun Xu, Yuan Huang, Ming-Ming Cheng, Li Liu, Fan Zhu, Zhou Xu, and Ling Shao. Noisy-as-clean: Learning self-supervised denoising from corrupted image. IEEE Transactions on Image Processing, 29:9316-9329, 2020.
[26] Kai Zhang, Wangmeng Zuo, Yunjin Chen, Deyu Meng, and Lei Zhang. Beyond a gaussian denoiser: Residual learning of deep cnn for image denoising. IEEE transactions on image processing, 26(7):3142-3155, 2017.