Unsupervised Image Denoising with Score Function
Authors: Yutong Xie
Mingze Yuan
Bin Dong
Quanzheng Li
Venue: NeurIPS 2023 poster (2023)
Keywords: unsupervised learning
image denoising
score function

Abstract: Though achieving excellent performance in some cases, current unsupervised learning methods for single image denoising usually have constraints in applications. In this paper, we propose a new approach which is more general and applicable to complicated noise models. Utilizing the property of score function, the gradient of logarithmic probability, we define a solving system for denoising. Once the score function of noisy images has been estimated, the denoised result can be obtained through the solving system. Our approach can be applied to multiple noise models, such as the mixture of multiplicative and additive noise combined with structured correlation. Experimental results show that our method is comparable when the noise model is simple, and has good performance in complicated cases where other methods are not applicable or perform poorly.

Unsupervised Image Denoising with Score Function
Yutong Xie
Mingze Yuan Peking University mzyuanšpku. edu cn
Peking University
yutongxie__essercchll.o
Bin Dong
Quanzheng Li
Peking University
Massachusetts General Hospital and Harvard Medical School
dongbin@bicmr pku. edu.
1i quanzhengemgghh...mm harvard.edi
Abstract
Though achieving excellent performance in cases, current unsupervised learn- ing methods for single image denoising usually have constraints in applications. this paper, propose new approach which is more general and applicable complicated noise models. Utilizing the property of score function, the gradient logarithmic probability, we define solving system for denoising. Once the score function of noisy images has been estimated, the denoised result can be obtained through the solving system. Our approach can be applied multiple noise models, such as the mixture of multiplicative and additive noise combined with structured correlation. Experimental results show that our method is comparable when the noise model is simple, and has good performance in complicated cases where other methods are applicable or perform poorly.
Introduction
has been studied for many years. Suppose æ is a clean image, y is a noisy image of æ; and p (y a) is the noise model. Supervised learning methods try train model representing mapping from y to æ. Due to the difficulty of collecting paired clean and noisy images in practice, methods in unsupervised learning manner focus of research. Noise2Noise [12] is the first to use pairs of two different noisy images constructed from the same clean image to train denoising model. Strictly speaking, Noise2Noise noisy pairs is still difficult. Despite all this, many other methods 22 10 24 25 are inspired from Noise2Noise or borrow the idea behind of it. These methods achieve good performance in some simple noise models.
Image denoising 4 20 26
an unsupervised learning method. Collecting
The main drawback of current methods is the constraint of application. Once the noise model is complicated, they either are not applicable or have poor performance. Noisier2Noise [16] can only handle additive noise and pepper noise. Recorrupted- to- Recorruppee (17) is limited to Gaussian noise. Neighbor2Neighborr 77 requires that the noise model is pixel-wise independent and unbiased (i.e. [y æ] = æ). Noise2Score g applies Tweedie's Formula to image denoising and provides a unified framework for those noise models that follow exponential family distributions. However, the practical noise model will be more complicated. It may contain both multiplicative and additive noise, even combining with structural correlation.
In this paper, we propose new unified approach handle more noise models. The key of our ap- proach is thcoretical property about score function, V, log p(y), which is shown in Proposition 3.1 This property indicates that the score function of the average of the score function of y under the posterior distribution p (x: y). Based on it, we define a system that the score function of y equals the score function of æ,, which turns out to be equation about since y is known. We
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

estimate
.(0)
x(y) loggp(())))
solve equation
function
noisy image
denoised result
Figure 1: The overall flow of our approach. The first step is to estimate the score function syy) and the second step is to solve an equation.
discover that the solution of this equation can be regarded the denoised result of y. Implementing our approach practically contains two steps: the first is to estimate the score function of y and the second is to solve the system defined above according the specific noise model. The overall flow is shown in Figure []] All the model training is the estimation of score function, which is represented by neural network. We adapt the amortized residual denoising autoencoder (AR-DAE) [3 to figure out it. More details can be seen in Section 3.3
Our approach is so powerful that long the score function of y is known and the system solvable, any kind of noise model is applicable. It means our approach is even able to solve sophisticated noise models such mixture noise. Another advantage of our approach is that regardless of noise models, the training process of the score function neural network is identical, Therefore, once the assumption of the noise model does not hold or parameters of noise corrected, we only change the equation system to be solved and resolve it without training the model again. In summary, our main contribution are: (I) We propose general unsupervised approach for image denoising, which is based on the score function. (2) Experimental results show that our approach competitive for simple noise models and achieves excellent performance for complicated noise models where other unsupervised methods is invalid.
Related Works
Here we briefly review the existing deep learning methods for image denoising. When the pairs of clean images and noisy images are available, the supervised training [26] is to minimize Eu.y [d (xx 0)), where f:: is neural network and d (-:-) is distance metrie. Though supervised learning has great performance, the difficult acquisition for training data hampers its
application in practice.
To avoid the issues of acquisition for paired clean and noisy images, unsupervised anssuprrisedaappoacchess proposed to use noisy image pairs to learn image denoising, which is started from Noise2Noise (N2N) [2]. While N2N can achieve comparable results with supervised methods, collecting noisy image pairs from real world is still intracTable Motivated by N2N, the followed-up works try to learn image denoising with individual noisy images. Mask-based unsupervised approaches [ design mask schemes for denoising individual noisy images and then they train the network to predict the masked pixels according to noisy pixels in the input receptive field. Noise2 100 also proposes
approaches are
the blind-spot network (BSN) avoid learning the identity function. Noisier2Noise [16]. Noisy-As-
Clean (25) and Recorrupted-to-RR Reeer-t---Rerrrpteee [17] require a single noisy realization of each training sample and a statistical model of the noise distribution. Noisier2Noise first generates a synthetic noisy sample from the statistical noise model, adds it to the already noisy image, and asks the network to predict the original noisy image from the doubly noisy image. Besides the blind-spot network design, Self2Self [19] is proposed on blind denoising to generate paired data from single noisy image by applying Bernoulli dropout. Recently, Neighbor2Neighbor 7 proposes to create subsampled paired images based on the pixel-wise independent noise assumption and then a denoising network is trained on generated pairs, with additional regularization loss for better performance. Noise2Score [9] is another type of unsupervised methods that can deal with any noise model which follows an exponential family distribution. It utilize Tweedie's Formula 21 and the estimation of score function to denoise noisy images.
Sometimes self-super used.

---

3 Method
In this section, provide more detailed description of the proposed approach. The organization as follows: In Section 3.1 we review Noise2Score, introduce the basic theory and present the relationship between method and Noise2Score. After that, derive the specific algorithms for different examples of noise models in Section [32] Finally, we describe the method for estimating the score function V, logp (y) in Section 3.3 All the proofs are in Supplementary Material.
3.1 Basic Theory
3.1.1 Noise2Score
Suppose the noise model follows some exponential family distribution and has the following form:
p (3 x) (y) exp [[(x) (B) - a (a=)
(1)
where T (3) and (z) have the same dimensions, and a (a) and b (y) are both scalar functions. One of the properties of exponential family distribution is that:
H' (z) [ (g) æ) = a a'))..
(2)
According to Bayesian Formula, we have the following derivation:
(x y) (xx) (y x) p (x) -=(=)
exp T (T)) H (x) + log
(v) p
(3)
Therefore, p (a: y) is also exponential family distribution. According to Eq. conclusion about V, log (y) (the score function of y). Tweedie's Formula, that
2
we can derive
Vy log (y) V log (y) + T' (y) E E!H (x) y
(4)
In Noise2Score, the denoised result è of is the solution of following equation:
Vy log (y) v, log (b)) + T (v) H (i)
Since Noise2Score is derived from Tweedie's 's Formula, it is limited to the noise models of exponential family distributions.
3.1.2 Proposed Approach
In this work, the proposed approach generalized Noise2Score to non-exponential family distributions. We begin with the following proposition.
Proposition 3.1 Let y ~ y and x ~ x where and y are two random variable, then the equation below holds:
V logp(y) = (ax y) V, log (y x)) dæ
The proof Proposition 3.1
is in Supplementary Material. the task of image denoising, suppose Rd is the noisy image and æ E TRd is the clean image where d represents the number of image pixels. p (ar y) represents the posterior distribution and p (y æ)) denotes the noise model. When p (y æ) follows some exponential family distribution, it is easy to verify that the right part of Eq, equals to the right part of Tweedie's 's Formula (Eq. Therefore, the equation (Eq. 5 to solved in Noise2Score in essence can be written as:
Vy log (v) = V. logp(y .).
We use è rather than æ to indicate that + the unknown variable to be solved in the equation. Hence, propose a new unsupervised denoising method that given noisy image y, the denoised result can obtained by solving Eq. 7 When the noise model follows some exponential family distribution, our method is the same as Noise2Score. The specific conclusions of Gaussian, Gamma and Poisson noise can be seen Supplementary Material. However, different from Noise2Score, our approach is applicable to some complex non-exponential family distributions as long the score function y and noise model (y x>) are known. For convenience, let s (ÿ) = V y log p (y) and The general
(ar, y) V, log p (y w). Our approach is consist of two steps shown in Figure 1
algorithm is illustrated in Algorithm[]
The following theorem explains why the solution of Eq. is good denoised result of y.

---

Algorithm The general denoising process
Input:
noisy image y and f (xx, y).
Output: Σ, the solution of Eq. Estimate s sz).
2: Solve s ( = f y).
Theorem 3.1 Given y, suppose the Hessian matrix f (z. y) is bounded, f (a,, y) is invertible with respect to x, and inverse function is Lipschitz continuous. Denote & as the solution of Eq. / then,
c - Euiy [x]]||2 I C'Tr (Cov [xe y)) o (Tr (Cov (x yl))
(8)
where Cov [ y is the covariance matrix and C a constant.
The proof of Theorem 3.1 is in Supplementary Material. Theorem 3.1 shows the lower noise level, the smaller the error between à and E [z:]. which guarantees the effectiveness of proposed method.
Next, in Section 3.2 will derive the specific algorithm for solving in Eq. under different noise models, including additive Gaussian noise in Section [32.1]] multiplicative noise in Section 3.2.2 and mixture noise in Section 3.2.3 Estimating score function (y) will be introduced in Section 3.3
3.2 Application Examples
3.2.1 Additive Gaussian Noise
Suppose the noise model y = x + e where e follows variable Gaussian distribution with mean of 0 and covariance matrix of Σ denoted by N (0, E), i.e.
exp
x)
Σ
(
xx)
p (y x)
22T d
Σ
Then, we derive that f (x, y) = Vy log = - (y x). We consider the following four kinds of Σ: (1) Σ o²1: (2) Σ = of (3) Ex) Σ (z)
diag (ax b1);: (4)
diag (aæ b1)² A. In the second and fourth cases, A usually represents convolution transform which describes the correlation between adjacent pixels.
For the first and second cases, Σ is constant matrix. By solving Eq. 7 have
a Es (y) + y-
(10)
While in the third and fourth cases, related to and Eq. is fixed point equation. Therefore, we use an iterative trick to solve it as shown in Algorithm 2
Algorithm 2 An iterative trick to solve Σ (x) 8 (y) + y
Input: noisy image y. (y), the parameters of Σ (-) and the number of iterations Output: &; the solution.
1: Initial value of z is set y.
2: for i = 1..... n do
3: t - E { y.
4: end for
3.2.2 Multiplicative Noise
Firstly, discuss three types of multiplicative noise model, Gamma, Poisson and Rayleigh noise. Then, we consider the situation where the convolution transform exists.
Gamma Noise constructed from Gamma distribution, & (a, 3): p(æ; a, 3) -aa=" -- and
defined as y = ② x, where 7/2 & (a, at), a > 1, and @ means component-wiss multiplication, i.e.
a
yk
a-1
03/i
p ( ②))
r (a)
J;
exp
JF;
(11)

---

Then, we derive that f (2, y) = Vy (y
x)) a=1 Here, the division is component-wise
division. In the residual part of this paper, we neglect such annotation if it is not ambiguous. By solving Eq. 7 we have
a
(12)
a -
Poisson Noise is constructed from Poisson distribution, P (A): Pr(x = k)) x k 1, and is defined y 1, T) P (Aæ;), À > 0, i.e.
Pr Pr x)) = II (Dr,,) Auyi (Aw,)) c Am;
a
(13)
i=1
Then, we derive that f (a. Eq. 7 we have
V, log Pr (y ax)
Alog (Aar) - Alog (>y + 22) By solving
1 c y
exp { ( (y)
X $
(14)
22
x² Rayleigh Noise is constructed from Rayleigh distribution, R (o): p(x; = a exp { defined as y (n + 1) ① x. lBs R (a). o > 0, i.e.
-
2n f. and is
1 yo 24 T2 2202
Ut x.
p x)) =
exp
(15)
2x 0²
Then, derive that (ax, y) V logp (y
Here we provide an iterative algorithm solve it. is illustrated in Algorithm directly is not easy.
d² Solving Eq.
y- x
Algorithm 3 An iterative method solve Eq. 7 in the of Rayleigh noise
Input: noisy image y. s (y), the parameter of Rayleigh noise o and the number of iterations n. Output: &, the solution of Eq.
1: Initial value & set as y.
2: for i = .... do
② at 4: Compute = (-b b
3: Compute = a² 8 (y)
5: è < y
t+1-
6: end for
Now, we consider the situation where the convolution transform A exists. Suppose the noise model represented by y Az, N (æ), where N (x) be any multiplicative noise model discussed above. Then, have V, log Py (y x) A A-1.T V_ log Pz (A-'y æ). To avoid confusion, use subscripts to distinguish different distribution. Therefore, we can apply Algorithm 4 to solve Eq. which is shown as follows.
Algorithm 4 The general framework to solve Eq. for correlated multiplicative noise model
Input: y. (y), A and f (x, z) Vz log (z x).
Output: &, the solution of Eq. 7
1: Computing 3 A 8 (y).
2: Computing z A -
3: Solve ă = f (x, =) by the corresponding Algorithm
3.2.3 Mixture Noise
In this paper, the mixture noise model is composed of multiplicative noise and an additive Gaussian noise. We denote it as y -
N (0, 2 x AN (æ), where N (æ)) is any multiplicative
5

---

noise model that can be solved by our approach and A is either convolution transform or iden- tity matrix. It is easy to derive that Py (y z) = Py (y 2) Pz (z x) dz. Generally speaking, Py (y æ) has not an explicit analytical form. In this paper, we assume that the additive Gaussian noise is far smaller than the multiplicative noise. Thus, we utilize Taylor expansion to approximate Py (y æ). We have the following conclusion:
Py (y x) R Pz (#
x) + VxP2 (# x)" (y- x),
(16)
where & = E [ y/- Then, we can further derive that
Vy log Py (y x)) R V= log Pz (= x)
(17)
The full and rigorous derivations of Eq. 16 and Eq. 17 are in Supplementary Material. Applying
Eq. in Section 3.2.1 we have & = y + t The full denoising process is illustrated in Algorithm 5
s (zy) Thus, the equation to be solve is (y) = f (2, 2).
Algorithm 5 The full denoising process for mixture noise y 2 +
Input: y. s (y).
Output: á., the solution of (y) (xx, z). I: Computing ž- = + o²g (y).
2: Solve s (y) = f (z, 2) through the corresponding algorithm discussed in Section 3.2.2
3.3 Estimation of Score Function
So far, assume that the score function of y, (y), is known. However, it is usually unknown and should be estimated from the dataset of noisy images {y). We use the same method, the amortized residual Denoising Auto Encoder (AR-DAE) [3] discussed in Noise2Score. Suppose (:: ())) is
neural network used to represent the score function of y. The following objective function is used to train the model:
I. = -= (y,,.~V(0.1)) lu + 0.8 (y + 0)
(18)
where da is a fixed value. Given da, the optimal model s(y; 8*) that minimizes L is the score function of perturbed y. y + u. In other words, we can approximate the score function (y) by using sufficiently small oa. Related analysis can also be seen in [8 23 [] During the training process, the value of a. will be decreasing gradually to a very small value. This progressive process helpful to numerically stabilize the model training. The only model training is estimate s (y) by (:; (), which is served as the first step of our approach. After the score function model is trained, we apply the denoising algorithms given in Section 3.1 to obtain denoised results and no more training required.
4 Experiment
Dataset and Implementation Details We evaluate the proposed method for color images in the three benchmark datasets containing RGB natural images: Kodak dataset, CBSD68 [15] and CSet9. DIV2K (22 and CBSD500 dataset 3 are used as training datasets. The synthetic noise images for each noise model are generated and fixed through the training process. For the sake of fair comparison, we use the same modified U-Net (5) for all methods. When training, we randomly clip the training images patches with the resolution of 128 128. AdamW optimizer [4 is used to train the network. We train each model for 5000 steps with the batch size of 32. To reduce memory, we utilize the tricks of cumulative gradient and mixed precision training. The learning rate is initialized to 1 x 10 for first 4000 steps and it is decreased to 1 X 10 for final 1000 steps. All the models are implemented in Py Torch (18 with NVidia V100. The pixel value range of all clean images is [o, 255] and the parameters of noise models are built on it. Noisy images will be scaled when fed into the network. When an iterative algorithm is needed to solve Eq. 7 we set the number of iterations as 10. The more details of implementation are described in Supplementary Material.
Baseline and Comparison Methods We use supervised learning with MSE loss as the baseline model. Noisier2Noise and Neighbor2Neighbor are used as comparison methods. Since our approach

---

Table 1: The application range of different methods including supervised learning (SL), Noisier2Noise (Nr2N), Neighbor2Neighhor (Nb2Nb), Noise2Score (N2S) and ours. v means applicable and x means not applicable incapable to perform. For Neighbor2Neighbor, v means that direct application is not feasible but indirect application is; means that the application is not feasible but model training is executable.
No.
Noise Model
SL
Nr2N
Nb2Nb
N2S
Ours
y = E, N (0.021)
2
y
N
0, A A)
3
y + ~ N
0, diag (ax b1)2)
4
y = x + 6, ~ N
0, A diag (ax + b1)² A)
X
5
y 1 0 Z. 9 (a,
6
y = An 10 (a,a)
7
y 77 ~P
8
y An,
9
y 011 (()))
10
y = A
11
y = 77 T (0,
12 y Ano (0,
13
y =
14 y
Px N(0,0011
15 y = 1n-1@mm=RRo
16 y = O (11)
is identical to Noise2Score when the noise model follows exponential family distributions, do not compare to it through metries. Because Noisier2Noise can only be applied to additive noise, do not train models by Noisier2Noise for other noise models. Though Neighbor2Neighbor not suitable for some noise models from the perspective of theoretical analysis, we still train corresponding models and report its results. Table|[]shows the comparison of application range for different methods, including additive Gaussian noise, multiplicative noise and mixture noise. Based on it our experiments conducted. Only supervised learning and our approach handle all noise models listed in Table
Parameters of Noise Models Here, emphasize that for all noise models in experiments, A set as a 3 x 3 convolution transform with the kernel of
0.05
0.1
0.05
0.1
0.4
0.1
(19)
0.05
0.1
0.05
it is used. The additive Gaussian noise in every mixture noise model set as / (0, 100T). Other parameters will be given later. Finally, all parameters are assumed to be known in our experiments.
Additive Gaussian Noise Using additive Gaussian noise, we consider four kinds of noise models with different Σ corresponding from No.1 to No.4 in Table | Our method is compared to supervised learning, Noisier2Noise and Neighbor2Neighbor as shown in Table For the first two noise models is 25, and for the rest 0.98 and 25. As expected, supervised learning performs best. In the the best among three other unsupervised learning methods. performs very poorly. Our approach outperform
cases without Neighbor2Neighhoo However, in the cases with A Neighbor2Neighte other unsupervised learning methods in the second noise model and competitive the whole.
We consider the combination of three various multiplicative noise model (Gamma, Poisson and Rayleigh) and a convolution transform A. They are corresponding from No.5 to No. 10 in Table [ Our method is compared to supervised learning and Neighbor2Neighbor and the results are shown in Table Because Noisier2Noise can not address such multiplicative noise models, neglect it. Though the noise is not pixel-wise independent when the convolution transform exists,
Multiplicative Noise
7

---

Table 2: Quantitative comparison for various parameters of Σ in additive Gaussian noise using differ- ent methods in terms of PNSR (dB)/SSIM. Bold indicates the best result among three unsupervised methods, while underlined indicates the second- best result.
No.1: Gaussian, 0 = 25 w/o A
No.2: Gaussian, 0 = 25 w/
Method
Kodak
CSe19
CBSD68
Kodak
CSet9
CBSD68
SL
32.4410.887 887
30.27/0.89
31.3210.892
34.80/0.99 928
32.53/0.922
34.08/0. 938
Nr2N
31.80/0.863
29.75/0.878
30.84/0.871
33.87/0.914
32.02/0.916
33.40/0.926
Nb2Nb
31.96/0.875
29.90/0.882
30.92/0.880
27.89/0.673
27.88/0.744 742
27.86/0.722
Ours
31.92/0.870
29.86/0.875
30.91/0.877
33.99/0.914
32.16/0.914
33.45/0.926
No.3: Gaussian, a = 0.98, b 25 w/o A
No.4: Gaussian, a = 0.98, = 25 w/ A
Method
Kodak
CSet9
CBSD68
Kodak
CSet9
CBSD68
SL
30.9210.850
28.69/0.660
29.68/0.856
23.02/0.904
30.76/0.900
32.1110.912
Nr2N
30.07/0.813
28.02/0.834
28,99/0.818
32.28/0.884
29.98/0.886
31.55/0.896
Nb2Nb
30.42/0.840
28.22/0.848
29.29/0.842
25.02/0.557
24.44/0.621
25.01/0.619
Ours
29.68/0.797
27.58/0.806
28.70/0.806
31.88/0.884
29.50/0.874
31.14/0.895
Table 3: Quantitative comparison for various multiplicative noise models using different methods in terms of PNSR (dBySSIM. For Neighbor2Neighhor (Nb2Nb), if the noise model is constructed with
A. it can be regarded as the one without A through A A Thus we do not provide the metrics. Bold indicates the better result between two unsupervised methods.
5o Gamma, a w/o A
No.6: Gamma, a 26 A
Method
Kodak
CSet9
CBSD68
Kodak
CSet9
CBSD68
SL
33.51/0.916 30.72/0.898
32.4410.922
33.0210.91 916 30.4110.897
32.1510.921
Nb2Nb
32.98/0.908 30.33/0.890
31.97/0.913
Ours
32.61/0.894
29.89/0.870
31.51/0.898
31.90/0.877 29.26/0.858
30.63/0.878
No.7: Poisson, À 0.2 w/o A
No.8: Poisson, À = 0.2 w/ A
Method
Kodak
CSet9
CBSD68
Kodak
CSet9
CBSD68
SL
32.9010.900 30.56/0.894
31.8710.90 908
32.55/0.900
30.2710.894
31.6410.907
Nb2Nb
32.50/0.893 30.17/0.886
31.48/0.899
Ours
32.38/0.886 29.98/0.874
31.31/0.891
31.84/0.872 29.48/0.864 30.56/0.873
No.9: Rayleigh, a = 0.3 w/o A
No. Rayleigh, o = 0.3 w/ A
Method
Kodak
CSet9
CBSD68
Kodak
CSe19
CBSD68
SL
35.2910.939 32.44/0.922
34.39/0.947
34.6310.939
31.9710.920 33.9410.946
Nb2Nb
16.55/0.865
15.16/0.844
16.74/0.861
Ours
34.25/0.915
31.45/0.892
33.34/0.923
32.87/0.894
30.30/0.869 31.85/0.901
we can execute its inverse transform on y so that the requirement of pixel-wise independence is satisfied for Neighbor2Neighbor. Therefore, tackling noise models with a convolution transform is equivalent to the situations without A. That is why we do not provide corresponding metrics result for in Table We set 26 for the Gamma noise, À as 0.2 for the Poisson noise, and a as 0.3 for the Rayleigh noise. When the noise model is based on Gamma or Poisson noise, it is unbiased, i.e. f/ ar when the noise model is based on Rayleigh noise which is biased our approach still has excellent performance while Neighbor2Neighbor is poor.
Neighbor2Neighben
In these cases, Neighbor2Neighbor is better than ours. However,
Mixture Noise We also consider the combination of three various multiplicative noise model (Gamma, Poisson and Rayleigh) and a convolution transform A. For each one, additive Gaussian noise with Σ = 100/ is added to construct mixture noise models. They are corresponding from No. 11 No. in Table Because of the same reason discussed before, our method compared to supervised learning and Neighbor2Neighhor, and Noisier2Noise is neglected. The experimental

---

Table 4: Quantitative comparison for various mixture noise models using different methods in terins of PNSR (dB)/SSI.. For each noise model, additive Gaussian noise with 100I is added. Bold indicates the better result between two unsupervised methods.
No.11: Gamma, a 26 w/o A
12: Gamma, a = 26 w/ A
Method
Kodak
CSet9
CBSD68
Kodak
CSe19
CBSD68
SL
32.86/0.900 30.23/0.88
31.70/0.900
33.0210.882
29.3010.875
30.4210.882
Nb2Nb
32.30/0.892 29.80/0.880
31.21/0.896
26.93/0.676
25.06/0.665
26.33/0.71L 710
Ours
32.13/0.880 29.61/0.861
31.08/0.887
30.74/0.853
28.40/0.841
29.58/0.854
No.13: Poisson, À 0.2 w/o A
No.14: Poisson, X 0.2 A
Method
Kodak
CSet9
CBSD68
Kodak
CSet9
CBSD68
SL.
32.49/0.892
30. 8870.887 31.4010.897
31.4410.877
29.3610.876
30.3210.776
Nb2Nb
32.03/0.884
29.74/0.880
30.98/0.888
26.82/0.646
25.48/0.669
26.32/0.688
Ours
32.10/0.879
29.70/0.870
31.03/0.883
31.11/0.855
28.80/0.851
29.85/0.856
No.15: Rayleigh, = 0.3 w/o A
1o. Rayleigh, a 0.3 w/ A
Method
Kodak
CSet9
CBSD68
Kodak
CSet9
CBSD68
SL
34.3810.926
31.79/0.913
33.39/0.933
32.83/0. 908 30.48/0.898
31.7410.910
Nb2Nb
16.54/0.856
15.15/0.838
16.71/0.851
16.40/0.704
15.06/0.738
16.56/0.736
Ours
33.54/0.902
30.94/0.883
32.68/0.913
31.14/0.867
28.94/0.847
30.23/0.876
33.5
33.0
32.5
32.0
31.5
31.0
30.5
Poisson Conv Gaussian Rayleigh Gaussian
30.0
0.00
0.02
0.04
0.06
0.08
0.10
Desurbance Rate
Figure 2: PSNR V.S. disturbance rate for 1 and No. noise models.
results are shown in Table 4 Due to the additive Gaussian Noise, Neighbor2Neighbor are not able to handle the cases with A through the inverse convolution transform. The correlation of noise hampers the performance of Neighhbor2Neighhoor Except the first noise model in Table 4 our approach all outperforms Neighbor2Neighbor and achieve excellent performance near to supervised learning.
Robustness Evaluation To apply our approach, the parameters of noise models have to be known beforehand. Therefore their precision may impact on the performance. We choose No. 14 and No. 15 noise models in Table a examples to show the robustness to parameters' precision. Suppose k is one of parameters, we disturb it by (a + 1)k where a ~ (00 r²) and r is called the disturbance rate. The larger the less precise the noise model. As is increasing, the PSNR of the denoised result is When r = 0.1, the PSNR reduces about dB, which displays the robustness.
shown in Figure 2
Conclusion
In this paper, we propose a new approach for unsupervised image denoising. The key part is Proposition 3.1, Based on it, we construct an equation, Eq. aboout the clean image and the noisy image y. After the score function of y is estimated, the denoised result can be obtained by solving the equation. Our approach can be applied to many different noise model as long as Eq. is solvable. The denoising performance competitive for simple noise models and excellent for complicated ones. We hope that this work is helpful to address sophisticated image denoising problems in practice.

---

Noisy
GT
SL
Nb2Nb
Ours
Noisy
SL
Nr2N
Ours
Noisy
GT
SL
Nb2Nb
Ours
Noisy
SL
Nb2Nb
Ours
Noisy
Nb2Nb
Ours
Noisy
NG2ND
Oars
Figure 3: Qualitative Comparison using CBSD68 dataset (cropped to 256 x 256). From the first
row the last: (1) Gaussian noise, = 25 w/o A; (2) Gaussian noise, 25 w/ A; (3) Rayleigh noise, o 0.3 w/o A; (4) Rayleigh noise, = 0.3 w/ A: (5) Poisson noise, = 0.2 w/ A added by Gaussian Noise with 10; (6) Rayleigh noise,
0.3 w/o A added by Gaussian Noise with a = 10. Noisy: noisy image, GT: ground-truth image, SL. supervised learning, Nb2Nb: Neighbor2Neighbor, Nr2N: Noise
10

---

References
[1] Guillaume Alain and Yoshua Bengio. What regularized auto-encoders learn from the data- generating distribution. The Journal of Machine Learning Research, ((1)::363--355 2014.
[2] Joshua Batson and Loic Royer. Noise2self: Blind denoising by self-supervision. In International Conference on Machine Learning, pages 524-533. PMLR, 2019.
[3] Lalit Chaudhary and Yogesh Yogesh. A comparative study of fruit defect segmentation techniques. In 2019 International Conference on Issues and Challenges in Intelligent Computing Techniques (ICICT), volume pages 1-4. IEEE, 2019.
[4] Kostadin Dabov, Alessandro Foi, Vladimir Katkovnik, and Karen Egiazarian. Image denoising with block-matching and 3d filtering. In Image processing: algorithms and systems, neural networks, and machine learning, volume 6064, pages 354-365. SPIE, 2006.
[5] Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. Advances in Neural Information Processing Systems, 34:8780-8794, 2021.
[6] Bradley Efron. Tweedie's formula and selection bias. Journal of the American Statistical Association, 106(4966:1002-1164) 2011.
[7] Tao Huang, Songjiang Li, Xu Jia, Huchuan Lu, and Jianzhuang Liu. Neighbornneighhor: Self- supervised denoising from single noisy images. In Proceedings of the IEEE/CVE conference on computer vision and pattern recognition, pages 14781-14790, 2021.
[8] Aapo Hyvärinen and Peter Dayan. Estimation of normaazzel statistical models by score matching. Journal of Machine Learning Research, 6(4), 2005.
[9] Kwanyoung Kim and Jong Chul Ye. Noise2score: tweedie's s approach to self-supervised image denoising without clean images. Advances in Neural Information Processing Systems, 34:864-874, 2021.
[10] Alexander Krull, Tim-Oliver Buchholz, and Florian Jug. Noise2void-learning denoising from single noisy images. In Proceedings of the IEEF/CVF conference on computer vision and pattern recognition, pages 2129-2137, 2019.
[11] Alexander Krull, Tomáš Vičar, Mangal Prakash, Manan Lalit, and Florian Jug. Probabilistic noise2void: Unsupervised content-awar denoising. Frontiers in Computer Science, 2:5, 2020.
[12] Jaakko Lehtinen, Jacob Munkberg, Jon Hasselgren, Samuli Laine, Tero Karras, Miik Aittala, and Timo Aila. Noise2noise: Learning image restoration without clean data. In International Conference on Machine Learning, page 2965-2974. PMLR, 2018.
[13] Jae Hyun Lim, Aaron Courville, Christopher Pal, and Chin-Wat Huang. Ar-dae: towards unbiased neural entropy gradient estimation. In International Conference on Machine Learning. pages 6061-6071 PMLR, 2020.
[14] Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. arXiv preprint 171 2017.
[15] David Martin, Charless Fowlkes, Doron Tal, and Jitendra Malik. A database of human segmented natural images and its application evaluating segmentation algorithms and measuring ecological statistics. In Proceedings Eighth IEEF International Conference on Computer Vision. ICCV 2001, volume 2, pages 416-423. IEFE, 2001.
[16] Nick Moran, Dan Schmidt, Yu Zhong, and Patrick Coady. Noisier2noise: Learning to denoise from unpaired noisy data. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 12064-12072, 2020.
[17] Tongyao Pang, Huan Zheng, Yuhui Quan, and Hui Ji. Recorupted-- unsupervised deep learning for image denoising. In Proceedings of the IEEF/CVF conference on computer vision and pattern recognition, pages 2043-2052, 2021.

---

[18] Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang. Zachary Viiioo, Zeming Lin. Alban Desmaison, Luca Antiga, and Adam Lerer. Automatic differentiation in pytorch. 2017.
[19] Yuhui Quan, Mingqin Chen, Tongyao Pang, and Hui Ji. Self2self with dropout: Learning self-supervised denoising from single image. In Proceedings of the IEEE/CVE conference on computer vision and pattern recognition, pages 1890-1898, 2020.
[20] Sathish Ramani, Thierry Blu, and Michael Unser. Monte-carlo sure: A black-box optimization of regularization parameters for general denoising algorithms. JEEE Transactions on image processing, 170911550-1154, 2008
[21] Robbins. An empirical bayes approach statistics, proceedings of the third berkeley symposium on mathematical statistics and probability, 1956.
[22] Radu Timofte, Shuhang Gu, Jiqing Wu, and Luc Van Gool. Ntire 2018 challenge on single image super-resolution: Methods and results. In Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pages 852-863, 2018.
[23] Pascal Vincent. A connection between score matching and denoising autoencoders. Neural computation, 23(7):661-1167, 2011.
[24] Xiaohe Wu, Ming Liu, Yue Cao, Dongwei Ren, and Wangmeng Zuo. Unpaired learning of deep image denoising. In European conference computer vision, pages 352-368. Springer, 2020.
[25] Jun Xu, Yuan Huang, Ming-Ming Cheng. Liu, Fan Zhu, Zhou Xu, and Ling Shao. Noisy-as- clean: Learning supervuue denoising from corrupted image. IEEE Transactions on Image Processing, 29:9316 9329, 2020.
[26] Kai Zhang, Wangmeng Zuo, Yunjin Chen, Deyu Meng, and Lei Zhang. Beyond gaussian denoiser: Residual learning of deep cnn for image denoising. IEEE transactions image processing, 26(7):312-33155, 2017.
12


Reviews:
[{"review_id": "JgyrOU1nQR", "reviewer": "Reviewer_Qfmm", "summary": "This paper proposes a unsupervised image denoising method based on score function. Compared with Noise2Score, it discards the noise distribution on exponential family, and derives the solutions for additive Gaussion noise, multiplicative noise, and mixture noise. The experiments verify the effectiveness of the proposed method.", "questions": "See weakness. ", "limitations": "As explained in the weakness, this work cannot handle the blind or real-world denoising task.", "rating": 5, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "1. The  proposed method is theoretically sound.\n2. It is a general unsupervised method, being able to deal with many kinds of noise types.\n3. The synthetic experimental results showcase that it is superior or comparable to recent SoTA method.", "weaknesses": "1. The proposed method is built on the core equation of Eq. (7).  The detailed mathematical derivation of this equation is not clearly presented. One of the main contributions of this work is that it dose not rely on the exponential family distribution. However, according to my understanding, Eq. (7) still follows the exponential family distribution. The writing of this part should be further improved.\n\n2. The introduction section states that \"Another advantage of our approach is that regardless of noise models, the training process of the score function neural network is identical.\"  In my opinion, it still requires to individually train the models on various noise types.\n\n3. The most limitation of this work is its practicality. It depends on the specific parameters of the noise distribution, thus cannot to deal with real-word denoising task. The experiments all focus on the synthetic datasets.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the positive feedback and constructive comments. We answer the raised questions below. We hope that our answers clarify the doubts and address the concern of the reviewer.\n\nQ1: The detailed mathematical derivation of this equation (Eq 7) is not clearly presented.\n\nA1: Equation 7 is derived based on the form of Equation 6, similar to how Equation 5 is derived based on the form of Equation 4. We apologize for any confusion stemming from certain notations in the paper. To clarify, the \"x\" in Equations 5 and 7 should be replaced with $\\hat{x}$. Equations 5 and 7 are not equations about the clean x and noisy y, but rather about an unknown denoised result $\\hat{x}$, where y is the given and known noisy image. Consequently, both Noise2Score and our approach aim to solve equations related to $\\hat{x}$. However, Noise2Score derives its solving equation using Tweedie's Formula, while our method constructs the solving equation based on Proposition 3.1. As Equation 6 remains valid for any distribution, it naturally holds for exponential family distributions. If the distribution belongs to the exponential family, Equation 6 can be reformulated as Equation 4. Hence, under an exponential family distribution, Equation 7 can be expressed in the structure of Equation 5. In essence, our method (solving Equation 7) serves as an important expansion of Noise2Score that enables image denoising for multiple noise models.\n\nQ2: The introduction section states that \"Another advantage of our approach is that regardless of noise models, the training process of the score function neural network is identical.\" In my opinion, it still requires to individually train the models on various noise types.\n\nA2: The underlying idea here is that when the noise model is provided, the training approach remains unchanged irrespective of its type and parameters. This approach focuses on estimating the score function, and the estimation method is independent of the noise model itself. In real-world scenarios, it's necessary to estimate the noise model's type and parameters in advance. If any changes occur in these estimations, the original score function estimation remains applicable as the noisy image remains unchanged. Consequently, retraining the model isn't needed; modifying Equation 7 and solving it anew would be sufficient.\n\nQ3: The most limitation of this work is its practicality. It depends on the specific parameters of the noise distribution, thus cannot to deal with real-word denoising task.\n\nA3: This is a limitation of our method, but it can be addressed by two approaches. For real-world denoising tasks: 1) Employ domain knowledge to estimate noise parameters; 2) Follow a method mentioned in Noise2Score, selecting an additional indicator that doesn't rely on ground truth, such as Total Variation (TV). By solving Equation 7 for various noise parameters, distinct denoised outcomes are achieved. The most optimal denoised outcome is then selected based on the indicator's performance. In fact, Noise2Score also has this limitation and Noise2Score addresses it by the second approach."}, {"review_id": "ASmdT7kDZP", "reviewer": "Reviewer_PKR9", "summary": "This paper proposes a general unsupervised image denoising approach by solving system with an estimated score function. The proposed system can be applied on multiple noise models by changing the equation system rather than retraining the model.", "questions": "The overall idea presented in this paper is interesting and the results are promising. Since I am not very sure about the application of the unknown noise parameters, I may defer my recommendation after authors' response. ", "limitations": "N.A.", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1)\tCompared to the Noise2Score limited to the noise models of exponential family distributions, the proposed method unlocks the limitation and generalizes to the non-exponential family distributions.\n2)\tFor different noise model, the proposed method only requires to modify the equation system to be solved and keeps the trained score function estimator unchanged.\n3)\tThe applications on the additive noise, multiplicative noise, and mixture noise have been proved in the manuscript and the supplementary materials.", "weaknesses": "1)\tQualitative comparison should be provided in the supplementary materials.\n2)\tThe Noise2Score can be applied to the unknown noise parameters, and whether the proposed methods can achieve it? Please provide the discussion and experiments.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the positive feedback and constructive comments. We answer the raised questions below. We hope that our answers clarify the doubts and address the concern of the reviewer.\n\nQ1: Qualitative comparison should be provided in the supplementary materials.\n\nA1: We will offer a one-page PDF comprising a qualitative comparison. Afterward, we will incorporate it into the supplementary materials.\n\nQ2: The Noise2Score can be applied to the unknown noise parameters, and whether the proposed methods can achieve it?\n\nA2: Noise2Score tackles unknown noise parameters by adopting a strategy of selecting an additional indicator that operates without relying on ground truth, such as Total Variation (TV). This method involves solving Equation 7 for various noise parameters, resulting in diverse denoised outcomes. The denoised outcome displaying the most favorable indicator performance is subsequently selected. This approach is equally suitable for our method."}, {"review_id": "ARo28jyGQg", "reviewer": "Reviewer_V5Ro", "summary": "The following work attempts to solve a single image denoising task in an unsupervised fashion. The authors propose to predict the score function and then denoise the input noisy image by solving the system of equations. Moreover, the proposed approach is a generalization of the Noise2Score method, which can work for complex non-exponential family distribution. Extensive experimental results demonstrate comparable and superior results in complex cases over existing unsupervised denoising methods.  ", "questions": "Suggestions:\n•\tIt would be better to move some of the details in the implementation part. For example, AR-DAE, which is used for score function estimation.\n•\tTypos (e.g. line 114)\n•\tI would suggest to add some visual examples in the Supplementary (if the main paper does not have enough space).\n\n", "limitations": "not applicable", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "+ Proposed extension of Noise2Score to images with complex noise distribution is very appealing and substantially increases the chances to apply the method to real-world scenarios.\n+ Extensive experimental results on additive, multiplicative, and mixed noises. ", "weaknesses": "- Proposed method is highly susceptible to noise parameter estimations according to Figure 2.\n- Moreover, one needs to understand the noise distribution type (Gamma Noise, Poisson Noise, etc.) of an arbitrary input image to estimate particular parameters of the distribution. Then those estimations are further used to solve Eq. 7.\n- It would be better to apply the proposed method to real image denoising given an estimated noise parameters. For example, raw image denoising (e.g. SIDD, DnD). \n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the positive feedback and constructive comments. We answer the raised questions below. We hope that our answers clarify the doubts and address the concern of the reviewer.\n\nQ1: Proposed method is highly susceptible to noise parameter estimations according to Figure 2.\n\nA1: Inaccurate estimation of noise parameters could lead to a decline in the performance of the proposed method. Nevertheless, a strategy akin to the one employed by Noise2Score for handling unknown noise parameters can be applied. This entails selecting an auxiliary indicator that operates independently of ground truth, such as Total Variation (TV). By solving Equation 7 with different noise parameters, a range of denoised outcomes is obtained. The denoised result exhibiting the most favorable performance according to the indicator is subsequently chosen. \n\nQ2: It would be better to apply the proposed method to real image denoising given an estimated noise parameters. For example, raw image denoising (e.g. SIDD, DnD)\n\nA2: We tested our approach on the SIDD dataset using the estimated parameters provided by the dataset, and produced a PSNR of 32.3. This result is better than the 32.2 achieved by unsupervised method Nr2N and comparable to the 33.9 achieved by supervised learning, demonstrating that our method is practical for real-world denoising problems."}, {"review_id": "bORWCI2OEl", "reviewer": "Reviewer_Y7fg", "summary": "This paper presents a self-supervised learning algorithm for image denoising which only requires the score of the noisy image distribution to perform denoising. The algorithm builds on ideas of a recent paper Noise2Score, extending the family of noise distributions that can be handled via an application of Fisher's identity. \n", "questions": "If I'm not wrong, the main formula in Proposition 3.1 is just Fisher's identity? It would be good to name this accordingly and cite some references.\n\nIn Theorem 3.1, what does \"f(x,y) is invertible\" means? Invertible with respect to all variables? There is also a typo in Lipschitz continuous.", "limitations": "The limitations are not discussed in detail. It would be good to specify that the method strongly relies on the knowledge of the noise distribution, whereas other methods such as Neighbor2Neighbor do not.", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- The paper presents a principled way of performing self-supervised denoising with a large family of noise distributions, which go beyond the exponential family. The previous Noise2Score only handles distributions belonging to the exponential family.\n- The paper demonstrates the good performance of the proposed approach on a large number of denoising experiments.", "weaknesses": "- The authors do not compare with SURE-based methods, which can handle mixture distributions, such as mixed Poisson Gaussian noise, e.g., see \"An unbiased risk estimator for image denoising in the presence of mixed Poisson–Gaussian noise\" by LeMontagner et al.\n\n- While the idea is theoretically stimulating, I don't see a lot of practical applications for the family of distributions where SURE or Noise2Score would fail. It would be good if the authors could specify some.\n\n- Some statements such as \"Our approach is so powerful that\" should be toned down. In particular, note that the approach cannot handle *any* noise distribution. See for example the discussion on Binomial noise in \"Least squares estimation without priors or supervision\" by Raphan and Simoncelli.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the positive feedback and constructive comments. We answer the raised questions below. We hope that our answers clarify the doubts and address the concern of the reviewer.\n\nQ1: It would be good if the authors could specify some distributions where SURE or Noise2Score would fail.\n\nA1: The noise models of many real world data are not exponential family distribution or a mixture of exponential family distributions; for example, all the ultrasound images in both medical and non-medical applications have speckle noise (or approximated as Rayleigh noise), and thus our method would be more effective for these applications. More importantly, a lot of real world data has correlated noise (although the structure of correlation could be simple), such as the noise in medical images is always locally correlated because of partial volume effect due to limited resolution. When SURE or Noise2Score is applied to this type of application, it usually has to approximate the noise to uncorrelated noise. \n\nQ2: Is the main formula in Proposition 3.1 just Fisher's identity?\n\nA2: No, they are quite similar, but there are some differences. \n\nFisher's identity:\n$$\n\\nabla_\\theta \\log p_\\theta\\left(\\boldsymbol{y}\\right)=\\int \\nabla_\\theta \\log p_\\theta\\left(\\boldsymbol{x}, \\boldsymbol{y}\\right) p_\\theta\\left(\\boldsymbol{x} \\mid \\boldsymbol{y}\\right) \\mathrm{d} \\boldsymbol{x}\n$$\nProp 3.1:\n$$\n\\nabla_{\\boldsymbol{y}} \\log p(\\boldsymbol{y})=\\int p(\\boldsymbol{x} \\mid \\boldsymbol{y}) \\nabla_{\\boldsymbol{y}} \\log p(\\boldsymbol{y} \\mid \\boldsymbol{x}) \\mathrm{d} \\boldsymbol{x}\n$$\n\n\nQ3: In Theorem 3.1, what does \"f(x,y) is invertible\" means? Invertible with respect to all variables?\n\nA3: \"f(x,y) is invertible\" means that with y given, it is invertible with respect to x. This property make it possible to find a unique solution for x in Eq 7.\n"}, {"review_id": "At2pw97mlo", "reviewer": "Reviewer_Gx7h", "summary": "The paper introduces an extension of Noise2Score for unsupervised image denoising and demonstrates its efficiency on several noise models, including non-exponential family distributions.", "questions": "* If possible, could the authors give numerical details on the approximation error between equations (6) and (7) in practical cases that differ from exponential family distributions, and elaborate on the hypotheses (f invertible, inverse function Lipschitz continuous, and bounded Hessian) ?\n\n* Could the authors exhibit ill-posed cases for which this approximation wouldn't be accurate, and show the implication on the denoising algorithm ?", "limitations": "See remarks above.", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 2, "strengths": "The extension of Noise2Score allows to deal with different kinds of noise, as opposed to prior works that only focus on exponential family distributions. The paper is clearly-written, with a fair amount of examples and experiments.", "weaknesses": "The contribution is relatively incremental compared to Noise2Score, and is based on an approximation (equation (7) replaces equation (6) in the resolution) that is not really discussed. Even though Theorem 3.1 gives some upper bound on the distance between the estimate and the conditional expectation, it would have been interesting to get insights on the constant $C$ and on the covariance of $x$ given $y$ in practice, in particular with complex noise models the paper aims to deal with. ", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the positive feedback and constructive comments. We answer the raised questions below. We hope that our answers clarify the doubts and address the concern of the reviewer.\n\nQ1: The contribution is relatively incremental compared to Noise2Score, and is based on an approximation (equation (7) replaces equation (6) in the resolution) that is not really discussed.\n\nA1: Our approach serves as an important expansion of Noise2Score. However it enables image denoising for multiple noise models (please refer to Table 1). Equation 7 is not an approximation of Equation 6; rather, it is derived based on the structure of Equation 6, akin to how Equation 5 is derived from Equation 4. We apologize if certain notations in the paper are causing confusion. To clarify, the \"x\" in Equations 5 and 7 should be replaced with $\\hat{x}$. Equations 5 and 7 do not pertain to the original clean x and noisy y, but rather they concern an unknown denoised result, $\\hat{x}$, with y representing the given and known noisy image. Consequently, both Noise2Score and our method aim to solve equations related to $\\hat{x}$. However, Noise2Score arrives at the solving equation via Tweedie's Formula (the cases of exponential family distribution), while our method devises the solving equation based on Proposition 3.1 (more general distribution). In the majority of cases, including most exponential family distributions, the solution of Equation 7 does not yield E[x|y] (in other words, the approximation error between E[x|y] and the denoised result also exists in Noise2Score). As such, we employ Theorem 3.1 to gauge the closeness between the solution and E[x|y]. \n\nQ2: Even though Theorem 3.1 gives some upper bound on the distance between the estimate and the conditional expectation, it would have been interesting to get insights on the constant and on the covariance of x given y in practice.\n\nA2: $C$ is related to the smoothness of score function, while the covariance of x given y is related to the noise level. A smoother score function and lower noise intensity, resulting in a smaller upper bound; conversely, a less smooth score function and higher noise intensity, leading to a larger upper bound.\n\nQ3: If possible, could the authors give numerical details on the approximation error between equations (6) and (7) in practical cases that differ from exponential family distributions, and elaborate on the hypotheses (f invertible, inverse function Lipschitz continuous, and bounded Hessian) ?\n\nA3: For an explanation regarding the approximation error, please refer to the response to the first question. Concerning hypotheses, please consult Proof 1.3 provided in the supplementary material. We list the explanations for them as follows:\n\n* f invertible: Given $\\boldsymbol{y}$, $\\boldsymbol{f}(\\boldsymbol{x}, \\boldsymbol{y})$ is invertible about $\\boldsymbol{x}$.\n* inverse function Lipschitz continuous: Denote the inverse function of $\\boldsymbol{f}(\\boldsymbol{x}, \\boldsymbol{y})$ as $\\boldsymbol{f}_{\\boldsymbol{y}}^{-1}$ where $\\boldsymbol{y}$ is given. Thus the variable of the inverse function is $\\boldsymbol{x}$. The Lipschitz continuous is an assumption on the property of $\\boldsymbol{f}(\\boldsymbol{x}, \\boldsymbol{y})$.\n* bounded Hessian: Denote each component of $\\boldsymbol{f}(\\boldsymbol{x}, \\boldsymbol{y})$ as $\\boldsymbol{f}_i$, for any $i$, the Hessian matrix of $\\boldsymbol{f}_i$ is bounded.\n"}]

Author rebuttal:
We thank the reviewer for the positive feedback and constructive comments. The qualitative comparison is in the attached PDF.

Meta-review: Overall, after the rebuttal round, the reviews are positive about the paper and all recommend it for (borderline) acceptance. I tend to agree that the paper has made a worth contribution and that the authors have made a convincing effort to address the shortcomings pointed out by the reviewers. Some lingering concerns remain about the mathematical derivation of Eq. 7 and the practicality of the proposed method, but these concerns are not sufficient to keep the paper from being accepted. That said, it would be very nice if the authors could, at least in future work, conduct additional experiments to demonstrate the practicality of the proposed method on real-world datasets.
Decision: Accept (poster)