GAN2GAN: Generative Noise Learning for Blind Denoising with Single Noisy Images
Authors: Sungmin Cha
Taeeon Park
Byeongjoon Kim
Jongduk Baek
Taesup Moon
Venue: ICLR 2021 Poster (2021)
Keywords: blind denoising
unsupervised learning
iterative training
generative learning

Abstract: We tackle a challenging blind image denoising problem, in which only single distinct noisy images are available for training a denoiser, and no information about noise is known, except for it being zero-mean, additive, and independent of the clean image. In such a setting, which often occurs in practice, it is not possible to train a denoiser with the standard discriminative training or with the recently developed Noise2Noise (N2N) training; the former requires the underlying clean image for the given noisy image, and the latter requires two independently realized noisy image pair for a clean image. To that end, we propose GAN2GAN (Generated-Artificial-Noise to Generated-Artificial-Noise) method that first learns a generative model that can 1) simulate the noise in the given noisy images and 2) generate a rough, noisy estimates of the clean images, then 3) iteratively trains a denoiser with subsequently synthesized noisy image pairs (as in N2N), obtained from the generative model. In results, we show the denoiser trained with our GAN2GAN achieves an impressive denoising performance on both synthetic and real-world datasets for the blind denoising setting; it almost approaches the performance of the standard discriminatively-trained or N2N-trained models that have more information than ours, and it significantly outperforms the recent baseline for the same setting, \textit{e.g.}, Noise2Void, and a more conventional yet strong one, BM3D. The official code of our method is available at https://github.com/csm9493/GAN2GAN.

GAN2GAN: GENERATIVE NOISE LEARNING FOR BLIND DENOISING WITH SINGLE NOISY IMAGES
Sungmin Cha', Taeeon Park', Byeonggoon Jongduk Baek² and Taesup Moon Sungkyunkwan University*, Yonsei University?, Seoul National University", South Korea (csm9493,tte12366 @skku. edu, b:kiim2006@eeave.... com, jongdukbaekeeonseein ac.kr, wtt.c..rrttttteeeeee.eeeeoooo
ABSTRACT
We tackle a challenging blind image denoising problem, which only single distinct noisy images are available for training denoiser, and no information about noise is known, except for it being zero-mean, additive, and independent clean image. In such setting, which often occurs in practice, it is not possible to train a denoiser with the standard discriminative training or with the recently developed Noise2Noise (N2N) training; the former requires the underlying clean image for the given noisy image, and the latter requires two independently realized noisy image pair for clean image. To end, Artificial-Nois to Generated- Generated-Ariffiial-- Artificial-Noise) method that first learns a generative model that can simulate the noise in the given noisy images and 2) generate a rough, noisy estimates of the clean images, then 3) iteratively trains a denoiser with subsequently synthesized noisy image pairs (as in N2N), obtained from the generative model. In results, we show the denoiser trained with our GAN2GAN achieves an impressive denoising performance on both synthetic and real-world datasets for the blind denoising setting; it almost approaches the performance the standard discriminatively-triie information than ours, and it significantly outperforms the recent bascline for the same setting, e.g., Noise2Voio and a more conventional yet strong one, BM3D, The official code of our method is available at https///tthhbccomm//////////////
the
propose GAN2GAN (Generated-
or N2N-trained models that have more
INTRODUCTION
Image denoising is one of the oldest problems image processing and low-level computer vision, yet it still attracts lots of attention due to the fundamental nature of the problem. A vast number of algorithms have been proposed over the past several decades, and recently, the CNN-based methods, e.g., Cha & Moon 2019); Zhang et al. (2017); Tai al. (2017): Liu et al. 2018). became the throne-holders in terms of the PSNR performance. The main approach of the most CNN-based denoisers is to apply the discriminative learning framework with (clean, noisy) image pairs and known noise distribution assumption. While being effective, such framework also possesses a couple of limitations that become critical in practice: the assumed noise distribution may be mismatched the actual noise in the data or obtaining the noise-free clean target images is not always possible or very expensive, e.g., medical imaging (CT or MRI) or astrophotographs.
Several attempts have been made to resolve above issues. For the noise uncertainty. the so-called blind training have been proposed. Namely, a denoiser can be trained with a composite training set that contains images corrupted with multiple, pre-defined noise levels or distributions, and such blindly trained denoisers, e.g., DnCNN-B in Zhang et al. (017)) were shown to alleviate the mismatch scenarios to some extent. However, the second limitation, i.e., the requirement of clean images for building the training set, still remains. As an attempt to address this second limitation, Lehtinen et al.) (2018) recently proposed the Noise2Noise (N2N) method. It has been shown that denoiser, which has negligible performance loss, can be trained without the clean target images, as long as two independent noisy image realizations for the same underlying clean image available. Despite its
"Corresponding author (F-mail: tamoonesnu. ac.kkr)))))

---

effectiveness, the requirement of the two independently realized noisy image pair for single clean image, which may hardly be available in practice, is a critical limiting factor for N2N.
In this paper, consider setting in which neither of above approach is applicable, namely, the pure unsupervised blind denoising setting where only single distinct noisy images are available for training. Namely, nothing is known about the noise other than it being zero-mean, additive, and independent of the clean image, and neither the clean target images for blind training nor the noisy image pairs for N2N training is available. While some recent work, e.g. Krull et al. (2019): Batson & Royer ) setting, we take generative learning approach. The crux of our method is first learn a Wasserstein GAN (Arjovsky et al. (017)-based generative model that can 1) learn and simulate the noise in the given noisy images and 2) generate rough, initially denoised images. Using such generative model, we then synthesize noisy image pairs by corrupting each of the initially denoised images with the simulated noise twice and use them to train a CNN denoiser as in the N2N training (i.e., Noisy N2N). We further show that iterative N2N training with refined denoised images can significantly improve the final denoising performance. We dubbed our method as GAN2GAN (Generated-Arttfical-- Artifical--ooss to Generated- Artifficial -oise)) and show that the denoiser trained with method can achieve (sometimes, even outperform) the performance of the standard supervised-trainee or N2N-trained blind denoisers for the white Gaussian noise case. Furthermore, for mixture/correlated noise or real-world noise in microscopy/CT images, for which the exact distributions are hard to know a priori, we show our denoiser significantly outperforms those standard blind denoisers, which are mismatch-trained with white Gaussian noise, as well as other baselines that operate in the same condition as ours: the SSL baseline, N2V (Krull et al. 2019), and a more conventional BM3D (Dabov et al. 2007).
:: Laine et al. (2019), took the self-supervised learning (SSL) approach for the same
2 RELATED WORK
Several works have been proposed to overcome the limitation of the vanilla supervised learning based denoising. As mentioned above, Noise2Self (N2S) (Batson Royer, 2019) and Noise2 Void (N2V) (Krull et al. (2019) recently applied super-ipe learning (SSL) approach to train denoiser only with single noisy images. Their settings exactly coincide with ours, but we show later that our GAN2GAN significantly outperforms them. More recently, Laine et al. (2019) improved N2V by incorporating specific noise likelihood models with Bayesian framework, however, their method required to know the exact noise model and could not be applied more general, unknown noise settings. Similarly, Soltanayev & Chun (2018) proposed SURE (Stein's Unbiased Risk Estimator)- based denoiser that can also be trained with single noisy images, but worked only with the Gaussian noise. Their work was extended in Zhussip et al. (2019), but it required noisy image pairs as in N2N as well as the Gaussian noise constraint. Chen et al. (2018) devised GCBD method learn and generate noise in the given noisy images using W-GAN Arjovsky et al. (2017) and utilized the unpaired clean images to build supervised training set. Our GAN2GAN related to Chen et al.) (2018), but we significantly improve their noise learning step and do not use the clean data at all. Table I summarizes and compares the settings among the above mentioned recent baselines. We clearly see that only our GAN2GAN and N2V do not utilize any "sidekicks" that other methods use.
Table Summary of different settings among the recent baselines.
Alg. Requirements
Clean image
Noisy pairs
Noise model
N2N Lehlinen et (2018)
HQ SSL Laine et al 2019
x
x
SURE Soluamaye Clun
Ext. SUKE Zhussip cl 2019
x
/
/
6CBD Chen
7
8
N2V [Kall 2019
8
8
GANZGAN (Uuss)
8
8
Additionally, there are recently published papers on blind image denoising but these also have a difference with ours. Anwar & Barnes (2019); Zhang et al. (2018) suggest effective CNN architectures for denoising, however, they only consider the setting in which clean images necessary for training. Zamir et al. (2020) considers the denoising of specific camera settings, and it also requires clean sRGB images as well as the knowledge of the noise level. Thus, it cannot be applied to the complete blind setting as ours, in which no information on the specific noise distribution or clean images is available.
2

---

More classical denoising methods are capable of denoising solely based the single noisy im- ages by applying various principles. e.g., filtering-bast basedd Buades et al. (2005): Dabov et al. (2007). optimization-base base-baadddddee EEad & Aharon 2006); Mairal et al. (2009), Wavelet-based Donoho & Johnstone (1995), and effective prior-based Zoran & Weiss| (2011). Those methods typically are, however, computationally intensive during the inference time and cannot be trained from separate set of noisy images, which limits their denoising performance. Another line of recent work worth mentioning is
the deep barning-based priors or regularizers, e.g., e.g., [Ulynov et al. (2018); Yeh et al. (2018): Lunz et al. (2018), but their PSNRs still fell short of the supervised trained CNN-based denoisers.
3 MOTIVATION
In order to develop the core intuition for motivating our method, we first consider simple, single- letter Gaussian noise setting. Let Z = X + N be the noisy observation of X ~ (0, x²),, corrupted by the N ~ (0, x. is well known that the minimum MSE (MMSE) estimator of X given Z is MMSE () = [ =
ox_ to Z. We now identify the optimality of N2N in this setting.
N2N Assume that we have two i..dd copies of the noise N: N1 and N2. Then, let Z, X N and Z2 = X + N₂ be the two independent noisy observation pairs of X. The N2N in this setting corresponds to obtaining the MMSE estimator of Zz given Z1.
o
JNZN (1) - arg min B(Z2 - - = E(X + N2 (0) E(( = xx +o
21
(1)
N
in which (a) follows from N₂ being independent of Z1. Note has the exact same form as hence, estimating X with (Z) also achieves the MMSE, in line with (Lehtinen et al.
J MMSE
(2),
2018
"Noisy" N2N Now, consider the case in which we again have the two i..dd N1 and N₂, but the noisy observations are of noisy version of X. Namely, let X' = X + No. in which No ~ NO, oī). and denote Zi = X' + Ni and Z X' + N22 as the noisy observation pairs. Then, we can define "Noisy" N2N estimator as the MMSE estimator of Z given Zj.
(x v)
NNoy NZN NEN v) - arg argmin f(Z))) = E(x 21) = x1 4)+ o
Z,
(2)
in which we denote Ao/ / and assume that 0 S < 1. Note clearly 2 coincides with
when y of 0. Following N2N, 2 is essentially estimating X' based on '' X' + N. An
interesting subtle question is what happens when we use the mapping JNoisy N2N (2, y) for estimating X given Z X + NN not X' given Z'. Our theorem below, of which proof is in the Supplementary Material (S.M.), shows that for a sufficiently large oū fNoisy NEN( Z, y) gives a better estimate of X than X'.
Theorem Consider the single-letter Gaussian setting and fNoisy NZN( Z, v) obtained in 2 Also, assume 0- < < 1. Then, there exists some Yo s.1. Vy E (yo, 1), E(X- Swoisy N2N (2 3) o
Theorem 1 provides a simple, but useful, intuition that mo-
tivates our method; if simulating the noise in the images is possible, we may carry out the N2N training iteratively, pro- vided that rough noisy estimate of the clean image is initially available. Namely, we can first simulate the noise to generate noisy observation pairs of the initial noisy estimate, then do the Noisy N2N training with them to obtain denoiser that may result in better estimate of the clean image when applied to the actual noisy image subject to denoising (as in Theorem D.. Then, we can refine the estimates by iterating the Noisy N2N training with the generated noisy observation pairs of the previous step's 's estimate of the clean image, until convergence.
PSNR (dB)
N2N
26.
Nolsy-N2N;
Noisy-N2N,
(26.3) (26.5) 125.2) 0445 (23.001 (2: (21 (3)53)
Figure 1: Iterative Noisy N2N.
To check whether above intuition is valid, we carry out fea- sibility experiment. Figure| [ shows the denoising results on The blue line is the PSNR of the N2N
BSD68 (Roth& & Black 2009) for Gaussian noise with 25.
3

---

model trained with noisy observation pairs of the clean images in the BSD training set, serving as an upper bound. The orange line, in contrast, is the PSNR of the Noisy N2N, model that is trained with the noisy observation pairs of the noisy estimates for the clean images. which were set to be another Gaussian noise-corrupted training images. The standard deviations (ao) of the Gaussian for generating the noisy estimates are given in the horizontal axis, and the corresponding PSNRs of the estimates are given in the parentheses, Although Noisy N2N, clearly lies much lower than the N2N upper bound, we note its PSNR still higher than that of the initial noisy estimates, which is in line with Theorem 1 Now, if we iterate the Noisy N2N with the previous step's s denoised images (i.e., /Noiso
for second/third iterations, respectively), we observe that the PSNR significantly improves and approaches the ordinary N2N for most of the initial op values. Thus, we observe the intuition from Theorem Saaareem@eeee generalizes well to the image denoising case in an ideal setting, where the noise can be perfectly simulated, and the initial noisy estimates are Gaussian corrupted versions. The remaining question is whether we can also obtain similar results for the blind image denoising setting. We show our generative model-base model-based approach in details in the next section.
4 MAIN METHOD: THREE COMPONENTS OF GAN2GAN
To concretely describe our method, we first set the notations. We assume the noisy image Z is generated by Z + N. in which x denotes the underlying clean image and N denotes the zero- mean, additive noise that is independent of x. For training a denoiser, we do not assume either the distribution or the covariance of N is known. Moreover, is available for learning denoiser. A CNN-based denoiser is denoted as X o (Z) with & being the model parameter, and we the standard quality metrics, PSNR/SSIM, for evaluation. Our method consists of three parts; 1) smooth noisy patch extraction, 2) training a generative model, and 3) iterative GAN2GAN training of (Z), each of which we elaborate below.
assume only database of n. distinct
noisy images. D (Z() 1
4.1 SMOOTH NOISY PATCH EXTRACTION
The first step is to extract the noisy image patches from D that correspond to smooth. homogencous areas. Our extraction method is similar to that of the GCBD proposed in (Chen et al., (018), but we make a critical improvement. The GCBD determines patch p (of pre-determined size) is smooth if satisfies the following for all of its smaller sub-patches, qj. with some hyperparameters 14, Y E (0, 1):
(V(g)) - v(p) < >V()), in which E(-) and V(-) are the empirical mean and variance of the pixel values.
[E(q;) - E(p) S μE(p),
(3)
Ours Bo.(4)
GCBD Ea.13)
GEBI Eq c3
23
1.0
52:5
5 1671
8 57.00
6 51.95
0 6450
Our Eqgé
0.5
0.0
48
0-
50
52
54
56
25:3
7581
2516
15.07
22.66
(a) Histograms of empirical ô.
(b) Examples of extracted noise patches (a 25)
Figure 2: Comparison of smooth noisy patch extraction rules.
While (3) works for extracting smooth patches to some extent, as we show in Figure 2(b it does not rule out choosing patches with high-frequency repeating patterns, which are far from being smooth. Thus, we instead use the 2D discrete wavelet transform (DWT) for new extraction rule; namely, we determine p is smooth if its four sub-band decompositions obtained by DWT, (W)(p)) satisfy
Σ
-(W.d)))-B(wwd - B(ow(p)) S (p)),
(4)
in which a(:) is the empirical standard deviation of the wavelet coefficients, ([) A Ei (WW((p)) and X E (0, 1) is hyperparameter. This rule is much simpler than which
4

---

N (
fu
OuGone Bik ReLU
C,, ur)
Smooth patch extraction
96
fun
Nosy
Cz O2. ws)
2i
cropping 2
(Z
Caye (02, 0g)
(a) Getting D & N
(b) The model architecture with three generators and two critics.
Figure 3: Overall structure of the W-GAN based generative model.
has be evaluated for all the sub- tabb-pptchs,, {q; }- Once N patches are extracted from DD using 4 we subtract each patch with mean pixel value, and obtain a set of "noise" patches, / (n()) j=1- Such subtraction is valid since all the pixel values should be close to their mean in a smooth patch, and the noise is assumed to be zero-mean, additive.
Figure 2 compares the rules 3 and (4) by showing the quality of the "noise" patches extracted from 1,000 Caussian-corrupted images The two plots in Figure [(a) show the normalized histograms of the empirical standard deviations, ã, of the extracted patches when true {25, 50). respectively. We clearly observe that while the â's for (4) are mostly concentrated on true a,, those of (3) have much higher variation. In addition, Figure 2(b) visualizes the randomly sampled patches of which a's were above the 90-th percentile among the extracted patches for each rule (when a = 25). Again, it is obvious that (3) also may result in selecting the patches with high-frequency patterns, whereas (4) is much more effective for extracting accurate noise patches. Later, we show (in Figure 5) that such improved quality of the noise patches by our (4) plays an essential role; namely, our pure unsupervised learning based denoiser using 4 even outperforms the clean target image based denoiser (Chen et al. 2018) using
4.2 TRAINING A W-GAN BASED GENERATIVE MODEL
Equipped with D { and the extracted noise patches /V = (n)) we train a generative model, which can learn and simulate the noise as well as generate initial noisy estimates of the clean images, hence, realize the Noisy N2N training explained in Section 3 As shown in Figure 3 our model has three generators, {ge,, g8.: } and two critics, { JwaJ. in which the subscripts stand for the model parameters. The loss functions associated with the components of our model are:
(n, w1) A En [w, (n)) - Er (r))
(5)
Lz w2) A Ez w0 [2) Ez, [fws (go, (Z) + 90, (r))]
(6)
Layc 0s) Ez go, (go. [1)
The loss (5) is standard W-GAN Arjovsky et al. 2017) loss for training the first generator-critic pair, (go, Fun, ). of which 9o, learns to generate the independent realization of the noise mimicking the patches in / = {n(
J= taking the random vector r ~ N/O0, I) as input. The second loss (6) links the generators, go, and go,, with the second critic, fung- The second generator go, is intended to generate the estimate of the underlying clean patch for Z. i.e., coarsely denoise Z, and the critic fwy determines how close the distribution of the generated noisy image. go, + ge, (r),
the that of 2 Our intuition is, if ge, can realistically simulate the noise, then enforcing ge, (Z) + ge, (r) to mimick Z would result in learning reasonable initial denoiser go.- One important detail regarding go, is its final activation must be the sigmoid function for stable training. The third loss (7). which
We assume go, implicitly has the cropping step for such that the dimension of go. (Z) and go, (r) match.
5

---

resembles the cycle loss in (Zhu et al. 2017). imposes the encoder-decoder structure between go: and ge., hence, helps g0, to compress the most redundant part of Z, i.e., the noise, and carry out the initial denoising. Once the losses are defined, training the generators and critics are done in an alternating manner, as in the training of W-GAN (Arjovsky et al. 2017), to approximately solve
min max 0,.02.03 w1.u22
(O,, w1)+ (Cz 02 w2) Ceye (02, 03))
in which (a, B) are hyperparameters control the trade-off's between the loss functions. The pseudo algorithm for training a generative model is given in Algorithm There are couple of subtle points for training with the overall objective (8) and we describe the full details on model architectures and hyperparameters in the S.M.
Algorithm Training a generative model, all experiments in this paper used the defaults values, Peritic = 5, Nepoch = 30, m 64, 0g = 4e Geritic = 5e a 5, B = 1, y = 10
1: Require D,
2: N + NoiisePatchExtraction// D,
3: for ePGAN + Nepoch do
4: Sample {n())) Ji=1 N, (r mm N(,, (2()) Ji=1
5: for eperitic s- 1, Meritic do
6: gren Vuy ((n w1 w1)), Iwg Vw, [C (1. 02 w2)l
7:
w1 Clip(u + Cleritic Adam (W1, 9w0 c)
8:
W2 +- Clip U22 + Clerisic Adamm (22 gwz C,
9:
end for
10:
9 90, + Vo. 0, an (02, w1 3Lz ( B2, W2) - (0,,
11: 01 s 01 aa- Adam (0,00,)))))))000)))))),.)))))( 82 a2 Adam (82, 90.) 0g t- 0g ag Adam(Os, ge,)
12:
end for
13:
return 01,05 02
4.3
ITERATIVE GAN2GAN TRAINING OF A DENOISER
With generative model, we then carry out the iterative Noisy N2N training described in Section 3 with the generated noisy images. Namely, given each ( E DD we generate the pair
(1) 2 (ge, (z) + go. (r go, (z()) + ge, (r12))
(9)
in which ril T12
(6)
(4)
1228 are i..d. NN,, Z). In contrast to the ideal case Section each
generated image 99 a noise-corrupted version of go, (Z),,, in which the corruption is done by
the simulated noise ge, (r). Denoting the set of such pairs as Dj { 2
) denoiser (Z )2) In Caza(-), we only X
(i)
(Z) is trained by minimizing (Ø Di) n
use the generated noisy images and do not use the actual observed ( hence, we dubbed our training GAN2GAN (G2G) training. Now, denoting the learned denoiser G2G, (with parameter i we can iterate the G2G training. For the j-th iteration (with j 2 2), we generate
(2 (1) (i) A (Xx; (Z()) + Se, (r (1) Xx, (Z) + 90,
(10)
j1:
for each ( and denote the resulting set of the pairs Dj-- Note in c10 update the noisy estimate of the clean image with the output of G2G Then, the new denoiser G2G, is obtained by computing Φ arg min& Laza( Dj), where the minimization is done via warm-starting from Φy-1- In our experiments, we show the sequence, G2G,21. successively refines the denoising quality and significantly improves the initial noisy estimate, similarly as in Figure ! Moreover, identify the benefit of the iterative G2G training becomes greater when noise is more sophisticated; i.e., for synthetic noise, the performance of G2G,21 converges after 1~3 iterations, whereas for the real-world microscopy noise, the performance keeps increasing until larger number of iterations.
EXPERIMENTAL RESULTS
5.1 DATA AND EXPERIMENTAL SETTINGS
Data & training details In synthetic noise experiments, we always used the noisy training images from BSD400 (Martin et al. 2001). For evaluation, used the standard BSD68 (Roth & Black

---

a test set. For real experiment, experimented on two data sets: the WF set in the microscopy image datasets in (Zhang et al. 2019) and the reconstructed CT dataset. For both datasets, we trained/tested each (Avg n. and each dose level, respectively, which corresponds different noise levels. For the generative model training, the patch size used for 'D and was 96 x 96, and and N were set to 20, 000 (BSD) and 40, 000 (microscopy), respectively. For the iterative G2G training, the patch size for D was 120 x 120 and n = 20, 500, and in every mini-batch, we generated new noisy pairs with g0, as in the noise augmentation of (Zhang et al. 2017). The architecture of G2G, was set to 17-layer DnCNN in (Zhang et al. 2017). We put full details on training, model architectures and hyperparameters as well as the software platforms in the S.M.
2009)
Baselines The baselines were BM3D (Dabov et al. 2007). DrCNN-B ZZang et al. 2018), N2N (Lehtinen et al.] 2018), and N2V (Krull et 2019). We reproduced and trained DrCNN-B, N2N and N2V using the publicly available source codes on the exactly same training data as our iterative G2G training. For DrCNN-B and N2N, which use either clean targets or two independent noisy image copies, we used 20-layers DrCNN model with composite additive white Gaussian noise with [0, 55]. N2V considers the same setting ours and uses the exact same architecture as G2G,; more details on N2V are also given in the S.M. We could not compare with the scheme in (Laine et a (019)), since their code cannot run beyond white Gaussian noise case in our experiments and they had an unfair advantage: they newly generate noisy images by corrupting given clean images for every mini-batch whereas we assume the given noisy images are fixed once for all. It is known that such noise augmentation significantly can increase the performance, and their code could not run in our setting which the noisy images are fixed once given. As an upper bound, we implemented N2C(Eq. Q denoting a 17-layer DrCNN trained with clean target images in BSD400 and their noisy counterpart, which is corrupted by our go, learned with
5.2 DENOISING RESULTS ON SYNTHETIC NOISE
White Gaussian noise Table 2 shows the results on BSD68 corrupted by white Gaussian noise with different o's. Several variations of our G2G, go, and the G2G iterates, G2G are shown for two different training data versions for learning the generative model. Firstly, we clearly observe the
Table 2: Results on BSD68/Gaussian. Boldface denotes algorithms that only use single noisy images. Red and blue denotes the highest and second highest result among those algorithms, respectively.
PSNRUSSIM
BMBD
DRUNN-B
Easolnes
N2N
NZV
GEG
GITT variation
GXSs
CBC5
Lyper Round
N2C(Fq741)
31,47/8.717
3144X18936
31 81.200088455
29.48N.8199
25.94N..5199
30.98A88552
32 $2.51188827
31.45.88825
31,640.8870
28.569..0013
28 8137
7400 8041
26.97007008
24.160066630
28.230.766
28.52/0 8016
28.9660 8010
118088888
27.78/0.7727
28.06/0.661
27.910.7720
26.38006657
23.4340.9667
27. 7783
23.00.0.7759
28.289.7890
25.600.6866
25.78006721
25.7100.712
24.360.5765
20.5%0.4482
25.08/.62155
25.55/0.6639
25.78/0.6749
26.05/0.6051
iterative G2G training is very effective; namely, it significantly improves the initial noisy estimate J0.-
particularly when the quality of the initial estimate is not good enough. This result confirms the result of Figure T indeed carries over to the blind denoising setting with our method. Secondly, we note G2G, already considerably outperforms N2V, which is trained with the exact same model architecture and dataset. Finally, the performance of G2G, is very strong: it outperforms BM3D, which knows true a, and even sometimes outperforms the blindly trained DrCNN-B and N2N, which is trained with the same BSD400 dataset, but with more information. This somewhat counter-intuitive result is possible since our G2G, accurately learns the correct noise level in the image, while DnCNN-B and N2N are trained with the composite noise levels, a E 55).
Mixture and correlated noise Table 3 shows the results on mixture and correlated noise beyond white Gaussian. Note our G2G, does not assume any distributional or correlation structure of the noise, hence, it can still run as long as the assumption on the noise holds. In the table, the G2G results
Table 3: Results on BSD68/ixaure & Correlated noise, The boldface and colored texts are as before.
PSNRUSSIM
BMID
DucNac-B
Hiselines
NIN Stet
NIV
Sh.
G2G,
G2G Viristion
G2G,
CIG,
Upper hound (21)E4
Misture
Casc
41.44% 9622
37.9778.9447
37.2300 9616
19.30199440
33.539.00 9368
32. 32775099488
31.88819522
42. 42.08899766
39.1308
39.649
42.516099888
42- 4LL0NOL
30,70000055
29.27/8.8198
2633 3547
30 20.59000
18.22007755 7755
29.78083345
$1.94088299
36.81184417
Comelied
31,36913984
76.699
27.3900.257
27.52/18594
26.1108.7348
25.651.7607
27.80088130
28.11898221
28.08118447
28.4006837
for (BSD) specified above. Moreover, DnCNN-B and N2N are still blindly trained with the mismatched white Gaussian noise. For mixture noise, we tested with two cases. Case A corresponds to the same setting as given in (Chen et al. 2018), i.e., 70% (N( 0.1 20% ~ NO 1), and
7

---

10% Unif] s, s] which means the random variable that is uniformly distributed between [- s] with 8 = 15, 25. For case B, we tested with larger variances, L.e., 70% Gaussian NV 152), 20% Gaussian (0, 25²), and 10% Uniform [-8, with 30, 50. For correlated noise, we generated the following noise for each E-th pixel,
N₂ = 7 + (1 - 17)
( NNBB
1
Σ mENB.
Mm): (= 1,2,...
in which {Mr} are white Gaussian NN0, ). WB, is the x l neighborhood patch except for the pixel E, and v/ is a mixture parameter. We set r = 1/v22 such that the marginal distribution Nr is also (0, o²) and set k = 16. Note in this case, Ne has spatial correlation, and tested with o o= 15, 25. From the table, we first note that DrCNN-B and N2N suffer from serious performance degradation for both mixture and correlated noises due to noise mismatch, and the conventional BM3D outperforms them for some cases (e.g., Case A for mixture noise). However, we note our G2Gg still denoise very well after just two iterations and outperforms all the baselines for all noise types. Note N2V seriously suffers and is not comparable to ours. Finally, N2C (Eq.44)) is sound upper bound for all noise types, confirming the correctness of the extraction rule
5.3 DENOISING RESULTS ON REAL NOISE
We also test our method the real-world noise. While some popular real noise is known have source-dependent characteristics, there are also cases in which the noise is source-independens and pixel-wise correlated, which satisfies the assumption of our method. We tested on two such datasets,
the Wide-Focal (WF) set in the microscopy image dataset (Zhang et al. 2019) and a Reconstructed CT dataset. A more detailed description and analysis on these two datasets are in S.M. The WF
38
48
Clean
PSNR
BM
$1.79
/SSIM
8.7342
(dD)
PSNR (dB)
44
PSNR
344
G2G Variations (411Ee((1)
C2G Variations
N2V
G2G,
N2C(Eq.14)
31,85
33
BM3D
38
BMBD
(0.6966
M.8365
DnCNN,
DrCNNN
32
625
G26;
626
34
96
626;
626;
62G,
G2G variations
G2G variations
(a) Wide-Focal (WF)
(b) Reconstructed CT
(c) Visualization
Figure 4: Results on real microscopy image denoising on WF and medical image denoising.
and Reconstructed CT data has 5 sets (Avg = 1, 4, 8, 16) and 4 sets (Dose=25, 50, 75, 100) with different noise levels, respectively. We did not exploit the fact that the images multiple noisy measurements of a clean image, which enables employing N2N, but treated them as noisy images of distinct clean images. Figure 4(a] and
4b) shows the PSNR of all methods for each dataset, respectively, averaged over all sets. The baselines were DrCNN-B, BM3D and N2V. We note BM3D estimated noise a using the method in Chen et al. (2015). We iterated until G2G; and N2C(Eq 4 was an upper bound for each set. We clearly observe that the performance of G2G, significantly improves (over ge,) as the iteration continues. In results, G2Gg becomes significantly better than DrCNN-B and N2V well as BM3D, still one of the strongest baselines for real-world noise denoising when no clean target images are available, for both datasets. We report more detailed experimental results (including SSIM) on both datasets in S.M. Moreover, the inference time for BM3D is about 4.5m~5.0 seconds per image since a noise estimation has to be done for each image separately, whereas that for G2G, is only 4 ms (on GPU), which is another significant advantage of our method. Figure 4(c) shows the visualizations on the WF, and we give more examples in the S.M.
5.4 ABLATION STUDY
Noise patch extraction Here, we evaluate the effect of the noisy patch extraction rules (3) and (4) in the final denoising performance. Figure 5 compares the PSNR of N2C(GCBD Eq.(3), re-implementation of (Chen et al. 2018), N2C(Ours Eq. (4) and the best G2G, for each dataset.
8

---

We note neither source code training data of (Chen et al. 2018) is publicly available, and the PSNR in (Chen al. 2018) could not be reproduced (with the exact same y and y as in (Chen et al. 2018). From the figure, we clearly observe the significant gap between N2C(Our Eq.44) and N2C(GCBD Ep. particularly when the noise is not white Gaussian. More- over, our pure unsupervised G2G with 4 even outperforms N2C(GCBD 3h) that utilizes the clean target images, confirming the quality difference shown in Figure ((b) significantly affects learning noise and a denoiser.
45.0
N2ciOurs (40)
42.5
620
N2C te
37.5
PSNR
32.5
25.0
Figure 5: Effect of noise patch extraction rule.
Generative model and iterative G2G training Figure ((a) shows the PSNRs of !Ø, on BSD68/Gaussian(
25) trained with three variations; "No Lz" for no fwy. "No Leye for no 77 and I0,, and "No sigmoid" for no sigmoid activation at the output layer of go.- We confirm that proposed architecture achieves the highest PSNR for I0., the sigmoid activation and fw, essential, and the eylce loss (7) is also important. Achieving a decent PSNR for ge, beneficial for saving the number of G2G iterations and achieving high final PSNR. More detailed analyses on the generative model architecture are in the S.M. Figure (6b) and ( show the effect of the quality of
30
(aB)
PSWR (d8)
PSWR (dB)
20.0
Gupsian (om
-8
17.5
9+
consaniowag 25.4)
94 (No Ez)
Notare u 10 pv)
(No
22
Motare u 16,
Cormaise 25,00
5
15
2s. simmoit
20
25
30
20
0./2 020, C2G, G20, 526, 62%,
Cormaines 6y- 25,21
24
9.12 G26, 62G, E2G, G2G, G26, G2G, G2G varlations
Epoch
G2G variations
(a) Variations of ge,
(b) Iterative G2G (Synthetic)
(c) Iterative G2G (Real microscopy)
Figure 6: Ablation studies. (b) and (c) compare the performances between starting from go, and
the initial estimate for the iterative G2G training. From Figure 1 one may ask whether ge, is indeed necessary, since even when dg R the iterating the Noisy N2N can mostly achieve the upper bound. Hence, for samples of synthetic and real microscopy data, evaluate how G2G, performs when the iteration simply starts with Z. Figure 6 shows somewhat surprising result that for synthetic noises, starting from achieves essentially the same performance starting from gø, with couple more G2G iterations. However, for real microscopy noise case in Figure [(c) WF(Avg= 1) in which starting from Z achieves far lower performance than starting from go,, justifying our generative model for attaining the initial noisy estimate.
6 CONCLUDING REMARK
Motivated by novel observation on Noisy N2N, we proposed a novel GAN2GAN method, which can tackle the challenging blind image denoising problem solely with single noisy images. Our method showed impressive denoising performance that even sometimes outperform the methods with more information as well as VST+BM3D for real noise denoising. As future work, we plan to extend our framework to more explicitly handle the source-dependent real-world noise.
ACKNOWLEDGMENT
This work was supported in part by NRF Mid-Career Research Program and UIT grant (N0.2019- 0-01396, Development of framework for analyzing. detecting, mitigating of bias in AI model and training data], funded by the Korean government (MSIT).

---

REFERENCES
Saced Anwar and Nick Barnes. Real image denoising with feature attention. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 3155-3164, 2019.
Martin Arjovsky, Soumith Chintala, and Léon Bottou. Wasserstein generative adversarial networks. In International Conference on Machine Learning (ICML), 2017.
Joshua Batson and Loic Royer. Noise2self: Blind denoising by self-supervision. In International Conference on Machine Learning (ICML), 2019.
A. Buades, B. Coll, and J. M. Morel. A review of image denoising algorithms, with SIAM Journal on Multiscale Modeling and Simulation: A SIAM Intendisciplinary Journal, 2005.
Sungmin Cha and Taesup Moon. Fully convolutional pixel adaptive image denoiser. In International Conference on Computer Vision (ICCV), 2019.
Guangyong Chen, Fengyuan Zhu, and Pheng Ann Heng. An efficient statistical method for image noise level estimation. In Proceedings of the IEEE International Conference Computer Vision (CVPR), pp. 477-48775 485, 2015.
Jingwen Chen, Jiawei Chen, Hongyang Chao, and Ming Yang. Image blind denoising with generative adversarial network based noise modeling. In Conference on Computer Vision and Pattern Recognition (CVPR), 2018.
K. Dabov, A. Foi, V. Katkovnik, and K. Egiazarian. Image denoising by sparse 3-d transform-domain collaborative filtering. IEEE Trans. Image Processing, 2088))2080-2955, 2007.
D. Donoho and I. Johnstone. Adapting to unknown smoothness via wavelet shrinkage. Journal of American Statistical Association,
1995.
M. Elad and M. Aharon. Image denoising via sparse and redundant representations over learned dictionaries. IEEE Trans. Image Processing, (1(12):336637444 2006.
Alexander Krull, Tim-Oliver Buchholz, and Florian Jug. Noise2void-leaming denoising from single noisy images. In Conference on Computer Vision and Puttern Recognition (CVPR), 2019.
Samuli Laine, Tero Karras, Jaakko Lehtinen, and Timo Aila. High-quality self-supervised deep image denoising. In Advances in Neural Information Processing Systems (NeurIPS), pp. 6968-6978, 2019.
Jaakko Lehtinen, Jacob Munkberg, Jon Hasselgren, Samuli Laine, Tero Karras, Miika Aittala, and Timo Aila. Noise2noise: Learning image restoration without clean data. In International Conference on Machine Learning (ICML), 2018
Ding Liu, Bihan Wen, Yuchen Fan, Chen C. Loy, and Thomas S. Huang. Non-local recurrent network for image restoration. In Neural Information Processing Systems (NeurIPS), 2018.
Lunz, O. Öktem, and C.-B. Schönlicb. Adversarial regularizers in inverse problems. In Neural Information Processing Systems (NeurIPS), 2018.
Mairal, Bach, Ponce, G. Sapiro, and A. Zisserman. Non-local sparse models for image restoration. In International Conference on Computer Vision (ICCV), 2009. 2009,
D. Martin, C. Fowlkes, D. Tal, and J. Malik. A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In International Conference Computer Vision (ICCV), 2001.
S. Roth and M.J Black. Field of experts. International Journal of Computer Vision. 82(2):205-229, 2009.
Shakarim Soltanayev and Se Young Chun. Training deep learning based denoisers without ground truth data. In Advances in Neural Information Processing Systems (NeurIPS), pp. 3257-3267, 2018.
10

---

Ying Tai, Jian Yang. Xiaoming Liu, and Chunyan Xu. Memnet: A persistent memory network for image restoration. International Conference on Computer Vision (ICCV), 2017.
Dmitry Ulyanov, Andrea Vedaldi, and Victor Lempitsky. Deep image prior. In Conference on Computer Vision and Pattern Recognition (CVPR), 2018.
R.A. Yeh, T. Y. Lim, C. Chen, A. G. Schwing, M. Hasegawa- and M. N. Do. Image restoration with deep generative models. In International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2018.
Syed Waqas Zamir, Aditya Arora, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, Ming-Hsura Yang, and Ling Shao. Cycleisp: Real image restoration via improved data synthesis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2696-2705, 2020.
K. Zhang. W. Zuo. Chen, D. Meng, and Zhang. Beyond a gaussian denoiser: Residual learning of deep cnn for image denoising, IEEE Trans. Image Processing, 26(7):3142 3155, 2017.
Yide Zhang, Yinhao Zhu, Evan Nichols, Qingfei Wang, Siyuan Zhang, Cody Smith, and Scott Howard. A poisson-gaussian denoising dataset with real fluorescence microscopy images. In Conference Computer Vision and Pattern Recognition (CVPR), 2019.
Yulun Zhang, Yapeng Tian, Yu Kong, Bineng Zhong, and Yun Fu. Residual dense network for image restoration. arXiv preprint 1rxiv: 10477, 2018.
Yan Zhu, Taesung Park, Phillip Isola, and Alexei A Efros. Unpaired image-to-image translation using cycle-consistent adversarial networks. In International Conference on Computer Vision (ICCV), pp. 2223-22322 2017.
Magauiya Zhussip, Shakarim Soltanayev, and Se Young Chun. Extending stein's unbiased risk estimator to train deep denoisers with correlated pairs of noisy images. In Advances in Neural Information Processing Systems (NeurIPS), pp. 1465-1477 2019.
D. Zoran and Y. Weiss. From learning models of natural image patches to whole image restoration. International Conference on Computer Vision (ICCV), 2011.


Reviews:
[{"review_id": "Zev9jC3gzOZ", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The proposed method uses the generative learning method to simulate the noise and synthesize noisy image pairs to train the proposed network. Experimental results show the effectiveness of the proposed method. \n\nLearning denoised network from noisy images has been developed by Krull et al., 2019; Batson\n& Royer, 2019; Laine et al., 2019. The main difference is the use of generative learning. One possible clarification is whether the aforementioned methods using generative learning can generate comparable results or not. If so, the contribution of the paper is limited.\n\nThe motivation of using the proposed network design is not clear. In addition, using more generators will lead to larger capacity models than existing methods. It is not clear whether the performance gains are due to use such larger capacity models or not. \n\nFor the real noise images, why do the authors evaluate the proposed method on the microscopy and medical images? How about the results on the real-world natural noisy images? In addition, the proposed results still contain significant noise residual as shown in Figure 4.", "extra_scores": {}, "extra_text": {}}, {"review_id": "SFdfblBvjs4", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper proposes a framework to train a network to remove noises, which are zero-mean, additive and independent of the clean image, with only noisy images and without knowing the noise statistics. They mathematically prove that a network, which is trained from pairs of images generated by adding simulated noises into the noisy image, can remove noises from the input noisy image. The proposed framework can remove the noises from the input noisy image. Then, it adds simulated noises into the denoised image to generate a pair of images and train the next network to further remove noises and it does this process iteratively. The proposed framework follows GCBD and utilizes flat textureless regions to train a network to simulate noises and proposes a wavelet based method to effectively distinguish flat regions from the ones that contain high-frequency repeating patterns. The experimental results show that the proposed method has good performance under simulated Gaussian noises as well as WT and CT datasets.\n\nI have several concerns and suggestions for this paper:\n\n(1) One assumption of the proposed method is the noises and signal are independent which is not true for most of the scenarios. The raw images captured by the camera contain both Poisson noises. I am wondering if the proposed method can deal with Poisson noises or not. Also, the final displayed images processed by ISP contain signal dependent noises with spatial correlation. Can the proposed framework remove this kind of noises?\n\n(2) In Sec. 3.3, why does the framework add simulated noises into the network 'ground truth' Z_j1^i. Will it be better if we directly treat X_phi_{j-1}(Z^i) in Eq. (10) as the 'ground truth'?\n\n(3) As claimed above Theorem 1, f_Noisy N2N (Z, y) gives a better estimate of X than X' for a sufficiently large σ_0^2. What the results will be if σ_0^2 is not large enough? That is to say, can the proposed framework remove small noises from the input images?\n\n(4) Similarly, according to Theorem 1, the iterative process in Sec. 3.3 will converge if y_0<y<1.  Can the network still effectively remove noises if y<y_0 which means the residual noises are not significant in the images?\n\n(5) g_theta1 generates noises with given random vector r. How does g_theta3 not need a random vector to add simulated noises into g_theta2(Z)?\n\n(6) More analyses are needed in Sec. 4.4 for ablation study. Especially, why is sigmoid so important?\n\nSome descriptions are not clear and confusing:\n\n(7) I cannot understand the relationship between N2C with Eq. 4. \n\n(8) What is Unif[−s, s] in Sec. 4.2?\n\n(9) It is not clear the described dataset for Table 3 for training or testing.\n\n(10) For a fair comparison,  do the two N2C networks have the same network structure?", "extra_scores": {}, "extra_text": {}}, {"review_id": "H-ADO2pY3JN", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper addresses a challenging task of blind image denoising where a single noisy image is provided with assumption that it is zero mean, additive and independent from the original image content. This is mostly the real-world scenario. Different from the recent N2N training, the authors propose a GAN2GAN based method since this blind setting cannot be trained by N2N. N2N or deterministic training needs explicit or implicit knowledge of clean image in order to be trained whereas the GAN2GAN method does not, leading to more realistic and efficient training. This method first attempts to simulate noise given the noisy image, generate rough and noisy estimates of the clean image, and iteratively train a denoiser with synthetic noisy pairs from the generator. For blind denoising, this work produces impressive results for synthetic and real world blind denoising. \n\nThis work provides a sound approach to denoising given only the noisy image.\nThe experiments were done with extensive datasets and baselines.\nAlthough the paper is solid, I am not sure how this work compares to well known algorithms for blind image denoising:\n- Residual Dense Network for Image Restoration\n- High-Quality Self-Supervised Deep Image Denoising\n- CycleISP: Real Image Restoration via Improved Data Synthesis\n- Real Image Denoising with Feature Attention\njust to name a few.\nIf the authors can provide comparisons or discussion, analysis on these algorithms and comparison, it would be helpful. \n", "extra_scores": {}, "extra_text": {}}, {"review_id": "DQhR6ol6g8J", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 7, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "\nThis paper proposed a new method for blind image denoising using a \"GAN2GAN\" network. Different from previous work noise2noise, the proposed method only needs single noisy images to train the network, without noisy pairs. Given a noisy dataset, a GAN generator is trained using the real noisy but smooth patches. With the noise generator, a pair of generated noise samples are added on the noisy image content to train a denoiser. This is trained iteratively, so that the image content is better and better after cleaned by the trained denoiser. The final results on synthetic and real data showed improvements over the baselines.\n\nOverall, this paper is very interesting to read. It solves a harder problem than noise2noise when the pairs are not available. The idea of refining the noisy image iteratively is reasonable, which makes the image content for training progressively better. The results showed significant improvements over its competitor noise2void.\n\nThere are still some areas that it could improve:\n\n1. It is unclear the purpose of the cycle loss. Although it improves the PSNR, the motivation is not clear.\n\n2. The idea of generating synthetic noise for image denoising has been studied already. This should be discussed.\n\nAbdelhamed, Abdelrahman, Marcus A. Brubaker, and Michael S. Brown. \"Noise flow: Noise modeling with conditional normalizing flows.\" Proceedings of the IEEE International Conference on Computer Vision. 2019.\n\n3. It would be great to have natural images in the experiments to validate the idea. In particular, showing that the proposed method can reduce and noise and reveal more image content and texture.\n\nAbdelhamed, Abdelrahman, Stephen Lin, and Michael S. Brown. \"A high-quality denoising dataset for smartphone cameras.\" Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2018.\n\nPlotz, Tobias, and Stefan Roth. \"Benchmarking denoising algorithms with real photographs.\" Proceedings of the IEEE conference on computer vision and pattern recognition. 2017.\n\n4. It is good to have a table or flowchart about the iteratively training, to make the paper easier to read.\n", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: Summary of discussion: Three reviewers rated the paper Good (7) while Reviewer2 disagreed. R2's criticism was focussed on how this work is placed within existing/related literature, and no technical problem was identified. The authors have addressed some of R2's comments/concerns, R2 has not participated in the discussion.

Novelty and contributions: Overall the reviews seem consistent with an incremental paper which is technically valid, improves the state of the art on a reasonably difficult task. However, it does not appear from the reviews that the paper substantially advances our understanding of machine learning more broadly beyond this specific application.

Experiments: There is some disagreement among reviewers on the adequacy of the experiments, with at least two reviewers calling for experiments involving 'natural photos'. I believe the author's responses adequately address these concerns: they pointed out that the key selling point of their paper is the ability to model structured noise which is less relevant in natural photos.

On the balance of things, I think this paper should be accepted, but I wouldn't argue if it did not make the cut due to its narrow scope. For this reason, I recommended poster presentation.
Decision: Accept (Poster)