On the Analysis of GAN-based Image-to-Image Translation with Gaussian Noise Injection
Authors: Chaohua Shi
Kexin Huang
Lu GAN
Hongqing Liu
Mingrui Zhu
Nannan Wang
Xinbo Gao
Venue: ICLR 2024 poster (2024)
Keywords: Image to Image translation
Noise robustness
f-divergence
Score matching
GAN
Primary area: learning theory
TL;DR: Analyze the impact of Gaussian noise injection on the GAN-based I2I translation model.

Abstract: Image-to-image (I2I) translation is vital in computer vision tasks like style transfer and domain adaptation. While recent advances in GAN have enabled high-quality sample generation, real-world challenges such as noise and distortion remain significant obstacles. Although Gaussian noise injection during training has been utilized, its theoretical underpinnings have been unclear. This work provides a robust theoretical framework elucidating the role of Gaussian noise injection in I2I translation models. We address critical questions on the influence of noise variance on distribution divergence, resilience to unseen noise types, and optimal noise intensity selection. Our contributions include connecting $f$-divergence and score matching, unveiling insights into the impact of Gaussian noise on aligning probability distributions, and demonstrating generalized robustness implications. We also explore choosing an optimal training noise level for consistent performance in noisy environments. Extensive experiments validate our theoretical findings, showing substantial improvements over various I2I baseline models in noisy settings. Our research rigorously grounds Gaussian noise injection for I2I translation, offering a sophisticated theoretical understanding beyond heuristic applications.

Published as a conference paper at ICLR 2024
ON THE ANALYSIS OF GAN-BASED IMAGE-TO-IMAGE TRANSLATION USING GAUSSIAN NOISE INJECTION
Chaohua Shi'
Kexin Huang? Lu Gan Hongqing Liu
Mingrui Zhu
Nannan Wang* Xinbo Gao! 1.4
Xidian University
National University of Defense Technology Chongqing University of Posts and Telecommunication @hshi2004ggmaill.ooo com, hkxin91 outlook.com, lu.gan brunel.ac.ut mmzzuu, nnwang @ xidian.edu.cn, {honggingliu, xbgao @ equut.eeu.cn
3 Brunel University
ABSTRACT
Image-to-image (12I) translation is vital in computer vision tasks like style transfer and domain adaptation. While recent advances in GAN have enabled high-quality sample generation, real-world challenges such as noise and distortion remain sig- nificant obstacles. Although Gaussian noise injection during training has been utilized, its theoretical underpinnings have been unclear. This work provides a robust theoretical framework elucidating the role of Gaussian noise injection in 121 translation models. We address critical questions on the influence of noise variance on distribution divergence, resilience unseen noise types, and optimal noise intensity selection. Our contributions include connecting f-divergence and score matching, unveiling insights into the impact of Gaussian noise aligning probability distributions, and demonstrating generalized robustness implications. We also explore choosing an optimal training noise level for consistent performance in noisy environments. Extensive experiments validate our theoretical findings, showing substantial improvements over various 12I baseline models in noisy set- tings. Our research rigorously grounds Gaussian noise injection for 12I translation, offering a sophisticated theoretical understanding beyond heuristic applications.
INTRODUCTION
Image-to-imet image (12I) translation has seen remarkable advancements in recent years and has emerged as a thriving field within computer vision. In particular, models based on Generative Adversarial Network (GAN) have gained significant attention due their ability generate high-quality images and fast inference speed (Goodfellow et al., 2020). However, their performance significantly suffers when handling noisy or distorted inputs, as shown in Fig. The degradation of input image quality is a common occurrence in real-world scenarios, spanning from low-light conditions to data transmission through noisy channels (Anaya & Barbu, 2018; Plotz & Roth, 2017; Yue al., 2020; Zamir et al., 2020), underscoring a critical vulnerability intrinsic to 121 models.
To address this challenge, we explore simple and widely applicable approach for boosting the noise resilience of 12I translation models. This involves injecting isotropic Gaussian noise into source domain images during training, as shown Fig. Our research tackles three core questions:
How does the variance of Gaussian noise introduced during training impact the divergence between real and generated distributions?
How does the presence of Gaussian noise in training data influence the model's ability to handle unseen noise distributions and intensities during inference?
Is it possible to identify an optimal noise intensity during training that guarantees consistent performance across diverse noise intensities during inference?
Correspondence lu. iu.gan& brunel..cc.uk and nnwang xidian.edu.cn

---

Published as a conference paper at ICLR 2024
121 Translation Results
Task
X
121 Translation Task
Task 1
Task 2
Task 3
Noise
Disturbance
Noise Injection
Task
Task
Task 3
G
Figure I: Overview of our framework. When dealing with noisy inputs, the original I2I translation model will get failed results (top). By applying the Noise Injection the original model during training, we successfully improve the noise robustness of the original model (bottom).
Our contributions to this discourse are multifaceted: Firstly, we establish a novel connection between f f-divergence and score matching, shedding light on the implications of Gaussian noise injection on 121 model training. This provides valuable insights into how injected noise impacts the alignment of probability distributions. Secondly, we demonstrate that for arbitrary source signal, the robustness of 121 systems to Gaussian noise implies resilience to other noise types with a matched covariance matrix, underscoring the advantages of Gaussian noise injection for enhancing general robustness. Thirdly, our study addresses the selection of the optimal noise variance, ensuring stability across diverse independent and identically distributed (i.id.) noise forms. Experimental results showcase the sigmificant performance improvement achieved by the Gaussian noise injection technique, effectively reducing sensitivity noise in various 12I translation models operating in noisy environments.
2 RELATED WORK
In the realm of reliable 12I translation, Chrysos et al. (2020) introduced the Robust Conditional GAN (RoCGAN), featuring dual-pathway generator architecture with shared decoder. Empirical evaluations face super-resolution and inpainting tasks demonstrated RoCGAN's ability to produce consistent outputs even in the presence of noise and perturbations. However, the dual-pathway architecture's 's computational demands and prolonged training duration raise concerns, particularly regarding its adaptability to larger generative models where computational efficiency is crucial. Moreover, while Chrysos et al. (2020); Wang et al. (2021); Jia et al. (2021) also explored noise injection for GAN-based 12I, the research mainly focused on empirical findings, lacking theoretical analysis. Additionally, the simulation results were limited to supervised 121 with paired training samples, leaving uncertainties about its effectiveness for unsupervised 12I models.
On the theoretical front, studies on unconditional GANs (Arjovsky & Bottou, 2017; Jenni & Favaro, 2019) have demonstrated that noise injection during training can significantly improve learning consistency and mitigate issues like model overfitting. Recently, this approach has gained interest in adversarial defence, with theoretical solid justification and promising empirical results in boosting robustness and resilience (Cohen et al., 2019; Goodfellow et al., 2014; Madry et al., 2017; Pinot al., 2019; Lee al., 2019; Xie al., 2023; Yang al., 2023a; Dong Xu, 2023). Yet it is essential to note that these successes in classification do not directly translate to the complex task of 12I translation. The distinction profound classification tasks culminate discrete outputs, while I2I models generate entire images, introducing unique challenges.
Advancements in diffusion probabilistic models (Song et al., 2021; 2020; Dhariwal & Nichol, 2021) have also explored controlled Gaussian noise addition to input images during the diffusion process. These studies have shown promise in both supervised (Saharia et al., 2022b;a; Batzolis et al., 2021; Li et al., 2022) and unsupervised (Sasaki al., 2021; Choi et al., 2021; Zhao et al., 2022; Kwon Ye, 2022; Su et al., 2023) 12I tasks. However, as we will demonstrate later, these models also exhibit vulnerability noisy inputs, highlighting the need to explore noise-robustned strategies.
2

---

Published as a conference paper at ICLR 2024
e
y
e
(a)
(b)
For each clean image æ in the training set, it is augmented with i.idd Gaussian noise, resulting in æ. The generator then produces the corresponding output, y. (b) Inference Phase: A noise-corrupted test image, ă, serves as the input to the generator, synthesizing the image ý.
Figure System diagram. (a) Training Phase:
In summary, while Gaussian noise injection can enhance the noise robustness of I2I models, comprehensive investigation of its effectiveness, implications, and limitations is required. The main aim of this paper is provide theoretical understanding of Gaussian noise injection's 's role in boosting the robustness of I2I translation models, rather than introducing new network architectures.
3 THEORETICAL ANALYSIS
Consider an image æ E Rdd from the source domain X. the GNI (Gaussian Noise Injection) system of Fig. 2a, the GAN-based generative model G is trained by adding isotropic Gaussian noise & (0, of Ia) to each æ E X. This produces a noisy training image ã = x + E. and the corresponding output from the generator is G( During inference, noisy input image is represented as a = x + e, and the corresponding output is ý = G depicted in Fig. 2b. Unlike prior methods such randomized smoothing for classification, which inject Gaussian noise during both training and inference, our framework only adds noise during training. This ensures quicker inference, facilitating easy integration with numerous 121 systems.
In what follows, we utilize f-divergence (Polyanskiy. 2019) to study the influence of training noise variance, adaptability unseen noise, and the identification of optimal training noise intensity for consistent performance. For brevity, notations, definitions, and proof's can be found in the Appendix.
3.1 RELATION BETWEEN f DIVERGENCE AND SCORE FUNCTION
Aligning the probability distributions of accurate and generated data crucial to 121 translation, and misalignment can result in unrealistic results. In this context, we examine the influence of Gaussian noise injected into the source domain on this alignment. The theorem below describes how the f-divergence between these distributions varies with the noise variance.
Theorem 1. Let Рx, and Qx,y be two joint distributions on X' x y representing real data and the data generated by a model, respectively. Define X X + oN, where N (O, Ia) is standard d-dimensional isotropic Gaussian noise. Let Px, and Q represent the corresponding distribations after Gaussian noise injection with their respective probability densities p(Ø. y) and q(æ, y). For the generator function f. if its second order derivative f" exists and Dy (Px,Y Qx,x)
finite, then D, (Px,y y || Qx,x) satisfies
d Dy (Px, Qx,x) -
d
(I)
in which n)(02) represents the weighted mean square error between two score functions
no(0²) E
((a.u)
(1
( VV log.(a;.) v) V log q(ū, 4)
(2)
q(
where V log p(r, y) and V a log @( y) are the score functions p(r, y) and q( y), respectively.
The above theorem unveils how the rate of change of D, (P Q) concerning a² is portrayed through ny ()())). In the case of KL-divergence, where f(t) log we can derive that
(o²) TEREY V/ a log p(. y) V log ( y)
3

---

Published as a conference paper at ICLR 2024
identifying it the Fisher divergence between p(a, y) and q( y) (Lyu, 2012; Verdú, 2010).
For small values of a = di, a Taylor series expansion yields
of
(Px.y |l Qx.x) = Dy (Px+o.N.Y Qx+aNN + 2 n + d(o)).
(4)
Through optimization of the noise-injected term Dj (Px+a,N,Y || Qx+o,N.) for minimizing the divergence between p(r + o,,, w) and q(æ + N,N, y). the term (y(a)) tends to decrease. In the ideal scenario where p(2 + o,N - q(x + N.N.y).. the first two terms on the right side vanish, leading to D (Px.y Y Qx,y) aligning the noise- seatt-pertrrbed distributions during training, the model is guided to align the original, noise-free distributions well, which results in coherent 12I transformation.
o( Hence, by injecting Gaussian noise with small of and
While previous research in information theory and machine learning has explored the relationship between KL and Fisher divergences for marginal distributions (Verdú, 2010; Lyu, 2012; Kong et al., 2023), we extend this understanding to f-divergence of joint distributions. This broader view deepens our insight into divergence and noise injection, thereby fortifying the theoretical base for modelling and manipulating complex dependencies within the 12I translation framework.
3.2 PERFORMANCE ANALYSIS FOR MISMATCHED NOISY INPUTS
This subsection explores the system's 's capability to handle unseen noise during inference, as depicted in Fig. 2b. Following the conventions of Theorem 1, let X and Y represent the clean source and target domain random variables, and X, Y G(X) denote the noisy counterparts during training.
Considering a new noisy input X = X + E during inference, where E is independent of X with zero mean and covariance matrix of Ee, the corresponding output is denoted by Y G(X). Marginal distributions are denoted by Px. Px. Qy Qy. and joint distributions by Q y and Qx Using data- patte-rocessing properties in f-divergence (Polyanskiy, 2019), we have
12(2x ll (xxxx = (, Px). 11(2g Il Qx) s (Px Px).
(5)
The equations reveal insights for 12I models handling unseen noise. They show that the joint input/output divergence equals the input marginal divergence. The output divergence is bounded by this value, with equality under reversible model. Many 121 models exhibit near reversibility, approximating translation inversion. Paired 12I scenarios (Isola et al., 2017) span seasonal shifts, sketch-to-realistic image conversion, face resolution alteration, and medical image translations (c.g.. MRI to CT scans). Unpaired models like CycleGAN (Zhu et al., 2017) establish near reversibility through sycle-consistency losses. Leveraging this property, the input marginal divergence D, (Px Px) can estimate model behavior with unseen noise. For general signal sources, obtaining a closed- form expression of the f-divergence is challenging. But with Gaussian signal source, we can explicitly derive the KL-divergence, as shown in the following Lemma:
Lemma 1. Let X be d-dimensional random variable with normal distribution N (Hs, E.). Assume that it is corrupted by noise E with zero mean and covariance matrix Ee, independent of X. Denote plo?, Ee) 2 DEL Ex+EE|N((s E. + +o)la) Then, plo? Ee)
can be expressed as
1 2
2l0 = (( +
Ex)=
log log(2r) (T)) + log E2l + Tr (T
(6)
in which h(X + E) denotes the differential entropy of X E. Ei E. and Eg E. oo
Eq. (6) allows for a closed-form solution of KL-divergence for a Gaussian source corrupted by arbitrary noise. In particular, the lower bound of p(a? E.) is achieved when e follows a Gaussian
distribution N' ).), i.e., p(o?, E.) 2 Pg(o?, E.) with
Σ Σ
Pg Ee) = Tr E E log
d
This indicates that given the same Σ., non non-Gaussian noise yields higher KL-divergence, as Gaussian distribution maximizes the entropy. The next Theorem characterizes the behavior of p(o? E.):
4

---

Published as a conference paper at ICLR 2024
Theorem 2. Consider the KL-divergences denoted by p( Ee) in (6) for general noise, and Pg(o?, Σe) in (7) Gaussian noise. Under these definitions, the following properties hold:
1. Let Σ. = o. which Eg normalized covariance matrix with Tr(Ez) d. Then, p is convex with respect to o². Additionally, for small of with o < 1, the following approximation is valid:
pl o. Pglot, -.E =(00))
(8)
2. If di Is, the inequality p(o Σ.) < p(0 ).) is satisfied.
Part 1 of the theorem implies that the KL KL-divergence first decreases and then increases with respect to o², owing to the convex nature of p(a? -z) Specifically, for Gaussian noise with Σ; La. the global optimum
o of Furthermore, Eq. (8) indicates that non-Gaussian noise with small of leads to KL-divergence close to that of Gaussian noise with the same covariance matrix. Hence, an I2I system that robust Gaussian noise can also tolerate other types of noises with the same Σ.
Part of the theorem establishes comparison between a system trained with Gaussian noise injection and one trained only on clean images. Specifically, it asserts that for certain noise levels, as
characterized by E. o Ia. the system trained with noise injection can better handle noisy inputs compared to a system trained only on clean images. This not only underscores the practical advantage of noise injection, but also provides sound theoretical foundation for its effectiveness.
The next Theorem discusses the case for a non non-nnusseee Gaussaassi signal.
Theorem 3. Let X be d-dimensional random vector with an arbitrary probability distribution and finite entropy h (.)). Denote ((o? of Ez) DEL (xx+elIXxooNN) where the definitions of E, ΣS, N, and o² are the same as those in Theorem 2. Let lg(0 Ex) denotes the special case of Mot 2z) when E is Gaussian noise. Then,
1. For small of with o²
of 0g(07, og olo?):
(9)
2. When E is also iid Gaussian, 0, DKL (Pxto.NlIPxtanl satisfies
d 0 02 Ia) = -E(0)) { {-2 Va log log + V ((t) Va logp(t) (10)
Eq. (9) generalizes the result of Eq. (8) to non-Gaussian sources, demonstrating that an 121 system's resilience to Gaussian noise ensures robustness against noises with the same covariance matrices, regardless of whether the input signals are Gaussian non- non-Gaussian In the special case when E also iid Gaussian, one can get 6y (o" o² La)a) by integrating the right-hand side of Eq. (10). In d
particular, døz (o? Ia = 0 when o² = of. Hence, when o of is small, using Taylor series expansion of, we have lg(o) o(o? o?))
3.3 SELECTION OF TRAINING NOISE INTENSITY
For ii.d. inference noise with bounded o². the following corollary provides insights into choosing the optimal training noise level by either minimizing worst-case KL-divergence or the average KL-divergence given uniform noise variance distribution:
Corollary 1. Given an i.i.d. input noise e with Σ. = o² la and a bounded variance 0 < o² Xmax define oi as the optimal noise level that minimizes the worst-case KL distance plo? La):
oi. = arg min max plot (of,o?1a)) L. OSS3SMM
(11)
o?
For this optimal level, it satisfies p(o7.0, 0a) plo7.00 Amxx la). Besides, of is uniformly distributed between O and Amas, iee.o2 ~ (1, Amax), the optimal training noise intensity of, that minimizes
5

---

Published as a conference paper at ICLR 2024
the average KL-divergence is 2mmas i.e.,
0 = arg min B02- of 1
> Amax-
(12)
This corollary offers theoretically sound method for determining the optimal training noise variance for an arbitrary type of i..dd inference noise. It implies that to determine of in Eq. (11), one can initiate the search at ot = 2max and proceed numerically until the condition p(o 02) = p(o Amax holds true.
EXPERIMENT
4.1 EXPERIMENTAL SETTINGS
Baselines & Datasets: We have employed the Gaussian noise-injected training methodology in various 121 translation models and contrasted them with their original baselines. Specifically, 1. HiFaceGAN (Yang et al., 2020), AAAN-based 121 model primarily utilized for Face SuperrRReolution task on real-life facial photographs:
GP-UNIT (Yang et al., 2023b), a generative prior-based image translation model for converting images between unpaired data domains, such as Cat->Dog 3. Sketch Transformer (Zhu et al., 2021), a Transformer-based photo-sketch paired transfer model. During training, we follow the default settings of each baseline model and use the corresponding datasets: FFHQ (Karras et al., 2019) (HiFaceGAN), AAHQ (Choi et al., 2020) (GP-UNIT), and CUFS (Wang et al., 2018) (Sketch Transformer). For more details, please refer to Appendix C.
Evaluation Metrics: In line with prior research (Karras et al., 2019; Yang et al., 2023b; Zhao et al., 2022), evaluation primarily relies the widely adopted Fréchet Inception Distance (FID) (Heusel et al., 2017) and Kernel Inception Distance (KID) (Binkowski et al., 2018) to assess the disparity between generated images and target datasets regarding their respective distributions. Additionally, for 121 translation models trained with paired datasets, we incorporate Learned Perceptual Image Patch Similarity (LPIPS) (Zhang et al., 2018) and Peak Signal-to-Ne to-iose Ratio (PSNR). This metric enables more accurate image similarity measurement, aligning with human perception.
To create noisy images, we first normalize pixel values the range [0, 1]. Next, we add noise and clip the pixel values to stay within [0, 1]. The pixel values are then rescaled to [0, 255) for image file creation. During training, zero-mean, isotropic Gaussian noise with of 0.04 (unless specified otherwise) is introduced. In the inference stage, we evaluate three models under five signal-independent noise types (Gaussian, Uniform, Color, Laplacian, and Salt & Pepper), each with six intensity levels applied to all input images. Furthermore, we explore the performance noises, blur, JPEG compression, and other corruptions, as modelled in Imagenet-C (Hendrycks & Dietterich, 2018). Additional details available in Appendix C, Table 4.
Degradation Settings:
under
4.2 EXPERIMENTAL RESULTS
Qualitative and Quantitative Evaluations: Tables and show the quantitative results of Cat-DDog and Photo->Skette translations, respectively, while Figs. 3 and 4 provide the corresponding qualita- tive results. Our empirical findings, obtained under the default setting of o/ = 0.04, closely mirror the insights from our theoretical analysis. First, as suggested by Theorem the Gaussian noise-injected model demonstrates the ability to effectively align noise-influenced distributions. ensuring a coherent transformation between source and target domains. outperforms GP-UNIT on Cat-+Dog translation across all settings, including clean inputs (Table 1). For Photo->Sketch, only marginal objective degradation is observed on clean data, which is nearly visually imperceptible. Second, while the baseline method excels on clean images, it experiences a significant decline under noisy conditions. In contrast, GNT method demonstrates remarkable resilience across diverse noise types and intensities. Though Theorem 2 considers Gaussian signals, experiments indicate Gaussian injection provides noise robustness even for non-Gaussian images. In summary, simulation results substantiate the theory of aligning noisy/clean distributions and showcase generalized noise robustness.
Fig. 3 also compares the noise injection approach to DiffuselT (Kwon & Ye, 2022) on Cat->Dog translation. Despite relying isotrophic Gaussian denoising. DiffuselT's performance drops sub- stantially under colored Gaussian noise. In contrast, GP-UNIT with GNI exhibits superior robustness across all noise types.

---

Published as a conference paper at ICLR 2024
Table 1: Quantitative comparison on the Cat->Dog image translation task (reference-guidedl) In the reference- guided approach, we randomly select 10 target domain images from the test set as additional guides and synthesize total of 5000 images. Furthermore, we employ the FID and x000xKID metrics to evaluate.
Noise Intensity
Noise Type
Metric
Method
Cleun
S2
S3
$5
S6
Baseline
17.82
18.28
18.95
20.82
22.13
26.95
36.84
Gaussian Noise
FID 4
+, 0.04)
16.47
16.21
16.24
16.23
16.19
16.26
16.31
KID 4
Baseline
7.08
7.91
8.79
10.53
11.35
15.48
23.03
(U, 0.04)
5.35
5.15
5.17
5.23
5.41
Baseline
17.82
19.17
21.19
22.31
28.42
43.31
FID 1
Uniform Noise
+, 0.04)
16.47
16.13
16.16
16.21
16.32
16.27
16.15
Baseline
7.08
7.98
8.81
10.76
11.45
16.42
28.83
KID 4
+U 0.04)
5.35
5.06
5.09
5.23
5.26
5.27
5.43
Baseline
17.82
18.48
19.94
22.91
25.11
31.85
47.26
Color Noise
FID 1
(0, 0.04)
16.47
16.31
16.15
16.21
16.19
Baseline
7.08
8.31
9.68
12.09
14.02
19.33
31.78
KID 1
+W"(U, 0.04)
5.35
5.09
5.26
5.13
5.16
5.36
5.22
Baseline
17.82
18.39
19.06
20.38
21.13
24.77
30.95
Laplacian Noise
FID
+0, 0.04)
16.47
16.16
16.13
16.12
16.15
16.14
16.19
Baseline
7.08
8.72
9,99
13.49
18.67
KID 4
+. 0.04)
5.35
5.07
5.15
5.16
5.17
5.26
5.27
Baseline
17.82
18.62
19.97
22.41
25.68
29.85
35.11
Salt & Pepper Noise
(0, 0.04)
16.47
16.11
16.28
16:26
16.08
16.07
16.99
KID 4
Baseline
7.08
8.49
9.62
11.76
14.58
17.93
21.99
(0, 0.04)
5.35
5.23
5.26
5.19
5.18
5.29
Table 2: Quantitative comparison on Skeech image translation tasks. For photo-to-sketch, we evaluate sketch synthesis using 338 test photos from the test set.
Noise Typo
Metrie
Method
Noise Intensity
Clean
S1
S2
53
S4
S5
$6
Gaussian Noise
FID J.
+Niia, 0.04)
Buseline
64.12
31.49
34.53
53.33
69.88
31,25
108.26
31,16
124.72
32.39
228.02
40.95
404.01
64.87
LPIPS L
Baseline +NV(0, 0.04)
0.3055
0.3991
0.4397
0.5123
0.5385
0.5994
0.6525
0.3601
0.3186
0.3129
0.3119
0.3192
0.3411
0,4043
Baseline +.V(0, 0.04)
31.49
53.14
70.36
112.59
133.71
250.89
431.58
Uniform Noise
FID
64.12
34.94
79.62
Baseline +Niia, 0.D4)
0.3055
0.4003
0.4423
0.5182
0.5445
0.6098
0.6641
LPIPS 4
0.3601
0.3185
0.3134
0.3161
0.3204
0.3488
0.4316
Baseline 0.DD)
226.99
400.63
448.24
Color Noise
FID 4
64.12
32.52
31.71
36.71
42.19
71.65
135.09
LPIPS L
Baseline (.(000
0.6931
0,3601
0.3194
0,3188
0.3227
0.3454
0.4125
0.5102
Buseline
31.49
51.48
68.01
101. 188
115.63
181.81
321.31
Laplacian Noise
FID J
+Nii,, D.D4)
64.12
34,93
31.37
31.11
32.18
38.26
51.61
LPIPS |
Baseline
0.3055
0.3965
0.4337
0.5001
0.5248
0.5835
0.6341
+.((0,0.0)) 0.04)
0.3601
0.3191
0.3134
0.3101
0.3169
0.3321
0,3702
Pepper
FID
+0 0.04)
Baseline
64.12
31.49
69.78
31.99
110.24
162.21
37.42
258.34
365 98
58.03
443.48
74.25
Noise
LPIPS
+.V 0.04)
Baseline
0.3055
0.3601
0.3149
0.4351
0.3187
0.5201
0.5777
0.3322
0.6199
0.3876
0.6481
0.3908
0.6671
0.4303
Ablation study of training noise intensity: We conduct an ablation study of of for the photo-to- sketch translation task. Fig. 4 shows the comparison visual qualities. Fig. 5 further illustrates the FID metrics for generated images by varying of with different types of i.i.d. noise at the inference stage. Here, the setting of "Leamnale of treats of as a tuned hyperparameter. We observe that of controls the balance between robustness and quality low values favor cleaner inputs, while high values prioritize noisy cases. This highlights the need to select an appropriate of balancing performance across expected conditions. With maximum o² 0.16 in these simulations,
7

---

Published as a conference paper at ICLR 2024
Clean
Severity
Source
Ref erence
Source Reference
Source
Re Referrnce
Figure 3: Noise injection method compared to DiffuselT (Kwon & Ye, 2022) generation on the Cat->Dog task. The baseline model GP-UNIT, and the noise environment Color noise. In each pair of source and reference image comparisons, the first row is the result produced by the baseline model, the second row is produced by DiffuselT, and the third is produced by the noise injection.
Corollary states the optimal of minimizing average KL-divergence is 0.16/2 0.08. Numerical FID results in Table (Appendix D) confirm that of = 0.08 yields the smallest average FID.
It is interesting note that although Photo->Sketch, the FID scores in Fig. 5 exhibit (near-) convexity in o². which follows a very similar pattern as theoretical KL-divergence in Fig. 6(a). Besides, for i.i.d. noise, Part 2 of Theorem indicates traiend systems can outperform -tained ones noisy inputs given of > 0.5g, FID scores in Fig. 5 agree with this theory. This empirically validates the value of our theoretical analysis for anticipating how models respond to unseen noises.
analysis uses the KL-divergence, for supervised
Due to space constraints, we provide more results Appendix D, which includes theoretical results of Gaussian mixture model (GMM) signal sources, further simulation results contrasting empirical and thcoretical findings, results on various types of image corruptions, out-of-domain tests, and comparisons between noise injection training and pre-denoising approaches. We also discuss theoretical and implementation limitations, highlighting certain failure scenarios.
Additional Results:
5 CONCLUSION
In this paper, we investigated the challenge of noise resilience in GAN-based 12I translation models, focusing the impact of injecting isotropic Gaussian noise into source domain images during training. By establishing a novel connection between ff f-divergence and score matching, we illuminated how Gaussian noise influences the alignment of probability distributions. We then demonstrated that for
8

---

Published as a conference paper at ICLR 2024
Glese
Gaussian
Serenity
Cloo
Color Noise
Seoerity
Eaput
001
000
-000000
Cleum
Uniform Noise
Semmnil
Cleam
Laplacian Noise
Semerity
Tapı
001
000
di
000
Figure 4: Photo Input photos with noise types, each 6 levels from clean to high severity, and their corresponding sketch outputs.
160
160
160
140
140
140
120
120
120
100
100
100
60
60
60
40
0.05
0.15
0.05
0.1
0.15
0.05
0.1
0.15
o2
o3
o2
(a) Gaussian Noise
(b) Uniform Noise
(c) Laplacian Noise
0.01
of
0.04
oz
0.08
0.16
Learnable of
Figure 5: FID score comparison on noisy input images for models trained with different Gaussian noise levels. Test Noise Types: (a) Gaussian (b) Uniform, (c) Laplacian.
arbitrary signal sources, the robustness of 12I systems to Gaussian noise implies resilience to other noise types with matched covariance matrix. Additionally, we addressed the selection of an optimal training noise variance. Extensive experimentation has validated our insights, showing a substantial performance improvement across diverse 12I translation tasks in noisy environments. Our results highlight the usefulness and efficiency of Gaussian noise injection for enhancing model robustness. This work offers valuable perspective into leveraging noise for more resilient 121 systems. Overall, it represents an important step towards reliable I2I translation in real- world noisy environments through rigorous theoretical grounding.

---

Published as a conference paper at ICLR 2024
ACKNOWLEDGEMENTS
We thank Dr. Cong Ling from Imperial College London and Dr. Yan Zhang from Chongqing University of Posts and Telecommunications for their invaluable assistance, support, and insightful feedback on this paper. We want to thank the three anonymous reviewers who spent a lot of time and effort and raised constructive questions and suggestions, which immensely helped us improve the theory and experiments of the paper. We would also like to thank the Program Chairs and Area Chairs for their approval and processing of this paper and providing valuable and comprehensive comments.
The theoretical framework for this work was primarily established during Chaohua Shi and Kexin Huang's undergraduate research within the CQUPT-Brure University London joint program. Mean- while, this work was supported in part by the National Natural Science Foundation of China under Grants U22A2096, 62036007 and 62106188 84; in part by the Fundamental Research Funds for the Central Universities under Grants QTZX23042 and YJSJ2401 I; in part by the Young Talent Fund of Association for Science and Technology in Shaanxi China under Grant 20230121; in part by the Youth Innovation Team of Shaanxi Universities; in part by the Technology Innovation Leading Program of Shaanxi under Grant 2022QFY01-15 and in part by the Innovation Fund of Xidian University.
10

---

Published as a conference paper at ICLR 2024
REFERENCES
John C Amazigo and Lester A Rubenfeld. Advanced calculus and its applications to the engineering and physical sciences. (No Title), 1980.
Josue Anaya and Adrian Barbu, Renoir dataset for real low-light image noise reduction. Journal of Visual Communication and Image Representation. 51:144-154, 2018.
Martín Arjovsky and Léon Bottou. Towards principled methods for training generative adversarial networks. In ICLR, 2017.
Georgios Batzolis, Jan Stanezuk, image generation with score-based diffusion models, Nov 2021.
Schönlieb, and Christian Etmann. Conditional
Mikolaj Binkowski, Danica J Sutherland, Michael Arbel, and Arthur Gretton. Demystifying mmd gans. arXiv preprint arXiv: 1ttf:11801.01111,,, 401, 2018.
Jooyoung Choi, Sungwon Kim, Yonghyun Jeong. Youngjune Gwon, and Sungroh Yoon. Ilvr: Condi- tioning method for denoising diffusion probabilistic models. in 2021 ieee. In CVF international conference on computer vision (ICCV), pp. 14347-14356, 2021.
Yunjey Choi, Youngjung Uh, Jaejun Yoo, and Jung- Woo Ha. Stargan v2: Diverse image synthesis for multiple domains. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 8188-8197, 2020.
Grigorios G Chrysos, Jean Kossaifi, and Stefanos Zafeiriou. Rocgan: Robust conditional gan.
International Journal Computer Vision, 128:2665-2683, 2020.
Jeremy Cohen, Elan Rosenfeld, and Zico Kolter. Certified adversarial robustness via randomized smoothing. In international conference on machine learning, pp. 1310-1320, PMLR, 2019.
Max Costa. A new entropy power inequality, IEEE Transactions on Information Theory. 31(6): 751-760, 1985.
Max Costa and Thomas Cover. On the similarity of the entropy power inequality and the brunn- minkowski inequality (corresp.). IEEE Transactions Information Theory, 30(6):837-839 1984.
Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. Advances in neural information processing systems, 34:8780-8799 2021.
Minjing Dong and Chang Xu. Adversarial robustness via random projection filters. In Proceedings of the JEEE/VFF Conference on Computer Vision and Pattrnn Recognition, pp. 4077-4086, 2023.
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial networks. Communications the ACM, 63(11):1399144 2020.
Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.657 4226572, 2014.
Dongning Guo. Relative entropy and score function: New information-estimatii relationships through arbitrary additive perturbation. In 2009 IEEE International Symposium on Information Theory, pp. 814 818. IEEE, 2009.
Dan Hendrycks and Thomas Dietterich. Benchmarking neural network robustness to common corruptions and surface variations. arXiv preprint 1107.v1807.01699 2018.
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash cquilibrium. Advances in neural information processing systems, 30, 2017.
Phillip Isola, Jun Yan Zhu, Tinghui Zhou, and Alexei A Efros. Image-to-ine -mmagg translation with conditional adversarial networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1125-1134, 2017.
11

---

Published as a conference paper at ICLR 2024
Simon Jenni and Paolo Favaro. On stabilizing generative adversarial training with noise. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12145-12153, 2019.
Zhiwei Jia, Bodi Yuan, Kangkang Wang, Hong Wu, David Clifford, Zhiqiang Yuan, and Hao Su. Semantically robust unpaired image translation for data with unmatched semantics statistics. In Proceedings of the IEEF/CVF International Conference on Computer Vision, pp. 14273-14283, 2021.
Takuhiro Kaneko and Tatsuya Harada. Noise robust generative adversarial networks. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8404-8414, 2020.
Tero Karras, Samuli Laine, and Timo Aila. A style-based generator architecture for generative adversarial networks. In Proceedings of the IEEE/CVF conference on computer vision and paltern recognition, pp. 4401-4410, 2019.
Xianghao Kong, Rob Brekelmans, and Greg Ver Steeg. Information theooeiciinntheooeiiccc diffusion. ICLR, 2023.
Gihyun Kwon and Jong Chul Ye. Diffusion-based image translation using disentangled style and content representation. arXiv preprint wwX.v:2209.15266 2022.
Guang-He Lee, Yang Yuan, Shiyu Chang, and Tommi Jaakkola. Tight certificates of adversarial robustness for randomly smoothed classifiers. Advances in Neural Information Processing Systems, 32, 2019.
Bo Li, Kaitao Xue, Bin Liu, and Yu-Kun Lai. Bbdm: Image-to-image translation with brownian bridge diffusion models, May 2022.
Siwei Lyu. Interpretation and generalization of score matching, arXiv preprint arXiv: 2205.6629, 2012.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. arXiv preprint 2017.
Ymir Mäkinen, Lucio Azzari, and Alessandro Foi. Collaborative filtering of correlated noise: Exact transform-done variance for improved shrinkage and patch matching. IEEE Transactions on Image Processing, 29:8339-8355 2020.
Aleix Martinez and Robert Benavente. The ar face database: Cve technical report, 24. 1998.
Kieron Messer, Jiri Matas, Josef Kittler, Juergen Luettin, Gilbert Maitre, et al. Xm2vtsdb: The extended m2vts database. Second international conference on audio and video-based biometric person authentication, volume 964, pp. 965-966. Citeseer, 1999.
Frank Nielsen and Richard Nock. On the chi square and higher-order chi distances for approximating f-divergences. IEEE Signal Processing Letters, 21(1):10-13, 2013.
Miquel Payaró and Daniel P Palomar. Hessian and concavity of mutual information, differential entropy, and entropy power linear vector gaussian channels. IEEE Transactions on Information Theory, 3613):361333628 2009.
Rafael Pinot, Laurent Meunier, Alexandre Araujo, Hisashi Kashima. Florian Yger, Cédric Gouy- Pailler, and Jamal Atif. Theoretical evidence for adversarial robustness through randomization. Advances in neural information processing systems, 32, 2019.
Tobias Plotz and Stefan Roth. Benchmarking denoising algorithms with real photographs. In Proceedings of the IEEE conference computer vision and pattern recognition, pp. 1586-1595. 2017.
Yury Polyanskiy. Information theoretic methods in statistics and computer science, 2019. URL https: / pwwple...... mids.mit........................ http:////honeepgg////atttteeeeee wttp//iiiiiiiii///////a//a//ieve online lec- ture notes, available at http://ppp..llii.mmmccuyyhhnnnng fdiv.pdf.
12

---

Published as a conference paper at ICLR 2024
Olivier Rioul. Information theoretic proofs of entropy power inequalities. IEEE transactions information theory, 57(1):33-333 050-33555520 2010.
Chitwan Saharia, William Chan, Huiwen Chang, Chris Lee, Jonathan Ho, Tim Salimans, David Fleet, and Mohammad Norouzi. Palette: Image-to-image diffusion models. In ACM SIGGRAPH 2022 Conference Proceedings, pp. 1-10, 2022a.
Chitwan Saharia, Jonathan Ho, William Chan, Tim Salimans, David J Fleet, and Mohammad Norouzi. Image super-resolution via iterative refinement. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(4):4713-4726 2022b.
Hiroshi Sasaki, Chris Willcocks, and Toby Breckon. Unit-ddpm: Unpaired image translation with denoising diffusion probabilistic models. arXiv preprint arXiv:2104.05358 2021.
Yang Song, Jascha Sohl-Dickstein, Diederik Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. arXiv preprint arXiv:2011.134456, 2020.
Yang Song, Conor Durkan, lain Murray, and Stefano Ermon. Maximum likelihood training of score-based diffusion models. Advances in Neural Information Processing Systems, 34:1415-1422 2021.
Xuan Su, Jiaming Song, Chenlin Meng, and Stefano Ermon. Dual diffusion implicit bridges for image-to-image translation. In International Conference on Learning Representations, 2023,
Xiaoou Tang and Xiaogang Wang. Face sketch synthesis and recognition. In Proceedings Ninth IEEE International Conference on Computer Vision, pp. 687-694. IEEE, 2003.
Sergio Verdú. Mismatched estimation and relative entropy. IEEE Transactions on Information
Theory, (6(8))371-)337777 2010.
Lidan Wang, Vishwanath Sindagi. and Vishal Patel. High-quality facial photo-sketch synthesis using multi-adversariin networks. In 2018 13th IEEE international conference on automatic face & gesture recognition (FG 2018). pp. 83-90. IEEE, 2018.
Xintao Wang, Liangbin Xie, Chao Dong, and Ying Shan. Real-esrgan: Training real- world blind super-resolution with pure synthetic data. In Proceedings of the IEEF/CVF international conference on computer vision, pp. 1905-1914, 2021.
Jiahao Xie, Chao Zhang, Weijie Liu, Wensong Bai, and Hui Qian. Towards optimal randomized strategies adversarial example game. Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pp. 10490-10498, 2023.
Jungang Yang, Liyao Xiang, Pengzhi Chu, Xinbing Wang, and Chenghu Zhou. Certified distributional robustness on smoothed classifiers. IEEE Transactions on Dependable and Secure Computing, 2023a.
Lingbo Yang. Shanshe Wang, Siwei Ma, Wen Gao, Chang Liu, Pan Wang, and Peiran Ren. Hifacegan: Face renovation via collaborative suppression and replenishment. In Proceedings of the 28th ACM international conference on multimedia, pp. 1551-1560, 2020.
Shuai Yang, Liming Jiang, Ziwei Liu, and Chen Change Loy. Gp-unit: Generative prior for versatile unsupervised image-to-image translation. IEEE Transuctions on Pattern Analysis and Machine Intelligence, 2023b.
Zongsheng Yue, Qian Zhao, Lei Zhang, and Deyu Meng. Dual adversarial network: Toward real- world noise removal and noise generation. In Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part 16, pp. 41-58. Springer, 2020.
Syed Waqas Zamir, Aditya Arora, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, Ming Hsuan Yang, and Ling Shao. Cycleisp: Real image restoration via improved data synthesis. In Proceedings of the IEFF/CVF conference on computer vision and pattern recognition, pp. 2696-2700 2020.
13

---

Published as a conference paper at ICLR 2024
Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 586-595, 2018.
Min Zhao, Fan Bao, Chongxuan Li, and Jun Zhu. Egsde: Unpaired image-to-image translation via energy-guided stochastic differential equations. Advances in Neural Information Processing Systems, 15:3609-3623, 2022.
Jun- Yan Zhu, Taesung Park, Phillip Isola, and Alexei Efros. Unpaired image-to-image translation using cycle-consistent adversarial networks. In Proceedings of the IEEE international conference on computer vision, pp. 2223-2232, 2017.
Mingrui Zhu, Changcheng Liang, Nannan Wang, Xiaoyu Wang, Zhifeng Li, and Xinbo Gao. A sketch- transforrre network for face photo-sketch synthesis. In I/CAI, pp. 1352 1358, 2021.
14

---

Published as a conference paper at ICLR 2024
A NOTATIONS AND DEFINITIONS
Notations: In this paper, capital letters indicate random variables or vectors, while lowercase letters represent their realisations. For random vector X, h(X) and J(X) denote its differential entropy and Fisher information matrix, respectively. For two random vectors X and Y. [(x; ; Y) corresponds their mutual information. The notation N (M, Σ) represents multidimensional normal (Gaussian) distribution with mean and covariance matrix Σ. For matrix A. Tr(A and |A| denote its trace and determinant, respectively. For symmetric positive-definite matrix A, A₂ (A) represents the i-th largest eigenvalue of A. For two real-valued symmetric matrices A and B, the notation A > B (or A B) indicates that A (or B A ) positive definite.
Definition of f-divergence: The f-divergence belongs to class of statistical metrics designed to quantify the discrepancies between two probability distributions. Let P and Q be distributions on measurable space X' with density functions p and g, respectively. If P < Q, the f-divergence between these absolutely continuous distributions defined as (Polyanskiy, 2019):
(P())) [ q())
p(z) ((x)
dx,
(13)
In this formula, f is strictly convex, continuous function satisfying ((1) = 0, and ís referred to as the generator function. Popular f-divergences and their corresponding generator functions are outlined in Table 3.
Table List of f-divergences Dy and corresponding generator functions (Nielsen Nock, 2013).
Name
(p())
Generator f(t)
Kullback-Leibler
DKL p(z) log q(z)
plz) dr
llog (1-e)a
Neyman x²
D2= = I (o(x)-p(a)) dx
q(a))
t
Total Variation
Drv = / 1x(x)= -x(x]]]] dz
/ - 1|
Squared Hellinger
DH= (Vp(z) - -v q(())) dr
(Vi - 1)²
Jensen-Shannon
DJSD DEL | - DEL (0) 5
(t + 1) log 1 + flog
Definition of small o: Let {(o?) be function o. We say {(o?) o(o?) as o² approaches 0 {(a() 0. This notation means that {(o?) becomes insignificant relative to o?
and only if: lim,g-+0 o as o² tends towards O.
B THEORETICAL PROOFS
B. PROOF OF THEBREM 1
The theorem's proof leverages the heat equation and Green's identities in vector calculus, which is similar to the proof of entropy power inequality in (Costa & Cover, 1984; Costa, 1985).
Proof.
Note that Eq. (2) can be expanded as
(P(()
d
f( v)
q(0, y)
dædy
d
7(5. v)
100
00
0q(a) σ y)
p( y)
8
p( y)
+ g ga m)
dady
d
q(2. w)
Do
g(x,, y)
0q(a, y)
p( y)
p y) 7 y)
p(æ, y)
Op(e,
(p(.,)) w) dødy. 7( y)
.f
2
7(
7 y)
(14)
15

---

Published as a conference paper at ICLR 2024
Using heat equation in diffusion, we know that
@p( y)
0ī( v)
1
V
Vap(a, y)
and
Vx
Vnq(a, y)
OG²
Og²
2
where V is the gradient operator with regard and Va Va Limi
#
This leads to
am?
d (PII))
da
p(a,y)
Va Vaj(a,)) v) -
dody
(15)
7(x, v)
+000
V V V y) [
p(a,, y))
p(a.y) / [((2 y)
(( y) ]
+
dædy
((., y)
q( y)
For simplicity of presentation, we set p = p(a, y), = q y). As g (ghY - -ghh, the above can be simplified (Guo, 2009)
V
f
V
V
9
9
(16)
E V m
qf
V ap- f"
p
v
# f"
p
9
For the first term, it can be simplified by Green's formula:
x /
Ø (
dædy = ( /Va.0/ as ( .s Va
/ qf
P
.d = 0, (17)
where S' piece-wiss smooth, closed, oriented surface in Rd. Since ( is finite, the above result can be obtained using a similar argument as Eq. (B.15) in (Costa, 1985).
The last two terms can be reduced to:
Vap 1d
P
P V 9
1" ②
(2)
- Vv
(
8"
--9"
75"
V =
(18)
Organizing the previous derivations, the final result can be written in the following form:
d D)(P((
- FA
pla,, y)
V
p(a, Vap (( v)
7( v)
(19)
-
1 E
p(a.y) y) pd
((..
|V log y) Va log v/
7(
((5,))
which completes the proof.
B.2 PROOF OF LEMMA
Proof. As shown in Table KL--divergence the special case of of /-divergence with S(1) (log in Eq. (13)
Dki(P1Q) -f p(z) In
p(z)
dr.
(20)
g(z)
Assume that ((t) is the probability density function of Px. the KL-divergence (KI(Px1 ( (M,, Σ. La)) can be expanded as
DxI(Px [K,,, E. + Ial)
= -(())) - / ((t) log (20))_/² d/2 det (Ez) (E2)-1/2_ exp
1 (ē u.)) En' (a u.) de (21)
--N(X) + d
2 log(2xr + log Ezl - ((0) (- (t ue))" Σn (t m.) dē,
2
16

---

Published as a conference paper at ICLR 2024
in which h(z) is the differential entropy of ã. The last term can be further simplified as
-f ((t) (-2
1 ( uoo)" Ez' (Ø= (so) dē
1 أン E (0- ue))" (0 k.))
Tr (a - -.))²² Σの (# - (a-1)))
2
(22)
ززز Στ}
where Σ, = E ( us)² (00 [n.)] Hence, Eq. (6) is proved.
B.3 PROOF OF THEOREM 2
Proof. Part 1 of the proof is based on the concave property of mutual information derived in (Payaró & Palomar, 2009), while Part 2 is on matrices trace and determinant properties.
. Part 1:
Proof of p(o? o dzEn'ss convexity in o²:
For E = o. E, with covariance matrix E. = 0. Eg. Eq. (6) gives:
d
plo7 oZ 1(X + 0 E) log(27) log Σ2
(23)
1 + 2
(T -.) + 0²
Tr [Σ² Ez)
2
in which Eg = =E, + L2 The term
c² Ir - is evidently convex in o² 2
To demonstrate the convexity of -h( (X + E) in of given X is Gaussian and independent of E, we express it as:
o. -h(X -o -
+ 12g(2 + log X|,
where 7(X E; F) denotes the mutual information between X +o, E and E. Given X is Gaussian, it was previously established in Corollary 1 of (Payaró & Palomar, 2009) that the mutual information 1(X + o. E; E) is concave with respect to o² This implies that - [(x + ag E; E) and therefore, -h(X + E) is convex in a². Remark: The work in (Payaró & Palomar, 2009) assumes Gaussian noise with an arbitrary signal. In contrast, our analysis considers a Gaussian signal with arbitrary noise. As a result, in our context, of serves the role analogous the SNR in Corollary 1 of (Payaró & Palomar, 2009).
- Proof of Eq. (8):
can be derived that plo?, E.) Po(ot Ex) 1 1( + Eg) - - + E)
(24)
= =h(X G.E.)) - M + E)
According to the generalized De Bruijn's Identity (Proposition in (Rioul, 2010))
d 1((X + E)
1 Tr (J
(25)
do?
2
in which (XX the Fisher information matrix of X. Hence, for small de. using first-order Taylor series expansion as in Eq. (37) of (Rioul, 2010), one can obtain
0² T (J())E)) +
h(X M
(26)
2
and similarly,
h(X - o. E,E.) - h(x) + 0²
Tr (J(X)E;,) + +oo0))
(27)
2
When Σ₁ = Σx,, we have h(x + o. Eyy - h h(x + ..E) E = o(o2). Combined with Eq. (24), Eq. (8) proved.
17

---

Published as a conference paper at ICLR 2024
Part 2: Given Eq. (6), the difference Py Σe) p(o?; Ee) can be expressed as:
P00, Ex) Pg(o?; Ex)
log 3 + o log |
(28)
+ 2 12, (E, + E.)) (E, -1 (E, + E.))
The proof of p(0,Ee) 2 pl Σe) when E. 2 o is equívalent to showing
Σ, + of
Tr, + ).) Tr(( o + E+)) log
(29)
E
Bounding the left-hand side (LHS) of Eq. (29), we have:
E.))
=
= - +
o
( E + La
+
Tr [E, + o -1)
2
(30)
of 2
TM + of Tr((2, + +0,10)
d
of
1 d
o
Σ
+
2
2 i=1
Σ
+ o
1=1
of ,
+1 1+ f 1+
2
where the inequality at (a) arises from Σe right-hand side of Eq. (29) can be written
2 2
Ia and E > (Eg + o2
-1 The
Σ, + of
1 of
log
log
Σg
(E,)
1=1
Next, consider the following function
1
(<x)
2 x +
log(1 + +x).
1 x
It is obvious that when x = 0, (0) = and dw(a) is strictly increasing for positive implying that
x
> 0 for x > O. Thus, (x))
de
21+27²
+1 1+ 1+07/>(E)) -1-+++1//M(M)) )
1
(1+ of >x(E))
> log
2 (EE)
This establishes Eq. (29) and completes the proof.
Example: We evaluate an AR(1) signal model with d 256 and its signal covariance matrix given by (k, o². k-11 (for 0 < k, < d 2d-17 1). Parameters o? 0.125 and p 0.95 are derived from the normalized CUFS dataset. Fig. 6 illustrates Pg (a? o Σe) across two different Σz- In Fig. 6(a), i.i.d. noise is considered with Σn Ig. Meanwhile, Fig. 6(b) shows non-i.i.d. noise with Σg = diag(1166a. 1.2 0.8 0.4 All curves display convex behavior in o2 which agrees with Part 1 of Theorem 2. Besides, as evident in Fig. 6(a), Pg o. Ia) (g o² Ia) whenever of 0.50?, reaffirming Part 2 of Theorem 2.
18

---

Published as a conference paper at ICLR 2024
Aglof.of2i)
500
300
KL divergence
Kl divorgence
200
200
160
150
0.05
0:15
iid Gaussian Noise
(b) Goussion Noise
0o 0
o
0.01 03
0.04
0/ 0.08 0 0.16
Figure 6: Visualization of Py(af, -a for AR(1) signal model with d = 256 and covariance matrix
E. (k,, l) = 0. 0.95plk-1| (for 0 < k, l < 255). (a) i.i.d noise with Σ; = -a- (b) Non-i.i.d. noise with
Σg diag(1.6 1.2 0.8 0.4
B.4 PROOF OF THEOREM 3
Proof. The previous proof in Theorem 2 is based on the assumption that the source signal is Gaussian distributed. Here we further consider the non- non-Gaussian signal with arbitrary noise.
Part 1
Expanding the KL-divergence,
000i DKL. ElttoEllPxxoonl
(31)
= + oe E) - f p( + o.E) log + N, da.
According the Lemma in (Guo, 2009)
d p( o E)
o,=0 = 2 VeV((a))
do?
where p(æ) is the probability density function of the clean data, and thus d p(x + o. E) log + o, dæ
V vn log + (32)
d
The first-order Taylor series expansion of the integral part of Eq. (31) can be derived,
p(a a. E) logø(z + N) dæ
(33)
p(x) logp(æ + N) dx + 02 V log +o,N) + o(02).
By gathering Eq (26) and Eq. (33), the Taylor series expansion of the KL-divergence
= - h(x) Tr (J(X)=) - / p(x) log + N) dar
2
(34)
v V. a( + o,N) o(o
2
The KI. divergence for Gaussian noise can be expanded according to the Taylor series expansion in Proposition (Rioul, 2010) as
0, of DKI.
PP (Px+o ll
(35)
82 J(X) + +(82).
2
19

---

Published as a conference paper at ICLR 2024
in which de oe 8. Obviously the A(0) dg only relates to the covariance matrix Le of the arbitrary noise instead of the noise distribution. When Σ; Σey- one have 0iot,o? Ez) = 0g (o d² Ee) + o( and thus Eq. (9) is proved.
. Part 2:
Derive of of o daI))
d
(36)
+00 p()) log (()) dæ.
h(x o.
d
d -00
Then we calculate derivatives one by one.
d
h X + No
d
+30
dop doz dá - (gσ2)(() (1) log dæ
(37)
+00
(va((t)) -0 2 oc
log (()) dā,
in which d 1980) if (xx) and (xx) twice continuously differentiable functions in R and if V is any set bounded by piecewise smooth, closed, and oriented surface S in R", then
26 p()) = 12 According to Green's 's identity (Amazigo & Rubenfeld,
[ VVAaaV - ov - do / VØ dV
(38)
where Veh denotes the gradient of t/, ds denotes the elementary area vector, and Vú ds is the inner product of these two vectors. This identity plays the role of integration by parts R" To apply Green's ' identity to Eq. (37), we let V, be the n sphere of radius r Green's ' identity
V,, and S, with á (i) = log P (œ) and vp (Ø) = P (Φ) and take the limit as r - cc. Hence we obtain
centered at the origin and having surface S,.. Then
d X + N. = - +o0 d
V
V log da
(39)
1
| V dé.
p())
The second term in Eq. (36) can further simplified by using Green's identity in Eq. (38),
do (()) log ((z) dx = V Va log dx.
(40)
Therefore, substituting Eq. (39) and Eq. (40) into Eq. (36),
d d 0
1
+
L Vap(t)Va log dæ
dē +
2 /
(41)
20000
2
-o0
p(t)
1
=
=Ep(a)
V. log (1
Volog((²) lologp())
20

---

Published as a conference paper at ICLR 2024
B.5 PROOF OF COROLLARY
Proof. We first prove that of satisfying plo 0a) p(o Amax is the optimal solution of
Eq. (11). For ii.d. inference noise with E. = o. it can be easily derived from Eq. (6) that
Öp(oř. Ee) do? 2
1
Eg - Li
1 ((Es -o, '(E + 0 oa
do²
2
0o²
2
2 + A₂ E,
Hence, we know that for fixed o the function plo? oaIa) is decreasing in of when of o² and increasing for of > o²--
Therefore, for of in the range of of or > 0, at of = 0, have p(o plo Oz). Likewise, for of of < Amax. at o² Amax, we have pl Amax La) p( Amx As result, oi is the optimal solution of Eq. (11).
Next, we derive the expression of t.o that minimizes the average KL-divergence when o² is uniformly distributed between 0 to Amax- Let ((o?) = ((3--((0)))e))) {p(of, o. From Eq. (6), we can obtain
Amax
oo do?
Amax
Ammax
Tr [T + La (Eg + [a + log Eg + oZLa llog E, + +ooLol
d do
2Amax J
1
[2E + Amax Is) (0 Σ,, + log Σ, + o +a,
4
where a - fo log Eg L d do is constant. Taking the derivative of d(a
d
p(a)))
do
1
Tr [2Σ, + Xmax Ia) (0 + +E,) |]
log Σ. + 07
do? 4
do 2
Tr [2Σ, 2X mama 1a) (0 Is E,) 2| Tr [I, + o3La)) -1
Tr (207 - >max) oo
As [E + and an increasing one if of Ammax. Therefore, the minimal value of o(o²) is achieved when of Amaax
(o.+o o, > 0, the above result implies that (0o;) ) is a decreasing function when of < Amax
B.6 EXTENDING TO GAUSSIAN MIXTURE MODELS (GMM) SOURCES
Gaussian Mixture Models (GMM) are often extended to fit a vector of unknown parameters. Given that numerous natural image signals can effectively represented by GMM, we therefore posit the fitting of the source signal through GMM assumption.
Consider a source signal represented by the Gaussian mixture model that px(x)) ΣN m M( Ex), EN= n and πV > Although we cannot get closed-for forme expression of the KL divergence for the GMM signal source, we can extend our results to its upper-bound, as listed below:
Extention of Lemma 1: By using the convexity property of K L--divergence, have DRL. (ex (Px+El ell Px+.,)) S ((o?, Ex).
21

---

Published as a conference paper at ICLR 2024
((o) M.)
Σ Th D
(42)
N Th pn 2e),
where is d-dimensional random variable the GMM represented source signal æ, Xx is the k-th Gaussian distribution with its weight тı for GMM and pk(o?. Σe) is the KL- divergence k-th Gaussian distribution for arbitrary noise. The above equation implies that for GMM fitted source signal, its Kl.-divergence concerning arbitrary noise is upper- bounded by the weighted sum of the KI--divergence of Gaussian source signals with respect arbitrary noise.
Extension of Theorem 2: For ((o De) given above, when E. =0? C(o,: Σe) is also convex in oe as each (a Ee) is convex in o? Likewise, ((o?: Σe) < ((0. Σ,) when
Σ. > /2/2.
Extension of Corollary 1: Under the assumptions of Corollary 1, the optimal solution
i0o
arg min (10 o
(43)
of
is still o7.00 = Amax, which remains the same as that of the Gaussian source.
C EXPERIMENT SETTINGS
C.I BASELINES
We have employed the Gaussian noise-injected training methodology in various image-to-image translation models and contrasted them with their original baselines. Specifically,
HiFaceGAN' (Yang et al., 2020), GAN-based I2I model primarily utilized for Face Super- Reselutisn task real-life facial photographs, was tested with the FFHQ dataset, specifically on a 16x (32 - 512) environment.
GP-UNIT (Yang et al., 2023b), a generative prior-based image translation model for converting images between unpaired data domains, was subjected to the Cat->Dog image translation task on the AFHQ dataset.
Sketch Transformer' (Zhu et al., 2021), 1), a Transformer based image translation model, was evaluated on its ability to convert photo-sketch paired data from the CUFS dataset in the Photo=+Sketch image translation task.
C.2 DATASETS
We validate the GNI method on various datasets the baseline models presented below: (1) FFHQ natural face dataset (Karras et al., 2019). It comprises 70000 high-quality facial images. We selected the 10000 images with the lowest serial number for training, while the final 1000 images were used for testing. We perform a 16x face super-resolution task on this dataset, where the HR resolution is 512x, and the LR resolution is k. (2) AFHQ animal dataset (Choi et al., 2020). contains high-resolution images of animal faces, including cats, dogs, and wild animals, from three domains with substantial variations. Each domain comprises 500 test images. We perform the Cat->Dog image translation task on this dataset. (3) CUFS face sketch dataset (Wang et al., 2018). It contains 188 identities from the CUHK student database (Tang & Wang. 2003). 123 from the AR database (Martinez & Benavente, 1998), and 295 from the XM2VTS database (Messer et al., 1999). We perform the image translation task on this dataset.
httpp://githubbco com/l Sataayuu/aac-Rennovaa hhtpss//github... www.giiuu..oo./maae http:/lllmmagg1999GGPP... shared by the authors
22

---

Published as a conference paper at ICLR 2024
Table Parameter configuration for noise intensity in the test set for testing I2I translation models.
Noise
Blur
Digital
Noise Type
Gaussian 22
Unilorm 6
Color
Laplacian <2
Sallee Density
Shox Density
Speckle Density
Glass Density
Delixcss Radius
Pixelate Density
JPEG
Quality
0.01
0.01
0.01
0.15
1
1
0.5
2
0.02
0.02
0.02
0.02
0.10
40
0.25
2
2
1.0
25
Intensity
4
0.05
0.05
0.05
0.05
0.20
12
0.45
4
5
2.0
15
6
0.16
0.16
0.16
0.16
0.30
0.65
7
7
3.0
C.3 DEGRADATION
We considered multiple types of image degradation, each with different intensities from weak to strong, detailed in Table 4. The parameter configurations for noise, blur and digital degradation based on those used in (Hendrycks & Dietterich, 2018) for ImageNetC. For colored noise, we adopt strategy similar to (Kaneko & Harada, 2020) by applying 2D Gaussian filter to i.i.d. Gaussian noise with a standard deviation of 0.5 and a x 7 window size. To generate noisy images, we start by normalizing the pixel intensities fit the [0, 1] first. Then, after introducing the noise, ensure the pixel values remain bounded within the same range by a clipping operation. After that, we adjust the pixel values the [0, 255) range produce an image file.
C.4 SETTING OF of
For i.i.d. noise, the peak inference noise level in our simulation Amax 0.16, indicated in Table 4. According to Corollary 1, the optimal value to minimize the average KL-divergence is ot, Amax/2 0.08. This value is noise eypeinndepeneent and straightforward compute. In our simulations, we set of 0.04 by default. Several reasons motivate this choice over 0.08:
Model integrity for clean Images: Eq. (4) suggests a smaller of ensures the model's efficacy on noise-free images. A larger value risks training noise bias, undermining its performance on clean data.
2. Near optimal Min-Max Value for Gaussian Noise: As depicted in Fig. 6(a), the optimal of from Eq. (11) to minimize worst-case KL-divergence falls below Amax/ /2. Specifically, of 0.04 stands as a near-optimal min-max solution.
3. Visual quality vs. KL-divergence: While KL-divergence offers statistical insight, it might not reflect the visual quality of translated images. Simulations suggest of 0.04 strikes balanced robustness-qualtt trade-off.
C.5 TRAINING
During our training process, we follow the default settings of each baseline model, only substituting clean images in the source domain with their noisy variants. Consequently, both training duration and memory requirements remain unchanged. Drawing insights from prior work on unconditional GANs, it's suggested that incorporating noise into training images improve convergence and enhance training stability. Indeed, our experiments, all three without any instances of model collapse-a frequent challenge in GAN training. Importantly, with noise intensity of o? = 0.04, the Gaussian noise-injected model does not bias the noisy inputs, and they demonstrate good performance clean inputs well, will show in the next Section.
12I models achieved convergence
D ADDITIONAL RESULTS
D.1 QUANTITATIVE RESULTS AND COMPARISON WITH THEORETICAL ANALYSIS
In the Cat->Dg translation task, the model, trained using Gaussian noise injection, shows superior noise robustness over the baseline across five different noise scenarios, as highlighted in Table (latent-guided). Even with significant noise interference, the GNI approach consistently delivers
23

---

Published as a conference paper at ICLR 2024
Garseian Noise
65
Unikerm Noior
Toplacionn
KL. Divergence
55
45
0.05
0.1
0.05
0.15
o]
o2
(a) Sinmlation
(b) Therretical Prediction
Figure 7: Comparison results of face super-resolution (a) FID results. (B) Theoretical results of KL-divergence with Gaussian noise.
stable results. Besides, Table 6 demonstrates that the noise-injected model outperforms the baseline popular evaluation metrics like PSNR and FID for face super-resolution tasks. These results echo our theoretical analysis on the following aspects:
Despite training only with noisy source images, the Gaussian noise-injected model effectively handle clean inputs, manifesting only slight objective result reductions. This consistent with insights from Eq. 4, as expounded Theorem
Clean inputs:
2. Resilience various types of Noises: Part of Theorem 2 indicates that for Gaussian signal sources, resilience to Gaussian noise implies robustness to other noise types with a matched covariance structure. Despite natural images not being strictly Gaussian, the model trained solely on isotropic Gaussian noise demonstrates efficiency against varied non- Gaussian distortions like Laplacian, uniform, colored Gaussian, and even erasure noises like Salt & Pepper. Experiments empirically exhibit this generalized capability gained from Gaussian noise injection.
3. Comparison with Baselines: Training with a noise intensity of of 0.04 (specifically at intensity level S3), Part 2 of Theorem suggests the Gaussian noise-injected model should surpass clean-trained counterparts for noisy inputs when the inference noise intensity satisfies - > 0.5a, 0.02. As anticipated, the Gaussian noise- coise-injected model excels for inputs with i.i.d. noises (e.g., Gaussian, Uniform, Laplacian) at noise intensity level $2 (corresponding to of = 0.02) and above, validating the theoretical expectations.
In addition, for the face super-resolution task, Fig. further compares the FID results vs. of against the theoretical KL-divergence Pg (0.04, of [a) predictions for Gaussian noise. To compute Pg (0.04, Ta. we employ an AR(1) model for the FFHQ data with d = 256 and covariance matrix The parameters of = 0.16 and 0.9 are
defined as (k, 1) o3 p k-1 for 0 < 1 - 1. informed by the normalized FFHQ dataset. Remarkably, the FID trends closely resemble the derived KL-divergence, Pg (0.04, of La). presenting convex nature in terms of o². With small o² values, the performance remains consistent acros various i.i.d. noises. This parallels the insights from Part of Theorem regarding KL-divergence.
In Table 7, we present average FID scores for Photo->Sketch task with different training noises variances of. The simulation results in this table demonstrate that of 0.08 yields the lowest average FID scores. This empirical finding closely aligns with the prediction Theorem 2, thus providing strong empirical support for theoretical analysis.
While our simulation results align in many respects with theoretical predictions, there are notable discrepancies between simulation outcomes and theoretical anticipations:
1. Gaussian Source Assumption: Lemma asserts that for a Gaussian source trained via Gaussian noise injection, Gaussian noise during inference would result in the smallest KL-divergence. Contrary this, Fig. reveals that Laplacian noise corruption often yields superior objective results. This might stem from the inherently non-Gaune Gaussian nature of image signals. Furthermore, we clip all test image pixel values to the range [0, 1) produce noisy
24

---

Published as a conference paper at ICLR 2024
Table 5: Quantitative comparison on the Cat->Dog image translation task. the latent-guided approach, randomly generate 10 latent codes additional guides, and synthesize total of 5000 images. Note: the random seed remains fixed at 777.
Noise Intensity
Noise Type
Metric
Method
Clean
S2
Baseline
22.82
22.35
24.29
26.12 12
26.98
31.33
40.34
FID
Gaussian Nnise
(0, 0.04)
25.04
23.81
22.89
22.29
22.21
Baseline
9.67
10.61
11.78
13.92
14.78
18.46
27.11
KID
(0, 0.04)
9.89
9.14
8.52
Baseline
22.82
23.61
24.28
26.42
27.74
32.42
45,38
FID
Uniform Noise
(0, 0.04)
25.04
23.72
22.87
22.32
22.21
21.97
21.71
Baseline
9.67
10.81
11.78
13.94
15.66
20.19
30.56
KID
+U, 0.04)
9.89
9.09
8.55
8.21
8.12
7.91
Bascline
22.82
23.47
24.85
27.49
29.17
35.31
48.61
FID
(0, 0.04)
25.04
23.57
22.86
22.45
22.19
22.13
22.11
Color Noise
Bascline
9.67
11.03
12.51
15.19
16.89
22.08
33.98
KID
(0, 0.04)
9.89
9.01
8.46
8.23
7.99
7.96
8.02
Baseline
22.82
23.51
24. 18
25.66
26.42
29.65
35.11
FID
0.004)
25.04
23.79
22.97
22.47
22.36
21.92
21.76
Laplacian Noise
Baseline
9.67
10.83
11.71
13.33
14.02
17.18
22.12
KID
+.V(0, 0.04)
9.89
9.18
8,58
8.25
8.09
7.79
7.74
Bascline
22.82
23.86
26.42
27.53
31.67
35.91
41.11
FID
Salt & Pepper
+0 0.000)
25.04
23.92
22.51
22.19
22.02
21.91
22.01
Noise
Baseline +O 0.04)
9.67
10.61
11.78
13.92
14.78
18:46
27.11
KID
9.89
9.14
8.52
8.09
7.99
7.82
7.88
Table 6: Quantitative comparison on the Face Super-Resolution image translation task. For face super-resolution, we adopt the configuration provided by HiFaceGAN (Yang et al., 2020), explicitly selecting the final 1000 face images from the FFHQ dataset as our designated testset.
Noise Intensity
Noise Type
Metric
Method
Clean
$1
S2
S4
S6
Baseline
34.83
96.21
122.48
166.94
191.21
263.96
320.41
FID
Guussian Noise
+0, 0.04)
43.94
37.15
36.62
36.98
38.04
46.81
60.01
Baseline
22.09
20.09
19.76
19.01
17.35
16.24
15.11
PSNR
+0, 0.04)
20.43
21.23
21.65
21.59
21.14
20.51
19.77
Baseline
34.83
95.74
124.22
173.31
197.33
279.81
329.63
Uniform Noise
FID 4
(0, 0.04)
43.94
37.09
36.79
37.01
38.21
42.27
66.96
Baseline
22.09
20.11
18.99
17.69
17.25
16.06
24.76
PSNR
(0. 0.04)
20.43
20.98
21.41
21.84
21.89
21.35
19.75
Baseline
34.83
99.91
130.08
182.57
208.83
282.85 822.82
328.49
Color Noise
FID 1
(.. 0.04)
43.94
37.91
37.39
36.81
40.11
48.99
65.11
PSNR
(0 0.04)
Baseline
20.43
22.09
20.95
19.98
21.31
21.74
17.56
21.83
17.25
21.71
20.94
14.95
Baseline
34.83
93.73
117.37
155.33
172.15
235.65
292.31
Laplacian Noise
(0. 0.04)
43.94
37.76
37.01
36.76
37.88
43.82
51.51
Baseline
22.09
20.21
19.21
18.04
17.67
16.69
15.72
PSNR
(0. 0.04)
20.43
20.95
21.31
21.74
21.83
21.71
20.94
Baseline
34.83
112.66
153.43
201.41
250. 88
291.66 62
317.24
Sult & Pepper
FID
(0, 0.04)
43.94
39.33
41.25
44.58
50.54
$7.21
64.78
Noise
Baseline
22.09
19.38
17.98
17.01
16.23
15.54
14.94
PSNR
(. 0.04)
20.43
21.23
21.65
21.59
21.14
20.51
19.77
25

---

Published as a conference paper at ICLR 2024
Table For Photo Sketch image translation task, average FID scores comparison on noisy input images of different intensities for models trained with different Gaussian noise levels.
Noise Injection of
Noise Type
0.01
0.04
0.08
0.16
Learnable
Gaussian Noise 182.55
78.89
42.17
36.18
36.64
59.19
Uniform Noise 199.17
101.68
45.37
36.79
36.89
64.81
Laplacian Noise
152.09
64.12
39.17
36.99
37.01
52.41
Table 8: More image degradation test comparisons Photo >Sketch task.
Intensity
Type
Metric
Method
Clean
S1
S2
S3
S4
S5
S6
FID
0.04)
Baseline
64.12
31.49
47.33
38.93
54.74
35.07
32.49
64.79
30.85
91.46
219.11
39.41
385.72
$8.06
Noise
Baseline
0.3055
0.3851
0,4036
0.4286
0.4848
0,5885
0.6434
Noise
LPIPS
+.V(0,0.01)
0,3601
0.3329
0.3267
0.3212
0.3196
0.3459
0.3954
Speckle
FID 4
0.01)
Baseline
64.12
31.49
42.91
41.14
34.44
58.76
32.22
84,59
125.21
34,08
194.61
41.16
270.52
51.73
Noise
Baseline
0.3055
0.3684
0.4186
0.4721
0.5213
0.5637
0.5955
LPIPS
0.100.004)
0.3601
0.3453
0.3338
0.3216
0.3321
0.364
0,3858
Baseline
31.49
31.72
48.45
52.53
64,82
83,08
91.04
Glass
FID
+.
64.12
63.39
51.59
52.01
53.47
57.18
58.54
Blur
Baseline
0.3055
0,3061
0.3512
0.3621
0.3869
0.4111
0.4231
Blur
LPIPS
0.04)
0,3601
0.3589
0.3521
0.3511
0.3559
0.3645
0.3704
Defocus Blur
FID
0.01)
Baseline
64.12
31.49
64.51
32.17
68.65
36.38
45.86
70.91
77.95
52.59
84.21
64.22
92.11
92.82
Baseline
0.3055
0.3098
0.3232
0.3516
0.3678
0.3901
0.4256
LPIPS
0.04)
0.3823
0.3931
0.4213
Baseline
31.49
39.77
57.61
77.33
89.87
108.11
145.32
Pixclate
0.04).0.0)
64.12
62.59
64.04
67.14
74.08
85.41
100.01
Digital
LPIPS
0.04)
Baseline
0.3601
0.3641
0.3671
0.3684
0.3987
0.3752
0.4341
0.3738
0.3869
0.4611
0,4011
0.4901
Baseline
31.49
34.79
36.02
36.89
41.16
47.39
71.35
FID 4
0.04)
64.12
62.51
61.22
60.63
59.21
55.52
53.72
Baseline
0.3I55
0.3192
0.3236
0.3301
0.3571
0.3699
0.4291
LPIPS
0.04
0.3631
0.3653
0.3665
0.3786
test images. Given the extended tails of Laplacian noise, this clipping could inadvertently reduce noise levels.
2. FID vs. KL-Divergence in 121 Tasks: For supervised 12I tasks such photo-to-sketch translation and face super-resolution, FID scores largely mirror KL-divergence trends. However, in the unsupervised Cat->Dog translation via the GP-UNIT model, FID results diverge from theoretical KL calculations. This discrepancy might arise because, apart from the source image, GP-UNIT integrates an additional latent code reference image for 121 translation. In contrast, our theoretical framework solely contemplates the source images as inputs. As noise intensities escalate, FID and KID metries for GP-UNIT models improve.
comprehensive analysis of these results, especially the intriguing behavior of GP-UNIT models under increased noise, will be the subject of our future investigations.
D.2 QUANTITATIVE RESULTS OF OTHER IMAGE DEGRADATION MODELS
We also assessed the effectiveness of Gaussian noise injection in handling other image degradation models outlined in Imagenet-C. C. These models include range of degradations, including signal- dependent noises such as shot and speckle noises, as well as various blurring operators and digital operations like JPEG compression and pixelation (as detailed in Table 4).
26

---

Published as a conference paper at ICLR 2024
Table 9: More image degradation test comparisons Dat task (latent-guided)
Intensity
Type
Metric
Method
Clean
S1
S2
53
54
S5
S6
Baseline
22.82
24,42
23.74
24.31
26.09
30.91
37.51
FID 4
Shot Noise
[0, 0.04)
25.04
24.03
23.44
22.85
22.39
21.76
21.88
Baseline
9.67
10.86
11.09
11.59
13.49
18.27
24.51
KID
Noise
+O 0.04)
9.89
9.31
8,98
8,46
8.12
7.79
7.91
Baseline
22.82
23.36
24.25
25.41
26.98
29.35
31,53
Speckle Noise
+0 0.04)
25.04
24.25
23.02
22.37
22.08
21.99
21.91
Baseline
9.67
10.69
11.64
12.97
14.53
16.97
19.15
KID
+O 0.04)
9.89
9.46
8.61
8.37
7.96
8.07
7.92
Bascline
22.82
22.85
23.91
23.04
23.12
23.51
23.98
Glass
+O'0, 0.04)
25.04
25.06
25.09
25.18
24.89
24.61
24.51
Blur
Baseline
9.67
9.69
9.82
9.86
10.06
10.44
10.56
Blur
(o 0.04)
9.89
9.88
9.92
10.03
9.98
9.91
9.77
Baseline
22.82
23.08
23.41
24.71
25.71
26.52
27.13
Defocus Blur
4
(0 0.04)
25.04
25.92
26.01
26.02
25.91
25.69
25.45
Baseline
9.67
9.81
9.98
10.71
11.31
11.97
12.45
KID
+N[0,0.00 0.044
9.89
10.44
10.61
10.67
10.59
10.48
10.29
Baseline
22.82
22.89
23.31
23.42
23.85
23.91
24.01
FID 4
+."00, 0.04)
25.04
25.29
25.84
26.01
25.81
25.71
25.69
Pixclate
Baseline
9.67
9.72
9.82
10.06
10.21
10.44
10.91
KID
Digital
+. 0.04)
9.89
10.03
10.35
10.51
10.54
10.45
10.42
Baseline
22.82
22.91
22.95
23.01
23.26
23.41
24.36
FID
+. 0.04)
25.04
25.19
24.94
25.21
25.41
25.49
25.53
Bascline
9.67
9.79
10.01
10.07
10.35
10.53
12.24
KID
+. 0.04)
9.89
10.03
9.77
9.96
10.11
10.25
10.31
Table 10: More image degradation test comparisons on Cat Cat->Dog task iiferrecce-uideede
Intensity
Type
Metrie
Method
Clean
S1
S2
S3
S4
S5
S6
Baseline
17.82
18.36
18.39
19.08
20.71
26.51
34.07
Shot Noise
FID
+. 0.04)
16.47
16.29
16.21
16.06
16.19
15.97
16.01
Baseline
7.08
8.04
8.06
8.72
10.03
14.61
21.08
KID
Noise
+O 0.04)
5.35
5.23
5.13
5.01
5.21
5.09
5.16
Baseline
17.82
18. 18.34
19.09
20.41
22.41
24.55
27.45
FID 1
Speckle Noise
+0[(0, 0.04)
16.47
16.39
16.11
16.03
16.04
16.07
16.15
Baseline
7.08
7.95
8.61
9.82
11.46
13.31
15.51
KID
(O 0.04)
5.35
5,25
5.11
5.08
5.12
5.26
5.27
Baseline
17.82
17.89
17.91
17.96
17.99
18.06
18.11
Glass Blur
+N 0.04)
16.47
16.45
16.44
16.46
16.55
16.48
16.47
Baseline
7.08
7.09
7.23
7.31
7,33
7.46
7.48
KID
Blur
+N 0.04)
5.35
5.34
5.32
5.33
5.44
5.38
5.39
Baseline
17.82
17.84
17.91
18.46
18.92
19.01
19.36
Defocus Blur
1
+N 0.04)
16.47
16.74
16.81
16.79
16.69
16.68
16.55
Baseline
7.08
7.09
7.11
7.15
7.53
7.86
8.31
+O(00, 0.04)
5.35
5.54
5.61
5.57
5.53
5.56
5.45
Baseline
17.82
17.86
17.93
17.94
17.96
17.99
18.16
FID 1
[N 0.04)
16.47
16.46
16.66
16.79
16.67
16.64
16.69
Pixelate
Baseline
7.08
7.09
7.12
7.17
7.26
7.43
7.89
Digital
+O 0.004)
5.35
5.32
5.48
5.62
5,54
5.56
5.58
Baseline
17.82
17.85
17.85
17.96
18.06
18.49
19.91
FID 1
+O 0.04)
16.47
16.49
16.43
16.53
16.54
16.69
16.74
JPEG
Baseline
7.08
7.10
7.25
7.53
7.56
7.91
9.56
KID 1
(O 0.04)
5.35
5.37
5.31
5.32
5.34
5.38
5.51
27

---

Published as a conference paper at ICLR 2024
Table 11: More image degradation test comparisons on Face Super-Resolution task.
Intensity
Type
Metric
Method
S2
54
Baseline
86.23 23
98.06
114.55
162.75
Shot Noise
FID
+N 0.04)
43.94
37.61
37.21
36,97
39.11
47.25
59.41
Baseline
22.09
20.56
20.04
19.32
18.07
16.62
15.76
PSNR
Noise
+."[0, 0.04)
20.43
21.02
21.16
21.29
21.32
20.43
19.45
Baseline
34.83
78.13
109.01
147.72
191.09
225.86
256.43
Speckle
FID4
+. 0.04)
43.94
38.25
38.15
39.33
43.71
49,43
56.81
Noise
Baseline
22.09
20.85
19.47
18.23
17.28
16.59
16.06
PSNR
(.,00,
20.43
20.99
21.19
21.21
20.63
20.18
19.75
Baseline
34.83
34.86
35.89
36.12
37.45
40,55
42.41
FID
Glass
(0, 0.04)
43.94
43.91
43.85
43.84
43.81
43.78
43.56
Blur
Haseline
22.07
22.05
22.03
21.98
21.89
21.87
Blur
PSNR1
0.004)
20.43
20.43
20.45
20.46
20.52
20.59
20.62
Baseline
34.83
34.84
34,88
34,91
34.96
35.01
35.06
Defocus
FID4
+0 0.044)
43.94
43.95
43.88
43.75
43.71
43.68
43.59
Blur
PSNR
Baseline
22.09
22.08
22.05
22.01
21.95
21.94
21.91
4., 0.04)
20.43
20.42
20.46
20.56
20.63
20.73
20.81
Baseline
34.83
34.93
35.68
35.86
38.31
38.89
40.78
FID4
0.044
43.94
44.01
43.64
44.02
43.95
43.97
43.96
Pixelate
Baseline 0.044
22.09
22.05
22.01
21.97
21.94
21.91
21.88
PSNR
Digital
20.43
20.52
20.62
20.67
Baseline
34.83
38.56
39.56
42.01
46.51
$6.16
81.39
FIDA
+,N(0,0.04
43.94
44.19
44.42
44.71
45.11
47.81
57.32
JPE
Baseline 0.004)
22.09
21.96
21.92
21.84
21.33
21.35
20.04
PSNRI
20.43
21.98
21.72
21.51
21.41
21.47
21.11
Table 12: Robustness evaluation to multiple image degradations on the Cat-DDgg image translation task (latent- guided). Note: +Noise! and +Noise represent using Gaussian and Shot noise to perturb the degraded image.
Blur
Digital
Mixing
Setting
Glass Blur
Defocus
Pixelate
JPBG
Glass Blur Pixelate
Defocus Blur JPBG
Baseline
23.12
25.71
23.85
23.26
26.35
25.93
Noise Injection
25.41
25.86
25.99
Basclinc+
26.98
28.99
27.81
26.75
28.51
29.51
Noise Injection Noise
22.21
22.75
22.53
22.41
22.44
22.61
Baseline+ Noise?
26.24
27.31
26.56
26.01
27.11
27.44
Noise Injection Noise?
22.67
22.84
22.81
22.49
23.05
22.75
The objective results for all three tasks are summarized in Tables 11. Table further provides results for combinations of these degradation models. Our key findings from these tables are as follows:
For corruptions involving signal-dependent noises alone, models trained with GNI consis- tently demonstrated substantial improvements over baseline models trained solely on clean images, mirroring those observed with indepedddde noises.
In cases where degradations included blurring, pixelation, JPEG compression without additional noise, models trained with GNI were occasionally outperformed by the baselines, particularly at low medium degradation intensities.
In combined degradation models, whenever the noise component was present (e.g., JPEG+noise Blur+noise). models trained with GNI consistently exhibited significant enhancements over the baseline models, as shown in Table 12. This observation is par- ticularly relevant real-world scenarios where noises are often present, emphasizing the effectiveness of GNI in 121 tasks.
Notably, as demonstrated in Table 12, in cases where the corruption model excluded the noise component, the addition of a small amount of noise the input images proved effective in producing
28

---

Published as a conference paper at ICLR 2024
Source
Clean
Severity
Baseline
+.V (0,0.04)
Figure 8: Comparison of reference-guided generation for Cat->Dog under the interference of Gaussian noise. Each source image is guided using two reference images. On the left is the result obtained by the baseline (GP-UNIT), and the right is the result obtained by applying noise injection to the baseline model.
outputs with stable quality. These findings provide further insights into the robustness and potential of Gaussian noise injection in 121 for various degradations.
D.3 ADDITIONAL QUALITATIVE RESULTS
In this subsection, we present additional comparative findings. Precisely, Figs. 8 through 12 depict the results of the Cat-:Dog image translation task when subjected to Gaussian noise, Uniform noise, Color noise, Laplacian noise, and Salt & Pepper noise. Meanwhile, Figs. 13 to 17 reveal results from the same noisy environment, guided by latent factors. Furthermore, Figs. 18 through 22 display the translation outcomes of the Face Super-Resolution image translation task when exposed to Gaussian noise, Uniform noise, Color noise, Laplacian noise, and Salt & Pepper noise. These visual representations confirm the significant enhancement in noise robustness of various image-to-image translation models achieved by the GNI training method, all without incurring additional resource overhead.
Fig. 23 and 24 show the image conversion results under different image degradation and their combinations tasks Cat->Dog and Photo-Sketch respectively. As be here, systems trained with Gaussian noise injection produce pictures with more stable visual qualities compared with those baseline models.
D.4 COMPARISON WITH DENOISING-BASED APPROACHES
An alternative to noise injection is to denoise the noisy input and feed these denoised images into an 121 model trained solely on clean images. In the context of photo-to-sketch translation, Table 13 and Fig. 25 compare the results of denoising-based approaches against the noise-injection method. Specifically, we employed CBM3D (Mäkinen et al., 2020) for images tainted by Gaussian noise and median filtering (using x 5 window) for those affected by Salt & Pepper noise. Visually, the quality of outputs from the noise-injection approach is comparable to those achieved through denoising. However, noise injection offers several advantages over the benoising-based pre- processessigg strategy:
29

---

Published as a conference paper at ICLR 2024
Source
Clean
Severity
Baseline
+.N (0,0.04)
Figure 9: Comparison of reference-guicee -uadded generation for Cat->Dog under the interference of Uniform noise. Each source image is guided using two reference images. On the left is the result obtained by the baseline (GP-UNIT), and on the right is the result obtained by applying noise injection to the baseline model.
1. Generality: Unlike denoising, which often requires precise noise details, the GNI mothod is broadly applicable without specific noise characterizations.
2. Adaptive Robustness: the GNI approach naturally conditions the I2I model to diverse perturbations, enhancing its resilience.
3. Efficiency:
Bypassing the denoising step reduces computational overhead, offering faster translations, especially for high-resolution inputs.
4. Scalability: the GNI mothod's adaptability ensures relevance against evolving noise chal- lenges without extensive re- re-engineering.
In essence, while denoising-based pre-processing attempts "clean" the input, noise injection empowers the model to "understand" and "adapt" to the noise.
D.S OUT-OF-DOMAIN RESULTS
The Sketch Transformer was trained on highly specific dataset. In this subsection, we demonstrate its capability in handling out-of-domaan (OOD) face photos when the model is trained using noise injection. As depicted in Fig. 26, the model reliably translates OOD face photos into sketches, even in noise perturbations. However, compared to its performance on the in-domain CUFS test dataset, the model's iss resistance to noise slightly drops. This decrease in resilience becomes especially evident at higher noise levels, revealing visible distortions and suggesting an opportunity for future enhancements.
Regarding the two remaining image translation tasks, the employed training datasets boast even greater expanses. As result, their capacity handle data beyond their designated domains is topic afforded less discourse. Instead, the focus leans towards assessing whether the data input model necessitates supplementary data processing and ancillary procedures.
30

---

Published as a conference paper at ICLR 2024
Source
Clean
Severity
Baseline
+.V (0,0.04)
Figure 10: Comparison of reference-guidee generation for Cat->Dog under the interference of Color noise. Each source image is guided using two reference images. On the left is the result obtained by the baseline (GP-UNIT), and on the right is the result obtained by applying noise injection to the baseline model.
D.6 LIMITATIONS
In this subsection, examine the limitations of the noise injection method. Our prior simulations predominantly employed uni-directional 12I models. Both visual and metric evaluations indicate that generated image quality may suffer, particularly in scenarios with clean low-intensity noise.
Besides, the method's efficacy appears less consistent for bidirectional I2I translations. This is demonstrated in Table 14, which presents FID scores, and Fig. 27, which depicts qualitative results when noise injection is applied to CycleGAN (Zhu et al., 2017) for Horse-Zeebra translation task. Notably, the Gaussian noise-injected model struggles with clean inputs, and despite better FID scores with noisy inputs, visual distortions, especially around the zebra's 's head and legs, are evident. It is, therefore, crucial to acknowledge ledge that when applied bi-directional 12I translation models, the GNT mothod cannot be considered a straightforward "plug-and-play solution. Adaptations network architectures and/or loss functions might be requisite achieve desired results.
31

---

Published as a conference paper at ICLR 2024
Source
Clean
Severity
Baseline
+.,.0.))04
Figure 11: Comparison of reference-guided generation for Cat->Dog under the interference of Laplacian noise. Each source image guided using two reference images. On the left the result obtained by the baseline (GP-UNIT), and on the right is the result obtained by applying noise injection the baseline model.
Source
Clean
Severity
Baseline
+N (0,0.0))
Figure 12: Comparison of reference-guided generation for Cat->Dog under the interference of Salt & Pepper noise. Each source image is guided using two reference images. On the left is the result obtained by the baseline (GP-UNIT), and on the right is the result obtained by applying noise injection the baseline model.
32

---

Published as a conference paper at ICLR 2024
Source
Clean
Severity
Baseline
+.N
Figure 13: Comparison of latent-guided generation for Cat->Dog under the interference of Gaussian noise. Each source image guided using two style latents randomly sampled from (0, 1). On the left is the result obtained by the baseline (GP-UNIT), and on the right is the result obtained by applying noise injection the baseline model.
Source
Clean
Severity
Baseline
+.,00,))))
Figure 14: Comparison of latent-guided generation for Cat-+Dog under the interference of Uniform noise. Each source image is guided using two style latents randomly sampled from (0, 1). On the left is the result obtained by the baseline (GP-UNIT), and on the right is the result obtained by applying noise injection the baseline model.
33

---

Published as a conference paper at ICLR 2024
Source
Clean
Severity
Baseline
+.0(0.0.44)
Comparison of latent-guided generation for Cat->Dog under the interference of Color noise. Each source image is guided using two style latents randomly sampled from (0, 1). On the left is the result obtained by the baseline (GP-UNIT), and on the right is the result obtained by applying noise injection the baseline model.
Figure 15:
Source
Clean
Severity
Baseline
+..00,))))
Figure 16: Comparison of latent-guided generation for Cat->Do under the interference of Laplacian noise. Each source image is guided using two style latents randomly sampled from (0, 1). On the left is the result obtained by the baseline (GP-UNIT), and on the right is the result obtained by applying noise injection the baseline model.
34

---

Published as a conference paper at ICLR 2024
Source
Cleam
Severity
Baseline
+0,00.))))
Figure 17: Comparison of latent-guided generation for Cat->Dog under the interference of Salt & Pepper noise. Each source image is guided using two style latents randomly sampled from (O, 1). On the left is the result obtained by the baseline (GP-UNIT), and the right the result obtained by applying noise injection to the baseline model.
Clean
Severity
Impat
Input
Imput
Impat
Figure 18: Comparison of Face super-resolution task under Gaussian noise corruption. Each example has the baseline in the 1st row and the (O, 0.04) in the 2nd. The same format is followed for results under other noise types in the subsequent figures.
35

---

Published as a conference paper at ICLR 2024
Cleon
Severity
Input
Imput
Input
Imput
Figure 19: Face Super-Resolution comparison under Uniform noise interference.
Clean
Severity
Input
Imput
Imput.3
Input
Figure 20: Face comparison under Color noise interference.
36

---

Published as a conference paper at ICLR 2024
Cleon
Severity
Input
Imput
Input
Imput
Figure 21: Face Super-Resolution comparison under Laplacian noise interference.
Clean
Severity
Input
Imput
Imput.3
Input
Figure 22: Face Super-Resolune comparison under Salt & Pepper noise interference.
37

---

Published as a conference paper at ICLR 2024
Input
Baseline
Noise Injection
Baseline Noise
Injection Noise
Baseline Noise
Noise Injection Noise
Glass Blur
(b) Defocus Blur
(c) Pixelate
(d) JPEG
(6) + (e)
(b) (d)
Figure 23: Image translation results to multiple image degradation on the Cat-DDg image translation task (latent-guide guiddd. Note: Each source image is guided using two reference images. The selected noise intensity is S3 and other degradation intensity is S4.
Table 13: Comparison with Denoising-based Approach FID scores for Photo-*Skete translation.
Salt Pepper
Gaussian Noise
Methods
Clean
$2
S4
S6
S3
S5
S6
Denoising-baset
31.47
61.11
114.32
241.08
36.46
41.02
45.35
(.,00,)
64.12
32.45
46.89
74.25
31.16
40.95
64.87
Clean
Salt Pepper Noise
Gaussian Noise
Input
Denoising
+0000,004)
(0)
(b)
(c)
(d)
(e)
f
Figure 25; Comparison of denoising bassed pre-processing. First row: Input images. Column (a): Clean. Columns (b)-(d): Images corrupted with Salt & Pepper noise densities of 0.1, 0.2, and 0.3. Columns (e)-(g:: Images corrupted with i.i.d. Gaussian noises of o?=0.04, 0.09, and 0.16. Second row: Results using denoising and baseline sketch transformer. Column (a): Unprocessed. Columns (b)-(d): Median filters applied. Columns(e)-(g): CBM3D (Mäkinen al., 2020) denoising applied. Third row: Outputs from the models trained using Gaussian noise injection.
38

---

Published as a conference paper at ICLR 2024
Clean
Severity
JPEG
Baseline
JPEG + Noise (0,000))
Baseline
1700000
(a)
(b)
(c)
(d)
(e)
(f)
(g)
Figure 24: Image translation results to multiple image degradation on the Photo->keechh image translation task.
39

---

Published as a conference paper at ICLR 2024
Impui
of =001
0.04
of
of = 0.08
0.16
of
Learnable
oz
(a)
(b)
(a)
(c)
Figure 26: Out-of-domain results of Photo->Sketch task under the interference of Salt & Pepper
noise and Uniform noise. (a) Input photos not disturbed by noise; (b) Input photo disturbed by Uniform noise of intensity S4; (c) Input photo disturbed by Salt & Pepper noise of intensity S4.
40

---

Published as a conference paper at ICLR 2024
Table 14: FID comparison on the Horse-Zeebri image translation task using Cycle-GAN model. We use the same FID calculation method in (Zhu et al., 2017).
0
0.01
0.04
0.05
0.09
0.16
Baseline
76.92
92.15
118.29
135.64
147.34
180.82
+0
283.97
1 114.91
58.32
54.11
54.54
50.45
Severity
Input
Bareline
Proposed
TANE
Figure 27: Failure results for Horse-7Zebr translation in Cycle GAN model. Noise injection techniques might require customization for diverse models; otherwise, one may attain inferior results.
41


Reviews:
[{"review_id": "9oYd0HMg15", "reviewer": "Reviewer_SHt1", "summary": "This paper provides a robust theoretical framework for understanding the role of Gaussian noise injection in image-to-image (I2I) translation models. The key contributions of this work include:\n\n(i) Analyzing the influence of noise variance on distribution divergence and resilience to unseen noise types.\n\n(ii) Proposing a method to choose an optimal training noise level for consistent performance in noisy environments.\n\n(iii) Connecting f-divergence and score matching to explain the impact of Gaussian noise on aligning probability distributions.", "questions": "(i) For the theoretical noise analysis part, if considering signal-independent and -dependent noise terms, what will happen to the analysis? How does it change?\n\n(ii) To measure the domain discrepancy, why not using the metric PSNR Gap proposed by the prior work DANet [3]?\n\n[3] Dual adversarial network: Toward real-world noise removal and noise generation. In ECCV 2020", "limitations": "", "rating": 3, "confidence": 5, "soundness": 2, "presentation": 3, "contribution": 1, "strengths": "(i) The writing is clear and easy to follow. The presentation is well-dressed.\n\n(ii) This paper conducts a detailed theoretical analysis in Section 3 to understand the role of Gaussian noise injection in I2I models from the perspective of alignment distribution.\n\n(iii) To validate the conjecture proposed in this paper, the authors conduct sufficient experiments including three types of I2I models, i.e., cat to dog, photo to sketch, and human face super-resolution, covering five types of noise: Gaussian, Uniform, Color, Laplacian, and Salt & Pepper.", "weaknesses": "(i) This paper has a fatal theoretical flaw, i.e., the authors think too simply and naively about real-world degradations. They set up additive Gaussian noise in the training phase and five synthetic degradations (Gaussian, Uniform, Color, Laplacian, and Salt & Pepper.) in the testing phase. However, the real-world degradations are much more complex and fundamentally different from these degradations. To be specific, as for the noise term, the real-camera raw noise produced by photon sensing comes from multiple sources (e.g., short noise, thermal, noise, dark current noise, etc.) [1, 2, 3] and is further affected by the in-camera signal processing pipeline to become spatio-chromatically correlated. The real-camera noise contains both signal-dependent and -independent terms. However, the authors only consider very naive and simple signal-independent noise types, which is far from their motivation because the degradation patterns they studied simply do not exist in real-world images. Similar to the face super-resolution problem, the blur kernel cannot be explicitly estimated. \n\n[1] CycleISP: Real Image Restoration via Improved Data Synthesis. In CVPR 2020\n\n[2] Variational denoising network: Toward blind noise modeling and removal. In CVPR 2019\n\n[3] Dual adversarial network: Toward real-world noise removal and noise generation. In ECCV 2020\n\nIf you want to continue studying this topic, which I think is valuable too, I suggest you to conduct experiments in real degraded images. Here are some suggested datasets:\n\n[4]  A high-quality denoising dataset for smartphone cameras. CVPR 2018\n\n[5] Deep retinex decomposition for low-light enhancement. BMVC 2018\n\n[6] Ntire 2020 challenge on real-world image super-resolution: Methods and results. In CVPRW 2020\n\n(ii) This work does not have any technical contributions. Training with additive Gaussian noise has been studied in the image denoising I2I task for a long time. It is a very common setting. The robustness of noise has also been validated by many prior works. This paper does not propose any new method.\n\n(iii) The idea of adding Gaussian noise and aligning distribution is highly similar to a prior work [7] that also studies real-camera degradations in I2I tasks. There is no discussion or comparison.\n\n[7] Learning to Generate Realistic Noisy Images via Pixel-level Noise-aware Adversarial Training. In NeurIPS 2021.\n\n(iv) Code and pre-trained models are not submitted. The reproducibility cannot be checked.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "xnXXlMHasJ", "reviewer": "Reviewer_Ceu7", "summary": "This work provides a robust theoretical framework elucidating the role of Gaussian noise injection in I2I translation models. They address critical questions on the influence of noise variance on distribution divergence, resilience to unseen noise types, and optimal noise intensity selection.", "questions": "- How do you quantize the real and predictive distribution? Furthermore, how do you measure the diff between them?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- This paper thoroughly investigates the Gaussian noise in I2I area from both the theory and experiment. \n- Extensive experiments and analysis make the effects of the Gaussian noise more clear to us.", "weaknesses": "- Analysis of the Gaussian noise has been widely investigated, the authors should give some comparison or analysis about them in related works.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "JE0TeH9cDh", "reviewer": "Reviewer_YTga", "summary": "## Summary\n* This paper studies the problem of noise-resistent I2I generation. Particularly, it studies the noise injection approach both theoretically and practically. The paper shows that the joint f-divergence does not change too abruptly when the source is polluted by Gaussian noise. Further, the paper finds optimal noise level for training when the source is Gaussian, and verify their results on Gaussian source and real images.", "questions": "## Questions\n* Is it possible to extend current theory (__Lemma 1__, __Theorem 2__, __Corollary 1__) to non-Gaussian source with known $\\mu,\\Sigma$? As Gaussian is the max-entropy distribution with known 1st and 2nd moment, can similar results be obtained?\n* Is it possible to extend current theory (__Lemma 1__, __Theorem 2__, __Corollary 1__) to mixture of Gaussian source? As we can approximate any distribution, including natural image distribution with GMM, it would be much better if we can extend the theory to GMM, even for finite mixture. In this way, the distance between the current theory and empirical results on natural image can be reduced.\n* Current analysis on joint distribution looks quite general, can it be extended into any conditional GAN, beyond I2I?\n* What about more real-life noise, e.g. JPEG compression?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "## Strength\n* The result of optimal training level in __Corollary 1__ is useful to practical training of noise resistent generative model, if it is not limited to Gaussian source. It provides theoretical justification to a  intuitative practice.\n* The empirical result show that their approach improves the noise-robustness in various cases.", "weaknesses": "## Weakness\n* The discussion after __Lemma 1__ only convers non-Gaussian noise, but the majority of experiment is about non-Gaussian source. The gap of source distribution should at least be explicitly discussed in the main text not appendix, if not well addressed. Currently the proof of __Corollary 1__ seems to rely on Eq. 6 of __Lemma 1__, which holds only when the source is Gaussian. If I understand correctly, in that case, it becomes unreasonable to derive the optimal noise level $0.08$ for actual image generation, which is obviously non-Gaussian. If the theory is not connected to the experiment, then this paper becomes a little bit unconvincing. As no new empirical approach is proposed and the empirical results alone is not enough for accepting this paper.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (poster)