On Error Propagation of Diffusion Models
Authors: Yangming Li
Mihaela van der Schaar
Venue: ICLR 2024 poster (2024)
Keywords: Diffusion Models
Error Propagation
Theoretical Explanation
Regularization
Primary area: generative models
TL;DR: We present a theoretical framework to explain why error propagation happens to diffusion models and a regularization to address this problem

Abstract: Although diffusion models (DMs) have shown promising performances in a number of tasks (e.g., speech synthesis and image generation), they might suffer from error propagation because of their sequential structure. However, this is not certain because some sequential models, such as Conditional Random Field (CRF), are free from this problem. To address this issue, we develop a theoretical framework to mathematically formulate error propagation in the architecture of DMs, The framework contains three elements, including modular error, cumulative error, and propagation equation. The modular and cumulative errors are related by the equation, which interprets that DMs are indeed affected by error propagation. Our theoretical study also suggests that the cumulative error is closely related to the generation quality of DMs. Based on this finding, we apply the cumulative error as a regularization term to reduce error propagation. Because the term is computationally intractable, we derive its upper bound and design a bootstrap algorithm to efficiently estimate the bound for optimization. We have conducted extensive experiments on multiple image datasets, showing that our proposed regularization reduces error propagation, significantly improves vanilla DMs, and outperforms previous baselines.

Published as a conference paper at ICLR 2024
ON ERROR PROPAGATION OF DIFFUSION MODELS
Yangming Li, Mihaela van der Schaar Department of Applied Mathematics and Theoretical Physics University of Cambridge y187 4 10cccmm..... ac.uk
ABSTRACT
Although diffusion models (DMs) have shown promising performances in num- ber of tasks (e.g., speech synthesis and image generation), they might suffer from error propagation because of their sequential structure. However, this is not cer- tain because some sequential models, such as Conditional Random Field (CRF), are free from this problem. To address this issue, we develop a theoretical frame- work to mathematically formulate error propagation in the architecture of DMs, The framework contains three elements, including modular error, cumulative er- ror, and propagation equation. The modular and cumulative errors are related by the equation, which interprets that DMs are indeed affected by error propagation. Our thcoretical study also suggests that the cumulattve error closely related the generation quality of DMs. Based on this finding, we apply the cumulative error as a regularization term to reduce error propagation. Because the term is computationally intractable, we derive its upper bound and design a bootstrap al- gorithm to efficiently estimate the bound for optimization. We have conducted extensive experiments multiple image datasets, showing that our proposed reg- ularization reduces error propagation, significantly improves vanilla DMs, and outperforms previous baselines.
INTRODUCTION
DMs potentially suffer from error propagation. While diffusion models (DMs) have shown impressive performances in a number of tasks (c.g., image synthesis (Rombach et al. 2022), speech processing (Kong et al. (021) and natural language generation (Li et al. 2022)), they might still suffer from error propagation (Motter & Lai 2002). a classical problem in engineering practices (e.g.. communication networks) (Fu et al. 2020 Motter & Lai 2002 Crucitti et al. 2004). The problem mainly affects some chain models that consist of many end-to-ens connected modules. For those models, the output error from module will spread subsequent modules such that emoos accumulate along the chain. Since DMs are exactly of a chain structure and have a large number of modules (c.g., 1000 for DDPM (Ho al., (2020), the effect of error propagation on diffusion models is potentially notable and worth a careful study.
Current works lack reliable explanations. Some recent works (Ning et al. 2023 Li Li al. 2023 Daras et al.) (023) have noticed this problem and named it as exposure bias sampling drift. However, those works claim that error propagation happens to DMs simply because the models of cascade structure. In fact, many sequential models (e.g., CRF (Lafferty et al.] (001) and CTC Graves et al. 2006)) are free from the problem. Therefore, a solid explanation is expected to answer whether(or even why) error propagation impacts on DMs.
Our theory for the error propagation of DMs. One main focus of this work is to develop theoretical framework for analyzing the error propagation of DMs. With this framework, we can clearly understand how this problem is mathematically formulated in the architecture of DMs and easily see whether it has a significant impact on the models.
Our framework contains three elements: modular error, cumulative error, and propagation equa- tion. The first two elements respectively measure the prediction error of one single module and the accumulated error of multiple modules, while the last one tells how these errors related. We

---

Published as a conference paper at ICLR 2024
first properly define the errors and then derive the equation, which will be applied to explain why enror propagation happens to DMs. We will see that it is a term called amplification factor in the propagation equation that determines whether enror accumilation happens. Besides the theoretical study, we also perform empirical experiments to verify whether our theory applies to reality.
Why and how reduce error propagation. In our theoretical study, we will see that the cu- mulative enror actually reflects the generation quality of DMs. Hence, if error propagation is very serious DMs, then we can improve the generation performances by reducing it.
A simple way to reducing error propagation is to treat the cumulative error as regularization term and directly minimize it at training time. However, we will see that this error is computationally infeasible in terms of both density estimation and inefficient backward process. Therefore, we first introduce an upper bound of the error, which avoids estimating densities. Then, we design a boot- strap algorithm (which is inspired by TD learning (Tesauro et al. 1995 Osband et al., 2016)) to efficiently estimate the bound though with a certain bias.
Contributions.
The contributions of our paper as follows:
We develop a theoretical framework for analyzing the error propagation of DMs, which explains whether the problem affects DMs and why it should be solved. Compared with our framework, previous explanations are just based
unreliable intuition;
In light of our theoretical study, we introduce regularization term to reduce error propa- gation. Since the term computationally infeasible, we derive its upper bound and design a bootstrap algorithm to efficiently estimate the bound for optimization;
We have conducted extensive experiments on multiple image datasets, demonstrating that proposed regularization reduces error propagation, significantly improves the perfor- mances of vanilla DMs, and outperforms previous baselines.
BACKGROUND: DISCRETE-TING DMS
In this section, we briefly review the mainstream architecture of DMs (i.e., DDPM), The notations and terminologies introduced below will be used in our subsequent sections.
DM consists of two Markov chains of T steps. One is the forward process, which incrementally adds Gaussian noises into real sample Xo 4(xo), a K-dimensional vector. In this process, sequence of latent variables X1:7 = [x1, 1x1,X,,--> x-. [x]] are generated in order and the last one x7 will approximately follow standard Gaussian:
T
q(xErr
Xp)
II gxxx xx-1), g(xx xx-1) = N(xx: 11 -1x,,1,,,,,
(1)
where N denotes a Gaussian distribution, I is an identity matrix, and Br, & t & T represents predefined variance schedule.
The other is the backward process, a reverse version of the forward process, which first draws an initial sample Xy from standard Gaussian p(xr) N/O,, I) and then gradually denoises it into
sequence of latent variables XT-1-0 = 1X Xo) in reverse order:
Po(XT.0) = p(xT) [ pa (x:-1 xt), pe (x:-1 xx) = N(x--i o),1),
(2)
where pe(x,-1 xx) is a denoising module with the parameter 0 shared across different iterations, and I is a predefined covariance matrix.
Since the negative likelihodooo E[- log pe (xo)] is computationally intractable, common practices apply Jensen's inequality to derive its upper bound for optimization:
Exg - x(x)) log (x)) & E,,LDLL((ee
xx) |l p(xx)) + E,, log (x xx)]
+
Σ 1s
EgLDKL(((X--1 x xo) (x x xx) = Call
(3)
2

---

Published as a conference paper at ICLR 2024
Propagation Equation
2
ecamu
Eg
Egumnu + E2
Emoo
E 2 Ez + E
Cumulative Error
3
Egumu
gamu
+
Modular Error
Egmoo
E2
eodd
g
Input Ertor
Input Ertor
Denoising Modules
(x(x212g)
(x(x11x2)
(x(xo)
Figure 1: A toy example (with T to show theoretical framework for the error propugation of diffusion models. We use dash lines to indicate that the impact cumulative error goumu on denoising module po(x;,1 Xx) is defined the distributional (not sample) level.
where DKL. denotes the KL divergence and each term has analytic form for feasible computa- tion. Ho et al. (2020) further applied some reparameterization tricks to the loss Lull reduce its estimation variance. For example, the mean Me is formulated by
3r d4
140 (x, 1)
x:
Eo (x.,
(4)
Vor
in which az 1 - 3r, ā. IT- ow, and is parameterized by neural network. Under this scheme, the loss Lall can be simplified
Call
eol + v1- ã,e, 1)12
Call
where the neural network is tasked to fit Gaussian noise c.
THEORETICAL STUDY
In this section, we first introduce our theoretical framework and discuss why current explanations (for why error propagation happens to DMs) are not reliable. Then, we seek to define (or derive) the elements of our framework and apply them to give solid explanation.
3.1 ANALYSIS FRAMEWORK
Elements the framework. Fig. illlstrrates preview of our framework. We can see from the preview that we have to find the following three elements from DMs:
Modular error Emod, which measures how accurately every module maps its input to the output. It only relies on the module iteration & and is independent of the others. Intu- itively, the error is non-negative: emod > 0, Vi [1,T]:
Cumulative error Ecumu, which is also non-negative, measuring the amount of error that can be accumulated for sequentially running the first L + 1 denoising modules. Unlike the modular error, its value is influenced by more than modules;
Propagation equation { Ecumu,... .... EE Eeram Emod E Emod Emodd 0. which
tells how the modular and cumulative errors of different iterations are related. It is generally very hard to find such equation because the model is non-linear.
Interpretation of the propagation equation. In a DM, the input to every module po(x;-1 x,), [1, T] can be regarded as gouma Since module po(x,-1 x,) is parameterized by a non- linear neural networks, it is not certain whether its input error ceumu is magnified reduced in the
3

---

Published as a conference paper at ICLR 2024
error contained in its output: Equmn, Therefore, expect the propagation equation to include or just take the form of the following equation:
Et - -, e_mo = p4C++1
coumu
(6)
which specifies how much imput error E++1 that the module po (x x,) spreads the subsequent modules. For example, P4 < 1 means the model can discount the input cumulative error Et4 which contributes to avoid error accumulation. We also term 14 as the amplification capliitcatioooacttor which satisfies 14 O because the cumulative error &g Eeunu at least contains the prediction error of the denoising module pe(X:-1 xx).
With this equation, we have N4 3 1, E [1,7]. then we can assert that the DM is affected by error propagation because in that case every module po (x xx) either fully spreads or even magnifies its imput enror the subsequent modules.
Why current explanations are not solid. Current works (Ning et al. 2023 Li ct al. 2023 Daras et al., 2023) claim that DMs are affected by error propagation just because they are of chain structure. Based on our theoretical framework, we can see that this intuition is not reliable. For example, if 1u 0, Vt [1, 77, every module po xx) of the DM will fully reduce its input cumulative error goumu which avoids error propagation.
3.2 ERROR DEFINITIONS
In this part, formally define the modular and cumulative errors, which respectively measure the prediction error of one module and the accumulated errors of multiple successive modules.
Derivation of the modular error. In common practices (Ho et al. 2020 Song et al. 2021b), every denoising module in a diffusion model pa( (x xx), L [1, T" is optimized to exactly match the posterior probability q(xx-1 x.). Considering this fact and the forms of loss terms in Eq. (3) we apply KL divergence to measure the discrepancy between those two conditional distributions: (RI(Po (-) II (()), and treat as the modular error. To further get ride of the dependence on variable we apply an expectation operation Exxxpexx)) to the error.
Definition 3.1 (Modular Error). For the forward process as defined in Eq. (I) and the backward process as defined in Eq. (2). the modular error Emad of every denoising module pa(x; xx), e [1,7] in a diffusion model measured as
emod
Exx (PLL xx) 4(xx xx))].
Inequality gooo > 0 always holds since KL divergence is non-negnn non-negative.
Derivation of the cumulative error. Considering the fact (Ho et al. 2020
q(x, xo)
N((x; Vo (1 - T.) we can reparameterize the variable x, as v Vo,oo N(O, I). Therefore, the loss term Call in Eq. (5) can be reformulated as
v1 046,6
X4 VaiXo
CTlll E Exnng(())))) x (xx
ca co(X,,))) ()
V1 oe
From this equality, we can see that the input x, to the module po (x 1 xx) training time follows a fixed distribution that is specified by the forward process q(XT=o):
[
q(xx)q(Xx xo)dxo = [[ g(xe, xo) = g(xx),
(8)
Jxg
which inconsistent with the input distribution that the module po px x,) receives from its previous denoising module (po(xe-1 xx) /' [t 1,T]] during evaluation:
2+1
p(xT) XT
Il $=7
(x x xx)xx1:411
po (T:1,x - (9)
JxT:e11
The discrepancy between these two distributions, g(x.) and (xx).. is essentially the input error to the denoising module po Px(x-11 x²). Like the modular error, we also apply KL divergence to measure this discrepancy DKL (pe (xx) g(xx)). Because the input emror of module pe(x x,) can be regarded as the accumulated output errors of its all previous modules, we resort to that input emoor to define the cumulative error.

---

Published as a conference paper at ICLR 2024
Definition 3.2 (Cumulative Error). Given the forward and backward processes that are respectively defined in Eq. [ and Eq. 2 the cumulative error Eq of the diffusion model at iteration [1, (caused by the first T t 1 modules) is measured as
Et = DKL (pe(pp( || q(x;-1)).
(10)
Inequality Et always holds because KL divergence non-negative.
Remark 3.1. The error E, in fact reflects the performance of DMs data generation. For instance, when = 1, the cumulative error Equmn = DKL(Lppp (po(xo) || g(xo)) indicates whether the generated samples are distributionally consistent with real data. Therefore, a method to reducing error propagation is expected improve the performance of DMs.
Remark 3.2. In Appendix [ we prove that &g Equmu 0 is achievable in an ideal case: the diffusion model is perfectly optimized (i.e., (x Xv) = q xy), A4' e [1,T].
3.3 PROPAGATION EQUATION
The propagation equation describes how the modular and cumulative errors of different iterations are related. We derive that equation for the diffusion models as below.
Theorem 3.1 (Propagation Equation). For the forward and backward processes respectively defined
in Eq. 4 and Eq. ( suppose that the output of neural network EΦ(-) )) (as defined in Eq. 44) follows
a standard Gaussian regardless of imput distributions and the entropy of distribution (xa) is non-
increasing with decreasing iteration i, then the following inequality holds:
&t
Eoumn
goumn + E gmod
(11)
where E [1,T] and we specially set ET+1 0.
Remark 3.3. The theorem is based two main intuitive assumptions: 1) From Eq. (5).
that neural network Eg aims to fit Gaussian noise e and is shared by all denoising iterations, which means it takes the input from various distributions [q(x)), ())xx)))...... .9(x)),, Therefore, it's reasonable to assume that the output of neural network eg follows a standard Gaussian distribution, independent of the input distribution; 2) The backward process is designed to incrementally denoise Gaussian noise XT ~ (O, I) into real sample Xg- Idcally, the uncertainty (i.c., noise level) of distribution pe (xx) gradually decreases in that denoising process. From this view, makes sense to assume that differential entropy Hp (x,) reduces with decreasing iteration f.
Proof. We provide a complete proof to this theorem in Appendix C
Are DMs affected by error propagation? By comparing Eq. 11 and Eq. (6), we can that Me > 1, vt [1. Based on our discussion in Sec. 3.1 we can assert that error propagation happens to standard DMs. The basic idea: Me > 1 means that the module po(x xx) at least fully spreads the input cumulative error gourn to its subsequent modules.
4 EMPIRICAL STUDY: CUMULATIVE ERROR ESTIMATION
Besides the theoretical study, it is also important to empirically verify whether diffusion models are affected by eиror propagation. Hence, we aim numerically compute the cumulative error & geumn and show how it changes with decreasing iteration t [1.7]- By doing so, we can also verify whether our theory (e.g.. propgation equation: Eq. ITT) applies reality.
Because the marginal distributions (x) 4(xx11) have no closed-form solutions, the KL diver- gence between them (i.e., the cumulative error & gcumu) computationally infeasible. To address this problem, we adopt the maximum mean discrepancy (MMD) (Gretton et al. 2012) to estimate the error e geumu and prove that is in fact tightly bounded by MMD.
With MMD, we can define another type of cumulative error Deumm t [1,7]:
Doumn
-x 10 ([((x=))) 1
- Ex.- lo [x(x
(12)
which also measures the gap between Po (x,)) 1) and q(x;-1). Here H. denotes a reproducing kernel
Hilbert space and IR H is feature map. Following common practices (Dziugaite al.
5

---

Published as a conference paper at ICLR 2024
(CIFAR-3)
6.041
0.0344
6.003
0.002
6.002
0.0322
0.001
0:31
0.0023
0.000
0.00
0.0320
100
(ImageNel)
8.013-
2/9125
0.0015
0.013
0.000
6,005
30350
0.0325
3/925
2000
000
500
420
1000
500
600
420
220
1908
800
600
420
200
Denoising iteration (Gaussian Kemel)
Denoising Iteration (Laplace Keree))
Denoising Iteration (Logi stic Kemell
Figure 2: Uptrend dynamics of the MMD error De w.r.t decreasing iteration t. The cumulative geumu might show similar behaviors since is tightly bounded by the MMD error.
error e
2015 adopt unbiased estimate of the error Dr Drumn:
Doumu
Σ
back, back.)
,x11
+ M
Σ
K(xx-1++ ,x==1
forw, forw.)
N
IENJEN
1
(13)
2
Σ
K(xx1111 ,x-1
back, JorwJ)
NM
ISiEN,EEJEM
backi po ) E iiie ig N is performed with Eq. (2. where x j & M is done by Eq. and KC(.) is the kernel function to simplity the inner products among (o(x)ockk1)) 6(x, back.)) o(x;-1 backk
forw.j q(x--1), & xt-1
back.NV (oxxoorr.)) (or "..... d(x/ory }).
Importantly, we show in the following that the cumulative error Equmn is tightly bounded by the MMD error Deumu from below and above.
Proposition 4.1 (Bounds of the Cumulative Error). Suppose that H. S Cz. (R) and the two contin- wous marginal distributions po (x--) (x+-1), (x.(((+))) are non-zero everywhere, then the cumulative error E, geumu is linearly bounded by the MMD error Doumu from below and above:
1 - DEumm & &.numn
(14)
Here (RRK) is the set of all continuous functions (with finite uniform norms) over R.K.
Proof. A complete proof to this proposition is provided in Appendix E
Experiment setup and results. We train standard diffusion models (Ho et al., 2020) on two
datasets: CIFAR-10 (32 x 32) (Krizhevsky et al. (2099) and ImageNet (32 x 32) (Deng al. 2009).
Three different kernel functions (e.g., Laplace kernel) are adopted to compute the error Doumu in terms of Eq. (3). Both and M are set as 1000.
As shown in Fig. 2 the MMD enror Deumu rapidly grows with respect to decreasing iteration in all settings, implying that cumulative error E also similar trends. The results not only empirically verify that diffusion models affected by error propagation, but also indicate that our theory (though built under some assumptions) applics to reality.
5 METHOD
According to Remark 3.1 the cumulative error in fact reflects the generation quality of DMs. There- fore, we might improve the performances of DMs through reducing error propagation. light of this finding, we first introduce basic version of our approach to reduce error propagation, which covers our core idea but is not practical in terms of running time. Then, we extend this approach to practical through bootstrapping.

---

Published as a conference paper at ICLR 2024
5.1 REGULARIZATION WITH CUMULATIVE ERRORS
An obvious way for reducing enror propagation is to treat the cumulative error & as a regular- ization term optimize DMs. Since this term is computationally intractable mentioned in Sec. we adopt its upper bound Dr based on Proposition 4.1
Drumu
L
Wy Lt
(15)
where We oc exp(p * (T- PE R and Ei=o 1. We exponentially schedule the weight we because Fig. suggests that error propagation is severe as t gets closer to O. Term L, is estimated by Eq. (13) in practice, with backward variable x,
back,5 denoised from a Gaussian noise via Eq. (2) and forward variable x; forw-J converted from a real sample with Eq. Q
The new objective _ is in addition the main _ (previously defined in Eq. (5) for regular- izing the optimization of diffusion models. We linearly combine them as = A C + mg CTD
where A s (0, 1) and A x 1, for joint optimization.
5.2 BOOTSTRAPPING FOR EFFICIENCY
A challenge of applying our approach in practice is the inefficient backward process. Specifically, to sample backward variable X; of T f modules to cast a Gaussian noise into that variable.
back, for estimating loss L have iteratively apply a sequence
Inspired by temporal difference (TD) learning (Tesauro et al. 1995), we bootstrap the computation of variable x back, for run-time un-time efficiency. To this end, we first sample a time point s > from uniform distribution (( + L, L,), t + }} and apply the forward process to estimate variable x back, in a possibly biased manner:
back.
V1- a.C, xo ~ ((xo), 6 NO,
(16)
where operation ((xo) means to sample from training data and L < is predefined sampling length. The above equation is based on a fact (Ho et al. 2020):
(xx xo) = Nxxx: V Voxoo (1 T)
Then, we apply a chain of denoising modules [po(x.-1 xx), Po(x x,-I), ... pa (x; xx+1)] to iteratively update &back, into x phack,i which can be treated as an alternative to variable x Each update can be formulated as
back,
x back,
1
( back,, &
3x
( h ,h)) OxE,E~ N(0,1,
(17)
Vau
VI
where 1 & s. Finally, we apply the alternative x, back, for computing loss C terms of Eq. [13 with at most L < T runs of neural network Eg.
For specifying the sampling length L, there actually a trade-off between estimation accuracy and time efficiency For example, large L reduces the negative impact from biased initialization x &back, but incurs high time cost. We will study this problem in the experiment and also put the details of our training procedure in Appendix F The source code of this work is pub- licly available personal repository: https://ithubb.co//uuuuutt http:///htbbb..ooo///vaeeeeeaaae
and our lab repository:
6 EXPERIMENTS
We have performed extensive experiments verify the effectiveness of our proposed regularization. Besides special studies (e.g., effect of some prper--arameter,, our main experiments include: 1) By comparing with Fig. 2 we show that the error propagation is reduced after applying reg- ularization; 2) On three image datasets (CIFAR-10 10, CelebA, and ImageNet), our approach notably improves diffusion models and outperforms the baselines.
7

---

Published as a conference paper at ICLR 2024
Ereer (CIFAR-10)
3,004
01003
3,003
0,002
2,002
0.001
0.001
5,003
0.003
3,0000
3030
1000
1000
230
Emrr (Imageliet)
0.05
3.0125-
2,025
5.0302
0.13
2,0000
20315
3.0025
0.02
1030
800
520
400
200
2,0003
5000
500
500
400
200
2,0000
2000
200
500
450
220
Denaising Iteration (Gaussian Kernel)
Denoising Iteration (Laplace Kernel)
Denoising Iteration (Logistic Kernel)
Figure 3: Re-estimated dynamics of the MMD error Drunu with respect to decreasing iteration t after applying our proposed regularization. These dynamics should be compared with those in Fig. 2 showing that we have well handled error propagation.
Approach
CIFAR-10
ImageNet
CelebA
ADM-IP Ning et al. 2023
3.25
2.72
1.31
DDPM Ho al. (2020)
3.61
3.62
1.73
DDPM w/ Consistent DM (Drrs et al.
2023
3.31
3.16
1.38
DDPM w/ FP-Diffusion LLai
2022
3.47
3.28
1.56
DDPM Our Proposed Regularization
2.93
2.55
1.22
Table I: FID scores of our model and baselines on different image datasets. The improvements of our approach over baselines are statistically significant with p < 0.01 under t-test.
6.1 SETTINGS
We conduct experiments on three image datasets: CIFAR-10 (Krizhevsky et al. 2009). Ima- geNet (Deng et al. 2009), and CelebA (Liu et al. (2155), with image shapes respectively as 32 32, 32 32, and 64 64. Following previous works (Ho adopt Frechet Inception Distance (FID) (Heusel et al.) 2017 the evaluation metric. The config- uration of our model follows common practices, we adopt U-Net (Ronneherger al. 2015) as the backbone and respectively set hyper-parameters T, or, L, rog A ; 1000, Br, 5, 0.2, 0.8, 0.003. All our model run on Tesla V100 GPUs and are trained within two weeks.
2020 Dhariwal Nichol (02T), we
Baselines. We compare method with three baselines: ADM-IP Ning al. (023). Consistent DM (Daras et al. (023), and FP-Diffusion (Lai et al. 2022). The first one under the topic of exposure bias (which related to our method) and the other aim regularize the DMs in terms of their inherent properties (e.g., Fokker-Plande Equation), which are less related to our method. The main idea of ADM-IP is to adds an extra Gaussian noise into the model input, which is not an ideal solution because this simple perturbation certainly can not simulate the complex errors at test time. Table shows that our method significantly outperforms ADM-IP though ADM (Dhariwal & Nichol 2021 is stronger backbone than DDPM.
6.2 REDUCED ERROR PROPAGATION
To show the effect of our regularization _ee to diffusion models, we re-estimate the dynamics of MMD error Deumu with the same experiment setup datasets and kernel functions as our empirical
study in Sec. The only difference is that the input distribution to module pe (x xx) at training
time is longer qxxx), but we can still correctly estimate error Drumu with Eq. (3 by re-defining forw. as the training-time inputs the module.
x
Fig. 3 shows the results. Compared with the uptrend dynamics of vanilla diffusion models in Fig. 2 we can see that the new ones are more like a slightly fluctuating horizontal line, indicating that: T) every denoising module po( p xx) has become robust to inaccurate inputs since measure DMMD is not correlated with its location in the diffusion model; 2) By applying our regulariza- tion Greg the diffusion model is not affected by error propagation anymore.
8

---

Published as a conference paper at ICLR 2024
3.6
100
3.5
Score
3.4
Time Cost
FID Score
120
Time Cost
3.2
100
3.0
2.5
60
80
2
3
Bootstrapping Steps (CIFAR-10)
Bootstrapping Steps (ImageNet)
Figure 4: A trade-off study of the hyper-parameter (i.e., the number of bootstrapping steps) on two datasets. The results show that, as L gets larger, the model performance limitedly increases while the time cost (measured in seconds) boundlessly grows.
Approach
CIFAR (32 x 32) CelebA (64 64)
DDPM w/ Our Regularization
2.93
1.22
Our Model w/o Exponential Schedule Our Model w/o Warm-start &back, (L 5)
3.12
1.35
3.38
1.45
Our Model w/o Warm-start X &back, (I.
3.15
1.27
Table Ablation studies of the techniques (i.e., the schedule of loss weight wi and the warm- start initialization of alternative representation xbback.i used in our approach.
6.3 PERFORMANCES IMAGE GENERATION
The FID scores of our model and baselines on datasets are demonstrated in Table 1 The results of the first row are copied from (Ning et al. 2023) and the others are obtained by our experiments. We draw two conclusions from the table: 1) Our regularization notably improves the performances of DDPM, mainstream architecture of diffusion models, with reductions of FID scores as 18.83% on CIFAR- 10, 29.55% on ImageNet, and 29.4 29.47% on CelebA Therefore, handling error propagation benefits improving the generation quality of diffusion models; 2) Our model significantly outper- forms ADM-IP, P, the baseline that reduces the exposure bias by input perturbation, on all datasets. For example, our FID score CIFAR-I0 lower than its score by 9.84%. An important factor tributing to our much better performances that, unlike the baseline, our approach doesn't 't impose any assumption the distribution form of propagating errors.
6.4 TRADE-OFF STUDY
The number of bootstrapping steps L. is a key hyper-paramest in our proposed regularization. Its value determination involves trade-off between effectiveness (i.e., the quality of alternative back- ward variable
x b and run-time efficiency.
Fig. shows how FID scores (i.c., model performances) and the averaged time costs of one training step change with respect to different bootstrapping steps L. For both CIFAR- 10 and ImageNet, can see that, as the number of steps increases, FID scores decrease and tend to converge while time costs boundlessly grow. In practice, set L for our model since this configuration leads to good enough performances and incur relatively low time costs.
6.5 ABLATION EXPERIMENTS
To make our regularization [ work in practice, we exponentially schedule weight w;, 0 & t T and specially initialize alternative backward variable x bback, via Eq. (16). demonstress the ablation studies of these two techniques. For the weighted schedule, we can see from the second row that model performances are notably degraded by replacing it with equal weight w, = 1/T. Firstly, by adopting random initialization instead of Eq. (16). the model performances drastically decrease, implying that it's necessary to have warm start; Secondly, the impact of initialization can be reduced by using larger number of bootstrapping steps T.,

---

Published as a conference paper at ICLR 2024
ACKNOWLEDGMENTS
We thank the anonymous ICLR reviewers for their kind and constructive reviews. Yangming Li also thanks Accenture for their sponsorship and support.
REFERENCES
Sitan Chen, Sinho Chewi, Holden Lee, Yuanzhi Li, Jianfeng Lu, and Adil Salim. The probability flow ode is provably fast, arXiv preprint arXiv:2305. 11798, 2023a.
Sitan Chen, Sinho Chewi, Jerry Li, Yuanzhi Li, Adil Salim, and Anru Zhang. Sampling is as easy as learning the score: theory for diffusion models with minimal data assumptions. In The Eleventh International Conference on Learning Representations, 2023b. URLhttps: htw.httpss//openevieew
Paolo Crucitti, Vito Latora, and Massimo Marchiori. Model for cascading failures in complex net- works. Physical Review E, (9(4):045144 2004.
Giannis Daras, Yuval Dagan, Alexandros G Dimakis, and Constantinos Daskalakis. Consis- tent diffusion models: Mitigating sampling drift by learning to be consistent. arXiv preprint arXiv:2302.09057, 2023.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hi- erarchical image database. In 2009 IEEE conference computer vision and pattern recognition, pp. leee, 2009.
Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan (eds.), Ad- vances in Neural Information Processing Systems, volume 34, pp. 8780-8794. Curran Asso- ciates, Inc., 2021. URL /ttp///ccc//prrrceeiinnn..ece neurips. cc/papper/22///ille/ 3Sac2aaeeeFFaaaaaa7777722saaaaa3
Gintare Karolina Dziugaite, Daniel M Roy, and Zoubin Ghahramani. Training generative neural networks via maximum mean discrepancy optimization. arXiv preprint wrxxv::505.039000 2015.
Xiuwen Fu, Pasquale Pace, Gianluca Aloi, Lin Yang, and Giancarlo Fortino. Topology optimization against cascading failures on wireless sensor networks using a memetic algorithm. Computer Networks, 177:107327, 2020.
Alex Graves, Santiago Fernández, Faustino Gomez, and Jürgen Schmidhuber. Connectionist tem- poral classification: labelling unsegmented sequence data with recurrent neural networks. In Proceedings of the 23rd international conference on Machine learning, pp. 369-376, 2006.
Arthur Gretton, Karsten M Borgwardt, Malte Rasch, Bernhard Schölkopf, and Alexander Smola.
A kernel two-sample test. The Journal of Machine Learning Research, (3(1):733-773 2012.
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to local nash equilibrium. Advances in neural information processing systems, 30, 2017.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems, 33:6840-6851, 2020.
Zhifeng Kong, Wei Ping, Jiaji Huang, Kexin Zhao, and Bryan Catanzaro. Diffwave: versatile diffusion model for audio synthesis. In International Conference Learning Representations, 2021.
net/ wwtp:neee//toimmeme....e..
XFK8Ymz5J
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
John Lafferty, Andrew McCallum, and Fernando CN Pereira. Conditional random fields: Proba- bilistic models for segmenting and labeling sequence data. 2001.
10

---

Published as a conference paper at ICLR 2024
Chieh-Hsin Lai, Yuhta Takida, Naoki Murata, Toshimitsu Uesaka, Yuki Mitsufuji, and Stefano Er- mon. Regularizing score-based models with score fokker-planck equations. In NeurIPS 2022 fott/forumm serummid
Workshop on Score-Based Methods, 2022. URL htttttt//ppereeeieeee
[qW7
Chieh-Hsin Lai, Yuhta Takida, Toshimitsu Uesaka, Naoki Murata, Yuki Mitsufuji, and Stefano Er- mon. On the equivalence of consistency-type models: Consistency models, consistent diffusion models, and fokker-planck regularization. In ICMI. 2023 Workshop on Structured Probabilistic Inference & Generative Modeling, 2023. URL https:://///mm. /ttpp::e///peeeviwww net forum?id witGsScvAO
Holden Lee, Jianfeng Lu, and Yixin Tan. Convergence of score-based generative modeling for general data distributions. In NeurIPS 2022 Workshop on Score-Based Methods, 2022. URL
net. forum? 3ddrgl 9A8mu8sv
Mingxiao Li, Tingyu Qu, Wei Sun, and Marie-Francing Moens. Alleviating exposure bias in diffu- sion models through sampling with shifted time steps. arXiv preprini arXiv:2305. /5583, 2023.
Xiang Lisa Li, John Thickstun, Ishaan Guirajani, Percy Liang, and Tatsunori Hashimoto. Diffusion- LM improves controllable text generation. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho (eds.), Advances in Neural Information Processing Systems, 2022. URL https: /wwpnrevieeeeeeeeeeeeeeeeeeeeeee net Forumeidu 3s9Irss
Ziwei Liu, Ping Luo. Xiaogang Wang. and Xiaoou Tang. Deep learning face attributes in the wild. In Proceedings of the IEEE international conference on computer vision, pp. 3730-3738, 2015.
Adilson E Motter and Bhng-Chegg Lai. Cascade-basee attacks complex networks. Physical Review E, 66(6):065102, 2002.
Mang Ning, Enver Sangineto, Angelo Porrello, Simone Calderara, and Rita Cucchiara. Input per- turbation reduces exposure bias diffusion models. arXiv preprint arXiv:2301.11706, 2023.
Ian Osband, Charles Blundell, Alexander Pritzel, and Benjamin Van Roy. Deep exploration via bootstrapped dqn, Advances in neural information processing systems, 29, 2016.
Marc' Aurelio Ranzato, Sumit Chopra, Michael Auli, and Wojeiech Zaremba. Sequence level train- ing with recurrent neural networks. ICLR, 2016.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pp. 10684-10695, 2022.
Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net:
Convolutional networks for biomed- ical image segmentation. In Medical Image Computing and Cosiuter-Assisted Intervention- MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceed- ings, Part III 18, pp. 234 241. Springer, 2015.
Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. In Interna- tional Conference on Learning Representations, 2021a. URL /tK.itttps://operreviiee net/
=orum?dd=Stlgiarchi
Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In Interna- tional Conference on Learning Representations, 2021b. URL https: //tp:::s///peneeveee htt.tiiiwwweeeeeeee.ooo...
Eooum?1d=PxTic1220
Yang Song, Prafulla Dhariwal, Mark Chen, and Ilya Sutskever. Consistency models. 2023.
Gerald Tesauro et al. Temporal difference learning and td gammon. Communications of the ACM, 38(3):58-68, 1995.
Chong Xiao Wang and Wee Peng Tay. Practical bounds of kullback- divergence using maxi- mum mean discrepancy. arXiv preprint arXiv:2204.02031 2022.

---

Published as a conference paper at ICLR 2024
Wen Zhang, Yang Feng, Fandong Meng, Di You, and Qun Liu. Bridging the gap between train- ing and inference for neural machine translation. In Proceedings of the 57th Annual Meet- ing of the Association for Computational Linguistics, pp. 4334-4343 Florence. Italy, July 2019. Association for Computational Linguistics. doi: 10. 1
URL https:
Yufeng Zhang, Wanwei Liu, Zhenbang Chen, Kenli Li, and Wang. On the properties of kullback- leibler divergence between gaussians. arXiv preprint arXiv: 202.::2002.04885. 2021.
12

---

Published as a conference paper at ICLR 2024
A IDEAL SCENARIO: ZERO CUMULATIVE ERRORS
The error Equmm, t [1, defined by KL divergence can be expanded as
Equmu DrI,(poxx-)) || x(xx-1)) JxL-- px(x:-1) In Px( dx:
(18)
((x,-1)
which achieves its minimum O exclusively at po(x{-1) q(x(-1). We show that Eq 0, Vt [o, 7] holds in an ideal situation (i.e., diffusion models are perfectly optimized).
Proposition A.1 (Zero Cumulative Errors). A diffusion model is perfectly optimized if "p((X=-) xz) = x(x-1 xt), Vt E [1, In that case, we have &q = 0, [1,T].
[) for perfectly optimized diffusion models. We apply mathematical induction to prove this assertion. Initially, let's check whether this is true at t T. Based on nice property of the forward process (Ho al. 2020
Proof. It suffices show px(x;) q(x,), 1A
q(x: Xo) Nxxx v V0 I
we can see that term (Xr Xo) converges to a normal Gaussian (O, I) as - 00 because limy-ro var Also note that (x(x)) p(x)) (Nxx;; O, we have
[
((x)) q(xT (x)g(x))dxx N(xx; 0,1) q(x))dxo px(XT)
(19)
X
Therefore, the initial case is proved. Now, we assume the conclusion holds for t. Considering the precondition po (xx-1 xx) g(xx-1 i xt), we have
po(x1-1,x))) x))xx; S( po(xx-11 xx)po(xx))ss
po (x)
xx
(20)
q(x (x) = ((x)-1;x))dxx q(xx-1)
Xx
Xx.
which proves the case for t - 1. Finally, the assertion (x, = g(x:) is true for every e [o, []. and therefore the whole conclusion proved.
In proof to this proposition, the use of that precondition T cc, is to let ((0T(20) ( 0, I). Considering the fact that
(<<e|<0)) N(a; vars, v1 a,E).
we can achieve the goal with much weaker assumption: lim,-r de assumption is a standard configuration in current diffusion models (e.g.. DDPM (Ho et al. 2020) and SGM (Song et al. 2021b)), which lets Xy contain no information about Xo.
Notably, this new
B DEFINITION OF THE MODULAR ERROR
The KL divergence is not symmetric, the modular error have another definition as
gmod = 1Ex-~gixx ) [DLL (q(x,-1 xr) || pe(x;-1 xx))]
However, the error propagation happens to the learnable backward process (instead of the prede- fined forward process q). Hence, it makes more sense to define the module error as
E (xxx
DKL. (po(xe-1 xx) || q(xx-1
xx))
px( (
(x xx) In
Po (x,
xx))
dxx-1) dx:
g(xx-1 xx)
po (x:-1 xx,)
Ex (x(x))
Ex Exx- In
g(xr-1
Xx)
such that expectation integrals operated the distributions pa (xx). ). po ( xx) of the back-
ward process to average the distribution gap In(-). For the reversed case, the expectation operations will be mistakenly applied the distributions ((x:). q(x,11 xx) of the forward process.
13

---

Published as a conference paper at ICLR 2024
C
PROOF TO THEOREM 3.1
The cumulative error mq can be decomposed into two terms:
Po (x(-1)
Equmu
po(x In q(x(-1)
dx Hp (xx-1) 1) + Ex, .I- In g(xx-1)]
(21)
where the second last term is the entropy of distribution (xx).. We denote the last term as T--1- According to the law of total expectation, we have
Ti-1 = Ex, -po (x,) [xxx i~no(x, 1/x1)] In (x)
(22)
Note that -xxx-1)((x x(-1) = g(xx-1, x1) = ((x))g((<c-
Xx): we have
Ex.- Pa (xx) In lxx(()) In gxx,11 X2)
T--1 Exi~po (xx)
( Xx-1)
g(xt-1 Xx)
Exix
- In (xx,) + Ex 1mpo |xx))
- In g(xt Xx-1)
(23)
(xx x(-1)
T + E ~p((x)
Bx (xxx-1x)(
In
g(x-11 Xx)
Now, we focus on the iterated expectations Ex,, [xx [:]] of the above equality. Based the defi-
nition of KL divergence, it's obvious that
E (xxxpo(xe) (xx-I-Po(xx In P Xe) x(x, x(-1)
x(x4-1
xx) po(x;-1 xx
= =E -p(i) B (x (xs)
In po xx(x111
Xx)
+ -xxxx1
In xx x(-1)
(24)
g
x(xe-1
xx)
poi (xt-1 Xx)
EXL upo (xt- In g(x: x1-1)
= = (xx [KKI(.)] + E (o(x))
Po (x--1 Xx
We denote term Ex, 1px (xx- I-] here and prove that it is a constant. Since the distribution q(x; X1-1) is predefined multivariate Gaussian, we have
q(x: x)= (2TT.:) exp (- |Ix: VI 3ixc-112
28,
(1 [.) (2 B:)) exp (- Xx-1 1x1/V1-1 23)13
(25)
2/3/ BB)
(1 3.) (x /I- Be, 3r/ B)
With this result and the definition of learnable backward probability px xx), convert the constant I, into the following form:
In M(xx-1: o,
]
K In(1 B,).
I
Exi
Ex
(26)
M(xx-ixx//V Br, 3,/ 3.)1
Note that term Ex--1- I.] essentially represents KL divergence between two Gaussian distribu- tions, which is said to have closed-form solution (Zhang et al., 2021):
DKL (N (N((x-11)
(x (xx-1:110xx oe I1.V |1.V(x
3e
X1-1
vI
B,
B
1 2 K In
Br B.)a,
(1 -
33
K
K
+
no
31
Be
VI-
3z
Considering this result, the fact that variance de is primarily set as Be in DDPM (Ho et al. and the definition of mean μas, we can simplify term I,
2020
1 3r E x
||
3
I,
Mx t)
23,
(27)
B1
3.
Ex
t)
2K
14

---

Published as a conference paper at ICLR 2024
Considering assumption that neural network Eg behaves standard Gaussian irrespective of the imput distribution, we can interpret term =xx- I as the total variance of all dimensions of multivariate normal distribution. Therefore, its value is K x 1, the sum of K unit variances. With this result, term Z can be reduced as
BOō at
1)
> O.
(28)
2
at
Combining this inequality with Eq. 23 and Eq. (24) we have
T--1 Exx->po(x)) DKL (po (x xx) |l g(xx-1 xx))] + It Ti + E gmoo + It:
(29)
and its derivation T.- 3 Te + Eqood are both true.
Considering the assumption that the entropy of distribution po(x;) decreases during the backward process (i.e., Hp (x,-) < H pa x,), & t & T), we have
Ei ecumu - pp (x+) + Te + et gomd ecumu + +c,
emad
(30)
which is exactly the propagation equation.
D WEAKENED ASSUMPTION TO THEOREM 3.1
Since neural network co Xx, 1) is designed to fit Gaussian noise e in the loss function Lall, we previ- ously supposed that its output distribution follows standard Gaussian. In proof to Theorem 3.1
that assumption is applied to indicate that [x,, })]²] = K. (K is the vector dimension). How-
ever, considering Eq. (27) and Eq. (88 in the proof, our theorem still holds for [x 0) K. To derive a new assumption, we first set term
Tt E Ea e + vI (), ])
(31)
which indicates the prediction error of neural network eg (x 1). The term will vanish to if the neural network is fully accurate in backward denoising. According to the Triangle Inequality, we then have
> E (c) ) = E[[e] | ?] [[(co
(32)
Since the second moment of Gaussian distribution E| 2 is K, we further have
(4/K) + -I- a,e, |
(rt/K).
(33)
Finally, consider the fact (Ho al. 2020 that x, be reparameterized as v 0g + +v1 ã.C. N(O,I) I) and let the prediction error 7, vanish, the above equation motivates us to make the following new assumption:
1/K) (1ox)E[le(x,, 0)] > 1,
(34)
which is much weaker than the previous assumption but still makes our Theorem 3.1 hold.
E PROOF TO PROPOSITION
Theorem of Wang & Tay (2022) indicates that the following equation is true:
- IIn(1 MMD (Q), !)) DKL
In(MMD P, +0)+))))))) 1),
(35)
if with the Assumption of Wang & Tay 2022 and
JP Zq
is continuous on Ω.
By setting Ω K IP(() Spe( x )dx, Q(-) = 1 )dx; and supposing that (x, 1).
q(x--1) are smooth and zero everywhere, the two conditions of Eq. (35) are both met. Further- more, Wang & Tay (2022) stated that C.2 (92) is a subset of C()) Q). Considering the definitions of the MMD and cumulative emors,, we can turn Eq. (35) into the following equation:
1 Drumu
IIn(1
< E coumu & In
( cumu + 1).
(36)
4
15

---

Published as a conference paper at ICLR 2024
Since In(x 1) & x. In(1 - 2))
4 we can get
Eq geumm
In D Deumu 1) Dr
(37)
geumu
In(1 Drumu
Doumu
-
which are exactly our expected conclusion.
Strictly speaking, also have to verify that Drama < such that the inequality - In(Inn11111 10) 2 is properly applied. Recall that the definition of Drummu as
[ - E (1x
where [6] = supx [6(x] < 1. According to the Triangle Inequality, we have,
Do (|Epo(x .) [[o(xx-1)]] + 1g nld(x(-1)l1)
(38)
Since |a|] is convex function, we can apply Jensen's 's inequality to the above equation:
Deumu
< (Epo(ar ,)[ Bg (aten))]l@((x))1111
(39)
(P Po( (1 1)2 4,
showing that our claim holds.
F TRAINING ALGORITHM
Compared with common practices (Ho et al. [020 Song et al. [021a), we additionally regularize the optimization of DMs with cumulative errors. The details are in Algo. [
Algorithm 1: Optimization with Our Proposed Regularization
Input: Batch size B, number backward iterations T, sample range < T.
while the model is not converged do
Sample a batch of samples from the training set So = (xb x
~ 1(xx))))
Sample time point for vanilla training E (1 T)
Estimate training loss Ci for every real sample x1 E Sp.
Sample time point for regularization s 4 (t L, T), 13.
Sample new batch of samples from the training set S/j = {x)
x {(x &
Sample forward variables Sforw [x forwo
x forw ~ q(x forw.j
x(), x E S6} at step
Sample forward variables Sforw [x
xx
forw,i q(x; for
x)), x E So} at step L.
Sample alternative backward variables at time step s as Sback (xbakk &back, ~ pa
Pa(xback, (bback,
x forw."), xh E ESforw) Sforw).
Estimate cumulative error Cres based on Sforw and Sback
Update the model parameter with gradient (o(Anll Ball + A ).
G RELATED WORK
Exposure bias. The topic of our paper is closely related to a problem called exposure bias that occurs to some sequence models (Ranzato et al. 2016 Zhang et al. (201)). which means that a model only exposed to ground truth inputs might not be robust to errors during evaluation. (Ning et al.) 2023 et al. 2023) studied this problem for diffusion models due to their sequential structure. However, they lack a solid explanation of why the models are not robust to exposure bias, which is very important because many sequence models (e.g., CRF) is free from this problem. Our analysis in Sec. actually answers that question with empirical evidence and theoretical analysis. More importantly, we argue that ADM-IP (Ning et al. 2023), which adds an extra Gaussian noise into the model input, is not an ideal solution, since this simple perturbation certainly can not simulate the complex emrors at test time. We have also treated their approach as a baseline and shown that our regularization significantly outperforms it in experiments (Section 6).
16

---

Published as a conference paper at ICLR 2024
Consistency regularizations. Some papers (Lai et al. 2022 Daras et al. 2023 Song al. 2023
et al. 2023). which aim to improve the estimation of score functions. One big difference between these papers and work is that they assert without proof that diffusion models are affected by emoor propagation, while we have empirically and theoretically verified whether this phenomenon happens. Notably, this assertion is not trivial because many sequential models (e.g.. CRF and HMM) are free from error propagation. Besides, these papers proposed regularisation methods in light of some expected consistencies. While these methods are similar to our regularisation in name, they were actually to improve the score estimation at every time step (i.e., the prediction accuracy of every component in sequential model), which differs much from our approach that aims to improve the robustness of score functions to input errors. The key to solving error propagation to make score functions insensitive to the input cumulative errors (rather than pursuing higher accuracy) (Motter & Lai 2002] Crucitti et al. 2004), because it's very hard to have perfect score functions with limited training data. Therefore, these methods are orthogonal to us in reducing the effect of error propagation and do not constitute ideal solutions to the problem.
Convergence guarantees. Another group of seemingly related papers (Lee et al. 2022 Chen al. 2023b a) aim derive convergence guarantees based on the assumption of bounded score estimation errors. Specifically speaking, these papers analysed how generated samples converge (in distribution) real data with respect increasing discretization levels. However, studies (Motter & Lai 2002 Crucitti et al. 2004) on error propagation generally focus on analysing the error dynamics of a chain model over time (i.e., how input errors to the score functions of decreasing time steps evolve). Therefore, these papers are of very different research theme from our paper. More notably, these papers either assumed bounded score estimation errors or just ignored them, which are actually not appropriate to analyse error propagation for diffusion models. There are two reasons. Firstly, the error dynamics shown in our paper (Fig. 2 have exponentially-Iin increasing trends, implying that the score functions close to time step 0 are of very poor estimation. Secondly, if all components of chain model are very accurate (i.e., bounded estimation errors), then the effect of error propagation will be insignificant regardless of the values of amplification factor. Therefore, it's better to set the estimation errors as uncertain variables for studying error propagation
17


Reviews:
[{"review_id": "ZODaO4t25T", "reviewer": "Reviewer_MiNJ", "summary": "In this paper, the authors investigate error propagation in diffusion models. They develop a framework to define error propagation for diffusion and connect error propagation to generation quality. This enables them to use the measured error as a regularization term during the diffusion model training, improving the generation results of models in small-scale experiments.", "questions": "- Would it be possible to fine-tune pre-trained diffusion models with the regularization term to mitigate this error propagation a-posteriori? If the training time of adding the regularization makes training larger models prohibitive it would be interesting to explore whether it is possible to tune the denoiser network after having trained with just the $L^{nll}$ loss.", "limitations": "", "rating": 8, "confidence": 2, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "- The proposed method is a novel approach to measuring error propagation in diffusion models and offers a new perspective on diffusion model training. The authors argue that apart from making the denoiser network more accurate, which has been the main focus of the literature so far, it is also important to regularize such that the denoiser is also robust to errors in the input during inference. This could have significant impacts on the broader diffusion generative model community.\n\n- The presented methodology is principled and well-explained. The experiments clearly demonstrate the success of the proposed solution in mitigating error propagation in diffusion models.", "weaknesses": "- The authors briefly address the trade-off between increased training time and reduced error propagation (resulting in better FID) for the 32x32 images of CIFAR and ImageNet but do not mention their CelebA experiments on 64x64 images. It is not clear if the benefits scale with the image sizes without an increased overhead as it is possible that the error estimate requires more samples or a larger sampling length $L$.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "1QLryMpgpp", "reviewer": "Reviewer_e2Ed", "summary": "This work analyze the error propagation of diffusion models by introducing the modular error (KL divergence between the reverse conditional distribution at each step), the cumulative error (KL divergence between the marginal distribution at each step). And then introduce an regularization loss based on MMD estimation for the cumulative loss to reduce the cumulative error and improve the sample quality of the trained diffusion model.", "questions": "1. Please address and fix the proof of the main theorem.\n\n2. Please add more detailed descriptions of the experiment settings to ensure reproducibility.\n\n=====================\n\nI've carefully read the authors' responses to other reviewers and AC. I deeply appreciate the authors' efforts to address the concerns, and now I don't have more questions. I think it is a quite interesting and useful technique for improving the training procedure of diffusion models, with the validation of the fine-tuning experiment results and the EDM results. So I raise the score to 6.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The proposed method is easy to understand and the writing is clean and easy to follow.\n- The error propagation of diffusion models is an important question and worth to be studied. The topic is important.", "weaknesses": "Major:\n\n- The assumption in the core theorem is absolutely wrong.\n  - \"suppose that the output of neural network $\\epsilon_\\theta$ follows a standard Gaussian\", which cannot be true. Because the noise-pred model corresponds to the denoising score matching loss, it is proved that the ground truth of such model is propotional to the score function of the distribution, i.e., $\\nabla_{x_t} \\log q_t(x_t)$. For a small $t$, such score function is quite complex and cannot be a simple and single-mode Gaussian distribution, and is far different.\n- Remark 3.2 is not rigorous. The proof requires $T$ goes to infty, but it is not true in practice.\n- Lack of detailed settings of experiments: what is the sampling algorithm for obtaining the FID results? What is the detailed network structure (e.g., layer structure and number of hidden neurons) and amount of parameters?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Ts6C7fokxK", "reviewer": "Reviewer_hknr", "summary": "This paper analyses the error propagation/accumulation in DMs across iterations. The authors prove and empirically verify that the error in DMs cumulatively increases. To minimize this cumulative error as regularization, the authors prove tractable estimates which tightly bound this error, which is then used as a proxy. The authors empirically show the proposed method reduces the cumulative error, and increases generation quality across multiple datasets.", "questions": "1. Contemporaneous work[1] (released after submission deadline) also analyses the sensitivity of DMs to error propagation, and aims to bound this error by scaling the long skip-connections. While the method in [1] is sufficiently different to the proposed method, could the authors comments on this? It would appear that [1] also bounds the error, without the computational overhead.\n2. The proposed method has significant computational overhead (Figure 4). How does the comparisons to  baselines (Table 1) change at equal wall-clock time?\n\n\n\n[1] https://arxiv.org/abs/2310.13545", "limitations": "", "rating": 8, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The theoretical framework and the bounds on error propagation through DMs are useful for analyzing robustness of DMs.\n1. The proposed method results in strong significant improvements across a range of datasets.\n1. The proposed method successfully decreases cumulative error in DMs", "weaknesses": "1. The proposed method requires significant compute overhead, so gains need to be weighed against this increase in compute.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "trhuJ1LAbT", "reviewer": "Reviewer_ebh3", "summary": "This paper presents a theoretical framework to quantify per iteration error in variational inference within the backward pass of difussion models.\nThe authors introduce metrics to theoretically quantify these errors and convincingly demonstrate their increase with iteration number. \nConsequently, the model's performance deteriorates as the number of iterations increases. \nTo mitigate this issue, the authors propose to add the error as a regularization term upon the original loss function. \nEmpirical experiments are conducted to validate the effectiveness of this proposed methodology.", "questions": "Please refer to the questions in the previous section.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "- The paper is very well written and I really enjoys reading this paper.\n\n- The concept of error quantification presented by the authors is both innovative and well-conceived.", "weaknesses": "- Modular Error Definition: The modular error is defined as the expected KL divergence from $p_{\\theta}$ to $q$. However, the KL divergence is not symmetric. Could the authors please explain why the modular error is defined in such a way? Why not using the reversed KL divergence as in the original loss function?\n\n\n- Assumptions in Theorem 3.1: The assumptions regarding the output distribution of the neural network (NN) following a standard Gaussian distribution and the entropy reduction with iteration number appear quite strong. Can the authors justify these two assumptions empirically?\n    \n\n- Technical Issues in the Proof:\n\n    - In the proof of Proposition A.1 in Appendix A, equation (19) holds only in the limit as $T\\to\\infty$. The current proof falls short of demonstrating Proposition A.1 for finite values of $T$.\n\n    - Several typographical errors in equation (24) in Appendix B raise concerns about the validity of the proof. Specifically, the $p_{\\theta}$ term in both the numerator and denominator should be consistent and denoted as $p_{\\theta}(x_{t-1}|x_{t})$ in order for equation (29) to hold. Additionally, the first term in the second line of (24) appears to be missing a logarithmic notation, and the third line of (24) appears inconsistent with the current notation.\n\n- Hyper-Parameter Selection: The authors should provide further clarification on the process for selecting hyper-parameters. Notably, the weight assigned to the regularization term is a critical factor that requires elucidation.\nHow to select the hyper-parameters? One such important factor is the weight of the regularization term, could the authors please clarify?\n\n- Minor Issues:\n\n    - The notation for conditional expectation should be corrected throughout the paper to $E_{X_{t}|X_{t+1}}$.\n    \n    - In Section 3.2, second paragraph, \"get ride of\" should be corrected to \"get rid of\".\n    \n    - The proof for (14), establishing the boundedness of the KL divergence by the MMD, rests on the assumption that $\\log(1-x/4)$ is well-defined, which only holds when $x<4$. The authors should address and clarify this point.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (poster)