O(d/T) Convergence Theory for Diffusion Probabilistic Models under Minimal Assumptions
Authors: Gen Li
Yuling Yan
Venue: ICLR 2025 Poster (2025)
Keywords: score-based generative model
diffusion model
denoising diffusion probabilistic model
sampling
Primary area: learning theory

Abstract: Score-based diffusion models, which generate new data by learning to reverse a diffusion process that perturbs data from the target distribution into noise, have achieved remarkable success across various generative tasks. Despite their superior empirical performance, existing theoretical guarantees are often constrained by stringent assumptions or suboptimal convergence rates. In this paper, we establish a fast convergence theory for the denoising diffusion probabilistic model (DDPM), a widely used SDE-based sampler, under minimal assumptions. Our analysis shows that, provided $\ell_{2}$-accurate estimates of the score functions, the total variation distance between the target and generated distributions is upper bounded by $O(d/T)$ (ignoring logarithmic factors), where $d$ is the data dimensionality and $T$ is the number of steps. This result holds for any target distribution with finite first-order moment. To our knowledge, this improves upon existing convergence theory for the DDPM sampler, while imposing minimal assumptions on the target data distribution and score estimates. This is achieved through a novel set of analytical tools that provides a fine-grained characterization of how the error propagates at each step of the reverse process.

Published as a conference paper at ICLR 2025
O(d/T) CONVERGENCE THEORY FOR DIFFUSION PROBABILISTIC MODELS UNDER MINIMAL ASSUMPTIONS
Gen Li*
Yuling Yan"
Department of Statistics
Department of Statistics University of Aaeconsin-Madisee
The Chinese University of Hong Kong
Hong Kong
Madison, WI 53706, USA
gen (eecuuk.edu hk
yul i iin. yan waan@wisc.ede
ABSTRACT
Score-based diffusion models, which generate new data by learning to reverse a diffusion process that perturbs data from the target distribution into noise, have achieved remarkable success across various generative tasks. Despite their superior empirical performance, existing theoretical guarantees are often constrained by stringent assumptions or suboptimal convergence rates. this paper, we establish fast convergence theory for the denoising diffusion probabilistic model (DDPM), a widely used SDE-based sampler, under minimal assumptions. Our analysis shows that, provided ly-accurate estimates of the score functions, the total variation distance between the target and generated distributions is upper bounded by (d/TT (ignoring logarithmie factors), where d is the data dimensionality and 7 is the number of steps. This result holds for any target distribution with finite first-order moment. To our knowledge, this improves upon existing convergence theory for the DDPM sampler, while imposing minimal assumptions on the target data distribution and score estimates. This achieved through novel set of analytical tools that provides fine-gu tne-grained characterization of how the error propagates each step the reverse process.
INTRODUCTION
Score-based generative models (SGMs) have emerged powerful class of generative frameworks, capable of lcarning and sampling from complex data distributions (Sohl-Dickstein et al., 2015; Ho al., 2020; Song et al., 2021b; Song & Ermon, 2019; Dhariwal & Nichol, 2021). These models, including Denoising Diffusion Probabilistic Models (DDPM) (Ho et al., 2020) and Denoising Diffusion Implicit Models (DDIM) (Song et al., 2021a), operate by gradually transforming simple noise-like distribution (e.g., standard Gaussian) into a target data distribution through series of diffusion steps. This transformation achieved by learning a sequence of denoising processes governed by score functions, which estimate the gradient of the log-density of the data at each step. SGMs have demonstrated remarkable success in various generative tasks, including image generation (Rombach et al., 2022: Ramesh et al., 2022; Saharia et al., 2022), audio generation (Kong et al., 2021), video generation (Villegas et al., 2022), and molecular design (Hoogeboom et al., 2022). See e.g., Yang et al. (2023); Croitoru et al. (2023) for overviews of recent development.
At the core of SGMs are two stochastic processes: a forward process, which progressively adds noise the data,
Xp - X1 4... Xx.
where Xo is drawn from the target data distribution Pdata and is gradually transformed into Xy that resembles standard Gaussian noise; and a reverse process,
Yr Yr-1 + Yo.
"The authors contributed equally.

---

Published as a conference paper at ICLR 2025
which starts from pure Gaussian noise Yr and sequentially converts it into Yo that closely mimics the target data distribution Pdata- At each step, the distributions of Y2 and X₁ are kept close. The key challenge lies in constructing this reverse process effectively to ensure accurate sampling from the target distribution.
Motivated by classical results on the time-reversal of stochastic differential equations (SDEs) (Anderson, 1982; Haussmann & Pardoux, 1986), SGMs construct the reverse process using the gradients of the log marginal density of the forward process, known as score functions. At each step, Y is generated from Y, with the help of the score function V logpx,() (-), where px, denotes the density of X,. Both the DDPM sampler (Ho et al., 2020) and the DDIM sampler (Song et al., 2021a) follow this denoising framework, with the key distinction being whether additional random noise is injected when generating each Y Although the score functions are not known explicitly, they are pre-trained using large neural networks through score-matching techniques (Hyvärinen, 2005; 2007; Vincent, 2011; ;; Song & Ermon, 2019).
Despite their impressive empirical success, the theoretical foundations of diffusion models remain relatively underdeveloped. Early studies on the convergence of SGMs (De Bortoli et al., 2021; De Bortoli, 2022; Liu et al., 2022; Pidstrigach, 2022; Block et al., 2020) did not provide polynomial convergence guarantees. In recent years, a line of works have explored the convergence of the generated distribution to the target distribution, treating the score-matching step focusing on the effects of the number of steps 7' and the score estimation error on the convergence of the sampling phase (Chen al., 2023c;a; 2024; Benton et al., 2023a; Lee al., 2022; 2023; Li et al., 2023; 2024b; Li & Yan, 2024; Gao & Zhu, 2024; Huang et al., 2024; Tang & Zhao, 2024; Liang al., 2024; Chen et al., 2023d). Recent studies have investigated the performance guarantees of SGMs in the presence of low-dimensional structures (e.g.. Li & Yan (2024); Tang & Yang (2024); Chen et al. (2023b); Wang et al. (2024)) and the acceleration of SGMs (e.g., Li al. (2024a); Liang et al. (2024)). Following this general avenue, the goal of this paper is to establish sharp convergence theory for diffusion models with minimal assumptions.
black box and
Prior convergence guarantees. In recent years, flurry of work has emerged on the convergence guarantees for the DDPM and DDIM type samplers. However, these prior studies fall short of providing fully satisfactory convergence theory due least one of the following three obstacles:
Stringent data assumptions. Farlier works, such Lee al. (2022), required the target data distribution to satisfy the log-Sobolev inequality. Similarly, Chen et al. (2023c); Lee et al. (2023); Chen al. (2024; 2023d) assumed that the score functions along the forward process must satisfy a Lipschitz smoothness condition. More recent work Gao & Zhu (2024) relied on the strong log- concavity assumption of the target distribution to establish convergence guarantees in Wasserstein distance. These assumptions are often impractical to verify and may not hold for complex distributions commonly seen in image data. Some newer studies on the DDPM sampler (e.g.. Chen et al. (2023a); Benton et al. (2023a)) and the DDIM sampler (e.g., Li et al. (2024b)) have relaxed these stringent assumptions, and their results applied to more general target distributions with bounded second-order moments sufficiently large support.
Slow convergence rate. We follow most existing works and focus the total variation (TV) distance between the target and the generated distributions.! I Let T be the number of steps and d be the dimensionality of the data. For the DDPM sampler, Chen et al. (2023c) established convergence rate of O(Lv(d M2)/T), where L is the Lipschitz constant of the score functions along the forward process, and M2 is the second-order moment of the target distribution. Later, Chen et al. (2023a) lifted the Lipschitz. condition, but this came at the cost of a rate (ddvTT) with worse dimension dependence. The Mfte-offthhe-at result for the DDPM samplers is due to Benton et al. (2023a), achieving a convergence rate of O O ddT)). However, this still slower than the convergence rate for the DDIM sampler, achieved in Li et al. (2024b), which attains (ddTT) in the regime >> dZ.
Additional score estimation requirements. Convergence theory for diffusion models must also account for the impact of imperfect score estimation on performance. While recent results for the DDPM sampler (Chen et al., 2023c;a; Benton et al., 2023a) require only accurrae score function
'Convergence rates in Kullback-Lecie Leibler (KI.) divergence in Chen et al. (2023a); Benton al. (2023a) are transferred to TV distance using Pinsker's inequality, because the KI. divergence not distance.
2

---

Published as a conference paper at ICLR 2025
Convergence rate (in TV distance)
Data assumption
Requirement' score estimates 8
Sampler
(No Paats
log PX,)
DDPM
L-Lipschitz s
LVd/T
8, ~ of in (pxx))
(Chen et al., 2023c)
E
Xo
00
DDPM
/T
||||
E
Xo ||
00
8t i in (px.))
(Chen al., 2023a)
DDPM
Va/T
Xool
<0
R. 8 in (pxx),
(Benton et al., 2023a)
DDIM
I--Lipschitz s/:
I-Liipshitzz st;
L'//dTT
(Chen et al., 2024)
E
Xx
00
8, 8 5 in (px,)
DDIM
9, S in (px.);
d
bounded support
(Li al., 2023)
Jr, x Js; in (pxx)
DDIM
9t = st in (pxx);
d/T when T T &
bounded support
(Li et al., 2024b)
JR, = Jsp in T.2 (px,)
DDPM
d/T
E Xolla]
8; = i L (px.)
(this paper)
Table 1: Comparison with prior convergence guarantees for diffusion models (ignoring log factors). Convergence rates in KL divergence are transferred to TV distance using Pinsker's inequality. Here J Rd
pdxd denotes the Jacobian matrix of a function / Rd - RRd.
estimates, another line of work on the DDIM sampler (Li et al., 2023; 2024b; Huang et al., 2024) achieves faster convergence rates, albeit under strieter requirements for score estimates. Specifically, Li et al. (2023; 2024b) require not only that the score estimates be close to the true score functions, but also that the Jacobian of the score estimates be close the Jacobian of the true score functions, which is significantly stronger condition. Additionally, Huang et al. (2024) assumes higher-order smoothness in the score estimates.
From this discussion, it is evident that while the state-of-the-ae convergence rates for the DDIM sampler surpass those for the DDPM sampler, they also rely on more restrictive assumptions. This motivates us to think whether it is possible to achieve the best of both worlds, namely,
Can we establish a convergence theory for diffusion models that achieves fast convergence rate under minimal data and score estimation assumptions?
As noted in Li al. (2024b), counterexample demonstrates that aa-accurate score estimation alone is insufficient for convergence of the DDIM sampler under TVV distance. The current paper answers this question affirmatively by focusing on the DDPM sampler.
This paper develops a fast convergence theory for the DDPM sampler under minimal assumptions. We show that the TV distance between the generated and target distributions is bounded by:
Our contributions.
d T
-
stx,)
up to logarithmic factors. The first term reflects the discretization eroor while the second term accounts for score estimation error. Compared to the two most relevant works (Benton et al., 2023a; Li et al., 2024b), which provide state-of-the-art results for the DDPM and DDIM samplers, main contributions are as follows:
(d/T) convergence rate. Under perfect score function estimation, establish Od/TT) convergence rate for the DDPM sampler T'V distance, improving on the previous best rate of O(/a/T) from Benton et al. (2023a). Our result also matches the convergence rate of the DDIM sampler achieved in Li et al. (2024b), but more general, as their result only holds when T > d? while applies for arbitrary T and d.
3

---

Published as a conference paper at ICLR 2025
Minimal assumptions. Our theory requires only that the target distribution has finite first-order moment, which, to the best of our knowledge, is the weakest data assumption in the current literature. Additionally, require only l- laaacurate score estimates, which is significantly weaker condition than the Jacobian accuracy required by Li et al. (2023; 2024b).
In summary, our results achieve the fastest convergence rate in the literature for both DDPM and DDIM samplers while requiring minimal assumptions. A comparative summary with prior work is presented in Table I.
PROBLEM SET-UP
In this section, we provide an overview of the diffusion model and the DDPM sampler.
Forward process. We consider Markov process in a starting from Xo Pdata. evolving according to the recursion:
X, V B.X,-1 + BBW, 11= 7).
(2.1)
where Wi...., Wr are independent draws from (00, Ia), and $I...., Br E (0, 1) are the learning rates, For each 1 < l < T, define ar == 3r and oe = I as. This allows us to express X, in closed form
X, = Vo + -1-a, W, where W, ~ La)
(2.2)
We select the learning rates such that (i) B, is small for every S < T; and (ii) or is vanishingly small, ensuring that the distribution of Xr is exceedingly close to (0, La). In this paper, adopt the following learning rate schedule
1 T
61 log T
min 31 (1+ Ci log T
B1
B.+1
(1= T (2.3)
T
for sufficiently large constants co: C1 > O. This schedule is commonly used in the diffusion model literature (see, e.g., Li et al. (2023; 2024b)), although the results in this paper hold for any learning rate schedule satisfying the conditions in Lemma
Reverse process. The crucial elements in constructing the reverse process are the score functions associated with the marginal distributions of the forward diffusion process (2.1). For each T, I,...,T, we define the score function as:
$ (x) = V logpx, (1=1,...,,
where Px,(5 (-) represents the smooth probability density of X,. Since the true score functions are typically unknown, we assume access to estimates s-(-) for each s (-). To quantify the error in these estimates, we define the averaged l score estimation eтror as:
-[ (5 12.)
score T
This error term quantifies the effect of imperfect score approximation in theoretical analysis. Using these score estimates, we can construct the reverse process, which starts from Yr ~ 10, and evolves as: and proceeds as
Y
(1- + -1- 20,2)) (T 1),
(2.4)
where Zi...., Zr are independent draws from NO, Ia). This is the popular DDPM sampler (Ho al., 2020). Although not the primary focus of this paper, we also include the definition of another widely-used ODE-based sampler (Song et al., 2021a):
1- a (Yi)) (Y,
Yı-1
(t = T, T....,1...
Yr N0 La),
(2.5)
Cxe
2
which frequently appears in discussions.
4

---

Published as a conference paper at ICLR 2025
3 MAIN RESULTS
In this section, we will establish fast convergence theory for the DDPM sampler under minimal assumptions. Before proceeding, we introduce the only data assumption that our theory requires. Assumption 1. The target distribution Pdata has finite first-order moment. Furthermore, assume that there exists some constant CMd > O such that
M1 Xoll2l TEN
Here require the first-order moment M1 to be at most polynomially large in T, which allows cleaner and more concise result that avoids unnecessary technical complicacy. Since см > can be arbitrarily large, we allow the target data distribution to have exceedingly large first-order moment, which is a mild assumption.
Now are positioned present convergence theory for the DDPM sampler.
Theorem 1. Suppose that Assumption I holds. There exists some universal constant c 0 such that the DDPM sampler (2.4) satisfies
TV(px,,Py)) S dlog $ T
+ CE VloTT
(3.1)
T
The two terms in the error bound (3.1) correspond to discretization error and score matching error, respectively. A few remarks are in order.
Sharp convergence guarantees. Consider the setting with perfect score estimation (i.e., Escore 0) and ignore any log factor. Theorem reveals that the DDPM sampler converges at the order of O(d/T) in total variation distance, suggesting iteration complexity of order d/s for achieving e-accuracy, for any nontrivial target accuracy level ε E (0, 1). This improves the state-of-the-art convergence rate O(yd/T) in TV distance for the DDPM sampler (Benton et al., 2023a). is important note that the bound in Benton al. (2023a) was originally stated in terms of KL. divergence, and here we apply Pinsker's inequality to translate their result into TV distance. Our theory does not, however, provide improved convergence rates under KL divergence. Turning to the ODE-based sampler (2.5), Li et al. (2024b) achieved the same O(d/T) convergence rate, but only the regime T d. Our result holds for general T and d. including the regime T d, hence is more general.
Stability vis-â-vis imperfect score estimation. The score estimation error in (3.1) is linear in Esoore which suggests that the performance of the DDPM sampler degrades gracefully when the score estimates become less accurate. In other words, our theory holds with fz-acuraate score estimates, consistent with recent work on the DDPM sampler (Chen et al., 2023e;a; Benton al., 2023a). In comparison, the convergence bound in Li et al. (2024b) for the ODE-based sampler reads
d TV(p S T Videscoee + da where Jacobi
1
Σ
T
E
Ost dx
(X,)
Os: dr
T
t=1
(3.2)
which exhibits worse stability against imperfect score estimation. First, the term involving Eccore Escoe in their bound (3.2) is amplified by factor of Vd compared to our bound (3.1). Second, their bound includes an additional term proportional to Eaacobi meaning their theory requires the Jacobian of S, closely match that of sf, which a more stringent requirement.
Minimal data assumption. The only data assumption is Assumption 1, which requires that the first-order moment Mj of the target distribution is at most polynomially large in T. In comparison, Assumption is weaker than the finite second-order moment condition in e.g., Chen et al. (2023c;a); Benton et al. (2023a) and bounded support condition in e.g., Li et al. (2023; 2024b). In fact, by slightly modifying the proof, we can further relax Assumption I to accommodate target data distributions with polynomially large 8-th order moment
M3 = X
1/8 TCM,
for any constant 5 > 0. The same bound (3.1) holds, provided that T maxx118-1]dlo TT
5

---

Published as a conference paper at ICLR 2025
Error metric. Theorem provides convergence guarantees to px, instead of the target data distribution (i.e.. the distribution of Xo), which is similar to the results in e.g. Chen et al. (2023a); Benton et al. (2023a); Li et al. (2023; 2024b). On one hand, since X1 v1 X + /31 W and 31 = T- is vanishingly small, the distributions of X and X, are exceedingly close. Hence (Vx,,, PY; ) is valid error metric. On the other hand, the smoothness of px, allows to circumvent imposing any Lipschitz assumption on the score functions, which provides technical benefit for the analysis.
It is worth noting that most previous studies on the convergence of the DDPM sampler (e.g.. Chen et al. (2023c;a); Benton et al. (2023a); Li et al. (2023); Li & Yan (2024)) typically begin by upper bounding the squared TV error using the KL divergence of the forward process from the reverse process. This done through the following argument:
1 (Vx, PY:) i KL (px. Ilpx) S 2 Il
(3.3)
where the first inequality follows from Pinsker's inequality and the second from the data- pata-procsssing inequality. The KL divergence on the right-hand side of (3.3) is more tractable and can be further bounded, for example, using Girsanov's theorem. In fact, (Chen et al., 2023c, Theorem 7) provides theoretical evidence that the KL divergence between the forward and reverse processes is lower bound by (dd/T), even when the target distribution is as simple as a standard Gaussian and perfect score estimates are available. This suggests that such an approach cannot yield error bounds better than O ((///T) in general.
To achieve a sharper convergence rate, we take a different approach by directly analyzing the total variation error without resorting to intermediate KL divergence bounds. Specifically, we establish fine-grained recursive relation that tracks how the error TV(px,,Py, Pr,) propagates through the reverse process as t decreases from to See Section 4 for more details.
PROOF OF THEOREM
4.1 PRELIMINARIES
For each 1 < < T and any x E RR, it is known that the score function (z) associated with px. admits the following expression
1
Cre f (xxlxx [x) Vā =: I 9r (x).
1
s1(x)
Let J(()) (1) (2) be the Jacobian matrix of (x), which can be expressed
1 1{(1))
{(/o Pxolxx
x) varoo)dro) ( (x0] vārzo)
J((x) 1- αi
(2o (2o
PXolX, 1x v (sr vārzo) " +T.
(4.1)
It is straightforward check that Jz(xz) 2 0. The following lemma will be useful in the analysis.
Lemma 1. Suppose that x E Rd satisfies - px, (x) S Od log T for any given 0 2 1. Then we have
(0 doggT
(0)(2 S 5
and
Tr(I (((=)((=)) < 12(0 co)d
where the constant co > is defined in (2.3). In addition, there exists universal constant Co > such that
T a: / Cke
Σ
(xx(==))lf S Codlog?.
2
Proof. See Appendix A.I.

---

Published as a conference paper at ICLR 2025
For some sufficiently large constants C1, C2.C2 O, we define for each 2 T the set
E111 = [x lx, I Cid T, || S Va,TZeR + C2v a+) log {
(4.2)
Define the extended d-dimensional Euclidean space IR U } by adding point oo to RR. From now on, the random vectors can take value in R {oo}, namely, they can be constructed in the following way:
X X', with probability 8, 0o,
with probability - 0,
where 0 E [0, 1] and X' is a random vector Rd the usual sense. If has a density, denoted by (x)()), then the generalized density of X is
px(x) 0px (z) 1f2 E Rd) (1 (1-0)8 8)
To simplify presentation, will abbreviate generalized density to density.
4.2
STEP 1: INTRODUCING AUXILIARY SEQUENCES
We first define an auxiliary reverse process that uses the true score function:
Y7 NNO, Ta); Y Va, (y; + az)ss(())) vI vl for ,....,,. (4.3)
To control discretization error, we introduce an auxiliary sequence {t T.... ,1 along with intermediate variables (Y, t T...., ...,,, as follows.
1. (Initialization) Define Y, Yr if Yr Er,1 and Y otherwise. The density of Y is
Pr, (UT) = PYr (VT) (UT Era] + yEEF,
(v)dy5
(4.4a)
2. (Transition from Y, Y,) For L = T,..., ,.....,,,.. 1, the conditional density of Y, given Y, = Vi is
px, (3)
1000r + (1 min
(x. Pr, (yE)
1))
PV, (yh (0l1))) min
800
(4.4b)
PT. (yr)
3. (Transition from Y, to F, For = T.... 2. the conditional density of Y given Y, = Yt is
defined follows: if yt Et,1, then
Py. F.. (Wi-1 v1) pY: Pr: (3
(4.4c)
otherwise, we let P (Vī= 111+ = -occ
This defines a Markov chain
+8, PI
(4.5)
An important consequence of the construction (4.4b) is that, for any 1/ oo,
Py, (B6) = ff PY,IH, (ye 9e )p, (yr Jdy = min (px. (v)) Pr, (1+)
(4.6)
To control estimation error, introduce another auxiliary sequence (Ýr :t ..... 11 along with intermediate variables (9, :t T,.., 1} as follows.
1. (Initializationn Let 8- -Y
2. (Transition from Y,- to Y.) For t = T, 1, the conditional density of Y, given 8,- = 3r is
P7.18. (v. 1y, = Pv,IV- (yr 1y5).
(4.7a)

---

Published as a conference paper at ICLR 2025
3. (Transition from Y, to Y For t defined as follows: if yt E Ei,,, then
,,,..,2 the conditional density of y given Y, =y1 is
Pr: 18, (Vi-1 lve) PY, IM
(4.7b)
otherwise, we let pp, IP, (VT-1 l1t) = 800-
This defines another Markov chain
Yr + 9
F- Pr-1
87- - Vi.
(4.8)
which is similar to (4.5) except that now the transitions from Y; to V; are constructed using the estimated score functions. We can use induction show that
(y:) 2 py, (y)),
V/y + 00
(4.9)
holds for all / = T,.... 1. First, it is straightforward to check that (4.9) holds for 1 = T. Suppose that (4.9) holds for t + 1. Then for any y * oc, we have
pg, p9,(31) (36) = ( PR.18, (yt 1 )PS (yt ddy @ min (px, (we) (2t), 1399, (y6)
$ pV.- (y6) [ PV:- 8i1a (vr 10t1)pp, (uc+))dy+1
3 / (we Me+ ()v (1++ )dy++ (W:).
Here step (i) follows from (4.7a) and (4.4b), while step (ii) follows from the induction hypothesis and (4.7b).
4.3 STEP CONTROLLING DISCRETIZATION ERROR
In this section, we will bound the total variation distance between px, and py, For each / T, T,..., 1,
(x = px. (x) PY, (x),
V
(4.10)
We emphasize that A.(-) is not defined at co. In view of (4.6), we know that A ( > 0 for any Tt # 00. The following lemma characterizes the propagation of the error f Ax(r)d through the reverse process.
Lemma 2. There exists some universal constant C1 > such that, for ......2
[ & / + C. ( (1 In addition, we have f Ar(x)dr < 4
/
1--4)) {(
Te JanEE. (dlogT +
+ T3
Proof. See Appendix A.2.
We can apply Lemma 2 recursively to achieve
f (i())dx s / Ar(ao)da +
Σ
0+
/
(dlog [1x(()))))))xx(x 7-3
0z
JZIEE,
< Cc
logT T
Σ
T
at
(u(e)))ixx.x + C,
d dlog T
+ T-
T
Ti
T
2m
dlogg 2 T
dlog T
7 < Cs dlog² 3T
8q
CoCo
+ 64
T
T
T
Here step (a) utilizes Lemma 7: step (b) follows from Lemma 1; while step (c) holds provided that Cs > CCC. This further implies that
dlogs T
TV(p (x)2p7 (x)
(px, (x) - -V, (r))dx = (1(2)) S Cs
T (4.11)
8

---

Published as a conference paper at ICLR 2025
4.4 STEP 3: CONTROLLING ESTIMATION ERROR
In this section, we will bound the total variation distance between py, and py, Note that
{ (Pr, (z) - pr, (x) 1 {p>, (2) > px, (2) + P( = 00)
PV(,
Rd
i
(D7, (x) - -Pp, (x)) 1 { (2) > Po (x) P(V = 00)
Rd
(11) TV + TV( S v KL ((pp;) + C3
dlogg
(4.12)
T
Here step (i) follows from (4.9); step (ii) follows from 1YY, oo) S TV(px, Py which holds
since X, does not take value at oo; step (iii) utilizes Pinsker's 's inequality and (4.11). Hence it suffices to bound KL(PV, p., ), which can be decomposed into
(2) (L ) S (L 1
1p9,, 8r.
(b) KL 1Ipp
+
E PemPy
KL (L
1-,1,,,=0 lIpp. (P
T
E
KKL (LY, =x, |1p.,1, =x,
=x,)
T
(c)
[KL ,/V. i
(4.13)
Here step (a) follows from the data data-processing inequality; step (b) uses the chain rule of KL divergence, where we use the fact that (4.5) and (4.8) are both Markov chains; step (c) follows from the facts that, by construction, Y = VT= and for any x of co, the conditional distributions of Y given Vi- = x and Y, given V, = x are identical. For any I E E,,,, we have
1 1- at (xx( st (2)))
(i) log T |/ s (1=)))
(4.14)
Here step (i) follows from the transition probability (4.4c) and (4.7b), which give
Te -1 a,)s (x,)) 1 at Ta
Y V Y. = =x
and
Vai
a2
P = = Et + (1 an)s) (1-
at Id
a₁ and the KL divergence between two Gaussian measures be computed in closed-form; step (ii)
Vae
utilizes Lemma 7. On the other hand, for any E Ef we have
KL (P (P=m)) =0.
(4.15)
Therefore we have
T
KL (p
|| pp.)
En,xxx 1=2
E
KL KL "i,,i,.== || pp. Pe =
5
C1 2
2
score
log T.
(4.16)
Here step (i) follows from (4.13) and the relation Py, (x) S px, (x) for any x * do (see (4.6)); while step (ii) follows from (4. and (4.. Substitution of the bound (4.16) into (4.12) yields
TV Py,) I log T E + Cs dlog" T
(4.17)
2
T
Taking the two bounds 11) and (4.17) collectively, we achieve the desired result
TV(px, ((px,.Pr)) S TV(px. (V(.x. SC dlog' for constant C> Vo + Cs.
ccdlog T
CEscore Tog
T

---

Published as a conference paper at ICLR 2025
5 DISCUSSION
In this paper, we establish an (dd/T) convergence theory for the DDPM sampler, assuming access 22-accurate score estimates. This significantly improves upon the state-of-the-art convergence rate of O /d/T)) in Benton et (2023a). Compared to the recent work Li al. (2024b) for another ODE-based sampler, which also achieves rate of O(d/T", our result relaxes the stringent score estimation requirements, such as the need for the Jacobian of the score estimates to closely match that of the true score functions.
This work opens several promising directions for future research. First, it remains unclear whether the Odd/T) is tight for the DDPM sampler; it would be of interest to develop lower bounds on certain hard instances. Additionally, when the target data distribution is concentrated on or near low-dimensional manifolds embedded in a higher-dimensional space such as in the case of image data - an important question is whether sharp convergence rate can be established based on the intrinsic dimension k, rather than the ambient dimension d? Existing work (Li & Yan, 2024) provides rate of Ol D(vk//T), and extending analysis improve upon this result would be highly valuable. Lastly, another intriguing direction is to explore whether the analysis in this paper can extend to developing convergence theory in Wasserstein distance (e.g.. Gao & Zhu (2024); Benton et al. (2023b)).
ACKNOWLEDGEMENTS
Gen Li is supported in part by the Chinese University of Hong Kong Direct Grant for Research.
REFERENCES
Brian DO Anderson. Reverse-time diffusion equation models. Stochastic Processes and their Applications, 12(3):31-3326, 1982.
Joe Benton, Valentin De Bortoli, Arnau Doucet, and George Deligiannidis. Linear convergence bounds for diffusion models via stochastic localization. arXiv preprint arXiv:2308.03666 2023a.
Joe Benton, George Deligiannidis, and Arnaud Doucet. Error bounds for flow matching methods. arXiv preprint arXiv:2305. 16860, 2023b.
Adam Block, Youssef Mroueh, and Alexander Rakhlin. Generative modeling with denoising auto- encoders and Langevin sampling. arXiv preprint arXiv:2002.00107, 2020.
Hongrui Chen, Holden Lee, and Jianfeng Lu. Improved analysis of score-based generative modeling: User-friendly bounds under minimal smoothness assumptions. In International Conference on Machine Learning, pp. 4735-4763. PMLR, 2023a.
Minshuo Chen, Kaixuan Huang, Tuo Zhao, and Mengdi Wang. Score approximation, estimation and distribution recovery of diffusion models on low- diw-einnsiiona data. International Conference Machine Learning, pp. 4672-4712 PMLR, 2023b.
Sitan Chen, Sinho Chewi, Jerry Li, Yuanzhi Li, Adil Salim, and Anru Zhang. Sampling is as easy as learning the score: theory for diffusion models with minimal data assumptions. In The Eleventh International Conference on Learning Representations, 2023c. URL net/Eorumt idd 2YUVMg
Sitan Chen, Giannis Daras, and Alexandros G Dimakis. cestoration-degraaatioo beyond linear diffusions: A non-asymptotic analysis for DDIM-type samplers. arXiv preprint 333::23033003844. 2023d.
Sitan Chen, Sinho Chewi. Holden Lec, Yuanzhi Li, Jianfeng Lu, and Adil Salim. The probability flow ode provably fast, Advances in Neural Information Processing Systems, 36, 2024.
Florinel-Alin Croitoru, Vlad Hondru, Radu Tudor Ionescu, and Mubarak Shah. Diffusion models in vision: A survey. IEFE Transactions Patlern Analysis and Machine Intelligence, 2023.
10

---

Published as a conference paper at ICLR 2025
Valentin De Bortoli. Convergence of denoising diffusion models under the manifold hypothesis. arXiv preprint 0rXiv:2208.05314 2022.
Valentin De Bortoli, James Thornoon, Jeremy Heng, and Armaud Doucet. Diffusion Schrödinger bridge with applications to based generative modeling. Advances in Neural Information Processing Systems, 34:17695-17709, 2021.
Prafulla Dhariwal and Alexander Nichol. Diffusion models beat GANs on image synthesis. Advances in Neural Information Processing Systems, 34:8780-8794, 2021.
Xuefeng Gao and Lingjiong Zhu. Convergence analysis for general probability flow odes of diffusion models in wasserstein distances. arXiv preprint arXiv:2401.1995 2024.
Ulrich G Haussmann and Etienne Pardoux. Time reversal of diffusions. The Annals of Probability, pp. 1188 1205,, 1986.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems, 23:6840-6835 2020.
Emiel Hoogeboom, Victor Garcia Satorras, Clément Vignac, and Max Welling. Equivariant diffusion for molecule generation in 3d. In International conference on machine learning, pp- 8867-8887. PMLR, 2022.
Daniel Zhengyu Huang, Jiaoyang Huang, and Zhengjiang Lin. Convergence analysis of probability flow ode for score-based generative models. arXiv preprint arXiv:2404.009730 2024.
Aapo Hyvärinen. Estimation of nonmnormdl statistical models by score matching. Journal of Machine Learning Research, 6(4), 2005.
Aapo Hyvärinen. Some extensions of score matching. Computational statistics & data analysis. 51 (5)2249-22112, 2007.
Zhifeng Kong, Wei Ping, Jiaji Huang, Kexin Zhao, and Bryan Catanzaro. DiffWave: A versatile diffusion model for audio synthesis. In International Conference Learning Representations, 2021.
Beatrice Laurent and Pascal Massart. Adaptive estimation of a quadratic functional by model selection. Annals of statistics, pp. 1302-1338. 2000.
Holden Lee, Jianfeng Lu, and Yixin Tan. Convergence for score-based generative modeling with
polynomial complexity. In Advances in Neural Information Processing Systems, 2022.
Holden Lee, Jianfeng Lu, and Yixin Tan. Convergence of score-based generative modeling for general data distributions. In International Conference on Algorithmic Learning Theory, pp. 946 985, 2023.
Gen Li and Yuling Yan. Adapting to unknown low-dimensional structures in score-based diffusion models. arXiv preprint arXiv:2405. 14861, accepted to NeurIPS 2024, 2024.
Gen Li, Yuting Wei, Yuxin Chen, and Yuejie Chi. Towards non-asymptotic convergence for diffusion-based generative models. In The Twelfih International Conference on Learning Representations, 2023.
Gen Li, Yu Huang, Timofey Efimov, Yuting Wei, Yuejie Chi, and Yuxin Chen. Accelerating convergence of score-based diffusion models, provably. arXiv preprint arXiv:2403.03852 2024a.
Gen Li, Yuting Wei, Yuejie Chi, and Yuxin Chen. A sharp convergence theory for the probability flow odes of diffusion models. arXiv preprint M0Xi:22408.0232000 2024b.
Yuchen Liang Peizhong Ju, Yingbin Liang, and Ness Shroff. Non-asymptotic convergence of discrete-time diffusion models: New approach and improved rate. arXiv preprint 13900 2024.
Xingchao Liu, Lemeng Wu, Mao Ye, and Qiang Liu. Let us build bridges: Understanding and extending diffusion generative models. arXiv preprint arXiv:2208 14699, 2022.
11

---

Published as a conference paper at ICLR 2025
Jakiw Pidstrigach. Score-based generative models detect manifolds. arXiv:2206.01018, 2022.
arXiv preprint
Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text- conditional image generation with CLIP latents. arXiv preprint arXiv:2204.0611 06112,,, 2022.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image synthesis with latent diffusion models. In IEEE/CVF Conference on Computer Vision and Pultern Recognition, pp. 10684-10695, 2022.
Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang. Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Bureu Karagol Ayan, Tim Salimans, et al. Photorealistic text-to-image diffusion models with deep language understanding. Advances in Neural Information Processing Systems, 35:36479-36494, 2022.
Jascha Sohl-Dickstein. Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In International Conference on Machine Learning, pp. 2256-22655 2015.
Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. In International Conference on Learning Representations, 2021a. WRRLtttps://openeeeeee net/ / Eorum? iidiiid--s St Str-tttlgiii StgiirrHL
Yang Song and Stefano Ermon. Generative modeling by estimating gradients of the data distribution. Advances in neural information processing systems, 32, 2019.
Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. International Conference Learning Representations, 2021b.
Rong Tang and Yun Yang. Adaptivity of diffusion models to manifold structures. In International Conference Artificial Intelligence and Statistics, pp. 1648-1656. PMLR, 2024.
Wenpin Tang and Hanyang Zhao. Score-based diffusion models via stochastic differential
equations-a technical tutorial. arXiv preprint arXiv:2402.07487, 2024.
Roman Vershynin. fligh-dimensional probability: An introduction with applications in data science, volume 47. Cambridge university press, 2018
Ruben Villegas, Mohammad Babaeizadeh, Pieter-Jan Kindermans, Hernan Moraldo, Han Zhang, Mohammad Taghi Saffar, Santiago Castro, Julius Kunze, and Dumitru Erhan. Phenaki: Variable length video generation from open domain textual deseriptions. In International Conference on Learning Representations, 2022.
Pascal Vincent. connection between score matching and denoising autoencoders. Neural computation, 23(7):1661-1674, 2011.
Peng Wang, Huijie Zhang, Zekai Zhang, Siyi Chen, Yi Ma, and Qing Qu. Diffusion models learn low-dimensional distributions via subspace clustering, arXiv preprint arXiv:2409.02422 2024.
Ling Yang. Zhilong Zhang, Yang Song. Shenda Hong, Runsheng Xu, Yue Zhao, Wentao Zhang, Bin Cui, and Ming-Hsuan Yang. Diffusion models: A comprehensive survey of methods and applications. ACM Computing Surveys, 56(4):1-39, 2023.
A PROOF OF AUXILIARY LEMMAS
A. PROOF OF LEMMA 1
For any pairs (x, 2o) E Rd x Rd satisfying
|| Varsal (60 3co)d(1 or) log T R²
(A.1)
12

---

Published as a conference paper at ICLR 2025
where co is defined in (2.3), we have
(xo x)) = Pxo (No)
PX,|X, (ro
(x) (x,1xo(20120 (2)
px. (oo) (2 (1+)) -d/2 exp
|| - Vauoll
log (x.
2(1 at))
|| VT
px. (zo) exp
(A.2)
3(1 oi.)
Here step (i) uses the fact that X, Xo = 30 ~ Nivav (1 T.) while step (ii) holds since
d
| - v Va log (2)
(ii) -
Co
| - va.vl 0d Adlog" T
log 2x(1 -aa)
dlog T
2
2(1 (i))
2
2(1 ae)
| - varzol
3(1 at))
where step (iii) follows from the fact that 1 - 02 2 1 01 = B1 for any 1 1 t S T, and
-logxx, (x) S Od log T' step (iv) follows from (A.1). Recall that
(x) - Ce /
Px,lx. (2ol xx)(xx Varzo)dro
(A.3)
20
and
Tr - J+())) 1 Ta (1/ Pxolx, [x)o=))|===-->0 (oloooolllll--oo Xxolx. (xl (o-voorao) dwo dwal
(A.4)
Then we have
(x)]l2
/
(xolx Varzo)
I-
I
1
Pxolx (00 -
0t
120
Pxo1x,
22)l2 Vauzo 1a {||0 Vausolla S R) dx
at
/
+
vārzol12 2 || varzol12 > R} dæo
1 - Ce
R
| -
(xo
|| Va,uolla {1|x varaoll2 > R} dr
04
Ci
3(1 xx,)
R
x - v /
+
(20) exp
1 {||> Voxoll2 > R} dro
1
6(1 Ti))
R
3
exp (-
R²
(d)
2R
(A.5)
0t
1 a4
6(1 -4)
at
Here step (a) utilizes Jensen's incquality; step (b) follows from (A.2); step (c) follows from the fact that 2 cxx(-2²²) I cxp (-22/2) holds for any 2 2 O; whereas step (d) holds provided that co is sufficiently large. In addition, have
/
Tr(T (x))) < E [[]x - Va.Xoll X, = x] 1 Cc
Pxolx, (oo |+)] - Varzoll3dro.
Then use the analysis similar to (A.5) to show that
@ <
1
/ pxolx, (2o xz) Vardollddro
Tr(I (xx))
Ti
<
1
/
"xxlxx
(20 [2)][= va {||z varsolla I R3 dro
1 C2.
13

---

Published as a conference paper at ICLR 2025
1
/ (201x(220|2)1122 varoo {1|x va.2ol12 > R3 dæo
04
(1)
R²
at / Pxo (xo exp
1| 110- VT
| va 13 1 {|| varxll2 > R} dzo
1 1-
3(1 - -ar)
(61)
R²
+3 / pxo(x1)) exp
| v /
1 {|| vārzolla > R} dro
Et
6(1 - a+))
R²
3 exp (- (-6(1
R2
(iv)
2R²
(A.6)
1 ai
6(1 a1
1-ax az
Here step (i) follows from ((A.4)); step (ii) follows from (A.2); step (iii) follows from the fact that
& exp(-xi S exp (-x/2) holds for any 2 2 0; while step (iv) holds provided that co is sufficiently
large.
Finally, we invoke Lemma 10 to achieve
T at Σ
< CydlogT,
(A.7)
di
where the matrix function Ea. (-) is defined in Lemma 10
Ex, (z) Cov(Z vā, 0,6 = vi-
for an independent (0, Ia). It is straightforward check that Jı(x) -I Ta-Ea, (x), therefore
we have
T
T
(Zx)
Tr (T(((T Jz(()))3)
1
T
T
at E
(A.8)
0x
Here the last relation holds since Tr(A?) || for any symmetric matrix A. We conclude that
T
at
/ 1
[[,((=))1Bxxx,(x (x))
at
E
(X))
Cte.
C2
T
E La -4 (X.) - 2|11
1
1=
2 2C log d T S Cog
Here step (a) utilizes the triangle inequality and the AM-GM incquality; step (b) follows from (A.7), (A.8) and Lemma 7; while step (c) holds provided that Co >> C. c1.
A.2 PROOF OF LEMMA 2
We first observe that
Py: Py-1(It-1) (Iz-1) 2 [
Bdd Pr...Y (fe-1 |x)py (xt)de; 2 [
py, 187 18 (Ft-1
(xx) (2x) )dr
(
JziEEen PYL, M2 ( dx, (
(A.9)
where we define
A- = f J.nEe.. PY: (me-1 xz A 2 0.
Here step (i) follows from (4.4c), while step (ii) makes use of the definition (4.10). It is straightforward to check that
f A 1 11-1 JZ
PY, (#+-1 w. A (sx))dd / A((s))d. (A.10)
14

---

Published as a conference paper at ICLR 2025
For any 21-1 such that 1 O, we have
Px, 1 (ax-1) - ( + A (
(
Pyz, (Ix-1) + Dese-
(b)
2
/
Pry ( xx)px (2x)d2z
21EE
(
S (x,(
0t
d/2
exp (
[|laa - (x+ (1 - a()a)s dx
2T((1 - ax))
2(1 - ar)
(4)
det
1 - det(1-1-00 J((2z)
Px px, (2o)
d/2 exp
|| - uel duy.
27 ax))
2(1 ar)
(A.11)
Here step (a) utilizes the definition (4.10) and Py (o1-1) = py (I(-1), which is consequence
of (4.6) and 1 > step (b) follows from (A.9); step (c) follows from the definition (4.3);
whereas step (d) applies the change of variable un = 2z - (1 - 0x2 ) (Ix). Moving forward, we need the following lemma.
Lemma 3. For any 2a Et,,, we have
1 0tt (1(z)) (x)
det 1
1- at
px, (20) exp (- ||ue - varwoll dro
= (2m - 1 -(+)) 4/2
2(2a; - - ai))
300
exp (Ee(ze)+
+ +0(()=
2
(d [J.(z))
(A.12)
where (x,)) < satisfies
J for some universal constant C3 > 0.
(2x))d < 03 1 an SIEEI
(d 1[+((11)))))) (xx + T
(A.13)
Proof. See Appendix A.3.
Taking the decomposition (2) and (A.11) collectively, we have
Px. (x A + A1-t-1(00--1 + 6 1
(A.14)
+ ((a(at)) (( ((((x))
2 ((-((1)))))) (dlog + (((((z))))) foe EnD ) pxx (zo)
2
exp
no m
an (4r2(11 ai) - -T,)
d/2
exp (-
2(20) - -Tr) exp (
1lue - var+ol 1|2
IIVanst-1 - -wel " Jduydro
2(1 - 2(1-a))
where we define
8 6x-1(x(1)== =
(ro) 4T72 - ar) (2ar - 1TT,)
04
d/2
10 1214E111
exp (-
|lu vārzol1
|| Uz 2
exp
Jdudoo (A.15)
2(20; - ar)
2(1 a1)
Moreover, it is straightforward to check that
/ pxo (20)
C2
||/ / V 122
exp
4T -az)(aa T.,)
2(2cx Tc,)
|| |/ u/ll
exp
dupdoo px._)(xt-1)
(A.16)
2(1 -aa)
15

---

Published as a conference paper at ICLR 2025
Then continue the derivation in (A.14):
px, (ct- 1) - A 11 + 1 1 + 6 (x4-1
1 + (x+))) + + (10((1=00) (1--0)) (dlog I' [1 1 faz E Ee.1t px. (TO)
50
| ue 1
d/2
|1ue V VOrTOl?
ddydo
1 (2xx - 1 [ip)
exp
2((2x
[Tp)
exp
2(1 ((1-
pxi
(II-1)
(((x)) + +0
(0((10110)) 01)
(1--)) (dlog (
[[(((z)) [1log++|J(((aa11888
p.o (20)
4T (2) -
04
d/2
exp
2(2cx: - varsoll?
a.))
exp
llvatte-1 - ue l|
2(1 a1s)
duydo-
Here step (i) follows from the fact that et 1+ for all R. while step (ii) follows from (A.16). By rearranging terms and integrate over the variable 1 we arrive at
J D ( S L/_ (A+(0r-1)+ + +
x212
/ J (15((22)) (0(()))++((110 0z
1-)
(d T (1)((2)) pxx(20))
ZIEEE
(2T(204 - -1
d/2
exp
1lue Vo 2(2a; - 1 - a,)
du,dxo
(A.17)
where we used (A.10) and for any fixed ue, the function
ale 4/2
a| Ue uel
2TT
exp
at
2(1 - az)
is a density function of T2-1. To establish the desired result, need the following two lemmas. Lemma 4. For E E,,1, we have
[ px. (x. (222 v.))
4/2 exp
(- 1/4e Vau1ol1 dzo S det (1-1-a0) ar
- aq Je(ae))
px (x)).
2(20x; (T.)
Proof. See Appendix A.4.
L
Lemma S. For the function 1-) defined in (A.15), we have
[ 1--1
61-1 T--1
Proof. See Appendix A.5.
Equipped with these two lemmas, we can continue the derivation in (A.17) as follows:
f A1-1(T1-1)dR--
(a) / (x(xe)dxe++ +2
[ks(zz)) ( ar)
(dlog [J<(az))
ae
m/
1 [e((z))))
01 (1-1-00
det
Px, (x:)du + T-
1- at
+7-+ + A((x))a +
/ [Ee(xt)) + 0 (((1100)) -0t
(d T + || (axe ) (ddlog)))
px.(ar))de
A((x))d,++7 + C.
(1-a))
1 at 2
(dlog T (1 (2,)d
1 - T.
JZEEE.
16

---

Published as a conference paper at ICLR 2025
which establishes the desired recursive relation. Here step (a) follows from Lemmas 4 and 5; step (b) follows from 24 = 2t + (1 ),)s s (2t), hence
1 - dit (z()))) dr: )dm)
du = det
Tte
whereas step (c) uses (A.13) in Lemma 3, and holds provided that C4 > C3 sufficiently large. Finally, we control the error / (r(x)dr in the initial step of the reverse process. Notice that
/ Ar(e))rr
S( JoTtO
(oxx(eT) py (ST)) @ TV(xX)
$ TV(px:: PYr) + (V
(A.18)
where step (i) follows from (4.6) and step (ii) utilizes the triangle inequality. The first term can be bounded by Lemma so it boils down bounding the second. By definition of YT in (4.4a), have
TV( = yEEE,
(vr(yddy
py, pr, 1 lor (V) > Cidlog 1lull2 S Vor + C2. - ar) log dd
+ f pr, (V) fllvl12 Var C2v d(1 ar log dy
(x) - log (v) Cid T. lllll S T + C2 d(1 σr) log T Jy
+V P([ (|| 2ar -cr) log
22 2C d( ar) log T exp(-Crdlog
PP((V/|12> > C2
2 dlogT) + TV(px,
(8) 2 exp ( ( - + P(IYT
C1
C2 dologT + TV(px,,Py.).
(A.19)
2
Here step (a) follows from the definition of Στ₁ in (4.2): step (b) follows from the definition of total variation distance, i.e., TV(p, supg p ( a( (B).. where the supremum is taken over all Borel set B in Rd; step (c) holds since गे < T 7 (see Lemma 7), provided that C2 is sufficiently large; whereas step (d) holds provided that C₁ > eR and >> d doogg T. By putting (A.18) and 19)1 together, we have
f Ar(z)dr 2TV exp ( G
C1
C2 dlogT) P(I Vdlog ST
2
where the last relation follows from Lemmas 9 and 8, provided that C₁ C2 > 0 are both sufficiently large.
A.3 PROOF OF LEMMA 3
Consider any I, Egp, Recall the definition u, x, + (1 (x)) and we decompose
||u - / Votrol
2(2a; 1 T+e)
| - v 0oroo1l
(1- (1-aa)le [24 v 00.0ol
+ (1 - [xxxxx vanzo)
(1 aaaa) 21)3 (2x)
+
2(1 at.)
(2ar 1 [1 at)
20h ar.
2(2a: -a,
|11 / Varrll
1- 0i
/ Pxolxx (20g [x))1105, Vor
2(1 aie)
(2a, [1, (1- [4,)
$1())) / Pxo x (20 (e)((ee - vārio) + (1 21s) + (0)),
+ 20, 1 ae
2(2a; a.)
17

---

Published as a conference paper at ICLR 2025
where we let
(e(21,z0) = (1-ai) (1) vā Szp Pxolxx (20 x1)]| varto (ddoo
(2ar -1 1 --) -at)
(1 - (z=)))0( ( (Io vā, - -Izo PNolx, (20 (x)) va dzo)
20x, - -1 T2
(A.20)
In view of (A.3) and (A.4), we can further derive
1|u, - v 0
1:08 - v va
1 - 02
Tr([ - Je(xx)) + (1 (1-a,) (1-0))(1b((1>))) + Ce (o))
2(20: 1 - σ4))
2(1 (1))))
204 - 1 - ae
2(2a; - 1 σx)
@ | - Vorsoll 11
1 1 ( 1- 0t )) (1-
100011110
1
at
(I - -(((z)) + (1 - a1))
+
1- -
+ Ce(xe. 20)
2(1 at)
1
(i)) || v V
1-
1 1- 02 Tr (1 Je(xe)) + +0
1 a4 (11 ax
d dogg (!--a1)
+ xo)
2(1 σ+)
1- 02
(in)
1x2 v varzo12
llo det
(I- 1 1- at Je(oe)
d
log
20, 1 - -
2(1 aa,)
I at
a4
(00) + o (1-a))
at) a2 (d [Je(m)))l()
(A.21)
Here, step (i) utilizes an immediate consequence of Lemma 7
1 1-
- a1)(a1)/(1-) - a,) =1==1:100 1+0 1- ar 1 + 2- oz)/
log T
= =1+0
(A.22)
20,-
Cr.
at
T
which holds provided that T > a1 log T; step (ii) follows from It E Er,1 and Lemma 1; whereas step (iii) follows from the following two facts:
1 - Ot Je(wt)
1
1 Tr(((((z))) +0 1 1 at ar
1| ) (x)s))) (xs))1)
log det
[
1 0t
and
d
20, 1 - C2
d(1 - at)
+0 ( d(1 -a,)2
= =0 ( dllog T
log
(A.23)
2
1 - Tه
1 1 T
(1 )2
T
Then we can use (A.21) to achieve
/ px. (20) exp (- | vārzoll 113
[| v a - (r((z, xo0 )do dro
dro
pxo (IO) exp
2(2cx; - 1 Ttp)
2(1 (.))
20
(- - -log del (1-
11-1--0 1-
1- as Je( (e
d
20, 1 01
+ +0 (1-ai) ai
(dlogg (1 ))
exp
+
log
2
1 - 0t
1 as
Define a function {r()) as follows
Seo pxo (x) exp (- 1101- Se (2,, mo))
(x)))) == - log
2(1-ā()
(A.24)
Jop Pxo (1t) (x)
exp exp (
||-r--v varzoll dur
2(1-a.)
Then we can write
[ Pxo (2o) (20) exp ( (- ||ue Vo 11
dro
(A.25)
2(2a; - 1 Ttp)
20
= =exp ( (E((at)) +0 (0((()))) (011- ar
(1-at)
(dlog T + || []
1 a,
/ pxo (x) exp
| VoLi 2
log det (1 (1-
1- at
Jı(wz))
+ d
log
20: 1 - T
d
2(1 1 o.
1 04.
2
1 0t
18

---

Published as a conference paper at ICLR 2025
and (((x)) I for any 24 Et since
[ Pxolxx (20 xx) exp ( Ce (2ie, moo
exp(-Cr(zz))
2 / ( 12 (2 To drop = 1,
x0
where we have used the fact that e² 2 + x. for any x R. Notice that
px. (2)) (2 (1 a.)) d/2
Px. (20) exp (- |1 v va 21
dro.
(A.26)
2(1 - (t,)
20
can rearrange terms in (A.25) to achieve
det (T 1- 04 J[
-1
Px. (244)
1 - Th,
(27(20-1-- 1- d/2 / Pxo (20) exp (- | va.0ol?
2(2cr, (x,) dro
exp ((((x)) + 0 (1-a1) T
((1-ar)) (dlog T + (1ee(s)))l))
(A.27)
which gives the desired decomposition (A. To establish (A.13), we need the following result. Lemma 6. We have
J (2m(2ar - - (x))) pxo (20) exp (- 2.f8..1
| vorsoll drndu ST (A.28a) 2(2ay axp)
and
Jnces Px.(2x)des ST
(A.28b)
Proof. See Appendix A.6.
Then we have
1 i
/ (2T(2at (1-1))-11aa))) -1-))) Pxo (20) exp
|/u V Vazzol dzodur
d/2
2(2ax Txp)
EE,,
60
det
-((()))) (1-1
Px, (2xe) exp
- (x(e) + +0 (0tt1a)) 2 (d T + (1((()))1)) )atue
Tr,
(i8)
Px. (xx) (E((xx))
- 10111000 -+
Id [|((((=))1))) dr.
T-
JZNEELL
(1-
+0
(10000+1111101 (1--0)²
)² (d T' + || l12) )) px. (2))dz..
2.E6111
ae
Here step (i) follows from (A.28a); step (ii) utilizes (A.27); step (iii) holds since + ar)s] (zt). namely
1 - at Jz ((x) dr:
du, det [
while step (iv) follows from the fact that ex > 1 + for any E IR. Recall that (xx)) for any Ie E E.1. By rearranging terms, we have
f JZ
[G((x)))xx
px, (xe)ddr + C3 Coa (a((1-1))
) J.EEE,, //- (d [1xx(()))))) (xe)dar
<
C3
1-0%
(dlog [1 +
Je,EE.
for some universal constant C3 0, where the last step follows from (A.28h).
19

---

Published as a conference paper at ICLR 2025
A.4
PROOF OF LEMMA 4
Recall the definition of Ge (T vo) from (A.20) in Appendix A.3. For any It E Er,1, we have
@
S
2
1 -
0t
1 - at (20 2x) va,.o Edo +
(1ze) T (ax voruo)
(1
ax,)
(
4
1
0t (6C1 + 30d 3o T + (1 -+))1+} (x:))2 +
1 -
1 Varzoll
1 04t
(1 xz)
(
1
co)d T + 1 0t
50
(C1
(1- ) || vazzol2
āt
1
||
20.
(A.29)
T.) j2
Here step (i) utilizes (A.3), (A.20) and (A.22); step (ii) follows from the AM-GM inequality and an intermediate step (A.6):
1 De / Pxolx, (20 -o S 2(6C; + 3co) (ol)d T,
where we also use the fact that - loggx, (2+) S Cjdlog T for 2, E E step (iii) follows from Lemma 1; while step (iv) follows from Lemma 7 and holds provided that T > c1(Ci + co). In addition, we also have
[++())) S 2 (x.))) + Tall (
- (x:)))) -
2d
(b) S 288(C1 + ) dL log T 2d,
(A.30)
for It E E1.1. where step (a) holds since Ia J1 (z+) = 0 and step (b) follows from Lemma 1. Substituting the bounds (A.29), (A.30) and (A.23) into (A.21) gives
1 /a
|| v ay 1
log det (I
1 1 at J{(2z)
2(2a; [e)
2(1 at)
1 T2e
ait
1 - Varvoll + 2,
(A.31)
(1 aq
provided that 7 >> CC( co)d T. Taking (A.31) and (A.23) collectively yields
det
1 - Ti
1 1
Pxo (20 - -1,)))
4/2 exp
|/ue - v vāgzoll dro
2(2ns - 1 - (5,))
< 10 / (xo) (20( aa))
d/2 exp (- ||00 Vorcoll
1 0t
+
1:00 - v Vazzol dro.
2(1 ai))
(1 )2
(A.32)
provided that T >> dlog T. To achieve the desired result, it suffices to connect the above expression
with
Px. (2x) = / (20) - -),,)) 4/2 exp (-
| - v varzol dxn-
2(1 (3,)
mo
For any E E,,,, define a set
1- aq A(x,)= = ro: (1 0.)2
1 ar
||= - vo > (6C) + +c0)
d log T
1 ai
We have
JncAter Pxo (xo oa))
4/2 cxp (- || Vorrall
1 - Il= v varzal33 dza
+
2(1 a.)
(1 - و,
=px,(e)) /
1 aq
Pxolx. (20 mx) exp
Ir
v vā,uol13 11 doo
(1 - oe )2
20

---

Published as a conference paper at ICLR 2025
| v va
1 1 at
$ px, (xr)
Pxo (20) exp
+
12 - va.vol dao
3(1 - a4)
(1 - a,)
IOEA((E))
@
|| varzoll
(xt)
pxo (xo exp
dz0
(ODEA(T.))
4(1 - (+t)
( I Px, (2:) exp ( (6C1 + 30o (oo)dlogg T>
1 S pPx, (2x))
px, Px, (xo)ddo
(A.33)
4
Here step (i) follows from (A.2); step (ii) utilizes Lemma and holds provided that > 01 log T, step (iii) follows from the definition of A(z;); while step (iv) holds provided that C₁ is sufficiently large. On the other hand, we have
Pxo (xo - a.))
d/2
exp
|| varzol
1- at
1 - vārvoll doo
2(1 - ((0))
(1-
())
I exp (BC) + 3co)
1- ar
1
/
Pxo (xo (2x (2
-d/2 exp (-
1 v vatcoll
dæo
2(1 (ia)
(b) exp (00) + 300)
8oi
10)
Px, (IT) S 2 px (3+)-
(A.34)
T
Here step (a) follows from the definition of A(xx): ); step (b) utilizes Lemma 7; whereas step (c) holds provided that T > (C co)dlog² T. Taking (A.32), (A.33) and (A.34) collectively gives
1 1 11110((((0))) J(((z)) det (T-1 1
/
pxo (2 (2at oz)) 4/2 exp
| var.oll3
dro S 20px, (3%))-
aie
2(20z 1 a3.)
Rearrange terms to achieve the desired result.
A.5 PROOF OF LEMMA 5
By definition of 6 (x in (A.15), have
f 1--1
81-1(x 61-1(31-1)d---1
(xo 17 oz)
a2
d/2
(T.)
eepp ( (- 1114--10 Va,20
exp (- 11var00011 iduddoo
2(20r TF,))
2(1 -a,)
||ue V 12 drodu
(2m - o.)) d/2 p/2 (xo) exp
dd1))
2(2cx 1 a,)
( T-1
(A.35)
Here step (i) holds since for fixed uy the following function
(ssp--Ilvve [Ivanee- 1el1
as
d/2
2m
exp
2(1 - -Oss)
is a density function w.r.t. 14-1, while step (ii) was established in (A.28a).
A.6 PROOF OF LEMMA 6
Proof of (A.28). We first prove (A.28b). Recall that
ε, {2+ : - (x. s dld T, ||æl|22 S Va,12en + C2vd(1 az) logT]
Then we can decompose
/ 2,,
Px, (x:))x
21

---

Published as a conference paper at ICLR 2025
- px. 1 log (xx) > C, Va,1 + +C2v ā.) log I dx.
/ px, (211 (1101112 > T2RT + C2v - a4) log T) dx
+
@
C1 dlog T)
$ exp
+ VāTTZeR + C2v d(1 - at)
2
C1 dllog T)
+ +P(IXoll2 > T2Ex) + P(II .112 > C2 C2V T)
exp
2
(
C1 dlogT) T)
+ Xoolal + P( Well > C2 CovdlogT) T)
(iv)
I exp
I T
2
T2cR
Here step (i) follows from simple volume argument
( logpx, I (x, 1
> Cid T, |12al22 S v Va, + C2 C2v d(1
a+) log 7 dax
S exp (- - logg dlggT) T):
C1
S (2VortTenn + 2C202 v d(1 - a4) a exp (-Cid (0gg))
2
provided that C1 > CR and T > dllo T; step (ii) follows from X; = VagXu + vI ai W15 step (iii) utilizes Markov's 's inequality; while step (iv) holds provided that Ci C2 eR > are large enough. This establishes (A.28b).
Then we prove (A.28a). Define
B, = {2: : || S vā,12eR + C2 C2Vd(2a; - 1 - ã.) log }
and for each & 124
Lr, = f Cidlg? < - loggx (2x) S 2 T}.
We first decompose
|/ee v /
(2m(2a) - 1 1--1-0 -x.))
-d/2 -i:)))///ppp px. (2o) exp
dzodu
2(2cx= - - aip)
|lue Va, 2 du,doo
pxo (xo (27 - 1- aa)) d/2 exp
2(20% - (x4)
50 N.
to
Σ
JSEELLLMEB Pxo (2 - 11 - k=1
exp
1|u - v
dzodu..
2(2a; - az))
=14
where step (a) holds since E = Up fi.k- The first term To can be upper bounded as follows:
To - 1 (Trallizza [., +
()xo(x))
vooi-vaaool22000 d(2a; 1- log
1
4/2
|//
v T
2n 12T(22 -1 1- -a.)
exp
2(2cu 1 a,
du,
(
< P (ol > T2()) + (1Z112 2 C2 C2 dlogT
( E Xol12] +P (12112 2 C2vdlogT)
< T
< 5-
(A.36)
Here step (i) holds since
- 1- (2 a.)) d/2 -d/2 pxo (20) exp (- |1ue - v vo.o
2(2a; - 1 - a.))
is the joint density of (Xo, Va,Xo Xo + /2ar - - 2) where Z ~ NO Ix) is independent of
Xo; step (ii) follows from Markov's 's inequality; whereas step (iii) holds provided that er and C2 are sufficiently large. Regarding 4s. we first show that
luu
(112, vāzzolle - (1 - an) |/ (x)))122
2(2or 1-11--00 a.)
2(2a:
(i.))
22

---

Published as a conference paper at ICLR 2025
01 V 0, 2
1 - at
20, - 1 -a; ||,
2(20, 1 a1
Ixe Va,to
1 1 1-a1 aq
| V 2012
2(1 [i)
(1 ū. (2ar 1 [.)
1 - a2s
+ (1- a: a1) [T))
+ (1 - Ci (2a - [+))
| vor
(1;))l
4(2a; - 1 (k,)
+ (1 - as) ||s;
(A.37)
2(1 ).)
Here step (a) utilizes the triangle inequality and '4 = It + (1 os)s; (are): step (b) invokes the AM-GM inequality; whereas step (c) follows from (A.22). Therefore we have
[ JNSEAA VIIELIN,
I px. (xo) n0
1
d/2
|| va 12
(1-))
2T(1 a+))
exp
2(1 σ+)
+ (1 ar) [1s (xx)) drydu
f InTECIAMEES P tee ar) ||o (1x) durachu
dlog2 I IEEEMMEE
( exp (200cl (2 + co)
px. (x=)du
(m)
I cxp (200c,(2* cu) dlog"1 T /
Cidlog T) du
u.EB
(1)
dlog" T
S exp (200c1 (2*C) co)
Cidlog 4deR log T + 4d log log(C2d)
T
(
exp (-
C1 21 2l (C1//44222d
(A.38)
Here step (i) follows from (A.37); step (ii) uses consequence of Lemma and Lemma 7: for It E Lekk.
(1 - ar)) ||
(2x) 11
1 1- - < 25
log s 200c, (2 200c, (2 (2
+ co) dlog² T
T step (iii) follows from the definition of Li,k, which ensures tht px, (1x) exp(-2 Cid T) for any x, Li step (iv) follows from
log vol (B.) S dloo (2Va, + 2C2 d(2a; - 1 (0,) log T)
< 4cgd log T + 4d log(C2d)
and finally, step (v) holds provided that C1 > CR + Co and T d dlog T. Taking (A.37) and (A.38) collectively yields
I To
T x 7
T- (C/ ST
R=1
k=1
provided that Cj is sufficiently large.
B TECHNICAL LEMMAS
In this section, we gather couple of useful technical lemmas. Lemma 7, When T is sufficiently large, for SLS T, we have
6 log T
04 2
T
2
For 2 < t < T, we have
1 -
1 a0
8c₁ log T
1 a
01 12
In addition, we have
5 12
or
23

---

Published as a conference paper at ICLR 2025
Proof. See Li et al. (2023, Appendix A.2).
Lemma 8. For 7 ~ N(0, 1) and any 2 1, we know that
P (12) 2 t) e--
44 2t2 1.
In addition, for a chi-square random variable Y (d), we huve
P(VY 2 vd+t) S 9/2
Vt 1.
Proof. See Vershynin (2018, Proposition 2.1.2) and Laurent & Massart (2000, Section 4.1).
Lemma 9. Suppose that Assumption holds, and that T and c2 are sufficiently large. Then we have (Vxxx, (lpr,) S TT99.
Proof. Define a random variable X5- == Xo Xo S 1 by truncating Xo. Let
X7 = Vār + vI- 2.
where ~ (0, La) is independent of XT Notice that X, has bounded support, which allows us to invoke (Li et al., 2023, Lemma 3) to achieve
PV,, = O
(B.1)
provided that 02 and T are sufficiently large. In addition, we have
TV(px, px,)
|xx, (x) (x)
2 /1I/ (Px, (mo) pxo (2o) (2x(1 (2x(1 - σT)) 4/2 exp (- || -|-=~v v xul1
1
duro dx
2(1 (T
|| v
$ 3z 300
|xx, (2o) - px. (xo) (2T(1 (2 ār.))
4/2 exp
)
2(1 στ))
1
Px, (FO) - Pxx = TV(PX, P(llXolla T
(
B [X| T
(B.2)
TeM- 100
Here step (i) invokes Tonelli's theorem, while step (ii) follows from Markov's inequality. Taking (B.1) and (B.2) collectively yields the desired result, provided that is sufficiently large.
Lemma 10. Suppose that Assumption holds, and that T is sufficiently large. Then we have
Σ 1
1 0z (E
S CodlogT
(B.3)
t=2
for some universal constant C > O. Here the matrix function Ex,, (.) is defined as
Ea, (x) = Cov(2 va.Xo + VI 0.6 =x), x),
where Z NO, La) is independent of Xo.
Proof. that
This result (B.3) was established in Li et al. (2024b, Lemma 2) under the stronger assumption
XollX < ) =
(B.4)
for some universal constant > 0. The assumption (B.4) is used to prove part (a) of their Lemma 2, which states that for any a', a E [Te, [. with < t < T. one has
E [[(>>< (v@xo+ + v1 2 σ2)
= & (2x + vi- + + exp(-c log Ta
for some universal constants eg cg > Through a similar truncation argument as the proof of Lemma 9, we can show that
E ( (
(ー && + vI vI-=z)) + 1-0 la-
GE
Armed with this result, we can use the same analysis for proving part (b) of Li et al. (2024b, Lemma 2) to establish (B.3) under our Assumption 1. The details omitted here for simplicity.
24


Reviews:
[{"review_id": "HC0CQ9BylE", "reviewer": "Reviewer_2VNk", "summary": "This work studies the sample complexity of diffusion models with a stochastic sampling process. By introducing two auxiliary sequences, they divide the discretization complexity and the approximated score error and achieve $O(d/T)$ convergence guarantee under a mild assumption on the score function.", "questions": "1.\tI wonder if the noise of the sampling process is pre-defined. More specifically, whether $Z_1,…, Z_T$ in (2.4) and (4.2) is exactly the same. In my opinion, if the noise of the ground-truth process and the approximated process are the same, the problem would become easier. Could the author discuss it in detail?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1.\tThe result is really interesting since this work achieves better results without the assumption on the Jacobian of $s_t$.", "weaknesses": "1.\tIt would be better to highlight the technical novelty compared to [1]. In fact, I think there are many points that are worth mentioning. More specifically, controlling the Jacobian of the ground-truth score and introducing two auxiliary sequences is the source to remove the Jacobian assumption (If I have any misunderstanding, please correct me.). The author can discuss them in detail to help readers to understand these papers.\n2.\tThe noise schedule is highly specific. Though this schedule has been used in some theoretical works [1][2][3], it would be better to discuss this schedule in a real-world setting.\n\n[1] Li, G., Wei, Y., Chi, Y., & Chen, Y. (2024). A sharp convergence theory for the probability flow odes of diffusion models. arXiv preprint arXiv:2408.02320.\n\n[2] Li, G., Wei, Y., Chen, Y., & Chi, Y. (2024, May). Towards non-asymptotic convergence for diffusion-based generative models. In The Twelfth International Conference on Learning Representations.\n\n[3] Li, G., & Yan, Y. (2024). Adapting to Unknown Low-Dimensional Structures in Score-Based Diffusion Models. arXiv preprint arXiv:2405.14861.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "MTPAVnhwCy", "reviewer": "Reviewer_34Q6", "summary": "This paper establishes a state-of-the-art convergence theory for diffusion probabilistic models, particularly focusing on SDE-based samplers. Under minimal assumptions, the authors derive an $O(d/T)$ convergence rate in total variation distance, substantially improving on prior results that typically require stronger assumptions or yield slower rates. The authors achieve this by introducing novel analytical tools that track error propagation across each step in the reverse process, leading to finer control over both discretization and score estimation errors.", "questions": "1. In line 269, it claims that the error bound holds provided that $T>> d \\log^2 T$ (here I simply choose $\\delta=1$). However, in line 247, it is stated that the result holds even when $T\\asymp d$. This appears to be a contradiction. Could you clarify the correct relationship between $T$ and $d$?\n\n2. What is the intuition behind introducing the generalized density in Section 4? It seems essential for the proof, but only the density of the auxiliary processes could become $\\infty$ at certain points? I am just curious about the reason of introducing those auxillary processes.\n\n3. What is the order of the constants in the error bound? e.g. in line 1127, it mentions that these constants need to be large enough. Could they be on the order of $O(d)$ or $O(T)$? If so, would this affect the order of the result of Theorem 1?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "The $O(d/T)$ convergence rate achieved for SDE-based samplers matches the rate of ODE-based models, bridging the gap between these methods under relaxed assumptions. This is a notable advancement for SDE-based models, particularly in high-dimensional settings. The analysis requires only finite first-order moments of the target distribution, which is a much weaker condition than those in previous studies. By developing new tools to capture the propagation of errors through the reverse process, the authors provide an elegant framework that simplifies the analysis without resorting to intermediate metrics like KL divergence. This results in more direct and interpretable bounds.", "weaknesses": "I did not identify any major weaknesses in this paper. However, I have a few minor questions. See the next section.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "xHvahxm3po", "reviewer": "Reviewer_i8U4", "summary": "This paper studies the (sampling) convergence of the DDPM, providing the status of the arts rate $O(d/T)$ under minimal condition.", "questions": "See weakness.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper is generally well-written, and I mostly enjoyed reading it. The results are insightful, and the \"minimal\" conditions enhances our understanding of what is really the key to the success of score-based diffusion models.", "weaknesses": "There are several weaknesses and questions/comments.\n\n(1) The authors consider $TV(p_{X_1}, p_{Y_1})$ instead of the target distribution $p_{X_0}$ -- this is because score matching is often very bad close to time $0$ (in the continuous limit). People usually do \"early stopping\" to avoid this bad region (as did in Chen et al.) The authors may make explanation to better guide the readers.\n\n(2) In Assumption 1, the authors assume that $E|X_0|^2 \\le T^{c_M}$, meaning that the data size (depending on $d$) is bounded by polynomial in $T$. Is this a valid assumption? In the original paper of Song et al., $T$ is not set to be too large. In some sense, this condition already assumes a tradeoff between $T$ and $d$ implicitly. \n\n(3) Theorem 1: often for the convergence analysis there are three pieces: (1) initialization error, (2) score matching error and (3) discretization error. (2) and (3) are there, where is (1)? Probably it is absorbed in one of the terms and the authors should explain.\n\n(4) Theorem 1: the score matching contribution $\\varepsilon_{\\tiny \\mbox{score}} \\sqrt{log T}$ is impressive, which is dimension free. I would point out another work https://arxiv.org/abs/2401.13115, which proposed a \"contractive\" version, which also make the score matching contribution to be dimension free. However, it is at the cost of possibly larger initialization error, which requires to choose the hyperparameters carefully to balance the two. This brings back my question (3) on the contribution from initialization error in this paper's setting.\n\n(5) The authors proved the results for DDPM (or VP in the continuous time). I wonder if the arguments/results are specific to DDPM/VP. It is known that e.g., other popular models as VE can be obtained by a reparametrization trick (https://arxiv.org/abs/2206.00364). I think it may be possible to get the results for general class of models, which may be even more significant.\n\n(6) The authors only stated the convergence results for SDE sampler. What about the corresponding ODE sampler? Is there any expectation on even improving the rate using deterministic sampler?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)