Unified Convergence Analysis for Score-Based Diffusion Models with Deterministic Samplers
Authors: Runjia Li
Qiwei Di
Quanquan Gu
Venue: ICLR 2025 Poster (2025)
Keywords: Diffusion Models
Probability Flow ODEs
Unified Framework
Deterministic Samplers
Primary area: generative models
TL;DR: A unified framework for convergence analysis of diffusion models with deterministic samplers.

Abstract: Score-based diffusion models have emerged as powerful techniques for generating samples from high-dimensional data distributions. These models involve a two-phase process: first, injecting noise to transform the data distribution into a known prior distribution, and second, sampling to recover the original data distribution from noise. Among the various sampling methods, deterministic samplers stand out for their enhanced efficiency. However, analyzing these deterministic samplers presents unique challenges, as they preclude the use of established techniques such as Girsanov's theorem, which are only applicable to stochastic samplers. Furthermore, existing analysis for deterministic samplers usually focuses on specific examples, lacking a generalized approach for general forward processes and various deterministic samplers. Our paper addresses these limitations by introducing a unified convergence analysis framework. To demonstrate the power of our framework, we analyze the variance-preserving (VP) forward process with the exponential integrator (EI) scheme, achieving iteration complexity of $\tilde{O}(d^2/\epsilon)$.
Additionally, we provide a detailed analysis of Denoising Diffusion Implicit Models (DDIM)-type samplers, which have been underexplored in previous research, achieving polynomial iteration complexity.

Published as a conference paper at ICLR 2025
UNIFIED CONVERGENCE ANALYSIS FOR SCORE- BASED DIFFUSION MODELS WITH DETERMINISTIC SAMPLERS
Runjia Li', Qiwei Di2, Quanquan Gu²
Schooo of Mathematical Sciences, Peking University, Beijing, China DDeparmment of Computer Science, University of California, Los Angeles, CA 90095, USA 1 lirunj jirunjiaestu pku. edu. en, {qiwei2000, @gu) agu)ecs.ucla.edu udla.edu
ABSTRACT
Score-based diffusion models have emerged as powerful techniques for generat- ing samples from high-dimensional data distributions. These models involve a two-phase process: first, injecting noise transform the data distribution into known prior distribution, and second, sampling to recover the original data distri- bution from noise. Among the various sampling methods, deterministic samplers stand out for their enhanced efficiency. However, analyzing these deterministic samplers presents unique challenges. as they preclude the use of established tech- niques such as Girsanov's 's theorem, which are only applicable to stochastic sam- plers. Furthermore, existing analysis for deterministic samplers usually focuses on specific examples, lacking a generalized approach for general forward processes and various deterministic samplers. Our paper addresses these limitations by in- troducing a unified convergence analysis framework. To demonstrate the power of our framework, we analyze the prrearce-reseeviing (VP) forward process with ((d2/o /c). Additionally, provide detailed analysis of Denoising Diffusion Implicit Mod- els (DDIM)-type samplers, which have been underexplored in previous research, achieving polynomial iteration complexity.
the exponential integrator (EI) scheme, achieving iteration complexity of (d)
INTRODUCTION
Score-based diffusion models (Sohl-Dickstein et al. 2015 Ho et al. 2020 Song et al. 2020a c have emerged as a powerful class of generative models, achieving significant success in image genera- tion tasks, such as DALL-E (Ramesh et al. 2022)), Stable Diffusion (Rombach et al. 2022), Imagen (Saharia et al. 2022). and SDXL (Podell et al. (023). Beyond that, these models have also demon- strated effectiveness in diverse applications, including structure-based drug design (Corso et al.] 2022 Guan al., 2024), text generation (Austin et al. 2021| Zheng et al. 2023 Chen et al., (023c), and reinforcement learning (Wang et al.) 2022] Lu et al. 2023). At the core of the score-based diffu- sion models is forward Stochastic Differential Equation (SDE) to diffuse the data distribution into a known prior distribution, and a neural network is trained to approximate the score function. Scal- able score- matching techniques, such as denoising score matching (Vincent 2011) and sliced score matching (Song et al., (2020b), enable efficient learning of the score function. Once learned, we can use numerical samplers to simulate the backward process and recover the original data distribution from noise.
To improve the sampling quality and efficiency of diffusion models, it is erucial to efficient samplers in addition to an accurate score estimator. Early developments of diffusion models, such as Denoising Diffusion Probabilistic Models (DDPM) (Ho al., (020) applied stochastic samplers. Later, empirical studies showed that diffusion models with deterministic samplers, such as Denois- ing Diffusion Implicit Models (DDIM) (Song al., (220a)) can still generate high-quality samples while achieving better efficiency. For instance, DDIM is more than 10 times faster than DDPM. Beyond DDIM, many novel fast ODE solvers have been developed for diffusion models, further improving the efficiency of the sampling processes (Lu et al.) 2022 Zhoou et al.) (2024).
The remarkable success of diffusion models has inspired extensive research interest in the math- ematical analysis of these powerful generative models (Block et al. 2020 De Bortoli et al., 202T
De Bortoli 2022
[Le et al. 2023 Pidstrigach] 2022
Chen et al. 2022 2023a b. Li al. 2024a

---

Published as a conference paper at ICLR 2025
Benton et al., 2024 Huang al., 2024a). Notably, many prior works studied the convergence of diffusion models with stochastic samplers (Chen et al. 2022 2023a Benton et al. 2024; Li et al.) 2024b). A key tool in their analysis is Girsanov's s theorem, which helps control the distance be- Tween the distributions of two stochastic processes. However, it relies on the smoothing effect pro- vided by stochasticity and therefore do not apply to deterministic samplers. To address this, (Chen iet al.] (024) introduced an additional corrector into the Langevin dynamics, which incorporates randomness to help smooth the distribution. Meanwhile, [Cenn al. (2023b) assumed the access to the exact score function and provided a discretization analysis for the probability flow ODE in Kullback- (KL) divergence. Moreover, under extra conditions, [Li et al. (2024b) proved a non-asymptotic convergence rate for specific deterministic sampler based on the probability flow ODE with elementary analysis. Huang et al. (2024a) considered the ward process with the Lunge-Kutta integrator and provided error bounds for both continuous- and discrete-time settings. However, all of these works focus on specific forward processes and sam- plers. Therefore, natural question arises:
(OU) for-
Can we develop a unified framework for the convergence analysis of diffisson models with deter- ministic samplers that accommodates those common forward processes and sampling algorithms? In this paper, we provide an affirmative answer to this question. We summarize our contributions as follows:
We develop a key technical tool (Lemma 3.2). which enables us to bound the time derivative of the total variation (TV) distance between the final states of two ODE processes, through the difference in their drift terms and the divergence. As a direct application, we establish convergence guarantees for the continuous-timg time reverse ODE in the case of the OU forward process.
We provide unified convergence analysis framework for diffusion models with deterministic samplers. For general forward processes and sampling algorithms, our framework decomposes the error of diffusion models into five distinct terms: two arising from score estimation and three from time discretization. This decomposition enables divide-and-conquer approach, allowing for improved analysis of each error component while maintaining the consistency of the unified framework.
To demonstrate the generality and effectivvness of our framework, we apply it to two typical diffusion model settings: we achieve Ö(d²/e) iteration complexity for the Variance Preserving (VP) forward process with exponential integrator (EI) numerical scheme. This theoretical guar- antee matches Li al. (2024b), where similar but different sampling algorithm is considered. Moreover, we establish polynomial iteration complexity for the Variance Exploding (VE) forward process with the DDIM numerical scheme. To the best of our knowledge, this the first con- vergence result for diffusion models employing DDIM-type samplers that can handle estimated scores, whereas prior works, such as Chen al. (2023b). only considered cases with access to accurate score function.
Notation: In this work, we use lowercase letters a, b to represent scalars, lowercase bold letters x, y to represent vectors, uppercase italic bold letters X, Y to represent random variables, and uppercase bold letters A, B to represent matrices. For a vector x E Rd and matrix E Rdxd, we denote by ||x|| the Euclidean norm of x and || the operator norm of A. We use fi S. fz to denote that there is a universal constant C' such that fi < C2.2 For two sequences {an} and {b, we write an = (b,,,) if there exists an absolute constant C such that S Cbb,, and we use O(.) to further hide the logarithmic factors. For vector operations, we use to denote the gradient, V. to denote divergence, and 2 to denote the Jacobian matrix.
PROBLEM BACKGROUND
The primary objective of diffusion models is to generate new samples given a set of examples drawn from the data distribution. In this section, we introduce the fundamentals of ODE-based diffusion models with deterministic samplers.
2.1 ODE-BASED DIFFUSION MODELS
A diffusion model typically consists of a forward process that perturbs the data distribution into noise, followed by a denoising backward process. In general, the forward process can be modeled Itō SDE:
dx, = fft, X,). (t) X go.
(2.1)
2

---

schedules have also been utilized in Published as a conference paper at ICLR 2025
where W, is Brownian motion in R f(t,-) -) is called the drift coefficient, g(t) is called the dif- fusion coefficient (Song et al., 2020c). It begins by sampling Xo from the data distribution qu. and evolves according to the forward process 2.I The law of X, is denoted by q (x)... Under mild reg- ularity conditions go. according to the following SDE:
construct a family of reverse processes (Y,²) rejo.ry which evolve
dx,' = - (E(T- - x²,, - ++ X2
g(T - -1) (o + >g(T - t)dW,, Yo ~ qr.
2
(2.2)
These processes hold the same marginal distribution as (x) at time t (Chen et al. 2023b). special case when 0, the backward process is deterministic, i.e.,
As
dY [f(T - t, Y,) 2
70( - 2))² log )tt, dt, Yo 4r,
(2.3)
which is called the probability flow ODE (Song et al. 2020c).
The goal of ODE-based diffusion model is simulate (2.3) However, we must make several ap-
proximations. First, as we do not have access to the true score V log g(l, x),, we learn an approxi- mation So (4, x) by minimizing
T ((so) = f/" Egg [llso(t, X,) V Voo qe (Xx))l dt.
(2.4)
When -().-) is affine, the transition is Gaussian, allowing for the closed-form solution of V log x). By applying integration by parts, (2.4) can be reformulated into tractable objec- tives, specifically through the methods of denoising score matching and implicit score matching (Hyvärinen & Dayan 2005; Vincent 2011]. For more general forward processes, these objectives can be estimated using samples drawn from the forward process. Then, using the estimated score function Se, we have the simulate reverse process:
dÿ. - -0 (,,)) Pa~ Gr-
(2.5)
Second, since the initial distribution qr of the reverse process is not directly accessible, instead initialize the reverse process with a given Gaussian distribution na. which is assumed to be a good approximation of qr.
Third, in practical implementations, the continuous time process is typically simulated using time discretization. Specifically, we choose time steps 0 = to < 4 <... < IN S T, sample Y, ~ aa and iteratively calculate Yi.,, from Vt. using a particular numerical scheme. In this way, we only need have access the value of the estimated score function st(,,, ), k 0, N, which can reduce the computational complexity of the sampling process. Common numerical schemes in- clude the Exponential Integrator scheme, the Euler-Maruyama scheme, and the DDIM-type -type sampler scheme, as we will discuss in Section 4.1
For non-smooth data distributions, the score function V log di can be unbounded as t + O, which will happen especially when the data distribution is supported on a lower-dimensional submanifold of Rdd. This will lead to difficulty when considering the distance reason, we consider an early stopping scheme, selecting i = T - for some small 8. Our analysis focuses on the distance between the final distribution of the sampling process and qs- When d is sufficiently small, the Wasserstein-p metric between and go is small. This shows that it is reasonable approximate q8 instead of go-
the data distribution. For this
In this paper, we propose unified framework that can handle different time schedules. The choice of the time schedule is crucial for obtaining good results when analyzing the convergence of specific samplers. In this paper, we follow the time schedule used in Benton et al. (2024). This allows us to provide more refined control over the discretization error. We will always assume T > 1 and 8 < 1. In the first stage when t E [O,T 1], we use uniform time steps, with step size m) = tx+1 - t2 S n- In the second stage for E [T - 1, T 8), we assume time steps satisfying 7 S (T (7(7-4+1), which results in an exponential decay at a rate of (1 + -) Using this schedule, we have
T/K S 1) min (1, T - 1++1)
(2.6)
It's worth noting that similar tat--unffoom-thee-exxxpeeee Chen et al. (2023a): Li et al. (2024b c)
3

---

Published as a conference paper at ICLR 2025
2.2 DIFFICULTIES IN CONVERGENCE ANALYSIS OF ODE FLOW
Compared with the convergence analysis of SDEs, the convergence analysis of ODEs more chal- lenging. For SDE, the prior results either apply Girsanov's Theorem (Chen al. 2022 or rely on a chain-rule-based argument (Chen et al. 2023a). Using these methods, we can see that for SDEs, when the drift term slightly perturbed, it is possible bound the distance between the final iter- ates or, more strongly, the entire distributions over the trajectories. Therefore, the assumption of the score estimation sufficient. However, as the following theorem illustrates, this fails the ODE case.
Theorem 2.1. Consider the 1-dimensional OU process (11))E(002] which starts at N(0, 1). satis- fies the following SDE
dx; = -x,dt + 2dd,,, 3o ~ N(.
Let the law of xe be denoted by dr. Its reverse process (3t))e( (see (2.3) can be represented with the following ODE:
dy: = [++ Vlog ( - t, ve)) ve)
Yo 9T-
For arbitrarily small O, there exists so(t, x)) which is smooth, and anywhere e-close the true score function, i.e,
1,x) v log (T' t, Se Vt, Va,
such that the corresponding simulated reverse process (Dr)1Ejo.2) (see 2.5 satisfies
TV VT)
4m
which indicates that the TV-distance of the final states is larger than constant matter how small the score estimation error is.
Remark 2.2. For the construction of the counter- counter-example, example, we consider a specific scenario where the data distribution is standard Gaussian, and the OU process maintains the distribution. In this case, adding an arbitrarily small perturbation to the drift term of the reverse ODE can result in 27 sin(2n)z)) with severe oscillations (n can be arbitrarily large), instead of a Gaussian distribution q x)) = / - " For details, please refer to Section B This demonstrates that the score estimation
sampling probability density function qT t, z) 1 Le
/2x
assumption is insufficient, even when ignoring the discretization error. For a similar purpose, an example has been given Li al.] (2024b) for the discrete- time sampling algorithms. Notably, our constructed Sy is smooth while theirs is not. This indicates that adding the score function's 's smoothness condition alone is insufficient guarantee the desired properties.
CONVERGENCE OF CONTINUOUS TIME REVERSE PROCESS
In this section, focus on the OU forward process, where ftt x) = - and g(l 22. Temporar- ily, we ignore the discretization error and focus on the true reverse process (2.3 and the simulated reverse process (2.5) starting at wd. Specifically, we want to study the TV-distance between the following ODEs.
dY: Y, + (ogg)r-((Y))) Yo GT, )dt, di,
(3.1)
dP.
9 So (T 1, V1)
Po Te-
(3.2)
We make the following standard assumption regarding the score estimation error. Assumption 3.1 (Score estimation error). The estimated score function so(:.)) satisfies:
E E,y, ||so (T 1, Y.) V log ( 1, Y-Il 2
This assumption guarantees that the drift terms in 3.1 and (3.2) will be close. However, as dis- cussed in Remark 2.2 this condition alone is insufficient for ODE flows. To address this problem, introduce the following lemma. It states that the time derivative of the TV distance between two ODE flows is determined by the distance between their drift terms and the divergence of these drift terms. Its proof is left Appendix C
4

---

Published as a conference paper at ICLR 2025
Lemma 3.2. Suppose X, and Y stochastic processes in R driven by ODEs:
dX;; = b(t, X2) dt dY; = (t, Yi dt, Yo 9o-
X Po:
Let p(t, x) be the law of X, and q(t, x) be the law of Y, the drift terms b. b* (0,xc) xR= R continuously differentiable with respect to E Rd then the time-derivative of the total variation distance between X, and Y, satisfies the following equation:
OTV(X,, Y.) 0t
[ (V.b(t,x) - 1-bb bbb'(t, (t,x))
-1 nz
(b(1,x) -b(,x))b((,,x)) V log x)g(1, x)dx,
where 22 :- (x [Rd [((())) x(.,)))
In the application of this lemma diffusion models, we can choose the true reverse ODE and sam- pling process ODE. Let Ps be the distribution of the sampling process, and Ør be the distribution of the true reverse process. Several works (Chen al. 2023b Albergo et al.) 2023), considered the time derivative of KI. divergence, deriving expressions that involve scores of the sampling pro- cesses V Vlgg x). However, as shown in Theorem B.1 the score of the sampling process can still explode given sufficiently small score estimation error (see Assumption 3.1) and divergence estimation error (see Assumption (.33). The crucial aspect of this lemma is that enables us to avoid the occurrence of the sampling process score V log xt), We apply Gauss's 's theorem to transfer the integral to aΩg. where we can replace Pt with qt because Pt = qi on òQ, Then, we apply Gauss's i theorem again convert back to volume integral. This allows us eliminate the depen- dence on V log p(t, x). Motivated by this lemma, we make the following assumption on divergence estimation error.
Assumption 3.3 (Divergence estimation error). For any t [0. - the estimated score function so (t,.) :) is sedond-order continuously differentiable. Moreover, it satisfies:
E
So
TT - Y,) V V ((T Y.) < Ediv-
Similar assumptions regarding the difference between the derivatives of true and estimated scores have been made Li et al. (2024b/alc). In comparison, those studies assumed the closeness of the entire Jacobian matrix, while we only require assumptions on the divergence. Under Assumptions
3.1 and 3.3 we have the following theorem.
Theorem 3.4. Let the true and simulate reverse process be defined as 3.11 and 3.2 Under As-
sumptions 3.1 and 3.3
we further assume that Xo has finite second-orde momentum, then we
have:
TV ex X&) TV Xo) + dT + dlog J Excore + Ediv
See Appendix D for the proof of this theorem.
Remark 3.5. Huang et al. (2024a) proves a similar bound on the TV-distance for continuous-time processes. Their result differs from ours in that, unlike Assumption 3.3 they do not assume the divergence estimation error to be small. Instead, they require the first two derivatives of the estimated score function to be bounded. Their estimation error term scales 0 strictly worse than our / + log (8- (( Moreover, their results rely on extra assumptions regarding the compact support of the data distribution. For detailed comparison with Huang et al. (2024a), regarding the settings and results, please refer to Appendix A.
which
UNIFIED ANALYSIS FOR DISCRETE-TIME REVERSE PROCESS
In this section, we conduct convergence analysis for the discrete-tine time reverse process. Specifically, we consider a sequence of time steps 0 = to < t N N
< ty < T. Starting from Yo ~ we to generate subsequent iterations. For any k, the
iteratively apply a deterministic sampler {Ti) Jk k sampling process can be expressed as:
Ts (Y.))
5

---

Published as a conference paper at ICLR 2025
where T3 acts as discrete-time simulation of the transition in the reverse ODE process (4.1) We
will present our results in three steps. In Section 4.1 we introduce some commonly used numerical schemes for diffusion models. In Section 4.2 provide unified framework encompassing these numerical schemes. We then employ an interpolation method to transform the discrete-tine sam- ODE, enabling us leverage Lemma from the previous section. In Section 4.3 we present the convergence analysis for this general framework.
pling into an equivalent continuous
4.1 NUMERICAL. SCHEMES
Recall that for general forward process 2.1 the continuous-time reverse process with the estimated score function Sy can be written as
- (T dı = - F(T - 1, -
P.) dt,
Yo ~ qr-
(4.1)
Forward Euler Scheme. The simplest method the forward Euler scheme, which directly replaces t E [kk [x+1] with the start point & in the equation above, i.e.,
aÝ -F((( .
g tx)² tg. ) for ty < tx+1.
Or equivalently, we have the following fiscrete-timg sampling algorithm, with 7e tx+1 txj
Pewes Y... Th (((T Lk: Vr.) 20(T 1xx) ) (T 1x Pr.))
(4.2)
Exponential Integrator (EI) Scheme. When f(t, Ly is linear function, we can apply the Exponential Integrator (EI) scheme by keeping Y; in the linear part, i.e., the ODE becomes
aÝ, - L.
(O - tx )r.))ot for la < 1 1 (T
2
In this way, we can integrate the linear part exactly. As a result, we have the following discrete-tima sampling algorithm, with 7)2- tx+1 tw:
LotH Y,,, e Lan -1
2L 9( 20 LR: Pt.).
(4.3)
DDIM-type Scheme, Song et al. (202aa) introduced a deterministic sampler for the probability flow ODE by considering non-Markon Markovann diffusion process. As interpreted by Chen al. (2023b), it can be viewed as a two-step process involving a restoration step that provides a rough estimate for past step and degradation step that simulates the forward process by progressively adding the estimated noise. Specifically, starting from Yi,- the restoration step provides an estimate of Y. for some 0, where 1R+1
1B 7, i.e.
Yuety Rs Y, |(T Iu. P,,,) g(T - So 1x, |r.)) z.
Next, the degradation step simulates the forward process during t E [T - tt - -T - twt1l. which can be expressed
YuE. (7
- 1x 7, g(T - 14 +7 1x116
where represents the noise estimated from Ý,. By substituting the form of and making some approximations, we can get the following sampling algorithm, with 7) t2+1 fg.
Pi0e -r, nef(T Lk Pr.) + Im v1 gT )s t
(4.4)
where / = 7//(tx+1 - tx). Please refer to Chen et al. (023b) for more details about the DDIM-type sampler. For linear diffusions, (.6). we have 1 2 (T 4++1)/ (t++1 4) 2 min / - (x) 2 1/1.
y = ty. Then, with selection of time schedule, by
4.2 INTERPOLATION METHODS
At the core of our analysis is to apply Lemma 3.2 which analyzes the divergence between two ODEs. However, two main challenges arise. First, the discrete-time nature of the sampling algorithm precludes direct application of the lemma. Second, the sampling process Y = Tx (Y,)) depends on the position Y, at time 4g while the proof of Lemma 3.2 utilizes the Fokker-Planck equation,

---

Published as a conference paper at ICLR 2025
which requires the drift term be function solely of time and the current position. To solve these problems, we first introduce unified framework encompassing all the numerical schemes in Section Next, present interpolation method to transform the sampling process into continuous-time ODE, enabling the application Lemma 3.2
For the numerical schemes defined in (4.2. (4.3) and (4.4), we naturally extend the definition to a continuous interval [tx, t&++1 by replacing t&+11 with t. More concrete examples of Fr can This yields continuous-time interpolation operator Fi.- Fi,-x(-), or simply Fr when no confusion arises. Moreover, let Y, (Y.( F(t, Y..). It is also equivalent to the following ODE:
be found in Section [
dY. OF Øt (1, Pr,,), )dl.
Moreover, we further assume F2 is invertible, which holds for many examples when sy Lipschitz and the time step 1/s 1R+1 la is small enough. Then we have the following ODE:
dy, = b(t, dt,
(4.5)
where b(t,x)
dE OF (1, (x))
4.3 MAIN RESULTS FOR GENERAL DIFFUSION PROCESSES
Using the interpolation method presented in the last section, we can now provide general conver- gence analysis for the discrete-time reverse process. Consider the time step [1x, (x+1]. Recall that for general forward process (2.1) the true reverse process is defined as
dY b(t, Yt,
2 2 log (x)). While the simulated reverse process where xt, (f(T t,x) [T -) given by:
dÝ. = E(t, f.)dt,
where b(t, x) 8F (t, F1 (x)). (x)) Sr
Definition 4.1. At each step [tx, 1x-1], let the interpolation operator of the sampling algorithm be Fi-st- The estimation-error operator defined by:
OF (t. x) = at (t, + f(T - ((x 2 g( - t)² log 91 -1 (x).
As we will show in the next section, it reflects the enror between true score V log 40997,, t, (x) and so (T 4g. x). Similarly, we can define the divergence-error operator, which reflects the error be- tween 2 log -r, (x) and (T' - x).
Definition 4.2. At each step [tx, 11, let the interpolation operator of the sampling algorithm be Fre-tt The divergence-error operator is defined by
L (t, v 0 F) (t.x)I+ [x[ -t, t, (x)) g =)" log -r (x).
Using these definitions, the next theorem shows the convergence for the general diffusion process with any numerical schemes. Theorem 4.3. Consider the true reverse process (Yi)seju,7 and reverse sampling process (%r, KEINNI Then,
TV(Yr-8, P(,) & (V(gr,Ta) N-1
da+1
log q(T x)xx))x, (x) 12 (PE)) (x)dx Pr, (x)
E [[ Yr. |[T]
k=0
Py, (x)
E tr V2 (1, 1n,))
(F,, (x))x,,)x))xxx (x)dx
PE. (x)
Divergence estimation

---

Published as a conference paper at ICLR 2025
+ 20(T t)² / (V Qo log (T-+(x))) V Vlog q(" xx)))(( x)dx
(III) discretization enror
+ 20(2 t)2
tr [r([2 log q(T tk. v² log q(T' x)) a(T x)dx
(IV) Divergence discretizaaion error
max
V
virr Q 0t
F
(t, F. (x)) + [x( (xx) I) dt,
x
(V) Bius error
where F. (xx.
Other than the distance between the initial distribution of the reverse process and Gaussian noise, our upper bound of TV distance is divided into five terms.
For (I) and (II), these terms rely the expectation of the defined estimation-eme operator and divergence-error operator over the true reverse process. Moreover, they depend on the density ratios between PY, (x), representing the distribution for the true reverse process at time t, and PF,(Y,,) (x). As the interpolation operator FÜ acts simulation of the true reverse process, ratio will be close to I, thus bounded. Consequently, these terms are expected to scale with the score he distance between
expect that the
and divergence estimation errors. For (III) and (IV), these terms depend lime-dicrettization algo- score functions and divergence evaluated at t and ty. originating from the time-
rithm. They will decrease as the time step gets smaller. For (V), we observe that F; close the identity when + tg. This results in VF, (x) - converging to the zero matrix when t + tg.
APPLICATION TO SPECIFIC DETERMINISTIC SAMPLERS
In this section, we apply Theorem 4.3 to analyze specific diffusion processes. We focus on the Vari- ance Preserving (VP) forward process with El schemes and the Variance Exploding (VE) forward process with the DDIM sampler. While prior work has shown that VP and VE can be connected through reparametrization (Karras al., 2022). such equivalence does not account for initialization and discretization errors. Therefore, we provide separate detailed analyses for both VP and VE processes. The analysis method can be easily extended other forward processes and numerical schemes.
We specifically focus on data distributions with compact support, as outlined in the following sumption.
Assumption 5.1 (Bounded Support of Data). For a constant R, the data distribution go satisfies:
go (x) O,
x||
R
equivalently, Xol| 0.
This assumption has also been made De Bortoli (2022); Chen al. (2022). In particular, we do not assume the smoothness of the data distribution. Therefore, it includes the setting where the data distribution is supported on a liver-dimensional submanifold of Rdd which, notably, does not possess smooth density.
Additionally, we make the following assumptions on the estimated score function.
Assumption 5.2 (Score Estimation Error).
Σ Th Bl 1,, Yu,) log - Yn) S Soore
Assumption 5.3 (Divergence Estimation Error).
Σ EusoEu(VVo(T - tt, Y.,) 2 log q tk, Yr,))² < Cdiv-
In addition, to deal with the discretization error for the ODE reverse process, we need the following regularity conditions for Sg.
Assumption 5.4 (sg is Lipschitz and bounded at 0). For all th, we have:
||50 Lg, X1) so le, x2) L ||x] x2|1
8

---

Published as a conference paper at ICLR 2025
|| - -t, Oll c.
for some constant L. and c. Without loss of generality, we assume 1, and c < I. to simplify later derivation. Similar assumptions on Sy have been made in Huang al. (2024a). Note that only need the Lipschitzness and boundness at O, while they require the boundness of high-order derivatives w.r. f and x.
5.1 VP+EI
In this section, we focus on the VP process where f(t, = -x and g(t) = V2. As is discussed in Section 4.1 the sampling algorithm is given by:
Pumm = c Vt, + + 1)soo(T tk P..)
x). Therefore, the estimation-error At each step [4x, 14+1 Fx(x) e + (e 1)80(( (T- -ks -R,xx).
operator (Definition 4.1 can be computed as
OF
1 50(T 1)2 log Qr (x)
Pu (1,
01 (1, x) + f(T (,
s6 (T' t x) V log -1
Similarly, the divergence-error operator (Definition 4.2) can be computed as:
V (1, x)
v
2 (1, (1.x) v. Pr (x)-)( 2 g(( - 1)2 or
ot r
Vs L2 x)
2 log QT (x).
Now, using Theorem 4 we have the following convergence analysis for VP + El. See Appendix for proofs of the following results.
Theorem 5.5. Consider a VP forward process with EI numerical scheme. Under Assump-
tions 5.1
5.2
5.3
and 5.4, suppose the time schedule satisfies (2.6). the step size nj satisfies
7) < min 1/ 27. d")
1/2A1 then the following bound holds:
R 8
TV( Xx) & TV πa) + cas Vdd/nNeecce + + LR (RR d
L'd
Lemma 5.6. Assumes the data distribution satisfies Cov(go) = La. Then for the VP process, we have TV(ær, TT) E Vde- At VP forward process case, we take va ~ Ia).Ia).
Corollary 5.7. For all T 2 1, < and N 2 log(1/5), there exists 7 = ( T + log N)) and time schedule satisfying (2.6). Under the same assumptions as Theorem 5.5 assume that the data distribution satisfies Cov(go) = Lg. When = 1/d and d 2 R/L L L/R LR (T T
additionally
for some e S T/L, we have TV(Ý IN X²)= S c, assuming Ö(E/ Vd) score estimation error and O(e) divergence error. Hence the diffusion model requires most Ö(LRAd d²/e steps to approximate 4 within c S 1/L in TV distance.
we take = log(d/<))/ and
Remark 5.8. We note that the e-dependence in iteration complexity of our deterministic sampler is cl. while stochastic ones usually exhibit slower iteration complexity proportional (Chen et al. 2023a] 2022 Li et al., (2224) [enton et al.) 2024). This aligns with general observation in dif- fusion model practice: deterministic samplers demonstrate higher efficiency compared stochastic samplers.
5.2 VE+DDIM
In this section, we focus on the process where f(1. x) = 0 and g(() = 1, which corresponds to the VE forward process with of = I. As discussed in Section 4.1 the sampling algorithm is given by:
T lx
P
Ye2 + (tx+1 - tt))(1 vI 1/ te. Yr.),
I
tx+1 - 4u
Let C: = 1(1 VI 1//1 < 1. At each step [tx 1.. Fx(x) = x + ci(t - to ss) J - tk x). Therefore, the estimation-error operator (Definition 4.1 can be computed as
0F
(t, Øt (t,x) + f(T - t, F(())) - 2 g0(T - 1) Mo 10
9

---

Published as a conference paper at ICLR 2025
1 V log qr-(
csso tg x)
Similarly, the divergence-error operator (Definition
can be computed as:
(t,,,)) = v[ F (t. x V - t, Fi, Fx,) 2 g( - -) log qr- (x)
= < (T tk. x)
2 log ar- (x).
Now, using Theorem 4.3 we have the following convergence analysis for VE + DDIM. See Ap- pendix for proofs of the following results.
Theorem 5.9. Consider VE forward process with DDIM numerical scheme. Under Assumptions suppose the time schedule satisfies (2.6). If the step size y satisfies S R2d), then the following bound holds:
5.1
5.2
5.3
and 5.4
min
LR* TV(Yr-8, Xr- & TV(Ta, (r) + ediv + v vd N/ Escore + N LR + L'd
52
Lemma 5.10. Assume the data distribution satisfies Cov(gu) Le- Then for the VE forward process, have (yT, Ta) X vd/vT. At VE forward process case, we take Td N TTa).
Corollary 5.11. For all T 1, < and N log(1/6), there exists n O log N)
and time schedule satisfying (266) Under the same assumptions as Theorem 3.3 we additionally assume that the data distribution satisfies Cov(gu) Ia. When
1/d and d 2 L/R if we take
T= d/e² and N = (d + log(1/8)) for some c S 1/ L., we have TV(Y,, IN Xx) S c, assuming sufficiently small score estimation error and divergence error. Hence the diffusion model requires at most Ö(LR*d /e²) steps to approximate q within e I 1/L in TV distance.
Remark 5.12. Compared to Corollary 5.7 VE SDE has larger iteration complexity, primarily due to the slow 1/VT decay in the distance between Qr and Td. compared with the exponential decay (Lemma 5.6) for VP SDE.
CONCLUSION
In this work, we introduce unified convergence analysis framework for deterministic samplers. We start by presenting counter-example to illustrate the main challenge in analyzing deterministic samplers compared to stochastic ones. Additionally, we provide a technical lemma that allows us bound the distance between distributions using score estimation error and divergence error. With this approach, we directly established convergence guarantees for the continuous-time reverse ODE. Moreover, we extend our analysis to the convergence of discrete-time deterministic samplers with unified framework. Finally, we demonstrate its effectiveness by applying it to two widely adopted sampling methods.
Limitation and Future Work. First, for the VP process with El schemes, our current results have quadratic dependence d, which leaves room for improvement compared to the d-linear state- of-the-art bounds ODE analysis (Li et al. 2024c). This discrepancy stems from two factors: different assumption that requires control of the divergence error, and our directly estimated Lipschitz. constants in the discretization error analysis. We believe that through more delicate methods for analyzing the discretization error and potentially stronger conditions, we could achieve linear dimension dependence within our unified framework. Second, since adding the divergence assumption can guarantee the convergence of the ODE sampler, an interesting future direction may be to design training methods that can obtain both small score error and small divergence error. One potential approach would be incorporate regularization terms corresponding to divergence error in the loss function, potentially leading to more effective diffusion model training algorithms. Third, for the VE process, we obtain only polynomial bounds due to the slow 1/VT decay in the distance between ęr and n.. It's important to note that this not a limitation of the DDIM sampler, applying DDIM to the VP processes yields results comparable to those obtained with the EI scheme. We leave an improved analysis for the VE forward process future work. Finally, the snscrete-time analysis currently relies bounded support assumption for the data distribution, which may be relaxed to less restrictive conditions, such as light-tailed distributions.
rather than the full Jacobian
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGMENTS
We thank the anonymous reviewers and area chair for their helpful comments. QD and QG are sup- ported in part by the NSF grants I1S-2008981, CPS-2312094 and Sloan Research Fellowship. The views and conclusions contained in this paper are those of the authors and should not be interpreted as representing any funding agencies.
REFERENCES
Michael S Albergo, Nicholas M Boffi, and Eric Vanden-Fijnden. Stochastic interpolants: A unifying framework for flows and diffusions. arXiv preprint arXiv:2303. 08797, 2023.
Jacob Austin, Daniel D Johnson, Jonathan Ho, Daniel Tarlow, and Rianne Van Den Berg. Structured denoising diffusion models in discrete state-spaces. Advances in Neural Information Processing Systems, 34:1 1981--1993, 2021.
Joe Benton, George Deligiannidis, and Arnaud Doucet. Error bounds for flow matching methods. arXiv preprini arXiv:2305./6866 2023.
Joe Benton, VD Bortoli, Arnaud Doucet, and George Deligiannidis. Nearly d-linear convergence bounds for diffusion models via stochastic localization. 2024.
Adam Block, Youssef Mroueh, and Alexander Rakhlin. Generative modeling with denoising auto- encoders and langevin sampling. arXiv preprint 20Xiv:2002.00107, 2020.
Hongrui Chen. Holden Lee, and Jianfeng Lu. Improved analysis of score-based generative modeling: User-friendly bounds under minimal smoothness assumptions. In International Conference on Muchine Learning, pp. 7735-4765. PMLR, 2023a.
Sitan Chen, Sinho Chewi, Jerry Li, Yuanzhi Li, Adil Salim, and Anru R Zhang. Sampling is easy as learning the score: theory for diffusion models with minimal data assumptions. arXiv preprint 2222:2209.11215.. 2022.
Sitan Chen, Giannis Daras, and Alex Dimakis. Restoration- beyond linear diffusions: A non-asymptotic analysis for ddim-type samplers. In International Conference on Machine Learning, pp. 4462 4484. PMLR, 2023b.
Sitan Chen, Sinho Chewi, Holden Lee, Yuanzhi Li, Jianfeng Lu, and Adil Salim. The probability flow ode provably fast. Advances in Neural Information Processing Systems, 36, 2024.
Zixiang Chen, Huizhuo Yuan, Yongqian Li, Yiwen Kou, Junkai Zhang, and Quanquan Gu. Fast sampling via de-randomization for discrete diffusion models. arXiv preprint arXh::3312.09193, 2023c.
Gabriele Corso, Hannes Stärk, Bowen Jing, Regina Barzilay, and Tommi Jaakkola. Diffdock: Dif- fusion steps, twists, and turns for molecular docking, arXiv preprint arXlv:2210.01776, 2022.
Valentin De Bortoli. Convergence of denoising diffusion models under the manifold hypothesis. arXiv preprint 2xXXv:2208.05314 2022.
Valentin De Bortoli, James Thorton, Jeremy Heng, and Arnaud Doucet. Diffusion schrödinger bridge with applications to scsre-based generative modeling. Advances in Neural Information Processing Systems, 34:17695-17709, 2021.
Xuefeng Gao and Lingjiong Zhu. Convergence analysis for general probability flow odes of diffu- sion models in wasserstein distances. arXiv preprint arXhv:2401.17958. 2024.
Jiaqi Guan, Xiangxin Zhou, Yuwei Yang, Yu Bao, Jian Peng, Jianzhu Ma, Qiang Liu, Liang Wang, and Quanquan Gu. Decompdiff: diffusion models with decomposed priors for structure-bast drug design. arXiv preprint arXiv:2403.07900 2024.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840-6855 2020.
Daniel Zhengyu Huang, Jiaoyang Huang, and Zhengjiang Lin. Convergence analysis of probability flow ode for score-based generative models. arXiv preprint arXiv:2404.09730. 2024a.
11

---

Published as a conference paper at ICLR 2025
Xunpeng Huang. Difan Zou, Hanze Dong, Yi Zhang, Yi-An Ma, and Tong Zhang. Reverse transition kemell: A flexible framework to accelerate diffusion inference. arXiv preprint 11XX:::4005.6638,, 2024b.
Aapo Hyvärinen and Peter Dayan. Estimation of non-normalized statistical models by score match- ing. Journal of Machine Learning Research, 6(4), 2005.
Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion- based generative models. Advances in neural information processing systems, 35:26565-265777 2022.
Gary Leal. Advanced transport phenomena: fluid mechanics and convective transport processes, volume 7. Cambridge university press, 2007.
Holden Lee, Jianfeng Lu, and Yixin Tan. Convergence for score-based generative modeling with
polynomial complexity. Advances Neural Information Processing Systems, 35:22870-22882, 2022.
Holden Lee, Jianfeng Lu, and Yixin Tan. Convergence of score-based generative modeling for general data distributions. In International Conference on Algorithmic Learning Theory, pp. 946-985. PMLR, 2023.
Gen Li, Yu Huang, Timofey Efimov, Yuting Wei, Yuejie Chi, and Yuxin Chen, Accelerating conver- gence of score-based diffusion models, provably. arXiv preprint arXiv:2403.03852, 2024a.
Gen Li, Yuting Wei, Yuxin Chen, and Yuejie Chi. Towards non-asymptotic convergence for diffusion-based generative models. In The Twelfth International Conference on Learning Rep- resentations, 2024b.
Gen Li, Yuting Wei, Yuejie Chi, and Yuxin Chen. A sharp convergence theory for the probability flow odes of diffusion models. arXiv preprint arXiv:2408.02320. 2024c.
Xingchao Liu, Lemeng Wu, Mao Ye, and Qiang Liu. Let us build bridges: Understanding and extending diffusion generative models. arXiv preprint arXiv:2208. 14699, 2022.
Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu. Dpm-solver: A fast ode solver for diffusion probabilistic model sampling around steps. Advances in Neural Information Processing Systems, 35:5775-5787, 2022.
Cheng Lu, Huayu Chen, Jianfei Chen, Hang Su, Chongxuan Li, and Jun Zhu. Contrastive energy prediction for exact energy-guidee diffusion sampling in offline reinforcement learning. In Inter- national Conference on Machine Learning, pp.
PMLR, 2023.
Jakiw Pidstrigach. bosred generative models detect manifolds. Advances in Neural Information Processing Systems, 35:35852 -38655 2022.
Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhom, Jonas Müller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion models for high-resolution image synthesis. arXiv preprint arXiv:2307.01952. 2023.
Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text- conditional image generation with clip latents. arXiv preprint arXiv:2204.06125, 1(2):3, 2022.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF confer- ence on computer vision and pattern recognition, pp. 10684-10695, 2022.
Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Bureu Karagol Ayan, Tim Salimans, et al. Photorealistic text-to-image diffusion models with deep language understanding. Advances in neural informa- fion processing systems, 35:36479-36994 2022.
Jascha Sohl- Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In International conference on machine learn- ing, pp. 2256-2265. PMLR, 2015.
12

---

Published as a conference paper at ICLR 2025
Jiaming Song. Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. arXiv preprint UrXiv:2010.02502, 2020a.
Yang Song, Sahaj Garg, Jiaxin Shi, and Stefano Ermon. Sliced score matching: A scalable approach to density and score estimation. In Uncertainty in Artificial Intelligence, pp. 574-584. PMLR, 2020b.
Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. arXiv preprint arXiv:2011.13456, 2020c.
Pascal Vincent. connection between matching and denoising autoencoders. Neural compu- tation, 1677)11661-167,, 2011.
Zhendong Wang, Jonathan J Hunt, and Mingyuan Zhou. Diffusion policies as an expressive policy class for offline reinforcement learning. arXiv preprint arXiv:2208.06193, 2022.
Lin Zheng, Jianbo Yuan, Lei Yu, and Lingpeng Kong. A reparameterized discrete diffusion model for text generation. arXiv preprint 2rXiv:2302.05737, 2023.
Zhenyu Zhou, Defang Chen, Can Wang, and Chun Chen. Fast ode-based sampling for diffusion models in around steps. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7777-7786, 2024.
13

---

Published as a conference paper at ICLR 2025
A RELATED WORK
Convergence Analysis of Stochastic Samplers. Early theoretical studies for diffusion models were either non-quantitative (De Bortoli et al., 2021; Liu et al., 2022, Pidstrigach 2022), or exhibited ex- ponential dependence on the dimension or other problem parameters (Block al. 2020 De Bortolii 2022]. Later, [ee et al. (2022) proved the first result with polynomial complexity, with the assump- tions L2-score estimate and log-Sobolev inequality (LSII). However, the LSII condition the data distribution is restrictive, prompting further studies to relax this assumption. [Chen et al. (2022) utilized Girsanov's theorem, and proved polynomial convergence bounds, assuming either the Lips- chitzness property of the forward process score function or bounded support of the data distribution. Lee et al. (2023) relaxed the smoothness condition to apply only the data distribution rather than the whole trajectory, and replaced the bounded support assumption with sufficient tail decay. Chen et al. (2023a) proved a result with both the advantages of these two works, achieving better convergence rate while assuming only the smoothness of the data distribution. Furthermore, they provided results for the non non-smooth setting using appropriate early stopping and decreasing step size. Benton et al. (2024) improved the dependency of the dimension d to linear via the stochas- tie localization method. Compared with the analysis of reverse SDE, Li et al. (204b)) utilized an elementary approach to analyze DDPM-type stochastic sampler. It also proposed stochastic sampler with better iteration complexity. More recently. Huang et al. (022b)) decom- posed the entire sampling process into several reverse transition kernel subproblems and proposed novel fast sampling algorithms.
accelerated
Convergence Analysis of Deterministic Samplers. The first non-asymptotic bounds for determin- istic samplers were derived by Chen et al. (2023b). which assumes access to the ground truth score function. Subsequently, Chen et (2024) proved the first polynomial-time convergence guarantees of probability flow ODE with estimation error, by incorporating an additional corrector Langevin dynamics. While this improved upon prior results, it introduced randomness, making the sampling processes non-deterministic. Li et al. 2024a b) applied an elementary analysis framework to study the convergence of a specific deterministic sampler with a given learning schedule, requiring an ad- ditional assumption on the Jacobian estimation error. Later, Li et al. (2024c) applied the same frame- work and achieved improved iteration complexity, which is linear in the dimension. Recently, Huang (OU) forward process, removing the assumption of Jacobian estimation error. However, they introduced higher-order boundedness assumptions on the derivatives of the estimated score function concerning time and space. They proved an upper bound for the total variation between the target and generated data at the continuous-time level while also analyzing the convergence rate for the Runge-Kute Kutta integrator. Further expanding the scope, [ao & Zhu (2024) studied the Wasserstein distance using general forward SDE with cog-ccicyvity data assumptions. Additionally, research has started to explore other deterministic generation methods, such as flow matching (Benton et al., 2023).
al. 2024a) examined the
While we prepared our manuscript, we noticed the concurrent work in (Li et al. 2024c). They ana- lyzed a specific discrete-time sampling algorithm that can be viewed as a deterministic counterpart of DDPM (Ho et al. 2020), achieving better (ddee) iteration complexity. The improvement arises from their fine- fine-grained analysis of discretization error through an elementary approach. The focus of their paper is orthogonal to ours, as they provide an improved analysis of a specific algorithm, while our work aims to develop a unified analysis framework. We conjecture that their techniques can be adapted into our framework, potentially achieving improved results. Nevertheless, it is beyond the goal of our paper and we will leave for future work.
A.1 COMPARISON WITH HUANG ET AL. (2024A)
Huang et al. (2024a) is the most related work to ours. It studied the convergence properties of diffusion models with deterministic samplers based on probability flow ODEs.
Theorem A. in Huang et al. (2024a) is similar to our Lemma 3.2 where they applied the charac- teristic line method for ODEs, while used the Fokker-Plane Planck equation and the Gauss's 's theorem twice. However, we use different methods when dealing with the divergence term
V [((b(t,x) - b" -t,))
We decompose it into score estimation error and divergence estimation error (see Lemma 3.2g In contrast, they applied the Gagllardo-Niremberg inequality to bound the integral of first-order deriva- tives using integrals of second-order and zero-order derivatives of sø. This led to different assump- tions regarding the convergence analysis of the continuous time reverse process. In addition to the standard estimation error assumption, we require small divergence error, while Huang et al.
14

---

Published as a conference paper at ICLR 2025
2024a) assumed the boundedness of up to the second order derivatives of sp. Furthermore, they quired a bounded support assumption, which not necessary for our continuous cottnuuuuu--meeeeeee analysis. As result, Huang et al. (2024a) proved upper bound on the TV distance: In comparison, result is O( dT d (1/6) Escore), notably only oog-dependent 6. For the discrete-time analysis, Huang et al. (2024a) studied the p-th Runge-Kutta discretization method. When p 1, it becomes the forward Euler method. In comparison, we study the EI scheme. Our analysis can be easily applied to the forward Euler method, yielding similar results. In the discrete-time case, assumed bounded support of the data distribution and the Lipschitz condition of so, which was already required Huang et al. (2024a) in the continuous-tine analysis. Additionally, they assumed that the mixed second-order derivatives of Sa with respect to / and are bounded by a constant L. For the forward Euler method, Huang et al. (202aa) proved iteration complexity of OII2REd remark that the constant in their result may be very large. For example, the derivatives of the true score function can depend on higher-order terms of 1/8 (see Lemma H.2) Moreover, also consider the VE forward process with the DDIM numerical schemes.
e
while our result for El scheme is O(LR** O(LIIdd for = 1/d. We
A.2 COMPARISON WITH OTHER WORKS ON DETERMINISTIC SAMPLERS
In this section, we consider different works on convergence analysis of diffusion models with de- terministic samplers and provide detailed comparison in Table [ The comparison focuses on three key aspects: major assumptions required by these works, the metrics used, and the iteration complexity in terms of dimension and accuracy
Major Assumptions
Metric
Complexity
Reference
Score Estimation Error
Lict 2024b
Bounded Support
Theorem
TV(P1.41
d
Liet al.
2024c
Vsal(t, V log X) X-m
I Cauobb
N
Theorem
Access to Exact Score
log g and V logg Lipschitz
KL(po.
Polynomial
Chen et
2023b
V/ log " (x)ll s 0|t (1 + ||x|| [x)()]]
Theorem 4.1
Score Estimation Error
d(p+1//p o c/p
sup max XEX 1gk+ Sp1+ Issdd
( max 10) 883 (t,x)l L
TV 96)
Huang et al. 2024a
Bounded Support
Theorem 3.10
Bounded Support: Assumption
5.1
Score Estimation Error: Assumption 5.2
TV(ps. 96)
This Work:
Divergence Error:
Assumption 5.3
Corollary 5.7
Table 1: Comparison of convergence analysis for diffusion models with deterministic samplers
B PROOF OF THEOREM 2.1
N for all t. Recall that we use xf,,x) to denote the law of æ4, then we have q(t, <2= and
Proof of Theorem 2.7 Since x, starts at N (0, 1), can casily show that xs
V log x(t,z) = = Define q( - t,x) = 1 c =1 2T sin(2nnxx)) and construct our es- timated score as:
V
(in(y)yy)
FT dy
So (T - f,2) /2
q t, x)
'This assumption is slightly different from ours. While it assigns uniform weights to estimation errors across all time steps, our approach applies smaller weights to time steps with smaller step sizes.
15

---

Published as a conference paper at ICLR 2025
Without loss of generality, can assume 2 0. Then we have:
000
sin(2nny)
(अ) dy
150(T - 1, V Vlog Vlog((7 1,2)l
x
v2n
(1
T-t sin (2TTx)
27
1 sin (2nry)dy
2
2T
sup
V2x
1 1
T TT
Taking n 2 1/(TTE), we have (T - x, V log t,,) S e. Moreover, since we have
0q(T - L, Øt
② (
((50 (T - x)) + x) q( -(T--t,7) t2z)). q( - -0,x) NO,,
Ox
which satisfies the Tokker-Planck equation. Therefore, know that ((t, x)) is the law of ūr Hence the law of Ør is
1+ 2 sin sin(2nzz) )
(0(2) = 22T e
The TV distance between yr and yr is:
1
TV (T)
e
1+ sin (2nTTT)
dr
J
V
2t
sin (2ntz))de.
(B.1)
/
To calculate the last integral, we consider the Fourier expansion
4
cos 4knuT
sin((nnxx)
41:2
k>1
For any & 2 1, we have:
/
# cos(Aknzz)der
+
=
cos (os(Annzz)de
R
2 sup N20
cos(AknTz)dr
2kn
Therefore, we can plug these integrals into B. and obtain
TV Fr) >
2
1
c
& cos(4knTr)ddr
4
π
4k2
-
2mT
121
1
1
2
4
1
π
4k2
22T22nm
k>1
1
2
1
π
2
36-
R>1
By taking n large enough, can easily show that TV FT) 1. This completes the proof of Theorem 2
Theorem B.1. Consider the same 1-dimensional OU process (s*))eto,T) and its reverse process (yo)e defined Theorem 2.1 Recall that we denote the law of x, by q.- Then for arbitrarily small > 0 and arbitrarily big N 0, there exists st,,, which is smooth, anywhere e-close to
16

---

Published as a conference paper at ICLR 2025
the true score function, and its divergence is anywhere c-close the divergence of the true score function, i.e.,
5/ (T - x, v log (T' tt,x se Vt, Vx,
V So (T -, V log t, x)) S e, Vt, Vz,
such that the law pt of the corresponding simulated reverse process (0e)xe(0,7) (see 2.5 satisfies
sup V logpr(z)) 2 N,
which indicates that the score the sampling process can be unbounded no matter how small the score estimation error and divergence enror is.
Proof of Theorem B. Substitute the oscillate term sin(2n in the proof of Theorem 2.1 by +1a2+1) sin(2n R.). In this case, we still have q(t, v2n T and Voog q x) -r Now
q(T ((T-) 2) becomes
(1+ 2T +[z sin(2n² mx)) ) and now our estimated score becomes:
V2
f e
ml= sin m2 (2T) dy
So [T t,x)=
V2n
2,
q(T t, x)
Without loss of generality, we can assume x 2 0. Then we have:
J
nly sin 2in(222Ty) Try (2t)) dy
22n
(T - x) - v IT - f, t,,)l
(1
T-t
sin (2n2Ty))
v2k
mly
/2m e
27 (1++1)
sup
N22
1 sin
sin(2n"my)dh
22
1 T n
(B.2)
Further more, we consider the divergence estimation error. Since this example is one-dimensional, taking divergence is the same as taking a derivative with respect to æ. Hence we have:
0 J
c
22 T
+(y=+1) sin(2n? Ty) ( 27 dy
1
V So (T - xx))) - V log (T - t, t,x:)
2m
Ox
Q(T - t,
sin fin(2n22r2 .T)) (2T) T(T - 1, x.)
V2n
M(*+1)
(T - t, xz
100
mlg++1| sin kyn (+r) dy 2 (T t,
87
+
2T
(B.3)
qT x)
Firstly, we have
22 I (T
- t,x)) s
2tT -
2 2m
Moreover, know that
(T - 2,2)
e -- 12
(=x(1111
+
t
sin(2n 2 Tx))
Ox
2TT
2T n(xx2 + 1)
1 2n n cos coscoonn 2 )
(x² + 1) 2x sin (2 TT))
(B.4)
2T n
(x² + +1)
1
1
[|0]
+ no +
V2n
4In
2T'
2
<
e
[(1z|
+ 2nT).
17

---

Published as a conference paper at ICLR 2025
Since 1 e- '
is strictly monotonically decreasing when 2 x 2 0, we have
/2
sin sin(2n Ty)
2T dy
2T
nly²
+ 1)
sin 2 dy
e
sup
nxx2
+ 1) 27
N2#
1
2
1
1
2m
x²
+
2T
Substituting back to B.3
We have
1
V (T x.
logg [,,:]]
4
(10 + 2nT)
2Tn 1
x²2
n
27
4
(B.5)
2In
Tn²
Tn²
For any >> O, using B.2 and (B.5). for n large enough, the score estimation error and the diver- gence estimation error are less than for every t and æ. Same as in the proof of Theorem 2.1 using Tokker-Plandee equation, know that is the law of the sampling process. However, by (B.4). the score of the sampling process clearly unbounded (there a coefficient proportional to n, and our is arbitrary large).
C PROOF OF LEMMA 3.2
Proof of Lemma 3.2
We begin by computing total variance distance between X, and Y, for any
t te []:
TV(X,, Y,) / p(t,x) (t,, dx,
(C.1)
S2
where 24 fx Rd of (C.I). By Theorem K.I
x(, > xtt, Now proceed to compute the time-derivative
we have
OTV(X,,
θ (p(t,x) x) - (t, x)) dx+ / (p(t,x) x) q(t, x)) v(t, x) n(t, x) as
Ōt
D. Øt
12
(C.2)
For term J1, by Fokker-Plande equation, we have
i at (v(l, - x(1,x)) x)((,)) b" (1,x))((1x)))
Therefore, have
11 4= S2
-V (b(t,x)p(t, b* (t,xx))q(t, x))
For term T2, by the definition of Ω = {x E d p(t,x) > q(t, x)), we have p(t, q(t,x) on 0Z, Hence la = O. Therefore, by substituting 11 and l2 into C.2). obtain
OTV(X, Y.) Qt
-V (b(t,x)p(txx)) - b" bb,,tbbb"ttiattt x)g(t,) dx
-(b(t,x)p(t, x) ba (t. x)) n(t,x)dt
80,
-(b(t,x)q(txx x) (z(t,x) x)) ntt, x)SS
100,
/ -v (lb(t.x) - b²(ttx)) dx
D.
18

---

Published as a conference paper at ICLR 2025
/ (b(t,x) b((,,) V g(t, x) dx,
where in the second equation we use Gauss's theorem to compute the integration of divergence, the third equation uses the fact that p(t,x) g(t, x) on U2, the forth equation holds by Gauss's theorem again. This completes the proof.
D PROOF OF THEOREM 3.4
In this section, provide proof of Theorem 3.4 which allows us to represent the time-derivative of TV distribution between distribution by score estimation emrr and divergence estimation
The core of the theorem's proof is Lemma 3.2
Proof of Theorem 3.4
Recall Y, and Y, ODE flows determined by (3. and
Define
btt, x) = V log t, x) and (tt, x) = x + (T t,x), then we have:
dy, = b(t, Yi)dt,
Yo ~ gr.
dÝ, b(t, Dddd,
Yo nd.
By Lemma 3.2
recall that the law of Y; is denoted by q(T - t,x), we have:
OTV(Y, Y)
(v.6(1,x) - v- b(l, ((T - 1, 1, dx
(v [(1,))
0t
5nc
(((t,)) - -b(t,))) v log (T t, x)x))(TT t, x)dx
V
"o(T x,x)
v log - - x)) q(T - t, x)dx
S4
(So(T t,x) Vogg (T t,x)) V - t, x)dx.
(D.1)
For I,, we know that:
11 S n.
V so(T -t,x) Vlogg(T t,x) q(T t, x)xxx
So t, Y)
log (T t, Yz)
(D.2)
For 12, using the Cauchy-Schwarde inequaliity we know that:
L $ V
[
||so(T - t, v log ( x)) q(T -t, x)xx
V (T -1, 1, q(T - x)dxx
Sz
V E E| t, Yi)
log 12)
V t, x
(D.3)
Substituting
D.2 and (.3) back (D.I). and taking the integral with respect to (0, T 8). we
have:
/ T-6 (14, + dt
TV X1) I TV(I,, Yo)
T-8 E v So (T 1, Y.) V (og - Yz) dt
< TV((,, Yo)
T-8
El|soo(T Y.)
Vlogg(T - Y:)
E V (T - Y.)
19

---

Published as a conference paper at ICLR 2025
-T-6
< TV(Y.. Yo) +
B V
So (T - Y.) V V log (T - 1, Y:))
T-6
B|
(T' t, Y2 log - t, Xi)l| dt
V log g(T - t, xx)l xt)l
(D.4)
Since Xo has finite second-order momentum, using Lemma K.2 we have that:
Eq Voogg(T - t, xi)l 2 d(1 - -(T
Thus we know that:
T-6
d
E lo (.)) <
1 e 2(T-1) dt
0
8 1 - c-2r dt
d log (e2T -1)
log (626 (628-1) -1)
0 dT + dlog 1) })
(D.5)
Substituting D.5 into (D.4) and using Assumptions 3.1 and 3.3
we can conclude with
TV( X8) S TV(Y. Yo) + V dT + dloog 8 Eworee + Cdiv-
E PROOF OF THEOREM 4.3
Proof Theorem In the time step t E [tk: (k+1). compute the time derivative of the total variation distance between Y, and Yt using Lemma 3.2
(V, Yi = / (V x(t,x) - v x(),,)))((TT - x)
Ot
/ (((t.)) b(t.x)) Vq(T x,x)xx.
(E.1)
n.
Recall that b(4, x) = -(P(T 1, T7 1)2 log (x)), (()), OF (1, Fx)().)
For I, we decompose tr tr(Vb((,x) - Vb((, x)] as follows:
tr /b((,tx) - Vb(t,x)]
F (t, F. (x)) VF, (x) + -
Ot
5( +) log -1 (Fr (x))
1
20(T
1)² 2 log fr (x) - V log - (Fr (x))
(v1
2
(t,
F (t,z) + - (VF - 1)
1 2 log or (x) - V 1o (2))
2 00 1)²
where (x).
Therefore, have
41/ S / tr [V((tx)) (b(t,x)) q(T - t, x)dx
20

---

Published as a conference paper at ICLR 2025
$ f V - -t xxx
+ / ur V Lan F (1. +
(VF - -D/eCT -
- -,,x)xx
+ / 20( - 173111 ir [V log log lol QT (2)])((((T - - xxx)xx
(1 Ir [rrVa(( 2) 2 "F.(),, (x)dx) 1/ (1
pr((s) 2
(PE. (x)
PE (PFCC,,
② F (t, F. (x)) + [L[ F(()))]
+ max
tr
v
t,
(VF,
(x) (xx 1)
Ot
+ = 20( - t)} tr [V (oggr-((x) - V2 logoog 10 -tx, xxxx
7
( [E[(trWa( Wal,, x))))1) 4 pr((s) (x)
)² 2 (x)xx))
[ tr oa [ (t, (T) + Vale(T t, [ (V (x) -1)
max
/ 20(0- () tr V log or (x) v2 log [2) t, x)dx.
(E.2)
where (i) holds due to the (r, (x) pY,, (z)dz for z F; (x).
inequality. (ti) holds due to the Jacobian transformation
For I2, we decompose b(t,,) x) - b(4, x) as follows:
OF
1 20(T - - log - (Fr (xx)
b(t,x) - b(t,x) =
Ot tt FT (x)) +
- ff
20 - 1)2 V Vlog Nogor-((x)) - V log tr (x)))
O0
1 - - -t) log QT cx (2)
Öt (t, + f(T -t,F(()))
2 20( - 1)2 ( 1 (rg))-((x) - Vloo to (F. (xx)) ].
1 7 1)2 Vlogor-((x) - Vog (F. (x))]
1x 2)
where z= = (xx) Therefore, we have
[22 S / [t((t,2) 2). Vq(T xxx
2
+ - - t)3 / | log (x) - V (F (F, (x))] Vg(T t,x)) dx
( | (4, 2)1 (x)xx) 1
V log log x)) (x) 12
PF (x)cxx)
+ - - z)2 /11 |1 | log ((())- - V log to (PT (x]() Vg(T - -1,)) dx
(1)
1 [B|| (EIl (t. Y (.))1111 (111)) 3 (/1
V Vlog q(T - x,x)pr( (x) 12 (x)()
-T 1)2 / | log (() Vlog (F. Vg(T t, dx. (E.3)
n.
where (i) holds due to the Cauchy-Schwarre innuii-a inequality. (ii) holds due to the Jacobian transformation ( (x)dx= pY,, (z)dz for F, (x). Substituting B.2) and [E.) into (E.1 and taking the
integral over we can complete the proof of Theorem
21

---

Published as a conference paper at ICLR 2025
F ANALYSIS OF VP+EI
In this section, we focus on the case where f(x) = - and g(t) = v2, i.. the VP-SDE. Specifi-
cally, the forward process X, satisfies:
dX, = -x, + v2dW, Xo ~ qo.
(F.1)
and the true reverse process Y satisfies:
dY; Y v log log Yo gr.
Then, the forward process has the following closed-form expression:
Y, = - (T--) Xo + v V1-
=
~ La)
(F22
Recall the exponential integrator provides discretized version of the reverse process:
P = th Yi, + 7 1) tsp Pea).
Therefore, we can define the following interpolation operator:
Fz(z) c' (c' 1) tk 2),
(F.3)
satisfying Frcms (fi))= Y 1 Under Assumption 5.4 we know so(t, -) is Lipschitz. Therefore, suppose 17 1/ then we have 1/2 S |VF S 3/2. In particular, F2 is invertible.
Denote e by Since - 4a - - 1, we have s 1 + 2(t - 4x). Then we know that:
| 2 all (x)] -a
( (0-
+ +c)
2 [x)]
2(t - tw.) (1lFFF [z)]] (c) +c)
(z)]l
2 1 (z)]l -
where the last inequality holds when we assume that 4 S S min(1/4L,1/44 Thus we
know that:
|F (z)]l 2| +
(F:4)
Before we start the proof, we introduce several important lemmas used in our proof. Let ej = (1 + d/2)4L² and C = (1 (L + c)². Using our assumption 17, know that
1
n S min
(F.5)
16d
20,d' 02
AR
The following two lemmas are related to the ratios pv, (x) (X)))))))
Lemma F.1. Let Y, and (z) be defined in (F2) and F.3 Under Assumptions 5.4 and 5.1 suppose our time schedule satisfies (2.6 Then, for S min 2(++)a Tea ond 201 og R2 have
we
Py, (x) (x) SeltaweillPP+0-1aa
Moreover, we have
( ( Py, (x)
(PF.lY,)(x)
PF, (x)dx S 1.
Lemma F.2. Let Y, and Fi(z) be defined in
F2 and
Under Assumptions 5.1 and 5.4 1
suppose our time schedule satisfies (.6). Then for 21 lod 2 201 we have:
4R
Vq(T ),,x) t,x) 1 (x)))
d
(x)d) min [T 1}
22

---

Published as a conference paper at ICLR 2025
The following two lemmas show the upper bound of the derivatives of the true score function with respect to and x separately.
Lemma F.3. When X₁ is defined as under Assumption 5.1 we have:
d
R² [[][]] + R)²
tr (v2 log
+
min {t. 1
min 1].4
v log qtt, x))
| + R
(||x]] + R)
[|x] + R)3
min (t. 1)2 +
0t
min 1}2
min 1]3
Lemma E.4. When X, is defined in F.I under Assumption 5.1 have:
| + R min 13
| lt, (t, x)
<
R
V logg(t, x)
min(t, 1}
min 1)
R3
VVtr(V2log(tt q(t, x))
mt 133
The proofs of the above lemmas can be found in later sections.
F.1 PROOF OF THEOREM 5.5
Proof of Theorem 5.5
From the discussion in Section 5. know that:
(x x) (T 122 x) V log -th (x);
(F.6)
(t, VsO(T tk, x) V2 (2log QT -r- (xt),
(F.7)
in this specific case. Recall that g(t) v2 and
and ((T x, -x=-Fi(a From F3
have:
i F(1, x) = Fx(()) + so -kk, x).
Øt
Thus we know that
a1 r (l. [F] + Vso 1),
(F.8)
By Theorem
we know that:
TV(Y--5 Pex) S TV Td)
dat1
[VB[l 1
V log qTT - t,x)pr (x) PF,(Y.,,
2
PF,(Y, (x)dx
(x)dxx
k=
Js: estimation ervor
/ E tr (V Xx.)) /
Pr,
2
P
PF.CY. (x) (x)
Divergence estimation
(V log QT- (z)
V log (x)) v log q(T t, x)]]((TT -t, xx
Ja: Scare discretization error
+
(r( (e(22log/ (T Lc, (6.22)=
log
q(T x)dx
Jg: Divergonce discretization error
(S tk. z) (VF r) dr.
+ max tr
(R9)
Ja Bias ervor
23

---

Published as a conference paper at ICLR 2025
Bounding the Score Estimation Error J1. We easily verify that 1j is small enough to satisfies the condition Lemma F2 Therefore, using Lemma F.2 and (F.6). we have=
d | - V min - t, 13 V
Be So(( - L. Y..)
V tr-a
qr- (Ya)
(F.10)
Bounding the Divergence Estimation Error J/2- Similarly, our v is small enough to satisfies the condition of Lemma F.1 we have
Using Lemma
F.1 and F.7
the Y - V22 Vo logg tk. Yi.)) ?
J2 V
Eq tr
Vsg
(F.11)
Bounding the Bias Term J5. From
we know that:
VF (x) = (FF
= = + (e- V) P,
Using Assumption 5.4 and t - t2 1 1 1, we have:
((VFF) (x)) Ialle $ (c'-tx - 11] Vsg (T tg. F, (x)ll2
2(1 (1)(1 D
Let A (VF.) (x)) L2. Then we have |A|| S 2( t1)(1 + L). Thus we know that:
|| (Fr (x)) Lalle = [Lv + +A)+ Lall2
8
1)"A A
(10
|
1=1
(iat)
- 4(t t1) L),
(F.12)
here (i) holds because of the series expansion for ||A|| || < 1. (ii) holds due to the Cauchy-Schwarz
inequality. (iii) holds due to our assumption 77 - 1/4(1 + L). Then we know that:
V (x:) I.) Ia)]
[Js max
tr
Vso - t4. z)
w8 max Vse (T t4 2)
- It) ||
5 max
Vso (T 4,, || / (1) Lall2
4dL(t (1 + L), where (i) holds due to tr(A) < A||A||2. (ii) holds due to Assumption 5.4 and (.12)). Thus we have:
|J S (t dLL
(F.13)
Bounding the Moments of Y,, Before we start estimating 1/3 and J4, we first do some preparation. Since we have Y, XT-1-
((--() Xo + v1 - e--2-(2-- Z. here Z ~ Lo). Our goal is to bound the moments of Y,. When T - < 1, we know that e (T-t) = 0(1) and v e 2(T O(VT t). Thus we have:
E Y, 2 = e 2(T D Xol1 2 + (1 - e 2(T 1) E E 2 + Ee (T- 1) /1 2-((T (Xo:
Xxo Z)
S X (T - F 2
= R + (T - t)d
(F.14)
where the first inequality holds due to the independence of Xo and Z. When T - > 1, we have:
E Y1 -
X E 2
24

---

Published as a conference paper at ICLR 2025
-2 R² d.
(F.15)
Combine F.14 and 1.155 together, we know that:
E| x R² + min - 1t.1dd
(F.16)
After computing the second-order moment, using the inequality (RI)Y - IR] Y. we can bound the first-order moment follows:
Y S R -,, 1.1V..
(F.17)
Moreover, consider the fourth-order moment using similar methods. When T - 1, have (T- (1) and vI - e-2
O(vT t). Thus know that:
- | X / + (T' - B|
S R + (T - df
(F.18)
where the first inequality holds due to ||x y| < (||x| + ||>l|) - C([|x|| + Ilyll*) for some constant C. Same as what we did earlier, when T - -t 2 1, we have:
EJ YIll e 1(T 1 Xoo + 1
Se 3(T 1) Rs - dd
(F.19)
Putting (F.88 and (..19) together, we have:
R min - m,,
(F.20)
After computing the fourth- momentum, using the inequality E Yi VEIY.PP.VE Y, we know that:
E Y S n8' + min - 14, +,,
(F:21)
Bounding Divergence Discretization Error J4. Let - We know that
We start by bounding ||z
x||-
|z - x|| = ||F (x) - x
= -c'-th 1) F.- (x) + =o (T - tx. F, F, (x))]
< 2(t - +x) F. (x) + (T txk F; (x))l
S 2(t (L 1) F, (x)]
tx)c,
where the first inequality holds due to 1/1
The second inequality holds due Assumption 5.4
From (.4) we know that |
(x)ll < 2| + 1. Hence we have:
Z x|| $ (t t L + 1>||x|| + (t
SV
(t tu L[|x 1).
(F:22)
Using Lemma F3 and Lemma
R4 we have
2 Vo (T L2, 2) 2 loga(T 1, )x)x)(((T))) - xxx)xx
R² R|([x|| R)2
R3
(t (t-11) (x)
+
xx 21 a(T)
9(T
t, x)xx
min(T (T' - 1, 12
min(T [ - 1, 1).
min [T - t, 113
3
(t tk)
d
R2 [ + 12)2
J0.
min(T - t, 11
min T - 11
133
(1 - (x)) ||| + 1) q(T - x)dx
min(7 1, 133
R²
S (t (x)
+ (t- (t)
E(1 2 + RP)
min
(T 1, 1122
min - 1,
25

---

Published as a conference paper at ICLR 2025
R
+ (t 1k)
L E + +)
min (T t, 1}
(1))
d
R + R Rinniin t, 1]d
x (t tk)
min (T - 1, 121 +tt (6)
min (T - 1,
LR
+ (t tk)
(R + VT 1),
min - t, 1]3
where (i) holds due to F.22p.
(ii) holds due to F.16 and F.177) Assuming R 2 1, we can further
simplify its form:
R
R min (T - t, 1]
Jal 2 (t tx)d
(t- t1)(1 Lmin(T -t11))
min
(T t, 1]3
LR3
+ (1 - 1x) min -1,15/22
R²
LR
LR
$ (t - tx
- t + (t - tk)
-t, 1,4 + (t - -tx)aa
-t,
min (T - -t. 1,///
(F:23)
Bounding Score Discretization Error J3. According to Lemma F.4 we know that:
| + R
log (T - x] 2
min [T t, 1}
R²2
R²2
x, - min (T' - t, 1}
min T t, 1
min - t, 1]2
Here (1) is because we assumed that R 2 1. Using Lemma
F3 we have:
Volo q(T -1x)
||x|| + R
[||x| + R) R2
(||x|| + R2)3
a
min (T 1,
min (T 1, 132
min (T' 1, 133
[|xl + R²
([|x|| + R2)
min(T (T - 1, 1,2
min(T t, 133"
Thus know that:
1
V log (z) - V log qr ( V (0 - ((x)))) 1, x)dx
(||x|| + R)2 R2
(||x + R)3
|||| + R
(t
(x)
+
min [ 1, 1,133) min 1, 11 9(T xxx)xx
min -1 1
Su
122
|x|| R
+
||x - z|| min - t, 1
min - t, 1} (( - x,x)xx
R22
S (1 - (x)
B([ + R2)3 + (1 - (x)
E[| Yrll + R2)
min(T - 11,
min (T - 1, 1]
R²2
||x + R.
+ (t - tx)
[||| + 1)
min(T - t, 13 9(T t, xx
min(T - t, 112
(
R22
$ (1 (x)
(R22 +
-1, 1,, 1,/22/28881)
min T t, 1113
1
(t (x)
(R² mini 13203)
min T- t, 1)4
LRR²
(t t+) min(T 11 (R² + min 1)d).
(F.24)
where we use F20 and F21 to bound the third and fourth moments of Y, (i) holds due to:
[ [ (|x] + [|x[ + R) - t, x)xxx S E Yill + R)² S R² + min(T - -t, 1)d.
26

---

Published as a conference paper at ICLR 2025
Reorganizing terms in F.24
we have:
R1
R²
R²
[Jal (t tx)
+ d3/2
min (T t, 133
min min(T- 1,, 1/3/2
min (T t, 1]a
1
LR
LR²
d min tt
(F:25)
min -
1,1 -,
min - t, 1]2
Putting J3 and J4 Together.
By F.23 and
F.25 and since we assume and L 2 1, we have:
R²
LR
LR
JJl
J1 (1 4))
+(1
())
+(1 - (4) Vi
min T - 131
min [T
1, 1,
min 135/2
Ci
C2
C3
R
R²
RA min
+
4k)
8/2
min (T t, 1)3
min [T - 1,133/2
t, 1]
C4
C5
Ca
1
LR
LR
d
min (T - t, 1
min (T - t, 1|3
min t, 1]
Cr
Cs
C
(40)
S (t (x)
Ci + C2 +
C4 + Cs + C7 + Co.
(t - tx) d
1
6/2
R2
+ d(
LR (T (T- - 1, + R2
min - t, 1,2
min (T t, 133/2
min - 1]3
min -T 1,552/2 + m' t, 1] + LK
LR
(F.26)
min 1, 1]
Here (ii) is because Cg S C2 and Cs < C2. Combining Everything Together.
F.26 back to F.9 and taking the integral, we can obtain the following Plugging (F.10 [E.11 F.13
inequality:
TV(PT 86) $ TV(Ta, QT)
Σ (tu41 - ta) Eotr
(T - th No.)
72 log tw. Nr.))
Ki
+
Σ (l++1 - 1s)
(o Lx Yr.) V log qr to (Yr.)
min tet1,
K2
+
> (tx41 - dlL
Kg
Σ (l++1 12)2
d
1
1/2/2
RR2
-T d [LR - 1,, R2
min t, 1
min -,, 1]3/2
min(T f, 1]3
LR3
LR
min(T'- - 1/55/
min T - t, 1
Using Assumption 5.3 we have Ki Ediv. For K2, using the
inequallty. we have
d
Σ ((x+1 - t2)) min(T to tk 13 Eq so - tk. Y..)
log fr -12 (Y,,)
27

---

Published as a conference paper at ICLR 2025
E (1++1 (k) min 1k+1, 11
d
Σ (le+1 (o)Bolls0( Lx, Y.,) V log -4 (Yt.) |
de
For K3, know that:
10 11) < 11 NdT3.
For Kg, using our assumption on time schedule x t S 17 (I' - (x+1) we have:
K4 Sn d + R + LR d + Rºd 8
1 LR 8z
+ LR
(0)
S n22N LRt
12 + LR + R² J
+ LR
5²
((8)
X n2N LR
LR 82
Ln where (ii) holds due to R2 - d + きF and 2T / S LRRdd + IR J. (iti) holds due to
2d0/2 R² S LR + LR d and 2LR < df + LR R.. Putting K1, K2, K3 and K₁ together,
have
TV(Ýr-8: -8) 2 TV(TT: ma) Ediv + vd (dd//Neen /Nesore
R 8
+ N + 5z + L +
This completes the proof of Theorem 5.5
F.2 PROOF OF LEMMA 5.6
AND COROLLARY 5.7
Proof of Lemma 5.6
By proposition 4 in Benton al. (2024). have:
(L((r))=a) S do 2T for 7 2 1.
Then by Pinsker's inequality, we know that:
TV Rd) S V KL & VT
This completes the proof.
Proof of Corollary 5.7 Please refer to Benton et al. 2202)'s Appendix D for a detailed deriva- tion of the existence of such time schedule. N (d log(1/8))//e). using < =1/L, we can easily show that n s
Since we take T = log(d/e))/2 and
min (1((1222d)) 1/2A,,,,)) Hence by Theorem and Lemma 5.6 we have:
TV(ÝT-5. ss) S dde -T + ediv + Vdvnnee + +12 LR ( +
R
8
Since we take 5 = 1/d and assumed that d 2 R/L + L/R*, we can further simply the bound of TV to:
TV S Vde- + eaiv + Eoore + N
Since T= lolggd//e)) and N LR ( + log (18)))))///)) we know that
Vde <e,
17 N/ (T + log(1 ( <..
Recall that we assume sufficiently small score estimation and divergence error, this completes the proof of Corollary 5.7
28

---

Published as a conference paper at ICLR 2025
G ANALYSIS OF VE+DDIM In this section, we focus on the case where f(x) = and g(1) = 1, i.e. taking o(1)² = in VE-SDE. Specifically, the forward process X, satisfies:
dx, dW
Xo go.
(G.1)
and the true reverse process Y, satisfies:
dy, = - (r Yo ~ dr- 4r-
(G.2)
2
Then, the forward process has the following closed-form expression:
Y - Xo VT iZ, Z N.)
(G.3)
Recall that we interpret DDIM as a numerical scheme, and provides the following discretized version of the reverse process:
1++1 Vr, + (tk+1 tx)1 1 (1-1)111111 SO tx, Pt,),
T tx t t
1=
(G.4)
Therefore, denoting 1(1 v1 1/1) by ct. we have a S Then we can define the following interpolation operator:
F((z) 2 (t- IN 2),
(G.5)
satisfying Fie.,, (V,.) Y 1 Under Assumption 5.4 we know (t, .) is Lipschitz. Therefore,
suppose 1/ L, then we have 1/2 $ || - 3/2. particular, F; is invertible.
Moreover, using Assumption 5.4 and Fr (x) - -(t ( (T - tx, Fi (oFF((()),, when
S Ltc we have
(x) x|| (t ((--t)) ta)aallse(( (T tx, Fi (x))] (x)]l I (t L1l (x)l + (1 4e)
I ||x|| + 1.
(G.6)
Thus we know that
| (x)Il S x||
(G.7)
Before we start the proof, we introduce several important lemmas used in our proof. Let Cj = (1 + d/2)4L² and c (1 d/2) c)2. Using our assumption on r7. know that
min
(G.8)
4c
The following two lemmas are related to the ratios py, (xx)///)),,,,) (X,
Lemma G.1. Let Y, and (z) be defined in G and G.5). Under Assumption 5.4 (Lipschitz condition on se), Assumption 5.1 (bounded support) and suppose the time schedule satsfies (2.6). assuming S min 2: L+o' 4o =2) RF we have
PF.(Y,,) (x) S ( Patt
PY, (x)
where ej = (1+ and 02 = (1 (1+d/2) d/2)(l c) Moreover, we have
Jn (PPLY..)
px, px,(x) (x)
PF, (x)dx $1.
Lemma G.2. Let Y, and (z) be defined in
G.2 and G.5
Under Assumption 5.4 5.1 and
2.6
d/2) and c2 = (1+ suppose & min पाे 2(442)e J. where (1+ then, we have:
(T t,x) 0r P)((,,,)x)
(x)m))(x))dx < d
-
29

---

Published as a conference paper at ICLR 2025
The following two lemmas show the upper bound of the derivatives of the true function of the VE forward process concerning and x separately.
Lemma G.3. When X, is defined as (G.I), then under Assumption 5.1
have:
v log (logg(t,x)) (Vlog(((.))
|
+
[|x + R2)
0t
2
t
Moreover, we have:
i 0t
R ([|x]
+ RR)
tr (v2 log q(t, a(t,x)))
2
+
Lemma G.4. When X, is defined in G.1 then under Assumption 5.1 have:
||*|| + R
V log (4, x)11 S
t
R22
V log
x)11 ||| S +
82
R23
Vir(VV log q(4, x)|[
The proofs of the above lemmas can found later sections.
G.1 PROOF OF THEOREM 5.9
Proof of Theorem 5.9
From the discussion in Section 5.
we know that:
4 (t, x) = c/s0 (T - x)
v log -r (x),
(G.9)
(x, = a Vs - L. x)
2 log Cr -r (x).
(G.10)
2
The interpolation operator can be expressed
Fx(x) x + cr(1 s,))))(( 1x, x);
(G.11)
this specific case. Recall that g(t) = and f(T - t, x) = 0. Denote Fr (x) by z. From (G.II) have:
0 [t, x) =c:So(T' t1, x).
Ø1
Thus know that
V
F (t, F. (x)) a VS 22,
(G.12)
Ôt
By Theorem 4.3
we know that:
TV(Yr s. Pex) TV(gr, ma)
N-1
V log q(T 1, 1)x (x) 1
PFRCY (x)dx
PFi(Y.,,
Jy: Score estimation ervor
PY, (x)
E (1,(l, Y(,))"
(x)(
PF. (x)
Jq: Divergence estimation errur
1 / (V log (z) V log (x)) V qT (t,))alt x,x)xx + 2
Jy: Score discrctization error
+ 1 / (r((2 log q[T t4, z) v log -,) xxxxx
Divergenee discrctization
30

---

Published as a conference paper at ICLR 2025
+ max tr at Vs (T tk: 2)
(VF,'(5) (x) - -1)
(G.13)
x
Js: Bias
Since we assumed s min (f 1/244 enough to satisfy the condition of Lemma G. and Lemma G.2
we can easily verify that vy is small
Bounding the score Estimation Error J1. For the first square root in J1, we know that:
E
Y., 01 Blllaso(T tk, N.,)
Vloger-4e
V llog or -u (Y%,))13
E
[[[1
2 (T Ls, N.,)
v Vlog (Y Il (er - 1 2)11s0(T las r
[1 so 4. 1r))
log qr. (Y, )1 + / T. (cr - 201(50(( 1/, Y,, ||..
2
(G.14)
Using Assumption
5.4 we have |so
tx: Yr, ) S L Y || + c. Since Nr. = Xo + vT - Z
we have:
E
Y 11 = EE| X + (T - tR E 2
S R + (T t
Moreover, recall that we take = (T' tk)/ (tx+1 tw) z 1/1.. We can show that 0 S a Hence we have:
S 17.
1 ) L, V,,),
1 -
2 (c: - _ R 1.2 Y 2 c
STVTRR + L2(T
4 )d
+ c²
(G.15)
For the second term, since our 7) satisfies the condition of Lemma G.2 we can apply Lemma G.2 and obtain
log q(T 1, x)xx, (x)
d (x)dxx S T-
(G.16)
PFI(Y,, (x)
2
Combining G.15).
(.14 and G.16
we have:
d
J1
E [|so Lu Y.,)
V Vlog GT -n 113
T
d
+ "V 1212
(G.17)
Bounding the Divergence Estimation Error Jz. For the first term in J2, we know that:
E tr (a: Vso - tk. Yr.) - 7 log - ux (Y
2
1 Vso(T - ts, 1.
1 2 + (a- 2) (tr Vso(T - tv. X(,))²
E tr
22 log qT (X
2
V B[tr (Vso(T
Lu Yr,)
22 log (Y..) 2] + (a (a-
9T
V (So - ln N..)) ))
1 E[(
(G.18)
Using Assumption 5.4 we know that V (T - tx x)112 I L. thus tr VTp(T - tk. x) 1 dL. Same we did with Jn have 0 I c n7. Moreover, Ki can be further bounded by
(a
1
E B[ Ls lx, Y),)) S ndL.
(G.19)
2
31

---

Published as a conference paper at ICLR 2025
For the second term, since 7) satisfies the condition of Lemma G.1
apply it and obtain
Saa ( PFIY (ts) (x)) x 1.
(G.20)
Combining (G.18. (G.19)
and (..00) together, we have:
2 J2 S V RQ ir (Vso(T - Lxs V,,) 2ol (( Lk, Yrn))
+ ndl..
(G.21)
Bounding the Bias Term J3. We know that:
VF = [[VF)(FF"(x)]]
= [T + (1(1-44)))00
(x) Vs) LR. F. (x)]
Using Assumption 5.4 and / - 44 S 1) S 1. we have:
I(VF:) (F: (x)) - Ialla s (t - (4)4 Vsp t, Fr (x))l2
(t - (1+ L).
Here we use oj 1. Let A (FE) (x)) La. Then we have || S (t (t-tx)(C tx) L). Thus we know that:
|| (V (x)) Ialla ( A) -1
Iall2
00
1)a A
(88)
| 1
(i14)
I 2(t t1. (11+00 L)+
here (1) holds because of the series expansion for |A < 1. (ii) holds due to the Cauchy-Schwar
inequality. (ti)) holds due to assumption 1/2(1 D). Then we know that:
J3 max tr a Vso
tk z) V ((F, (xx) It)
2 dmax
V 4. 2) V (F. (x)) Ia] l12
< d Vso(T - 44 2) ||2/|V (F (x)) Tall2
S dL(t 21) + L),
here (i) holds due to tr(A) < ||2 and < 1. Thus we have:
|Ja| (t ddT.
(G.22)
Estimating z - x. By Assumption 5.4 we have:
-|z- x = |.Ft (x) x||
x||
(I - 1x) |s0 (T L1. Fx)(()))
< (t - (x) Ll| F. (x)] + (t 1x)c.
Since we assume that f. 44 < 1 < min az: : ]. From G.7 we know that |IF s 2||[||++ +1.
1
Hence we have:
|| 2 x|| S (1- -1- + 1) + (1 - (x)c
(9)
2 (t- (t-+))(((x) +
(G.23)
32

---

Published as a conference paper at ICLR 2025
Bounding the Moments of Y. Before we start estimating Ja and J,, we do some preparation work. Our goal is to estimate the
moments of Y, Since we have Y, Xr-t = Xo + / 12, here (O, I2). We have
E Y|lll = |E| X 2 (T - 21|
- R + (T t)d.
(G.24)
After computing the second order momentum, by the inequality (E) Y,11)* N Y|I know that:
E Yill S R+ VT tyd.
(G.25)
Moreover, we consider the fourth order momentum as we will need to estimate it later: we have
B||| x E X $ + (T' ()-1)
x R² ()
(G.26)
After computing the foruth order momentum, by the inequality E|| S VE V Y. we know that:
Y.
Y 3 S R² + (T 11-131220122
(G.27)
Bounding Divergence Discretization Error Ja. Using Lemma G.3 and Lemma G.4] we know that
8
Lr (V2 log loga(T - -1,x))) ~ (T 1)2
d
[² + R2)²
Øt
(T (T-
RB
x))| S (T - t)3
Thus we know that:
1/1
tr V2 1T - tx, - 22 log q(T - t, (1xx))9(TT) q(T - t, x,x
94
(tt) tx)(
d
[ + R2)2
(T (T-t)a t)3 ||x 211)0(TT - -t,x)dx
R3
+
(T -t)
(T - -t)
(9)
((-ta)) [x)(
d
R (||x|| + R))
(T-t)* (t - [==x)[((xx] + 11)) -1, xxxxxxx
R3
[T t)²
(T t)a
d
R²
R3
$ (t (k)
+ (t th)
B(1
+ R²²) 2²)+ + (t-
L -E([| + 1)
(T' - t)2
(T t)s
(T t)3
(10
Rd + RT t)d
LR3
$ tk)
(t - tx)
+ (t 65-ta tx)
(R + VT - tvd (T t)3
(T t)2
(T t)
Here (i) is due to
G.23 and (ii) is due to G.24
and G.25
Assuming R 2 1, we can further
simplify its form:
R²
Rd
LR
[J4 S (t tw) (T -T 1)2 T 8)3 + (1 + L(T -t)
+ d
(1- + Vd
(G.28)
- f)5/2
Bounding the Score Discretization Error Js. Using Lemma G.4. we know that:
|x|| + R
V log x1,,
S
1
1
R²
2 log x)
0/12 <
Moreover, by Lemma G.3
we have:
(logg(1x))
x)
|
+
+ R)3
ot
t2
t
33

---

Published as a conference paper at ICLR 2025
Thus we know that:
1/.
V Plogrr-ce (z) V log (xx) V log q(T - xx, - t, xxxx -txxxx
R2
(1-1x) (x)) ( |x||| + R
+ [||x|| + R))0
||X|| + R
T - q( - L, x)dx
(T - 1)2
(T - -1)3
1
+ (T' - t)2 ) ||x z|| ||||| + R q(T - t, x,x
R²
+
T
T t
1
$ (t- tb)
B([ + R)² + (t- 1x)
(T t) E(|ly + +R)
(T - -t)3
/
1
R²
|| + R
+ +11 - 4x)
+ (T- - 1)2
[[| + 1)
T -t Q(T - 1, -1,)dxx
00
T t
1
2 (1- (n)
(R² + (T l)d) + (1 (x)
(T - 1)1 (R* + +(T - (1)adP
(T- 1)3
L
LR
+ (t- (k)
RR2- + (T t)d) + (t - tx) (T o t)s (R² + (T - -t)d..
(G.29)
(T t)2
Here we use G.26 and (G.24) to bound the second and fourth momentum of Y, and (i) is due to:
(||x| + x|x + RT -1, & E(I + R) x R² (T - t)d.
Reorganizing terms in G.29. since we assume that L 2 1 and R 2 1. we have:
LR
L
Jul S (t - tk)
d
(T - t)2 + d
+ d
+ d
(T
t))
(T - -t
T - t
R³
R²
LR?
LR
+ (T t)4
+ (T - t)3
+
+ (T f)3
(T t)?
St - th)
d
1
+ d
LR
+ d
L
+
R4
+
LR
+
LR
(T t)2 (T - t)2
T-t
t
(T
t)3
(T-t)* t)2
(G.30)
Putting J3 and J4 Together. Combining (G.3 and G.28)
together, we have
LR
L
Rt
LR
LR
JJa + [Jal $ (t- tx)
d
+
d
d
+
+
(T t)2
TT
t)2
T - t
(T - t)
(T t)3
(T t)2
C1
C2
C2
C
CN
R2
RA
LR3
+ (t- - [x)
d
(1 + L(T - 1))
Vd
(T
1)2
(T
1)3
(T 1)11
(T - t)5/2
Or
Cs
Co
C20
①
S (t 4))
C1 + C2 + C3 + Ca + Cs + Cs + cco
(t - tx) d
1
LR
L
R2a
+ +d-
+ d
+
(T -t3 (T -12
T-
(T' tt-
LR
R22
LR3
(T 1)3 (T- -13 Vd
d
(G.31)
TT - 1)6/2
where (i) is due to C7 < C2. Co C1 + Co, Co C2. Combining Everything Together.
Plugging (..7). (G.2T). G.22) and G.31 into G.13). and taking the integral, we can obtain the
following incquality
TV(Yr-8. sr- S (V(
34

---

Published as a conference paper at ICLR 2025
+ Σ (tx+1 - (4) / BQ tr
(Vsa(T
tk Ni.) -
4 log tk. x
R4
+
Σ
(l++1 - 1.)ddL
R
R2
d
+
Σ (t&+1 t1e)
EQQ
5o(T the M.. - V Vlog (V
T tx+1
Rs
+
Σ (t&+1 - -1/1 - -x+1
1 IIRR2 L?(T t c²
124
+
(1x41 - 1o12
k
+
Σ (l++1 - (1) 68
(T 112 + d
1
LR²
+
L
Rd
(T (T-1)3
1)2
T
(7- t)
h
LR
d t33 + Vd LR3
R²
+ -T- - t)3
(T t)5/2
We further denote the last term by Rs. We know that Ri is Ediv. For R3. by Cauchy-Shwartz inequality, we have:
Ra S Σ (tu+1 - (x) T tk+11 V
d
Σ k
(th+1 - tx BB (T tu, v.,) V log -1 (Yr))
A
S VnNd= Escore.
Since we assume 1, for R2 + Rs. we have
Σ (lK+1 - 4.dd +
$ ?
For the R4, we have
d Σ (1u+1- -w)
T-4+11
(1)
S 11 2(1mF -tw)l. 11+1 + - tx)d
(11)
2 " Σ VVLLR++ Lt
N(VaLR Ld).
here we omit e (0), and (ii) is because T-le T-
< 1 + 5 Hence we know that
RR R4 + Rs S N L2d LRVd
Finally, for the RG, we know that:
R
LR.
R2
d
Rs s 1 & LRdd+ Ld+
d
LR
82
8
8
35

---

Published as a conference paper at ICLR 2025
R S 1 d LR
d R$ +
LR*
8 dd LR
d+
Putting together, know that
TV (T(:). + Ediv + nnd
Gocore
+ N d LR I R² 1d+LR
R
LR
V8 +
+
82
8
(40 x TV(Ta: (T) + Ediv + Vnd Goore + 1 LR + L'' LR
8
here (dit) is due to 2LR & LR d LR' 2 L LR and LR
S d + LR
(and < as well as R 2 and L 2 1). This completes the proof of Theorem 5.9
0
G.2 PROOF OF LEMMA 5.10 AND COROLLARY 5.11
Proof of Lemma 5.10
Since qejo (x xx0) (xx; xo, tla), we have
(= 7Ia)) |
(G.32)
27
By the convexity of the KL divergence.
(/
(g [N TIa)) KL
TIa)
KL (el(ero((t
3
Eg
|||X
27
d
(G.33)
2T
where have used that Ego [|
d. since Cov(go) Is. Applying Pinsker's inequality
to (G.33,, we have:
Vd
TV(Xo Xo NO, <
(G.34)
2T
This completes the proof of Lemma 5.10
Proof of Corollary5.11 5.77 Please refer to Benton al. (024)'s 's Appendix for detailed derivation of the existence of such time schedule. Since we take T d/e² and N = (d2 log(1/8))²/e), using S 1/L, we can easily show that S
min (in11//(D22T)))1122211 vd ViP7-,YY--s) vT Cav + CBVNNNCu + 1 N LR'd + LR
Hence by Theorem 5.5 and Lemma we have:
11
82
Since we take 5 = 1/d and assume that d 2 T./ E/RA, we can further simply the bound of TVV to:
Vd
TV(ÝT-5. Yr-) <
VT Ediv +
+ N
Since T = =de² and N LR (2 + log we know that
Vd
S E, VT
1) N LR &
(T + log(1/)) <e.
Recall that we assume sufficiently small score estimation and divergence error, this completes the [
proof of Corollary 5.111
36

---

Published as a conference paper at ICLR 2025
H LIPSCHITZNESS OF SCORE FUNCTIONS
In this section, we consider the forward process by gradually adding noise to data distribution, i.e..
X, f() Xt)x g(t)
(H.1)
where Z ~ I), is standard normal random variable independent of data- and f(t) and g scaling functions determined by the forward process that modulate the influence of the data and noise over time. Both VE and VP forward processes can be represented in this form. The distribution of X, is denoted by xt. We will show the Lipschitzness property of V log qt,, x) under Assumption 5.1
H.1 GENERAL RESULTS
The following lemma regards the upper bound of the derivatives concerning
Lemma H.1. Suppose Assumption 5.1 holds. Let X, be the forward process defined in H. and its distribution is denoted by g(t,x) x). We have the following inequalities:
xx| + S(DR
v log xt,x
9(1)²
1 f(t)2
2 log x x)1 -
g(t)2 2n2
9(1)2
vir(vv (t, x)) S 6f(t)²
g(1)6 Ra
Proof of Lemma H.I the score function step by step. A key point in our proof is that we bringing the terms involving x outside of the integral and canceling them, resulting in expression where only appears in the function's evaluation. Firstly, we derive the form of our score function V In q(t, x).
We prove the lemma by gradually computing the higher-order derivatives of
Expression of the Score Function.
To start with, using the expression of the forward process X₁ in (H.] we represent q(t, x) as the convolution of the data distribution and Gaussian distribution. Specifically, we have:
[ Pranxo (x) - y)dy
g(t, x)
[ f(t) f(t)Px
pxo f(t) ) P0122(x y)dy
px. (Y) (x - (x-xxxxxxx0 f(OY)dy
[ 90(y)pz
x --((fl) dy
g(t) g(t)
x f(t)y 1
qo (v
dy.
(H.2)
g(t)
g(t)
where we represent the probability density function of standard Gaussian distribution <(x) (e [|x]1/2)/ (2))d Besides, in the second and fourth equations, we use the change of variable formula of the density function, i.c., Pax (x) = Px (x/a) ///a/a where X is a random variable and is a constant. By directly taking the gradient, we have the following property of ((-):
Vo(x) = -(xxxx
Moreover, by the chain rule, we can calculate the gradient with respect to x: f(t) g(t)
f(t)))
(x-((thy) f())
x
g(t)
g())
g(t)
X f(t)
f(t)y
Φ g(t)
(H.3)
g(t)
Thus, we can express the score function as follows:
x))
V log x).
g(t, x)
37

---

Published as a conference paper at ICLR 2025
JRa Pdata (y)
x-S(t)y dy
9(t)
g(t)
Jee Paatal (y)
x 5(1)y dy
9(1)
Jia Pdata (y) Vx
x- f(lly) dy
9(()
Sga Pdato (x)
x --(tty dy
g(()
(6) J Paatta (y)
x x-1(1)))
)xb( x- (0)) dy
0(8)²
9(2)
SEze Pduta (y) Φ5 x f(tly)
dy
g(t)
1 Sg Patata (y) ) (x f(t)y) (x= f(tly ddy
g(()
(H.4)
g(8)²
Sma Pdata (v)
x- f(t)y)
dy
g(t)
where (i) holds due to (H.2) and (ds) holds due to (H.3).
Under Assumption 5.1 we have Pdata (y) = for |
> R. Thus, we have
Ssr Pdata (v) (x f(t)y) 0 x=/(0x dy
log (t, x),1
g(1)2
SR. Paana (y) x- x-110x dy
Sg Pettaa (a)(t))lx x [(t))|1b x x-1U0 Jdy
a(t)
g(t)2
J Daata (y) x- sitflyy dy
JR Petta ( (1|x( + ((t)lly)) o (x- f(tly Jdy
)dy git)
g(t)2
SE P X- f(Ely ) g(()
|||||
S())
g(1)²
This shows that, under Assumption 5.1 the norm of the score function can be bounded by linear function.
First-order Derivative of the Score Function.
We proceed by computing the Jacobian matrix of V Vlog qt,, x). Directly calculating the gradient of (H6 we have
SEu Pdata (x) f(t)))&( x f(tly dy
(lgg((,x) v[-
1
g(t)
g(122
SE Pdata (x) x-1(0) )dy
9(1)
1
V ( Pitana (y)(x ((1))) x f(t)y x-5(t)y)
dy x(xx)
gt) {xx)2
g(t)
Pdata (y) ((())) -(((tt)
)oibayy dy (v (v/
Pdata (v) (v) (0)0(= x-f((ty f(t))
dy
g(1)
g(()
Rd
12
(H.5)
where the shorthand expression (xx)) SKs Paa (y) x f(tly
dy, and the last equation
s(t)
holds due to the gradient formula of vector-valued functions, i.e., V [xx vs(x) f(x)Vo(x)T Firstly, we compute Iı:
o(x) 9(x)²
Jpe Pdata (x)))))))
x x- f(ty) dy
1 =
g(t)
(6 x (x-s(t) f(t))
I+ (x (x-)()))VV (0(x- )7)) dy
J Pdnna (y) (y)
g(t)
9(1)
(xxx)))
g(t)2 )) dy
Pd (y)
ΦI (x -(t)))
Bd
38

---

Published as a conference paper at ICLR 2025
J Panta (y) I (x (xx)))(x (0--(()))) f(t)))
odyy
g(1)2
/ Pdata (y)ody .
xx
Pdntaa (y)
f(t)
Pdota (y)y T ody
g(t)²
((1)2
JR
Rd
11.2
f(t)
5(t)2
JEc Paat (y Jyy ody,
Pdatal (y) (yyddy
x
(H.6)
g(1)²
9(1)2
where we use the shorthand expression xx=0=-1000000000000000000001 Next, we compute T2:
( and (i) holds due to H.3 g(t)
U/e Pdttaa (x)(xx((())))
x -((1))
x s(t)y
12
dy
V
Pdata (v)
dy
9(1)
9(1)
Rd
1 Petatu (()))x(((()))) x -((()))
x -((())
x f(()
T
dy
Pussa (an
dy s(1)
9(1)
g(1)2
Rd
Rd
Pataaa (y) f(t))) dddy
(x) Py
Pdtca
f(t)y) (ddy)
g()}
Ba
xx'
Pataaa (y) Jody + [
[(1)
Pautal Panaay) (y)ody
Passa (y)y Jdy.
Panta (y Jody
g())²
g(t)2*
Rd
Rd
Bad
12.1
f(t)
(1
f(t)²
[
Pdata (y)y Jydy [ Pytaaal)) Joyy
Pdata (y) vydddy
x
Pdata (y) (ydyy
91)2
Rdd
g(t)²
12.3
(H.7)
where the second inequality holds due to (H.)). the third inequality holds due to the short- hand expression ④ s(² x =1(00x) ). Substituting (H.6) and (H.7) into H.5). notice that 4(x)
9(1)
Ja Pamaa (v) (0=~0(100000 dy, ,,y, I1,1 12,1, I12 12,2 and 11,3 12,3. Thus, have the following a(t)
equation:
f(t)² Sax Pdata Paaca (y (v)yy" ody SRe Pasalyy Jody
V2 (t, = g(t)²
g(t)2
[squ Pdata (y)y
[ Paata (v)y ddly] )}
(H.8)
(Jiea Pdata (y)dd)))
Using Assumption 5.1 when Pdata (y) * O, we have Ilyy 1|2
||y||2 2 I R². Moreover, we have
| Patata (y) dydy
/
-I Pitata (y (y)yddy
Petsaa P (y) çidy
=
2
fee Pentta Py)na(y)ll>((
(fee Prataa (x)dy
< R2
Substituting into H.8 we have
11 f(t)2 g(t)2 2R²)
log log gtt, x x)2 S g(t)²
This implies that V log q(t, x) is Lipschitz.
Derívatives of the Divergence of the Score Function. Next, we consider the divergence the score function, i.c.,
[ log q(t, x)] tr (V qt, x)).
39

---

Published as a conference paper at ICLR 2025
To start with, using (H.8). have:
f(t)2
(Sga Pauaa 1
I| Jae Petnal (y) (yyy byddyl1
tr(2 log x))
(H.9)
g(1} g(1)
Jg Pdata (y) odl
(Jza Pauaa (y) Dody)
where we use the fact tr(xx) ) )}- = ||x||²
Directly computing the gradient, we have
f(1)2
V Jus Dauna )ly p
Il Saee Pdutaa (y cycyyyy1
Vtr log g(t)4
Sgd Pasta (y) coly
(Sre Petuaa (y )oddy)
J2
Firstly, for J1, we have
V Sga Pdata (y)lly/odd
fRr Patata (yddyyy
V VJza Pdata (y)ody
-za Pdataa (y) ody
J1
(See Pdata (y) cddly
(H.10)
We calculate the two gradients separately. First, we have
/ Panaa (y) (
Pessa (y) llyl|
pd
Pduaa (y))l>|?
-x-1()) 5(()
x-
Jody
g(1)
Rd
((1)
Jec Patsaa (1) @pyy +
Pdua (y) (lyllyl| ylyllyyddy, (H.11)
yody,
g(t)
g(1)²
where the second equality holds due to H..) Next, we have
Paata (y) oydy
Penna (y)
Vody
RRa
f(1)
Pitana
(y)
ody
Red
g(t)2
x
f(t)
Pdota (y)dy +
Pdnta ddy.
(H.
g(t)²
g(1)2
JRd
Substituting HH.11 and (H.12) into H. we have
J1 s(1)² Jiga Ddata (y)l byy
f(t) o(t)z
SE Paau ( (yyl)/ld SSg Petata (y) Jody
(Jga Pdatal (auuu(y)err )ody)
s(tfe Jga Paata ody + g(EfT Sge Paata (y) yyddyy -Jg Pdata (v) Ily / ooy
f(t)
( (Sea Pama pddy)
ftt
(Jga Patatu (y) 2 yody
Sg Pattua (y ddd Sga Paata Pamull)) (y) y (Pdy )
g(t)²
Jzu Pdata ood
(Jee Pdata ( pddy)
Next, for Jz, we have:
Jow Pdata (y)yody/12 (Sge Paata (y) ( Pdata (v (d)d) -1. Pdata (aanaay)a oddd?
J2
(Jrv Pdata (y Jody)
(H.13)
Again, we calculate the two gradients as follows. First, we have:
/ Rd
d
(Jr Pitaaa (y) Jy
Pdtau (dy
im
40

---

Published as a conference paper at ICLR 2025
/ Pdttaa (y) V Vody
2
Pdata (y) (y)):ody
Rd
1=1
/ Pdata (y)y
f(t)y
Pdata (onnaly)yyee (y)yydyy
ody
g(t)²
JRa
i=1
2x
2
2f(1)
Pdtta (y) oy) / Pdata (ouu(y)yss)) (yyyody
Pdaatl (y)ysddd))
g(t)²
g(t)²
2x
25(()
[ Pdutu (y)yody, (H.14)
Pdaat Pd dyyyyy
Pama (y)yy" ody
g(t)²
g(t)²
where (i) holds due Moreover, we have
H.3 and (it) holds due the coordinate expression of matrix multiplication.
Paiaa (y) Jody 2
Pattaa ( ( V
Patuta (y (yddy
Rd
ggd
Jec Pautlly (daualy)ddy +
f(t) g(t)
Pdata (y) kody
Rdd Paata y y
g(t)²
2x
21(1)
Pesta ( dody
+
Pdata (y) oody
Jg- Poata ( Jyyody
_()2
g(122
JRd
(H.15)
where (i) holds due to (H.12) Substituting H.14 and H. into H.13). we have
2x 0(1)
2)(() Ja Pdata (y) lo)dyll + o(1)²
fgx Pdata (y)yy" ody SR Peat (y)ddd (Jze Pdata (y Dody)
J2
(Se Paata (y) jódy)"
(-
2x [t)²
(Jiza Pdata (d))) + 2 20(1) s(t)²
SR Pdat (v dody Spee Peata (y)yddy) Il fg Pdata (y) 1
(Jes Pdata (v dodd)
2f(1)
(See Pdata (idua(y)yy) (y)yy ody SE Paam yy)yddy
+ Sue Pama (y yyddy Sza Pdanaa (y) sody11
g())
(SEa Pataaay sody))
(See Paisaa (y v(ddy)
We can conclude with
f(t)
SR Pama (y) (yy))lllyy See Paata (y) Jdy
Jg Putta (y) lody fga Pdnaa lyyll |yyll (Fzv Pdata (y (ddy)
||1ll
g(1)2
_(L)
RR3 + R. RR2)
9(1)²
2t(t)
123.
g(122
Moreover, have
2f(t)
[R. Panaa Pamaay)yy (v)yy" odyylelll fga Pdata ( (Jpe Pdata (V) ody) Jau Pdata (y) pody fg Pdata (y y 12
| -
g(t)²
+
(SEd Penna Pa (ody) 3
2f(t)
Sge Pdata llyy
11addy Jgo Pamaa Iyy Sige Pama (v) by)d
g(t)2
+ Ja Pdmal (amalyylll (y)l lody (Jga Pama (y) lyyl|sldyy lsddy)
Sge Patatu (y) (v)
41

---

Published as a conference paper at ICLR 2025
@ 2f(()
Jgn Pdata 1ylllyl/ddd Jo Peata (y) (y) |yyll cody ( (Jas a Putta (y) pody) ?
g(t)²
Jus (y lood (Jau Pamaa (y) lbbdy) ( (See Pataua (y)
( 2t( g(t)
4f(t) R.a
R² R R R²)
g(t)²
where (i) holds due to the fact lly 2|2 1lyl1 for any y E IRd, and (ii) holds due Assump-
tion 5.1 Putting everything together, we know that
VVr((V (()] 6f(1)² g(1] R
This completes the proof of Lemma H.1
The following lemma considers the upper bounds for the time-derivative our score function. We similar techniques to obtain in the proof of Lemma H.
Lemma H.2. Suppose Assumption 5.1 holds. Let X, be the forward process defined in H.I and
its distribution is denoted by gtt, x). Then we have:
0 Ot
v log q(t, ((t,x))) x))
2g' (4)
g())3 (IxII + (5(0RR) + (
S
g(t)2
(0))) (IX|| + (())(2) 21)( ([|xII + S()))
g(1)s
① (tr (V
2d [g(()) 4f(t) (f())((t) - 2f(()g (t) R²
s
Øt
g(t)3
9())
+ 6R f(t)
(())5((()))
[| (()) + (g)( ([xl + S(()R)
g(1)a
g(1)3
Proof of Lemma H.2 chain rule, we have
To start with, we first compute the time-derivative of de ((>-f
Using the
2(1)
x 1(0N)
0
fitly
Dt
g(t)
Ot
(v 2T)
fitly D103
1 9 x - ʃ(t) 2
(H.16)
(v2m) ja
2 0t
g(t)
Since we know that:
0 x
f(t)y)²
0 ||X|| - 2f(x)x 1lt)
Ot
9(1)
It
g(t)²
2f'())xxxx>))) (10xx) 2 F(0)
g(t)a
g(133 (000000T-- - -(01l + (0]||
(H.17)
Substituting H.117 into H. and recall our shorthand expression Φ- = 60(-1100) ), we know that: x011 (()) /(11y)))) + [(]][x 5(1))12 Φ.
(H.18)
Ot
g(t}3
Therefore, assuming
||y|| < R, we have
i Φ =
g(t) f(t)f())
f(t)]ly|13 -
0t
g(t)3
42

---

Published as a conference paper at ICLR 2025
(010(0 (Ixxllly [(t)lly)1) + (0) |X f(t)y|1
g(1)a
(00 [ (()) (t)) ([xx + +(())))
Wn
6.
(H.19)
9(1)3
Teme-Derivative of the Score Function. By H.4). we know:
JRe (ax ((t)))((*'))) x-1(0)) )dy
V log (t,
9(t)
g(1)²
Sue Pdmaa v))
x-f(tly )dy g(()
Taking the derivative with respect to I, have:
v logg Vooga(t, = (t)(t) Jea Putaa (x) f(t)y) odyy
Ot
g(t)3 JR. Panta (y)a
0 fga Paata Pd (y) (t)) ody
g(t)2
Jga Pdata y'dy
Using Assumption 5.1
we can easily see that:
Si Pdlta (y) 100--f(te f(tyy)
|x||| + 15(11R
(H.20)
JEs Pdltta (y) oddy
For I1, we have:
L Sge Petuta (y) ((x 1( (
Jo Petaaa (y) f()) Joly Jq Paata ( oy (Jor Peatal (odataly)ooc) )
(JE. Pelata (y)d)
From H.19 and Assumption 5.1
we know that:
| (()) (1|x R R(t)R²) + (d(())) ([xx]| + f(t)R)2
2( |x|| + (f(t)(R) (f)()))
g())3
(H.21)
Combining H.20 and H.21). we have:
V log (t. x) x)]|
= 2g'(t)
(|>x|) + (f(()(R)
Ot
g(t)3
(t))) (|x|||++ R() + (g)(()) (||xI + f())))
2([|x| + R) + +1(()))))
0(1)2
0(1)3
(t)
||x|| + (1())R)) + 2g(1))5(()) (IX||| S()R2)2 (1) ([x + f()))
g(t)3
g(t)5
[S(()RR
0(t)2
Ceme--Drivvtive of the Divergence of the Score Function. In this section, we consider the t-derivative of the divergence of the score function, i.e.,
i
v
|V log (lg ((1,))
0i tr (V2 log q(t,, x)).
a
To start with, using (H.9).
we have:
d
f(t)2 (Sza Pdata (y) || pyy
1 Pdata (y) (y)
v log (l,
(H.22)
2()²
g(t Sge Paata (y) ooly
(Szv Pdota (y Jody)²
43

---

Published as a conference paper at ICLR 2025
Taking the derivative with respect to t, we have:
0
2d (1(
(V2 (ogg(((x)))
0t
g())3
(t))))( 44(())g'(()
[TR Pdata (y) 11
11 IR Pdata (yyydy 11
g(1)s
Ja Pdma (y Jody
(Sra Pdataa (y) (ody)) 2
Ki
f(1)²
0
Jae Pauta (y) byy ody
Il Jiza Puuta (y yy Jyddyl
(H.23)
g(1
01.
JRe Pa y rody
(Jzv Pdatta (y Jody)
K2
Ks
For J,, using Assumption 5.1
we have
(t)g)) - of(t))
45(() (1)))(())))))))))))))))))))))))) (2(()))(((((((((((((()))) (((1)g(())
K1 S
(R²
R2)
R (H.24)
g(t)s
g(t)s
For K2, Using the definition of K2 (H.3))
we have
Ja Pattaa (y) Ily olyy
K22
01
Ju Paata Joly
SE Panta y)) ||
Ja Puttaa (y) Ily?? pdy ( Ja Paltaa (y) (rdd
JR Pdata y odl
(Sga Pamaa (ana(y)))y)))
Se Petata (y) || ody
SRr Paata (y) 1lyll? (JaaPanll))
Pamaaly) ar odly)
+
SR Peata (y) oody
(Sz. Paata (y)
S 2R (t))
(Ixx|
R()) + (g'(t) (||x| + (t)
(H.25)
g()}3
where the last inequality holds due to H..19) and Assumption 5.1 For K3, we have
Sg Pdata (y) yy
K3l =
Ot (fzr Pdlna (y Jody)
21 Jg. a Pdtnaa (y) 2
1. So Pdtta (y) yddy
06 (Jgr Pdata (odd))
(Jee Peima (y) )2
Se Patta (y)
JR Pdatta (y) yy oddy 11
Sged Pdata (y dyy
Jga Pdtta (y) oolly)
(H.26)
(Je Pauaa (y )ody)2
( Painaa P )4
Moreover, have:
19/
(L
Pdina (y) (y:bddy) 2
Paaaa (v) (y)ddyy
Öt
Ot
2 ( Pdata (haualy)u (y)y)
Pdita (y)y
ody
JRd
Ft
d
( Rd
Pdata (y))a/yl
Pdttaa (x))1/
0 dy
gdd
Øt
1=1
g(t) ((()) [| A(t) + (g)(t)) (||x|| + F()
Piata (y) Ilslldyyy 2
2
Σ
g())3
Using the Cauchy-Schwarz inequality, have
d
(1e Paaaa (y) (y))ys
2 l lédy) < Σ
(J Pamu (y) (oddy) ( ( (yla 1ual
44

---

Published as a conference paper at ICLR 2025
- -( Paata ( (y) Jody
(1
Pdanaa Py)na())lyy// ooly
(fee Pdinta (y)n
S R²
where the last inequality holds due Assumption 5.1
Therefore, have
i
/ Pdata (y) (y)yddy
0t 3d
S 2R 0(0))(01) (IXII ((()RE) + (0) J(OR))
(1er Pautaa (y Joddy
2
g(t)3
(H.27)
Moreover, have:
i
)
=2 )|= =
/ Pdata (y) ①
Ot / Pama vody
Paam (y) (ody)
Ot
[/c Re
Passaa (y)dddy
Puataa (y)
dy
Ot
2 ((t)))((t) (Ix + F(t) + (g)(t)) (||x + )())
(1
2
Pdata (y)
(H.28)
g())3
Ra
where the last inequality holds due H.18). Substituting (H.27 and H.28 Assumption 5.1. we have:
H.26 and using
(()))5((()) (Ixx R f(()R²) + (0((t) (Ixx {())R² ?
Kgl < 4R²
(H.29)
9(1)3
Combining (H.23).
(.224.
H.25
and H.29).
we have the following inequality:
0 Ot
2d
lg((t)
+ 4f(() ['(()((()) - 2f(())((() R²
tr 2 log (t, x))
g(t}3
g())
+ 6R f(t)²
((t))
([|x R f(t) + (t)( ([|x)) f(t)
(H.30)
9(1
g(1)3
This completes the proof of Lemma H.2
H.2 PROOF OF LEMMAS F.3
AND
F.4
Proof of Lemma F.3 Since X, e Xo (O - ) we have f(() = e and g(()
vI -e- - in this case. When
we know that f(t +(), (t) @(1), g(t) O(VI).
g'(t) = 0 1.) Hence by Lemma H.2
we have:
① Ir (V2 log g(1, ((1, x)))
d
R
1
R)R 3/4 ([| + R2)
R22
Ø6
12
d
R2
RD
+ +R)
([([[ + R)²
12
13
13
11
(
d
R2
[[xx]
+ +R)2
1 1.
12
Here in (i) we use 2 Rd
We next consider the case 1: Denote ( by a, then we know that f(t) O()), ((( O(a). g(t) O g((t) O(a²). By
<
R)]
Lemma H.2 we have:
tr (V (oglog((1,)))))
sddd R²a² + R² [a[[|/]| + aR)R+ a² + +R)))
0t
45

---

Published as a conference paper at ICLR 2025
(0)
I a'd R + RRa ([|x] R)R+ (1xx + R))
Sadda + (1 + Rt)
(00)
S d'd d2 RD | + R)²
<d R2 x|| + R)3 12 1.
Here (ii) is due to < and (iii) is because 2R - $14 1 R - d ( + R)2. This proves the first inequality of Lemma F3
Same as above, When &
1.
we know that f(t) ( (t) 0(1,, g(t) O(vE), g(()) =
0 ( ). Hence by Lemma
H.2 we have:
V log ((xx))
x
R
(||x| + R)
(||x| + R)3
R
82
8
t
x R
(||x + R)²R2
(||x + R)
t< 1.
1
12
13
We next consider the case 2 1. Denote e-f by a, then we know that f(1) = O(a), (((1 = @(a).
g(() ((1), ((a²). By, Lemma
H.2 have:
@ Ot
V log ((tx)())))
- aa² + R a[|| + aR) R² + (/xx[| aR)² aR
$ a[[|x|| + R) + a[||x|| + R)² + a|||=|| + R)3
$ (||x| + R) + (||x + R)² R² + (||x + R)3, t21.
Here (i) is due a = = < given 2 1. This completes the proof of Lemma
F3
Proof of Lemma F.4 Because X, = e Xo + N (o, (1 - e 21 La). we have f(t) = e and g(t) =
vI- e- in this case. When < 1, we know that
f(t) @(), (t) O g(t) =(V)),
Then by Lemma H.T we can easily show the three inequalities are valid when <
Moreover, when 1 1, we have
f(t) = ), (t) ((e-)))((t) = O(1., = 0(e-2)).
Then by Lemma H.I we can easily show the three inequalities are valid when & 2 This completes
the proof of Lemmal F44
H.3
PROOF OF LEMMAS G.3
AND G.4
Proof of Lemma G.3 Since X, X N (0, ViLa), we have f(t) and g(t) = Vt in this case.
Hence (t) = 0 and g' g'(1) 0 +z ). By Lemma H.2 we have:
i v log xx)
| +
(|x|| + R)3
ot
t2
t
θ 0t
d
R²
R2 [||x|| + R2)²
Ir (v2 log ((1, ((,x))))
24
12
13
2A
R2
(2
+ RT).
12
11
here (i) is due to 2 pr
< F
+
n² pr.
This completes our proof.
Proof of Lemma C.4 Since X; Xo+N Vit.), we have f(t) and g(t) = v in this case.
Hence (t) = 0 and
(t) e (.))). Substituting into Lemma H.I completes our proof. O
46

---

Published as a conference paper at ICLR 2025
1 PROOF OF REMAINING LEMMAS IN SECTIONS AND G
I.1 PROOF OF LEMMAS F.1 AND F.2
We first present the following two technical lemmas.
Lemma I.1. Let Y and (z) be defined in F22) and
E33
Then for any 0
K A
11
t [lx 1x+1] and Rd, we have:
Pr,
(F (x))
1 - c 2(T-()
+
1-2/d
a²-1
d/2
2(a--1)
(x))
1 e- 2-((T-1)
where a = el and g(x) = aPp''(x)
(x) -x
Lemma I.2. Suppose Y; e (T- Xdata
- (1----1--))) and p(1 p(Il < R) 1.
Then for m }). we have:
EC S S
Using these lemmas, we can start the proof of Lemma
Proof of Lemma F. To start with, we have the following lemma about the ratio: Firstly, using the expression of the interpolation operator
we have
Fx.(x) e t La - 1) V)
Lxs x).
The operator norm of VF((x) can be bounded by
[VF((x)]2 S <e'- + (c-=th 1)
=a (a 1)b
Using the fact that |A| S |A|l", we know that:
VF. (x))
S aa + (a - 1)
< (1 + 2n + 21)T.) a
S
Here the last inequality is because 77
S 2(L+1)d'
With our time schedule, by (2.)). we have
4- S k+1 12 < 17 min (1, - 1}.
We consider two cases when T - and T - - 1. First, when T 2 1, we have fx S 17. Therefore, we have
e ()- + a²-1 d/2
2(e-1())
6/2
1+2/d
1 1+ 1 c 2(T-1)
1 - c- 2(T-Tr)
27 d/2
1 1 c-2
Since e" -1 2x when x S 1 and 1 - e-
>
1
2,
when S 1/(8d), we have:
(1+ 2
d/2
1 d/2
<
1
Ve.
Secondly, when T' - tt < 1, since
- e
2z >
x 4 and ex - 1 < 2x when 0 < < 1, when
1 I 1/16d, we know that:
2(T + 2-1
d/2
1 + 2(1-(s) _
d/2
1+2/d
1 --T
T rー
1 + 4(t - tx)
d/2
T -
47

---

Published as a conference paper at ICLR 2025
(1 16n) a/2) S Ve.
Combining the two cases, we can summarize that when S 1/ 16d, we have
1 - 2(T-)) + a²-1 1+2/d
d/2
5 Vē.
1- 1- 2(T-1)
Since g(x) = aF) - x, we have:
g(x) = .aF; (x)
aFF (x) oF;" 1 (x) (a 1) - Fx (x))
(x))
-(a 1)80(T (T 1k, Fr- (x)).
Hence,
(1 (1/2))l((()]1 (1+ d/2) 1)|[so [T Lx, Fr (x))
2(a² 1)
2(a + 1)
Using the assumption on and inequality F.4 we have that:
(1 + da - (T Lx. F. (x)
(1+ (/2) - (4) F, [xx)] + tc²
+1a+1)
2(a 11)
(1 + (a2))(( (2||| + L+
2( 11)
+ d/2)(a-1)
2(a 11
d/2)(a 1)
AL |x|1 + (1 + d/22({a 1)
< a a+1
(L c)²
a a+1
(t tw) (1+ d/2)44 |x| + (t tw) (1 + dL2)(L + cc²
where the 1st inequality holds due to and 15 2(t t&)). Thus, we know that:
Py, (x) (..()..))))
S ((t-1))o211x (2tc:olllx11((000-00110 (2+
This proves the first part of Lemma F.I Next prove the second part, we know that:
Jaa (
px((x) (pr)))))1))
2
(x) $ / cll cl--Wa)lll(PP(--eee (x)dx
(t
1.
where the (i) holds due to Lemma 1.2 since our f - t S 17 S min
cid: 20,
And (Mi) holds due to
1 14 S S min
og Rog }
By Lemma I.I we know that:
(x)
e VF. (x))
11-e-2(T- +
I+2/d
2
d/2
-11
(x)
2-T
I 1.
here = e- and g(x) aF, (x) X. This completes the proof of Lemma
F.I
Next, we begin our proof of Lemma
48

---

Published as a conference paper at ICLR 2025
Proof of Lemmal F.2
To start with, using Lemma F.I we know that:
Py, Py, (x)
S e((-(x)oilIx ( (10-
where o₁ = (1 + d/2)4L² and C₂ (1 d/2)(L+
Thus, we have:
Vq(T' (,3)
pr, (x)
PF(( (x)dx= =
V Vlogg -t,,x
pr. (xx pr. (x)dx
PM.(), (x)
12
/
Vlogo(T - x2
e (1()))l)1(+00)) to2) PY, (x)dx
x 12Q V -t, 2
where the last inequality holds due to v S Next, we select a constant M 2R. We have the following inequality:
Bi V (T - t, v.) 2 [
Fe V log q(T 1, |1 enasl1 |1 + EeQ
V log Q(T 1, Yo)l| 7ell
BQl V log q(T t, x.) + EeQ || V/VVogg (TT - Y1)
2 nec YIl? 11x111MM
1.
where the last inequality holds due to 17 <
1
For 4, using Lemma K.2 we have:
TR
a 1/
11 = EQ
V
((T - Y.)1
d 1
((T-
d
(L.I)
min - [in(T-(,) 1,
where (i) holds because 1 e > 2 when x < 1 and 1 - e > 1/2 when 2 1.
For I2, using Lemma H.l with ((1) - ()) g(() v1 -(- have:
|||| + R
q -T, 5 1 2(((T-t)
| + R
or
here we denote -2(T-)- by or- Therefore, we have:
12 s Eo( Yill + R
n mrel|xxlll |XIl
(I.2)
or-t
Let = (T--)) we have Y, = X7 =ax000X0 + v1 - a1-a2Z. Thus, ||||| 5 R + v1
Hence, (.2) becomes
1 < (or EQ (2R+ + /1 x²|12]]) 2
)²
1 >
M- R
Vi-o?
(or- Be
RR2 +11 a a²)l| 2010 2|
1212
M-n V1-az
(1.3)
where the last inequality holds due (a+ b) < 2a + 262 and n - 1/c, Let A 2 (1-a²). Then we have
R²
e- 1n12 Al2ll
I <
M-R dz vI-w?
(or-1)
Rd
(V2
xxl
49

---

Published as a conference paper at ICLR 2025
2
1xj2
||z| Tel/eeAlle]] 1 1e12 M-n dz,
(1.4)
OT
JRd V2T
v1-02
here we use that 1 a² =
2-((1-1) -t- Let 22 (x) xxp(-||x|P/2o2)//
When À smaller then 2. we have @d||e|?? (1)2) = 1 (z) 1-2x ) Therefore, we know that:
K
(2) ||||| > v1 a²
M-R dz
2X
Rd
TP
| 2
M- - R
Z' N N(1,,
1 - 22 Ta)
2A
VI a
1
vI - 2X(M - R)
IP
|| 2
z~ L()
2X
v1 - a²
1 a²
(1- 2M - It)2 1=12
1 1 2X
3+1 or-[d
2X
(M- - (
or-id
(I.5)
R²
where (i) holds due to the Markov's ' inequality. Since S 2nj S 2oT cad we have À = 2m/11 (1 a²)= s 1/(d+ 2). This implies (1/(1 2x)) < e, thus (ii) holds. Moreover, for K2 we have:
K₂
Rd T (z)1|2| |||| M-- dz v1 az
27
Ez b 1.) 12
1 2X
2
1-2X 2X
$ d.
(1.6)
The last inequality holds due to (1/(1 21))d/2 s e when X s 1/(d + 2). Substituting (L. and (I6 into (.2) we have
d
12 x oT
d
(I.7)
min ITinTT-Ou 17--,,1 1}
Combining I.1 and 1.7
we have:
Vg(T
x,x)
d
PF. (x)dx 2 min t, 1)
n.
Pr. (x)
which completes the proof of Lemma F2
I.2 PROOF OF LEMMAS G.1 AND G.2
We first present three technical lemmas.
Lemma L.3. Let Y, and Ft (z) be defined in G.2 and
G.5
For any k t E [1x, (441] and x E Rdd,
we have:
PY, (x)
T - 1+2/d
d/2
(144/2111)) 100--16
PF. (x) e VF.
e
21c (2)
T
Lemma L.4. Suppose Y. Xo + N(O, (T - 1) and p(1 < R) = 1. Then for <
min
1) R²
1 ). we have:
Bed Y.I
50

---

Published as a conference paper at ICLR 2025
Lemma L5. Recall that X, is our forward process defined in G.1 law, under Assumption 5.1 we have:
and we use ((t,x) to denote its
d 7
E V (l, x,)
Proof of lemma G.1
By Lemma
I.3
we know that:
PY, x
(F
a/2/1 (x) xii 2(1-(()
T - t +
T+2/d
1-
d/2
( (x)
T -
2
Firstly, using the expression of the interpolation operator G.5 we have
(x) = Ia + (t Jez Vsg (T tx x).
The operator norm of (x((x) can be bounded by
[V)l(x)) I 1 + (t - tt)eLL
I (t - L.
Using the fact that | S ||||s, we know that:
(F((FT(())) (1+ (t- - tn)
Since: t to S 17 1, when 17 S 1/(Ld), we have
(FF = (1+) S c.
With our time schedule. by (26)) we have
t S tk+1
S mmmin -t.
We have t tx S Therefore, we have
T ++
I-tu T+2/a
d/2
S 1 t - tk
4/2
--
--
(1 1)/2
When v) 7.S. 3. we have:
(1 (1+1) 1/// (1 + 2 y0/2 S Vē.
Then we have:
(1 + d/2) F, (x)
x|
(1+ d/2)(t - tw) 0i2 21|ss (T tu F. (x))11
2(1 (x)
2
(1 + d/2)(t (n)
(LIIFT'(x)) + +c)²
2
(1 + d/2)(t - tx)
(2L| +L+c)2
2
(1 (x)
(1 (1+d (1+d/2)a1² |x|| + +
(1+d/2) d/2)
c)²
Putting together, we know that:
PY, (x)
S Selttteilll
(F.(Y,,) (x)
Moreover, since t - tx Sn S 1/02, we have:
1 Pr, (x)
Pr, (x)
Pr((),,) (x)dx (x)dx=
Pxx, (x)dx
((x)
Ja. (F:(X,, (x)
(t
(1)
$
1
where (i) holds due to Lemma 1.4 (we have (t - tx)c1 S min 4d(7-13 Ra }).
51

---

Published as a conference paper at ICLR 2025
Proof of Lemma G.2 To start with, recall that assume that L 1, easily verify that
satisfies Lemma G.
condition. Using Lemma G.I we know that:
(x, (x)
S < (1
where C1 = (1 d/+2//) and c2 - 12==11 d/2) + tc)². Thus we have:
Vq(T 1,,) 1/
Vloga(T- - xxx) 2 pr, (x)
l
PFo( (x)dx
PF. (x) pr, (x)dx
14 Pri(ys,))
22
/
Vlogo(T - x,x)
"erles ||x|] (xlcs1/x1(++taa) to2)
py, (x)dx
52
V v.) (eviil11)
where the last incquality holds due to 7 S following inequality:
1. Next, we select a constant M 2R. We have the
EQ V IT t. Yr) ( /
FQ V log ( 1, k1) 2 nesll
V log ((7 - ?
|
+ E.g
E2l V log ((T - t, Yo)l| + EQQ
V log q(T - t, Yi)
2 enelllelll 1
where the last inequality holds due to 17 S
I RT.
For 41 using Lemma I.5 we have:
V log tt 1))1
(L.8)
T-
For 12, using Lemma H.l with f(t) = 1, g(t) = VT - t, we have:
V - 1,,) 1 ||x|| + R
T-t
Therefore, we have:
12 S Eg/ [ | + Ri
-- 2 n 1|
1|181112
(1.9)
We have Y, XT-r Xo VT ttz Thus, | S /T Z| Hence, 1.9
becomes
1 <
1
M-R EQ (2R + VT t Z11 (R+ 1
(T' 1-tj" t)2
$ (T' - f)2
1
EQ (R² -T- +)|| 22c
2 M-B
(I.10)
where the last inequality holds due to (a + b)2 < 2a + 2b2. and 17 S Then we have
1. Let A = 2ncc( - t).
R²
1
I 2
T -
t)2
Rdd (V2 )
||l||
dz
Ki
+
T
2
2d
/ 12m
d
2
|z
A|
12
M-- dz, -
(I.11)
K2
52

---

Published as a conference paper at ICLR 2025
Let (x) exp(-||x]] /2/l///22///// (2 When X smaller then 2/2 we have elllll Φ1(z) Φ 1x (2) 1-21 I.. Therefore, we know that:
K1
[z) ||2 M dz
1 22
M R '' N(1, 1- 22 lt) 1
1
1
2X
VT -
v1 2X( R)
| Σ
Z~ (~.N(..)))
21
VT
1
T-
(1- RT2 2
1- 2X
++1 (T - ((dd
1 2X
(M - R)2
2A3
(T l)d
(1.12)
R
where (1) holds due to the Markov's inequality. (tt) holds because 17 S (T)))) thus À
<
1/(d+ + 2), then we know that (1/(1 -22))a/22 < Moreover, for K2 we have:
K₂ = 1 - 22
(z)|z]|²1
|| M-R dz
Rd
< 1 1-221 22
E E 122x 1'12
1
4
d
2X
2X
S d.
(I.13)
The last inequality holds due to (1/(1 - 21)) < e. Substituting (I.12) and (1.13) into have
we
d
12 < T-
(I.14)
Combining
and 1.14 we have:
Vq - x,x)
P (x)dxx X d T-
PF.(Y.,) (x)
which completes the proof of Lemma G.2
J PROOF OF LEMMAS IN SECTION
Proof of Lemma I.I Using the Jacobian transformation of probability densities, we have
Pr. Pr. (x) PY,, (F (x)) (FF (x))
(J.1)
Moreover, the forward process indicates Y., (t- Zet., where Z,,, N(0, 1 (2((1(2)) is independent of Y.. Using the Jacobian transformation of probability densities, have
PY,, (FT (x)) Pxite 1 kein (c' F, (x)) e
- [
o(t.)))200---1( ge
-y)dy.
where 2,2 (x) = cxp(-1[x|]//207) ( the probability density function of Gaussian dis- tribution with variance o². The last equality holds due the formula for the sum of independent
53

---

Published as a conference paper at ICLR 2025
variables. For simplicity, denote ct- and we ignore the dependency of t and ta when it will not cause any confusion. Then we have
{ Q( -t,))))=--(aaF yy
PYa, (x))
(J.2)
R
Since aF' (x) - y = x - (at, (x) - xx), with the shorthand notation g(x) aFF aFFT(xx) we have
X,
Φa-- (ali - yy = - y + g(x))
Ixx 1x(x)1
2(a²-1)
(a² - 1))
1lx Ilx-yl1 (p(x) 22(xxyy 2(a"-1)
(2T(a² - 11)aa/2 e
(1/2/11x vI (114/1)
2 (2 (a² - 1))d/2
21a
044/2
(x y)
-1a--1)
(J.3)
(1 2/d)d/2
112/d
where the first inequality holds due to the Young's 's inequality. In the last equality, we use the defini- tion of (-) with 0² = (a² 1) 2/d). Combining U.I. (J22) and (J.3), have
PP.(),,) 2 (FF (x)) a. / g(t,y) (1+ 2//////2
(+422))0001
1 y)e 14272
2t 1) dy
(+1/3))00011
a
(F, (x)) Y, x2 (x)
2(a-11)
where use (1+ 2/d)/d)//² the fact that (F,, (x)) (F.
(x)) (x))
and the formula
for the distribution of the sum of independent random variables. Then we have
(x)
VF. (F (x))
0+d/3)/0001
Py, (x)
2
(F. (x)
P Y, N(, 22
Since Y, = e-(-t) data + NNO, 1 - e-- 2(T by Lemma K.3
we know that
e 2(T-t)
2-1
(x)
I+2/a e- 2(7- ()
d/2
Py,+N(U (1 (x)
Combining the above two inequalities and we can complete the proof of Lemma
Proof of lemma [.2 Since (T data + (0, (1-1011110 e- 2() denote e- (('-t) by and use Z to represent standard normal distribution, we know that:
Ee Alle -
Ee Xlu
Ee A 12 Be All 2
where (1) holds due to the Cauchy-Sehwarte inequality. (it) holds due to the independency of X X and Z.
Since p(1 X 1 < R) = 1, then Ae Xamil 12 < Att For À < f we know that:
Ee Z
/ (21) - llxil
dx
| 2 1-21 - - 2 2x dx
f11 (2
54

---

Published as a conference paper at ICLR 2025
(1 - 2
When X < 2. (1-2A)
2 < _ thus we have:
Be S -AR
Proof of Lemma
1.3 Using the Jacobian transformation of probability densities, we have
P ,Y,, (Fr (x)
(J.4)
Moreover, the forward process indicates Yts Yt + Zr, where Ze,ta N(, tx) is indepen- dent of Y, Using the Jacobian transformation of probability densities, we have
PY,, (F. = PY,+2 (F. (x))
| Jm g(T - (1,))rr 24 (x)
=
where Φa2 (x) = exp( cxp(-I|x 12/=)///22)// / the probability density function of Gaussian dis-
tribution with variance o². The last equality holds due the formula for the sum of independent variables. We have
De-ta (F- y) = - (x y F. (x)
Ix X 1x-xtF- 000-11
(2T((1 41) (a//
w1 11FT (6) wil 12tx x.F x)
20 20111)
(2T(1 1x))
x1 3/- (x)-x13
(2m(t t4 42)2
(1+d/2) 1 xx0) 2(6-(2)
(x
(J.5)
(1 + 2/d/d/2
where the first inequality holds due to the Young's 's inequality. In the last equality, we use the defini- tion of (.) with x² (t 41 2/d.. Combining (I.4) and (.5). have
P.,(Y,,)) (x) 2 ((FF
y)e / ((T - -(T-1 1,y) (1+ (1-2/d) 2/d)a/22 I127a
11221
(0)
x
dy
1
(1-d/211, (x0-11
(x)) Py, N 1- (x) e
(c)
c V.F (x))
where we use (1+ 2/d)//2//2 e, the fact that (x) for the distribution of the sum of independent random variables. Then we have
(F (x)) and the formula
(x)
(11//2111 x
Py (x)
(F
(
PF(Y, (x)
Py, PY,+Noo, (x)
Since Y. Xo NO, t), by Lemma K.3
we know that
t- tu
Py, (x)
1+2/a --
d/2
P PY, f
(x)
Combining the above two inequalities and we can complete the proof of Lemmal
Proof of lemma I.4 Since Y, Xo + VT - fZ, we know that:
Be < Ee 2A1lxol1'' ()
2AR 2A(T-1))121
55

---

Published as a conference paper at ICLR 2025
2A(T 2
here (i) because we assumed À I/R². Moreover, we have S (ग(- this implies 2X(T-
- thus have
2 22(TT (2T) $
dx
RR
3z (2T 1 4X(T -1) -1 41(T-1)) -1)) 4
1
4x12
dx
(1 - 1))
$
This completes the proof of Lemma 1.4
Proof of Lemma
1.5
By tweedic's 's formula we know that:
- {xx, Egon Xo.)
1
log q(t, X1)
B ge (x,))
12
E8
X; E E[Xo|Xal]] x
12
1
X 2EX; [X, + E
Since X, = + ViZ, we have E| X 1| = E E + td. Here the second order momentum is finite because our bounded support assumption. Moreover, we know that:
EX, E X X EX,, Xo = E (Xo + Viz) Xo X
We next consider the trace of the covariance matrix of X, given X,:
tr Cov son1 (xxx. E X X. [ x.]11
Thus know that:
E E x
E Cov ( (11x, x,) (Xo) X
E Xol1².
Putting together, know that:
< d
E V log (t, X,)
t
K
AUXILIARY LEMMAS
Theorem K.1 (Reynolds Transport Theorem Leal 2007). For a function F(t, x) IR x Rd + IR that continuously differentiable with respect both x and the following equality holds:
( F(1,x) = / 2 F(t, xx
F(I.x) vtt, n(t ds,
Øt .n
Øt
Jon.
where n is the outward-pointing unit normal vector, v is the velocity of the area element, and d$ is area element.
Lemma K.2 (Lemma 6 in Benton et al. (024). Let X, be the OU forward process defined in When Xo has finite second moments, we have:
d
E V qt,, X1II
e-
56

---

Published as a conference paper at ICLR 2025
Proof of Lemma K.2 This lemma is the same Lemma 6 in Benton al. 2024 We provide proof for the completeness of our paper.
Denote -e e -2t by or. By Tweedie's 's formula we know that:
v log g(t, Xt) 1 d2
(
X: te e (-
Taking the expectation of the square, have
E / log 9r (x, = o "Elee Xi] X,I
= o
20 'EX, [x X +
Since X, = = +
Z we have E X 2 = e E Xol + ozd. Recall that we assume that X has finite second moments. Moreover, know that:
EX, E X X EX, X₂ = E (e (ee'X Xo + Vo, Xo = -
We next consider the trace of the covariance matrix of Xu given X,:
tr Cov 2011 (xx.) = E XoX X | x
Thus we know that:
E | x
Xoo12 |||
E tr Cov (1 xx) (Xo))
E Xol1².
Putting together, we know that:
d
B V log xt) at
Lemma K.3. For any data distribution Pdata and positive parameters and h. Let X, Zs. Zn be three independent random variables in IRd satisfying ~ Pdatas Z8 ~ N(0, 5) and Zn ~ N(0, h). Then have:
(x) Px+2++2, (x)
h d/2
where we use py (x) to denote the probability density function of random variable Y.
lx12 20 which is the probability density fune-
Proof of Lemma K.33 We define (x) (2n02)a/EE
tion of normal distribution (0,, o²Ix).
First, we provide an upper bound of 65(x y-/ 10///(1xxxx1 y))
(x - y) | + h d/2
(x-xi2 (x--))
8 + h 4/2
2(0-1)
(K.1)
Φ8+4 - y)
8
Using the independence property, we have
(x) [ Pdota (y) 0x(x - y)
Jan
[ Py 884h(xx y) dy.
px+ Px+Z+t2,
Then we have
Je Pdlatu (y) -yddy
(1e Paitta (csts))c Φ5(x - y)
dy
Ø541 (x -y)
(066()))6+66x1--1 y)dy 8 h d/2
which completes the proof of Lemma K.3
57


Reviews:
[{"review_id": "6xlACtdWpE", "reviewer": "Reviewer_HJsC", "summary": "The paper addresses a previously unexplored question: bounding the total variation between the true distribution and the distribution generated by an ODE sampler of a diffusion model. The contributions are twofold: 1) Identifying that standard assumptions are insufficient for constructing tight bounds; 2) Laying out an intuitive framework that builds the bound under sufficiently strong assumptions, accounting for both approximation errors in learning and the ODE solver.", "questions": "The dependency on the time schedule appears to manifest solely through Equation (3.6) in various Lemma in Appendix F and G. While it may be due to my limited understanding of prior work, are there other schedules that could fulfil the condition? Can a similarly concise bound be proven with a different time schedule?\n\nThe construction of the counterexample is pivotal in motivating the paper. Would it be beneficial to discuss how one might construct such a counterexample within the main text?\n\n[1] Albergo, M. S., Boffi, N. M., & Vanden-Eijnden, E. (2023, March 15). Stochastic Interpolants: A Unifying Framework for Flows and Diffusions. arXiv.org. https://arxiv.org/abs/2303.08797", "limitations": "", "rating": 8, "confidence": 3, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "The paper tackles a fundamental question regarding ODE solvers for probability flow ODE, starting with a strong motivation by cleverly constructing a counterexample that demonstrates the insufficiency of assumptions on score estimation error to prove a tight bound on total variation. Adding an extra divergence approximation error, the proposed analytical direction is clear and intuitive, resulting in a three-step procedure:\n\n1. Bound the total variation between two ODEs.\n2. Define an extended ODE that aligns with the discretization scheme used in an ODE solver.\n3. Bound the difference between the extended ODE and the true ODE.\n\nAdditionally, the paper showcases the instantiation of this framework through VP-SDE with an exponential integrator and VE-SDE with a DDIM integrator, providing a very natural path to follow in defining the extended ODE for other diffusion models or samplers.", "weaknesses": "While the paper is well-written, it would benefit from a discussion of potential applications for this bound. The main extra assumption required is a bound on divergence, which results in an additive term in the resulting bound. However, divergence is not typically controlled during training, so exploring whether this can be managed would be interesting.\n\nMinor Remark:\n\nI am uncertain about the claim that Lemma 4.2 serves as an ODE version of the Girsanov Theorem, as the Girsanov Theorem concerns the Radon-Nikodym derivative. The use of a time derivative of the divergence of the marginal distributions of an ODE is also common (as seen in the proof of Lemma 2.21 in [1]). Although such a bound may not be practical within the context of this paper, it might be worthwhile to cite or discuss these bounds.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "HBA6YxQzJ0", "reviewer": "Reviewer_usTt", "summary": "This paper investigates score-based diffusion models that utilize deterministic samplers. It introduces a unified framework for convergence analysis by leveraging divergence estimation of the score function. Additionally, the analysis explores the convergence properties of variance-preserving SDEs employing the exponential integrator scheme, along with variance-exploding SDEs in conjunction with the DDIM sampler.", "questions": "1. **Typos and Missing Definitions**  \n   a. Line 176: What is the definition of $q_{\\delta}$, and how does the distance between $q_{\\delta}$ and $q_0$ depend on $\\delta$?  \n   b. Assumption 4.1: What is the definition of $Q$?  \n   c. Theorem 4.4: What is the definition of $X_{\\delta}$, and what is the total variation distance between $X_{\\delta}$ and $q_0$?  \n   d. Equation (5.2): The definition of the step size $\\eta_k$ is missing.  \n   e. Line 334: Should $\\eta$ be $\\eta_k$?  \n   f. Line 349:  Should $F$ be $F_t$?\n\n2. **Corollary 6.7**: The paper states, \"assuming sufficiently small score estimation error and divergence error.\" Could you quantify how small $\\epsilon_{\\text{score}}$ and $\\epsilon_{\\text{divergence}}$ should be in relation to the target error level $\\epsilon$?\n\n3. **Effectiveness of the Framework**: In Section 3, the author discusses results from Section 6.1, suggesting that the dimension dependence is worse than that of the state of the art. Does this arise from limitations within the proposed framework, weaker assumptions, or other factors? Furthermore, the results in Section 6.2 indicate a slow decay for the variance-exploding SDE. Is this limitation due to the proof technique, or does it stem from other reasons? Given these considerations, despite the generality of the proposed framework, could the author provide additional evidence to demonstrate its effectiveness? Specifically, can this framework achieve a sharp convergence rate or match the state-of-the-art convergence rates?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper is clearly written and presents a novel unified convergence analysis of ODE flows that integrates divergence estimation of the score function. Furthermore, this framework provides fresh insights into the convergence analysis of variance-preserving forward processes employing the exponential integrator scheme, as well as variance-exploding forward processes utilizing the DDIM sampler.", "weaknesses": "There are a few typos and missing definitions. Additionally, regarding the assumption of divergence—particularly the assumption of the divergence estimation error—are there any studies that provide estimators for divergence that support the validity of this assumption? How can practitioners verify assumptions like those in Section 6.3 in real-world applications? \n\nFurther details can be found in the Questions section.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "DkgWooEfsW", "reviewer": "Reviewer_Pg4X", "summary": "This paper consider the convergence of score-based diffusion with deterministic sampler. The paper starts with a counter-example (using 1d OU process) to show that the problem of convergence may be subtle. The paper then shows a $O(d^2/\\varepsilon)$ bound for the convergence, yet another polynomial complexity.", "questions": "See weakness.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "The paper provides a systematic/unified study of the convergence of the score-based diffusion models using deterministic or ODE samplers (mostly, EI). The paper is generally well written, and I enjoyed reading it.", "weaknesses": "There are several weaknesses of this paper:\n\n1. There have been now a handful of papers, e.g., Chen et al, Li et al., and Huang et al,... providing the polynomial convergence of the diffusion models via ODE/deterministic sampling. It is not too clear how the results in this paper are compared to the existing literature. For presentation, the authors may provide a table to showcase.\n\n2. The authors claimed \"polynomial iteration complexity for the VE forward process with DDIM numerical scheme, both of which have been relatively underexplored in the literature. It is known that VE and VP are connected by a reparametrization. In most of cases, it suffices to derive the convergence for VP or VE. This viewpoint is highlighted in the important paper of https://arxiv.org/abs/2206.00364, which the authors fail to cite.\n\n3. The main theorem, Theorem 5.3, basically decomposes the error into three pieces: (1) initialization error, (2) score matching error and (3) discretization error. This idea is not too novel (and of course, the analysis may be involved). The theoretical insight from this paper is not significant, and the contribution of the paper is purely technical.\n\n4. In Theorem 5.3, the authors include a term involving the second order derivatives, which echoes Assumption 6.3. I find this assumption is somewhat too strong.\n\n5. Out of curiosity, I wonder if the authors expect the rate derived in the paper is optimal (especially regarding the dimension $d$).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "lRmteImelU", "reviewer": "Reviewer_EFQA", "summary": "The paper studies the error of generative models based on deterministic samplers, that is when the backward process is represented by an ODE. Error bounds are presented both for the continuous time and for the discretisation schemes, giving a general result that holds for a wide class of schemes. Several instances of samplers are then considered, giving error bounds for each case.", "questions": "In lines 756-761, why is the TV distance equal to the integral over the entire real line? Can the authors clarify how this is obtained, and also why the absolute value is inside the integral? It cannot be that one of the two densities is larger than the other over the entire domain, else it would not integrate to 1. For this reason I am doubtful of the proof.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "The results in the paper are interesting and cover a good range of algorithms that are of practical interest in generative modelling. The results complement well with the existing literature. \n\nThe main idea, Lemma 4.2, is to bound the derivative of the total variation distance of the laws of the true and approximate backward processes, giving an error that depends on the score and its divergence. As far as I know this is original in this literature.\n\nThe presentation is overall clear enough and the goal and results are simple to understand.", "weaknesses": "The paper presents several times inconsistent notation, e.g. lines 134-137, 151-154, 349-350, repeated equations (e.g. lines 204 and 206).\n\nSome aspects should be explained better in the paper, for instance: why is Assumption 6.1 used in the application the specific examples of samplers? Or also, can the authors give a clearer idea of what F can be in lines 345-350 for specific samplers? As it is now, the reader has to trust that this function can be found easily, while it would be much better to have one or two examples. \n\nSeveral times I had the impression that sentences could be written better, e.g. the statement of 6.5 just to name one, or several times the authors write \"Lipschiz\" instead of Lipschitz.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)