Leveraging Drift to Improve Sample Complexity of Variance Exploding Diffusion Models
Authors: Ruofeng Yang
Zhijie Wang
Bo Jiang
Shuai Li
Venue: NeurIPS 2024 poster (2024)
Keywords: Variance exploding diffusion models
Convergence guarantee
Manifold hypothesis
Primary area: generative_models

Abstract: Variance exploding (VE) based diffusion models, an important class of diffusion models, have shown state-of-the-art (SOTA) performance. However, only a few theoretical works analyze VE-based models, and those works suffer from a worse forward convergence rate $1/\text{poly}(T)$ than the $\exp{(-T)}$ of variance preserving (VP) based models, where $T$ is the forward diffusion time and the rate measures the distance between forward marginal distribution $q_T$ and pure Gaussian noise. The slow rate is due to the Brownian Motion without a drift term. In this work, we design a new drifted VESDE forward process, which allows a faster $\exp{(-T)}$ forward convergence rate. With this process, we achieve the first efficient polynomial sample complexity for a series of VE-based models with reverse SDE under the manifold hypothesis. Furthermore, unlike previous works, we allow the diffusion coefficient to be unbounded instead of a constant, which is closer to the SOTA models. Besides the reverse SDE, the other common reverse process is the probability flow ODE  (PFODE) process, which is deterministic and enjoys faster sample speed. To deepen the understanding of VE-based models, we consider a more general setting considering reverse SDE and PFODE simultaneously, propose a unified tangent-based analysis framework, and prove the first quantitative convergence guarantee for SOTA VE-based models with reverse PFODE.
We also show that the drifted VESDE can balance different error terms and improve generated samples without training through synthetic and real-world experiments.

Leveraging Drift to Improve Sample Complexity of Variance Exploding Diffusion Models
Ruofeng Yang
Zhijie Wang John Hoperoft Center for Computer Science Shanghai Jiao Tong University
John Hopcroft Center for Computer Science Shanghai Jiao Tong University wanshuiyingsj tu. edu. cn
w edu. cn
Bo Jiang
Shuai Li'
John Hopcroft Center for Computer Science Shanghai Jiao Tong University bj ajiang0sjtu. edu. en
John Hopcroft Center for Computer Science Shanghai Jiao Tong University
shuaili80sjtur het.jtt..uu...oo cn
Abstract
Variance exploding (VE) based diffusion models, an important class of diffusion models, have shown state-of-the- (SOTA) performance. However, only few theoretical works analyze based models, and those works suffer from a worse forward convergence rate 1 (T' than the exp (-T) of variance preserving (VP) based models, where the distance between forward marginal distribution @r and pure Gaussian noise. The slow rate is due to the Brownian Motion without a drift term. In this work, we design new drifted VESDE forward process, which allows faster exp (-T) for- ward convergence rate. With this process, we achieve the first efficient polynomial sample complexity for a series of VE-based models with reverse SDE under the manifold hypothesis. Furthermore, unlike previous works, we allow the diffusion coefficient be unbounded instead of constant, which is closer the SOTA mod- els. Besides the reverse SDE, the other common reverse process is the probability flow ODE (PFODE) process, which is deterministic and enjoys faster sample speed. To deepen the understanding of VE-based models, we consider more general setting considering reverse SDE and PFODE simultaneously, propose a unified tangent-based analysis framework, and prove the first quantitative convergence guarantee for SOTA VEE-based models with reverse PFODE. We also show that the drifted VESDE can balance different eтror terms and improve generated samples without training through synthetic and real-world experiments.
the forward diffusion time and the rate measures
Introduction
Recently, diffusion modeling has shown impressive performance in different areas [Ho et al. 2022 Rombach al. 2022 Esser al. 2024 Li al. 2024). Diffusion models consist of two processes: the forward and reverse process. The forward process gradually converts data qo to Gaussian noise, which can be described by an intermediate marginal distribution sequence (4. The reverse process sequentially predicts noise and removes it from data to generate samples.
There are common forward processes: (1) Variance preserving (VP) SDE and (2) variance exploding (VE) SDE. The VPSDE corresponds to an Ornstein-Uhleneen process, and the stationary distribution (00, I). The VESDE has exploding variance in the forward process. In earlier
"Corresponding author
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

times, VP-based models (Ho et al. 2020 Lu et al., 2022 provide an important boost for developing diffusion models. Recently, VE-based models have shown the ability to generate data distribution supported on low-dimensional manifolds (Song and Ermon 2019 2020]. Since image and text datasets typically exhibit a low-dimensional manifold nature [Pope et al., 2021 Tang and Yang. 2024). VE-based models have achieved great performance in image generation. one-step generation, and reinforcement learning [Teng et al. 2023, Song et al. 2023 Ding and Jin 2023]. Furthermore, Karras et al. [2022) unify VP and VESDE and prove that the ODE solution trajectory of a specific VESDE is linear and directly towards the data manifold, which makes the denoise process easy.
After determining a forward SDE, diffusion models reverse it and generate samples by running the corresponding reverse process. Since the reverse drift term contains the gradient of forward logarithmic density V log (a.k.a. score function), we estimate it by using the score matching technique [Vincent 2011]. After that, diffusion models discretize the continuous reverse process and run this discrete process starting from pure Gaussian. There are two widely used reverse processes: reverse SDE (Ho et al. 2020] and probability flow ODE (PFODE) [Song et al.) 2020a]. The reverse SDE usually generates higher quality samples (Kim al. 2022]. The reverse PFODE always has a faster generation speed and is useful in other aspects such as calculating likelihoods [Song et al. 2020b or obtaining one-step generation models [Song et al. 2023]. Hence, these processes are both critical, and providing the guarantee for VE-based models with these processes is necessary.
Recently, many works analyze the convergence guarantee of the VP-based diffusion models under the reverse SDE setting and prove that the VP-based models can sample from the target data distribution
with polynomial complexity [Chen et al. 2023c b Lee et al. 2023 Benton et al., 2023). As the first step of this work, we also analyze ViE-based models under the reverse SDE setting. Different from the VP-based models, only few works consider VE-based models and all of them suffer from slow I/Poly(T) forward convergence rate [Lee et al. 2022 Gao et al. 2023 Gao and Zhu 2024). which is worse than exp(-7) one for VPSDE. A slow forward convergence rate makes a large distance between qr and pure Gaussian noise, which leads to a large reverse beginning error. From the theoretical perspective, this error introduces hardness to balance three error sources, as shown in Section []] From the empirical perspective. [in et al. [2024) show that this error introduces a data information leakage problem, which leads to bad performance. To deal with this problem, De Bortoli et al. 2002 introduce a small drift term to obtain exp (- TV) reverse beginning error. However, they introduce an additional exp (T)) in the discretization error term. Furthermore, their results do not allow unbounded 3g, which is the key point of the optimal solution trajectory and used by the SOTA models [Karras et al. 2022. Song et al. 2023). Therefore, the following question remains open:
Is it possible to design VESDE with faster forward convergence rate than 1 (Tpoly(T) and achieve the polynomial sample complexity when the diffusion coefficient is unbounded?
In this work. for the first time, we propose new drifted VESDE forward process, which enjoys a faster forward convergence rate and allows unbounded coefficients. We first show that the drifted VESDE has similar trends but performs better than the original SOTA VESDE on synthetic data (Section 7. After that, we analyze the sample complexity of drifted VESDE under the realistic manifold hypothesis. The manifold hypothesis means the data go is supported on a lower dimensional compact set M, and much empirical evidence shows that image and text dataset satisfy this hypothesis [effferman et al. [0166] Pope et al. 2021 Tang and Yang [024]. Furthermore, as shown in Section 2 the manifold hypothesis is more realistic than previous data assumptions since allows the blow-up phenomenon of the score function at the end of the reverse process, which matches the empirical observation (Kim et al. 202T]. Under the manifold hypothesis, we prove that the drifted VESDE with suitable larger 3r balances the reverse beginning, discretization, and approximated score errors and achieves the first efficient polynomial sample complexity for VE-based models with reverse SDE.
To better understand VE-based models, we analyze reverse SDE and PFODE simultaneously after obtaining polynomial complexity for reverse SDE. Despite the great performance, a few theoretical works consider reverse PFODE [Chen et al.] b Gao and Zhu 2024), and these works either focus on VPSDE or have strong assumptions. Hence, we propose the tangent- based framework for ViE-asedd models and achieve the first quantitative convergence for the SOTA VE-based models with reverse PFODE. conclusion, we accomplish the following results under the manifold hypothesis:
1.
We propose new drifted VESDE forward process, which allows exp (-T) forward conver- gence guarantee with suitable 3, With this process, we achieve the first polynomial sample complexity for series of VE-based models under the reverse SDE setting.

---

2. When considering the general setting, we propose the tangent- based unified framework and analyze reverse SDE and PFODE simultaneously. Under this framework, we prove the first quantitative guarantee for SOTA VE-based models with reverse PFODE.
3.
We show that the drifted VESDE balances different error terms and improves generated samples without training via synthetic and real-world experiments.
2 Related Work
Before providing current results, we first discuss different assumptions about the data distribution from strong to weak. The strongest assumption the log-concave distribution. While the log-Sobelev inequality (LSI) assumption slightly weaker, does not allow the presence of substantial non- convexity, which is far away from the multi-modal distribution. Recently, some works assume the score function is L-Lipschitz allow the multi-modal distribution. However, this assumption can not explain the blow-up phenomenon of score [Kim et al. 2021]. The last assumption is the manifold hypothesis, which supported by much empirical evidence and allows the blow-up score.
Analyses for VP-based models. For the reverse SDE, Lee et al. 2022] achieve the first polynomial
complexity with strong LSI assumption. Chen al. [202cc remove the LSI assumption, assume the Lipschitz score and achieve polynomial complexity. Bortoli [2022 is the first work to focus on the
sample complexity of diffusion models under the manifold hypothesis, and it is the most relevant work to our unified framework. However, as discussed in Section 6.1] the original tangent-based lemma can deal with reverse PFODE even in VPSDE. We carefully control the tangent process to avoid additional exp (T') by using the exploding variance property of VESDE. Recently, Chen al. [023aa] and [Benton et al. 2023] also remove the Lipschitz score assumption, and Benton et al. [2023] achieve optimal dependence on d. More recently, Conforti et al. [2023] use bounded Fisher
information assumption and replace with a Fisher information term.
For the PFODE, Chen et al. [2023d] propose the first quantitative result with exponential dependence on the Lipschitz constant. Chen et al. (2023h) achieve polynomial complexity by introducing
corrector component to inject suitable noise. More recently, [Li et al.] [2023) remove the additional
corrector. However, their results rely heavily on the very specific Sr, which goes to 0 as 7 + +00.
Since VE-based models have an unbounded 3r, this method is not suitable for our models.
Analyses for VE-based models. When considering VESDE, most works focus constant 3 and reverse SDE. De Bortoli et al. [021] provide the first convergence guarantee with exponential dependence T. Lee et al. [2022] analyze constant diffusion coefficient VESDE and achieve polynomial sample complexity under the LSI assumption. When considering the reverse PFODE, Chen al. (2023d] only consider the discretization error and provide a quantitative convergence guarantee. However, their results introduce additional exp (T) compared to ours (Section 6.1) Recently, Gao et [2023) Gao and Zhu (022)) provide the polynomial results for series of VESDE with reverse SDE and reverse PFODE under the log-concave assumption, respectively.
3 The Drifted Variance Exploding (VE) SDE
Diffusion models usually consist of forward process and a reverse process. The forward process gradually injects noise to convert the data distribution to pure noise. To generate samples, diffusion models reverse the forward process and run the corresponding reverse process.
This section first recalls two previous forward processes: VPSDE and VESDE, Recently, the VE- based models achieve great performance in application [arras et al., 2022 Song et al. 2023 However, unlike the widely analyzed VP-based models [Benton et al. 2023 Chen al. 2023b), J, the
VE-based models suffer from challenges in obtaining an efficient sample complexity due to the slow forward convergence rate. To address this limitation, we introduce new drifted VESDE forward process, which has a faster forward convergence rate, balances different error terms and achieves the first efficient polynomial sample complexity (see Section Finally, we introduce how to reverse this new forward process and obtain an implementable algorithm.
3.1 The VP and VESDE of Diffusion Models
We first introduce the general form of the forward process and then recall two common forward processes, VPSDE and VESDE, adopted in previous works (Ho et al. 2020 Karras et al. 2022 Let
3

---

Qu be the data distribution. Given X E R., the forward process is defined by
- f(x, dt + g(t dB,, Xo ~ qo
where (Bz) is the standard Brownian motion in R2 s(X,, () is drift coefficient and g(() is diffusion coefficient. Let ạt. be the density function of X, at time I. With a suitable choice of drift and diffusion terms (e.g. Section and 3.2). the forward process gradually converts the data distribution into Gaussian noise. More specifically, the conditional distribution X is exactly N (m,, Xo, If given Xo. where my is determined by the drift term and of is determined by the diffusion term.
The VPSDE forward process. Let {5e 1220 be a non-decreasing sequence with bounded range []. The VPSDE has the following formula:
dx;- = X dt + 223, dB, where X. q0
(1)
In this case, my = exp(- So 3,ds) and of = -mm.. Note that my I exp (-T/3) ,which indicates a fast forward convergence rate TV(a7) (0, I)) & exp (-T/3) CChnn et al. 2023c].
The VESDE forward process. The VESDE forward process is defined without a drift term:
dx, = ddo? /dt dB,, where X, To
Two common choices for of are and t², with the latter achieving SOTA performance [Krrras et al. 2022,
Teng et al. 2023). However, VESDE only has slow polynomial-decay forward convergence rate (Theoeem 4.2). which introduces hardness to obtain an efficient sample complexity (see Section This motivates us to design an improved VESDE process with a fast forward convergence rate.
3.2 The Drifted VESDE Forward Process
Note that the forward convergence rate the general process is upper bounded by my/ /o (Theo- rem 4.2). In practical applications [Ho al. 2020 Karras et al. 2022 Song et al. 2023]. the variance of the forward process of at time T, which is determined by the diffusion term, does not exceed T This indicates the contribution of o the forward convergence rate is only 1 1/Poly(T). Hence, the exponential-decat forward convergence rate of VPSDE comes from the drift term, which introduces an aeoonential-decca m, I exp (3T Due to the absence of the drift term in VESDE, the data information, such as expectation [F[o]] and covariance Cov(gn) [no]. does not decay and m, = 1, which is key an only polynomial-dest
forward convergence rate mr/d? & 1/Poly(TT. With the drift term, the VPSDE gradually removes the data information from q: during the process, which makes q: quickly converge to pure Gaussian noise. Inspired by this elimination effect of the drift term, we propose a drifted VESDE forward process:
dX;- - 3,X, dt 2B, dB,, Xo 9o
where T E [T, T2] is the coefficient used to balance the drift and diffusion term and {8eh2o is positive non-decreasing sequence. In this case,
m, exp (- 3./T ds and of = 711 (1-m?))
We show that the drifted VESDE is not only an effective representation of the existing VESDE but also extends beyond it (see Section 7 and Appendix A. We also prove that this process with suitable 3e has a exp (-T) forward convergence rate and enjoys an efficient polynomial sample complexity.
3.3 The Reverse Process of the Drifted VESDE
To generate samples from Gaussian noise, a diffiusion model reverses the forward process. Let of be the density function of the drifted VESDE forward process at time i and (Y, x)eeoo.7) ( As shown in Cattiaux et al. 2021 the reverse process of drifted VESDE has the following form
dY, = - Y, Y,/T + (1 7² log 47 (Y.)) dt + ny23 dB,
(5)
We note that the choice [2' is used guarantee the exploding variance of the forward process. In fact, our drifted VESDE is general formula that covers the VP and VESDE by choosing 7 1, 1, More details are shown Section
Now (Be) is the reversed Brownian motion, and abuse this notation here for of notation.

---

The parameter 1) E [0, 1] is used to determine the type of reverse processes. There are two common reverse processes in the application: reverse probability flow ODE (PFODE) (1 = 0) [Song et al.) 2020b
2023 and reverse SDE (1) = =1 (Ho et al.) 2020].
To generate distribution To through running the above reverse process, diffusion models need the true score function V log (Ye) and the accurate reverse beginning distribution 97. However, v log -- and 97. contain the data information and usually can not be exactly calculated. For the score function, diffusion models approximate it using a score network s(T' - t by minimizing the score matching objective function |Vincent 2011]. For the initial distribution of the reverse process, since 97 should be close to a pure Gaussian, we choose I = =0, I) as an approximation. Then, the continuous reverse process (Y c) incorporating s(T - .. and then is defined as:
- - 3r (V./T + (1 L, (, + 7/ 2,3- dB,, , where Yo Thx
Since diffusion models can not run continuous process due the nonlinear score function, these models usually discretize the above continuous process and freeze the approximated score at the beginning of each interval. Let {7x ]E([K] be the stepsize and fx+1 Σド= 7/-- As shown in Kim et al. 2021), V Vlog - (Y.) goes to +00 the end of the reverse process. To mitigate this issue, they use the early stopping technique 'x- = T - ō, and we also employ this technique in this work. With the stepsize, we choose the exponential integrator discretization scheme (Zhang and Chen 2022] to discretize the above process, which runs the following process:
k
dY, = - (1 )T tRo Y., dtt + 28T dB,, where t E (R+)) (6)
As shown in Karras et al. 2022]. J. the choice of 3r significantly affects the performance models, and we need to determine 3, before running the reverse process. The state-of-the-art diffusion models adopt 3, = t, which increases rapidly and has an unbounded range. However, current theoretical confined to bounded interval [1/3, 3]
works assume 3y to be a constant [Chen et al. 2023c]
[Bortoli 2022] to match the setting of VPSDE. To align more closely with practical applications allow an unbounded 3e in this work. Furthermore, make detailed assumption on 3g when considering different reverse processes.
of VEEeaaee models,
Assumption 3.1. Let {32 be a positive, non non-decreasing sequence. For any r E [T,T*). there exists constants 3 and C, such that for any / E T]: (1) for 7) O, then 1/3 s S max{33 (3,)) and 50" Be/T dt S C:; (2) for 17 1, then 1/3 s 3r S max{B,
As shown in Chen et al. 2023b), due to the absence of the stochasticity, the small errors for quickly accumulate and are magnified. Hence, we assume conservative 3, for the reverse PFODE, whose growth rate most t, to avoid an additional exp (T) in the convergence guarantee (see Section 6.1 We note that this choice of B3 is satisfied in practical applications [Song et al. 2020b Karras et al. 2022]. For the reverse SDE setting, we assume the growth rate of 3 can depend on T instead of at most linear. For example, when = T we can choose B, = 12, which has the same order as As shown Theorem 4.2 the drifted VESDE with aggressive 33 12 has an exponential-decay forward convergence rate, which leads to the first efficient polynomial complexity for VE-based models.
Notations. For I E Rd and A E Raxd we denote by || and ||A|]] the Euclidean norm for vector and the spectral norm for matrix. We denote by 5 argmax (o....,KK 7 the maximum stepsize
for k E [0, K 1]. We denote by Pr the distribution of Xr, Q the distribution of Yike R% the distribution of Y'x and Q20or the distribution which does reverse process starting from 47. (Eq. 5). We denote by W1 and W2 the Wasserstein distance of order one and two, respectively.
4 The Faster Forward Convergence Rate for the Drifted VESDE
This section shows that the drifted VESDE has a fast forward convergence rate. Since o7 contains the data information, we first introduce the manifold hypothesis before controlling TV 9x) Assumption 4.1. go is supported on a compact set M and 0 E M.
We denote the diameter of the manifold by = sup l - y/l x, y E M) and assume R >>> As shown in Section I the manifold hypothesis is supported by much empirical evidence (Bengio

---

et al. 2013, Fefferman et al. 2016 Pope et al.
2021 and allows the blow-up phenomenon of the score. Recently, Tang and Yang 2024 show that diffusion models can adapt to the intrinsic manifold structure. With Assumption 4.1 we obtain the forward process guarantee for the drifted VESDE.
Theorem 4.2. Assume Assumption 4.1
and 3.1 Let t'o A0 o With mer or defined in
Equation (4). we have TV Yoo) /mT /mrD//r, where D de Elmo R and is the eigenvalue of Corlgol with the largest absolute value.
Recall that mry = exp(- --" B./T dt), the previous VESDE [Song et al. 2020b Karras et al. 2022
Lee et al. 2022 chooses a conservative & satisfies J Be/T dt I C. However, with an aggressive Br, the drifted VESDE will have faster convergence rate. To illustrate the accelerated forward process, we use T = 7 as an example and discuss different 3: = t aa E [1, 2J. Due to the definition of oT, aT T, and the forward convergence rate mainly depends on v VITT. When 0y is conservative, my is a constant, and the convergence rate is 1/T. When 01 = 1 + In(2r in (T)) In()) is slightly aggressive, the convergence rate is 1/T for r > O. When 01 2 1 + In (anTT-((T))) (nT))//n(T) is aggressive, the convergence rate is faster than exp( (-)). In our analysis, whether 5g can be aggressive depends on the reverse process (see Section (6). When choosing an aggressive Br, the drifted VESDE achieves an improved sample complexity compared with pure VESDE (see Section 5)
In(T)
5 The Polynomial Complexity for a Series of VESDE with Reverse SDE
In this section, we first pay attention to the reverse SDE (1) = 1) to show the power of the drifted VESDE. More specifically, show that our general drifted VESDE form covers the current models (VP and VESDE). After that, we show that drifted VESDE can go beyond the current models and achieve improved complexity with aggressive 3g. Since the objective function minimizes the Le distance between the ground truth and the approximated score, we assume that the approximated score is Lg-accuracy, which is exactly the same with Chen al. 2023c] and Benton et al. [2023].
Assumption 5.1. Eqg,
Sta.
V In q., [1]
S L.core for Vk E []]
With this assumption, we provide a universe convergence guarantee for T E [1, and 3 E [1, and discuss the sample complexity of VP and VE-based models in detail.
Theorem 5.2. Assume Assumption
3.1
4.1
5.1
Let defined in Theorem 4.2 7K =
ke (o......I) K-1 = T and Be E
1,t"
Then, we have that
D mm
Rdvd
TV R 9b)
v 3TTTTT + Cscore V BT
Or
To guarantee the above convergence guarantee smaller than Ö(erv), each component of the result needs to be smaller than érv. As shown in (0mark(5.3) it is difficult for pure VESDE to balance the approximated score and the first two error terms to achieve efficient sample complexity. Hence, discuss how to balance the reverse beginning and discretization error. More specifically, we require D D/TT//o /oT S eTV and 5x S o (RrdBTTTT where the first inequality determines the order of T and the second inequality determines the stepsize 7к. After that, with sample complexity
K T==77 T//kk, have that TV .95) Ö(TTv). The last step is to guarantee Øs and go is close enough (2 Qs) S S which requires of S /wv RR/)..
Following the above process, this general convergence guarantee leads to the polynomial sample complexity for VP and VE-based models. When 3, = I and 7 1, the drifted VESDE becomes which achieve exactly the same order compared with Chen et al. (2023c). When Br = 1 and 77, 7,, our formula is similar but slightly better (Figure 2 and 3) to pure VESDE (o = () and achieves O(1 / w; e result. For B, = L and - T the
VPSDE and achieve the complexity ((I/(=V,
general formula is similar to SOTA pure VESDE (o? 82) and achieves the first polynomial results ((1/c W3 cTyy for this model under the manifold hypothesi
Although we achieve the first polynomial sample complexity for VE-based models under the manifold hypothesis, it is clear that the results of the VE-based models are significantly worse than the result
WW note these results still hold for pure VESDE with o² = t and 2², and we use the general drifted VESDE for simplicity. Here, only consider the dependence of e. Readers can find detailed results in Appendix C.I

---

of VP-based models since the slow L/Poly(T) forward convergence guarantee. More specifically, the forward convergence rate of VPSDE exp (-T), which means T has the order log(1/erv) and can be ignore. When considering the pure VESDE with of 12 the forward convergence rate is D/T. which indicates T 2 D/ETV is polynomial term and can not be ignored. Hence, the results K = R R/TT (of of pure VESDE (o² = (²) is heavily influenced by T. When considering pure VESDE with of = I, it suffers from slower forward convergence guarantee D/vT, which indicates T 2 D2/eTy and = RA'T eTy) This is the first work to explain why pure VESDE (o? = f-) performs better than pure VESDE (o? = =) from a thcoretical perspective.
Remark 5.3. In this part, we explain the reason why the pure VESDE fails to balance the reverse beginning and the approximated score. We use the pure VESDE with of = tf as an example. Under this setting, the guarantee has the form 1/T + V + Éscore vT², which requires '' 2 1/eTv. Then, Cecore VT2 is larger than Escore/ / c+v. Hence, it is hard achieve non-as con-ssympootic results.
Drifted VESDE with an aggressive Be balances different error terms. This part shows that our drifted VESDE with a suitable aggressive 3, can balance the above three error terms. More specifically, we show that introducing aggressive 3r only slightly affects the discretization error and significantly benefits in balancing reverse beginming and approximated errors. As result, we obtain an efficient polynomial complexity for series of VE-based models with unbounded Sr.
Corollary 5.4. Following the setting of Theorem 5.2 "W2
When considering T = T -T, = 1,, by
choosing 8 S (d+Rvayis) T 2 2In(D/erv), 7K- S 8 ey (D/erv)))) and assuming
(score ÕCrv)) RK is (erv (score) close to qs. which ew2 close qo. with sample complexity
K ad RVaja
Gw.er
Defined by R2 the output Ri projected onto B (0, Ro) for Ro (RR.. Then, we achieve pure
K W2 guarantee W2 qu) S ew, with sample complexity Ō (dnn (dn' (d-Raa))
WW
Since our drifted VESDE with r = 2 and B. = f2 has a fast forward convergence rate exp (-T)/T² T becomes logarithmic term and does not influence the discretization term, which is the source of the improved sample complexity. Furthermore, the requirement <eore has the same order with etv, which indicates the drifted VESDE balances the reverse beginning and approximated score error. In fact, we only require the forward convergence rate of drifted VESDE is exp (-T), which indicates series of VE-based models can achieve this sample complexity. We
= T as example. When considering the 3 = tr) we require 2 2 a1 2 1 + In - (T)) In(T) to enjoy exp (-T) forward convergence rate and achieve K < Ô (1/(ci erv)) sample complexity. For 3r = t and
T T, we also obtain complexity K SŌ (1/ (ciz (tv) (Appendix C.1
Lee Remark 5.5. Recently, Lee et al. 2022) and Gao et al. 2023 consider the sample complexity of 2022 consider VESDE with of =
VESDE with reverse SDE under strong assumption. Lee et al.
and achieve O(L2/Ctv) result under the LSI assumption. Under the manifold hypothesis, the result is Ö(L/Ew,, (ty) which is worse than Corollary 5.4 Gao et al. [0023 achieve pure W2 guarantee Ō(1/<²)) under the log-coneave distribution, which is even stronger than I.II assumption and ignore the influence of Hence, they ignore an additional 1/ I/Poly(W,, (Detail in Appendix A.2
6 The Tangent-based Analysis Framework
To deepen the understanding VE-based models instead of the specific reverse process, we introduce the unified framework for VESDE with reverse SDE and PFODE. Similar to previous PFODE work CCen al. 2023d], we assume an accurate score and consider the other errors.
Theorem 6.1. Assume Assumption 3.1 and 4.1 8 < 1/32 and "yk sup - 41).T4))) Be/03 < 1/28 for Vk ..,... K 13. Let 7K- 6. Then, for VT E [1:
7

---

(1) If (the reverse SDE), choosing Br =1= W1 (Ri .90) bounded by
R + vajvo+ exp
{
R²
3
+ ))
(G(TTTAAT (13) nn + +1)
De
2112
1/2
2
83
VT
where (T( (1/T 3/8a) and C1(T) is linear in 7?.
1 choosing conservative Be (Assumption 3.14. Wi (R9fe.9) ) is bounded by
R + + cxp
R22 ( +) (2(T))B(T)
( ( Mik
+1
1/2 PK
+
VF
D
where -T) 3/6²) and C2(T) is linear in 73.
Theorem 6.1 proves the first quantitative guarantee for VE-based models with reverse PFODE using the unified tangent-based framework. Correspondingly, the Girsanov-based method [Chen et al. 2023c can not deal with reverse PFODE since the reverse process diffusion term is not well-defined. Recently, Chen et al. (2023d) employ the Restoration- Restoration-Deggraation framework to analyze VESDE with reverse PFODE, which also has exponential dependence on R and 8. Furthermore, their results have exponential dependence on Be (Amax in Chen et al.) (023d]), which corresponds to T. However, our dependence on r appears in the polynomial term. Hence, our framework is suitable unified framework. Furthermore, we emphasize that our tangent- based unified framework is not a simple extension BBrtool (2022). We carefully control the tangent process according to the variance exploding property of VESDE to avoid exp (Z") term when considering PFODE 6..ction66.1)
Theorem 6.1 has exponential dependence on I and 8, which is introduced by the tangent process. Similar to Bortoli 2022]. if we assume the Hessian || | log qi (Ix) S. T/ we obtain a better con- trol on the langent process and replace the exponential dependence on by a polynomial dependence on 8 and exponential dependence on I' when considering reverse PFODE.
Corollary 6.2. Assume Assumption 3.1 4.1 and || log q: (z:))l S LT/O.. Let 1j 0 (reverse PFODE), E (0, 1/3)), =T3, 3, = and (T( C2(T) defined in Theorem 6.1 we have
R W1 R 90) S vajvo+
3 8T exp
T + 2 (C(())N((TTT(( +
B 82
1 1
D
+ 11nk
+ VF
2
Though the additional assumption is strong, many special cases, such as hypercube M [-1/2, 1/2]2 satisfy this assumption. We emphasize that our analysis also holds for VESDE (a² = (2)
with reverse PFODE, which means our results can explain the SOTA model in Karras et al. [2022]
6.1 The Discussion on the Unified Framework
In this section, we introduce the unified langent-based framework for reverse SDE and PFODE and discuss key steps to achieve the quantitative guarantee for PFODE. Firstly, we decompose the goal
W1
R 40 into three terms: W,
+ W, (0% Q200 + W, (Q .90)
These terms correspond to the discretization scheme, reverse beginning distribution, and the early stopping parameter ò. We focus on most difficult discretization term and first recall the stochastic flow of the reverse process for any Idd and s, t [0, 7] with 2
dY n, -r { Y (1 +7) V log -T (Y,,)) } di +1/ 72/237- -BB, where Y. x.
With VY= I, the corresponding tangent process is
d x. = x, / + 3r- (1 +7) 2 log Y VY
The key of the discretization error is to bound tangent process V e.t For this term, we consider the reverse SDE and PFODE simultaneously and propose a general version of Bortoli 2022
Lemma 6.3. Assume Assumption
3.7 and
4.1
For Vs [0, tx] and x E Rd, we have
R2
(1 (1- 172)
dx 3T u du
VY s.tg
S exp
20
2
-Ix
8

---

(a) Original Figure
(b) VESDE (T 100)
(c) Dritted PrftedVESD VESDE (T = 100)
Figure I: Experiment results of Swiss roll with Euler Maruyama Method (Reverse SDE)
We emphasize that the general bound for the tangent process is the key to achieving the guarantee for VESDE with the reverse ODE. Recall that in the original lemma for the tangent processes. since independent T and Be is bounded in small interval [1/ 3. ftx 9r-u/rde means there is an additional exp (T) when considering VPSDE with revere PFODE. However, our tangent-based lemma makes use of the variance exploding property VESDE to guarantee that So' 3//rdt & C' with a conservative 3t when considering reverse PFODE. When = 1, we choose aggressive Bt t since the choice of 3, does not affect the bound of the tangent process.
O which
For the early stopping term, it corresponds to and smaller than 2 RR/ + vd)vō. Since we can the data processing inequality in Wasserstein distance, the reverse beginning terms consists of the bound of tangent process term and the forward process term:
Wi (Q Qt0
m
R
(1
tK BT du
exp
OT
T
20
One notable future work is introducing the short regularization technique [Chen et al. suitable corrector to remove the above exponential dependence.
2023b and
Experiments
In this section, we show the power of the drifted VESDE forward process through experiments. Section shows that aggressive one achieves good balance in different error terms. After that, we consider the approximated score and show that the conservative one can improve the quality of the generated distribution without training in the synethetic and real-world setting.
7.1 The Aggressive Drifted VESDE Balances Errors
In this section, we do experiments on 2-D Gaussian to show that the aggressive drifted VESDE balances different errors. Since the ground truth score of the Gaussian can be directly calculated, we use the accurate score function to discuss the balance between the other two error terms clearly. We show how to use approximated score in Section 7.2
As shown in Figure the process with aggres- sive 3, = 12 achieves the best and second per- formance in El and EM discretization, which supports our theoretical result (Corollary 5.4) The third best process is conservative 3, with the small drift term. The reason is that though can not achieve exp (-T) forward guarantee, it also has a constant decay prior information, which slightly reduces the effect of the reverse beginning error (Section 3.I). The worst process pure VESDE since is hard to balance different error sources. Our experimen- tal results also show that is better than EMM.
EM, Br ==
EM,B.=t,T=TT
Divergence
5
EM, pure VESDE, of =t²
EI, A==
4
EI, =
EI, pure VESDE, o² =
3
2
0.005 0.010 0.015 0.020 0,025 0.030 0.035 0,040 Stepsize
Figure 2: Results of 2-D Gaussian

---

(a) Pure VESDE
(b) Drifted VESIE
Figure Experiment results of CelebA dataset
7.2 The Conservative Drifted VESDE Benefits from VESDE without Training
As shown in Figure the red and orange lines have similar trends. Hence, for conservative drifted VESDE, which satisfies (2) of Assumption 3.1 we directly use the models trained by pure VESDE to improve the quality of generated distribution. We confirm our intuition by training the model with pure VESDE with of and directly use the models to conservative drifted VESDE with 3g = and T = T. From the experimental results (Figure [ it is clear that pure VESDE has a low density on the Swiss roll except for the center one, which indicates pure VESDE can not deal with large dataset variance, as we discuss in Section 4 For conservative drift VESDE, as we discuss in the above section, it can reduce the influence of the dataset information. Figure I (c) supports augmentation and shows that the density of the generated distribution is more uniform compared to pure VESDE, which means that the drift VESDE can deal with large dataset mean and variance.
Beyond the synthetic data, we show that our conservative drifted VESDE can improve the generated images of pure VESDE without training the real-world CelebA256 dataset. From the qualitative perspective, as shown Figure 3 the images generated by our drifted VESDE have more detail (such as hair and beard details). On the contrary, since pure VESDE not deal with large variance, the images generated by pure VESDE appear blurry and unrealistic in these details. From the quantitative results,
aesthetic score [Schuhmann et al. 2022] and Inception Score to measure the quality of generated images. Our drifted VESDE achieves aesthetic score 5.813, and IS 4.174, which is better than the results of baseline pure VESDE (aesthetic score 5.807 and IS 4.082). There more examples on 256 and more experiments on Swiss roll and 1D-GMM to explore different sampling methods (RK45, PFODE) and different T. We refer to Appendix G for more details.
8 Conclusion
In this work, we analyze the VE-based models under the manifold hypothesis. Firstly, we propose new forward drifted VESDE process, which enjoys faster forward convergence rate. Then, we show that with an aggressive 3r, the new process balances different errors and achieve the first efficient polynomial sample complexity for series of VE-based models with reverse SDE.
After achieving the above results, we go beyond the reverse SDE and propose the anggnt-basee unified framework, which considers reverse SDE and PFODE the same time. Under this framework, make use of the variance exploding property of VESDE and achieve the first quantitative convergence guarantee for SOTA VE-based models with reverse PFODE. Finally, we show the power of the new drifted forward process through synthetic and real-world experiments.
Future Work and Limitation. This work proposes the first unified framework for VE-based models with an accurate score. After that, we plan to consider the approximated score error and provide a polynomial complexity for the VE-based models with reverse PFODE under the manifold hypothesis.
Broader Impact. Our work focuses on the convergence guarantee the SOTA diffusion models and deepens the understanding of diffusion models. Therefore, this work can be viewed as a fundamental step in improving the quality of diffusion models and the societal impact similar to general generative models MMrsky and Lee 2021]
10

---

Acknowledgments and Disclosure of Funding
The author Bo Jiang supported by National Natural Science Foundation of China (62072302).
References
Vladimir Alekseev. estimate the perturbations of the solutions of ordinary differential equations. Vestn. Mosk. Univ. Ser. I. Math. Mekh, 222-36, 1961.
Yoshua Bengio, Aaron Courville, and Pascal Vincent. Representation learning: review and new perspectives. IEEE transactions on pattern analysis and machine intelligence, 35(8):1798-1828, 2013.
Joe Benton, Valentin Bortoli, Arnaud Doucet, and George Deligiannidis. Linear convergence bounds for diffusion models via stochastic localization, arXiv preprint arXiv:2308.0365: 2023.
Valentin Bortoli, Convergence of denoising difffusion models under manifold hypothesis. Truns. Mach. Learn. Res., 2022, 2022. URL /RRp:tttp://ppeeerevveee wtt.:/wwweeee.oeeeeeeeeee hi-horrammniddkKKaaX33a3332
Patrick Cattiaux, Giovanni Conforti, Ivan Gentil, and Christian Léonard. Time reversal of diffusion processes under finite entropy condition. arXiv preprint 1rX.::2004,07008, 2021.
Hongrui Chen, Holden Lee, and Jianfeng Lu. Improved analysis score-based generative modeling: User- friendly bounds under minimal smoothness assumptions. In International Conference on Machine Learning, pages 4735-4763, PMLR, 2023a.
Sitan Chen, Sinho Chewi, Holden Lee, Yuanzhi Li, Jianfeng Lu, and Adil Salim. The probability flow ode is provably fast, arXiv preprint arXiv:2305. 2023b.
Sitan Chen, Sinho Chewi, Jerry Li, Yuanzhi Li, Adil Salim, and Anru Zhang. Sampling is as easy as learning the score: theory for diffusion models with minimal data assumptions. In The Eleventh International Conference Learning Representations, ICLR 2023, Kigali. Rwanda, May 1-5, 2023. OpenReview.net, 2023c. URL https://openzevieuu http://wwwwwneedeeeeeeeeeeeeeeee 2iy./yys
Sitan Chen, Giannis Daras, and Alex Dimakis. Restoration-ddgradatiin beyond linear diffusions: A non- asymptotic analysis for ddim-type samplers. International Conference on Machine Learning, pages 4462 4484. PMLR, 2023d.
Giovanni Conforti, Alain Durmus, and Marta Gentiloni Silveri. Score diffussion models without early stopping: finite fisher information is all you need, arXiv preprint arXiv:2308. /2240, 2023.
Valentin De Bortoli, James Thortoo,, Jeremy Heng, and Arnaud Doucet. Diffusion schrödinger bridge with applications to score-based based generative modeling. Advances in Neural Information Processing Systems, 34: 17695-17709, 2021.
Pierre Del Moral and Sumeetpal Sidhu Singh. Backward itô ventzell and stochastic interpolation formulac. Stochastic Processes and their Applications, 154:197-255 2022.
Zihan Ding and Chi Jin. Consistency models as rich and efficient policy class for reinforcement learning. arXiv preprint arXiv:2309. 16984, 2023.
Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz. Axel Sauer, Frederic Boesel, et al. Scaling rectified flow transformers for nighrresolution image synthesis. arXiv preprint 2403033000 2024.
Charles Fefferman, Sanjoy Mitter, and Hariharan Narayanan. Testing the manifold hypothesis. Journal of the American Mathematical Society, 29(4):9331104) 2016.
Xuefeng Gao and Lingjiong Zhu. Convergence analysis for general probability flow odes of diffusson models in wasserstein distances. arXiv preprint 2401::2401. 17958, 2024.
Xuefeng Gao, Hoang M Nguyen, and Lingjiong Zhu, Wasserstein convergence guarantees for general class of bosee generative models. arXiv preprint arXiv:2311.1100: 2023.
Jonathan Ho, Ajay Jain, and Picter Abbcel. Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems, 33:6840-668 0651, 2020.
Jonathan Ho. Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi, and David Fleet. Video diffusion models. arXiv preprint 2rXiv:2204.033500 2022.

---

Tero Karras, Miik Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion-based generative models. Advances in Neural Information Processing Systems, 35:26565-26577, 2022.
Dongjun Kim, Seungjae Shin, Kyungwoo Song, Wanmo Kang, and II-Chul Moon. Soft truncation: A universal training technique of score-based diffusion model for high precision score estimation. arXiv preprint arXiv:2/06.05527, 2021.
Dongjun Kim, Ycongmin Kim, Wanmo Kang, and I1-Chul Moon. Refining generative process with discriminator guidance in score-based diffusion models. arXiv preprint arXiv:2211.17099 2022.
Chich-Hsie Lai, Yuhta Takida, Naoki Murata, Toshimitsu Uesaka, Yuki Mitsufuji, and Stefano Ermon. Fp- Improving score- score-based diffusion models by enforeing the underlying score fokker-plance equation. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett, editors, International Conference on Machine Learning, ICMI. 2023, 23-29 July 2023, Honolulu, Hawaii, USA, volume 202 of Proceedings of Machine Learning Research, pages 18365-183998. PMLR, 2023.
diffusion:
Jean-François Le Gall. Brownian motion, martingales, and stochastic calculus. Springer, 2016.
Holden Lee, Jianfeng Lu, and Yixin Tan. Convergence for score-based generative modeling with polynomial complexity. Advances in Neural Information Procexsing Systems, 35:22870-22882, 2022.
Holden Lee, Jianfeng Lu, and Yixin Tan. Convergence of score-based generative modeling general data distributions. In International Conference on Algorithmic Learning Theory, pages 946-985. PMLR, 2023.
Gen Li, Yuting Wei, Yuxin Chen, and Yuejic Chi. Towards faster non asmmptottooiic convergence for diffusion- generative models, arXiv preprint 2rX1v:2306.092355 2023.
Haoran Li, Haolin Shi, Wenli Zhang, Wenjun Wu, Yong Liao, Lin Wang. Lik-hang Lee, and Pengyuan Zhou. Dreamscene: 3d cauusian-based text-to-3d scene generation via formation pattern sampling, arXiv preprint arXiv:2404.03575. 2024.
Shanchuan Lin, Bingchen Liu, Jiashi Li, and Xiao Yang. Common diffusion noise schedules and sample steps are flawed. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 5404-5411, 2024.
Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu. Dpm-solver: A fast ode salver for diffusion probabilistic model sampling around steps. arXiv preprint 1/Xiv:2206.00927, 2022.
Yisroel Mirsky and Wenke Lec. The creation and detection deepfakes: A survey, ACM Computing Surveys (CSUR), 54(1):1-41, 2021.
Jakiw Pidstrigach. Score-based generative models detect manifolds. arXiv preprint arXiv:2206.010/8. 2022.
Phillip Pope, Chen Zhu, Ahmed Ahdelkader, Micah Goldhlum, and Tom Goldstein. The intrinsic dimension of images and its impact on learning. arXiv preprint 0rXiv:2104.08894 2021.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10684-10699 10695, 2022.
Christoph Schuhmann. Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, et al. Laion-5b: An open large-scale dataset for training next generation image-text models. Advances Neural Information Processing Systems, 35: 1-278-25294 2022.
Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusson implicit models. arXiv preprint arXiv arXiv:2010.02502, 2020a.
Yang Song and Stefano Ermon. Generative modeling by estimating gradients of the data distribution. Advances in neural information processing systems, 32, 2019.
Yang Song and Stefano Ermon. Improved techniques for training score-based generative models, Advances neural information processing systems, 01112243-1244 2020.
Yang Song. Jascha Sohl-Dickstein, Diederik Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. arXiv preprint arXiv:2011. 13456 2020b.
12

---

Yang Song, Prafulla Dhariwal, Mark Chen, and Ilya Sutskever. Consistency models. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett, editors, International Conference on Machine Learning, ICML 2023, 23-29 July 2023, Honolula, Hawaii, USA, volume 202 of Proceedings of Machine Learning Research, pages 32211 1 32252. PMLR, 2023.
Rong Tang and Yun Yang. Adaptivity diffusion models manifold structures. In International Conference on Artificial Intelligence and Statistics, pages 1648-1656. PMLR, 2024.
Jiayan Teng, Wendi Zheng, Ming Ding, Wenyi Hong, Jianqiao Wangni, Zhuoyi Yang. and Jie Tang. Relay diffusion: Unifying diffusion process across resolutions for image synthesis. arXiv preprint arXiv:2309.03350 2023.
Pascal Vincent. A connection between score matching and denoising autoencoders. Neural computation, 23(7):
1661-1674, 2011.
Qinsheng Zhang and Yongxin Chen. Fast sampling of diffusion models with exponential integrator. arXiv preprint arXiv:2204. 13902, 2022.
13

---

Appendix
A More Discussion Drifted VESDE and Current Works
A.1 The Drifted VESDE is Representative Enough.
In this work, we consider [T, T and show that this choice is enough represent the current.
VESDE. More specifically, since VESDE with of = f. has 9r (V Cov [go] + 7T) and drifted VESDE with 7 7 and 3e = 1/2 has qi = [exp (- 전 [b exp I)), these two setting almost identical when T +oo. The second choice of VESDE of = 12, which achieves the state-of-the-ar performance |Karras et al. 2022]. almost identical to r = T2, Br = t. The simulation experiments also show that VESDE and drifted VESDE with specific Br and T have similar performance (Figure 2
[go] exp (-
(1
A.2 The Detailed Calculation of Previous work
The results of Lee et al. [2022] Lee ct al. [022] consider VESDE (o² t) with reverse SDE under the LSI assumption with parameter CLs- The LSI assumption does not allow the presence of substantial non-convexity and is far away from the multi-modal real-world distribution. Furthermore, they use unrealistic assumption Cocure < 11/CCss + T) avoid the effect of the approximated score, which is stronger than Assumption 5.1 Under the above strong assumption, [ee et al. (2022) achieve the polynomial sample complexity O(L2d(d)c) + R)2 /))/<tt). Under the manifold hypothesis, by
Lemma H.2 we know that
L = R² d2/eww
Then, the result is O(Add( + R)2/ei, vtv). which is worse than Corollary 5.4
The results of Gao et al. 2023]. Gao et al. 2023 analyze series of VESDE with reverse SDE and achieve 1/ W. sample compelxity for VESDE with of 12 in 2-Wasserstein distance. However, they assume the data distribution is log-concave, which is even stronger than LSI assumption. Furthermore, under this assumption, log (-) do not blow-up at the end of the reverse process, which do not match the empirical phenomenon and ignore the influence of early stopping parameters. To transfer their results to our the results under the manifold hypothesis, we need to consider the influence of 8, which would introduce additional 1/Poly(W,)) term.
B The Proof for the Faster Forward Process
Lemma B.I. The minimization problem mina, v, KL (ff [Wi,, V.)) is minimized by ma m.E [oo] and V = mi Cov [go] + I where my and os defined in Equation (4)
Proof. For simplicity, denote the mean and covariance of qo by a and C". We also define the optimize variable ne = N (m,, Ct). We can directly compute the KL divergence (L:q:(m)):
KL (gelme) H (1:) / log (n,( ge (x)dr
1 = - (q:) + log(2T)) + log (det (V)) + 1/2 / ((x m; (z)d
2
For the last term, we directly compute
/ (x - mi)" V (x m.) pt (x)drr
-E (Xe - ) V (X, me)
E [mixo t m²)" V, (m,Xo 0, - m.)
-E m? (Xu - a)T V.- (Xo - 0)] + (mia mi) Vi-1 (m4a - mi) + oiE 2 VTV
= tr (C'V,-) ) + of tr (V,-1) (maa mi) V.- (mra -m+)
14

---

where the second inequality follows that X, = m, + o, It is clear that the optimal solution of my is m,a. In the next step, we focus on the optimization problem for Vi:
L (V, = log (det (Vi) + ir (mm]C' + o v) log (det (V;- + tr (m + of V.
Since the above optimization is a convex optimization problem, we use the method similar to Pidstrigach| 2022]. we obtain that the optimal solution of V₁ is mîC² + ofl.
Lemma B.2. Let mh and V be the optimal mean and covariance operator from Lemma B.1 Then
1
I (mfc, + 00,)
R2m
KL (g/ Vi)) - 2
log
+
(oz)a
o7
dmge
Rºm+
+ o mfo
20
oî
oi
KL (N (o) of) =
m Σ
-i=1
Co
m, (E(9)))
1
log
(0
20
20
2
(mQ cs
+
of)
m Σ i= Cj
d
(1(9o)))
dmfc
mic o?
ol
2
20?
20?
where e, are the eigenvalues of Cov [go], and e is the eigenvalue with the largest absolute value.
Proof. For 2 0, we directly calculate the KI. divergence for this term:
KL (ge [m V.)) = -H (qt) + log (det (2 +
2 tr + o V,-1
2
-- H1 (gt) + 2
log (det Vi)) +
2
- (g:) + d
log (2T) + log
1
II
(mic c + o:)
+ d
2
2
2
f=1
where G, are the eigenvalues of Cov [go]. Now, we only need to calculate H H())):
1
|x - Xoll? )1)
-H (qi) = Ex, [log qe (Xt) = Ex, log Ex. (2To?)
4/2 exp
(-207
203
By Assumption 4.1 it is clear that
||xe xol12 ) S exp
(-1
1
(1x01l + 22xX, Xoi)
exp
(-20² 20,
20,
Then, we know that
(-1 1
E
log
Exo (2TO exp
(- 207
|| - -Xoll)
<E log (2To,)=a14
(Ix.112
+ 2(Xe, .oi)
207
<
d
2 log(2rr)
log ((02)d)
1
E ||xX,|] 1|
+ of
Rºm,
207
we also know that
E ||x,||]] = m
| 1| + of [1
= E |xoo|
+ EE [| = m3
Vo + o²d.
Finally, put these terms together, we have:
1 KL (g Vi) S 2 log
I (mfc, of) (o
R2 m
+
of
where & are the eigenvalues of Cov [go]. Then by choosing the largest absolute value eigenvalue largest absolute value, we can use the Taylor expansion to obtain the first results of this lemma. For the second result of this lemma, we directly compute the KI. divergence between M(m, V.) and (O, of) to obtain the final results.
15

---

Theorem 4.2. Assume Assumption 4.1 and 3.1 Let Toc = o With my, or defined in
Equation (4) we have TV(Tr, Ioo) S / D/ar, where = d/c + E[mo] + and e is the
eigenvalue of Covigol with the largest absolute value.
Proof. We know that
llgr 92 TV
$ l|gr (m Covl)))) DIvV
[[[(m) m/ Cov
o Gellrv
By directly using the Pinsker's inequality and Lemma B.2 we complete the proof.
C The Proof of the Polynomial Complexity for Reverse SDE
In this section, we prove Corollary 5.4 First, we recall the Girsanov's Theorem Chen al. [2023c]:
Le Gall 2016 used
Lemma C.1 (Girsanov's theorem). Let Pr and Qr be probability measures path space (Io,T); ())) Suppose that under Pr, the process (Xe)eeto follows
ax, b, dt at dBt
where B a Pr-Brownian motion, and under QT, the process (Xe)rEjo.T) follows
dX; b dt + at dBt
where B is a Qr-Brownian motion. We assume that for each > 0, at is a d x d symmetric positive definite matrix. Then, provided that Novikov's 's condition holds,
(
/ 1 2
10o." (b. b,) b.)1 dt at)
EQr exp
< <0,
we have that
exp (f"² ar (b, (b,-b)) dB, |0110 (b.--1)]' dQr
aPr
If the Novikov's condition is satisfied, we apply the Girsanov theorem by choosing Pr Q by
Rot QT
Br ( + 2s(T LR, Y.) (for [1, (+))), b.
3T-1 (,Y, + (1+ Vlog (Y,)). and a = 2B
Then, similar to Chen et al. 2023c]. we have the following lemma.
Lemma C.2. Assuming that Ro and Qix satisfy Novikov's 's condition, it holds that
KL Q
(
aQ tg
K-1
23r-r11s(T tb. Y..) Vnng-- dt. ('at 1/
= E oF
In
E
oF
4th
dR
k==
Before using the Girsanov's Theorem, we need to check the Novikov's condition. We use almost the same proof process compare to Chen et al. (023c). The key proof of the Novikov's condition is Lemma 19 of Chen et al. [023c Hence, we give complete proof of Lemma 19 in Chen et al. 2023c under our drifted VESDE. Before the proof, we first introduce smooth cuttoff function for truncating the drift terms.
Lemma C.3 (lemma 17 of Chen et al. 2023c ) For any R > 0, there is smooth function
DR Rdd >> [0, 1] satisfying: 1. @g(x)= for all | & R, Φ (x(x) ffr all || 2 2R, 3. &R
is
16

---

Note that R is not R in Assumption 4.1 and will goes to +oo in the proof of Chen et al. [023c]] Similar Chen et al. [023c]. we also introduce Lo and modified process with truncation argument for E [1, [1 Define the bad set
B, = {Il st V
In qi Excore ,
where Escore, oc > O is parameter to be chosen later. We define the - accurate score estimate to be
ii se1 Vlng..
We note that || V In I Excore .00-
The modified process with truncation argument for t [b,, tx+1] is
dY; dY, -Br- / tk. ) / 2/ dB,,
log 97 (Y? dt 2 dB.
where Y Yp is obtained by sampling Yi of and setting Y XT if | S R and setting Yo otherwise. Then, we ara ready to prove the following lemma.
Lemma C.4 (Modified key lemma for Novikov's condition).
K-
Ant1
op exp Qik
Br |laa (Y sT (YF- oR (YF- Viner (YF- dt < 0
Proof. We note that due to the manifold hypothesis (Assumption 4.1 if R 2 vdT2 R² then the marginal distribution of T-ts is exactly the same compared to X We also recall that R + to0
in Chen al. [003cc (Theorem 21 ofen et al. 2023c]). Hence, we can use Lemma to prove that
Vrr-uud (Y sT- (Y
sup
Brr-r | (Y =: A' <0.
and
| A Vin (Y (YT-
sup -]
VB-- | (Yr-r)ll =: B' <0.
Then, the left hand of this lemma is at most exp (2T (A + B²)) < do as claimed.
After obtaining the above inequality, the remaining proof for Novikov's condition are exactly com- pared to Chen al. 2023c].
Since we assume the accurate score function in this work. this lemma need to control
sup 23 |/ Ingr-r (x )| =: B < 0o.
x () E
As shown in Lemma E.1 know that with the early stopping parameter 8, |/ In -r (x*) Fr -- < B. Finally, with similar process to proof that the Novikov's condition is satisfied. The following lemma show the discretization error for our drifted VESDE with reverse SDE.
is controlled. By using Assumption 4.1 we know that
Chen et al. [023c].
Lemma C.5 (Discretizationn. Suppose that Assumption 4. and Assumption 5. Tholds. Let 7K = argmaX K-1] Tk, "K = 5. If [1 and Be E [1, then with Q and B defined in
Lemma
RT RLTTèrd
R TV
NK +
7 + 20ore TBT
o3
o
17

---

Proof. First, we control the discretization error in an interval t E [+x. tx+1]:
E
|s (T
ths Y.,)
Vingr-
Qig
K
S Soore
+
V In qr- (Y..) V (x
0ik KK
+
E
In
-t (Yo))
Vin (Y,))
of
E
In -T (Y:.))
L Qix
|Y -Y.11² + Ezooe
Qik
-
X T L drik + L (TT + 122) + + + LL (Srddi + (P7k)) + Eccore
S T dik + T 1 R²= 7px + scone :
where L,
[ -5] V log QT + (Y.)l - (1+ + R2-) /03 and the third inequality follows Then, know that
Lemma
E.5
K-1
1--1
Σ k=0
E
28r-e11s( (T tk. Y.,) - v Vingr- (Yi)l dt
of 0xk dt
LL dryk +
+ 2ame TB
T Brd d
RT
3r
k + 2000 T T3.-
7K
o3
After obtaining the general discretization error for our drifted VESDE, we focus on two special cases. For T and 3r = t², we have that
K 1
E
28T-e1ls(T tk. Y.,) V (Y:)) dt
Qig Jt
$ T L d5k
+ L 12R2T3BT B + 2 TT
R Byd
R BT
PK +
i T
o3
o3
Rd
d
1877
7K +
1K scone
T3
o3
o
For T and B, we know that
K-1
1
E Qik
28T-e1ls (T tx. Y.,) - V l dt
k=0
lt
3 L2dFik
L2 RRT + 2 T2
R d
ROT3
āK +
σ3 1k e T22
Combined with the reversing beginning error controlled by Theorem 4.2 we can obtain the conver- gence guarantee for the general drifted VESDE with reverse SDE.
Theorem 5.2. Assume Assumption
3.1
4
I
5.1
Let D defined in Theorem 4.2
7K
argmax &E (O.....(oo..)) (O..... K-1) lh. T = =T and B
E
1, t
Then, we have that
D Dmt
Rdva
TV Roi .46) 5
/TT + Escore V BT
oT
o²
18

---

Proof.
By the data processing inequality, we know that
TV
R K
18
S TV
Ri Rei
+ TV [ Qik
$ TV (or Poe) + TV (R 0ix)
Combined with Theorem 4.2 and Lemma C.5 we achieve the final result.
C.1
The Sample Complexity for Drifted VESDE
As shown in Theorem 5.2 the general convergence guarantee is
Dvmrr RRvd
TKBTTT Escore V BTT.
oT
o²
In this section, we provide the sample complexity for different Br and T.
The results of r and 3 becomes VPSDE and mr exp (-T), which indicates T is logarithmic term and the dominated of the convergence guarantee is the discretization term Ó vd /ot). To make this term smaller than erv. require ýK S o To make sure that W (1o, 98) ew,, require
When considering 3r and = 1, the drifted VESDE
L o3 S J+RVd'
Then, we achieve the final sample complexity
(dR Rvd)
K K
W. ciy
8
We note that this results is exactly the same with Chen et [023c which means our drifted VESDE covers VPSDE setting.
C.1.1
The results of 2 with different 3g
In this part, we analyze the influence of 3r under setting T and show the power of our drifted VESDE.
The aggressive TT When considering aggressive 3, = = where a1 2 1+ In(T
ln(T))) In(T",, /mT/orr 2 exp (-T/2), which means I' is a logarithmic term and can be ignored. After that, the analysis process is exactly the same with the above VPSDE setting, and we achieve the sample complexity
dR RVa) 8 ww cTv
K Ō
R Defined by R, the output R% same proof process Chen et al. (2023c) (Corollary 5), we have that
projected onto (0, Ro) for R, = ē(R). Following exactly the
W₂ (R K.R. go) S CW2
with sample complexity
dR Rva)
K
CW2
The conservative 3r = t. In this case, the first term is D/T. To make this term smaller than etv. we require 7' D/erv. For the stepsize, we require 7K o which means the sample complexity is
(R +RVV) 8 cw, cTv
K KSO
19

---

The most conservative 5t = 1. In this case, = exp (-1/T) and of = 7(1-m²) mm) = (1 exp (22/T). When T is large enough, my (1) and T which indicates the first term is D/VT. To make this term smaller than crv, we require T z D2 /cvv For the second term, require RKK I (R Then, the final complexity is
(d RVd) D
K KSO
EW2 TV
C.1.2
The results r with different Be
At the remaining part, we show the sample complexity of setting T with different Sr.
The results for setting T and Be to discretization error
For this setting, as shown in Lemma C.5 the
2 HTT3d 2 og
R&T3 + of
TV R
"K
+ T
Furthermore, we choose an aggressive Be, which indicates T a logarithmic term. Then, by choosing is and FK S ofctv In (D/crv) / (Rtd), we obtain the sample complexity
of S (d+RVā)
dRR (o+ddRR) (dvay)
K
7K
The results for setting T and 3: 1. In this setting, the reverse beginning error is bounded by D/VT, which indicates T> 12/e2rv For the discretization term, we require JK S of 2 (R Then, the sample complexity is bounde by
T KK
(d+ RVa)ADo CT
00
D The Proof of the Convergence Guarantee in the Unified Framework
In this work, we introduce an indicator E [1,2) for or-tx to represent different fr. We use r = 7 example. When 3r t2 is aggressive, we choose i 1 7- 1 and o 2 (i = + gs-
When 3, = is conservative, we choose i 2, [0, 1) and -2 (i = 2) < + 7.. In the proof process Lemma 6.3 Lemma D.I. Lemma D.2 and Lemma D.3 we ignore the indicator since this lemma does not involve the specific value of of xx (t). Before the proof of this section, first -
recall the stochastic flow of the reverse process for any Rdd and 8, E [) with
d dY stt = Br (Y + (1 + + V log -r (Y dt + /V dd
Y 2,8 = I.
and the interpolation of its discretization for any k E (0,..., K} and L E [Sk, (x+1):
dY (k) _r { Y ",,//T + (1+ sk, Y dt + n/ 2,
where sy max (s. tx). To deal with the discretization error, we use the approximation technique used in Bortoli [2022]. Hence, we introduce the tangent process:
dVY Br (I/r (1 (1+ n²) V lroggr-((YE,... (Y,))) VY dt,
VY= I.
Then, we discuss the interpolation formula, which is used to control the discretization error. Proposition 1. For t [0, 7) with s < t, any & E (0. K} and (wo)eejer)) we define that
1
(w) = Br u Ww + (1 +++1 n log (ww))())w)
1
bu(w) = Br- Wa + (1 ) (T - sk W) Abu(w) = bu(w) - bw)
20

---

where ms t&) and u [sk, tx+1). Then, for any I E R,,, we have that
Y s.1
VY u.e 8,0
Ab, ( (
du,
where for any u E [0, T), there exists a k E (0...., K satisfies u E [8k, 1k+1).
For reverse SDE, the augmentation is similar to Bortoli [2022] (Appendix E). When 1/ = 0, the stochastic extension of the Alekscev Alekscev-Gröbnn Gröbner formula [Del Moral and Singh 2022 degenerates into the original version [Alckseev] 1961]. After that, we control the tangent process.
Lemma 6.3. Assume Assumption 3.1 and
4.1
For Vs E [0, [x] and E IR, we have
R²
(1 - 1²) pek Br-udu)
/
VY
sxxal S exp
20
2
[r][v" log a (#t) S T/o7, VVY xll
S
OT-
(1+12
tx
exp
(((1+1) r So 2r-" du
Proof. Using the definition of the tangent process and Lemma E.1
we have
d 2 ?|
$ 2
VY a.l (1+ (1 R (2))))()))))) de.
Using Lemma
we have
[ BT-u (7 (1-(1+7²) / + (1 ( u22-2122/25 du
< ((1 + (11++(²)0 R2/4) (oT-1 - -o7-)) + 1 - 11² p 37- du
(1+ 12) R²
1-11
p 3T-u du.
+
407
2
Note that VY *. = I, we get
(1+
R²
//d pKx :r- du
IV k.i 1 < exp
+ +11 (1-
20
When we assume V log of (rt)) S <//03, we know that
2. 11 < 2,7 (
(1 + 17)
d V
VY dt.
2
T-
Using Lemma we have
{
2 BT- on-n/of_nde
T Br-u du -1 -1
T-t
Br du -
< log exp
2
log
exp
2
-0
5
00
7
T-- Bn du.
$ log (o7-)) log (07_1) +
T--
Then have
/ (xx BT du
VY 2 of -I stg
(11+7)) exp ((1+)²) [ +2)
Thus complete our proof.
21

---

After bounding the gradient of the tangent process, the remaining term is ||
| s || bl ( + || bl1
(7)
where ((a) = b and ((d) = b. Moreover,
6(0 (w) Pr-u( We + (1 + n²) V or
6x (Wu));
( (w) Br-u(( (ww + (1 + 17 V log oz (wox)) (Wax))
Agh ((a) b1b), D.c = b b(0), a.d = =ble)_ - b(a).
We then control ( bll, || bl. || (o.
separately. In this section, || = 0 since
we assume that the accurate score function is achieved. For || (a.b)b (w)]]: we have the following lemma.
Lemma D.1. For 8, u E [0, T) such that 2 8, [SK, (a-1 1) and = (Wo)eEls;Ty we have
| (o.b)), (w) ||
s + 1²) BT-u velT -u,T--t
sup
(Be/o (2+ R2) (R + [|wall) 7k-
Proof. Without loss of generality, we assume s S tx. Then
|A (o.b) bu (w) || S (1 n²) 8T-u V log or (wa) - V log - (wa) s (1 + n²) BT-
sup
100 V log -T (ww) ||.
WE T
Then by Lemma E.4 we have
A (a (w) ||
< (1 + 12)
Br-u
sup
(Bo/o (2 + R2) (R + [|wall) 7k-
vElT-u. u..
For b, we have the following lemma.
Lemma D.2. For s, u E [0, T) such that s, [st, tx-) and (Wn)eels:r we have
(brr) by (w) (1 + n?) (Br-a/ot (1 AA2) |w Woull.
Proof. Without loss of generality, we assume s I ty. In this case 8 = tkk Then
A (b.e) by (w)
< (1 + n²²) Br-u
V log - (wr.) - Vlog tt to (wa) II
S (1 n?) Br-u sup
|| log -r (wo) I|lwwr Wen ll.
Using Lemma E.2 we have that
|| (b, (w)
S (1) (Br-w/of-a) (1+ R²) I|wio Weall
Then the proof is complete.
We need to control the reverse process when dealing with Ab. The following lemma shows an upper bound for the reverse Yr.
Lemma D.3. Assume Assumption 3.1 Assumption 4.1 and there exists 8 > O such that S4 S
o
8 < 1/28 for any E (0,..., KK) then we have
B||| U
rd rd B(1AA
22

---

where
A 4n/² 2 26 4(1 r1) 8)
== 42 + 2(1 (1) 8) R
4np rd
P
and μ is an arbitrary positive number which makes 0. particular, if S 1/28, then [ ||2] - Uo(T) = 111 + 13rd.
Proof. Recall the discretization of the backward process (the explicit form of Equation (
( + (1+ )s)s (T' ts Yx)
Yx+1 = Ys + n.k
77 212,
[ T-111
T
- -1)12. 12:
71,k = exp
Bg ds
1, 72.6 =
exp
Bg ds
T-1211
where {Z*} kEk independent Gaussian random variables. It is clear that 71 S 72,k S 271,k- we have
and using Lemma E.
(2i, s(, oz)) (z,, V log ((z))
-- | + Rl
S (-1 + pm, /ot + (m,R/p))o?.
where the first equality follows that we assume the accurate score function. For any > 0. Again using Lemma F. we have
[ts 21) = |/ log 4e (xx))l
< 1| /0 2m 12/R//0 /0
Combining the results above, we have
= (1 2 ) [| (1 (2) [[]|s(T 1xs Yx)l
+ +2(1+1))) + 7i. > B((B (T - Lu, Yx)) + ra
< ((1 + n1. + +2 + /o
2(1 + )²) (1 + 7.k m + p R)/o7 ). 1|
2m2 to R2
(1 + 12 r/, +
(1 + 17 )1 + 71.8 YTi + 4nj 4n m.d.
TT
1
If we denote 6x 7/,k/// 02 - and notice the fact that m, [0, 1] of E [0, -). E [0, 1]. then
have
21]| (1 + 28 + || 12 + 86
2R
+ 2( + 5 (- / B 61
+ 5₂ + 5x)) + 4T6ad. p
We also have that
1" T- T-41
F = =xpp
Bids] S exp_3r-1,
S 28
where the last inequality follows that 7/4 = exp (-T), -1 7/ < 1/2 for small enough stepsize,
and e 1 < 2w for any 1/2]. We get dR < BT /0/ < 26. Thus
E E|| 1| (1 + 25, + 6 Y 16
R 8h 4rddk.
4(1
16R 5.5 4(1 + 8)
23

---

Hence, we have
|||| [1] (1 - 140 4(1 8) 1
R /
+ + 4(1 4rd]
denote A 148 4(1 + 6) and B 16 + 4(1 + 8) R
+ 4rd, then
/
B 119 _ (1 6x Y B.
Notice that 2[ dr and if { [12] B/A it is decreasing, if ||] S B/A have B|||+++1| [|-] & B/A++ 5B. so
E| - + B(1A++ B
Notice that when å $ 1/28,iff we choose
A (1 R) B S 37R + 4rd.
1/2, and
Then, the proof is complete.
The following lemma shows discretization error in the k-the interval.
Lemma D.4. Assume Assumption 3.7 Assumption 4. and 7t.3t-u/off /o7-10 S 1/28 for any k
(O.....A K 1-11. 1}. Then for any k, E [ [x+1] and {1,2), we have that
BI Y 119 (r
8 UO(T) + 64R² 9Kmi(F) of ix())
where Lx(T) = TicK(T)( 07 ex
64
4d, Jx. Ks(T) is defined in
Lemma D.5 and UT(T) is defined in Lemmal D.3
Proof. Recall the discretized backward process
Y, Yr, + (expl / T-4
B,ds] 1)( + +1 (1+ - 14 Y..))
T-t
f T-
T-th
+ n)(epp(2
B,ds (dds-1)) 11)
where 7 is a standard Gaussian random variable. By directly calculating, we have that
B - Y,, [1 =2(exp
1 Bods) -
(1){ -) Y, 119 + E(] Lx Y. )13
T-t
T-4x f B,ds] 1)d.
+ n (cxpl2
T-E
By Lemma H.I and accurate score function assumption,
[T tk, YY,)) )|12 < 2 Y 10//Y,,,1///5 12/04 (1) 2m² R²/ R2/04-1, (1).
So have that
T-x [T B,ds] - 1))
8R²
Elll Y, Y.a < 2(expl
+
EE|| N.2.1 11
T-t
T (i)
o7
--
T- T B.ds) 1)d.
+ (exp(2
By 2w - S + 4w for any w 1/2) and 74 sup B,/0² S 1/28 for any
E (0..., - 1}, we have
T- explp
3,ds] - 1 5 2p3r-u477
T-t
24

---

for p 1, And using Lemma D.3 and Lemma F.2 we have
B|llY; 1ie [1]]
$
647k
8. 7h
(o( 64R
7k
bu- + 4d.3, -1
T-
72
T-
We denote L((T) = FKkK(TT
+
T (o(T) + 64R (g(((())
+ 4d for E {1,2} and the
07 ix
(6)
I ix (1)
proof is complete.
Lemma D.5. Assume Assumption 3.1 and Assumption 4.1 Th.
80/0 I 1/28
for any k {0,.... K 1}. Let PK- ke K- Yke K;(T) m T I - (i), and
T
Ci(r) 2 R2) (
3/2 + R²3,, (0 2T, (T)
for E {1,2}. Then, for any s, [0. tk] with a 2 s and i {1.2], we have
Bl Bbl (Y. (( (1) + n((())
where Y 5.s I.O.I).
Proof. Combining Lemma D.I Lemma D.2 and the exact score function, we get
Abu (w)][ < (1+
sup
(2)
( + ||w 11)m
wE 44
+ (1+ (3T
dimm( - Weell-
For any u E [T' [T, T], using Lemma F.3 we have B.// S (T). Hence,
(w), $ (1+ 17)
wE(T v (i)) + diam(M + ([wull)
sup
+ + (i - weu
0
$ (13+ (1+ diam + ||
+
+ + R²) -1,
Combining this with Lemma D.3 and Lemma D.4
(1
=1 max ( [3, 3//23/2211// 1/2
We denote = [, E {1,2), 2], then we have
E CC (T)) 1/2
Lemma D.6. Assume Assumption
3.1 and Assumption
4.1
Th CoupuelT-tae t1+1 21.1. T-t+)) 8./o - 1/28
for any l E (O,.., K - 11.. Let 7K
argmax (O......
Tkk #K 8, and 1/32. Then
R²
11-
1/2
Wi R 1% C.(T)) exp
1/5
(i)
2
- tx
ii)
where (() (1,2) the same terms Theorem
6.1
25

---

Proof. By Proposition we have
|\xxx Yxll = |Y Yxx S 1xx11 (" IVYuux Yo.))llAAb,,l ((
(. o.eooe Ildu.
|Y -Yxll
[1+ R²
(1 (1-17) 1²) ('m pix 3r-u du Soee [Y
] pKc
$ eexp
(
Then by definition of Wasserstein distance, we have
Wi (9 Qtk 4o0 Rx)
E Y tK
YK Il]
(1 + 17²) R2
(1-1p²) (dh Pr-u ddu
$ <xxp
+
J 7
- (t)
2
R²
(1 m²))
CC(T)) exp
(x)())oT21x O +x]()))
1/2
40
1x
Theorem 6.1. Assume Assumption 3.7 and 4.1 1. 8 S 1/32 and Th - -41. Bo/o? S 1/28 for Vk {0, ..., K 13. Let "KK-
Then, for VT E [T, [T:
(1) If (the reverse SDE), choosing Br = = W1
R (0) is bounded by
R valvo exp
R2
3 83
+ :)
(10)
3 + mi +1 7x
De-
T
VT
where (T) = (1 + 3/52) and Ci(T) is linear in 7².
12) (PFODEE) choosing a conservative 3e (Assumption
3.14 W1 [Rfk .40)
is bounded by
R + Va)vo exp
(
(
R2 B +
2 82
(2(())((T)) (1 +))
82
B
+ nik
1 1/2
+1
7K
1/2
+
VT
D
where (2(T) = T (1/T 3/8²) and C2(T) linear in +?
Proof. To obtain the convergence guarantee, we need to control three error terms:
W1 R 40 W1
R Q Wi (0% Q?a + Wi
(Qrx"" .90)
For term Wi
[2 0% we use Lemma D.6
For the second term, we define (Y (ejee and (Y re(u,2] condition and y. Then have
be the reverse processes with initial
Yo, Y s | vll IIV
where 2> = Ax (1 Ajy. In this work, we choose x ~ Toc and ~ PoPr. Combined with the above inequality, Theorem 4.2 and Lemma 6.3 we know that:
Wi (0%
Q000T
R²
- 172)
AK BT-u du I|goPr goel
< exp
207 2
2
Jo
V
R²
(1
2)
xxx Br du
exp
OT
20
-tg
i
2
26

---

For the last term, we use exactly the same process with Bortoli 2022 with bounded --
Wi (oex
-00)
X- -T- X + or -r Z1]
1 R vd)oT-
R
2 + vd)Vo,
where the second inequality follows that - + TWT--K
In the end the section, we provide the proof of Corollary 6.2
2 log at (axe)l S l/o.. Let nj O (reverse PFODE), 8 E (0.1/32), T 3r i and (2(()), C2(T) defined in Theorem 6.1 we have
Corollary D.7. Assume Assumption 3.1 4./ and
Wi
{ .2) S { vdjvo+
R
32 8r
exp
T + 2
B C2(T))
1
1)
1/2 D VF
2
82
Proof. The proof of this corollary is almost identical to the proof of Theorem 6.1 replace the first bound for the tangent process in Lemma 6.3 by the second bound.
We just need to
E Lemmas for the Logarithmic Density
In this section, we introduce auxiliary lemmas to control the gradient and Hessian of the logarithmic density under the manifold hypothesis. Lemma E.1 Lemma [.2] and Lemma E.3 come from Lemma C.I, Lemma C.2, and Lemma C.S Bortoli 2022]. Since these lemmas do not involve the
relationship between my and dre we can directly use the results from |oortoli [2022] Following
2022]. we also define a empirical version qo with N datapoints, i.c. q = (1/N) Σ X with
[x*] k=1 40 We denote by (g )e2o such that for any measure of the distribution of X and when N - +oc, qi
the density w.r.t. the Lebesgue
Lemma E.1. Assume Assumption 4.1 Then for any t E (0, and x4 E R we have that
V (1)) s -1/==11//0
+ m
In addition, we have
V log qt (xx) 11 < 21|,,| 1//0//////// + 2m n² /oi
Lemma E.2. Assume Assumption
4.1 Then for any t E (0, 24 Rd and M Ma (R.)
2
V log qe (xx) M) S - - 1--m@r -1 R² (207)) /
In addition, we have
V log a (æ:) s (1 + R²) la
The following lemma shows that the derivatives up to the fourth order are uniformly bounded since 7 E [T, T Thus we can use the stochastic extension of the Bokkeev-Gröbbee formula (Del Moral and Singh [022].
Lemma E.3. Assume Assumption 4.1 Then, there exists 2 0 such that for any t E (0, we have
Log + log q (x) + V log (=)))) < C/o
The following lemma shows that |0, V log de (It) is bounded. The proof before using the relation- ship between o, and m, is identical compared to Lemma C.3 in Bortoli 2022]. For the sake of completeness, we also give the proof process of this part.
Lemma E.4. Assume Assumption
4.1 Then for any t E (0, T] and E IR we have
| V logg: (xx)
(Br/o?) (2 + R²) (R ||x,]))
27

---

Proof. Let N N and E (0, TT.
We denote for any x R", q." (x) = q.h² (x) (2
with
N
K (x) - (1/N) Σ (a),
- (xp]==ep--ls-mxx---- (2o?)]
Next we denote f & log
Then we have
3 log ah (x)
Therefore we have
V.V log le (x)
N
N (x)
Σ (((x)) + (aM
Σ (x)
k=l
k=1
a.5 VfV / (s))(())
k-=1
k.j=1
(
Ed((z)
(1/2) Σ (a.5((a) (vI+()) VV/(()))))))))))))))))))))))))() (()) Σ (o)d()))) AJ=1
k
In what follows, we provide upper bounds for I a.f(l, [V vsll and & V... First we notice that ff((() (z x, /az, and using m, S we get
(x) - (x((l S mn/o < R/0
and
(t) 0,03/ (20;) || m + a.mi/ of (x*, xx*xx-mx*" mm,x*)
Bt. m,, combined with the above
Notice the fact that 0,07 - - =28mm and O.m. equality, we know that
a. (1) - Bemi/o?
(me/o?) m4 m (s 15-mi m, xty
x m- - (m///7) (x mx + 1 x
Bemi/o²
x x (x-mixt, mixx -m+ 1 + m + of x+)
= B
= 3/ (mell + m. || + (1 + m,) (5r, x+1)
where the last equality holds that rm + o² r. The rest of the proof is identical to the Lemma C.3 in Bortoli [2022].
So using m, I we have
(0.5% (x) - (0))( s 28 R² /0 Bem, m1+ R|
2 2 / R(R+|<
Now we compute va,st (x) for any E R
Vo.f. (x) 23m /0 x + (Beme/o?) (1 + m²) x².
28

---

So we can bound the norm of it by
| 2 (R ||=|))
Combining results above we get for any E Rd
| log ah (z)] - (B./ 0 (R+ ||||| + (3//of) R²a (R + ||]l]
- (3./o,) (2 + R²)
(R + |
Note that
lim V.V log 9 N (rt) = 0 V log 4 N t
and the proof complete.
In the following lemma, similar to Chen et al. 2023c we obtain better control the time discretization error instead of controlling |0 log dt (a+) for Vx, E Rd.
Lemma E.5. Assume Assumption 4. and X, satisfies the forward process Equation 3 Define
L= m
0.7
V log -- +
(1+ R22) for then we have that
- (Y
V In
Quk
9r-
TTL2dTk + T L 2 (dr + 122) + LL + (BTdTx + 11277))
Proof. Due to the property of the forward process, we know that if S Rd - IR is the mapping
S(x) = exp(-(t (x)),, then Gr-t4 = ST normal (0, - exp( -2 J.v pl++1 Ba/Tda))
Similar to Chen et al. [203cc], we define a exp [ 8. ds = + 0 and 0² =
exp( 2 Stx
plx+1 B./Tds)) = O(Tāk). Then we can use Lemma C. of Lee et al. 2022 to
obtain
-T (Yoz)
E
V In
Qik
4T
< TL2
- 112 Y 1/4 11 + T V VIngr- (Y,,))
TL2 + L 2 (dr R2) L + 7L T (Brddik + R*T2).
The last inequality follows Lemma F.4]and and the fact that
(Y.)) S V Ingr- (Y 21 + IIV - Vingre (X.))
3 V (Y, L'(dddix + R22
$ L+ (3 + R2
F Auxiliary Lemmas
Lemma F.1. For any s,t E T] we have
1
B du
- log
exp
2
- -1
T- B
(1/2T)
1- exp
-2
du
29

---

Proof. We directly compute
S: BT Or-u/oj--0 du = 7 1.² ! Br-
T
T-u
3.
1 - exp
exexp
2
du
du
7
/ Br-u oxp
T --u
Iu
Bu
du
cxp
2
du
du
T
T
1
T-u
Bu
Oy log
exp
2
du
-1
du.
2
0
Similarly
1. BT mm-
1 {-¹ 3r-uexp exp
T Bo du
/ (1 (1-exp
T'
Bu
J2
-2
exp
-2 j
T du
du
du
7
=(1/27) (-¹ Ou (1 (1-exp
T- Be drl)
exp
exp
2
du.
Lemma F.2. Assume Assumption 3.1
For i E (1,2}, we have ox (1) S 25 and o (1) <
(=2x(() 1
B
Yu [T tk,
81-1
Proof.
T T-xx
3r
(x (1) =
1 - exp
-2
ds
J0
T T-ix
2
B, ds S 25,
J
where the first inequality follows from for any 2 0, expl -a] 2 from Assumption 3.1 and 8 < 1.
the second inequlity follows
1 1111 7
T T-tx
Bo
-1
T-tx
3n T
T-- (1)
1-exp exp
-2
ds
1
2
ds
0
3
84-1
where the first inequality follows from for any a 2 0, (1 1+1/a the second inequal-
ity follows from Assumption 3.1 is easy to check that (i) S OT (i), Vu E [T - Lx, T].
Using the bound on or- (i) immediately yields the following control of B,) Lemma F.3. Assume Assumption 3.1
Then, we have for any u E [T - tx, tx T]: (1) ifi 1, then
Bn
{ + 83
3
$ <x((T) max(3, 2
:
o 1)
(2) if = 2, then
3n
S k2((T) = max 33,3
( 8z
0. (i = 2)
30

---

Generally speaking, T 2 3 2 1. Hence, We can further simplify the above inequality by removing max.
In the rest of this section, we provide the useful lemma to achieve polynomial sample complexity for VE-based models with reverse SDE. As shown in Lemma E.1 also need control x,|| in the forward process. The following lemmas shows that this term is bounded by the R and exploding variance.
Lemma F.4. Suppose that Assumption 4.1 hold. Let (Xt)ee0. denote the forward process Equa- tion Then, for all 2 0,
E
| 1??
do v R
Proof.
As shown in Equation
X
E
X
[|=] + +ofd S do? R².
Lemma FS5 (movement bound for VESDE). Let (Xe)xe(o.7) denote the forward process Equation
For 0 S < t with == t-s, if 8 I 1, then
|X, x.11] & 28,od + R2
Proof.
Xr x.1l] SE ||| (B, (B.BB.BBB)))) + J: E [[1xm1P] dr 225230 23,dd 88
E
Similar to Chen et al. [203cc], we can also show that if we do forward process for time 8, will be close to 90 in W2 distance.
Lemma F6.. Suppose Assumption 4.1 holds.
Let ewz O. 18
12 and T T2, choose
2/a
the early stopping parameter 8 <
<w
Br and T, we choose ò S
<wa
(d+R/o)1/5
(d+R/ojurz
If consider pure VESDE (SMLD) (Equation with of = t, we choose Srz Then we have W2 (46, 90) S EW2
Proof. For the forward process Equation 13
know that X, m,X, + o.Z, where Z 1. Hence, for S 1,
normal (0, Ia) is independent of Xo and m.
W (go, 9s) S (1 - FB [
+ E [|,
For 8, t2 and 2 we have that
W3 (9o. q5) - 8 R28
T
2/3
Hence, can take 8 <
For B t and T. have that
R28
W3 (go, gs) < 8dd +
T2
Hence, we can take 8 <
For pure VESDE (Equation 22 with a; = 1, we have
R/dja/2
W2 (go, q8) < åd.
31

---

G Additional Synthetic Experiments
In this section, we do synthetic experiments show the power of our new forward process with small drift term in different setting.
G.1 The Synthetic experiments with accurate score function
In this section, do numerical experiments on 2-dimension Gaussian distribution show the power of our new VESDE forward process in balancing different error sources.
Experiment Setting.
We set the mean of target distribution E[gu] [6, 8]. the covariance matrix
25 5
Cov[ua]
4- the diffusion time T = 2, T and the reverse beginning distribution is
(0, We choose uniform stepsize 7 h, Ak [K] where h {0.005, 0.01, 0.02, 0.04). For score functions, we directly calculate the ground truth score function instead of learning by the score matching objective. We calculate the KL divergence between the generation distribution and target distribution go as the experiments.
The implementable algorithm. We choose three different VESDE forward processes in the experiments: (1) aggressive 3, = i² with т T2; (2) conservative 8, with = 72 and (3) VESDE without drift term Equation 2 with of = t.. After determining the forward process, we run the reverse SDE with the above 7k, E [K]. For the discretization scheme, we choose two common method: exponential integrator (EI) [Zhang and Chen 2022] and Euler-Maruyama (EM)
discretization [Ho et al. 2020].
The experimental results are shown in Figure 2 We note that the red line (EL, VESDE without drift, of = (²) and orange line (conservative drift VESDE, B, = and r = T has similar trend. Furthermore, the conservative drift VESDE has better performance compared to pure VESDE without drift term. Hence, our new forward process is representative enough to represent current VESDE, discussed in Section 3.1
Observations.
The experimental results also support our theoretical results and show the power the new forward process in balancing different error terms. As shown in Figure the process with aggressive 33 = 12 with small drift term achieves the best and second performance in and EM discretization since can balance the reverse beginning and discretization. The third best process is conservative 3r = t with the small drift term. The reason is that though it can not achieve a exp (-7") forward process guarantee, it also has a constant decay prior information, as shown in Section 3.1 This decay slightly reduces the effect the reverse beginning error. The worse process is VESDE without drift term since it is hard to balance different error sources. Our experimental results also show that EI discretization better than EM discretization.
G.2 The Synthetic experiments with approximated score function
In this section, instead of using an accurate score function, we train an approximated score function on the pure VESDE (Equation (2) without drift term on two synthetic datasets: multiple Swiss rolls and 1-D GMM. Then, for the drift VESDE, we do not train the approximated score corresponding to Equation 3 we directly use the approximated score learned by pure VESDE and show that the drift VESDE can improve the generated distribution without the training process.
Datasets.
The 1-D GMM distribution contains three modes:
3 V (-8, 0.01) + 10 N (-4,00001) 10" VV
3
10
Lai For multiple Swiss rolls, we use similar code compared to Listing of Lai et al. (2023), except Line We change Line 6. to data /=10. to obtain a larger variance dataset. Each dataset contains 50000 datapoints.
The implementable algorithm. In this subsection, we choose two forward processes: (1) conserva- tive B, = 1 with T; (2) pure VESDE without drift term (Equation (2) with of = t. To match
32

---

Table 1: The KL divergence for pure VESDE (Equation 2 and conservative drift VESDE with different sampling method.
1-D GMM
Swiss roll
Forward Process
Reverse SDE
PFODE
Reverse SDE
PFODE
Pure VESDE (T' = 100)
0.082
0.434
9.58
21.05
Drift VESDE (T = =100)
0.043
0.249
8.71
7.77
Pure VESDE (T- = -625)
0.027
0.057
8.00
8.20
Drift VESDE (T -625) 0.025
0.031
7.95
7.21
our analysis, we choose two sampling methods for the reverse process: Fuler-Maruyama method for reverse SDE and RK45 ODE solver for the reverse PFODE method.
We note that although aggressive setting and the experiments with accurate score (Figure 2). other sampling issues may arise in practice. We leave the experimental exploration for drift VESDE with aggressive 3, future work.
t and T has shown its power in theory (Lemma C.2
The training detail. For each dataset, train score funetion with pure VESDE (Equation 2) of =)= t). We train for 200 epochs with batch size 200 and learning rate 10-4. For both training and inference, the start time is 5 = 10-5. For the conservative VESDE, learned by the pure VESDE since the conservative drift VESDE has a similar trend compared to pure VESDE, shown in Figure The above experiments are runned over 5 random seed and present the average over these seeds in Table
directly adapt the checkpoint
The above experiments are conduct a GeForce RTX 4090. It takes 25 minutes to train score function of pure VESDE.
Observation. We experiments with T 100 and lager 625 and these two choice show similar phenomenon. In this paragraph, we first use T = 100 as example to discuss the results. As shown T the conservative drift VESDE has smaller KI. divergene compared to pure VESDE under all sampling methods and datasets. From Figure and Figure it is clear that pure VESDE has low density on the Swiss roll except the center one, which means that though pure VESDE can deal with small B[go]. it is hard to deal with large dataset variance Cov[go], as discuss in Section 4 For conservative drift VESDE (3, and 5 = T), as we discuss in Section 3.1 there a constant decay the prior information E[go] and Cov[go], which is helpful in deal with large dataset mean and variance. The experimental results support our augmentation. Figure (c), Figure 4 (c) and Figure 5 (c) show that the density of the generated distribution is more uniform compared to pure VESDE, which means that the drift VESDE can deal with large dataset mean and variance.
We also do experiments with larger T = 625. As we discuss in Section 4 larger 7 will reduce the influence of the prior data information and have greater generated distribution, as shown Figure (c) and 4 The experiments of ID-GMM (Figure show similar phenomenon compared to the multi Swiss rolls.
G.3 The
Experiments on CelebA 256
After achieving great performance under the synthetic data, we show that our conservative drifted VESDE can improve the results of pure VESDE without training.
Setting. In this experiment, we adapt well-known VESDE implementation (Song et al., 2020b and do experiments on CelebA datasets (size: 256 256 3). More specifically, we use ve/celebahq 226eaahgg________2_ 3:56_ccspp____ aesnpp_continuous continuous checkpoints provided by (Song et al. (020b) and modify the sampling process strictly according to our drifted VESDE. To fair comparsion,
fix the random seed and use the reverse PFODE process. Then, we generate 10000 face images to calculate metrics. We note that when using this checkpoint and pure VESDE pipeline provided by [Song et al.] 2020b), the models would generate almost pure noise with certain probability. Hence, we use an aesthetic predictor [Schuhmann al. 2022 (aesthetic score 2 5.5) to filter the generated images to ensure that the images are clear faces.
33

---

(a) Original Figure
(b) VESDE (T 100)
(c) Dritted VESDE (T 100)
whed Dit VMO7
(d) VESDE (7 625)
(e) Drifted VESDE (7 625)
Figure 4: Experiment results of Swiss roll with reverse PFODE
Visuume Couasion
MCC/E wimglr
500
(a) Original Figure
(b) VESDE (T = 625)
(c) Drifted VESDE (T = 625)
Figure 5: Experiment results of ID-GMM with reverse PFODE
Discussion. From the qualitative perspective, as shown in Figure 3 (Figure and 7). the images generated by our drifted VESDE have more detail (such as hair and beard details). On the contrary. since pure VESDE can deal with large variance, the images generated by pure VESDE appear blurry and unrealistic in these details. From the quantitative results, our drifted VESDE achieves aesthetic score 5.813, and IS 4.174, which is better than the results of baseline pure VESDE (aesthetic score 5,807 and IS: 4.082). In conclusion, the real-world experiments show the potential of our drifted VESDE.
We note that the goal of these experiments is to show that our conservative drifted VESDE is plug- and-play without training instead of achieving SOTA performance. Hence, we focus on the relative improvement compared to the baseline (Song et al. 2020b]. There are two interesting empirical future works. For the conservative drifted VESDE, we will experiments on the SOTA pure VESDE models [Krrras et al. [2022] and improve their results without training. For the aggressive drifted VESDE, since this process makes larger modification compared with the conservative one, we need to train new score function instead of directly using pre-train one achieve better results.
34

---

(a) Pura VESDE (More Examples)
[b] United VESUL (More Examples)
Figure The real-world experiments on CelebA256 dataset (More examples)
(a) VESDE
(b) Drifted VESDE
Figure The real-world experiments on C25 dataset (Detail)
NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: We propose new drifted VESDE forward process and achieve the first polynomial complexity for reverse SDE (Corollary 5.4). For reverse PFODE, we propose the firs quantitative convergence guarantee for VESDE (SOTA) (Theorem 6.1 We also do synthetic experiments to support our results (Section
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals attained by the paper.
2. Limitations
35

---

Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
We discuss future work and limitation at Section
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model sell-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full of assumptions and a complete (and correct) proof?
Answer: [Yes]
Justification: All assumptions, Theorem, Corollary, and proof sketch have been clearly stated in the main content. The detailed proof appears in the appendix.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
36

---

We has shown all experiments detail including dataset and training detail in Appendix G Furthermore, we discuss why these experiments results support our theoretical results in Section 7
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
specific model and empirical evaluation, may
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [No]
Justification: As a theoretical work, we only do simple synthetic experiments to support our results. All detail and the used checkpoint are shown in Appendix G
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (nttps://nips.co cc/
Souidees/CodSuumissttt
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https for more details.
/nips.cc/ppull wttp:/cccpppccccccctcc.cec.cccct otg.aaeeee/oeeeuuuissiiiooooo..o
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
37

---

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Answer: [Yes]
Justification: We has shown all experiments detail including dataset and training detail in Appendix G
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: The results of Figure and Table are calculated over 5 random seed.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation or the standard error of the mean.
It is OK to report 1-sigma error bars, but should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We have shown the compute works and computation time in Appendix G
38

---

Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compule required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics httppp////aaaipss..cppcppl//////
ines?
Answer: [Yes]
Justification: We have checked the code of ethics and make sure that our work satisfies the code of ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive socictal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
We have discussed the broader impacts of our work at the end of main paper.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (c.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Question: Does the paper describe safeguards that have been put in place for responsible release of data models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
39

---

Answer: [NA]
Justification:
This paper poses no such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [NA]
Justification: The paper does not use existing assets.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
The paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/moodll as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
40

---

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country or institution) were obtained?
Question:
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
41


Reviews:
[{"review_id": "DGjqHb7VN5", "reviewer": "Reviewer_ZwJR", "summary": "In this paper, the authors propose an analysis of the convergence of diffusion models under the manifold hypothesis in a similar setting as [1]. The main contribution is the extension of the analysis to the case of VESDE (Variance Exploding SDE) contrary to [1] which is limited to VPSDE (Variance Preserving). The rates obtained by the authors are better than the ones obtained in [1] (although in a different context). They also extend their analysis to ODE samplers which are notably more difficult to deal with than SDE samplers from a theoretical point of view. The decomposition of the error is the same as in [1] but with a more careful analysis of the tangent process (see Section 6 \"The Tangent-based Analysis Framework\"). Experiments in toy settings are presented. \n\n[1] De Bortoli -- Convergence of denoising diffusion models under the manifold hypothesis", "questions": "* l.34 \"Furthermore, Karras et al. [2022] unify two processes and show that the optimal parameters of the general formula correspond to VESDE.\" --> Not clear what the authors are referring to here.\n\n* l.48 \"leads to a large reverse beginning error\" --> I disagree as one could argue that most of the error in diffusion models arises from the approximation of the score. The \"large reverse beginning error\" is a strong statement here. \n\n* One relevant work that is not discussed is [1]\n\n* l.296 \"Furthermore, we emphasize that our tangent-based unified framework is not a simple extension of Bortoli [2022].\" --> Can the authors provide more details here?\n\n* Since there is a one-to-one mapping between VESDE and VPSDE could the authors have leveraged this connection? There is also a connection with Stochastic Localization as pointed out by [2]. Below we explicit the connection between VESDE and VPSDE.\n\nAssume that $(X_t)_{t \\geq 0}$ satisfies a VESDE $\\mathrm{d} X_t = g(t) \\mathrm{d} B_t$ then we have that $(Y_t)_{t \\geq 0}$ given for any $t \\geq 0$ by $Y_t = \\exp[F(\\phi(t))] X_{\\phi(t)}$ satisfies $\\mathrm{d} Y_t  = F(\\phi)'(t) Y_t \\mathrm{d}_t  + \\exp[F(\\phi(t))] \\phi'(t)^{1/2} \\mathrm{d} B_t$.\n\n[1] Conforti et al. -- \"Score diffusion models without early stopping: finite Fisher information is all you need\"\n\n[2] Montanari -- \"Sampling, Diffusions, and Stochastic Localization\"", "limitations": "Limitations are not really addressed in Section 8 (\"Conclusion\") in the paragraph \"Future Work and Limitation\". I think a more in depth discussion of the benefits of VESDE and VPSDE is needed.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "* These results are the first results obtained for the convergence of diffusion models in the VESDE setting under the manifold hypothesis. \n\n* The analysis of the tangent process represents an improvement over the results of [1]. This is an interesting development in itself.\n\n* The introduction of the drifted VESDE (Equation 5) is interesting and represents a good avenue for future studies of the VESDE process. \n\n\n[1] De Bortoli -- Convergence of denoising diffusion models under the manifold hypothesis", "weaknesses": "* Experiments are only toyish. I actually don't think they benefit the paper. This is mostly theoretical work and I'm struggling to understand what point is made here. If this is to illustrate the validity of the samplers this is already well established. If the point of the experiment is to illustrate the benefit of drifted VESDE then I would have appreciated a more challenging setting (like CIFAR10 or Imagenet in image processing or larger models like Stable Diffusion). This does not require pretraining a large model since this is a modification of the sampler.\n\n* There is actually not a lot of discussion on how drifted VESDE relate to VESDE. Can one obtain convergence results for  VESDE (classical) based on drifted VESDE?\n\n* As of now, the analysis is limited to drifted VESDE. It would be interesting to analysis if the improved results can transfer to the VPSDE framework and improve on the results of [2].\n\n* The paper is easy to follow and clearly presented. \n\n[1] Karras et  al. -- Elucidating the Design Space of Diffusion-Based Generative Models\n\n[2] De Bortoli -- Convergence of denoising diffusion models under the manifold hypothesis", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your valuable comments and suggestions. We provide our response to each question below.\n\n**W1: The real-world experiments.**\n\nWe do experiments on the CelebA 256 and show that our drifted VESDE improves pure VESDE **without training** from the quantitative and qualitative perspectives. Please see the experiment details in the **global rebuttal**.\n\n**W2: The link between drifted VESDE and VESDE and universe error analysis.**\n\n(i) For reverse PFODE, our analysis holds for pure VESDE with $\\sigma_t^2=t$ or $t^2$, which covers the SOTA VESDE.\n\n(ii) For reverse SDE, we extend our general drifted VESDE formula and provide a universe complexity for general $\\tau \\in [1,+\\infty)$ and $\\beta_t\\in [1,t^2]$. \n\n$$\n\\begin{align}\n    \\frac{\\bar{D} \\sqrt{m_t}}{\\sigma_T}+\\frac{R^2\\sqrt{d}}{\\sigma_\\delta^4}\\sqrt{\\bar{\\gamma}_K\\beta_T\\tau T}+\\epsilon\\_{\\text{score}}\\sqrt{\\beta_TT},\n\\end{align}\n$$\nwhere $m_t=\\exp \\left(-\\int_0^t \\beta_s/\\tau \\mathrm{~d} s\\right)$ and $\\sigma_t^2=\\tau\\left(1-m_t^2\\right)$.\n\n(a) The general formula covers current models (including VE and VPSDE). \n\nWhen $\\beta_t=1$ and $\\tau=1$, the drifted VESDE becomes VP and $m_t=\\exp{(-T)}$, which leads to a logarithmic $T$ and achieve $\\tilde{O}(1/\\epsilon\\_{W_2}^8\\epsilon\\_{TV}^2)$ (the same with [1]). When $\\beta_t=1$ and $\\tau =T$, our formula is similar but slightly better (Fig. 2 and real-world experiments) to pure VESDE (\\sigma_t^2= t) and achieves $1/\\epsilon\\_{W_2}^8\\epsilon\\_{TV}^6$ results ([2] achieve $1/\\epsilon\\_{W_2}^8\\epsilon\\_{TV}^4$ since they assume strong LSI holds). For $\\beta_t=t$ and $\\tau =T^2$, the formula is similar to SOTA  pure VESDE ($\\sigma_t^2=t^2$) and achieves the first polynomial results $1/\\epsilon\\_{W_2}^8\\epsilon\\_{TV}^6$ under the manifold hypothesis. We also note that the above results holds for pure VESDE with $\\sigma_t^2= t$ and $t^2$.\n\n(b)  Go beyond: When given a $\\beta_t$, the optimal $\\tau$ has the same order with $\\beta_T$ for reverse SDE.\n\nThe key part of analysis is balancing the reverse beginning and discretization error (please see approximated score in Q2). As in (a), pure VESDE has a worse $\\epsilon_{TV}$ than VP, which comes from the large reverse beginning term. For example, if $\\beta_t=t$ and $\\tau =T^2$, $m\\_T= e^{-1/2}$ and $\\sigma\\_T^2=(1-e^{-1})T$, which leads a polynomial $T$ and heavily influence the discretization term. Hence, the optimal choice of $\\tau=T$ instead of $T^2$ and $m\\_T=e^{-T}$. Then, we achieve the same guarantee with VPSDE [1]. Similarly, the optimal $\\tau$ is $T^2$ for $\\beta_t=t^2$ (Thm. 5.2).\n\n**W3: The improved results for [3].** \n\n[3] considers VPSDE with the reverse SDE and achieves a guarantee with an exponential term $\\exp{(1/\\delta)}$. [1] achieve a pure $W_2$ guarantee $1/\\epsilon_{W_2}^{12}$ by using a projection technique=, which is a direct improvement of [2]. Our general formula can also recover this result (W1), although the main point of our work is not VPSDE.\n\n**Q1: Discussion on Karras et al.** \n\nThis work unifies the reverse PFODE of VP and VESDE (Eq. 4 of their work) and proves that the ODE solution trajectory of VESDE ($\\sigma_t^2=t^2$) is linear and directly towards the data manifold (Fig. 3 of their work). On the contrary, the trajectories of VP and VESDE ($\\sigma_t^2=t$) are not linear in most regions, which makes the denoise process difficult.\n\n**Q2: The error terms.**  \n\nThe reverse beginning, discretization, and approximated score error are both important. Since the sampling and learning process are relatively independent, current works usually decouple these parts. For the sample process, previous works assume an $L_2$ accuracy score [1] [3]. For the learning process, some works analyze how to use a NN to learn score [4]. We will add a discussion about error terms. \n\n**Q3: The discussion of Conforti et al.**  \n\nThis work considers VPSDE with reverse SDE under the finite fisher information assumption. Though this work relies heavily on the property of OU process, it is an interesting future work to analyze whether the connection in Q5 can be used to improve the results of VESDE. We also note that our work analyzes a broader area (reverse SDE and PFODE). We will add a discussion.\n\n**Q4: The novelty of our tangent-based method.** \n\nThe technique novelty is our tangent-based lemma. For the PFODE, Lem. 6.3 contains $\\exp{(\\int_{0}^{t_K}\\beta_{T-u}/\\tau du)}$. For VPSDE ($\\beta_t =1$ and $\\tau =1$), there is an additional $\\exp{(T)}$, which indicates that the previous lemma can't deal with reverse PFODE even under the VPSDE setting. To avoid this term, we use the variance exploding property of VESDE (for example, the above term is a constant for $\\beta_t=t$ and $\\tau = T^2$) and achieve polynomial $T$ in final results. \n\n**Q5: The connection between VP and VESDE.**\n\nAs shown in Montanari, VE and VPSDE are equivalent to a change of time, which indicates the discretization analysis of these models is similar under the reverse SDE. Our universal analysis (W2) also reflects this phenomenon. However, the other key point is the balance of the first two terms, and pure VESDE performs badly. Hence, we further propose the general drifted VESDE formula, prove the optimal choice of $\\tau$ faces a $\\beta_t$, and provide better results. We will make the discussion of W1 and Q5 clearer. \n\n**Limitation.**\n\nThanks for the comments on Limitation. We will discuss the benefit of our drifted VESDE formula (including VPSDE and pure VESDE) and its potential to achieve the SOTA performance.\n\n[1] Chen et al. Sampling is as easy as learning the score: theory for diffusion models with minimal data assumptions. ICLR 2023.\n\n[2] Lee et al. Convergence for score-based generative modeling with polynomial complexity. NeurIPS 2022.\n\n[3] De Bortoli, V. Convergence of denoising diffusion models under the manifold hypothesis. TMLR.\n\n[4] Chen et al.. Score approximation, estimation and distribution recovery of diffusion models on low-dimensional data. ICML 2023.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Z5p3Kl8ply", "reviewer": "Reviewer_a6C8", "summary": "This paper focuses on variance exploding (VE) based diffusion models and proposes a drifted VESDE forward process with an unbounded diffusion coefficient. This choice of coefficients allows an exponential-decay forward convergence rate, and the authors establish the first polynomial sample complexity for VE-based models with reverse PFODE. Moreover, the authors propose a tangent-based unified analysis framework with reverse SDE and PFODE and prove the first quantitative guarantee for SOTA VE-based models with reverse PFODE.", "questions": "1. In Assumption 3.1, why for reverse SDE and PFODE, the choices of $\\beta_t$ are different? Could the authors provide some intuitive explanation?\n\n2. In Theorem 5.2, the sample complexity has the same dependence on $\\epsilon_{W_2}$ and $\\epsilon_{TV}$ as that in Chen et al. (2023c). However, the dependence on $d$ is worse. Is it a consequence of a more aggressive $\\beta_t$? \n\n3. Could the theoretical results reflect the superiority of VE-based diffusion models over VP-based diffusion models?", "limitations": "The authors discuss future work and limitations in Section 8.", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "In terms of originality, this paper proposes a new variance exploding (VE) based diffusion model and establishes the corresponding convergence guarantees. The theoretical results are solid. Moreover, this paper is well-organized and clearly written.", "weaknesses": "1. The convergence guarantees for VE-based models with reverse PFODE are relatively weak. For example, in Assumption 3.1, the choice of $\\beta_t$ is more conservative for reverse PFODE; In Theorem 6.2 part (2), the last term is $\\bar{D}/\\tau$ instead of $\\bar{D} e^{-T/2} / \\sqrt{\\tau}$ in part (1). Does this mean that the $e^{-T}$ forward convergence rate can only achieved by the reverse SDE?\n\n2. The numerical results only include synthetic experiments.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your valuable comments and suggestions. We provide our response to each question below.\n\n**W1 & Q1: The different $\\beta_t$ for reverse SDE and PFODE: the balance between different error terms.**\n\n(a) We first recall the reverse beginning error term when considering the unified tangent-based framework:\n$$\n\\begin{align}\nW_1\\left(Q_{t_K}^{q_{\\infty}^{\\tau}},Q_{t_K}^{q_0P_T}\\right)\\leq \\frac{\\sqrt{m_T}\\bar{D}}{\\sigma_T}\\exp\\left(\\frac{R^2}{2\\sigma_{T-t_K}^{2}}+ \\frac{(1-\\eta^2)}{2}\\int_0^{t_K}\\frac{\\beta_{T-u}}{\\tau}\\mathrm{d}u\\right)\\,.\n\\end{align}\n$$\nWe note that the forward and reverse processes determine the above bound simultaneously, where the exponential terms come from the bound of the tangent process (Lem. 6.3, reverse process), and the first part comes from Thm. 4.2 (forward process). \n\nFor the reverse SDE ($\\eta=1$), the exponential term becomes $\\exp\\left(\\frac{R^2}{2\\sigma_{T-t_K}^{2}}\\right)$, which is independent with $\\beta\\_t$. Hence, choosing an aggressive $\\beta_t=t^2$  (here we use $\\tau =T^2$ as an example) will introduce $\\frac{\\bar{D}e^{-T/2}}{\\tau} \\exp\\left(\\frac{R^2}{2\\sigma\\_{T-t_K}^{2}}\\right)$ without introducing additional terms.\n\nFor the reverse PFODE ($\\eta=0$), we can also choose an aggressive $\\beta_t=t^2$, which leads to a $\\bar{D}e^{-T/2}/\\tau$ for $\\frac{\\sqrt{m_T}\\bar{D}}{\\sigma_T}$. However, since $\\eta = 0$, $\\exp{\\int_0^{t_K}\\frac{\\beta\\_{T-u}}{\\tau}\\mathrm{d}u}$ term  will introduce an additional $e^{\\frac{T}{6}}$ to the bound of the tangent process. **We note that the key part of the convergence guarantee is to balance the discretization, reverse beginning, and early stopping error terms.** Though the reverse beginning still enjoys an $e^{-T/4}\\exp\\left(\\frac{R^2}{2\\sigma\\_{T-t_K}^{2}}\\right)$, the aggressive $\\beta_t$ will introduce an additional  $e^{\\frac{T}{6}}$ in the final result (since the tangent process also influences the discretization error term.). Hence, a better choice for the reverse PFODE is a conservative $\\beta_t$.\n\n(b) An interesting future work: the PFODE predictor and suitable corrector.\n\nThe above discussion shows that since the reverse beginning error is determined by the forward and reverse process at the same time, an aggressive $\\beta_t$ can not be used under the PFODE setting. The complex dependency is due to the fact that data processing inequality is forbidden when considering the Wasserstein distance. As a next step, we discuss how to improve the results of Thm. 6.1 with an aggressive $\\beta_t$ (the PFODE setting).\n\nWe first recall the data processing inequality: Consider a channel that produces $Y$ given $X$  based on the law $P_{Y \\mid X}$. Let $P_Y$ be the distribution of $Y$ when $X$ is generated by $P_X$ and $Q_Y$ be the distribution of $Y$ when $X$ is generated by $Q_X$. Then we know that for any $f$-divergence $D_f(\\cdot \\| \\cdot)$, $D_f\\left(P_Y \\| Q_Y\\right) \\leq D_f\\left(P_X \\| Q_X\\right)$.\n\nWhen choosing $f(x)=\\frac{1}{2}|x-1|$, the $f$-divergence is TV distance. Hence, by viewing $Q_{t_K}$ as the channel, the inequality $\\operatorname{TV}\\left(Q\\_{t_K}^{q\\_{\\infty}^\\tau}, Q\\_{t_K}^{q_T^\\tau}\\right)\\leq \\operatorname{TV}\\left(q\\_T^\\tau, q\\_{\\infty}^\\tau\\right)$ holds, which indicates that for the $\\mathrm{TV}$ distance, the influence of reverse process can be ignored when considering the reverse beginning error term. However, the data processing inequality does not hold for Wasserstein distance. To overcome this problem, an interesting future work is to introduce a suitable corrector (such as the Underdamped Langevin process in [1]) to inject some small noise into the PFODE predictor, which allows the use of the data processing inequality and achieve a polynomial sample complexity.\n\nWe will add a discussion to make it clearer.\n\n**W2: The real-world experiments on CelebA 256.**\n\nWe do experiments on the CelebA 256 dataset (a common face dataset) and show that our drifted VESDE can improve the results of pure VESDE **without training** from the quantitative and qualitative perspectives. Please see the experiment detail, discussion, and generated images in the **global rebuttal** part.\n\n**Q2: The dependence on $d$ and $R$.**\n\nWe recall that the result shown in [2] is $\\tilde{O}\\left(\\frac{d^3 R^4(R \\vee \\sqrt{d})^4}{\\varepsilon_{\\mathrm{TV}}^2 \\varepsilon_{W_2}^8}\\right)$ and our results is $\\tilde{O}\\left(\\frac{dR^4(d+R\\sqrt{d})^4}{\\epsilon\\_{W_2}^{8}\\epsilon_{\\text{TV}}^2}\\right)$. Since the image datasets are usually normalized into $[-1,1]^d$, $R\\leq \\sqrt{d}$. Hence, our results have the same order as [2]. We will add a discussion part about the dependence on $d$ and $R$.\n\n**Q3: The superiority of VE-based models over VP-based models.**\n\nWhen considering the reverse PFODE, we show the superiority of VE-based models over VP-based models. More specifically, Lem. 6.3 contains $\\exp{(\\int_{0}^{t_K}\\beta\\_{T-u}/\\tau du)}$ for the reverse PFODE. For VPSDE ($\\beta_t =1$ and $\\tau =1$), there is an additional $\\exp{(T)}$. On the contrary, VE-based models make use of the variance exploding property of VESDE, avoid this term (for example, the above term is a constant for $\\beta\\_t=t$ and $\\tau = T^2$), and achieve polynomial $T$ dependence in final results. We note that our results also hold for VESDE ($\\sigma\\_t^2= t^2$), the SOTA models proposed by [3]. We will add a discussion part in the next version.\n\n\n\n[1] Chen, S., Chewi, S., Lee, H., Li, Y., Lu, J., & Salim, A. (2024). The probability flow ode is provably fast. *Advances in Neural Information Processing Systems*, *36*.\n\n[2] Chen, S., Chewi, S., Li, J., Li, Y., Salim, A., & Zhang, A. R. (2022). Sampling is as easy as learning the score: theory for diffusion models with minimal data assumptions. *arXiv preprint arXiv:2209.11215*.\n\n[3] Karras, T., Aittala, M., Aila, T., & Laine, S. (2022). Elucidating the design space of diffusion-based generative models. *Advances in neural information processing systems*, *35*, 26565-26577.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "llIIZZkhVN", "reviewer": "Reviewer_Duca", "summary": "The paper analyzes the Variance Exploding diffusion model under the manifold hypothesis. By a slight modification to the VESDE process, the authors propose a method whose convergence guarantees are better than prior best known rates in this regime.", "questions": "The final rates are only given in total variation. Is it possible to improve the metric to KL or to state W2 bounds with similar complexities? What is the analytical challenge otherwise?\n\nTheorems 6.1 and Corollary 6.2 could have their exposition simplified in the main text; the intuition provided in the subsequent section is much clearer.\n\nMinor:\n\nThe font used for \"KL\" is not consistent in the equations in the appendix.\n\n97: about data -> about the data\n\n102: score function -> the score function\n\n107: with strong LSI -> with a strong LSI\n\n108: assume the Lipschitz score -> assume the score is Lipschitz\n\n108: first work focus -> first work to focus\n\n138: introduces -> introduce\n\n187: \"reversing\" does not make sense here. Perhaps \"initial distribution of the reverse process\" is meant\n\n216: \"does reverse\" -> \"which reverse the process\"\n\n345: \"support\" -> \"supports\"\n\n364: \"support our\", \"show that\" -> \"supports our\", \"shows that\"", "limitations": "The authors have already addressed all major limitations of their work.", "rating": 7, "confidence": 2, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "The rate obtained is state-of-the-art for Variance Exploding models, which are notorious for being difficult to analyze.\n\nAlthough the paper borrows parts of its analytic framework from the work of Bortoli et al. (2022), the adaptation of the analysis in this context appears to be non-trivial.\n\nThe paper is clearly written and the exposition of the proof is overall quite clear.", "weaknesses": "The experiments in the main text and appendix are not particularly thorough. This is acceptable since the primary claimed contribution by this article is theoretical, but some kind of practical benchmark might also be beneficial.\n\nThe dimension and diameter dependence are quite severe. Is there any sense of how close these parameters are to their optimal values?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your valuable comments and suggestions. We provide our response to each question below.\n\n**W1: The real-world experiments on CelebA 256.**\n\nWe do experiments on the CelebA 256 dataset (a common face dataset) and show that our drifted VESDE can improve the results of pure VESDE **without training** from the quantitative and qualitative perspectives. Please see the experiment detail, discussion, and generated images in the **global rebuttal** part.\n\n**W2: The discussion on the diameter.**\n\nSince there is no existing work to discuss the lower bound of the diameter of VE-based models, we start from the VP-based models and discuss the improvement space of VE-based models to achieve the same order results compared with VP-based models.\n\nFor the reverse SDE, VP-based models achieve an optimal $d$ dependence by using the stochastic localization technique and exponential-decay stepsize [1]. As a first step, we use a uniform stepsize for VE-based models, which leads to a slightly worse dependence on $R$ and $d$. It is an interesting future work to use a more refined, time-dependent stepsize to achieve improved results for VESDE and show that VESDE performs better than VPSDE from the theoretical perspective.\n\nFor the reverse PFODE,  since the Wasserstein distance can not use the data processing inequality, Thm. 6 has an exponential dependence on $R$. As discussed at the end of Sec. 6, it is possible to introduce a suitable corrector (such as the Underdamped Langevin process in [2]) to inject some small noise into the PFODE predictor, which allows the use of the data processing inequality and replaces the exponential $R$ with a polynomial one.\n\nWe will add a discussion paragraph on the diameter to make it clearer.\n\n**Q1: The guarantee under stronger metric.**\n\n(a) The pure $W_2$ guarantee.\n\nSince we assume the manifold hypothesis, we first show how to obtain a pure $W_2$ guarantee at each setting. For the reverse SDE setting, similar to Corollary 5 of [3], by using the projection technique, we can achieve pure $W_2$ guarantee $\\tilde{O}(1/\\epsilon\\_{W\\_2}^{12})$, which has the same order with [3]. We note that the slightly worse $\\epsilon$ dependence of our work and [3]  is due to  the relationship between $W_2$ and $\\mathrm{TV}$ [4]:\n\n$$\nW\\_2(R\\_K^{q^{\\tau}_{\\infty}},q\\_{\\delta})\\leq R\\sqrt{\\mathrm{TV}(R\\_K^{q^{\\tau}\\_{\\infty}},q\\_{\\delta})}+R\\exp{(-R)}.\n$$\nFor the reverse PFODE setting, similar to the tangent-based method for VPSDE [5], our results can extend to $W_p$ for any $p\\ge 1$. \n\n(b) The $\\mathrm{KL}+W_2$ guarantee.\n\nWhen considering reverse SDE setting, similar to [6], we can use the chain rule of $\\mathrm{KL}$ divergence instead of the triangle inequality to obtain a $\\mathrm{KL}+W_2$ guarantee $\\tilde{O}(1/\\epsilon_{\\mathrm{KL}}^2\\epsilon\\_{W\\_2}^8)$.\n\nWe will discuss the guarantee under stronger metrics in detail.\n\n**Q2 and Minor Question: the presentation.** \n\nThanks for your helpful comments on the presentation. For our tangent-based unified framework part, we will simplify the formula of Thm. 6.1 and Coro. 6.2 and highlight the technique novelty. For the typos, we will polish our presentation according to your comments.\n\n\n\n[1] Benton, J., De Bortoli, V., Doucet, A., & Deligiannidis, G. (2023). Linear convergence bounds for diffusion models via stochastic localization. *arXiv preprint arXiv:2308.03686*.\n\n[2] Chen, S., Chewi, S., Lee, H., Li, Y., Lu, J., & Salim, A. (2024). The probability flow ode is provably fast. *Advances in Neural Information Processing Systems*, *36*.\n\n[3] Chen, S., Chewi, S., Li, J., Li, Y., Salim, A., & Zhang, A. R. (2022). Sampling is as easy as learning the score: theory for diffusion models with minimal data assumptions. *arXiv preprint arXiv:2209.11215*.\n\n[4] Rolland, P. T. Y. (2022). *Predicting in uncertain environments: methods for robust machine learning* (No. 9118). EPFL.\n\n[5] De Bortoli, V. (2022). Convergence of denoising diffusion models under the manifold hypothesis. *arXiv preprint arXiv:2208.05314*.\n\n[6] Chen, H., Lee, H., & Lu, J. (2023, July). Improved analysis of score-based generative modeling: User-friendly bounds under minimal smoothness assumptions. In *International Conference on Machine Learning* (pp. 4735-4763). PMLR.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "LsmRb6s6Ct", "reviewer": "Reviewer_bHCt", "summary": "Diffusion models are powerful tools in generative modeling.  As the paper pointed out, very few theoretical works in the literature\nconsider variance exploding diffusion models. Among those works, forward convergence rate $1/\\text{poly}(T)$ is achieved compared to $\\exp(-T)$ from the variance preserving models.  This paper proposes a drifted variance exploding diffusion model that allows a faster $\\exp(-T)$ forward convergence rate.  With this process, the polynomial sample complexity for a series of variance exploding models is achieved under the manifold hypothesis.  In addition to the reverse SDE, probability flow ODE is a popular alternative in the literature. The paper considers a more general setting and proves a convergence guarantee with probability flow ODE.", "questions": "(1) You studied both total variation distance and 1-Wasserstein distance. Since your data is supported on a compact domain, would it be possible to study more general $p$-Wasserstein distance guarantees?\n\n(2) After Assumption 3.1. you wrote that the choice $\\tau\\in[T,T^{2}]$ is used to guarantee the exploding variance of the forward process. This is not that accurate. I understand when $\\tau$ is large, the mean-reverting effect of the forward SDE gets very weak, and the variance tends to grow. But that does not explain why you need $\\tau\\leq T^{2}$. I think you should also add some discussions on why you need $\\tau\\leq T^{2}$.", "limitations": "Limitations have been discussed in the conclusion section.", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "(1) The paper proposes a so-called drifted variance exploding forward process and to the best of my knowledge this is new. \n\n(2) When the data is supported on a compact set, the paper manages to derives convergence guarantees in total variation distance, and 1-Wasserstein distance.\n\n(3) The analysis seems to be rigorous.", "weaknesses": "(1) The model seems to be an interplay between the VE-SDE model and VP-SDE model  in the literature in the sense that when $\\tau=1$, it recovers the VP-SDE model and then $\\tau=\\infty$, it recovers the VE-SDE model. To better compare the model with the VP-SDE and VE-SDE models in the literature, I am wondering whether it is better to assume $\\tau=[1,\\infty]$ and for a fixed $\\tau$, to see what is the convergence guarantees and how it depends on $\\tau$? If you do that, will you find the optimal range of $\\tau$ to be $[T,T^{2}]$ which is what proposed in the paper? My guess is that for given $\\beta_{t}$, the optimal choice of $\\tau$ should depend on $\\beta_{t}$ instead of being in the range $[T,T^{2}]$.\n\n(2) Since the model proposed is a new model, it would be even more convincing if the paper can include some experiments beyond synthetic experiments.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your valuable comments and suggestions. We provide our response to each question below.\n\n**Weakness 1: The universe error analysis for general $\\tau$ and $\\beta_t$.**\n\nIn this part,  we first provide a universe complexity for general $\\tau \\in [1,+\\infty)$ and $\\beta_t\\in [1,t^2]$ under the reverse SDE, which covers current diffusion models. Then, we discuss the influence of different $\\tau$ and $\\beta_t$ in detail. \n$$\n\\begin{align}\n    \\frac{\\bar{D} \\sqrt{m_t}}{\\sigma_T}+\\frac{R^2\\sqrt{d}}{\\sigma_\\delta^4}\\sqrt{\\bar{\\gamma}_K\\beta_T\\tau T}+\\epsilon\\_{\\text{score}}\\sqrt{\\beta_TT},\n\\end{align}\n$$\nwhere \n$$\n\\begin{align}\nm_t=\\exp \\left(-\\int_0^t \\beta_s/\\tau \\mathrm{~d} s\\right)\\, \\text{and } \\sigma_t^2=\\tau\\left(1-m_t^2\\right)\\,.\n\\end{align}\n$$\n(a) The general formula covers current diffusion models. \n\nIn this paragraph, we show that our general formula can cover current models and provide the sample complexity. The key part of the sample complexity is balancing the first two terms: the reverse beginning and the discretization term. When $\\beta_t=1$ and $\\tau=1$, the drifted VESDE becomes VPSDE and $m_t=\\exp{(-T)}$, which leads to a logarithmic $T$ and would not influence discretization term heavily. Then, we achieve $\\tilde{O}(1/\\epsilon\\_{W_2}^8\\epsilon\\_{TV}^2)$, which has the same order with [1]. When $\\beta_t=1$ and $\\tau =T$, our formula is similar but slightly better (as shown in Figure 2 and our real-world experiments) to pure VESDE (\\sigma_t^2= t) and we can achieve $\\tilde{O}(1/\\epsilon\\_{W_2}^8\\epsilon\\_{TV}^6)$ results ([2] achieve slightly better results $1/\\epsilon\\_{W_2}^8\\epsilon\\_{TV}^4$ since they assume strong LSI assumption). When considering $\\beta_t=t$ and $\\tau =T^2$, the general formula is similar to pure SOTA VESDE (\\sigma_t^2=t^2) and achieve the first polynomial sample complexity $\\tilde{O}(1/\\epsilon\\_{W_2}^8\\epsilon\\_{TV}^6)$ under the manifold hypothesis. We also note that the above results holds for pure VESDE with $\\sigma_t^2= t$ and $t^2$.\n\n(b)  When given a fixed $\\beta_t$, the optimal $\\tau$ has the same order with $\\beta_T$.\n\nAs shown in (a), pure VESDE has a worse $\\epsilon_{TV}$ dependence compared to VPSDE, which comes from large reverse beginning terms (the first term). For example, when considering $\\beta_t=t$ and $\\tau =T^2$, $m\\_T= e^{-1/2}$ and $\\sigma\\_T^2=(1-e^{-1})T$, which leads a polynomial $T$ and heavily influence the second discretization term. Hence, the optimal choice of $\\tau=T$ instead of $T^2$. With $\\tau = T$, $m\\_T=\\exp{(-T)}$ and the complexity is $\\tilde{O}(1/\\epsilon\\_{W_2}^8\\epsilon\\_{TV}^2)$ (Thm. 5.2), which has the same with VPSDE (this result also shows that the choice of VPSDE is optimal.). For $\\beta_t=t^2$, the optimal $\\tau$ is $T^2$, which has the same order with $\\beta_T$.\n\nWe will make the above discussion clearer in the next version. \n\n**Weakness 2: The real-world experiments on CelebA 256.**\n\nWe do experiments on the CelebA 256 dataset (a common face dataset) and show that our drifted VESDE can improve the results of pure VESDE **without training** from the quantitative and qualitative perspectives. Please see the experiment detail, discussion, and generated images in the **global rebuttal** part.\n\n\n\n**Q1: The general $W_p$ distance.**\n\nFor the reverse SDE setting, similar to Corollary 5 of [1], by using the projection technique, we can achieve pure $W_2$ guarantee $\\tilde{O}(1/\\epsilon\\_{W\\_2}^{12})$, which has the same order with [1]. \n\nFor the reverse PFODE setting, similar to the tangent-based method for VPSDE [3], our results can extend to $W_p$ for any $p\\ge 1$. We will make it clearer in the next version.\n\n\n\n**Q2: The choice of $\\tau$.**\n\nThanks for the helpful comment on our general formula. As shown in Weakness 1, our general formula can consider $\\tau\\in [1,+\\infty)$ and achieve polynomial sample complexity under the reverse SDE. We also show that our formula can recover current VPSDE and pure VESDE models and go beyond with $\\tau \\in [1,T^2]$. \n\nFor the reverse PFODE, as shown in Lem. 6.3, if considering   $\\tau =1$ (the VPSDE setting), there would be an additional $\\exp{(T)}$ term. To avoid this term, we need to use the variance exploding property of VESDE. Hence, we choose $\\tau\\in [T,T^2]$, which represents two common VESDE choices under reverse PFODE (We note that Thm. 6.2 and Coro. 6.3 also hold for pure VESDE.)  \n\nWe will make it clearer in the next version.\n\n\n\n[1] Chen, S., Chewi, S., Li, J., Li, Y., Salim, A., & Zhang, A. R. (2022). Sampling is as easy as learning the score: theory for diffusion models with minimal data assumptions. *arXiv preprint arXiv:2209.11215*.\n\n[2] Lee, H., Lu, J., & Tan, Y. (2022). Convergence for score-based generative modeling with polynomial complexity. *Advances in Neural Information Processing Systems*, *35*, 22870-22882.\n\n[3] De Bortoli, V. (2022). Convergence of denoising diffusion models under the manifold hypothesis. *arXiv preprint arXiv:2208.05314*.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
# The Real-World Experiments and Discussion (CelebA 256)

Once again, we thank all reviewers for their valuable suggestions on real-world experiments. In this part, we show that our conservative drifted VESDE can improve the quantitative results (IS (higher is better), and Aesthetic score [1] (1-10, higher is better)) of pure VESDE **without training** on the CelebA256 dataset (a human face dataset). From the qualitative perspective, similar to our synthetic experiments (Sec. 7.2), we observe that drifted VESDE can generate more details compared to pure VESDE.

(a) Setting. In this experiment, we adapt well-known VESDE implementation [2] and do experiments on CelebA datasets (size: 256\*256\*3 ). More specifically, we use ve/celebahq_256_ncsnpp_continuous checkpoints provided by [2] and modify the sampling process strictly according to our drifted VESDE. To do a fair comparsion, we fix the random seed and use the reverse PFODE process. Then, we generate 10000 face images to calculate the metrics. We note that when using this checkpoint and pure VESDE pipeline provided by [2], the models would generate almost pure noise with a certain probability. Hence, we use an aesthetic predictor [1] (aesthetic score>=5.5) to filter the generated images to ensure that the images are clear faces.

(b) Discussion. From the qualitative perspective, as shown in the experiment results (**please click our PDF to see the generated images**), the images generated by our drifted VESDE have more detail (such as hair and  beard details). On the contrary, since pure VESDE can not deal with large variance, the images generated by pure VESDE appear blurry and unrealistic in these details. From the quantitative results, our drifted VESDE achieves aesthetic score **5.813**, and IS **4.174**, which is better than the results of baseline pure VESDE (aesthetic score 5.807 and IS: 4.082).

In conclusion, the real-world experiments show the potential of our drifted VESDE, and we will make it clearer in the next version of the paper.

We note that the goal of these experiments is to show that our conservative drifted VESDE is plug-and-play without training instead of achieving a SOTA performance. Hence, we focus on the relative improvement compared to the baseline [2]. There are two interesting empirical future works. For the conservative drifted VESDE,  we will do experiments on the SOTA pure VESDE models [3] and improve their results without training. For the aggressive drifted VESDE, since this process makes a larger modification compared with the conservative one, we need to train a new score function instead of directly using a pre-train one to achieve better results. We will add these discussions to the future work paragraph.

[1]  Christoph Schuhmann. Laion-aesthetics. 2022.

[2]  Song, Y., Sohl-Dickstein, J., Kingma, D. P., Kumar, A., Ermon, S., & Poole, B. (2020). Score-based generative modeling through stochastic differential equations. *arXiv preprint arXiv:2011.13456*.

[3] Karras, T., Aittala, M., Aila, T., & Laine, S. (2022). Elucidating the design space of diffusion-based generative models. *Advances in neural information processing systems*, *35*, 26565-26577.

Meta-review: The paper received positive reviews from most reviewers and the several concerns have been clarified during the rebuttal period. After discussing with reviewer bHCt, I have come to the conclusion that the paper merits an acceptance, given all the promised changes/clarifications being implemented. Please make sure you carefully incorporate all the reviews/discussions into account.
Decision: Accept (poster)