Improved Discretization Complexity Analysis of Consistency Models: Variance Exploding Forward Process and Decay Discretization Scheme
Authors: Ruofeng Yang
Bo Jiang
Cheng Chen
Shuai Li
Venue: ICML 2025 poster (2025)
Keywords: Consistency Models
Discretization Complexity
Primary area: deep_learning->generative_models_and_autoencoders
TL;DR: The state-of-the-art discretization complexity for consistency models.

Abstract: Consistency models, a new class of one-step generative models, have shown competitive performance with multi-step diffusion models. The most challenging part of consistency models is the training process, which discretizes the continuous diffusion process into $K$ steps and trains a one-step mapping function on these discretized timepoints. Despite the empirical success, only a few works focus on the discretization complexity $K$, and their setting is far from that of empirical works. More specifically, the current theoretical works analyze the variance preserving (VP) diffusion process with a uniform stepsize, while empirical works adopt a variance exploding (VE) process with a decay discretization stepsize. As a result, these works suffer from large discretization complexity and fail to explain the empirical success of consistency models. To close the gap between theory and application, we analyze consistency models with (1) VE process and (2) decay stepsize and prove the state-of-the-art discretization complexity for consistency models. This result is competitive with the results of diffusion models and shows the potential of consistency models. To balance the computation and performance, previous empirical work further proposes a $2$-step consistency algorithm. In this work, we also analyze the role of $2$-step sampling and show that it improves the discretization complexity compared with one-step generation.

Improved Discretization Complexity Analysis of Consistency Models: Variance Exploding Forward Process and Decay Discretization Scheme
Ruofeng Yang'
Bo Jiang'
Cheng Chen Shuai Li
Abstract
generation (Rombach et al., 2022; Esser et al., 2024; Ho et al., 2022; Ma et al., 2024; Chen et al., 2024). The mathe- matical mechanism of diffusion models is made up of two processes: the forward and reverse process. The forward process gradually injects noise into data till the marginal distribution is close to pure noise. The reverse process is iterative sampling process, which sequentially removes noise from data to generate samples. At each denoised step, diffusion models only need to predict and remove a small noise, making the training process more stable than Genera- tive Adversarial Networks (GAN) (Goodfellow et al., 2014). However, the iterative sampling process requires diffusion models evaluate a large neural network predict noise at each step, leading higher computational cost than other one-step algorithms such GAN, Variational Auto- Encoder (Kingma and Welling, 2013), and Normalizing Flow (Papamakarios et al., 2021).
Consistency models, a new class of one-step gen- erative models, have shown competitive perfor- mance with multi-step diffusion models. The most challenging part of consistency models is the training process, which discretizes the contin- uous diffusion process into K steps and trains a one-step mapping function on these discretized timepoints. Despite the empirical success, only a few works focus on the discretization complex- ity K, and their setting is far from that of em- pirical works. More specifically, the current the- oretical works analyze the variance preserving (VP) diffusion process with uniform stepsize, while empirical works adopt variance explod- ing (VE) process with a decay discretization step- size. As a result, these works suffer from large discretization complexity and fail to explain the empirical success of consistency models. To close the gap between theory and application, we ana- lyze consistency models with (1) VE process and (2) decay stepsize and prove the sfate-of-the-arr discretization complexity for consistency mod- els. This result is competitive with the results of diffusion models and shows the potential of consistency models. To balance the computation and performance, previous empirical work fur- ther proposes a 2-step consistency algorithm. In this work, we also analyze the role of 2-step sam- pling and show that it improves the discretization complexity compared with one-step generation.
To solve the computational issue, a series of works try to accelerate the sampling process of diffusion models (Song et al., 2020a; Zheng et al., 2023). One notable algorithm in these works Consistency Model (Song al., 2023), which tries to find mapping function (a.ka. consistency function) to directly map any points at any time of the forward process to the target data distribution. Consistency models have shown state-of th-th-arr (SOTA) performance compared to other one-step generative models in image generation (Song et al., 2023; Kim al., 2024; Lu and Song. 2024), video generation (Wang et al., 2023) and music generation (Fei et al., 2024). Furthermore, it is also widely used in other areas, such as reinforcement learning (Ding and Jin, 2024) and medical image segmentation (Zhang et al., 2024).
To obtain the consistency function, consistency models dis- cretize the timeline into K discretization points and hope the consistency function outputs similar results at two ad- jacent points in the training phase. When the adjacent points are far away, the learned consistency function is far away from the ground truth (Thm. 1, (Song et al., 2023)). However, it is also necessary to avoid large K since it will make the training phase cone-consuming. Hence, one core of consistency models is the discretization number K in the training phase, which is helpful effectively training consistency function with great performance.
1. Introduction
Recently, diffusion models have shown impressive perfor- mance in different areas such as image generation and video
'John Hoperoft Center for Computer Science, Shanghai Jiao Tong University "East China Normal University. Correspondence to: Shuai Li <shuaili8 @duuuccdu.cn>
Proceedings of the 4 International Conference on Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Improved Discretization Complexity Analysis of Consistency Models
Despite the empirical success of consistency models, no existing works explain why consistency models achieve comparable performance to diffusion models. Though some impressive works analyze the discretization complexity of consistency models (Lyu et al., 2024; Li et al., 2024a; Dou et al., 2024), the setting is far away from the consistency models with great performance in application:
(Karras et al., 2022). On the contrary, with time-dependent score perturbation lemma, we achieve SOTA complexity / 3+2/a for consistency models, where the influence of a highlighted (More details in Sec. 4.4).
To improve the sampling quality of consistency models, Song et al. (2023) further provide a 2-step sampling algo- rithm, which has been widely used in current works (Song and Dhariwal, 2023; Lu and Song, 2024). The algorithm first adds noise to clean samples generated by consistency models and obtains the noised samples. Then, the algo- rithm again maps the noised sample to the clean samples (Eqq. 6). The great performance of this method indicates that the requirement of K can be relaxed if the 2-step sampling method is used. Recently, Lyu et al. (2024) show that the 2-step sampling algorithm can reduce the W₂2 error. How- ever, they do not obtain an improved K, which does not match the empirical observation. In this work, we show that with a VESDE process, the 2-step sampling algorithm can effectively reduce K to O / /W,, than Thm 4.7 and explains the role of 2-step sampling.
The forward process. Previous theoretical works choose variance preserving forward process (VPSDE). On the contrary, empirical consistency models adopt the variance exploding forward process (VESDE) with specific noise schedule whose solution trajectory is linear. As discussed in Karras et al. (2022) and Liu et al. (2023a), a linear trajectory is important for a one-step model (Details in S.c.D.1).
The discretization scheme. The discretization scheme of consistency models is EDM (Eq.5), which first uses a large stepsize and gradually reduces the stepsize. However, Lyu et al. (2024) and Dou et al. (2024) use a uniform stepsize, and Li al. (2024a) use a scheme that relies heavily on VPSDE (Details in Sec. D.2).
343/2a), which is better
In conclusion, we explain why consistency models have competitive performance compared to diffusion models from the theoretical perspective. More specifically,
Due to the mismatch between the theoretical and empirical setting, the current discretization complexity results are significantly worse than diffusion models (Table 1). Hence, the following natural problem remains open:
We close the gap between theory and application by an- alyzing consistency models with the VESDE forward process and EDM discretization scheme.
Under a setting closer to empirical works, is it possible to achieve a discretization complexity comparable to diffusion models and explain the success of consistency models?
Under the above setting, we achieve the SOTA dis- cretization complexity for consistency models, which is also competitive with the results of diffusion models.
In this work, we answer this question by analyzing consis- tency models with VE forward process and EDM stepsize.
For the first time, we also show that the 2-step sampling algorithm can effectively reduce the discretization com- plexity of consistency models.
With mild assumptions on pre-trained score function, consistency function, consistency models require'
Theorem 1.
K (=O(LT/ 3+2
2. Related Work
discretization steps in the training phase to output a dis- tribution which is Wasserstein-2 (W₂) close to the target distribution in the sampling phase, where I.g is Lipschitz constant of consistency function and a is EDM parameter.
Since the mathematical mechanism of consistency models is close diffusion models, we discuss the discretization com- plexity of diffusion and consistency models. For diffusion models, we summarize the results for reverse PFODE due to the deterministic sample process of consistency models.
As shown in Table 1, this result better than the previous theoretical guarantee of consistency models, shows the bene- fit of suitable a, and achieves competitive results compared to diffusion model results (Remark 4.9). The core step of this result is to make full use of the time-dependent score perturbation lemma instead of the previous uniform one. More specifically, we first show that with a uniform one, we achieve I, / discretization complexity for any EDM parameter a and does not match the empirical observation
Before the discussion, we first discuss the log-concave as- sumption, which is much stronger than our bounded support assumption since it precludes the existence of multi-modal real-worke data. Furthermore, under log-concave distribu- tion, V log (-) dose not go to +oo at the end of the reverse process, which does not match the blow-up phc- nomenon of score function in application (Kim et al., 2021). As a result, Gao and Zhu (2025) ignore the influence of early stopping parameter 5 and an additional Poly(ew,) (Table 1).
1 'Here, we ignore data diameter and dimension for clarity.
2

---

Improved Discretization Complexity Analysis of Consistency Models
Forward
Model
Stepsize
Complexity
Reference
Process
1/ew,
VESDE
Uniform
1/(ewr, Poly(ew;)) (+)
Gao and Zhu (2025)
Diffusion
1/ews
VPSDE
Uniform
//ee,,oly((eee (+)
VPSDE (Reverse SDE)
Exponential Decay
1/cls
Chen et al. (2023a)
Uniform
Lg/ews
Lyu et al. (2024)
VPSDE
Specific to VPSDE
L
Li et al. (2024a)
Consistency
To00e /
Uniform
Dou et al. (2024)
L2y(ew, 10
) (*)
Lg/CW,
Theorem 4.7
VESDE
EDM (5), a E (0)
Ly/cW2 3+
Corollary 4.12
(2-step Sampling)
Exponential Decay
L//o,,
Corollary 4.8
Table 1: The discretization complexity for consistency and diffusion models with reverse PFODE in Wasserstein distance.
To make thorough comparison, we also provide the SOTA discretization results for diffusion models with reverse SDE, (+) means that we transform the log-concave distribution (discussed in Section 2) to our bounded support distribution. (*) means that we transform the results with Lycore into the results under our setting, We present more detail in Appendix D.2.
Diffusion models with reverse PFODE. With strong as- sumptions or additional components, a series of works achieve polynomial complexity for reverse PFODE (Li al., 2023; Chen et al., 2023c; Gao and Zhu, 2025). More specif- ically, Li et al. (2023) assume accurate enough Jacobian matrix, and Gao and Zhu (2025) assume the target data distribution is log-concave. Without any strong assump- tion, Chen et al. (2023c) introduce predictor-correcte algorithm, which switches between a Langevin corrector and a PFODE predictor. Then, they prove a polynomial discretization complexity for this algorithm.
3.1. Diffusion Models
Diffusion models consist of two processes: the forward process and the reverse process (Song et al., 2020b). The forward process gradually converts data distribution to pure noise. To generate samples, diffusion models reverse the forward process and run the corresponding reverse process.
The forward process. Let go denote the data distribution. The general forward process is
dx, = I(X,, di g(() dB, Xo ~ 40
Consistency models. Lyu al. (2024) and Li et al. (2024a) analyze the discretization complexity of consistency dis- tillation and consistency training paradigm, respectively. More recently, Dou et al. (2024) analyze the estimation er- ror and discretization complexity of consistency distillation and training paradigm at the same time. Though these works deepen the understanding of consistency models, the setting of these works is far away from consistency models in the application and suffers from large discretization complexity compared to SOTA results of diffusion models (Table I).
where (Bt)120 is d-dimensional Brownian motion, f(X,,t) is a drift coefficient, and g(t) is a diffusion co- efficient. Let q be the density function of X, at time t and {3x he be non-negative non-decreasing sequence. When f(X,,t) = -3,X, and g(t) = V23,, the general forward process is instantiated as a widely used variance preserving forward process (VPSDE) (Ho al., 2020). We note that though VPSDE plays important role in develop- ing diffusion models, the solution trajectory of VPSDE is curved instead of linear, which prevents it from becoming the basis of one-step generation models.
3. Preliminaries
obtain one-step generation models, consistency models choose variance exploding (VESDE) forward process the training phase (Song et al., 2023: Song and Dhariwal, 2023). which has a linear solution trajectory under specific noise schedule. Let {aï he be non-decreasing sequence and g(t) = /doi/dt Then, VESDE is defined by:
Since the training phase of consistency models relies heavily on the diffusion process, we first introduce diffusion models in 1. After that, Sec.3.2 introduces how to train consistency model with pre-trained diffusion model.
dX. = s(1) dB,, Xo go
(I)
3

---

Improved Discretization Complexity Analysis of Consistency Models
As shown in Karras al. (2022), when choosing of = 12, the solution trajectory of VESDE is linear, which indicates it is possible to generate samples with single Euler step (Detail in Sec.D.1) (Song et al., 2023; Liu et al., 2023b). Hence, consistency models adopt VESDE (o? = 19) as the forward process, and we also choose VESDE with of = t² as the forward process to match the empirical setting.
exact vector field with the ground truth score function. Then, the reverse PFODE (Equation (2)) has the following form:
dYp v(Yv, t')dt' Yo -r.
Let f" R + + Rd be the associate backward mapping of the above PFODE. Then, know that:
The reverse process. Let T be the reverse time and (Yr) (E0))) = (Xr-z))
f (Ye,t') Yr X Vt' E
To generate samples, the model reverses the forward process (Eq. 1) and obtains the reverse process (probability flow ODE, PFODE) 2:
where d is the early stopping parameter. The above equation is equivalent to the following conditions:
f (Y,, t') = f (Yx,,")), vO vo ST VY R
dy, - -0T V log or dt', Yo QT- (2)
Since the ground truth score function V (og and qr contain the data information, we can not directly run the above PRODE to generate samples. For the reverse begin- ning distribution gr. we choose [00, Ia) to approximate it due to of- = T2. For Voog (-). Vincent (2011) propose the following score matching objective function to learn an approximated score function s (X,,t), Vt [o;
The goal of consistency models is to train a consistency func- tion fo to approximate f" and do onc-step generation. Let (Y, t') be a free-form deep neural network. To satisfy the boundary condition (the second equation of Equation (3)), Song al. (2023) parameter fo with the following form:
fo(Y,)) = {
Y
L' t'= T - FolY,E) e'E 10
min /' Ex. Ex,,xx V qe (X,) - sΦ (X,, 1)l dt. ΦED
/
There are two paradigms for consistency models: consis- tency distillation (CD) and consistency training (CT), where CD requires pre-trained score function & & (Y,t') and CT trains independently (Song et al., 2023; Lu and Song, 2024). Since CT can not take information from pre-trained score function, its hyperparameters need to be carefully selected to achieve great performance (Song and Dhariwal, 2023). On the contrary, the CD paradigm has a stable training process (Song et al., 2023). Hence, we analyze the CD paradigm and discuss the current results of CT paradigm in Remark 4.10.
With the approximated score function s.).), diffusion mod- els discretize the reverse process and generate samples. Let = to S t S ... S fx = T be the discretization points in the forward time and ha = le 4--1 be the stepsize. When considering the reverse process, we de- fine by 1/2 T - -x and M4- = hx- & the discretiza- tion points and stepsize in the reverse process, respectively. Since the score function V Ploggr-- goes to too at the end of the reverse process, we adopt the early stopping technique by setting to = 8 to avoid this issue, which is widely used in the application (Song al., 2020b; Kim et al., 2021). Then, starting from Yo ~ (N(OTT2La)) dif- fusion models run the following process in each interval t t ig, 1 E [0, K - 1] to generate samples:
Consistency Distillation Paradigm. Let ÿ 1/11 be the out- put by running step PFODE from t,2 to t with initial distribution Yú, and approximated score function ss:
Yig + (2T t/ - -t )h/ K11
P
(Y T 12)
dY. 9(T t22
so T 1h)
dt', di', it thrtl
2
Motivated by Eq. 3, Song et al. (2023) propose the following consistency distillation objective function:
3.2. Consistency Models
This part introduces how to obtain consistency function to
CKD
directly map pure noise to the target distribution by using the
above PFODE process and pre-trained score function. As a beginning, we first introduce the learning goal of consistency models. Let v(Y t') =T' V log - (Y) be the 2
Ex, Ey 1xo
Fo (
fo (8 Se4s (
[]] (4)
where 4 is the time discretization points in the reverse pro- cess and A is uniformly distributed over (0, 1,.,, K - 1}. Since Yg, is equal to XT we calculate Yg, Ix, using the forward process Xo + (T - ZZ),, where Z is the stan- dard Gaussian noise. To make the training process more
We note that there are two kinds of the reverse process: (a) SDE and (b) PFODE (Song et al., 2020b). Since the consistency models adopt the PFODE process in the training phase, we only present this process for simplicity.
4

---

Improved Discretization Complexity Analysis of Consistency Models
stable, (Song et al., 2023) introduce an additional parameter 0 which is updated using an exponential moving average (EMA) strategy 0- stopgrad (ny + (1 μ))0,, where E [o. 1) is the decay rate. We also note that this objec- tive function has already been adopted by many one-step flow-based models, like InstaFlow (Liu et al., 2023b).
ing process (called discretization complexity) to guarantee W2 (fo.otN (o, Ta) .q0) S <W2- This section provides state-of-the-art discretization complexity for one-step con- sistency models (Sec.4.2) and shows that the multi-step (2-step) sampling algorithm proposed by Song et al. (2023) can further reduce the discretization complexity (Sec.4.3).
Recently, Dou et al. (2024) discretize the interval Itte #1-1 in M smaller intervals and run multi step PFODE to obtain y- ",+1" We note that though this operation makes theoretical analysis easier, it is far away from the real-world application and consumnnn Since our work aims to explain the empirical success of consisteney models in application, we exactly follow the empirical operation, which does one-step PFODE instead of multi-step PFODE.
4.1. Assumptions
Before showing our results, we introduce some suitable assumptions on data distribution, pre-trained score function, and consistency function.
A Assumption 4.1. qo is supported on compact set M, where 0 E M and It = sup(l|= yll 2, E M} 2 1.
The bounded support assumption is naturally satisfied by the image dataset and is widely used by theoretical works (De Bortoli, 2022; Lyu et al., 2024; Yang et al., 2024).
The stepsize of consistency model. When training the con- sistency model, Song et al. (2023) and Song and Dhariwal (2023) use EDM stepsize
For the approximated score, similar to previous works, we assume it is L2-accurate (De Bortoli, 2022; Lyu et al., 2024). Assumption 4.2. There exists a constant Escore such that for any h E [K].
fx = (6 + and h = (T -8)/K,
(5)
with a- 7. As discussed in Karras et al. (2022), since VESDE has large variance at the end of the forward pro- cess, it is more suitable for VESDE to use a large stepsize at the beginning of the reverse process instead of uniform stepsize (a = 1). When a goes to +00, the EDM stepsize becomes& a theoretically friendly exponential decay stepsize h4 = rtg. where r is small coefficient corresponding to accuracy parameters e. We note that the exponential decay stepsize is widely used in theoretical works (Chen et al., 2023a; Benton et al., 2024). In this work, we simultane- ously analyze the EDM and exponential decay steps and achieve state-of-the-art discretization complexity.
Ex., ~ack [|Iso| (Xis V log 9t, (X,,,)1 S /ccore/oh.
We also assume after the one-step reverse PFODE process. the output of the learned consistency function are still close. Assumption 4.3. There exists a constant Com such that for any 1s E [K] and Yıg ~ 9%
t2 ||]]
E
( - fo N+1
S 2 (f++1 - 122)
Notation. We denote by W1 and W22 the Wasserstein dis- tance of order one and two, respectively. Note that W1 guarantee is weaker than W2 guarantee since (1(p,)) 2) s W2(P, 4). The push-forward operator t is associated with measurable map J MM' - ... For any measure over M'. we define the push-forward measure fīp over N by: fdu(A)) = / (A)). for any A be measurable set in A. A complete notation part is provided in Appendix A.
The above assumption is also used by Lyu al. (2024). We note that this assumption has the same form with L and can be satisfied when LKD is small enough.
Similar with previous theoretical analysis, we also assume fo (Y, t') is Lipschitz.
Assumption 4.4. fo(Y is Ly-Liscchitz for 1' E [0,T 8) and is -. with Ly.0 R/T when t' = 0.
The first part is a standard one used by all previous theo- retical works on consistency models (Lyu et al., 2024; Li et al., 2024a; Dou et al., 2024). We further assume the sec- ond part for VESDE forward process. In the following two paragraphs, we first provide empirical evidence support our additional assumption (Example 4.5). Then, we discuss why the additional assumption is necessary for consistency models with VESDE (Remark 4.6).
4. Discretization Complexity of Consistency Model
The goal of consistency models is to train a consistency func- tion fo (Y,t') to directly map pure noise Y ~ NN(0, La) and t' = 0 to target distribution qo. Let fa.o be the learned consistency function at t' = 0 and fa.utN (0.T2Ia) Ia) be the generated distribution. In the training phase, consis- tency models discretize the continuous time t' E [0,7- - 6] into K intervals and use the CD objective function to learn a consistency function (Eq.4). The goal of this work is to determine the requirement of K in the train-
Example 4.5, As a start, we first provide f for a 1-d Gaus- sian target distribution go = -N(,, o²):
Yo p
(Yo. = + Vo2 + T2
5

---

Improved Discretization Complexity Analysis of Consistency Models
where Yo ~ 4r N(,, T²). It is clear that the Lips- chitz constant t. of for is a ve/vaz 02 + T2x < R/T at t' = O 3. We note that this result also holds for high-dimension multivariate Gaussian, as shown in Li et al. (2024b).
Taking the derivative of Yo, we know that the L, have order 1/T. The intuition of I,5,0 Q(/T)) is that the variance of (0, I21) is much larger than data variance. Hence, one necessary step to map pure noise to the target distribution is to remove large variance by multiplying a I/T,, which leads to an O(1/T) Lipschitz constant.
For the highly multi-modal target distribution, the consis- tency function for these distributions does not have closed form. There are two choices to overcome this hardness. The first choice to run experiments to simulate the so- lution and calculate the order of Lipschitz constant. In Appendix F, we run simulation experiments on the multi- modal Gaussian mixture distribution (3-modal and 4-modal GMM) and verify the Lipschitz constant f of f has an order of 1/T. The second choice is to add some as- sumptions on the multi-modal target data to simplify the PFODE process and then show the order of the Lipschitz constant. In this part, tribution Xo ~ (2, + 1 1/2N(- o² Ta) an example to support our discussion. The score of the target distribution has the following form (Appendix A.2 of Shah et al. (2023))
Remark 4.6 (The necessity of L,0 = R/TT). As shown in Eq. 7, one enror term is W2 (fe ofoo (0. fo which corresponds to the forward process and is bounded by 10 W2 (O, aa) qr). For VPSDE forward process, as shown in Lyu et al. (2024), (2, Ia), ) S (Vd + R) exp (-T) since VPSDE converges to (O, Ia) with an exponential convergence rate. On the contrary, for VESDE, we know that W2 (W (0, (0, .qT) is bounded by
2-modal GMM target dis-
E go
| + -T- T) S RR.
where & ~~ (0, Ia). is clear that this term is independent T. As a result, we need L,.0 depends on T and show that 5 has order of 1/T (Example 4.5).
X, of + o2)
X of + o²
We also note that even with Assumption 4.4, the polynomial T 1/ew, for VESDE is much larger than the logarithmic T log(1/ew,) for VPSDE. However, in Remark 4.15, we show that VESDE has better order on early stopping 8, which offsets the influence of large T. Furthermore, as shown in Thm.4.7, the dependence of T is T1/o, which indicates EDM stepsize further reduces the influence T'.
V logg (X,) tanh
of 002
Since PY,,, 0) the associate backward mapping of the following PFODE (in the following part, we ignore the superscript of ('):
(T -) dY, (T- t)2 d² tanh (T' t)2 + 02
Y
Yn
dt,
4.2. Improved Results for Consistency Models
we need to solve it to obtain fex
1%..0) 0). To simplify the PFODE process, we assume ju is smaller enough to guaran- (7-1)++07 can be approximated by -
With these assumptions, we obtain the following results under the VESDE and EDM stepsize.
tee tanh a TY,
Theorem 4.7. Assume Assumption 4.2, 4.3, 4.4 holds and consider the EDM stepsize (5). Then, one-step genera- tion error W2 (fa.otN T I0,T ,4) is bounded by
which simplify PFODE to a linear ODE (in fact, the distri- bution gradually closes to Gaussian)
dy, - ay= (ITNNX(T-) -1)
( (
Yi(T -)) (T (T- 1)2 02
R² L (P(R+Va)(TT)) (T) T
dt,
+ LLyorse Team + Vd8.
K82
which have the following solution
Furthermore, by choosing T 2 R R2/ews. 5 ewz/Và. Cam S 'Ta/T and Escare S (T.g R²), the output is ew, -close to go with discretization complexity
Y2= = Yo (
02 (T- -t)2
0² 42
L L 1 VR
2 exp
K ==O
2
0 + (T - 1)2
02 + T2
EW2
3+
and indicate
Song et al. (2023) choose a 7 for the EDM scheme, which leads to / 23/7) result. When considering exponential decay stepsize, we improve the results as follows.
σ2 Yr Tn=Yo Yo (V 02 +2 exp
(1 2
2
o²
+ T
Corollary 4.8. Assume Assumption 4.1, 4.2, 4.3, 4.4 holds and consider the exponential decay stepsize h2 = rtp- where = =/,// (R (o2log((T8) (T/8)). Following T, 5, cam in
Since Gaussian is log-concave, the score does blow-up (Details in Sec.2), and we do not use early stopping technique here.
6

---

Improved Discretization Complexity Analysis of Consistency Models
Thm. 4.7 and choosing eccore S 5 (R² (o)( the output is <W2 -close 90 with discretization complexity
4.3. 2-step Sampling Further Reduce Complexity
To achieve better performance, a multi-step sampling method is used by consistency models (Song et al., 2023; Lu and Song, 2024). Let T = T1 2 72 2 ... 2 TN 8 be sequence of time points, P1 be the one-step generated distribution Sa,otN (0, Ia) and X ~ Pl. The n-step sampling process has the following procedure:
K O (L log (TT])
As shown in Table 1, the above discretization complexity ((g)/ewg) for CD paradigm significantly improves the cur- rent results (w, provided by (Iyu et al., 2024) (We discuss current results for CT paradigm in Remark 4..... Furthermore, as shown in Remark 4.9, this result achieves competitive results with diffusion models in both reverse SDE and PFODE settings by taking full use of the time- dependent score perturbation lemma (Sec. 4.4), which shows the potential of consistency models.
X fo(X + to, Tm): Z ~ I), (6)
which first adds noise to the (n - 100-1)-ssoe s)-step sampling data using the VESDE forward process and then generates X Let Pa. be law (X**). Recently, Lyu et al. (2024) make an important step in understanding the multi-step samplingg mechanism in consistency models and prove that this opera- tion can reduce the W2 error with suitable N. However, as shown in Corollary 3.14 of Lyu et al. (2024), both the dis- cretization complexity of one-step and multi-step sampling are Ó This result means the multi-step sampling can not reduce the requirement of discretization, which does not match the empirical observation. Hence, it motivates us to do a more refined analysis under a realistic setting and show the role of multi-step sampling. Since n = 2 is enough to generate high high-quality samples in application (Song et al., 2023; Lu and Song. 2024), we analyze the 2-step sampling and improve discretization complexity 4.
Remark 4.9 (The Diffusion Model Results). The SOTA results for diffusion models under W₂ distance is O(1/et,)) Coro. 4.8 achieve ( (w,,) discretization and will be better than O()/w(,) if Ly S O(1/ewz). As shown in Thm. 4.7, the early stopping parameter has order ew.- Hence, we require f, S 1/8 for VI' E [0, T 6), which can be satisfied if L, grows slower than 1/(T t'). As shown in Example 4.5, the Gaussian distribution satisfied this growth rate. For multi-modal real-world distributions, as shown in Appendix F, the growth rate is much slower than 1/(T - f'), which also satisfies our requirement.
Remark 4.10 (The CT Paradigm Results). Li et al. (2024a) and Dou et al. (2024) analyze the discretization complexity of CT paradigm. As shown in Table 1, their discretization complexity results are worse than our results. More specif- ically, Li et al. (2024a) achieve a / result, which adopt weaker W1 guarantee and has worse L, depen- dence. Dou et al. (2024) also adopt the W1 guarantee and 10 results. More importantly, their paradigm is different from the CT paradigm in application. Li et al. (2024a) use an iterative consistency training method, which trains a consistency function for each k E [K] and is time- consuming. To avoid this problem, Dou et al. (2024) only train a consistency function. However, they run multi-step PFODE in the training phase, which does not match the operation (only run one-step PFODE) of the empirical con- sistency models (Song et al., 2023; Song and Dhariwal, 2023). Hence, it is an interesting future work to explain the empirical success of CT paradigm from the theoretical perspective in the setting used by empirical works.
Corollary 4.12. Assume Assumption 4.1, 4.2, 4.3, 4.4 holds and ,,1/2 @RR/T). Choosing EDM stepsize and T2 T/2 (2-step sampling), W2 (P2, 90) is bounded by
Vdo + (T2(( + L (R + va)(T/8)* /KT)))))((K()))))))))))))
achieve /2W
+ (RT/T2 + T2) (L + Cem)
By choosing T 2 R1.5 /VE2. 8 ewa/Vd, cam S
ew,/TT /T and Escore S /L, the oulput is -close to qu with discretization complexity
K ( 2
20 (R+ vdld + to
3 /ew, 20
The error bound comes from three sources: the early stop- ping term, the previous error W2(p1, qs) and the discretiza- tion error at this sampling phase. The core observation is that the multi-step sampling can reduce the requirement of T due to the 1² (Tg2') term, which further improves the discretization complexity.
Remark 4.11. Recently, some works (Lyu et al., 2024; Li et al., 2024a; Dou et al., 2024) assume the second moment of data distribution E|l [/l///l1 | is bounded, which is slightly weaker than Assumption 1. The dependence of R comes from our -eeeddepnddent score perturbation lemma, which is an important part of making full use of EDM stepsize (Sec. 4.4). We also note that to achieve a refined analysis, Lemma 3.13 of Lyu et al. (2024) and Lemma D.2 of Dou et al. (2024) also assume Assumption 4.1 to obtain the Lipsheitz constant R² /4 for the score function.
We note that compared to Thm. 4.7, we further assume L = O(RTT) which is also supported by our simula- tion experiments. The intuition is that since T/2 and T have the same order and are both large, the Lipschitz constant at T/2 also should have the same 1 1/T order with T.
*Our analysis can be directly extended to multi-step sampling.
7

---

Improved Discretization Complexity Analysis of Consistency Models
Remark 4.13 (N-step Sampling). We note that our analy- sis can be extended to N-step sampling algorithm and can achieve nearly / /t/// (which is better than Thm. 4.7 and Coro 4.12) under the EDM stepsize. We use 3-step sam- pling algorithm as an example (Ti = T2 = 3T/4, T3- = T/2. Here, we still further require ,,,37/4 (R/T)). Under this setting, the result becomes (here we ignore Sccores fem: R, d and focus on the dominated term)
where the expectation is taken over Yo ~ er and the first equality follows the fact that f 0Y) Xs fo (YT-6, - 6). After that, we aim to control the er- ror bound of one step PFODE, which relies heavily on Ex,~q [[|@, V log dt (Xx)
Byowes [Ixae Yeg [13] Chithi
S lig 2
Soore
/ 4(T -(T t) 110.Vlog de. (9)
8 8+ + + + (T/6) (K6²)
To guarantee the above term smaller than EW2- we require 5- = EW; and K= 2 L,T\// / /(62+1/-ews), which is the same with the one-step and two-step sampling algorithms. However, 3-step algorithm only require T 2 1/W,, which is better than 1/ew, of 1-step and 1/ of 2-step. Hence, the discretization complexity for 3-step sampling algorithm is /.y/wa // 3+<4/(3a) which is better than 2-step algorithm. The above steps can be extended to N steps, and the influence of T decreases, and finally, T does not affect the discretization 3+1/a results. complexity, leading to /
In the following paragraph, we focus on the second term and discuss why a time-dependent lemma is suitable and necessary for EDM stepsize to achieve an improved results.
Time-dependent Score Perturbation Lemma. Following previous works, we call the following lemma score pertur- bation lemma. However, different from previous lemma uniformly holds for t E [ō, T] (Lyu et al., 2024; Chen et al., 2023c), we provide the following time-dependent score per- turbation lemma, which leads to an improved results.
Lemma 4.14, For VESDE (Eq. 1), we have that
4.4. Proof Sketch and Technique Novelty
Ex,og 11 V log q (x,) - R R2/O + RVaaft.
In this section, we first provide proof sketch for Theo- rem 4.7 to introduce the dependence of score perturbation lemma. Then, we highlight our technique novelty to take advantage of the time-dependent score perturbation.
With this lemma, we can control the one step PFODE error (Eq. 9) and then the discretization error.
Before showing the advantage of the denendepnddet lemma, we first provide the result with the uniform score perturba- tion lemma, which has large discretization error and does not reflect the role of EDM. By setting ( = 5 in Lem. 4.14, we obtain an uniform version lemma holds for t E [6,
Proof Sketch. We first decompose the target error term W2 (fo.o II, 90) as:
W2 (fe (0, Taa) foo + W2 (fe,oder, 96) + W2 (gs, q0)
(7)
Ex,ng. [1 V log q (XiIla) & RA Va)
where the first term is due to the forward process, the sec- ond term is the discretization error, and the third term is due to the early stopping 5. The first term is discussed in Re- mark 4.6, and the third term is smaller than vd5. This part focuses on the discretization term W2 (fe,oter, 2)), which is controlled by the following inequality.
Combined with Eq. 8 and Eq. 9, we know that the first term of Eq. 8 is smaller than
L (R + Va)
K-1
h
2
85
[lfe (Yo,)) 0)- 8" (Yo. (Y.)))1)) 1/2
k=0
E
We know that
K Σ k=
(oo(Y,, th)) - fo fo the ((41)) [(]) 1/2
K-1 1
T1/
T
T2
hgt
t
x
(10)
K
K
: (2+1)) - So(Yi (++111133 1/2
k=
E (F 1/211
where the first equality follows the fact that h2/h / x tx
K-1
1/2
and the equality comes from h (T1/aa 5) K. To make this term smaller than ew,, we require K > "T/T%/(ew,, With T and ò in Thm. 4.7, we know that K has order of L / which has large dependence on EW.-
E (1Fo(Ye,+h - fo (E po th
R=
K-1
Σ Ls
E ||| 1 441 - -Ye,, 11)
1/2
+ tem T,
'Here, we only focus on ew, and ignore R and d for clarity.
8

---

Improved Discretization Complexity Analysis of Consistency Models
More importantly, it is clear that the result with uniform version lemma is not influenced by EDM parameter a. How- ever, Karras et al. (2022) have shown that a suitable = 7 can significantly improve the generation quality, and consis- lency models adopt this setting (Song et al., 2023). In this part, we show that with the time-dependent score perturba- tion lemma, the discretization error can be well control to achieve an improved complexity. More specifically, with Lem. 4.14, we have that
algorithm proposed by Song al. (2023) and show that this algorithm further improves discretization complexity to O TT
//g//W 3+ 2u which explain the widely used of multi- WWa
step sampling algorithm in application.
In conclusion, we achieve competitive results for consis- tency models compared with diffusion models and show the potential of consistency models.
Future work. In this work, we directly assume the score and consistency function are accurate enough. For the score function, some works analyze it learning process Chen et al. (2023b); Yuan et al. (2023). For the consistency function, Dou et al. (2024) analyze its estimation error. However, as discussed in Sec. 3.2, they run multi-step PFODE instead of one-step PFODE, which is time-consuming and does not match the empirical operation. Hence, it is interesting to analyze the learning process of consistency models un- der a realistic setting and achieve an end-to-end analysis. Very recently, Lu and Song (2024) train consistency mod- els in continuous time and achieve great performance. Since the continuous time models use dfa- (X..)) instead of so (X,-ar, - At)) (At is h4+1 - h& in our work. Here we use the uniform stepsize for convenience), there are not well-defined discretization complexity K = T/ for continuous-ting models, however, due to the absence of At, the training process of continuous time models is less stable than discrete time consistency models, which is the core problem for continuous time models. Hence, it is also an interesting future work to analyze how to make the training process of continuous time consistency models more stable from the theoretical perspective.
K- Σ K=
1/2
Tif (Eooarr
Fk41
- -
12
K-1 Lig(T -1)
Ra
Vd
(T 1/)0
(T (T- 1/))*
Lg (R vā)
K-1
1
82
(T - -1%)
k=
We know that
K
K
T
(T/5) K
x h
dt x h8
di
56 1.
1=1
which has a better dependence on T and 8 compared with Eq. 10 and achieve an improved result / 3422/a
Remark 4.15. To make clearer discussion on the role of VESDE, we choose a uniform stepsize (a = 1), which is used by Lyu et al. (2024). Under this setting, the difference between our work and Lyu et al. (2024) is the forward process. Lyu et al. (2024) use the VPSDE forward process and achieve L,/(ew,8") result. We adopt VESDE forward process and achieve L L,T/(cW,,0) the uniform stepsize, our better result is due to better 8 of VESDE. More specifically, to guarantee W2 go) I ew; (the third term of Eq. 7), we require ō has order of ew; for VESDE and ew, for VPSDE. Hence, Lyu et al. (2024)
Ly/(<w, 8²) 6.6. Under
Impact Statement
Our work aims to deepen the understanding of consistency models from the theoretical perspective. Since consistency models are a new class of one-step generative models, the so- cietal impact is similar to general generative models (Mirsky and Lee, 2021).
achieve /,y/ result for VPSDE and we achieve Tog / for VESDE (uniform stepsize setting).
5. Conclusion
References
In this work, we make the first step to explain why con- sistency models perform well from a theoretical perspec- tive. More specifically, we bridge the gap between theory and application of consistency models by analyzing the discretization complexity of the consistency model with VESDE forward process and EDM discretization scheme. Under this realistic setting with great empirical performance, we first show the of-te-of-theaar discretization complexity 3+2 O (Ls/ew," by making full use of time-dependent score perturbation lemma. Then, we analyze the 2-step sampling 'Here, we ignore the dependence on R and d.
Joe Benton, Valentin De Bortoli, Arnaud Doucet, and George Deligiannidis. Nearly d-linear convergence bounds for diffusion models via stochastic localization. In The Tweffih International Conference on Learning Representations, 2024.
Hongrui Chen, Holden Lee, and Jianfeng Lu. Improved analysis score-based generative modeling: friendly bounds under minimal smoothness assumptions. In International Conference Machine Learning, pages 4735-4763. PMI.R 2023a.
Minshuo Chen, Kaixuan Huang, Tuo Zhao, and Mengdi Wang. Score approximation, estimation and distribution recovery of diffusion models on low-dimensional data, In International
9

---

Improved Discretization Complexity Analysis of Consistency Models
Conference on Machine Learning, pages 4672-4712. PMLR, 2023b.
Dongjun Kim, Chich- Lai, Wei-Hsiang Liao, Naoki Murata, Yuhta Takida, Toshimitsu Uesaka, Yutong Hc, Yuki Mitsufuji, and Stefano Ermon. Consistency trajectory models: Learning probability flow ode trajectory of diffusion. In The Twelfth International Conference Learning Representations, 2024.
Qihua Chen, Yue Ma. Hongfa Wang. Junkun Yuan, Wenzhe Zhao, Qi Tian, Hongmei Wang, Shaobo Min, Qifeng Chen, and Wei Liu. Follow--yur--cnnas:: Higher-resolution video outpainting with extensive content generation. arXiv preprint arXiv:2409.0005,, 2024.
Diederik P Kingma and Max Welling. Auto-enending variational (Bayes). In Inf. Conf. on Learning Representations, 2013.
Sitan Chen. Sinho Chewi, Holden Lee, Yuanzhi Li, Jianfeng Lu, and Adil Salim. The probability flow ode is provably fast. Advances in Neural Information Processing Systems, 36:68552- 68575, 2023c.
Gen Li, Yuting Wei, Yuxin Chen, and Yuejie Chi. Towards faster non-asymptotic convergence for diffusion-baselt genera- tive models. arXiv preprint arXiv:2306.092 2023.
Gen Li, Zhihan Huang, and Yuting Wei. Towards a mathemati- cal theory for consistency training in diffusion models. arXiv preprint arXiv:2402.000 078007 2024a.
Sitan Chen, Sinho Chewi, Jerry Li, Yuanzhi Li, Adil Salim, and Anru R Zhang. Sampling is as casy as Icarning the score:
theory for diffusion models with minimal data assumptions. In International Conference on Learning Representations, 2023d.
Xiang Li. Yixiang Dai, and Qing Qu. Understanding generalizabil- ity of diffusion models requires rethinking the hidden gaussian structure. arXiv preprint arXiv:24 24060, 2024b.
Valentin De Bortoli. Convergence of denoising diffusion mod- cls under the manifold hypothesis. Transactions Machine Learning Research, 2022.
Xingchao Liu, Chengyue Gong, et al. Flow straight and fast: Learning to generate and transfer data with rectified flow. In The Eleventh International Conference on Learning Representations, 2023a.
Zihan Ding and Chi Jin. Consistency models as rich and effi- cient policy class for reinforcement learning. In The Twelfth International Conference on Learning Representations, 2024.
Xingchao Liu, Xiwen Zhang, Jianzhu Ma, Jian Peng, et al. In- staflow: One step is enough for high-quality diffusion-based text-to-image generation. In The Twelfth International Confer- ence on Learning Representations, 2023b.
Zehao Dou, Minshuo Chen, Mengdi Wang, and Zhuoran Yang- Theory of consistency diffusion models: Distribution estimation meets fast sampling. In Forty-first International Conference on Machine Learning, ICML 2024, Vienna, Austria, July 21-27, 2024. OpenReview.net, 2024.
Cheng Lu and Yang Song. Simplifying, stabilizing and scal- ing continuous- consistency models. arXiv preprint arXiv:2410.1 2024.
Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz. Axel Sauer, Frederic Boesel, al. Scaling rectified flow transformers for high-resolution image synthesis. In International Confer- ence on Machine Learning, pages 12606-12633, PMLR, 2024.
Junlong Lyu, Zhitang Chen, and Shoubo Feng. Sampling is as casy as keeping the consistency: convergence guarantce for consistency models. In Forty-first International Conference on Machine Learning, 2024.
Zhengcong Fei, Mingyuan Fan, and Junshi Huang. Music consis- tency models. arXiv preprint arXiv:2404. 13358, 2024.
Yue Ma. Yingqing He, Hongfa Wang. Andong Wang, Chenyang Qi. Chengfei Cai, Xiu Li, Zhifeng Li. Heung- Shum, Wei Liu, et al. youlow-your-llikk: Open-domain regional image animation via short prompts. arXiv preprint arXiv:2403.08266. 2024.
Xuefeng Gao and Lingjoon Zhu Convergence analysis for gen- eral probability flow odes of diffusion models in wasserstein distances. In The 28th International Conference on Artificial Intelligence and Statistics, 2025.
Yisroel Mirsky and Wenke Lee, The creation and detection of deepfakes: A survey, ACM Computing Surveys (CSUR), 54(1): 1-41, 2021.
Ian Goodfellow, Jean Pouget-Abes Abadie. Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. Advances in neural infor- mation processing systems, 27, 2014.
George Papamakarios, Eric Nalisnick, Danilo Jimenez Rezende, Shakir Mohamed, and Balaji Lakshminarayanan. Normalizing flows for probabilistic modeling and inference. Journal of Machine Learning Research, 22(57): 2021.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in Neural Information Process- ing Systems, 33:6840-6851, 2020.
Robin Rombach, Andreas Blattmann, Dominik Lorenz. Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffiusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2022.
Jonathan Ho, Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi, and David J Fleet. Video diffusion models. arXiv preprint arXiv: 2204-03488,48 2022.
Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine. Eluci- dating the design space of diffusion- based generative models. arXiv preprint 2r06v:2206.000 00364, 2022.
Kulin Shah, Sitan Chen, and Adam Klivans. Learning mixtures of gaussians using the ddpm objective. Advances in Neural Information Processing Systems, 36: 19636-196499 2023.
Dongjun Kim, Seungjae Shin, Kyungwoo Song, Wanmo Kang, and II-Chul Moon. Soft truncation: A universal training technique of score-based diffusion model for high precision score estimation. arXiv preprint Nix:220660053 2021.
Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502. 2020a.
10

---

Improved Discretization Complexity Analysis of Consistency Models
Yang Song and Prafulla Dhariwal. Improved techniques for train- ing consistency models. arXiv preprint arXiv:2310.1413: 2023.
Yang Song, Jascha Johl-Dicksteinn Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based genera- tive modeling through stochastic differential equations. arXiv preprint arXiv:201 13456, 2020b.
Yang Song, Prafulla Dhariwal, Mark Chen, and Ilya Sutskever. Consistency models. arXiv preprint arXiv:2303.0142 2023.
Pascal Vincent. A connection between score matching and de- noising autoencoders. Neural computation, 23(7):1611-1674 2011.
Xiang Wang, Shiwei Zhang, Han Zhang, Yu Liu, Yingya Zhang. Changxin Gao, and Nong Sang. Videolem: Video latent consis- tency model. arXiv preprint urXiv:2312.09100 2023.
Ruofeng Yang, Zhijie Wang. Jiang, and Shuai Leverag- ing drift to improve sample complexity of variance exploding diffusion models. In The eiihtt Annual Conference Neural Information Processing Systems, 2024.
Hui Yuan, Kaixuan Huang, Chengzhuo Ni. Minshuo Chen, and Mengdi Wang. Reward-directed conditional diffusion: Provable distribution estimation and reward improvement. Advances in Neural Information Processing Systems, 36:60599 60635, 2023.
Kejia Zhang, Lan Zhang, Haiwei Pan, and Baolong Yu. Cts: A consistency- medical image segmentation model. arXiv preprint 240.::2405.0900 2024.
Kaiwen Zheng. Cheng Lu. Jianfei Chen, and Jun Zhu Dpm- solver-v3: Improved diffusion ode solver with empirical model statistics. Advances in Neural Information Processing Systems, 36:55502-55542, 2023.

---

Improved Discretization Complexity Analysis of Consistency Models
Appendix
A. Notation
We denote by Wi and W2 the Wasserstein distance order one and two, respectively. Note that W1 guarantee weaker than W2 guarantee since (p(p. q) I (2( q). The push- forward operator is associated with measurable map f M' N. For any measure μ over M', define the push-forward measure fīj over N by: fip(AA 1 (f)) for any A be measurable set in ..
Before introducing our theoretical guarantee, we first introduce the notation for the diffusion and consistency models. Let qo be the data distribution.
Diffusion models:
Let (Xt)te(o,r) be the random variable of the forward process (Equation (1)). We define by = to 44 S ... S Lx = T and h2 = tt - t the discretization points and stepsize.
Let 1' = be the reverse time and (Yooo (Xr-e)eeto,.) be the random variable of reverse process
(Equation (2)). We define by 1/2 = T - b-- and hig = h the discretization points and stepsize in the reverse process.
Let Pk be the distribution generated by running the discrete PFODE process (the last equation of Section 3.1) with Sg the complexity of the sample is the requirement K guarantee W2(KK, qu) S eW,
Consistency models:
The goal of consistency models is to learn a consistency function fa (Y,t)) to directly map pure noise Y ~ (0,,0 and e' (the start of the reverse process) to go-
We denote by fe.oiN (0,7%I) the generated distribution of the above operation. Since the consistency function is one step, the discretization complexity is the requirement of K in the training process (Equation (4)) guarantee W2 W2ffo,oN (0, Ia) .40) S eW2
B. The Analysis for Consistency Model
Theorem 4.7. Assume Assumption 4.1, 4.2, 4.3, 4.4 holds and consider the EDM stepsize (5). Then, one-step generation error W2 Nfo.otN (0, Ia)) (o) is bounded by
R2
R
R R + (T)
L Team vd5.
T
K82
Furthermore, by choosing T R2/ew,, 8 = ew,/ Vd. fam S ewn/T and escore S (L, the output -lyse to qo with discretization complexity
L (R+ +
K K=O
3+
Cw2
Proof. We first decompose W2 (fa.otN ()) .90):
W2 (fe. (0. 9o) - W2 (fe II.) fe. + W2 (fe.odor, 95) + W2 (gs.
where the first term is the reverse beginning error due to the forward process, the second term is the discretization error due the discretization training. and the third term is due to the early stopping technique. We first define a joint distribution y r (W (0, .qr) between / (0, T l2) and qr and take couple of (Yo, Yo) 7, which indicates
[ (.,Yo)d = N (N To,
Rd
[ 7Yo, -dY = GT
12

---

Improved Discretization Complexity Analysis of Consistency Models
Then, have that
W2 (fo,oEN (0, .96)
1/2
_ E, [l (Yo, 0) - f (Yo, 0) [
= E, [l (Yo, 0) - fo (Yo, 0) [b:] ) 1/2
1/2
+ (E, [lfe 0) - fr (Xo, 0)
- 100 (B, 1116 - Yoll2) 1/2 + EE,, fe (Yo. 0) fr (Yo, 0)1B3) 1/2
R s 7 Wa( IT): (a)) + (E,, [l (Yo, - 8 (Yo, 0) 1]]]] 1/2
R
+ (Byongr [lifo (Yo, fr (Yo, 0) 0)][]] [1]
where the first inequality follow the fact f (Yo, 0) = X8 when Yo ~ qr and the last inequality follows the fact that ?y can be any coupling between (0, (O,TT2L) Ia) and qr-
The reverse beginning error. have that
We first control the reverse beginning error term. When considering the W₂ guarantee, we
| + (T -T- (T) 1/2 s R.
Wg(N(O,T"La..r))
Ego
The discretization error.
In this section, we control the discretization error:
EYower [I (Yo, 0) - fr (Yo, (x6.0)112) 1/2
-(Brrvr
K Σ So (K) - Sa (Yr (Vi (h11 K=
27
1/2
BYooerr
||500 (YR,th) fo (Xurn) 119)
1/2
Exaner
(het)) - So ( 1(2) 1/2
Eyo-or
|1500 (Ya. fo (yrg ck4ss" (ktt)) + So
(yro (Vren
50 (Yr th) fo (pr0 (ktt2) [1=]
1/2
Exaner
(vo 11 (oon.) - fo (Ve: )113]
1/2
Eyo-or
so
= Es E2,
where the first inequality follows the fact that for (Yo, 0) Xs So T For term Ed, since we assume the learned consistency model is accurate enough (Assumption 4.3), then have that:
(fee (/rt1) )11] 1/2
[|50 (ve,, fo (v
E
Byo-er
K
||:50 (Xx,,t)) fo (ro (Pi ( (1²)) 1/2
Bx ~o
k==
6cm
1/s Ecm
(T-8) -8)-
k=
13

---

Improved Discretization Complexity Analysis of Consistency Models
For term Eg, know that
K
1/2
[| (g (ee,)))))))) fo +
E2
Bygner
K
[1000 - -Yes
1/2
Li
Ero-or
122
2
-Vecu (21)
1/2
Li
Byg
1/+11
k=0
K-
R
Ra (T
L
hi
11 Crcore
(T)
where the last inequality comes from Lemma C.2. When considering EDM stepsize
T 8
1. (6 kh) h
K
2-1 we know that hy x k
hi
S"
1 _
(T/8)=
dt
hd-
K
Then, term E2 is control by the following inequality:
R + vaylit
Eg
L
haCscore
t2.02
k=
(R+
+ Lecore
K82
For term W2(gs, qo), by using Lemma E.3, is smaller than vda, vd5. Combined with the reverse beginning, discretization and early stopping term, we have that
We (fe.u (f. go) S R²
LR Va)(11))
Lycorc + Cem + vds.
K82
To make the above inequality smaller than eW;, we choose T RR/eww, 8 Sws/Vd. fem S Sww//T and guarantee
L + vayrt
S SW2
K82+
which indicates that K n (R+ (R+ Vaja
After determining the discretization complexity K, we can also obtain the
34 2
requirement of the approximated score function Gscore I R2)
Corollary B.I. Assume Assumption 4.1, 4.2, 4.3, 4.4 holds and consider the exponential decay stepsize by rtg, where r e (R² log" (T))) Following T, 8, cem in Thm. 4.7 and choosing Cscore S 6 log(T/8)), the output is <cloce to yo with discretization complexity
K = (L, vd) log log (T/8)/
14

---

Improved Discretization Complexity Analysis of Consistency Models
Proof. For the theoretical-friendy] exponential decay stepsize, we have that
K=
(T/8)
Rº + vd)hi
E2 S LF
Escore
12/52
1=
D vd) log (T/5)
+ Ly
K8a
Combined with the reverse beginning term and discretization term, know that
R²
L R vd) log (T/O) + L + Cem I + ddo
W2 (fo. (0, II) 90)
(fo.
Kd²
In order to guarantee the right hand of above inequality smaller than eWs, we choose T 2 R RR/ews> = ewa/ 2 S TT², and guarantee
R Vd) (T/6)
S CW2
K8²
which indicates discretizaiton complexity is 2 Va (T/6), After obtaining the requirement of K, we can
W2
also obtain the requirement of upproximated score function Eacore I /
At the end of this part, we provide the proof of multi-step sampling.
Corollary B.2. Assume Assumption 4.1, 4.2, 4.3, 4.4 holds and L = (RR/T). Choosing EDM stepsize and T2 = T/2 (2-step sampling). W2 (P2, Qu) is bounded by
Vdō + R (T2T) + R Va)(T/6))/(K3))
(RT) T2 + T2) (Lo + Cem)
By choosing T 2 R1.5/ 8 ew,/Vo, ecm S Eww,/T and Escore S / (L the output is ew, -cloe to with discretization complexity
k= 0 (1 R² (R+ vdld // 1/
Proof. Take couple of (Y, Z) 2),,2) where y (, take & N (0, Ia), then have
Y = Y Tn pn.
Ż = Z + T ~ 9rn
Similar with Corollary 10 of Lyu et al. (2024), we have that W2 (p2, go)
L D (0)
$ W2(gs, go) + -. W2 (Jin (7,)
12 + 6em72
K8²
1/2
L (R + (T2
$ W2(gs. go) + L [ - -2113)
+
I1g22cccor + Cem
K8²
+ L + V (T2
S We(gs.90) + - EE, Y Z
Z1b)
1/2
L + Com T2
K8²
R2
L R+ (T)()
S Vdo +
L + Cem
T
K82
Re + (T)
LyT2Cucure + Cem
K8
where the first line of the last inequality is introduced by the previous sampling process, and the remaining term is the discretization error of this phase.
15

---

Improved Discretization Complexity Analysis of Consistency Models
C. The Error of One Step PFODE for the VESDE Forward Process
As a beginning, we control -x, 11 V log a: (X:) previous work, the following lemma is named the score perturbation lemma, which depends on the uniform Lipschitz constant Lscore for the score function. As discussed in Section 4.4, time-dependent results is important for better results for the EDM and exponential decay stepsize.
Lemma C.1. For VESDE (Eq. 1), we have that
Exirg [| Vld.Vloge log (X+))12 S R +
Proof. For the VESDE forward process, we know that (here, we use the forward process notation):
X; - log q: (Xt)
which indicates
V log qi (X₂) [0, V log log 1
- log (X,) V log di (X,)
Hence, the following inequality holds
||a V log di (Xt) || S || [0: V log 9e (v)])yxxx 1l2 v/ log a (xx) log Iog (X2)l12
R + R Xelll
R²2
|Xell2 + R
+
1
1
12
R3
Rd
ta
where the second inequality follows Lemma E. and Lemma E.2 and the last inequality follows Lemma E.4.
When considering the consistency distillation training paradigm, we need to a one-step reverse PFODE starting from Yig to obtain ÿo " Hence, need to control one step starting from the same distribution q.
o2-02 of
Lemma C.2. Suppose Assumption 4.1 and Assumption 4.2 hold and assuming for the small interval & E (411 vk E [0, K 11. we have that
S t/ for any SSSS 8 S T, then
W
T Phi k/ S h
R25
+
R'd (T-
+ 122 score
(T-
where q P means the output of one-step PFODE with fixed approximated score starting from q.
Proof. Fort E 11/, 1. the reverse PRODE is
y, 9(T - t)2
V logg 91-1 (Yi)
2
Ý, g(T 1)2
Yg T t/
ss
for 1/ 515 1/41 with Yú: = Pp, q, Yo,
&QODE, and Piithg, q PODEE Then, have that
th/ &4,+
de y - Poli 2 Y Y,, Yr ).)
t)2 Y; Pr, g(T -)2
(Vlog - s0 T
Yi 1/ +
h hgg(T t)a
Vlog -r- (Yi) ss (Prg T - t//s 11.
$
N/
4
As the next step, we use the Grönwall's inequality to control the one-step discretization error.
16

---

Improved Discretization Complexity Analysis of Consistency Models
Grönwall's inequality. Let my(-) be nonnegative, absolutely continuous function [0, Ng]. which satisfies for a.e. / the differential inequality
(1(1) - (())((()+++++ + ((t)
where (1) and (1) are nonnegative, summable function on [0, k(g. Then
n(()) + L ((s)ds)
(1() S edo o(o)d=
By setting (tt) |r: 1 and (t) -))
V log qr (Yr) - s6 (FeL 1
We note that Ye. Ptg q
starts from the same distribution, r700) 0. Then, we can obtain that
E | - -Y -Pattiil 1
s exp
1/pp// 1
dt)
g(T 19--) h) E
V log (r)) s6 (Ve, T
1/2 {/ df
1/4
1/
4
( t
1/] dt
g(T 1)
V log qr-c (Yt)
86
Then, by using Lemma 4.14, we have that
Ye: thg - Fe, +h4 [|]
E
16t/4
v log - 80 Tth) 16)113] de
-1)"11 F.
(Yi) V logr-- (P:z) ) 112
MT -1)2 dt
g(T 1)
o
g(T M3
|10, log Qr (X))l1 + 1 Score
16
Rdd (T - f2)s (T
R6
1i6 2
t6,
sccore
where the last inequality follows Lemma 4.14.
D. The Discussion on the Previous Work
D.I. The Linear Solution Trajectory of VESDE (o? (2)
In this part, we show why is possible for VESDE (a² (2) generate samples with single Euler step. In this part, we use the forward process notation X, and t to keep consistent with empirical works (Karras et al., 2022: Song et al., 2023). We first provide the exact form of V log q (-). As shown in Karras et al. (2022) and Benton et al. (2024), we have that
E X
X2
X,
V log ge (X,)
t
where E. [X,] | X2) is the posterior mean given X,.
17

---

Improved Discretization Complexity Analysis of Consistency Models
By choosing VESDE (ož (2) the forward process, the reverse process becomes
dx, V log ge (Xx) dt, XT gr-
With the exact form of V log (:(),, we have that
dX;- X1 E [Xo Xi]
dt, XT ~ Qr.
I
Then, starting from Xr, if doing one-step Euler, we have that (we denote the generated samples as Xo)
Xo = XT + (0 XT E No Xr)
[Xo
XT]
As discussed in Karras et al. (2022) and Liu al. (2023a), linear trajectory indicates is possible generate samples with single Euler step and is the basis of one-step model. Hence, consistency models adopt VESDE (o? t²) the forward process (Song et al., 2023; Song and Dhariwal, 2023).
D.2. The Detail Calculation of Previous Consistency Models Results
In this part, show current discretization complexity results and discuss the reason why the noise schedule of Li et al. (2024a) relies heavily on the VPSDE forward process.
The results of Lyu et al. (2024). As shown in Corollary 8 of Lyu et al. (2024), the discretization complexity of consistency distillation (under the bounded support) is
δ d R
R V d3)) L3
EW2
The results of Li et al. (2024a). In this part, we first show the noise schedule of Li et al. (2024a) and discuss the reason why this schedule depends heavily on VPSDE. Li et al. (2024a) deseribe the VPSDE in a discrete perspective instead of a continuous forward process:
Xo Qo:
X₃ v1 - 11-BeXu-1 + Va. l
Let
0%
8k,
0%
II ag: 1 K. ==1
Then, we know that X Vāg.oo + VI- 0g We for some We A (0. Ia), which indicates Xx approximately N (0, Ia) (VPSDE) with suitable noise schedule. Li al. (2024a) choose specific noise schedule
81 = 01
K
e log
C1 log K
k
3x = 1 - 0g
min 31 1
1
2 < K
K
K
where co. C1 are large enough numerical constants. We note that when goes to +oo, 3h will goes to is quietly different from VESDE forward process since of = t² would goes +oo when T goes +oo (K +oc). Hence, this noise schedule heavily depends the form of VPSDE.
With the above specific noise schedule for VPSDE, Li et al. (2024a) show the discretization complexity of consistency training paradigm with VPSDE forward process:
L3db/2
K= δ
eW,
Note that the coefficient of the above PFODE is negative, which slightly different with Eq. 2. This is hecause the forward process notation and the generation timeline Xy + Xo-
18

---

Improved Discretization Complexity Analysis of Consistency Models
The results of Dou et al. (2024). (a) Consistency Distillation Paradigm.
As shown in Theorem 4.1 of Dou et al. (2024), when considering Wi distance, the total error has the following form
L Leconee VM + Vo,
where M is the step number discussed in Section 3.2. To make the above term smaller than ewi we require
L3 and S ei..
M
W,
As shown in (Chen et al., 2023d), we know that o3 = 5 for VPSDE forward process. Furthermore, as shown in Lemma E. we need to choose Lscore = R2/ /o8 R² /82, which finally requires
1 M 2
W
(b) Consistency Training Paradigm.
As shown in Remark 4.5 of Dou et al. (2024), they choose ew, =n=//d where is the size of the training dataset and M = n 20/d Hence, the discretization complexity of the CT paradigm is M 2 1/e
E. Auxiliary Lemmas
In this section, we first prove some regularity results for log (e()) under the VESDE setting. The following two results
come from Yang al. (2024) (Lemma E.1, E.2 and E.4) and are used control Ex [ V log q (Xx)l which is useful controlling the one-step error W3 on P/t..i
Lemma E.1. Assume Assumption 4.1. I. Then for any t (0, T] and X E Rdd we have that
(Vlog X) S - X / + X
In addition, we have
log (X) < 21 + + /o;
and
log qi (x)112 < <(1+ + R²) /0
Lemma E.2. Assume Assumption 4.1. Then for any t (0, and X E Rd we have
g)) R² + [x
1 log qe (X) 11 < 6
Lemma E.3. Suppose Assumption 4.1 holds. Let cw, > 0. (1) If considering VESDE with of =t, we choose the early stopping parameter 8 <wa,
Vá (2) If consider VPSDE, we choose eiv, (Vd(R vdl) then we have Wa (g8, qn) S ewy-
Proof. For the VESDE forward process Equation (1), we know that X, == No + o,2, where normal (0, Ta) is
independent of Xo. Hence, for d $ 1,
W (go. 98) S E ||as2|12 = odd.
Then, for the setting of = 12, we can take 8 < For the VPSDE forward process, directly use Lemma 20 of Chen et al. (2023d) to obtain the final results.
EW2 Vi
'Here, we only discuss dependence on ew, and ignore R,
19

---

Improved Discretization Complexity Analysis of Consistency Models
Lemma E.4. Suppose that Assumption 4.1 hold. Let (Xi)cEjo.T denote the forward process Equation (1). Then, for all
t t 0,
[ I do + R
Proof. We know that X, = Xo + o,2, where Z (0, 7). Hence, we have
E
X =
X 1 + oofd do? + R².
F. Simulation Experiments and Simplified PFODE discussion
In this section, we support our Assumption 4.4 by using the simulation experiments. Since we do not have the closed-form consistency funcciin for highly multi-modal target distribution, we simulate the consistency function f by running the following process starting from (O, I2)a) (We choose T = 50 our experiments):
dY, 9( - (t) 56 T dt' dt',''s t'
It' tk
2
In this part, we use multi-modal Gaussian Mixture distribution as the target distribution. Since the ground-truth score function of Gaussian Mixture Distribution can be calculated, we directly use the ground-truth score function log (:()) (where the target consistency function f also determined with the ground-truth score function) to run the above process approximate f" (In Figure fax is f in the main content).
(" as an approximation.
For the Lipschitz constant of f at a fixed time t', we calculate Y.
Since we want to show that the Lipschitz constant of (' (() has order 1/(T - +') at the beginning of the reverse process (t' is small, / large), we provide reference line (brown line in Figure 1) with 1/t Lipschitz constant. More specifically, after obtaining the Lipschitz constant T, of (", (which always the smallest one), we calculate the points at reverse time t' (forward time T - t') on the brown line by times T/(T f') on L
Baseline.
Observation and Discussion. In this experiment, distribution. As shown in Figure I, the brown and the blue lines almost the same at the beginning of the reverse process (the forward diffusion time is large). Hence, the simulation experiments support our Assumption 4.4.
use 3-modal and 4-modal Gaussian mixture distribution the target
Referenece with L,,t 1/t
Referenece with 4 1/t Lipschiz Constant for fex
Lipschiz Constant for fex
0.02
0.1
0.00
0.0
20 Diffusion Time t
40
20
Diffusion Time
(a) 3 modes GMM
(b) modes GMM
Figure I: Simulation Experiments on the Lipschitz Constant.
20


Reviews:
[{"review_id": "ZFo6PQ1Rck", "reviewer": "Reviewer_KKHE", "summary": "This paper aims to provide a theoretical explanation for the strong empirical performance of consistency models — specifically focusing on how many discretization steps $K$ are needed during training to guarantee high-quality one-step sampling at test time. Prior theoretical analyses of consistency models typically used variance-preserving (VP) forward processes with uniform steps, leading to large and possibly unrealistic complexity bounds. This work, instead, targets the variance-exploding (VE) forward process and the EDM (decay) time-step scheduling. Under these more practical assumptions (matching real applications in, e.g., Karras et al., 2022 or Song et al., 2023), the authors derive improved discretization complexity bounds – polynomial in $O(1/\\varepsilon)$ with exponents significantly better than previous results. They also show that 2-step sampling (a widely used trick in consistency models) can further reduce the required number of steps to achieve a given Wasserstein-2 error.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your valuable comments and suggestions. We provide our response to each question below.  \n\n**Weakness 1: The approximated score and consistency function error (end-to-end analysis).**\n\nIn this work, we assume the pretrained score and consistency function are accurate enough to achieve the final discretization complexity. Though they are standard assumptions in the complexity analysis area, as the friendly reviewer mentioned, the end-to-end analysis is also important, and we can use the current estimation error analysis results to achieve this goal. More specifically, for the approximated score, we use the results of [1] and replace $\\epsilon_{score}$ with $n\\_{score}^{-2/d}$ (where $n\\_{score}$ is the number of data used to train the score function). For the approximated score function, we use the result of [2] and replace $\\epsilon\\_{cm}$ with $n\\_{cm}^{-1/2(d+5)}$. Then, we can obtain the end-to-end complexity analysis.\n\n**Weakness 2: The Results of Multi-step Sampling Algorithm.**\n\nIn fact, our analysis can be extended to $N$-step sampling algorithm and can achieve nearly $L_{f}/\\epsilon\\_{W_2}^{3+1/a}$ (which is better than Thm. 4.7 and Coro 4.12) under the EDM stepsize. We use $3$-step sampling algorithm as an example ($\\tau_1=T,\\tau_2=3T/4, \\tau\\_3=T/2$). Under this setting, the result becomes (here we ignore $\\epsilon\\_{score}$, $\\epsilon_{cm}$, $R,d $ and focus on the dominated term)\n$$\n\\delta+1/T^3++L_f (T / \\delta)^{\\frac{1}{a}} /\\left(K \\delta^2\\right).\n$$\nTo guarantee the above term smaller than $\\epsilon_{W_2}$, we require $\\delta=\\epsilon\\_{W_2}$ and $K\\ge \\frac{L_fT^{1/a}}{\\delta^{2+1/a}\\epsilon\\_{W_2}}=\\frac{L_fT^{1/a}}{\\epsilon\\_{W_2}^{3+1/a}}$, which is the same with the one-step and two-step sampling algorithms. However, 3-step algorithm only require $T\\ge 1/\\epsilon\\_{W_2}^{1/3}$, which is better than $1/\\epsilon\\_{W_2}$ of 1-step and $1/\\epsilon\\_{W_2}^{1/2}$ of 2-step. Hence, the discretization complexity for $3$-step sampling algorithm is $L_f/\\epsilon\\_{W_2}^{3+4/(3a)}$, which is better than $2$-step algorithm. The above steps can be extended to $N$ steps, and the influence of $T$ decreases, and finally, $T$ does not affect the discretization complexity, which leads to a $L_{f}/\\epsilon\\_{W_2}^{3+1/a}$ results. We will add the above discussion in our next version.\n\n**Weakness 3: The Discretization Complexity for Piecewise Discretization Scheme (beyond the EDM with single $a$).**\n\nBefore providing the complexity result for the piecewise discretization scheme, we first discuss the performance of the EDM scheme in the application (Consistency models follow the choice of $a$ of EDM.). EDM [3] shows that when $1\\leq  a\\leq 7$, a larger $a$ will help the diffusion models to achieve better performance (Figure 13 (c) of [2]), which also matches our theoretical results. However, when $a$ is larger than $7$, the improvement is not significant and even becomes worse [3]. As a result, the exponential decay stepsize is theoretically friendly and is not widely used in applications. One possible explanation is that at the end of the reverse process, diffusion models generate image details and require a small stepsize (However, the exponential decay stepsize is too large.)\n\nHence, we can design a two-stage discretization scheme: (a) when $t'\\in [0, T-1]$, we use the exponential decay stepsize; (b) when $t'\\in (T-1, T-\\delta]$, we use the EDM stepsize. With this scheme, the discretization complexity becomes  $L_{f}/\\epsilon\\_{W_2}^{3+1/a}$, which is better than Thm. 4.7 with EDM (single $a$). This result shows the improvement of the two-stage discretization scheme from the theoretical perspective, and we leave the empirical application of this scheme as an interesting future work. We will add the above discussion in our next version.\n\n[1]Oko, Kazusato, Shunta Akiyama, and Taiji Suzuki. \"Diffusion models are minimax optimal distribution estimators.\" In *International Conference on Machine Learning*, pp. 26517-26582. PMLR, 2023.\n\n[2]Dou, Zehao, Minshuo Chen, Mengdi Wang, and Zhuoran Yang. \"Theory of consistency diffusion models: Distribution estimation meets fast sampling.\" In *Forty-first International Conference on Machine Learning*. 2024.\n\n[3]Karras, Tero, Miika Aittala, Timo Aila, and Samuli Laine. \"Elucidating the design space of diffusion-based generative models.\" *Advances in neural information processing systems* 35 (2022): 26565-26577.", "claims_and_evidence": "In this paper, the authors claimed that analyzing VESDE plus EDM steps yields a polynomial discretization bound for consistency models that is significantly smaller than in previous theoretical studies, and this complexity is close to that of the best known diffusion results. In addition, 2-step sampling further reduces the exponent in $\\varepsilon$. \n\nTo show these claims, the authors provided rigorous proof in the main text and appendix. They compared the final complexity expressions to older results, showing strict improvement. Besides, simulation experiments for multi-modal Gaussian distributions illustrate that their key assumption on Lipschitz constants is possible.", "methods_and_evaluation_criteria": "As a theoretical paper, there is no benchmark or datasets needed.", "theoretical_claims": "The paper states each assumption explicitly, references prior standard assumptions (like bounded support for data or Lipschitz continuity of the consistency function). The proofs revolve around standard SDE manipulations, approximate PDE expansions, and the idea that “time-dependent” bounding of the score drift is more precise.", "experimental_designs_or_analyses": "There are no experiments needed in this theoretical paper.", "relation_to_broader_scientific_literature": "This is the first analysis that specifically uses VE forward SDE plus a decaying step approach for consistency. It corrects prior mismatches in theoretical assumptions vs. real usage. The authors connect the final complexity to that of diffusion, bridging a gap that older works left open. They cite relevant works on diffusion complexity (Song et al., Gao & Zhu, Chen et al.), on prior consistency theory (Dou et al. 2024, Li et al. 2024, Lyu et al. 2024). They also mention Karras et al. for EDM steps. Therefore, I would like to say the references are quite comprehensive.", "essential_references_not_discussed": "Nothing crucial seems missing. The standard relevant theoretical diffusion or consistency references appear.", "other_strengths_and_weaknesses": "Strengths:\nThe focus on VE and EDM steps is precisely the realistic setting used in modern SOTA consistency models, bridging earlier theoretical-limitation criticisms. It is a good progress to achieve $\\tilde{O}(\\frac{1}{\\varepsilon^{3+2/a}})$ with 2-step sampling compared with the previous $O(1/\\varepsilon^7)$. The paper is well-organized, with main results clearly stated and strictly proved.\n\nWeaknesses:\n1. The entire analysis relies on an assumption that the score approximation is sufficiently accurate. Is it possible for the authors to remove this assumption and handle the end-to-end training complexity? \n2. The multi-step analysis is restricted to 2 steps; though that’s the main empirical scenario, I still want to ask about the scenario with more steps or sampling schedules. Have you considered a more general $N$-step approach for consistency? Could that yield further improvements or do you expect diminishing returns after 2 steps?\n3. Could your “time-dependent lemma” approach be extended to other step-size patterns beyond EDM (like a piecewise approach)? Are there potential further gains if we do a more sophisticated scheduling than a single exponent $a$.", "other_comments_or_suggestions": "Please refer to the \"Weaknesses\" section.", "questions_for_authors": "Please refer to the \"Weaknesses\" section.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {"ethical_review_concerns": "No ethical concerns since it is a completely theoretical work."}}, {"review_id": "eyVnzI5paz", "reviewer": "Reviewer_a46D", "summary": "This paper examines the convergence of the consistency model under the VE process with a decaying step size. It focuses on consistency distillation and establishes convergence results based on the Wasserstein distance between the generated and target distributions. Additionally, it demonstrates that 2-step sampling enhances discretization efficiency.", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your valuable comments and suggestions. We provide our response to each question below.  \n\n**Q1: Theoretical Claims: The discussion on $L_{f,0}$ assumption and remove it.**\n\nFollowing the suggestion of the professional reviewer, we consider the $L_{f,0}$ in 2-mode GMM in the following **Suggestion 1** and show $L\\_{f,0}$ has the order $1/T$, which is necessary for a $W\\_2$ guarantee. In this part,  we mainly discuss how to remove this assumption. We prove that if considering  a weaker $W\\_1$ guarantee,  **we can remove $L\\_{f,0}=O(R/T)$ assumption and achieve a  $L_f^{1+1/a}/\\epsilon\\_{W_1}$ result**:\n\nWhen considering $W_1$ guarantee,  the first term of line 615 (appendix B) becomes $L_fW\\_1(\\mathcal{N}(0,T^2I_d),q_T)$ (Using uniform $L_f$ instead of $R/T$). Different from the $W_2$ distance, the $W_1$ distance can be bounded by weight TV distance (Case 6.16 [1]):\n$$\nW_1(\\mathcal{N}(0,T^2I_d),q_T)\\leq R\\mathrm{TV}(\\mathcal{N}(0,T^2I_d),q_T)\\leq R^2/T,\n$$\nwhere the second inequality follows the fact of [2]. Hence, we do not require $L_{f,0}=O(R/T)$. The other proof is exactly the same with $W_2$ distance. To guarantee $L_fW\\_1(\\mathcal{N}(0,T^2I_d),q_T)$ smaller than $\\epsilon_{W_1}$,  we require $T\\ge L_fR^2/\\epsilon\\_{W_1}$, which is the source of additional $L_f^{1/a}$. We will add the above result in the next version.\n\n**Q2: Experimental Analysis: the calculation of $L_{f,0}$ in simulation experiments.**\n\nSince $L_{f,0}$ can be obtained by calculate the F-norm of $\\nabla_{Y_0}\\boldsymbol{f}(Y_0,0)$, we calculate the following equal to approximate it\n$$\n\\left|\\frac{\\boldsymbol{f}^{\\boldsymbol{v}}\\left(Y_{t^{\\prime}}, t^{\\prime}\\right)-\\boldsymbol{f}^{\\boldsymbol{v}}\\left(Y_{t^{\\prime}}+\\Delta Y, t^{\\prime}\\right)}{\\Delta Y}\\right|,\n$$\nwhere $Y_{t'}\\sim q\\_{T-t'}$ (sample $1000$ times and take average) and $\\Delta Y = 0.01$. \n\n**Suggestion 1: The Lipschitz constant (Mixture of Gaussian).** \n\nWe sincerely thanks again for the comments. We consider 2-mode GMM $X_0 \\sim 1/2N(\\mu, \\sigma^2I_d)+1/2N(-\\mu, \\sigma^2I_d)$. The score has the following form (Appendix A.2 of [3], we transform it from VP to VE)\n$$\n\\nabla \\log q_t(X_t)=\\tanh(\\frac{\\mu^{\\top} X_t}{\\sigma_t^2+\\sigma^2}) \\frac{\\mu}{\\sigma_t^2+\\sigma^2}-\\frac{X_t}{\\sigma_t^2+\\sigma^2}.\n$$\nSince $f^{\\mathrm{ex}}(Y_0,0)$ the associate backward mapping of the following PFODE (in the following part, we ignore the superscript of $t'$):\n$$\ndY_t=\\tanh(\\frac{\\mu^{\\top} Y_t}{(T-t)^2+\\sigma^2}) \\frac{\\mu(T-t)}{(T-t)^2+\\sigma^2}-\\frac{Y_t(T-t)}{(T-t)^2+\\sigma^2}dt,\n$$\nwe need to solve it to obtain $f^{\\mathrm{ex}}(Y_0,0)$. Since the score is highly nonlinear, it is hard to obtain a closed-form solution. There are two choices to overcome this hardness.  The first choice is to do simulation experiments to simulate the solution (Appendix F). \n\nThe second choice is to add some assumptions on the target data to simplify the above ODE. We assume $\\mu$ is smaller enough to guarantee $\\tanh \\left(\\frac{\\mu^{\\top} Y_t}{(T-t)^2+\\sigma^2}\\right)$ can be approximated by $\\frac{\\mu^{\\top} Y_t}{(T-t)^2+\\sigma^2}$, which simplify PFODE to a linear ODE  (in fact, the distribution gradually closes to Gaussian)\n$$\n\\mathrm{d} Y_t=\\left(\\frac{\\mu^{\\top} \\mu Y_t(T-t)}{\\left((T-t)^2+\\sigma^2\\right)^2}-\\frac{Y_t(T-t)}{(T-t)^2+\\sigma^2}\\right) \\mathrm{d} t,\n$$\nwhich have the following solution\n$$\nY_t=Y_0\\underbrace{\\left(\\sqrt{\\frac{\\sigma^2+(T-t)^2}{\\sigma^2+T^2}} \\cdot \\exp \\left(\\frac{\\mu^2}{2}\\left(\\frac{1}{\\sigma^2+(T-t)^2}-\\frac{1}{\\sigma^2+T^2}\\right)\\right)\\right)}\\_{C(t)}.\n$$\nThe above results indicate\n$$\nY_T=Y_0\\left(\\sqrt{\\frac{\\sigma^2}{\\sigma^2+T^2}} \\cdot \\exp \\left(\\frac{\\mu^2}{2}\\left(\\frac{1}{\\sigma^2}-\\frac{1}{\\sigma^2+T^2}\\right)\\right)\\right)\n$$\nTaking the derivative of $Y_0$, we know that the $L_{f, 0}$ have order $1 / T$. This result also matches our intuition that $Y_0$ has a large variance (order $T^2$ ), and we need to multiply a $1 / T$ to avoid the influence of large variance (lines 282-287).\n\n**Further Discussion on the error of linear approximation.**  The above part makes a linear approximation to simplify the ODE when $\\mu$ is close to $0$, which introduces some small errors. Assuming $Y_0\\sim q_T= 1/2N(\\mu, (T^2+\\sigma^2)I)+1/2N(-\\mu, (T^2+\\sigma^2))$. For the variance, $Y_T=Y_0C(T)$ recover $\\sigma^2$. For the mean, the recover $\\mu$ of the above consistency function is approximately  $\\mu\\sqrt{\\sigma^2/(\\sigma^2+T^2)}$, which is smaller than $\\mu$. However, since assuming $\\mu$ is close to $0$, this error term is small and is possibly introduced by the nonlinear term.\n\nWe will add the above discussion in the next version.\n\n[1]  Villani, Cédric. *Optimal transport: old and new*. Vol. 338. Berlin: springer, 2008.\n\n[2] Yang et al,. Leveraging Drift to Improve Sample Complexity of Variance Exploding Diffusion Models. NeurIPS 2024.\n\n[3] Shah et al,. Learning mixtures of gaussians using the DDPM objective. NeurIPS 2023.", "claims_and_evidence": "The main result, Theorem 4.7, heavily depends on Assumption 4.4, which lacks supporting evidence. See below for details.", "methods_and_evaluation_criteria": "Theory paper, not applicable.", "theoretical_claims": "I find Theorem 4.7 to be not very informative. According to Appendix B, the first term in the error decomposition is $ L_{f,0} R $, which does not asymptotically converge to zero. Therefore, the condition on $ L_{f,0} $ must be strict: even if $ L_{f,0} = O(1) $, the error bound in Theorem 4.7 remains $ O(R) $. Since $ R $ represents the diameter of the target distribution’s support, an error bound of $ O(R) $ is trivial. This paper only establishes $ L_{f,0} = R/T $ for the Gaussian distribution, which is quite limited. To derive a more meaningful result, the paper should rigorously demonstrate that $ L_{f,0} = R/T $ holds for a broader class of distributions rather than merely assuming it (Assumption 4.4).", "experimental_designs_or_analyses": "In Appendix F, the calculation of the Lipschitz constant is not clearly explained.", "relation_to_broader_scientific_literature": "This paper investigates the convergence of the consistency model under the variance-exploding process, whereas prior work primarily focuses on the variance-preserving process.", "essential_references_not_discussed": "No.", "other_strengths_and_weaknesses": "This paper examines the Lipschitz coefficient of the consistency function, a crucial step in understanding consistency models.", "other_comments_or_suggestions": "The author could analyze the Lipschitz constant of the consistency function at $ (x,t) = (0,T) $ for the bimodal Gaussian mixture model $ 0.5 N(-1,\\sigma^2) + 0.5 N(1,\\sigma^2) $ for a small $ \\sigma $.", "questions_for_authors": "None", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "x2q8SG6TTq", "reviewer": "Reviewer_iimn", "summary": "The paper proposes a novel discretization complexity analysis of Consistency Models, by incorporating the variance exploding kernel and the non-uniform step size. The results are closer to diffusion models than previous methods, providing a better analysis of conistency models.\n\n## Update after rebuttal\nThe several reviews and answers clarified some parts of the paper, and I will maintain my previous score.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your valuable comments and suggestions. We provide our response to each question below.  \n\n**Weakness 1: The guidance on the design of better consistency models.**\n\nThis paper makes the first step to elucidate the design space of consistency models under the different diffusion processes and reveals their different advantages and disadvantages, which will heavily influence the discretization complexity and is fundamental in designing the consistency models. For the VP-based consistency models, the early stopping parameter $\\delta$ has order $\\epsilon_{W_2}^2$, which is worse than the VE-based consistency models ($\\delta$ has order $\\epsilon_{W_2}$) and is the source of large discretization complexity. However, the VE-based consistency models also have their disadvantage: the polynomial diffusion time $T$, which is much larger than $T=\\log(1/\\epsilon)$ for the VP-based consistency models and introduces additional $\\epsilon_{W_2}$ dependence. \n\nHence, from the discretization perspective, a better consistency model should enjoy a Logarithmic $T$ and a $\\delta$ with order $\\epsilon_{W_2}$, which would lead to better complexity results. We note that the rectified flow-based one-step models have this potential. We will add the above discussion to our next version and view the design of better consistency models as important future work.\n\n**Question 1: The choice of $a$.**\n\nOur results show that with a larger $a$, the discretization complexity is better than the uniform discretization scheme ($a=1$).This phenomenon is also observed in the empirical work EDM [2], which observed that when $1\\leq a\\leq 7$, a larger $a$ will help the diffusion models to achieve better performance (Figure 13 (c) of [2]). When $a$ is larger than $7$, the improvement is not significant. Consistency models follow the choice of $a$ in EDM. In our Theorem 4.7, we also show that with $a=7$, the discretization complexity  has order $1/\\epsilon_{W_2}^{23/7}$, which is close to the $1/\\epsilon_{W_2}^{3}$ of exponential decay stepsize. \n\n[1] Lyu, Junlong, Zhitang Chen, and Shoubo Feng. \"Sampling is as easy as keeping the consistency: convergence guarantee for Consistency Models.\" In *Forty-first International Conference on Machine Learning*. 2024.\n\n[2] Karras, Tero, Miika Aittala, Timo Aila, and Samuli Laine. \"Elucidating the design space of diffusion-based generative models.\" *Advances in neural information processing systems* 35 (2022): 26565-26577.", "claims_and_evidence": "The claims of achieving better complexity analysis are supported by the proofs. The framework represents more closely what is commonly done in the practice, which results in complexity results more close to the ones of diffusion models, which could help motivating the great empirical performance of consistency models.", "methods_and_evaluation_criteria": "Besides appendix F, there are no evaluations, as the claims are theoretical and supported by proofs.", "theoretical_claims": "The assumptions (4.1 to 4.4) are reasonable and in line with related literature. The results from theorem 4.7, corollaries 4.8 and 4.12 with proofs in appendix B seem correct.", "experimental_designs_or_analyses": "As the paper is mostly theoretical, there is no real experimental section, besides some simulations in Appendix F.", "relation_to_broader_scientific_literature": "Consistency Models are a novel generative modeling framework which can achieve performance similar to diffusion models with significantly less sampling steps. Deepening our theoretical understanding of these models is relevant to further improve their performance.", "essential_references_not_discussed": "N/A", "other_strengths_and_weaknesses": "Given the analysis from the paper, I wonder if one can derive practical considerations to design better consistency models. Having a discussion about this could make the paper more relevant for applied research.", "other_comments_or_suggestions": "Would be useful to name corollaries and lemmas in the same way between main text and appendix.", "questions_for_authors": "1- From your results, is it correct that the complexity decreases as $a$ approaches $\\infty$? Would that mean that in practice, schedules with big $a$ should be preferred?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "E84G3y2IPb", "reviewer": "Reviewer_Fjk9", "summary": "The paper analyzed the consistency model of VE process and decay step size, and proved the discretization complexity of the consistency model.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your valuable comments and suggestions. We provide our response to each question below.  \n\n**Weakness & Suggestion: The analysis for consistency training and  continuous time consistency models.**\n\nAs shown by the professional reviewer, the consistency training and continuous-time consistency models are both important part of of consistency models. In this part, we discuss some possible method to obtain the discretization complexity for this method.\n\n**Consistency Training.** If we can not obtain the pre-trained score function, we can construct an empirical score by using $n$ samples from the target data distribution $\\\\{X_{0,i}\\\\}\\_{i=1}^n$\n$$\ns\\_{\\mathrm{emp}}(X_t ; t)=-\\frac{1}{ \\sigma_t^2}\\left[X_t- \\frac{\\sum\\_{i=1}^N \\mathcal{N}\\left(X_t ; X\\_{0,i}, \\sigma_t^2 I\\right) X_{0,i}}{\\sum\\_{i=1}^N \\mathcal{N}\\left(X\\_t ; X_{0,i}, \\sigma_t^2 I\\right)}\\right],\n$$\nwhich has an explicit formulation, needs no additional training ([1] also use this formula) and converges to the ground-truth score function with a rate $n^{-1/d}$. Hence, we can replace the pretrained score function $s_{\\phi}$ in eq. (4) with $s_{\\mathrm{emp}}$. Then, the $\\epsilon_{\\text{score}}$ becomes $n^{-1/d}$ and achieve the guarantee for consistency models without a pre-trained score function under the VE process and EDM stepsize.\n\nWe note that though this results does not rely on the pretrained score function, it use the reverse PFODE process of diffusion. On the contrary, the consistency training paradigm only use the forward diffusion process. Hence, the above result is not the discretization complexity of consistency training paradigm. To achieve this goal, one possible way is to use similar method with [1], which use $s\\_{\\mathrm{emp}}$ to construct a baseline consistency function (a bridge between the target data distribution and the consistency function learned by the consistency training paradigm)  instead of directly using it in the training objective function.  However, as shown in our Remark 4.10, the construction of the baseline consistency function run $M$-step PFODE instead of one-step PFODE (used in application), which leads a large discretization complexity $1/\\epsilon_{W_1}^{10}$. Since this result is significant larger than our $1/\\epsilon\\_{W_2}^3$ results, we left the discretization complexity analysis (compareable with the CD paradigm) for the CT paradigm under the setting used in application.\n\n**Continuous-time consistency models.** Since continuous time model use $\\frac{\\mathrm{d} \\boldsymbol{f}\\_{\\theta^{-}}(X\\_t, t)}{\\mathrm{d} t}$ instead of $\\boldsymbol{f}\\_{\\theta^{-}}(X\\_{t-\\Delta t}, t-\\Delta t))$ ($\\Delta t$ is $h_{k+1}-h\\_{k}$ in our work. Here we use the uniform stepsize for convenience), there are not well-defined discretization complexity $K=T/\\Delta t$ for continuous-time models. However, due to the absence of $\\Delta t$, the training process of continuous time models is less stable than discrete time consistency models, which is the core problem for continuous time models. Recently, [2] make a great effort to stabilize the training process of continuous time models. \n\nThanks again for the comments on a broader area of consistency models and we will add the above discussion in our next version.\n\n\n\n[1] Dou, Zehao, Minshuo Chen, Mengdi Wang, and Zhuoran Yang. \"Theory of consistency diffusion models: Distribution estimation meets fast sampling.\" In *Forty-first International Conference on Machine Learning*. 2024.\n\n[2] Lu, Cheng, and Yang Song. \"Simplifying, stabilizing and scaling continuous-time consistency models.\" *arXiv preprint arXiv:2410.11081* (2024).", "claims_and_evidence": "The paper bridges the gap between theory and application of consistency models by analyzing the discretization complexity through mathematical derivation and support the main claims.", "methods_and_evaluation_criteria": "This paper is a theoretical work and does not contain any empirical results.", "theoretical_claims": "As I am not an expert in diffusion model theory, it is difficult for me to keep up with some parts of the paper, so I did not check the correctness of all the theorems.", "experimental_designs_or_analyses": "This paper does not include experiments. They provided a discretization complexity analysis of the consistency model in mathematics.", "relation_to_broader_scientific_literature": "In my opinion, this work is the first time it has closed the gap between the discretization complexity analysis for the consistency model and the practical setting. And it overcomes some limitations of prior work such as [1][2].\n\n[1] Zehao Dou, Minshuo Chen, Mengdi Wang, and Zhuoran Yang. Theory of consistency diffusion models: Distribution estimation meets fast sampling. In Forty-first International Conference on Machine Learning, ICML 2024, Vienna, Austria, July 21-27, 2024.\n\n[2] Junlong Lyu, Zhitang Chen, and Shoubo Feng. Sampling is as easy as keeping the consistency: convergence guarantee for consistency models. In Forty-first International Conference on Machine Learning, 2024.", "essential_references_not_discussed": "The author has thoroughly discussed the relevant literature.", "other_strengths_and_weaknesses": "**Strengths**\nThe authors also provide the 2-step Sampling analysis, which is widely used.\n\n**Weaknesses**\nThe conclusion in the article is important for consistency distillation. As is well known, consistency training can independently train consistency models, which is missing in the article.", "other_comments_or_suggestions": "I find no typos currently.", "questions_for_authors": "Please see the weakness. Can the author clarify whether their conclusion can be extended to consistency training and continuous time consistency models.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper studies discretization complexity of consistency diffusion models. Compared to the existing results, the settings in the paper closely aligns with the practice: 1) The forward process is a variance exploding SDE and 2) the time discretization follows EDM. Under assumptions, the established complexity bounds improve over existing ones.

Theoretical contributions are well recognized by reviewers, including the problem setup to reflect real applications, novel analysis to address the challenges, and clear presentation of the main results. These strength indeed justifies an acceptance of the paper.

However, there are outstanding weaknesses preventing a firm acceptance. The concern centers around the assumptions and the scope of Theorem 4.7. In particular, the authors assume that the clean data distribution is compactly supported and the consistency function $f$ is Lipschitz continuous in $Y$ at $t = 0$. These two assumptions on their own do not directly contradict. Nonetheless, later examples (example 4.5 and the simulation in Appendix F) violate the compact data assumption to argue the Lipschitz continuity. This leaves the study much weaker, even though the theoretical results remain valid. Additional simulation results do not help to reinforce the theoretical part.

In order to resolve the issues, my suggestion is two-fold: 1) relax Assumption 4.1 to light-tailed distributions. In that case, we can use a truncation argument to restrict our attention to a bounded domain. Then the discussion on Gaussian and Gaussian mixture models are supported under the assumption, or 2) remove the discussion on Gaussian and Gaussian mixture models, but try to show the clean data regularity leads to Assumption 4.5 (at least, provide a reasonable example, say the clean data distribution is Holder continuous on the compact domain). That being said, either of the fix requires an another round of review. Therefore, I am recommending a weak acceptance.
Decision: Accept (poster)