Adaptive Stochastic Coefficients for Accelerating Diffusion Sampling
Authors: Ruoyu Wang
Beier Zhu
Junzhi Li
Liangyu Yuan
Chi Zhang
Venue: NeurIPS 2025 poster ()
Keywords: diffusion models
solver-based acceleration method
Primary area: general_machine_learning

Abstract: Diffusion-based generative processes,  formulated as differential equation solving, frequently balance computational speed with sample quality.    Our theoretical investigation of ODE- and SDE-based solvers reveals complementary weaknesses:  ODE solvers accumulate irreducible gradient error along deterministic trajectories, while SDE methods suffer from amplified discretization errors when the step budget is limited. Building upon this insight, we introduce AdaSDE, a novel single-step SDE solver that aims to unify the efficiency of ODEs with the error resilience of SDEs. Specifically, we introduce a single per-step learnable coefficient, estimated via lightweight distillation,  which dynamically regulates the error correction strength to accelerate diffusion sampling. Notably, our framework can be integrated with  existing  solvers to enhance their capabilities. Extensive experiments demonstrate state-of-the-art performance: at 5 NFE, AdaSDE achieves FID scores of $4.18$ on CIFAR-10, $8.05$ on FFHQ and $6.96$ on LSUN Bedroom. Codes are available https://github.com/WLU-wry02/AdaSDE.

Adaptive Stochastic Coefficients for Accelerating Diffusion Sampling
Ruoyu Wang"
Beier Zhu
Junzhi Li
Liangyu Yuan
Chi Zhang
AGI Lab,
Westlake University *Nnyynng Technological University
University of Chinese Academy of Sciences
4 Institute of Software, Chinese Academy of Sciences Tongji University
(wangruoyu71 satrlloyy7,,cizhhnneee www.ihanggnwweeeeeee @westlake edu.
beier www.:tu..uuuuuuuuuu
1i jijuuzhi220maalss.uemm uca.aaaiiss.uu.scoooooo.coeem.ee ac cn
Hangyuy001@gmal1.com
Abstract
Diffusion-basee generative processes, formulated as differential equation solving, frequently balance computational speed with sample quality. Our theoretical inves- tigation of ODE- and SDE-based solvers reveals complementary weaknesses: ODE solvers accumulate irreducible gradient error along deterministic trajectories, while SDE methods suffer from amplified discretization errors when the step budget is limited. Building upon this insight, we introduce AdaSDE, a novel single-step SDE solver that aims unify the efficiency of ODEs with the error resilience of SDEs. Specifically, we introduce single per-step learnable coefficient, estimated via lightweight distillation, which dynamically regulates the error correction strength to accelerate diffusion sampling. Notably, our framework can be integrated with existing solvers to enhance their capabilities. Extensive experiments demonstrate state-of-the-art performance: at 5 NFE, AdaSDE achieves FID scores of 4.18 on CIFAR-IO, 8.05 on FFHQ and 6.96 on LSUN Bedroom. Codes are available https://gitai witgggthhbb.... www.eosttaaeeeaattooeoooo AGI Lab/AdaSDDADDD
Introduction
Diffusion Models (DMs) 2 3 5 have revolutionized generative modeling, achieving state-of- the-art performance across a broad range of applications 6 g non-equilibrium thermodynamics, DMs learn to reverse a diffusion process: data are first gradually corrupted by Gaussian noise forward phase, and then reconstructed from pure noise through learned reverse dynamics. This principled design offers stable training and exact likelihood model- ing 15 resolving long-standing challenges in earlier approaches, e.g., GANs 16 and VAEs [7].
12 13
14].
Rooted in
Recent advances diffusion models have highlighted the role differential-cquation solvers in balancing sampling speed and generation quality. We first develop unified error analysis that decomposes the total approximation error into two orthogonal components: (1) gradient error, the discrepancy between the learned score function and the ground-truth score; and (2) discretiza- tion error. introduced by time discretization during sampling. Viewed through this lens, existing solvers exhibit complementary behaviors. Ordinary differential equation (ODE) based methods offer deterministic trajectories with modest discretization error for low number of function evaluations (NFEs), but their performance is fundamentally constrained by the irreversible accumulation of
Equal contribution. CCorresponding author.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

19 20 21 In contrast, stochastic differential equation (SDE) based methods gradient error [18
inject stochasticity that can mitigate gradient error and enhance sample diversity; however, effectively suppressing gradient error in practice usually requires large step counts (e.g.. 10001,00000 1,000 NFEs) 22 Hybrid strategies such as restart sampling(2) [3] alternate forward noise injection with backward ODE integration to combine these advantages, yet they still operate in high-NFE regimes.
Building on the above analysis, we introduce AdaSDE, novel single-step SDE solver that unifies the computational efficiency of ODEs with the error resilience of SDEs under low-NFE budgets. Unlike traditional SDE solvers [24 2 that inject fixed noise based on a predetermined time schedule, AdaSDE employs an adaptive noise injection mechanism controlled by a learnable stochastic coefficient % each denoising step To effectively optimize %%, we further develop a process-supervin optimization framework that provides fine- fineigrained guidance at each intermediate step rather than only supervising the final reconstruction. This design inspired by the observation that diffusion trajectories exhibit consistent oow-dimensional geometric structures across solvers and datasets [25] 26]. By aligning the geometry of the trajectories using nr. AdaSDE reduces gradient error through adaptive stochastic injection, while preserving deterministic efficiency of ODE solvers.
Extensive experiments on both pixel-space and latent-space DMs demonstrate the superiority of AdaSDE. Remarkably, with only 5 NFE, AdaSDE achieves FID scores of 4.18 on CIFAR-I0 (27] and 8.05 on FFHQ 64 [2]], surpassing the leading AMED-Solver 20 by 1.8x. Our contributions are threefold:
We conduct theoretical comparison of SDE and ODE error dynamics, demonstrating that SDEs offer more robust gradient error control.
We introduce AdaSDE, the first single-step SDE solver that achieves efficient sampling (<10 NFEs) by optimizing adaptive y-coefficients. Moreover, AdaSDE serves as universal plug-in module that can enhance existing single-step solvers.
Extensive evaluations on multiple generative benchmarks show that our AdaSDE achieves state- of-the-art performance with significant FID gains over existing solvers.
2
Related Work
Recent advancements in accelerating DMs primarily progress along two directions: improved numerical solvers and maining-based distillation.
2 24 accelerated sampling by improving noise- schedule design, and DDIM (29) later introduced a non-Markovian formulation that enabled de- terministic and much faster sampling. The establishment of the probability-fie ODE view (I5) further unified diffusion formulations and paved the way for higher-order numerical schemes and practical preconditioning strategies, exemplified by EDM [30]. Following this insight, series of ODE/SDE integrators have emerged to push the accuracy-spect frontier. For instance, DEIS [3]], DPM-Solver [21). and DPM-Solver++ (22) exploit exponential integration, Taylor expansion, and data-prediction parameterization to achieve robust few-step sampling. Linear multistep variants, including iPNDM 32 31 and UniPC B3), further enable efficient DMs sampling with 10 NFE.
Improved numerical solvers. Early studies
Hybrid and stochastic extensions extend beyond deterministic solvers: Restart Sampling (3] alter- nates ODE trajectories with SDE-style noise injection: SA-Solver (34 introduces training-fres stochastic Adams multi-step scheme with variance-controllee noise.
Training-based distillation. Two main paradigms dominate this research direction. The first is trajectory approximation, which uses compact student networks to approximate trajectories generated by teacher models, reducing computational steps. This can be achieved offline: by curating datasets from pre-generated samples [35). or online through progressive distillation that gradually decreases the number of sampling steps 36 [ The second paradigm is temporal alignment, which enforces coherence across sampling trajectories by aligning intermediate predictions between adjacent timesteps (3), or by minimizing distributional gaps between real and synthesized data [39 40). While these methods improve generation quality and efficiency, they typically require substantial computational resources and complex training protocols, limiting their practicality. Recent distillation- based solvers-suce as AMED 20 EPD 41 and D-ODE 37 achieve few-step sampling through lightweight tuning rather than full retraining. Complementary efforts on time schedule optimization, including LD3 42]. DMN 43), and GITS (26), further improve efficiency. While most few-step

---

samplers are rooted in ODE formulations, our approach introduces few-step SDE-driven generation by learning stochastic coefficients under a computationally lightweight objective.
3 Preliminaries
3.1 Diffusion Models with Differential Equations
DMs define forward process that perturbs data into noise distribution, followed by a learned reverse process that inverts this perturbation to generate samples. The forward process is designed as stochastic trajectory governed by predefined noise schedule, which can be described by:
dx
3(0 =(0) x + s()) s(t) (t))
where (t) is the monotonically increasing noise schedule, and w denotes a standard Wiener process. This formulation ensures that the marginal distribution Pt (x) at time corresponds to the convolution of the data distribution Po Pdata with Gaussian kernel of variance at) By selecting a sufficiently large terminal time Pr converges to an isotropic Gaussian (Q00, o² (T))), serving as the prior. Sampling is performed by reversing the forward dynamics through either reverse-time SDE in Eq. T or an ODE [5]
dx = (t)) log (x))dt
(2)
Here, the score function Vx Pe(x) is the drift term that guides samples toward high density regions of po- Following common practice [19]. the noise schedule is simplified to (t)) t, reducing (t) to A neural network sx(x,t) t) is optimized through denoising score matching [[5] to
estimate the score function. The training objective minimizes the weighted expectation:
E2 xx, [A(1) (1so(x,,)))) () - Vxx log xo)]?3
where X(() specifies the loss weighting schedule and Pt. (xt xo) denotes the Gaussian transition kernel of the forward process, During sampling, s0(x, t) serves as surrogate for the true score in the reverse-time dynamics, reducing the general SDE in Eq. (2) to the deterministic gradient flow:
dx se (Xe, t)
4
Analysis of ODE and SDE
4.1 Trade-offs Between ODE and SDE Solvers
The choice between ODE and SDE solvers in DMs entails trade-offs among sampling speed, quality, and error dynamics. ODE solvers, characterized by deterministic trajectories, offer computational efficiency and stability through compatibility with compatibility with higher-order numerical methods, egg, iPNDM 332 31.. Such solvers reduce local discretization errors and achieve competitive sample quality with few 10-50 steps 21.19 However, their deterministic nature limits their ability to correct errors from imperfect score function approximations, leading to performance plateaus as step count increases (23 Furthermore, the absence of stochasticity may suppress fine-grained variations, potentially reducing sample diversity compared to SDE-based methods [2).
In contrast, SDE solvers leverage stochasticity counteract accumulated discretization and gradient errors over time, enabling superior sample fidelity in high-step regimes (33 The injected noise further encourages exploration of the data manifold, improving diversity 2 However, these benefits come at the cost of significantly larger step counts (typically 100-1,000) required to suppress errors that scale as (633/2), compared to (62)) for ODEs [23] 44). Moreover, SDE trajectories are highly sensitive to suboptimal noise schedules, particularly in low-step settings (24.. While reverse-time SDEs theoretically guarantee convergence to the true data distribution under ideal conditions [45), their computational cost often renders them impractical for real-time applications.
Recent hybrid approaches, such as Restart sampling 23 reconcile these trade-offs by alternating deterministic steps with stochastic resampling, leveraging ODE efficiency for coarse trajectory simu- lation while resetting errors via SDE-like noise injection. This strategy highlights the complementary strengths of both methods, positioning hybrid frameworks at the forefront of quality-speed Pareto frontiers in diffusion-bace generation. However, Restart sampling still performs under high-step regimes (>50 steps).
3

---

4.2 Error Propagation in Deterministic and Stochastic Sampling
The trade-offs discussed in Section 4.1 raise a key question:
Can SDE-based approaches achieve efficient sampling with substantially fewer steps?
To answer this, we build on the theoretical frameworks of [2 to analyze the total sampling error of ODE and SDE formulations under the Wasserstein- I metric. We begin with the discretized ODE system gD governed by the learned drift field sg. and examine its approximation behavior over the interval it, t Ar] [0, Theorem formalizes this analysis and establishes upper bound on the Wasserstein-1 I distance between the generated and true data distributions (proof in Appendix [B.].
Theorem 1. (ODE Error Bound (23)) Let AL >> denote the discretization step size. Over the interval + At), the trajectory X, = ODE, (x,+At, t + At + t) is generated by the learned drift 80, and the induced distribution is denoted by Pt. poDE,, We make the following assumptions:
AI. Lipschitz and bounded drift: tse(x, t) Lz-Lipschitz x, Lo-Lipschitz and uniformly bounded by L1.
A2: The learned drift satisfies uniform supremum bound: supx,1 ||tso 1) IV log (x)] S ex.
A3. Bounded trajectories: ||xell S B/2 for all t. [t, + At).
The Wasserstein-1 distance between Pt.
ODE, and the true distribution pt satisfies:
ODEd Pt PE
B TV (PEtEL ( tel (At(LzI + Lo) + t
W1 total error
(5)
discretization ervor bound
@ gradient evror bound
where TV(,,)) denotes the total variation distance.
The bound in Eq. consists of two term distinct interpretations. The first term ① is the gradient bound which reflects the discrepancy between the learned score function and the ground-truth one the start time At. also captures the propagation of errors accumulated from earlier time steps. The second term @ is the discretization error bound, which represents the newly introduced errors within the current interval [t, t + At). Since the ODE process is deterministic, any discrepancy between the generated and true distributions at t + At is directly carried forward to time t, without stochastic mechanisms to dissipate it.
Next, we introduce our AdaSDE update over the interval [t, 1 + At). defined as:
AdaSDE,(X*+Ar; At At, 7),
which inserts a stochastic forward perturbaation followed by a deterministic backward process. x At X++ = Xx+ -4-
(Forward process)
x = ODEd A 1+ (1 dt + +),,
(Backward process)
where
(o, ((1+ (1 + -1+
Different from deterministic ODE, AdaSDE introduces controlled noise injection to mitigate error accumulation. Theorem establishes error bound between the generated and the true data distribution for AdaSDE (proof in Appendix B.2
Here, E (0, 1) is tunable coefficient with its optimization deferred in Section 5
Theorem 2.
Under the same assumptions in Theorem Let pt
AdaSDEs denote the distribution after
AdaSDE update over the interval [t, t At). Then
AdaSDE BB TV Wi p AdaSDE pi)
(r+(L+T) (
(6)
gradieut ervor bound
+ (1+)))/221 (1 7) ((1 +t (L + Lo) + + At
discretization error bound
B
where =QQ
2 2 (1 Atj2 12 ()
Q(r Pr(a 2 r)r a

---

0.030
SDE: y- 0.001
SDE: 0.001
y- 0.301
SDE: 0.005
SDE: y 0.005
0,040
5DE. 0.005
0.015
SDC:
ODF: y
ODE: ym0
ODE: y
0.018
0.012
0.024
0.006
0,016
15
steps
steps
steps
(a) Gradient Error
(b) Discrctization Error
(c) Total Erro
Figure 1:
Gradient error, Discretization error and Total error on synthetic dataset across various
steps (measured in 1-Wasserstein Distance). 7 indicates adding no stochasticity (ODE), 7 indicates SDE variants, figures are plotted in Pareto Frontier.
As shown in Theorem 2 the decoupled formulation tightens the Wasserstein- 1 error bound through a reduced coefficient B (1)-) We next formalize this improvement by comparing the gradient-error terms of ODE and AdaSDE formulations in Theorem
Theorem 3. Under the same assumptions as Theorem and Theorem
we denote:
CODE
(V TV Pitsi: P(+ae):
(ODE gradient error)
AdaaSEE grad
B (1- ((1)) TV TV (P++(L+)) P+40+11))
(SDE gradient error)
Then we have @AdaSDE
where the inequality is strict when > 0.
grad grad
Proof sketch. (full proof in Appendix B.3 For the ODE update, Egad depends on the total-variation distance between the distributions time At. For AdaSDE update, EAdASDE includes contraction factor (1 - X(>i) and evaluated at the higher noise level + (1 + A).. Define the Gaussian kernel
|
(>(²) = (2mo?) 4/2 exp
o = (t (1+1
21)²
20?
The distributions after the noise injection satisfy
P++(1+))Ar Pr+Ar Oy:
(c+(1+7)Ar Cr+Ar * Qy.
By Lemma 6 in Appendix, convolution with the same Gaussian kernel does not increase total variation
distance:
TV by D+ Φyy) & TV(p (+4+).
Consequently,
AdaaDE grad
< (1- (1-)()))
CODE
grad
with strictly smaller bound whenever y >
of AdaSDE enjoys tighter bound through B(1 X())), its dis- cretization eиror grows rapidly under large time steps (At) with noise schedules scaling as (t) x At. Specifically, the exponential growth factor d combined with the quadratic At-dependence in + AAt (LzL1- Lo) creates error amplification that scales asymptotically O(Ate CAt)) when 7 ~ O(At). This dominates the improved gradient error control, particularly during critical initial denoising steps where the product + At violates discretization stability conditions. This amplification offsets the benefit of gradient-error contraction, causing total error accumulation along the trajectory and explaining the degraded few-step performance of SDE-based sampling in practice.
Although the gradient
4.3 Synthetic Validation
To verify the error-mitigation capability of stochastic updates in AdaSDE, we conduct experiments on 2D double-circle synthetic dataset, comparing the total, gradient, and discretization errors.
5

---

Setup. As illustrated in Figure we use 2D double- circle dataset consisting of 20, 000 samples uniformly distributed along two concentric circles with radii of 0.8 (outer) and 0.6 (inner), each perturbed by Gaussian noise with a standard deviation of 0.1. We follow the training and sampling procedures of EDM [30] to model the data distribution, employing the second-order Heun method for SDE integration. The stochastic coefficient y is varied over {0, 0.001, 0.005, 0.01}, ), where y 0 corresponds to the deterministic ODE sampler.
To quantify different types of errors, we measure the 2D Wasserstein-1 distance between corresponding distribu- tions. The total error is computed as the distance between the ground-truth data distribution and the generated distri- bution. To estimate gradient and discretization errors, we first construct an intermediate regenerated distribu- tion. Specifically, given the dataset of 20, 000 samples,
Figure 2: Illustration of the 2D double-
circles: two concentric rings with radii 0.8 (outer, blue) and 0.6 (inner, green). We uniformly sample 20,000 points and add isotropic Gaussian noise (o = 0.1).
we perturb each point by Gaussian noise according to Xtm Xy tmid where mid 0.8 and perform one-third of denoising step to obtain the regener- ated samples. The gradient error is defined as the distance between the regenerated distribution and the model-generater distribution at T 80.0, while the discretization error is defined as the distance between the regenerated distribution and the ground-truth distribution.
Result. The gradient error, discretization error, and total error over the steps range E [15, 40] are illustrated in Figure [ is observed that the discretization error of ODES is less than that of SDE variants (in Figure (b)), corresponding to the derived result that the upper bound for ODE sampling error (stated in Theorem M is less than that for SDEs (stated in Theorem 2 by a multiplication factor. However, the gradient error (i.e., error caused by network approximation) of SDEs (r > 0) drops compared to ODE counterparts (in Figure | (a)), validating the Wasserstein-1 distance bound in Theorem The stochastic step is effective in alleviating the gradient eπror made by network approximation. Consequently, as shown in Figure (c). the total error accumulated throughout the sampling process decreases due the reduction of gradient error brought by stochasticity, confirming the effectiveness of our approach in improving sampling accuracy. Given the above theoretical analysis and synthetic validation Wasserstein-1 distance, we present the following remark.
Remark 1. Let (No 7) represent the accumulated sampling error for discretization of N steps with parameter 7.. Then for VN E Z+, Fy E (0, 1) such that:
Erotal( (N,7) S Ecotal (N,ta
5 Methodology
Building on the above theoretical and empirical validation, we introduce AdaSDE, a single-step SDE solver that parameterizes the stochastic coefficient y as learnable variable. This design unleashes the potential of SDE-based solvers under low-NFE regimes.
5.1 Sampling Trajectory Geometry
The trajectories generated by Eq. 4 exhibit low complexity geometric features with implicit con- nections to annealed mean displacement, as established in previous work [26] 25). Each sample initialized from the noise distribution progressively approaches the data manifold through smooth, quasi-linear trajectories characterized by monotonic likelihood improvement. In addition, under identical dataset and time schedule, all sampling trajectories demonstrate geometric consistency across different sampling methods. This geometric insight motivates a discrete-time distillation frame- work. By strategically inserting intermediate temporal steps within student trajectories, we construct high-fidelity reference trajectories. This enables process-superviee optimization that rigorously determines the governing 7 parameters for trajectory segments. Specifically, given student time schedule Taw = (fo, t1,..., ....,,,yy f.3 with N steps, we insert intermediate steps between fn and f,+1 (1) (M)
(denoted Tec (to, to ......o ,,.,...,tty tNl to generate refined teacher trajectories. Notably,

---

Initial
GT/Teacher
Sumpled Trijectory
Deterministic
Stochastic
Supervision
Noise
Trijectory
Step
Step
Lr (R.)
(6) DDPM
(b) EDM-SDE
(6) AdaSDE (ours)
The proposed AdaSDE framework. AdaSDE diverges from traditional heuristic noise injection methods used in DDPM (2 and EDM-SDE [9) Instead, we use intermediary supervision
Figure 3:
from a teacher sampling path to learn and optimize the noise injection process.
Algorithm 1 Optimizing 1
Algorithm AdaSDE sampling
Given: time schedules Tstu and Trea
Given: parameters OI.N. student time schedule Tstu
2: Repeat until convergence
3: Sample Xtx = Yty ~ NV 1)
2: Initialize Xt ~ (N(0,, 1
4: for n N to do
3: for - to do
5:
Sample en ~ IO,
4: Sample En ~ (.
6:
17.5,,,11 t On
5: 1y.E.A, t On
7:
in + tn + Antn
6: in tn + int
8:
X,, Xt P2 12 en
7: X1,, Xt, V 12 17E en
9:
Compute Xta-- using Eq.
8: Compute Xtm- using Eq. (
10: Update On via Eq. (10)
9: end for
11: end for
10: Return: Xto
interpolation scheme employs flexible strategy that allows for selecting different time schedules based on various solvers. This adaptability enhances the fidelity of teacher trajectories.
5.2 Fast SDE-based Sampling
We extend the midpoint-based correction mechanisms Eq. (8) from AMED- MoED-Solver 20 to SDEs, proposing sampling framework that strategically aligns stochastic perturbations with learned trajectory geometry.
X1, X1 + (In (x+1) 80 ( Sn), En E [b]+1,
(8)
midpoint gradient
The parameterization adopts the design from DPM-Solver's intermediate time step construction, formally defined as En. = VEntn+1. This square-root formulation guarantees smooth geometric interpolation between adjacent time steps in the noise scheduling process. Building insights from [46] 47 showing network scaling mitigates input mismatches, we propose learnable parameters {)n, (n to adaptively adjust both exposure bias and timestep scales. The parameters O, are optimized through our discrete-time distillation framework described in Section 5.1 Consequently, Eq. (8) can be reformulated in the following form:
(7us En, An.
X1. Xt + An) (tn, tx+1) (Xe,. + pmm)
(9)
Let (yr,, N denote the reference states of teacher trajectories. Starting from the identical initial noise Yt,, we generate student trajectories by optimizing the parameter sequence ) resulting in student states {x,,, J that align with the teacher trajectories under predefined metric d(.;)) )).
Crucially, since x,, depends on all preceding parameters {O,) N through the iterative sampling process, we implement stagewise optimization by minimizing the cumulative alignment loss at each timestep ln
Lr., (0x) dl (Xt... Vt..)
(10)
7

---

Table 1: Image generation results across different datasets. (a) CIFARIO
55
(unconditional), (b)
FFHQ 28 (unconditional), (c) ImageNet [49] (conditional), (d) LSUN Bedroom
50
(unconditional),
We compared Solver and the training-required method AMED-Solver
20 as well as other
training-free methods. AdaSDE achieves superior performance across all datasets.
CIFARTO 32 32
(c) ImageNet 64 64
49
NFE
Method
Method
5
7
9
Multi- Step Salvers
Multi Step Sulvers
33M-
P2
110.0
24.97
6.74
3.42
DPM-Sober++K3M)
22
91.52
25,49
10.14
6.48
UniPC
3
109.6
23.98
5.83
3.21
UniPC (3
91.38
24.36
9.57
6,34
iPNDM
IPNDM B
58.53
18.99
5.91
Single-Step Solvers
Single-Step
DDIM 29
93.36
49.66
27.93
18.43
DDIM 29
82.96
43.81
21.46
19.27
19
16.76
DPM-Solver-2 Solver 2
21
153.6
43.27
16.69
8.65
DPM. Solver-2 2
21
140.2
59.47
22.02
11.31
DPM-Plugin (ours)
13.75
DPM-Plugin (ours)
108.9
17.03
8.06
AMED-Solver
20
18.49
7.59
4.36
3.67
AMED-Solver
20
38.10
10.74
6.66
5.44
AdaSCE (ours)
12.62
4.18
2.88
2.56
AdaSDE (ours)
18.51
6.90
5.26
4.59
(b) FFHQ 64 64
28
(d) LSUN Bedroom 256 x 256 50
Method
NFE
Method
NFE
3
5
7
Multi-Step Solvers
Multi-Stop Solvers
DPM Solver+<(3)
22
86.45
22.51
8.44
4.77
DPM SoMer++(M)
22
111.9
23.15
8.87
6.45
UniPC
UniPC (3
23.34
8.73
iPNDM
K 31
45.98
17.17
7.79
4.58
iPNDM K 3
80.99
26.65
13.80
8.38
Single-Sier Solveer
Single Step Solvers
21.01
20
86.13
34.34
19.50
13.26
Heun
10
356.5
116.7
54.51
28.86
Heun 19
291.5
175.7
78.66 78.66
35.67
DPM-Solver-2 21
215.7
74.68
36.09
16.89
DPM-Sober-2
227.3
47.22
23.21
13.80
DPM Plugin (ours)
66.31
20.80
14.51
10.48
DPM Plugin (ours)
97.13
21.02
13.68
10.89
AMED-Solver
20
47.31
14.80
8.82
6,31
AMED-Solve 20
58.21
13.20
7.10
5.65
AdaSDE (owrs)
(ours)
In each training loop, we perform backpropagation N times. The comparison with existing SDE
solvers are presented in Figure The complete training algorithm is detailed in Algorithm while the inference procedure is outlined in Algorithm| 2 AdaSDE serves as plug-and-play module for existing solvers. To implement this, train the AdaSDE predictor Algorithm [ by replacing the mean update in Equation (8) with the target solver's formulation.
Experiments
6.1 Experiment Setup
Models and datasets. We apply AdaSDE and DPM-Plugin five pre-trained diffusion models across
diverse domains. For pixel-space models, we include CIFARIO (32 32) [27) FFHQ (64 x 64) [48), and ImageNet (64 x 64) [49]. For latent-space models, evaluate LSUN Bedroom (256 x 256) (50] with a guidance scale of 1.0. Additionally, we consider texl-to-image high high-resolution generation models, including Stable Diffussin v1.5 [5] at 512 x 512 resolution with guidance scale of 7.5.
Solvers and time schedules. We compare AdaSDE against mate-of-the-aee single-step and multi-step ODE solvers. The single-step baselines include training-fre free methods -DDM [29], EDM [99] and
DPM-Solver-2 (2I), as well the hethtweighhtttningg approach AMED-Solver 20 For multi-step
methods, we evaluate DPM-Solver++ (3M) [22), UniPC [33, and iPNDM
32 31 To further
demonstrate the effectiveness of our method, we also conduct a head-to-heal comparison between DPM- and DPM-Solver-2 (11.

---

Table 2: FID results on Stable Diffusion v1.5 [ with classifier- free guidance weight = 7.5.
Table Ablation study of time schedules on CIFAR-10 [27].
NFF
NFF
Method
schedule
MSCOCO 512x512
CIFAR-I0 32x32
DPM-Solvere= (2M)
21.33
15.99
14.84
14.58
Time Uniform t
12.62
4.18
2.88
2.56
AMED-Plugin 20
18.92
14.84
13.96
13.24
Time Polynomial
hp
11.61
10.05
5.14
3.35
DPM- DPM--olver-v3
5
16.41
15.41
15.32
Time LogSNR
21
23.38
10.42
7.96
4.84
AdaSDE (xurs)
30.89
13.99
13.39
12.68
To ensure an equitable and consistent comparison, study faithfully adheres the time scheduling strategies as recommended in the related work [19 (3) Specifically, we implement the logarithmic signal-to-noise ratio (logSNR) scheduling for DPM-Solverf -2, ++(M)) and UniPC algorithms. For other baseline algorithms, EDM time schedule with set to 7 has been employed. For AdaSDE and DPM-Plugin, implement time-uniform schedule.
Learned perceptual image patch similarity While some search-based frameworks employ LPIPS as their distance metric [52]. observed that using LPIPS during the intermediate steps of our method provided no significant performance gains and substantially increased training duration. Consequently, to balance efficiency and final quality, approach utilizes Mean Squared Error (MSE) for optimizing intermediate steps, while applying the LPIPS metric in the final stage to enhance the overall training outcome.
Training details. Our AdaSDE is assessed at low NFE settings (NFE & (3, (3,5, 7,9)) with AFS 53 implemented. Sample quality is gauged using the Fréchet Inception Distance (FID) [54]
over 50k images. For Stable-Diffusion, We evaluate FID as (54), using 30k samples from fixed prompts based the MS-COCO (28) validation set. The random seed was fixed to 0 to ensure consistent reproducibility of the experimental results.
6.2 Main Results
In table I we benchmark AdaSDE against single- and multi-step bascline solvers CIFAR- 10, FFHQ, ImageNet 64x64, and LSUN Bedroom across varying NFF. We observe consistent and substantial improvements in the low-step regime (3-9 NFE). For example, at NFE=9 obtain FIDs of 4.59 (ImageNet) and 5.16 (LSUN Bedroom), while the second-best single-step baseline (AMED- Solver) reaches 5.44 and 5.65, respectively, indicating clear gains. In few-step setting (NFE=3 on LSUN Bedroom), AdaSDE achieves 18.03 FID, markedly outperforming AMED-Solver's 58.21. On CIFAR-IO. NFB=5 yields 4.18 FID (vs. AMED-Solver's 7.59); FFHQ, NFE=5 yields 8.05, substantially better than DPM-Plugin's 's 20.80 and DPM-Solver-2's 74.68. Overall, AdaSDE maintains and often widens- its advantage as the number of steps decreases.
more challenging
We further evaluate AdaSDE on Stable Diffusion v1.5 with classifier-free guidance set to 7.5, reporting FID the MS-COCS validation set (see table 2 At NFE=8/10, AdaSDE attains 13.39/12. surpassing
(2M at 14.84/14.58 and AMED-Plugin at 13.96/13.22 13.4, while remaining competitive with DPM-Solver-v3 across multiple step counts. These results indicate that our adaptive stochastic coefficient not only improves pixel-space diffusion models but also transfers robustly to high-resolution text-to-image generation in latent space. Additional quantitative results are provided in Figures 5 to
6.3 Ablation Studies
Efffec of the stochastic coefficient. We quantify the contribution of the learned stochastic coefficient by comparing AdaSDE with and without 7n on 10, FFHQ, and Stable Diffusion v1.5 (MS-
COCO); see tables [ad Removing "h. consistently degrades FID, with the effect most pronounced
in the few-step regime. On CIFAR-10, FID rises from 12.62 to 13.32 at NFE=3 and from 4.18 to 4.36
NFE=5. On FFHQ 64 64, we observe similar trends: FID increases from 23.80 to 25.85 at NFE=3
and from 8.04 to 8.11 1 at NFE=5. The benefit is especially clear on SD v1.5 (MS-COCO 512 x 512):
when "nn removed, FID rises from 30.89 to 37.23 at NFE=4 and from 13.79 to 16.34 at NFE=6,
while the gap narrows as steps increase (12.68 with in versus 12.82 without at NFE=10). These

---

Table 4: Ablation of "/n CIFAR-10
27 and
Table 5:
Ablation of %n. Stable Diffusion
FFHQ 28
v1.5 5
Training configuration
NFF
Training configuration
NPE
CIFAR-10.324
MSCOCO 512x512
AdaSDE
12.62
4.18
2.88
2.56
AdaSDE W.O. 7
30.89 37.23
13.79
13.39
12.68
W.O. 7
13.32
4.36
2.91
2.63
16.34
14.18
12.82
FFHQ 64x64
MdSDE
23,80
8.04
5.11
4.19
W.O. 7
25.85
8.11
5.12
4.27
results support that injecting learned stochasticity stabilizes few-step trajectories and mitigates error accumulation in low-NFE sampling.
Effect of time schedule. We further compare common time schedules on CIFAR- 1AAR-100La Lo,SS,, EDM (polynomial), and time-uniform summmarrzed in table 3 The time-uniform schedule is the most reliable once NFE is at least 5, achieving FID scores 4.18, 2.88, and 2.56 at NFE=5, 7, and respectively, clearly outperforming the polynomial (10.05, 5.14, 3.35) and LogSNR (10.42, 7.96, 4.84) schedules. At the extreme NFE=3 setting, the polynomial schedule attains a marginally lower FID than the uniform schedule (11.61 versus 12.62), but its performance degrades rapidly as NFE increases. Overall, we adopt the time- lime-rniform schedule as the default for few-step experiments due its robustness across moderate step counts.
Conclusion and Limitation
Conclusion. In this work, we present AdaSDE, novel framework using adaptive stochastic coefficient optimization to fundamentally address the efficiency-qualiiy trade-off in diffusion sampling. It achieves new state-of-the tatte-of-theaald the-art results. such 4.18 FID on CIFAR-10 with only 5 NFE (a 1.8x improvement over prior SOTA). AdaSDE acts as lightweight plugin, compatible with existing single-step solvers and requiring only 8-40 parameters for tuning, enabling practical deployment without full model retraining.
Limitation. When the step size is large and stronger stochastic injection is used (higher 7), local errors can amplify across steps and dominate the total sampling error, leading to instability. In practice, the admissible range of y is constrained by both the dataset and the step schedule, often necessitating conservative time discretization or 7 clipping. Our method's per-step distribution resets and geometric alignment break the linear recurrence assumptions underlying multistep (c.g., iPNDM 31 32), UniPC (33) and predictor-corrector frameworks.
Acknowledgements
This work was supported by the National Natural Science Foundation China (No. 6250070674) and the Zhejiang Leading Innovative and Entrepreneur Team Introduction Program (2024R01007).
References
[1] Jascha Dick-Dics,,ei, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsuper- vised learning using nonequilibrium thermodynamics. ICML, 2015.
[2] Jonathan Ho, Ajay Jain, and Pieter Abbecl. Denoising diffusion probabilistic models. In NeurIPS, 2020.
[3] Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. In NeurlPS, 2021.
[4] Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily I. Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, Jonathan
10

---

Ho, David J Fleet, and Mohammad Norouzi. Photorealistic text-to-image diffusion models with deep language understanding. In NeurIPS, 2022.
[5] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High-resolution image synthesis with latent diffusion models. In CVPR, 2022.
[6] Kesen Zhao, Jiaxin Shi, Beier Zhu, Junbao Zhou, Xiaolong Shen, Yuan Zhou, Qianru Sun, and Hanwang Zhang. Real-time motion-conttoolabble autoregressive video diffusion, 2025.
[7] Lifeng Chen, Jiner Wang, Zihao Pan, Beier Zhu, Xiaofeng Yang. and Chi Zhang. Detail++: Training-free detail enhancer for text-to-image diffusion models, 2025.
[8] Zhanxin Gao, Beier Zhu, Liang Yao, Jian Yang, and Ying Tai. Subject-consistent and pose- diverse text-to-image generation, 2025.
[9] Mingkun Lei, Xue Song, Beier Zhu, Hao Wang, and Chi Zhang. Stylestudio: Text-driven style transfer with selective control of style elements. In CVPR, 2025.
[10] Xin Jin, Yichuan Zhong, and Yapeng Tian. TP-blend: Textual- precise object-style blending in diffusion models. TMLR, 2025.
eextual-prompt prompt attention pairing for
[11] Chenxi Song, Yanming Yang, Tong Zhao, Ruibo Li, and Chi Zhang, Worldforge: Unlocking emergent 3d/4d generation in video diffusion model via training-free guidance, 2025.
[12] Xiangdong Zhang, Jiaqi Liao, Shaofeng Zhang, Fanqing Meng, Xiangpeng Wan, Junchi Yan, and Yu Cheng. Videorepa: Learning physics for video generation through relational alignment with foundation models, 2025.
[13] Wenyu Mao, Zhengyi Yang, Jiancan Wu, Haozhe Liu, Yancheng Yuan, Xiang Wang, and Xiangnan He. Addressing missing data issue for diffusion-based recommendation. SIGIR, pages 2152-2161. ACM, 2025.
[14] Wenyu Mao, Shuchang Liu, Haoyang Liu, Haozhe Liu, Xiang Li, and Lantao Hu. Distinguished quantized guidance for diffusion- based sequence recommendation. In WwW, pages 425-435 ACM, 2025.
[15] Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In ICLR, 2021.
[16] Ian J Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In NeurIPS, 2014.
[17] Diederik P Kingma and Max Welling. Auto-encoding variational bayes, 2022.
[18] Chenlin Meng, Robin Rombach, Ruiqi Gao, Diederik Kingma, Stefano Ermon, Jonathan Ho, and Tim Salimans. On distillation of guided diffusion models. In CVPR, 2023.
[19] Tero Karras, Miik Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffuusion-baed generative models. In NeurIPS, 2022.
[20] Zhenyu Zhou, Defang Chen, Can Wang, and Chun Chen. Fast ode-based sampling for diffusion models in around steps. In CVPR, 2024.
[21] Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li. and Jun Zhu. Dpm-solver: A fast ode solver for diffusion probabilistic model sampling in around 10 steps. In NeurlPS, 2022.
[22] Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li. and Jun Zhu. Dpm- solver++: Fast solver for guided sampling of diffusion probabilistic models. arXiv preprint arXiv:221..01095, 2022.
[23] Yilun Xu, Mingyang Deng, Xiang Cheng, Yonglong Tian, Ziming Liu, and Tommi Jaakkola. Restart sampling for improving generative processes. In NeurIPS, 2023.

---

[24] Alexander Quinn Nichol and Prafulla Dhariwal. Improved denoising diffusion probabilistic models. In ICML, 2021.
[25] Defang Chen, Zhenyu Zhou, Jian-Ping Mei, Chunhua Shen, Chun Chen, and Can Wang. geometric perspective on diffusion models. arXiv, 2024.
[26] Defang Chen, Zhenyu Zhou, Can Wang, Chunhua Shen, and Siwei Lyu. On the trajectory regularity of ode-based diffusion sampling. ICML, 2024.
[27] Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers features from tiny images. Technical Report, 2009.
[28] Tsung- Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In ECCV. Springer, 2014.
[29] Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. In ICLR, 2021.
[30] Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion-based generative models. In NeurlPS, 2022.
[31] Qinsheng Zhang and Yongxin Chen. Fast sampling of diffusion models with exponential integrator. In ICLR, 2023.
[32] Luping Liu, Yi Ren, Zhijie Lin, and Zhou Zhao. Pseudo numerical methods for diffusion models on manifolds. ICLR, 2022.
[33] Wenliang Zhao, Lujia Bai, Yongming Rao, Jie Zhou, and Jiwen Lu. Unipe: A unified predictor- corrector framework for fast sampling of diffusion models. In NeurlPS, 2024.
[34] Shuchen Xue, Mingyang Yi, Weijian Luo, Shifeng Zhang, Jiacheng Sun, Zhenguo Li, and Zhi-Ming Ma. Sa-solver: Stochastic adams solver for fast sampling of diffusion models. In NeurlPS, 2024.
[35] Xingchao Liu, Chengyue Gong, and Qiang Liu. Flow straight and fast: Learning to generate and transfer data with rectified flow. In NeurlPS 2022 Workshop on Score-Based Methods, 2022.
[36] David Berthelot, Arnaud Autef, Jierui Lin, Dian Ang Yap, Shuangfei Zhai, Siyuan Hu, Daniel Zheng, Walter Talbott, and Eric Gu. Tract: Demoising diffusion models with transitive closure dime-distillation. arXiv preprint arXiv:2303.04225: 2023.
[37] Dongjun Kim, Chieh- Lai, Wei-Hsiang Liao, Naoki Murata, Yuhta Takida, Toshimitsu Uesaka, Yutong He, Yuki Mitsufuji, and Stefano Ermon. Consistency trajectory models: Learning probability flow ode trajectory of diffusion. In ICLR, 2024.
[38] Simian Luo, Yigin Tan, Longbo Huang, Jian Li, and Hang Zhao. Latent consistency models: Synthesizing high-resolution images with few-step inference. arXiv preprint arXiv:23/0.04378. 2023.
[39] Axel Sauer, Dominik Lorenz, Andreas Blattmann, and Robin Rombach. Adversarial diffusion distillation. In ECCV, 2024.
[40] Zhengyi Wang, Cheng Lu, Yikai Wang, Fan Bao, Chongxuan Li, Hang Su, and Jun Zhu. Pro- lifiedreamer: High-fidelity and diverse text-to-3d generation with variational score distillation. In NeurIPS. 2023.
[41] Beier Zhu, Ruoyu Wang, Tong Zhao, Hanwang Zhang, and Chi Zhang. Distilling parallel gradients for fast ode solvers of diffusion models. arXiv preprint arXi:25507.47977 2025.
[42] Vinh Tong, Dung Trung Hoang, Anji Liu, Guy Van den Broeck, and Mathias Niepert. Learning to discretize denoising diffusion ODEs. In ICLR, 2025.
[43] Shuchen Xue, Zhaoqiang Liu, Fei Chen, Shifeng Zhang, Tianyang Hu, Enze Xie, and Zhenguo Li. Accelerating diffusion sampling with optimized time steps. In CVPR, 2024.
12

---

[44] Arnak S. Dalalyan and Avetik Karagulyan. User-friendly guarantees for the langevin monte carlo with inaccurate gradient. Stochastic Processes and their Applications, 129(12):5278-53)) 2019.
[45] Brian D.O. Anderson. Reverse-time diffusion equation models. Stochastic Processes and their Applications, 12(3):313-33 -31326,,,111 1982.
[46] Mang Ning. Mingxiao Li, Jianlin Su, Albert Ali Salah, and Itir Onal Ertugrul. Elucidating the exposure bias in diffusion models. In ICLR, 2024.
[47] Mingxiao Li, Tingyu Qu, Ruicong Yao, Wei Sun, and Marie-Francine Moens. Alleviating exposure bias in diffusion models through sampling with shifted time steps. In ICLR, 2024.
[48] Tero Karras, Samuli Laine, and Timo Aila. A based generator architecture for generative adversarial networks. In CVPR, 2019.
[49] Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al. Imagenet large scale visual recognition challenge. IJCV, 115:211-252, 2015.
[50] Fisher Yu, Ari Seff, Yinda Zhang, Shuran Song, Thomas Funkhouser, and Jianxiong Xiao. Lsun: Construction of large-scale image dataset using deep learning with humans in the loop. arXiv preprini arXiv: 1506.03365, 2015.
[51] Kaiwen Zheng, Cheng Lu, Jianfei Chen, and Jun Zhu. Dpm-solver-v3: Improved diffusion ode solver with empirical model statistics. In NeurlPS, 2023.
[52] Zhenyu Zhou, Defang Chen, Can Wang, Chun Chen, and Siwei Lyu. Simple and fast distillation of diffusion models. In NeurlPS, 2024.
[53] Tim Dockhom, Arash Vahdat, and Karsten Kreis. Genie: Higher-order denoising diffusion solvers. In NeurIPS, 2022.
[54] Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea Voss, Alec Radford, Mark Chen, and Ilya Sutskever. Zero-shot text-to-image generation. In ICML, 2021.
13

---

NeurIPS Paper Checklist
The checklist is designed to encourage best practices for responsible machine learning research, addressing issues of reproducibility, transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and follow the (optional) supplemental material. The checklist does NOT count towards the page limit.
Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:
You should answer [Yes], [No]. [NA].
[NA] means either that the question is Not Applicable for that particular paper or the relevant information is Not Available.
Please provide short (1 (1-2 sentence) justification right after your answer (even for NA).
The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include it (after eventual revisions) with the final version your paper, and its final version will published with the paper.
The reviewers of your paper will be asked use the checklist one of the factors their evaluation. While "[Yes] is generally preferable to "[No]" ", it perfectly acceptable to answer "[No] provided proper justification is given (e.g., "error bars are not reported because it would be too computationally expensive" or "we were unable to find the license for the dataset we used"). In general, answering binary way, we
"[No] or "[NA]" is not grounds for rejection. While the questions are phrased acknowledge that the true answer is often more nuanced, so please just use your best judgment and write justification to elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] to a question, in the justification please point to the section(s) where related material for the question can found.
IMPORTANT, please:
Delete this instruction block, but keep the section heading "NeurIPS paper checklist",
Keep the checklist subsection headings, questions/answers and guidelines below.
Do not modify the questions and only use the provided macros for your answers.
1. Claims
Question:
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
The abstract and introduction clearly articulate the primary contributions of the paper. These include the proposal of AdaSDE, the first single-step SDE-based solver designed to unify the efficiency of ODEs with the error resilience of SDEs. The paper makes specific claims that are substantiated by both theoretical analysis (error decomposi- tion into gradient and sampler errors, with provable improvements under stochastic noise injection) and empirical results (e.g., SOTA FID scores CIFAR-10 10 and FFHQ at NFE). Additionally, the adaptive noise coefficient y and the process-supervised training method are explicitly introduced in the introduction key components. The scope and impact the method are realistically positioned, and there are no unattainable aspirational goals presented achieved results.
Justification:
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
14

---

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations the work performed by the authors?
Answer:
[Yes]
Justification: We acknowledge both theoretical and practical limitations. Theoretically, AdaSDE relies assumptions such as Lipschitz continuity and boundedness of the learned score function, which may not hold in all diffusion models or datasets. Additionally, the method assumes the geometric regularity of sampling trajectories across solvers and datasets, which might be violated in more complex or high-dimensions generative tasks.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model sell-specificatinn, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
Justification: We offer a formal theoretical analysis in Section 4 and Appendix to support the claims made about AdaSDE's sampling enror decomposition and contraction proper- ties. Each theorem is stated with all necessary assumptions, including Lipschitz conti- nuity, bounded score approximation error, and norm constraints sample trajectories. These assumptions are explicitly enumerated before each theorem (c.g., Theorem on the Wasserstein-1 bound for ODEs and Theorem 2 for AdaSDE). The corresponding proofs either outlined in the main text or fully detailed in the appendix with supporting lemmas (e.g., Lemma and Lemma 2), and referenced accordingly.
Theorems involving the role of y in modulating error bounds are rigorously derived, with explicit expressions of how y affects both gradient and sampler errors. Furthermore, the
15

---

paper provides interpretation and intuition alongside the formal derivations (e.g., contraction factor -()))) and confirms theoretical insights with synthetic experiments (Figure 1).
All notation defined in Appendix A, and every proof follows a clear structure with mathematical rigor and consisteney. The manuscript adheres closely to NeurlPS standards for theoretical completeness and reproducibility.
Guidelines:
The answer NA means that the paper does not include theoretical results, All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: We describe all experimental setups in detail, including datasets, evalua- tion metrics (FID), solver configurations, time schedules, and training hyperparameters. Algorithms are provided in pseudocode (Algorithm & 2), and the number of function eval- uations (NFE) is specified for each result. In addition, the complete code and reproducibility instructions will be released upon publication, ensuring that all results can be independently verified.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access to hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily the architecture clearly and fully.
model architecture, the paper should describe
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
16

---

(d)
We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: The datasets used (CIFAR-10, 10, FFHQ. ImageNet 64x64, LSUN Bedroom, MS-COCO) are standard and publicly accessible. We will release the full codebase and configuration files upon publication, including scripts for reproducing all reported results. The supplemental material will include detailed setup and usage instructions to ensure faithful reproduction of the main experiments.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.cm cc/
public/guides/ Sttpees/DooeeSummisiiiinnPPeee
for more details.
While encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
/ //nips. //nips.ccppubl...../ http://cppp.cggcieeeeeoooooeoeea
for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing much information possible in supplemental material (appended the paper) is recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We specify the datasets used (CIFAR-10, 10, FFHQ 64x64, ImageNet 64x64, LSUN Bedroom, and MS-COCO for Stable Diffusion), the number of evaluation samples (50k or 30k), and the metric (FID). We describe the number of sampling steps (NFE), guidance scales, and time schedules (uniform, logSNR). For training, we detail the process- supervised optimization of the stochastic parameters 7, including the student-teacher time schedules and stage-wise loss. The solvers and comparison baselines are also listed clearly, along with ablation study settings. All relevant experimental settings are included in the main text or supplemental material.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can provided either with the code, appendix, or as supplemental material.
17

---

7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: To ensure the reproducibility of experimental results, we fixed the random seed O throughout all experiments. Under this controlled condition, the observed variations across multiple runs were found to be statistically insignificant (p > 0.05). While the paper does explicitly report error bars for every experimental result, the consistency achieved through fixed random seeds provides an alternative approach to demonstrating result stability.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall
with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report I-sigma eгror bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error that would yield results that are out range (e.g. negative error rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: We conducted our experiments using NVIDIA A800 and 4090 GPUs with 80GB and 24GB memory, respectively. We train θ for 10K images, which only takes 5-10 minutes on CIFAR10 with single 4090 GPU and about 20 minutes on LSUN Bedroom with 4090 GPUs. We did not use large-scale scge-scale distributed clusters or proprietary cloud services, and all experiments can be reproduced on a single modern GPU.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported the paper (e.g., preliminary or failed experiments that didn't 't make it into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics htttpp////aarrrpp.cc/////////s/s
18

---

Answer: [Yes]
Justification: We have reviewed the NeurIPS Code of Ethics and ensured that our work adheres all relevant principles, including responsible use of computational resources, transparency of methodology, acknowledgment of potential societal risks, and respect for reproducibility and openness. Our study does not involve human subjects, sensitive personal data, or deployment in high-stakes applications. We take care to clarify limitations and ensure that our findings are not misrepresented or overstated. All datasets used are publicly available, and no private or proprietary data were involved.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations their jurisdiction).
Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: We acknowledge that diffusion- based generative models, including our pro- posed AdaSDE framework, may be used to synthesize realistic images at scale, which could potentially be misused for generating misleading or harmful content (e.g., deepfakes or misinformation). Although our work focuses on improving sampling efficiency and does not directly involve content generation pipelines or personalization, we recognize that enhanced accessibility and speed may lower the barrier to misuse. We encourage future research to pair such generative techniques with detection, watermarking, and auditing tools to ensure responsible deployment.
Guidelines:
The answer NA means that there is no socictal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g.. deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative socictal impacts, the authors could also discuss possible mitigation strategies (c.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, scraped datasets)?
Question:
Answer: [Yes]
19

---

Justification: While our work does not release new pretrained model or dataset, it proposes an efficient sampling strategy (AdaSDE) applicable to existing diffusion models. As such methods could be used to accelerate the generation of high-quality synthetic media, we recognize potential risks such as misuse in producing harmful or misleading content. To address this, we include clear statements in the paper discouraging the application of AdaSDE in domains involving safety-critical or deceptive use without human oversight. We also plan to release code only for academic and research use under an appropriate license, and we do not release any generated content or fine-tuned models that could pose direct safety concerns.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: We use publicly available datasets, including CIFAR-10, FFHQ 64x64, Ima- geNet 64x64, LSSU Bedroom, and MS-COCO. Each dataset is cited in the paper with its original reference, and we confirm that all are released under licenses permitting academic research use (e.g., MIT, CC-BY, equivalent). We also build on standard pretrained diffu- sion models (e.g., EDM, DDPM, Stable Diffusion), which are properly cited and used under their respective open-source licenses. No scraped or proprietary data used. We include version references and license information where applicable in the paper and supplemental material.
Guidelines:
The answer NA means that the paper does use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that souree should be provided.
assets are released, the license, copyright information, and terms use in the package should be provided. For popular datasets, thcode thcode. com/ con/dataseta
has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors encouraged to reach out to the asset's 's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
20

---

Justification: We do not introduce any new datasets, pretrained models, or other standalone assets in this paper. All our experiments are conducted on existing, publicly available datasets and pretrained diffusion models, which are properly cited.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to amonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: Our work does not involve any human subjects or crowdsoureing data collection, annotation, experimentation. All datasets used are publicly released and do not contain personally identifiable information.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [Yes]
Justification: All contributions to this work, including ideas, implementation, writing, and experiments, were made by the listed authors. We have additional contributors or external collaborations that need to acknowledged. If relevant acknowledgments arise later (e.g.. code release support), we will include them in the final camera-ready version.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
21

---

Appendix
A Notation and Symbols for the Proof
This subsection provides comprehensive list of notations and symbols specific the theoretical proof. The definitions align with the conventions in stochastic calculus and diffusion model analysis. We build on the notations [23].
A.1 Common Terms
ODEa(-) Approximate ODE trajectory using the learned score sø (x,x t).
Pr : Truee data distribution at noise level
P: Distribution generated by simulating ODEs.
ODEs
B Norm upper bound for trajectories, satisfying VI, ||x,|| < B/2.
Pt : x2 is sampled from distribution pt-
A.2 AdaSDE Terms
Al ODE discretization step size.
Hyperparameter controlling the noise injection ratio in the AdaSDE process.
AdaSDE forward process: X++ + (++2)--c+((+))):
e: Gaussian noise ~ (),
x7
AdaSDE backward process: ODE, (x & + (1 (1+7)44 7)At + +t).
AdaSDE, (x, 7) Applies the AdaSDE operation with parameter y state
Xy: The solution to d&, = -180 (x++ / + At) di,
A.3 Lipschitz and Error Bounds
Lo Temporal Lipschitz constant:|[sso(x./ 1) 180 8)ll S Loll s)
L Boundedness of the learned score: ||,00( 1)ll s Ty--
L2 Spatial Lipschitz constant:|| t) tso (y, t)ll (y,
L2[[x yll
6 Score matching error: [V log (x()) - t5 (x, t)ll
A.4 Special Operators
ODE (x, f1 - 12) Ground Truth backward ODE evolution under the exact score from f to 1z2
ODE, (x, f2) Approximate ODE evolution using the learned score sø- #: Convolution operator between distributions, e.g., P * denotes the convolution P and
R.
+ Time-reversal marker, c.g., x
A.5 Key Process Terms
pr
Distribution noise level f. after applying the AdaSDE process starting from state
AdaSDEe Distribution generated by the AdaSDE algorithm.
p,
E, iid Gaussian noise: & N o2) N 10).
22

---

A.6 Error Dynamics
<(t) X x Error dynamics in the time-reversed coordinate system in t.
A(7): Noise merging probability:2Q
( where Q(r) = Pr(a 2 r) for
a .V(,,)) 1).
:i(+:)) Wasserstein-1 distance.
TV (++): Total Variation (TV) distance.
Where +== N (o, (1+ (t+ At)? I). For the sake of simpli- fying symbolic representation and facilitating comprehension, in the following proof, Ad, ?) to denote x in the above processes. In various theorems, we will refer to a function Q(r): R 1/2), defined the Gaussian tail probability Q(r) Pr(a 2 for NO0, 1).
B Proofs of Main Theoretical Results
Lemma 1 (Upper Bound on ODE Discretization Error). [23] Let ODE (X++se, At + 1) denote the solution of the backward ODE under the exact score field, and X; ODEg (R+Ai + At t) denote the discretized ODE solution using the learned field sg- As
sume Se satisfies:
1. Temporal Lipschitz Continuity:
[|t,o txx((x S Lalt - s| Vx,t,
2. Boundedness:
||(0((x)))| by Vx,
3. Spatial Lipschitz Continuity:
[xtsa(xt)) (x, t) (y)lly L2|1x yll Vx,y,t
Then the discretization error satisfies:
Xx xx S eba (||X+AI - X +
Lo) + EE) At)
Proof. Step 1: Definition of Hime-Reversed Processes
Introduce time-reversed variables x and & governed by: where the integer satisfying
t E [1', + At), corresponding to discrete timesteps.
Step 2: Error Dynamics
Define the error e(t) == ||x; x1- Its derivative satisfies:
e(t) < || log (xx - -sa (Xt+ + At)ll
dt
Decompose the right-hand side:
I log Pa (xt) tsp (xr +I
Approximation Error
+ | (xi, .1) - 1800 (x, 1)1
La]lx] -x-1
|ss (xf t) tsg (x t' At)
Temporal Discrctization Error
23

---

Step 3: Temporal Discretization Error Bound
Further decompose the temporal discretization error:
< Ilso (xt ()) - 18g (x 4) + |leso (x - 1se (xi+a0 l' + A!)
Ln Il' + Al - 1'] + L2 |x - x (Lipschitz continuity)
Ln + L2 (| - x
Using the boundedness condition ||dx; /dtll S L1, have:
+ $
|xx; ds At
Step 4: Composite Differential Inequality
Combining all terms, the error dynamics satisfy:
_ c(t) S Lze(t) + (er + LoAt L2L1
dt
Step 5: Gronwall's Inequality Application Integrate over [t, + At) and apply Gronwall's inequality:
(t) cl (e At) (a ALo L2L1)) At)
Lemma (TV Distance Between Gaussian Perturbations). Let &nr NN(0,02la) and &y (0, o²Ia) be independent noise vectors. For x' x Ev and y' = + Ey, their total vari- ation distance satisfies:
|xx |X--yl
TV(x', = 1- 2Q
20
where Q(r) = Pr (a
Proof. Let =x y. The TV distance is:
TV(x' f IN(2:x, -N(z;y, 2 JR'
W(z - 0;0,021a) VMzz-80,,021 m
01 02 dz
Through orthogonal transformation U aligning å with the first axis:
U8 [1[8] 0, O,....) 0)
By rotational invariance Gaussians:
TV(x', (') TV (V(118)) (2)
For 1D Gaussians (iu, 2²) and N(, 2)
1
2
2
TV
dz
2
+
(By symmetry)
20
20
11
H
2Φ
2Q
20
2
where μ = _/||x> -|x-|Xx0 |/l- then:
|8||
(IIx [|x-yl) yll
TV(x' = 11-22
-Q
20
2a
24

---

Lemma 3. Let P and " denote the densities x,, and y? respectively. After applying AdaSDE with noise injection from t (1 At followed by backward ODE evolution, we have:
TV (p. Kn (1- (pt
B
where X()) 2Q
+ (1+ f2
14.-yele yell S B. The AdaSDE process first perturbs both states to noise level / + (1 + 7) through Gaussian noise injection:
Proof. Consider states x, and y, noise level with |x,--
X++(1+7) = Xx Eas & NO, [(t + (1 (1+)) (2)T)
(r+(I+))Ar = Yt + Ey- Ey ~ (O [t (1 + 7) (2)))
We construct a coupling between the noise injections: when x, = set &. = Eyy otherwise use reflection coupling. By Lemma the merging probability satisfies:
||x - yrll
B
X()) = 2Q
2 2Q
(since ||x - y:ll - B)
20,(7)
(20((1))
where Q(r) = Pr Pra~N(o.1) r).
This implies:
((X(+(1+)))r # (++(1+)) x # S -1)(
where X(7) quantifies the minimum merging probability between the Gaussian perturbations.
The subsequent backward ODE evolution preserves this coupling relationship because both trajecto- ries are driven by the same learned score 8g. Therefore:
P(x + v?) s (1 - ((7)) yr)
Through the coupling characterization of total variation distance, we conclude:
TV(p .r (1 A(7))TV(P) p()
Lemma 4 (AdaSDE Error Propagation). Let X++At E Rd be an initial point. Define exact and approximate ODE solutions:
ODE (x++(1)))); ++ -)) (*+((1+)))))) O t (1+ () t).
& ODE,
Under AdaSDE with noise injection t + At - t + (1 At and ||x -xe1l S B, there exists coupling such that:
Xx+(1++)Ar- *14(1+))ad1 < (((+++)))(111
+)) 7 (A4 + To) + 6 Ar,
where Lo, LA, L2, ct are the Limshitcoonneenessapppostttttt constants for so and discretization errors.
Proof. By Lemma T (OEE Discretization Error), the local truncation error satisfies:
||x x S (a(()))))
[ 81+(1+))4ell
+ ((L + Lo) + c1) A)
Local discretization
Applying AdaSDE's noise injection with variance o² = -1+ (1+ - - Lemma 2 gives:
B|xx++(47)) 8e+(1-))ar) 5 [(1))x &
B
where the merging probability A(r) =2Q(
dominates the coupling
2 (t+ (1+ Atj
effectiveness.
25

---

Multiplying by (1 - X())) from partial revert and adding the local ODE approximation error leads to the stated bound:
X++(1+>00) (+(++000)
x - -(7) || x--- - &
(22(1)+)) (1+ 71+7) +1))(( + La) + Ee] At
=e(allt))ar (1 L2l + Lo) + ad] ai Al
Lemma 5 (Connection of Wasserstein-1 distance and Norm). Let Pı and P2 be two probability distributions over space X Rd, and let T(pip2)) denote the set of all joint distributions with marginals and p2. The Wasserstein-1 distance between p1 and p2 satisfies:
Wi(p1. inf (x |||x x2lll.
(P
where || l1a is the LI norm. Furthermore, for independent samples X1 P1 and X2 ~ p2, we have:
W (1 P2) S S E - x2
with equality if and only if the coupling &b is optimal.
Lemma 6. TV(P R, Q Q R) < TV(P, Q) for independent distributions P Q, and R. The inequality
TV(P * R, Q R) = Q) holds if and only if R is degenerate distribution.
Proof. 1. Total Variation Distance Definition
The total variation distance between two distributions and Q is defined as:
TV(P, Q)
lp(x) 4(x))dxx
where p(x) and g(x) are the probability density functions of P and Q, respectively.
2. Convolution Definition
The convolution two distributions and R is defined as:
(P* R)(x) / p(x - y)r((r)
Similarly, for Q and R:
(Q R)(x) / 00 -(x-x) y)rr))
3. TV Distance for Convolved Distributions
We want compute TV(P R R), which is:
TV(P R2Q * R) 2
I(P R)(x) (Q R)(x) da
00
(p(x y) 9(x ())r()) dx
Applying triangle inequality, we obtain:
TV(P R) / (
1p(x 1p(x-10 y) g(x y))r(r)dy) dx
Using Fubini's theorem, we can swap the order of integration:
TV(P * R2Q R) S 1 f
|p(x y) -g(x y)1dx) r(y)dy (
26

---

For fixed y, the inner integral is:
/
1p(x - -) y)- 1(x y)]dx = / (x(x) =xx)]xxx
Thus, we obtain:
TV(P R Q R) S / ( [x(x) o(x)]axx) r(y)dy
TV(P R. TV(P.Q
The inequality TV(P R,Q R) = TV(P, Q) holds if and only if R is degenerate distribution.
B.1 Proof of Theorem 1
Theorem 1. Lert + At be the initial noise level. Let = ODEs (X++Ai, At + t) and ODE, denote the distribution induced by simulating the ODE with learned drift sg. Assume:
1. The learned drift tss (x, t) is Lz-Lipschitz in x. bounded by L1. and Lo-Lipschitz in t. The approximation error |[tsg( (x. t) tV log (x))l S ct.
3. All trajectories are bounded by B/2.
Then, the Wasserstein-1 distance between the generated distribution Pt Pt is bounded by:
ODEs and the true distribution
W1 (pt ODEo .P.) BBBTT TV (P + clase (St (L271 + Lo) + Ec) At
where At is the step size
Proof. Let X; = ODE, (X++a t + At t) t) with the corresponding distribution P and x, ODE (X++Ar,t t At +) (simulated under the true score). The proof bounds W1 pr oDEs Pt via triangular inequality:
ODE, pp PP S Wi ODEa pb) W, (pe.Pt)
W1
(11)
Then we can bound two terms seperately.
1. gradient error: By bounded-dianee inequality,
Wi ODEE, p..) < B TV (poPEr ( PerSt)
2. discretization error: Using Lemma (discretization bound), given x, Dr, ~ Pr.
|x x La (At (LoL1 + Lo) +t
where the exponential factor arises from Gronwall's inequality applied to the Lipschitz drift. Accord- ing Lemma 5 combine terms via triangular inequality:
ODEs Pt
PI) < B TV (p PODEE PL+A)) + LeAt (At (L2L1 + Lo) + Ea) At
Wi
gradient error
27

---

B.2 Proof of Theorem 2
Theorem 2 (AdaSDE Error Decomposition), Consider the same setting as Theorem denote the distribution after AdaSDE iteration. Then
Let pt
AdaSDEn
W1 (pt AdaSDE,
AdaSDE .Pt) pp)) - B (1 (()))TV (1+((++)) Pc+(1+))
gradient error
el+t))2at(11 + 7) ((1 a) (I2I4 Lo) + ()) At
discretization ervor
B
where x(7)=20
= 2Q
+t+ -
Proof. Let (++(1+))ar ()+(1+)) and (1+()+))
(++(1+)) denote exact and generated
AdaSDE
distributions respectively. And (1++))) Pi-(147)A)' The proof contains three key components:
By Lemma 3 the AdaSDE process contracts the TV distance:
| xel1 (1 X(1)) &+-(+))aell
(1 X(1)) x())) X(-(1+))A)))
Since X2 ~ pl and &2 ~ pt AdaSDE, we obtain:
TV Pr, AdasDEo)
< (1 X(7))
=(1 (1 T()))VVVVVVV) (BL+(I+))A Pc+
Using the bounded trajectory assumption ||x|| - B/2, we convert TV to Wasserstein-l:
Wi (fr. AdaSDEa)
< B
TV
< B(1 TV ((444)))) Pr. Pr AdaSDEa)
From Lemma the local ODE error satisfies:
xx]
x s e(++))aas! (1++)))) (1+ +))(((221 + Lo) + ea] At
According to Lemma and Apply triangle inequality to Wasserstein distances:
Wi AdaSDE, .Pt)
< Wi (pr. Pt
AdaSDE,
Wi (p:,
< B(1 (()))TV
AdaSDE (4H0H)))-PP+04-))00
+ (1+))aaa111 +
+ Lo) + +c]
This completes the error decomposition.
B.3 Proof of Theorem 3
Theorem 3 (TV comparison: AdaSDE vs. ODE). Assume the same conditions as in Theorem | and Theorem 2, and in particular that there exists a compact K Rd with (iam(K) B such that the relevant one-step distributions are supported in K. Define
ii) ODE gradient:
CODE : B TV( ODE,
Pr+Ae: P(+At).
(ii) AdaSDE gradient:
AdasDE : B (1 X(1)) TV AdaSDE
(P++0+7) Pc+(1-9)at)
where =0
(0, and B > 0 is the diameter bound. Then
21 (1
A1
edasSE grad
Egad
28

---

Proof. By Theorem |
CODE grad
B
Pi+S0: Pi+se).
By Theorem
AdaaSDE =
(1- ((>)) TV
AdaSDE (1+()+))A P++(1+7)
grad
From ++ to (1+ At, AdaSDE injects Gaussian noise (a common Markov kernel) into both
branches. By Lemma connolltion/puushoowwaa is nonexpansive in TV),
AdaSDE TV
(1++(++) P+(0+1))))
ODE, TV(P+AV P(+de).
Since 0 < (1-)())) 1, we get
CAdaSDE = (1 - A(7)) TV (11+a2t M P(+(I+7)0)) S B T. ODE, "grad
AdaSDE
Pi+AC: P(+Ae) = Egaa
Remark 2 (When the inequality is strict). If 7 > O, the Gaussian kernel is nondegenerate, and TV(pi-t: P(+ae) > (equivalently, the two pre-smoothing distributions are not u.e. equal and admit L densities), then
TV
AdaSDE (++(1+)) P(+(+4))))) < TV TV Pe+ pt+
and hence EAdaSDE 'grad
CODE
grad
C More on AdaSDE
C.1 Experiment details.
Experiment detail in main result
Since AdaSDE has fewer than parameters, its training incurs minimal computational cost. We train θ for 10K images, which only takes 5-10 minutes on CIFARI0 with a single 4090 GPU and about 20 minutes on LSUN Bedroom with four 4090 GPUs. For generating reference teacher trajectories, we use DPM-Solver-2 with M=3. For tuning across all datasets, we employed learning rate of 0.2 along with a cosine learning rate schedule (coslr). The random seed was fixed to to ensure consistent reproducibility of the experimental results. To ensure the robustness of our experimental results, we conducted ten independent runs for each NFE (Number Function Evaluations) setting on the CIFAR10 dataset. Across these runs, the FID (Fréchet Inception Distance) scores consistently varied by no more than 0.1.
C.2 Time uniform scheme
proposes a discretization scheme for diffusion sampling given the starting maxx end time miin and 6.. Denote the number of steps as N, then the time uniform discretization scheme is:
a(t) 0.5 Ba +Bmm -1) 0.5
B + 2,3a In(o2 + Bmin
(g)
Ba
+ +1) (In(on
In In(o2 ( + 1))
84
6x 11
Bmin
ln(02 +
0.. Bd
i
(2/0--1) "
tremp
1; = =<(empp)
We set maax 80.0, amin 0.002, and 6, 10-3 across all datasets in our experiments.
29

---

C.3
Supplementary experimental results
Table 6: Evaluation on MSCOCO 512x512 (Flux.1-dev).
Model
NFE
Sampler/Method
FID 4
CLIP (%) 1
DPM-Solver-2
54.09
28.49
AdaSDE
35.32
29.94
Flux.1-dev 512x512
DPM-Solver-2
30.17
29.75
AdaSDE
26.51
30.51
DPM-Solver-2
26.32
30.32
AdaSDE
23.54
30.77
DPM-Solvart H2MD
AdaSDE(OOus)
12 NFE
16 NFE
20 NFE
Figure 4: Comparison of image synthesis quality under identical NFE constraints using AdaSDE (ours) and DPM-Solver++ (2M). Both methods generate images with Stable Diffusion v1.5 (5) and classifier-free guidance (scale = 7.5) for the prompt "A photo of some flowers in a ceramic vase".
30

---

Table 7: Unconditional generation results CIFARIO 32x32.
NFE
Method
AFS
4
10
DPM-Solver-yy
15.10
11.39
8.96
8.27
109.6
45.20
23.98
11.14
5.83
3.99
3.21
2.89
UniPC
54.36
20.55
9.01
5.75
4.11
3.26
2.93
2.65
110.0
46.52
24.97
11.99
6.74
4.54
3.42
3.00
DPM-Solver+++MM)
55.74
22.40
9.94
5.97
4.29
3.37
2.99
2.71
47.98
24.82
13.59
7.05
5.08
3.69
3.17
2.77
iPNDM
24.54
13.92
7.76
4.04
3.22
2.83
2.56
93.36
66.76
49.66
35.62
27.93
22.32
18.43
15.69
DDIM
67.26
49.96
35.78
28.00
22.37
18.48
15.69
13.47
205.41
45.32
12.93
10.65
DPM-Solvve-22
227.32
47.22
13.68
10.89
17.18
7.04
5.56
4.14
AMED-Solver
18.49
7.59
4.36
3.67
10.16
4.67
3.18
2.65
AdaSDE (ours)
12.62
4.18
2.88
2.56
Table 8: Unconditional generation results on ImageNet 64x64.
NFE
Method
AFS
91.38
55.63
54.36
14.30
9.57
7.52
6.34
5.53
UniPC
64.54
29.59
16.17
11.03
8.51
6.98
6.04
5.26
91.52
56.34
25.49
15.06
10.14
7.84
6.48
5.67
DPM-Solver+++(MM)
65.20
30.56
16.87
11.38
8.68
7.12
6.25
5.58
58.53
33.79
18.99
12.92
9.17
7.20
5.91
5.11
iPNDM
34.81
21.31
15.53
10.27
8.64
6.60
5.64
4.97
82.96
58.43
43.81
34.03
27.46
22.59
19.27
16.72
DDIM
62.42
46.06
35.48
28.50
23.31
19.82
17.14
15.02
140.20
59.47
22.02
11.31
DPM-Solver-2
163.21
62.32
23.68
11.89
32.69
10.63
7.71
6.06
AMED-Solver
38.10
10.74
6.66
5.44
18.53
7.01
5.36
4.63
AdaSDE (ours)
18.51
6.90
5.26
4.59
31

---

(a) DDM-DDMMSoEee 2. NFE=5, FID 43.27
(b) 2 NFE=9, FID 8.65
(c) AdaSDE. NFE=5, FID 4.18
(d) AdaSDE. NFE=9, FID 2.56 Figure 5: Qualitative result on CIFARIO 32x32 (5 and NFEs)
(a) So NFE=5. FID 74.68
(b)
NFE=9, FID 16.89
(c) AdaSDE. NFB=5, FID = 8.05
(d) AdaSDE. NF FID 4.19
Figure 6: Qualitative result FFHQ 64 64 (5 and NFEs)
20M---- NFE=5. FID 59.47
DPM-Solver22 NFE=9, FID 11.31
(c) AdaSDE. NFE=5, FID 6.90
(d) AdaSDE. NFE=9, FID 4.59
Figure Qualitative result on ImageNet 64x64 (5 and 9 NFEs)
32


Reviews:
[{"review_id": "TKQBpScBQo", "reviewer": "Reviewer_wUwf", "summary": "This paper introduces AdaSDE, a novel single-step SDE solver designed to accelerate diffusion sampling while maintaining high output quality. The authors first analyze the limitations of existing ODE and SDE solvers in diffusion models, revealing that ODE solvers accumulate irreducible gradient errors due to deterministic path dependence, while SDE methods suffer from amplified discretization errors when reducing step counts. To address these issues, AdaSDE employs an adaptive noise injection strategy controlled by a learnable coefficient, which dynamically regulates the error correction strength to enhance sampling efficiency. The method integrates a process-supervised optimization approach to tune the parameters effectively. Extensive experiments demonstrate AdaSDE's superior performance, achieving state-of-the-art FID scores on datasets with fewer function evaluations compared to existing solvers.", "questions": "1.\tThe paper demonstrates strong performance on several datasets. How does AdaSDE perform on datasets with more complex distributions or higher variability, such as real-world medical imaging datasets or highly diverse datasets? Are there any specific adjustments or considerations needed to ensure robust performance across such diverse data?\n2.\tHow does AdaSDE compare in terms of overall computational time and resource usage compared to other solvers? What is the concrete computational overhead of lightweight tuning? Could the training cost offset sampling acceleration benefits?", "limitations": "yes", "rating": 4, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "strengths:\n1.\tThe paper tackles the important challenge of balancing speed and quality in diffusion sampling, a key issue in generative modeling.\n2.\tThe manuscript is well-written, making complex concepts accessible and ensuring that the innovations and methodology are clearly communicated.\n3.\tThe method is tested on multiple datasets and consistently shows superior performance, demonstrating its effectiveness and robustness.\nWeaknesses:\n1.\tThe paper could benefit from a more comprehensive review of existing methods, particularly alternative stochastic approaches and hybrid solvers.\n2.\tThe adaptive noise injection mechanism in AdaSDE is mathematically complex and lacks intuitive visualizations or explanations, potentially limiting its accessibility to practitioners.\n3.\tWhile AdaSDE performs well on tested datasets, its scalability to more complex tasks is not fully explored.", "was_revised": true, "final_justification": "I thank the authors for their detailed response, which has addressed my concerns about the paper's weaknesses and provided the requested discussion. I believe the work now meets the threshold for acceptance. Accordingly, I will keep my score, 4.", "rebuttal": "We sincerely thank you for your thorough evaluation and encouraging assessment of our work. We are particularly grateful that you highlighted our paper's clarity, the importance of the problem we address, and the robustness of our method's performance. We address your points below.\n\n> $\\textcolor{red}{\\text{Weakness 1: }}$ The paper could benefit from a more comprehensive review of existing methods, particularly alternative stochastic approaches and hybrid solvers.\n\n$\\textcolor{blue}{\\text{Response} :}$ Thank you for this valuable suggestion. Following your feedback, we have conducted an extensive new survey and have updated our related work section to include a discussion of alternative stochastic and hybrid solvers, including the methods in [1-7].\n\n> $\\textcolor{red}{\\text{Weakness 2: }}$ On the Clarity and Intuition of the Method\n\n\n$\\textcolor{blue}{\\text{Response} :}$\nThank you for this valuable feedback. We agree that making our method more accessible and intuitive for practitioners is very important. The theoretical exposition in the main text is not sufficiently streamlined for clarity. Intuitively, while SDE solvers correct for this using noise injection, our method, AdaSDE, enhances this principle by adaptively controlling the intensity of the noise at each step to improve generation quality. In our revised manuscript, we will restructure the presentation by moving some of the dense theoretical derivations to the appendix. We will use this space to add more intuitive explanations and visualizations of the adaptive noise injection mechanism to better illustrate how it works in practice.\n\n\n> $\\textcolor{red}{\\text{Questions 1 and Weakness 3:}}$  On Robustness and Broader Applicability\n\n$\\textcolor{blue}{\\text{Response} :}$ Thank you for the question. In specialized domains like medical imaging, large-scale, standardized benchmarks are often unavailable due to the significant challenges in data collection. For this reason, general-purpose datasets like MS-COCO are widely regarded as the standard benchmark for text-to-image tasks, serving as the closest and most reliable proxy for diverse, real-world generation scenarios in academia. On this benchmark, as shown in Table 2 of our paper, AdaSDE consistently outperforms existing advanced solvers when paired with Stable Diffusion v1.5.\n\nFollowing your recommendation, we conducted an experiment between our method and DPM-Solver-2 on the **Flux.1-dev** model.\n\n### **Evaluation on MSCOCO 512x512 (Flux.1-dev)**\n\n**Table 1: FID Score (Lower is Better)**\n\n| Method | 3 steps | 4 steps | 5 steps |\n| :--- | :---: | :---: | :---: |\n| DPM-Solver-2 | 54.09 | 30.17 | 26.32 |\n| **AdaSDE** | **35.32** | **26.51** | **23.54** |\n\n---\n\n**Table 2: CLIP Score (Higher is Better)**\n\n| Method | 3 steps | 4 steps | 5 steps |\n| :--- | :---: | :---: | :---: |\n| DPM-Solver-2 | 28.49 | 29.75 | 30.32 |\n| **AdaSDE** | **29.94** | **30.51** | **30.77** |\n\n---\n\n> $\\textcolor{red}{\\text{Questions 2 :}}$ On Computational Overhead and Training Efficiency\n\n$\\textcolor{blue}{\\text{Response} :}$ In short, a key advantage of our method over other light-weight approaches like AMED-Solver lies in our optimization strategy. While they often require training an auxiliary network—a process that is both slower and prone to instability—we optimize only a small set of **8-40 parameters**.\n\nThis parameter-based tuning is fundamentally more stable and significantly faster, requiring just **3-10** minutes on a single RTX 4090 for CIFAR-10. In contrast, Distillation-based methods demand massive computational resources.(e.g. 1000+ A100 hours for Consistency Models [8] and 100+ A100 hours for Consistency Trajectory Models [9] on CIFAR-10. Crucially, this tuning is a one-time cost for a reusable solver. At inference, it adds **zero computational overhead**—for the same NFE, our sampling time is identical to standard solvers, while our output quality is vastly superior. We thank you for raising these important points about the practical implementation of our algorithm. Following your suggestion, we will expand our paper to include a detailed comparison of our method's time consumption, convergence speed, and computational resource requirements against other lightweight-training approaches and training-bsaed distillation methods.\n\n---\nThank you again for your time and valuable feedback. We hope this addresses your main points and welcome any further discussion on our work or related topics in this area.\n\n[1] Dc-solver: Improving predictor-corrector diffusion sampler via dynamic compensation, Zhao et al.\n\n[2] Dpm-solver-v3: Improved diffusion ode solver with empirical model statistics, Zheng et al.\n\n[3] Efficient Integrators for Diffusion Generative Models, Pandey et al.\n\n[4] Bespoke Solvers for Generative Flow Models, Shaul et al.\n\n[5] SA-Solver: Stochastic Adams Solver for Fast Sampling of Diffusion Models, Xue et al.\n\n[6] SEEDS: Exponential SDE Solvers for Fast High-Quality Sampling from Diffusion Models. Gonzalez et al.\n\n[7] Bespoke non-stationary solvers for fast sampling of diffusion and flow models. Shaul et al.\n\n[8] Song, Y., Dhariwal, P., Chen, M. and Sutskever, I., 2023. Consistency Models. arXiv preprint arXiv:2303.01469.\n\n[9] Dongjun Kim, Chieh-Hsin Lai, Wei-Hsiang Liao, Naoki Murata, Yuhta Takida, Toshimitsu Uesaka, Yutong He, Yuki Mitsufuji, and Stefano Ermon. Consistency trajectory models: Learning probability flow ode trajectory of diffusion. arXiv preprint arXiv:2310.02279, 2023."}, {"review_id": "6JZ1aLk0Pl", "reviewer": "Reviewer_23GH", "summary": "Based on the theoretic results, this paper introduces AdaSDE, which intelligently adjusts the stochasticity in every sampling steps by learning a small set of parameters. Empirical gain is significant.", "questions": "-", "limitations": "-", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "My standard for a paper to be at least at the poster level in a top-tier conference is as follows:\n- (1) The idea must present a method that has not been attempted before in the field.\n- (2) The idea should be novel (which is pretty subjective).\n- (3) The idea must significantly outperform the baseline it targets.\n\nI have the following opinion on this specific paper:\n- (Originality & Contribution) Although the core idea seems straightforward and is likely something many students have attempted before, I believe its successful build and implementation is recommendable. The idea itself is not groundbreaking on its own, but considering its universality, I believe this kind of simple and direct approach that others can easily adopt is what will ultimately gain widespread adoption.\n- (Demonstration) Many distillation-based papers report outperforming their teacher diffusion models. However, in practice, at a service level, this report is not always true. They may show better FID scores (or even human evaluation results) on benchmark datasets, but they struggle to perform well in the wild. Thus, I believe the question of \"how to develop a diffusion-based sampler that maintains the original diffusion performance\" remains a valid and important research question.\n- In this sense, I strongly recommend experimenting on a large-scale model, such as Flux-dev (with NFEs 8, 12, 16, and 20), to compare the proposed sampler against existing distillation-based methods (Flux-Schnell with NFEs 4). Testing on this realistic setup would be a compelling way to demonstrate that the proposed method really works in real-world scenarios, beyond an experimental setup.\n- Add: I have few questions regarding the experiments:\n    - The teacher seems to be DPM-Solver 2. I am curious if this choice is the reason why AdaSDE underperforms DPM-Solver++(2M) at 8 NFE in Table 2. What would the result be if DPM-Solver++(2M) were used as the teacher?\n    - I recommend to report the performance of the teacher model, clearly labeled as such in every tables. If the student performs worse than teacher, a detailed analysis is required.\n    - SD and ImageNet has same dimensionality, but I wonder why SD requires a higher NFE to achieve good results? Does this performance difference coming from the difference on the manifold shape (that latent manifold is expected to be condensed and curvy)?", "was_revised": true, "final_justification": "I've asked several tricky and technical questions regarding the proposed method and the author properly addressed all my concerns. I recommend acceptance for this paper.", "rebuttal": "We sincerely thank the reviewer for their positive assessment and valuable, constructive feedback. We are encouraged that you see the value in our method's universality and have provided excellent suggestions to strengthen our paper. We address your points below.\n\n> $\\textcolor{red}{\\text{Suggestion 1:}}$  **On Comparison to Distillation and Large-Scale Model Evaluation**\n\n$\\textcolor{blue}{\\text{Response} :}$ \nThank you for the suggestion to compare our method against distillation on a large-scale model. We agree that this is a crucial test for real-world viability and share your view that distilled models often \"struggle to perform well in the wild.\"\n\nWe acknowledge that Flux is a widely-used and powerful generative model, and its distilled version, Flux-Schnell, serves as a popular baseline. However, we believe a direct comparison between distillation model and our lightweight method is unfair, especially when considering adaptability. When faced with a novel model architecture (e.g. unify model), distillation requires a costly redesign and massive compute resources (1000+ H100 hours), whereas our model-agnostic method is significantly faster and cheaper to adapt because it focuses on sampling dynamics rather than internal architecture. Under standard experimental settings, our method significantly outperforms the baseline AMED-Solver.\n\n* **Advantages over Distillation:** We believe the phenomenon that distilled models \"struggle to perform well in the wild\" is due to a fundamental drawback of the distillation process. By altering the model's original parameters, distillation can reduce generative diversity and impose a quality ceiling, meaning more inference steps often do not lead to better results. Furthermore, the training cost is enormous (e.g., **1000+ A100 hours** for Consistency Models [1] and **100+ A100 hours** for Consistency Trajectory Models [2] on CIFAR-10). Our method avoids these issues by preserving the original model and requiring minimal training (**3-10 minutes** on CIFAR-10). We see them as distinct pipelines, where our method offers a highly efficient and effective alternative.\n* **New Experiments on Flux.1-dev:** Following your recommendation, we have conducted new experiments on the **Flux.1-dev model**. After just **3 hours of training on a single H800 GPU**, our solver significantly outperforms the baseline second-order sampler, DPM-Solver-2, demonstrating our method's effectiveness and efficiency.\n\n### **Evaluation on MSCOCO 512x512 (Flux.1-dev Model)**\n\n**Table 1: FID Score (Lower is Better)**\n\n| Method  | 3 steps | 4 steps | 5 steps |\n| :--- | :---: | :---: | :---: |\n| DPM-Solver-2 | 54.09 | 30.17 | 26.32 |\n| **AdaSDE** | **35.32** | **26.51** | **23.54** |\n\n---\n\n**Table 2: CLIP Score (Higher is Better)**\n\n| Method  | 3 steps | 4 steps | 5 steps |\n| :--- | :---: | :---: | :---: |\n| DPM-Solver-2 | 28.49 | 29.75 | 30.32 |\n| **AdaSDE** | **29.94** | **30.51** | **30.77** |\n\n> $\\textcolor{red}{\\text{Suggestion 2:}}$ **On Specific Experimental Questions**\n\n$\\textcolor{blue}{\\text{Response} :}$ \nWe are happy to answer your specific questions regarding the experiments:\n\n1.  **Regarding Performance at 8 NFE:** Thank you for this observation.  The performance difference at 8 NFE is due to a discrepancy in the number of integration steps between our second-order method and multi-step solvers like DPM-Solver++(2M). At 8 NFE, our method completes 2 full steps, whereas DPM-Solver++ completes 4 steps. This creates a performance bottleneck for our method at this extremely low step count.\n\nHowever, we have conducted preliminary visualization tests with our plugin version, and the qualitative results show that it significantly outperforms DPM-Solver++ even at this low NFE. \n\nMoreover, our method has a much lower truncation error per step. This allows it to quickly overcome the initial bottleneck and surpass the baselines at **12+ NFE**. More importantly, as shown in Appendix Figure 6, traditional solvers can still produce poor or unstable results even at **20 NFE**, due to the accumulation of truncation error from their early steps. Traditional solvers often require **50+ NFE** on  Stable Diffusion 1.5 to achieve good generation quality.  In contrast, our method achieves high-quality results at just **12 NFE**.\n\n2.  **Regarding Teacher Model Performance:** Thank you for this recommendation. We confirm that in all our experiments, our student method (AdaSDE) **significantly outperforms its teacher (DPM-Solver-2)** when compared at the same NFE. To make this clear, we will add the teacher's results to every table in the final version of the paper.\n\n3. **Regarding NFE for SD1.5 vs. ImageNet**: To clarify this point, we must first explain our **NFE counting rule**. While it is common to count the parallel cond and uncond passes in CFG as 1 NFE, we count them separately as 2 NFE. This is the primary reason for the apparent difference: for a given NFE, the SD 1.5 model (with CFG) completes only half the number of sampling steps as the ImageNet EDM model (conditional, no CFG). \n\nMore importantly, as we analyze in our paper, the total sampling error is a combination of **sampler error** (from discretization) and **intrinsic model error** (the discrepancy between the score network $s_\\theta$ and  gradient of log probability $\\nabla_{\\mathbf{x}_t} \\log p_t(\\mathbf{x}_t | \\mathbf{x}_0)$). The pre-trained ImageNet model targets a simpler, \"toy\" distribution with low intrinsic model error. In contrast, SD 1.5 is a far more complex model for real-world scenes and thus has much higher intrinsic model error.\n\nOur method's adaptive stochasticity is particularly effective at mitigating this high intrinsic model error. This is why AdaSDE achieves strong results on SD 1.5 at only **~12 NFE**, a task where traditional solvers (e.g. DPM-Solver++) often require **50+ NFE** to produce stable, high-quality images.\n\nThank you again for your time and insightful feedback, which will be invaluable in improving our manuscript. We hope our responses have clarified our position and would welcome the opportunity for any further discussion concerning the paper or related research areas.\n\n[1] Song, Y., Dhariwal, P., Chen, M. and Sutskever, I., 2023. Consistency Models. arXiv preprint arXiv:2303.01469.\n\n[2] Dongjun Kim, Chieh-Hsin Lai, Wei-Hsiang Liao, Naoki Murata, Yuhta Takida, Toshimitsu Uesaka, Yutong He, Yuki Mitsufuji, and Stefano Ermon. Consistency trajectory models: Learning probability flow ode trajectory of diffusion. arXiv preprint arXiv:2310.02279, 2023."}, {"review_id": "qPaFdX8wQJ", "reviewer": "Reviewer_RSdV", "summary": "The authors propose AdaSDE which learns parameters per diffusion timestep to dynamically regulate noise injection during diffusion sampling. More specifically, the authors introduce a distillation based method for learning the sampler parameters which makes discretization in the low timestep regime more efficient leading to better empirical performance on standard image synthesis benchmarks.", "questions": "See weaknesses", "limitations": "The authors have not discussed the limitations of their work. I would highly recommend to discuss potential theoretical limitations and empirical performance gap with state of the art distillation based methods.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "**Strengths**:\n\n1. The method is plug-and-play and can work with existing diffusion samplers.\n2. The method enables good quality synthesis in the ultra low-timestep regime (< NFE 10 or so) as illustrated in Table 1.\n3. The distillation method for learning the solver parameters is relatively straightforward to understand. Moreover, the approach has a lot less parameters to optimize (in the scale of 8-40) so the distillation of sampler parameters is much more efficient than diffusion model distillation for few step synthesis.\n\n**Weaknesses**:\n\n1. **Presentation**: I think it has been well established in the literature that ODE based diffusion solvers have error accumulation due to discrepancies in modeling the score function accurately. While SDE solvers correct for this using noise injection, they have higher sampler discretization error in the low timestep regime. Therefore, I think a lot of theoretical fluff in the main text can be moved into the Appendix while just retaining the key points. Moreover, im not sure if the theory presented in earlier sections is relevant to understanding the core distillation algorithm presented in Section 5? I strongly encourage the authors to introduce the core methodology earlier followed by the only key theoretical results and moving the rest to the Appendix. Currently the major contributions of the paper start from Page 6 which strictly hinders the readability of the paper.\n\n2. **Discussion of some related work is missing**. Interestingly, the authors don’t cite DDIM [1] in Section 2 which is one of the seminal work in speeding up diffusion model sampling for ODE. More recent works on generic frameworks for fast sampling for diffusion/flows [2,3] and on fast SDE solvers [4,5] is also missing in Section 2. I would recommend the authors to revise the related work accordingly.\n\n3. While the empirical results are impressive, I don't think using smaller scale benchmarks like CIFAR-10 makes sense anymore. Since the distillation stage is relatively inexpensive, can the authors prioritize results from larger scale models like Flux? I see some results on StableDiffusion but its not clear how well the sampler performs for these models in ultra low NFE like 5 steps. Including results on large scale tasks would provide more clarity to the practitioner when choosing different sampling algorithms.\n\n[1] Denoising Diffusion Implicit Models, Song et al.\n\n[2] Efficient Integrators for Diffusion Generative Models, Pandey et al.\n\n[3] Bespoke Solvers for Generative Flow Models, Shaul et al.\n\n[4] SA-Solver: Stochastic Adams Solver for Fast Sampling of Diffusion Models, Xue et al.\n\n[5] SEEDS: Exponential SDE Solvers for Fast High-Quality Sampling from Diffusion Models.", "was_revised": true, "final_justification": "The authors addressed most of my concerns in their response. I continue to maintain my rating.", "rebuttal": "We are very grateful for your comprehensive review, which provided both an encouraging assessment of our method's strengths and valuable suggestions for improvement. Your valuable suggestions on improving the paper's structure, related work, and focus on large-scale experiments provide a clear path for our revision. Below, we address each of the points you raised.\n\n> $\\textcolor{red}{\\text{Weakness 1:}}$ **On Paper Structure and Presentation**\n\n$\\textcolor{blue}{\\text{Response} :}$ Thank you for the suggestion on improving the paper's readability. We agree with your assessment. In our revision, we will significantly restructure the paper by moving a large portion of the theoretical analysis from the main text to the appendix. This will allow us to introduce our core methodology much earlier and expand upon its details, which will substantially enhance the paper's clarity and flow.\n\n> $\\textcolor{red}{\\text{Weakness 2:}}$ **On Missing Related Work**\n\n$\\textcolor{blue}{\\text{Response} :}$ Thank you for pointing out these important references; we acknowledge this was an oversight in our work. We have now added the suggested papers [1-5] to our related work section. Furthermore, we have conducted a systematic survey of related methods and have comprehensively updated this section to better position our work within the current manuscript.\n\nThank you for this valuable feedback. We agree that demonstrating performance on large-scale models is crucial, and we have conducted new experiments on the Flux model to address this.\n\n\n> $\\textcolor{red}{\\text{Weakness 3:}}$ **Performance On  Large-Scale Models**\n\n$\\textcolor{blue}{\\text{Response} :}$ Thank you for this valuable feedback. We agree that demonstrating performance on large-scale models is crucial, and we have conducted new experiments on the Flux model to address this.\n\n\nBefore we discuss the performance of our sampler in the ultra-low NFE regime (e.g., 5 steps), it is important to first explain our NFE counting convention. In line with a stricter accounting practice, we count each conditional and unconditional forward pass for CFG separately. Consequently, one complete integration step of our method equates to 4 NFE. This differs from some other conventions and means that at 8 NFE, our method performs 2 full steps. To ensure a fair comparison in our experiments on the Flux model, we benchmark our method against DPM-Solver-2, which is also a second-order solver, reporting results at corresponding sampling steps (1 step = 4 NFE).\n\n### **Evaluation on MSCOCO 512x512 (Flux.1-dev Model)**\n\n**Table 1: FID Score (Lower is Better)**\n\n| Method | 3 steps | 4 steps | 5 steps |\n| :--- | :---: | :---: | :---: |\n| DPM-Solver-2 | 54.09 | 30.17 | 26.32 |\n| **AdaSDE** | **35.32** | **26.51** | **23.54** |\n\n---\n\n**Table 2: CLIP Score (Higher is Better)**\n\n| Method | 3 steps | 4 steps | 5 steps |\n| :--- | :---: | :---: | :---: |\n| DPM-Solver-2 | 28.49 | 29.75 | 30.32 |\n| **AdaSDE** | **29.94** | **30.51** | **30.77** |\n\n\nThank you once again for your time and for providing such constructive feedback. We hope our responses have clarified our position and would welcome the opportunity for any further discussion concerning the paper or related research areas.\n\n[1] Denoising Diffusion Implicit Models, Song et al.\n\n[2] Efficient Integrators for Diffusion Generative Models, Pandey et al.\n\n[3] Bespoke Solvers for Generative Flow Models, Shaul et al.\n\n[4] SA-Solver: Stochastic Adams Solver for Fast Sampling of Diffusion Models, Xue et al.\n\n[5] SEEDS: Exponential SDE Solvers for Fast High-Quality Sampling from Diffusion Models."}, {"review_id": "aEn9CkKl0h", "reviewer": "Reviewer_m3gL", "summary": "This paper proposes AdaSDE, a novel single-step SDE solver that combines the speed of ODE solvers with the error robustness of SDE methods for diffusion-based generative models. By introducing a lightweight, learnable parameter to adaptively correct errors, AdaSDE reduces sampling cost while maintaining quality. It can be easily integrated with existing solvers through minimal tuning.", "questions": "see comments", "limitations": "see comments", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\n\t1.\tThe paper is well-written and clearly structured.\n\n\t2.\tFrom my perspective, the proposed idea is novel and interesting.\n\n\t3.\tThe algorithm demonstrates strong performance compared to the provided baselines.\n\nWeaknesses:\n\n\t1.\tThe comparison with related work could be more comprehensive. It would be valuable to see results against other lightweight, learning-based solvers — for example, [1] and [2] could serve as meaningful baselines. That said, I appreciate that the authors included a comparison with AMED-Solver, which is also a lightweight, training-based approach.\n\n\t2.\tI am also interested in understanding the training efficiency of the proposed algorithm. How long does it typically take for the student model to converge? Additionally, is there any comparison with teacher-student distillation methods to contextualize the training cost and performance trade-offs?\n\n        3.  Lastly, I encourage the authors to consider evaluating their method on larger-scale datasets to better demonstrate its practical efficiency. For example, showcasing generation results using an SD3 [3] checkpoint — even without numerical metrics — would be informative, as visualizations alone could highlight the method’s strengths. Additionally, incorporating results with EDM2 [4] could further demonstrate the advantages of the proposed approach. Overall, the current benchmarks feel somewhat limited and may not fully reflect the algorithm’s potential.\n\n[1] Zhao, Wenliang, et al. \"Dc-solver: Improving predictor-corrector diffusion sampler via dynamic compensation.\" European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2024.\n\n[2] Zheng, Kaiwen, et al. \"Dpm-solver-v3: Improved diffusion ode solver with empirical model statistics.\" Advances in Neural Information Processing Systems 36 (2023): 55502-55542.\n\n[3]Esser, Patrick, et al. \"Scaling rectified flow transformers for high-resolution image synthesis.\" Forty-first international conference on machine learning. 2024.\n\n[4]Karras, Tero, et al. \"Analyzing and improving the training dynamics of diffusion models.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024.", "was_revised": true, "final_justification": "I feel all my issues are resolved. Thus I recommend for the accept.\n\nWhy not a higher score? In my view, the significance of the work is insufficient, even though it is an intersting and valuable paper.", "rebuttal": "We sincerely thank the reviewer for their positive assessment and valuable, constructive feedback. Your suggestions have provided a clear path for us to substantially improve our manuscript.\n\n>**$\\textcolor{red}{\\text{Weakness1:}}$ On Comparison with Other Solvers** \n\n$\\textcolor{blue}{\\text{Response} :}$ Thank you for the valuable suggestion to include more baselines. In response to your feedback, we have added comparisons against DC-Solver [1] and DPM-Solver-v3 [2], based on the results reported in their respective papers. The preliminary results below show that our method remains highly competitive.\n\nWe wish to clarify that a direct comparison on the LSUN-Bedroom dataset was not feasible, as the base models for DC-Solver [1] and DPM-Solver-v3 [2] in their original publications differ from the one used in our study. To address this, we will incorporate comparisons with these solvers on relevant datasets in a future version of our work. Under standard experimental settings for the LSUN-Bedroom dataset, our method significantly outperforms current solvers. For example, at just 3 NFE, AdaSDE achieves a 69.1% improvement in FID score over the state-of-the-art AMED-Solver.\n\n **FID result of FFHQ 64 × 64**\n| NFE      | 5    | 6    | 7    | 8    | 9    | 10   |\n| -------- | ---- | ---- | ---- | ---- | ---- | ---- |\n| DC-solver | 10.38 | 8.39 | 7.66 | 7.14 | 6.92 | 6.82 |\n| **AdaSDE**   | **8.91** | **7.82** | **6.06** | **5.83** | **4.83** | **4.61** |\n\n**FID result of CIFAR10 32 × 32**\n| NFE              | 5     | 6     | 8    | 10   |\n| ---------------- | ----- | ----- | ---- | ---- |\n| DPM-solver-v3(S) | 21.83 | 16.81 | 7.93 | 5.76 |\n| DPM-solver-v3(M) | 12.76 |  7.40  | 3.94 | 3.40 |\n| **AdaSDE**   | **4.79**  | **4.67** | **3.18** | **2.56** |\n\n**FID result of MSCOCO 512 × 512 on Stable Diffusion v1.5**\n| NFE  (CFG)   | 12    | 16    | 20    |\n| ------------- | ----- | ----- | ----- |\n| DPM-Solver-v3 | 16.41 | 15.41 | 15.32 |\n| **AdaSDE**   | **13.99** | **13.39** | **12.68** |\n\n> **$\\textcolor{red}{\\text{Weakness2:}}$ On Training Efficiency and Comparison to Distillation**\n\n$\\textcolor{blue}{\\text{Response} :}$Thank you for these important questions. We will add a detailed analysis to the appendix covering convergence times and a cost-performance comparison against traditional distillation.\n\nIn short, our training time is significantly shorter than existing lightweight-training methods because we optimize only a few parameters instead of an auxiliary network. On CIFAR-10, our method converges in just **3-10 minutes** with high stability.\n\nThis efficiency stands in stark contrast to traditional distillation methods, which require massive computational resources (e.g., **1000+ A100 hours** for Consistency Models [3] and **100+ A100 hours** for Consistency Trajectory Models [4] on CIFAR-10). Furthermore, those methods often suffer from training instability and can degrade the base model's performance due to altered parameters, leading to a limited quality ceiling where more inference steps do not yield better results.\n\n>**$\\textcolor{red}{\\text{Weakness3:}}$  On Evaluation with Larger-Scale Models**\n\n$\\textcolor{blue}{\\text{Response} :}$Thank you for encouraging us to evaluate on larger-scale models; we agree this is crucial for demonstrating practical impact.\n\nWe have conducted new experiments on the **Flux.1-dev**, comparing our method against DPM-Solver-2. After just 4 hours of training on a single H800 GPU, our solver significantly outperforms the baseline second-order sampler. We will include qualitative visualizations from these experiments in the revised paper and also add results validating our plugin approach on the FLUX model.\n\n\n### **Evaluation on MSCOCO 512x512 (Flux.1-dev Model)**\n\n**Table 1: FID Score (Lower is Better)**\n\n| Method | 3 steps | 4 steps | 5 steps |\n| :--- | :---: | :---: | :---: |\n| DPM-Solver-2 | 54.09 | 30.17 | 26.32 |\n| **AdaSDE** | **35.32** | **26.51** | **23.54** |\n\n---\n\n**Table 2: CLIP Score (Higher is Better)**\n\n| Method | 3 steps | 4 steps | 5 steps |\n| :--- | :---: | :---: | :---: |\n| DPM-Solver-2 | 28.49 | 29.75 | 30.32 |\n| **AdaSDE** | **29.94** | **30.51** | **30.77** |\n\nThank you once again for your time and thoughtful feedback. We are very encouraged by your positive assessment of our work's potential and believe your suggestions will help us substantially strengthen our manuscript.\n\n[1] Zhao, Wenliang, et al. \"Dc-solver: Improving predictor-corrector diffusion sampler via dynamic compensation.\" European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2024.\n\n[2] Zheng, Kaiwen, et al. \"Dpm-solver-v3: Improved diffusion ode solver with empirical model statistics.\" Advances in Neural Information Processing Systems 36 (2023): 55502-55542.\n\n[3] Song, Y., Dhariwal, P., Chen, M. and Sutskever, I., 2023. Consistency Models. arXiv preprint arXiv:2303.01469.\n\n[4] Dongjun Kim, Chieh-Hsin Lai, Wei-Hsiang Liao, Naoki Murata, Yuhta Takida, Toshimitsu  Uesaka, Yutong He, Yuki Mitsufuji, and Stefano Ermon. Consistency trajectory models:  Learning probability flow ode trajectory of diffusion. arXiv preprint arXiv:2310.02279, 2023."}]

Author rebuttal:

Meta-review: Tl;dr: Based on the reviews, rebuttal and ensuing discussion I recommend accept.

### Paper Summary

The paper introduces AdaSDE, a novel single-step SDE solver for fast sampling in diffusion-based generative models. The key thesis is that an optimal sampling trajectory can be achieved by balancing the trade-off between two fundamental error sources: the “gradient error” (from inaccurate score estimates) and the “sampler error” (from discretization). Paper proposes a method with a learnable and adaptive coefficient that dynamically modulates the magnitude of stochastic noise injected at each sampling step, allowing the solver to correct for inaccuracies in the learned score function while mitigating the large discretization errors.

### Key strengths and weaknesses

Strengths
- Theory: The decomposition of error into orthogonal gradient and sampler components is elegant and powerful and enables principled justification of the proposed method.
- Novelty and efficiency: The core contribution of adaptive stochasticity is novel and is relatively cheap to incorporate.
- Empirical validation: Initial and new results (during discussion phase) are convincing, demonstrating SOTA performance on several benchmarks.

Weaknesses

- Readability: The paper structure made it difficult to understand the core contribution with the dense theoretical exposition. The authors have committed to restructuring the paper to improve its flow and clarity.
- Insufficient discussion of relevant work: Reviewers pointed several relevant works that would be useful to include in the discussion.
- Experimental scope: The initial experiments were smaller scale. Though rebuttal addressed this concern.

### Decision justification

Accept recommendation is made primarily because all the major weaknesses of the initial submission were effectively addressed during the author-reviewer discussion period. All the reviewers were positive leaning.

In the initial review, reviewers were largely in agreement, liking the core idea but raising concerns about the manuscript’s readability, the completeness of the related work section, and the lack of larger scale experiments. The authors addressed most of the concerns in their rebuttal. New results on the larger Flux.1-dev model were also provided.
Decision: Accept (poster)