Tortoise and Hare Guidance: Accelerating Diffusion Model Inference with Multirate Integration
Authors: Yunghee Lee
Byeonghyun Pak
Junwha Hong
Hoseong Kim
Venue: NeurIPS 2025 poster ()
Keywords: diffusion model solver
diffusion model inference
multirate method
classfier-free guidance
Primary area: deep_learning
TL;DR: Tortoise and Hare Guidance accelerates diffusion sampling by integrating the guidance term on a coarser timestep grid.

Abstract: In this paper, we propose Tortoise and Hare Guidance (THG), a training-free strategy that accelerates diffusion sampling while maintaining high-fidelity generation. We demonstrate that the noise estimate and the additional guidance term exhibit markedly different sensitivity to numerical error by reformulating the classifier-free guidance (CFG) ODE as a multirate system of ODEs. Our error-bound analysis shows that the additional guidance branch is more robust to approximation, revealing substantial redundancy that conventional solvers fail to exploit. 
Building on this insight, THG significantly reduces the computation of the additional guidance: the noise estimate is integrated with the tortoise equation on the original, fine-grained timestep grid, while the additional guidance is integrated with the hare equation only on a coarse grid. We also introduce (i) an error-bound-aware timestep sampler that adaptively selects step sizes and (ii) a guidance-scale scheduler that stabilizes large extrapolation spans. 
THG reduces the number of function evaluations (NFE) by up to 30% with virtually no loss in generation fidelity (∆ImageReward ≤ 0.032) and outperforms state-of-the-art CFG-based training-free accelerators under identical computation budgets. Our findings highlight the potential of multirate formulations for diffusion solvers, paving the way for real-time high-quality image synthesis without any model retraining. The source code is available at https://github.com/yhlee-add/THG.

Tortoise and Hare Guidance: Accelerating Diffusion Model Inference with Multirate Integration
Yunghee Lee
Byeonghyun Pak
Junwha Hong
Hoseong Kim
Agency for Defense Development
[yhl, byeonghyun_pak, qbit, hoseongkim)@add, kr
Abstract
In this paper, we propose Tortoise and Hare Guidance (THG), a training-free strategy that accelerates diffusion sampling while maintaining high-fidelity gen- eration. We demonstrate that the noise estimate and the additional guidance term exhibit markedly different sensitivity numerical error by reformulating the classifier-free guidance (CFG) ODE as a multirate system of ODEs. Our error-bound analysis shows that the additional guidance branch is more robust approximation, revealing substantial redundancy that conventional solvers fail exploit. Building on this insight, THG significantly reduces the computation of the additional guidance: the noise estimate is integrated with the tortoise equation on the original, fine-graine timestep grid, while the additional guidance is integrated with the hare equation only on a coarse grid. We also introduce (i) an error-bound- aware timestep sampler that adaptively selects step sizes and (ii) guidance-scale scheduler that stabilizes large extrapolation spans. THG reduces the number function evaluations (NFE) by up 30% with virtually no loss in generation fidelity (AlmageRwward < 0.032) and outperforms state-of-the-art CFG-based training-free accelerators under identical computation budgets. Our findings high- light the potential of multirate formulations for diffusion solvers, paving the way for real-time high-qualit image synthesis without any model retraining. The source code is available at lttps:: //gpttbttt/tt///////// com hta/yyllee- wanlee-add/aae add/THG
Introduction
Diffusion models (DMs) have become the state-of-the-art generative model for images 10 and, more recently, for video (20 quality, sampling costly: each output is obtained by iteratively denoising noisy sample, and the latency scales with the total number of function evaluations (NFE) required by the solver.
[ 52 21 and audio-via visual content 41. Despite their impressive
Many practical scenarios, such text-to-image synthesis, class-controlled synthesis, in-context image editing, require conditional generation. The dominant technique for high-quality conditioning is classifier-free guidance (CFG) (18), which improves perceptual quality and controllability. However, CFG runs the denoising network twice per timestep-once conditional and once unconditional- thereby doubling the NFE. For real-time applications, such as interactive editing and large-scale serving, evaluating deep backbone at every timestep remains major bottleneck.
A large body of work accelerate these models has focused two main approaches. Some approaches reduce the number of steps using higher-order ODE/SDE solvers 45]
46 30 or distilla-
tion 43 34 while others-such cache-based strategies like DeepCache K3 and Learning-to-
Cache 32
lower the cost per step by reusing intermediate features. Nevertheless, both approaches still perform forward passes whenever CFG is enabled, implicitly assuming that conditional and unconditional calls are equally indispensable.
Cooresponding author.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

INFE
1NFE
NPE
2 NFE
2 NFE
2 NFEE
X1.
30
ATL12
Xiçez
X1e43
NFE
(a) Standard diffusion ODE (CFG)
(b) Multirate system of ODEs (Ours)
Tortoise ODE
CFG
&
E
Numerical Error Fine-grained
1.43 x
Hare ODE
Ours
6c,
Numerical Error
Coarse
50
75
100
(c) Total NFE on Stable Diffusion 1.5
Figure 1: Conceptual illustration of Tortoise and Hare Guidance. We decompose the standard diffusion ODE into tortoise branch (Eq. 6 which is numerically sensitive and thus integrated on a fine-grained grid, and a hare branch (Eq, 7 which is comparatively less sensitive and can be integrated with larger step sizes, Our multirate scheme evaluates each branch at different timestep grids, skipping unnecessary evaluations, thereby boosting inference efficiency without sacrificing sample quality.
Through the lens of numerical analysis, revisit CFG by reformulating the reverse diffusion process as a two-state multirate system of ODEs whose trajectories are governed by the noise estimate and the additional guidance term. Our error-bound analysis reveals a pronounced asymmetry: the additional guidance term is more robust to approximation than the noise estimate, exposing substantial redundancy that conventional solvers fail to exploit. This finding raises natural question: Do we need to compute the neural network twice at every fine- grained timestep?
Leveraging this asymmetry, introduce Tortoise and Hare Guidance (THG), training-free free sampler that bypasses most additional guidance computation. The noise estimate is integrated with the tortoise equation the original fine-grained timestep grid. Meanwhile, the additional guidance is integrated with the hare equation only on coarse grid. We further introduce (i) an error-bound-awe timestep sampler that adaptively determines the coarse grid, and (ii) guidance-scale scheduler that keeps the trajectory stable over significant gaps.
With these components, THG achieves sampling speeds up 1.43x faster by reducing the NFE budget from 100 to as low as 70 while maintaining virtually identical generation fidelity (ewaageReward < 0.032). Moreover, across Stable Diffusion 1.5 [40]. 3.5 Large [47]. and Audi- oLDM 2 (28), our method outperforms sfate-of-the-ar art Casebased training-free free accelerators under identical computation budgets. Our study highlights the potential of multirate fommulations for accelerating diffusion models and brings us step closer to achieving real-time performance and high-quality image synthesis without retraining the model.
In summary, our contributions are threefold:
We are the first to cast the reverse diffusion ODE two-state multirate system of ODEs and to provide an error-bound analysis showing that the additional guidance term can be safely approximated at a much coarser temporal resolution.
We design Tortoise and Hare Guidance (THG), a training-free sampler that eliminates the need for significant amount of additional guidance term evaluation. THG is compatible with any diffusion backbone.
Using image-text pairs from the COCO 2014 4 dataset, we demonstrate that THG can reduce NFEs up to 30% with virtually no loss in generation fidelity (ATmageeeward S 0.032). THG outperforms state-of-the-art CFG-based accelerators under identical compute budgets.

---

2 Related work
Diffussin models Denoising Diffusion Probabilistic Models (DDPMs) [9] laid the foundation for modern diffusion models by introducing a probabilistic framework. A forward Markov process gradually corrupts data point zo into Gaussian noise. In the reverse process, each timestep t, neural network & (Zr,. t) estimates and removes the noise component in 2; to recover 11 ultimately reconstructing xn- The denoising trajectory can be interpreted either as a stochastic differential equation (SDE) or its deterministic counterpart, the probability flow ODE (PF-ODE) [46]. Denoising
Diffusion Implicit Models (DDIMs) [5 drop the strict Markov assumption of DDPMs and apply
Tweedic's ' formula [9] to jump directly from I' to Ee. cutting sampling steps from hundreds of steps to as few as 50 and effectively solving the PF-ODE a single deterministic pass 46
ODE-based integrators Viewing diffusion sampling as an initial-value ODE problem enables high-order integration techniques. Concretely, DPM-solver [30] observes that the diffusion ODE
(1) has semi-linear term f()))) The need for approximation for the linear term is eliminated by solving the semi-linear ODE using the variation of constants formula. This semi-linear integrator then affords large step sizes with minimal approximation error. Inspired by these semi-linear methods, we introduce multirate formulation for the classifier-free guidance (CFG) scheme (18) that adjusts the step size of each component of CFG to its own dynamics, achieving further reductions in the number of function evaluations (NFE) without degrading sample quality.
dxa/dt = f(1)== + (t)
Classifier-free guidance and its variations In real-world applications, diffusion models must produce samples that satisfy a given condition (e.g., class label or text prompt). Classifier Guidance [8] achieves this by incorporating pre-trained classifier (c(r)) effectively sampling from the sharpened density p(z) p(=)p(()z)) where w controls the strength of the bias towards class c. Classifier- Free Guidance (CFG) 18] eliminates the need for an external classifier by training single denoising network that gives both conditional and unconditional outputs. Concretely, if &o (0 c) and & (z,, Ø) denote the network's 's noise predictions with and without condition c, respectively, then CFG defines
Co C c) = čo (24, Ø) + w. (Za e) ép (z+, 0)).
(2)
Subsequent variants focus on finding the optimal strength and timing of guidance for balancing condition fidelity against sample diversity. Guidance Interval (26] restricts the of CFG to mid- level noise steps, avoiding over-conditioning at the beginning and final stages of the sampling process.
CADS and Dynamic-CFG [42] slowly anneal either the conditioning vector or the scale w during the early denoising steps, preserving diversity in the final samples. PCG [2] reformulates CFG as predictor-corrector method (with w' = 2 1) that alternates between denoising and sharpening phases. CFG++ 77) treats guidance as an explicit loss term rather than a sampling bias, splitting each DDIM iteration into "denoising" and "renoising" phases. Unlike these methods, we reformulate the diffusion ODE using multirate method, integrating the noise estimate on fine- fine-grained grid and the additional guidance term on a coarse grid, reducing the NFE while preserving sample quality.
Efficient diffusion models Beyond advanced ODE/SDE solvers, various methods have been proposed to speed up pre-trained diffusion models. Distillation methods 4 34] compress a pre- trained "teacher" model into a "student" model that can advance multiple timesteps in one forward pass. While these methods reduce the number of sampling steps, they incur substantial retraining costs. Cache-based techniques exploit feature redundancy within the denoising neural network ég. DeepCache 3 reuses high-level U-Net activations across adjacent steps. Learning-to-Cadee Cache (32) introduces a layer-wise caching mechanism that dynamically reuses transformer activations across timesteps via timestep-condttioned router. A-Dit [4] leverages stage- adaeeadee caching of block- specific feature offsets in DiT models to speed up inference without retraining. These methods deliver inference speedups without retraining but depend heavily on the model's internal architecture. More recently, several works have noted that CFG doubles the NFE per denoising step and have proposed methods to reduce this extra cost. Adaptive Guidance B adaptively skips redundant guidance steps based on cosine similarity between conditional and unconditional predictions. FasterCache [I reuses attention features and conditionalluncondiiie these methods reduce the NFE, they lack rigorous theoretical foundation and leave further savings on the table. Our approach delivers more efficient and theoretically grounded method of guided diffusion by directly exploiting the CFG's intrinsic dynamics.
residuals to mitigate CFG overhead. Although
3

---

We plot the L2 norms of the time derivatives di (x(e)) and di ( across diffusion timesteps for Stable Diffusion 1.5 and 3.5 Large. The results confirm that the noise estimate exhibits greater Stable Diffusion 1.5
Stable Diffusion 3.5 Large
10a
10a
Derivative Norm
Derivative Norm
10a
10a
10-1
10"
400
800
0.0
0.4
Timestep
Timestep
Figure 2: Tire-derivative norms of the noise estimate (x,,,) and additional guidance (a:)..)
temporal sensitivity compared to the guidance term. Shaded areas denote two standard deviations over multiple prompts.
Method
In this section, we introduce Tortoise and Hare Guidance (THG), which accelerates diffusion model inference by leveraging the asymmetry between the noise estimate and the additional guidance terms. Since the additional guidance term varies more slowly w.r.f. the denoising timestep f. than the noise estimate term, we apply a multirate integration scheme that uses a coarser timestep grid for the additional guidance term (Sec. 3.1 and Sec. 3.2). We then perform an approximation error-bound analysis to determine the appropriate grid granularity (Sec. 3.3 Finally, we propose an adaptive guidance scale to compensate for any performance degradation resulting from the reduced number of evaluation points (Sec. 3.4
Preliminaries To accommodate different definitions of the diffusion process 19 46 49], we adopt
general notation (30] so that the forward process and the diffusion ODE are described as follows:
dæ dt f(t)2, + 20,
g2(1) co(x)) zr ~ -
((ze|=a) == NN of
(3)
d oxe (1) dof 2d of and E [0. T] (v-prediction models are covered in Appendix A los
where /(1)
dr
ar and dr are the predefined noise schedule of the diffusion model. Although modern diffusion models primarily operate the latent space [40], we adopt x (instead of z), as our framework is agnostic to this choice. For brevity, denote the unconditional noise estimate &o (Ix) == (It, 2). the conditional noise estimate (Z( = Ee( the difference of the two ( (xx,) é (zz), and the CFG noise estimate è: (xx) Cg (2 c) following [7].
3.1 A multirate formulation
We propose a multirate formulation 339, in which the reverse diffusion process is decomposed into numerically sensitive and less sensitive components to reduce the number of function evaluations (NFE). We begin by writing the diffusion ODE in Eq. By explicitly separating it into two distinct terms, the noise estimate and the additional guidance term. By the definition of CFG, we have
(Φ(x²) = 25d (ax) εœ( + w- (dc( (x)) + (w (xc(
(4)
Substituting Eq. into Eq. yields the following:
di tt f(t)xt + 20, (x)) f(t)x + (1)
(2(()
(2())
& (x))
(w (1))
(5)
20,
20,
sensitive
less sensštive
We observe significant difference in temporal sensitivity between the noise estimate term and the additional guidance term. Figure 2 plots the time-derivative norms of (xx))) and (x.) confirming that the noise estimate varies more rapidly than the additional guidance term. This result

---

Algorithm Tortoise and Hare Guidance Algorithm
Require: 2T 07()
Initial noise
Require: wi 2.20 0
b Guidance scale
Require: (fiosiiNN
T, Require: C (4/0 N),0EC,TEC
Fine-grained timestep grid D Coarse timestep grid
1: oT t FT
2: 2r f-
3: for
i = to N do
4:
é f é0(2) + x. c)
P 1 NFE D. Compute x given #7.
5:
Them - Solver( co, 4, 444)
6:
E then
7:
êz C + + 0)
1 NFE (only if E C)
8:
ACc + 60 E0 jti
9:
10:
repeat
> Compute æ up to the next coarse timestep H > Compute 2i given æt, equals the next coarse timestep at inner loop exit
j +1
12:
H 21, t Solver( (w - 1) Ddc; tz,
13:
until tj E C
end if end for
15:
Zo
< mb + x
17:
return Zo
clearly demonstrates that the noise estimate exhibits greater numerical sensitivity than the additional guidance.
This motivates the use of a multirate method 44 where the sensitive term is integrated on a fine- grained grid, and the less sensitive term is integrated on a coarse grid. We split the diffusion ODE (Eq, 5) into the following system of ODEs:
dt x 1(1)= + g2(1) _ + a (x
d
(6)
20,
d
f(t)= + g²) (w- 1)--)) +
dt
20,
where 27. Dr, 27 = 0, and x, := ... + at The tortoise ... covers the noise estimate part of the diffusion ODE, while the hare x takes care of the additional guidance term. We call the ODE integrated the ane-ggained grid the tortoise equation (Eq. and the ODE integrated the coarse grid the hare equation (Eq.7). Intuitively, the hare equation uses coarser timestep intervals-i.e. larger steps-aae alloowng it to skip unnecessary computation and thus significantly improve the efficiency of integrating the diffusion ODE. Moreover, because both equations retain the standard diffusion ODE form, existing solvers such as DDIM 45 can be applied each equation without modification.
3.2 Tortoise and Hare Guidance
Solving the hare equation (Eq. 7 on the coarse grid is straightforward, since every coarse timestep is also fine-grained timestep. By contrast, because the tortoise equation (Eq. requires the full state J, = x! + a at every fine-grained timestep, we must infer = at those intermediate points [39]. Instead of using generic extrapolation methods [31], we exploit property of diffusion model solvers: given H, and (x(2)) they can deterministically compute Ie for any < t by running the chosen solver from / s. From each coarse timestep, we run the solver not only to compute x!f for the next coarse timestep but also to populate zt for all intermediate fine-grained timesteps, thereby constructing the full trajectory of 2 the fine-grained grid for use in integrating the tortoise equation.
Building on this formulation, we propose an implementation strategy summarized in Algorithm [ While the standard diffusion solver evaluates both (xxx) and (x, at every fine-grained timestep, our scheme evaluates (a,) only on the coarse grid C C (lo..... (NN). thereby significantly reducing NFE. At each coarse step t E C, the updated guidance term is used to integrate the hare
5

---

Stable Diftusion 1.5
Stable Diffusion 3.5 Large
1.00
8
0.75
100
1000
(1)
0.25
0.00
600
0.2
Timestep
Timestep
Figure 3: Approximation error bounds of the tortoise o. and the hare a. We show the per- timestep error bound of the tortoise and the hare terms across sampling steps. The consistently higher bounds for the tortoise curve indicate that the noise estimate more sensitive to timestep resolution than the additional guidance. Shaded areas denote two standard deviations over multiple prompts.
equation across the fine- fine-grained grid until the next coarse step. then use the resulting a values during the subsequent tortoise equation steps. As result, the NFE is reduced from 2N to N -1 while preserving the dynamics of the original diffusion ODE, Moreover, it slots seamlessly into existing diffusion pipelines without any changes to their core logic.
3.3 Approximation error bound analysis
To determine an appropriate coarse grid C' for the hare equation, we now turn to an error-based criterion. Our objective is to ensure that the integration error of æ remains sufficiently small relative to that of x. To this end, we adopt standard multirate strategy [11. We select coarse step sizes such that the ratio between the hare's approximation error and the tortoise's approximation error does
not exceed a user-specified threshold such that R 1:
P.
(8)
Here, _r_ and w denote the analytical solutions to the tortoise and hare equations at timestep while A and aH are the corresponding numerical solutions obtained using the diffusion model solver. Given that the solver has order p, the local integration error single step scales as [14]
a. Ig = c. (At) + ((At))+))
where At is the fine-gres fine-grained step size and is unknown constant. Let the coarse step size be mAt, meaning the hare leaps m tortoise steps per update. Then, the local integration error of the hare equation over one coarse step becomes:
- = CH. (mAt)=+1 + +0 ((A))*+2).
(10)
In contrast, the tortoise equation accumulates error over m fine-grained steps:
-. CT m(At)++1 O ((A+))+2)
(11)
Taking the ratio from Eq. and ignoring higher-order terms, we obtain:
||A))tt m²
||cH|| ||TT|| S... :.m S (p||<T||/||>1)) 1/2
(12)
| -|||
||e [|m()))+
Since m must be positive integer, we define the maximum allowable value as:
Myxx max (I. (elleT II/l///
(13)
Estimating the error constants To compute m-axx we need estimates of ||cT]] and ||cH|| without relying on the analytic solution Z.. We accomplish this using the Richardson extrapolation method 14 First, solve the ODE once using step size At:
(1) I. (At) + +0 ((A6)pt)).
(14)

---

Algorithm 2 Look before you leap
Require: (lmx((1)
Calculated Mmmx for each timestep Fine-graines timestep grid > The result is initially empty set > Start advancing the fine-grained grid from the first timestep
Require: (4JosisNN =7,6N =
1: C
2: i+0
3: while < do
4:
C CUftt
> Add current position Advance mmax (t:) steps
5: 1 Minax (t:)
6: end while
7: C CU {0}
> Include last timestep
8:
return C
Next, solve again using two steps size At/2:
g J. = c. 2(241122)+))) (((A))))
((Ar)+t2).
(15)
Subtracting Eq. 14 and Eq.
15 yields
=!? c (1-21 (Al) + ((AI)-))
(16)
we ignore the higher-order terms, the norm of this difference provides a direct estimate proportional
||c|]. We apply this procedure independently to both the tortoise and hare equations estimate
||c
and ||c [|. respectively. Empirical results (Fig. 3) 30,000 prompts from the COCO 2014
dataset [27 37 show that ||c is greater than ||ctt for most cases, confirming that the tortoise
equation more sensitive to timestep resolution.
Example usage scenario To generate samples using Tortoise and Hare Guidance, we first calculate Mmax of each fine- fine-grained timestep (Eq.
13) using the average ||cT|| and ||c*]] over batch of inputs. Then we build the coarse timestep grid C' via the "look before you leap" strategy (Algorithm 2 Starting at the first fine-grained timestep la. we insert coarse timesteps so that they lie Mmax (44) steps ahead, keeping the local error ratio below Finally, we generate samples using Algorithm
(More details are covered in Appendix
!
Note that C' could be reused for all subsequent inferences
without any additional NFEs.
3.4 Adjusting Guidance Scales
Approximating the hare at fine-grate fineggrined timesteps can lead to a degradation in output quality. To compensate for this, we propose adjusting the guidance scale whenever the additional guidance term used more than once per timestep. In particular, we introduce a constant boost factor b and scale the guidance term: A b. A This simple multiplicative adjustment improves sample quality, especially in cases where the inner loop (which integrates the hare equation) is repeated multiple times for each coarse step. Our method draws inspiration from prior work such amplifies guidance in the frequency domain using FFT. However, unlike FFT-based methods, our approach avoids the overhead of spectral transforms, which can be computationally expensive for high- dimensional latent variables. The additional guidance term predominantly contains low-frequency information in the early stages of sampling and vice versa [15. Therefore, selectively enhancing the frequency components of the additional guidance term per timestep has low significance.
[31], which
Furthermore, CFG and the additional guidance term are of low significance the later phase of the reverse diffusion process 226
3 We leverage this fact by introducing threshold timestep index in and substituting Ade <- 0 if ini This simple adjustment helps reduce the NFE even further.
4 Experiments
4.1 Experimental Settings
Compared methods To demonstrate the effectiveness of our approach, we compare against CFG- Cache 311, training-free acceleration technique that reuses conditional and unconditional outputs in video diffusion models. Given that CFG-Cache exploits timestep-adapttie enhancement technique
7

---

Table Comparison of methods in terms of distributional similarity and prompt fidelity. Our method
marked in blue whereas vanilla CFG is marked in gray The best results are highlighted.
Distributional similarity
Prompt fidelity
Method
NFE
FID +
CMMD
CS
IR f
Stable Diffusion 1.5 with DDIM
CFG [8]
100
14.057
0.58885
26.294
0.14765
CFG-Cache w/o FFT [1
70
14.240
0.59187
26.141
0.08757
CFG-Cache 31
70
14.367
0.59556
26.180
0.09735
THG (Ours)
70
14.165
0.59223
26.189
0.11499
Stable Diffusion 1.5 with DPM-Solve-22
30
CFG 18
100
13.255
0.60379
26.254
0,16148
CFG-Cach w/o FFT 31
70
13.387
0.60665
26.107
0,10513
CFG-Cache [3]
70
13.468
0.60880
26.147
0.11474
THG (Ours)
70
12.909
0.60868
26.205
0.14926
Stable Diffusion 1.5 with 2nd-order Linear Multistep Method [29]
CFG 18]
100
13.540
0.60653
26.260
0.15966
CFG-Cache w/o FFT 31
70
13.686
0.60844
26.107
0.09881
CFG-Cache [31
70
13.798
0.61142
26.144 144
0,10805
THG (Ours)
70
13.686
0.61094
26.204
0.15184
Stable Diffusion 3.5 Large with Euler method
CFG [18]
56
68.158
0.81106
26.624
1.03569
CFG-Cache w/o FFT [3
38
67.931
0.76448
26.643
1,00715
CFG-Cache BI
38
67.914
0.75324
26.668
1.00745
THG (Ours)
38
68.252
0.80092
26.672
1.02365
Distributional similarity FAD
Prompt fidelity CLAP Score t
Method
NFE
AudioLDM with DDIM
CFG [18
100
2.596
0.2409
CFGCCache w/o FFT 31
70
2.901
0,2251
THG (Ours)
70
2.764
0.2342
to mitigate fine-detail degradation, we evaluate both the full CFG-Cache (with enhancement) and a variant without this enhancement (denoted "CFG-Cache w/o FFT"). All variants adapted to the diffusion model's 's modality for a fair comparison. More comparisons with CFG variants are given in Appendix B
Implementation details We build Tortoise and Hare Guidance with PyTTrch [36], Diffusers 48 and Accelerate [13.. We evaluate three pretrained diffusion models-Sttan Diffusion 1.5 [40. Stable Diffusion 3.5 Large [47 10], and AudioL.DM 2 28) For Stable Diffusion (SD) models, we use prompt-image pairs randomly sampled from COCO 2014 4 27 37 30,000 pairs for SD 1.5 and 1,000 pairs for SD 3.5 Large. For AudioL.DM 2, we use 2,230 prompt-audio pairs from the validation set of AudioCaps [25]. Experiments are run on server with an AMD EPYC 74F3 26934-core CPU, 1 TB of RAM, and NVIDIA 100 80GB GPUs. Hyperparameters (N, W, p. b, dni) are set to (50, 7.5, 1.1, 1.1, 38) for SD 1.5, (28, 3.5, 1.0, 1.2, 21) for SD 3.5 Large, and (50, 3.5, 0.9, 1.15, 39) for AudioLDM 2.
4.2 Main Results
variants in terms of distributional similarity metrics such as FID 17 35 CMMD (2), and FAD 24), together with
Quantitative comparison
our method to the
prompt fidelity metrics such as CLIP Score (CS) 16). ImageReward (IR) [51 and CLAP Score 50 under the same number of function evaluations (NFE). Refer to Appendix for more details on metrics. On SD 1.5, all methods cut NFE from 100 to 70; ours lowers FID ( (14, 165 vs. 14.240).

---

SD 1.5 with DDM,, Prompt: A group of zebras grazing in the grass.
NFE 100
NFE 70
NFE 70
NFE 70
SD 1.5 with DDIM, Prompt: Two hill above valley and mountains the other side.
NFE 100
NFE 70
NFE 70
NFE 70
3.5 Large with Euler method, Prompt: A single giraffe standing in the middle of tall grass
NFE 56
NFE 38
NFE 38
NFE 38
SD Large with Euler method, Prompt: that sign reads "Crosstown", a metro bus.
NFE 56
NFE
NFE
NFE 38
CFG
CFG-Cache w/o FFT
(c) CFG-Cache
THG (Ours)
Figure 4: Comparison of visual results for prompts from the COCO 2014 dataset.
matches CMMD, and improves CS and IR over CFG-Cache w/o FFT, and beats full CFG-Cache CS and IR while keeping FID competitive. On SD 3.5 Large, all cut NFE from 56 to 38; although CFG-Cache slightly leads FID and CMMD, method delivers nearly equal FID/CMMD with the highest IR and tied CS. On AudioLDM 28), all cut NFE from 100 to 70; ours lowers FAD (2.764 vs. 2.901) and improves CLAP Score CFG-Cache w/o FFT. CFG-Cache is excluded since its enhancement is inapplicable to the audio domain. These results show that THG generalizes across solvers and scales, preserving sample distribution and text alignment under aggressive step reduction. The tradeoff of distributional similarity and prompt fidelity is further discussed in Appendix G
Qualitative comparison Figure compares images generated by our method and the two CFG- Cache variants. The results demonstrate that THG effectively preserves image fidelity and fine details. More visual comparisons shown in Appendix I

---

Table Ablation study for the hyperparameter Table 3: Ablation study for the hyperparameter p.
Method
NFE 4 FID + CMMD 4 CS f
IR 1
Method NFE 4
FID + CMMD + CS
IR 1
6 = 1.00
70
13,811
0,58364
26.137 0,09395
0.9
75
14.128
0.59044
26.193 0.1 0.11942
6 1.05
70
13.988
0.58794
26.162 0.10456
P 1.0
73
14.148
0.59068
26.200 0.11949
b 1.10
70
14.232
0.59354
26.197 0.11576
p 1.1
70
14.232
0.59354
26.197 0.11576
1.15
70
14.472
0.59783
26.221 0.12639
P 1.2
69
14.336
0.59306
26.221
0.1
b 1.20
70
14.729
0.60260
26.246
0.13478
1.3
67
14,280
0,59521
26.197
0.10849
4.3
Ablation Studies
Boost factor We perform ablation studies hyperparameters using SD 1.5 with DDIM. Table shows how varying the boost factor affects inference quality at 70 NFE budget with the same latents Fr- As increases from 1.00 to 1. 1.20, observe a steady rise in IR from 0.09395 up to 0. 0.13478. indicating stronger image-text alignment, and a modest gain in CS. However, this comes at the cost of higher FID and CMMD values, reflecting gradual drop in distributional similarity. We select b = 1.10 as our default because it strikes the best balance: it substantially boosts IR (0.11576 1576) with only moderate increase FID (14.232) and CMMD (0.59354) relative to lower b values.
Error-ratio threshold Table summarizes the effect of varying p with the same latents ry- Lowering P from 1.1 to 0.9 results in more conservative hare leaps NNpp rise from 70 to 75-and 14.232) and CMMD (0.59044 vs. 0.59354), at the expense of marginally lower IR (0. 1942 vs. 0.11576). Increasing P to 1.3 reduces NFE to 67 but degrades FID (14.280) and IR (0. 10849). We choose p 1.1 our default since it achieves the best trade-off: 30% NFE reduction (70 NFE) while maintaining competitive fidelity and alignment metrics.
yields slightly better FID (14.128
5
Conclusion
We present Tortoise and Hare Guidance, training-free acceleration framework for diffusion sampling that leverages a multirate reformulation of classifier-free guidance (CFG). Exploiting the asymmetric sensitivity of the noise estimate and the additional guidance term to numerical error, Tortoise and Hare Guidance integrates the noise estimate on a fine-grained grid while integrating the additional guidance term coarse grid. This approach allows for substantial reduction in the number of function evaluations (NFE) without sacrificing generation quality. With an errorbbouddaww -wware timestep sampler and guidance scale adjustment, our method achieves up 30% faster sampling while preserving fidelity across models like Stable Diffusion 1.5, 3.5 Large, and Audiol.DM 2, demonstrating the effectiveness of multirate integration for real-time high-quality generation.
Limitations Our experiments currently limited to latent diffusion models and few bench- mark datasets such as COCO 2014 and AudioCaps. Extending the evaluation to wider range of architectures, modalities, and downstream tasks will help assess the generality and robustness of our method.
Broader Impact By reducing sampling cost without retraining. Tortoise and Hare Guidance lowers the barrier deploying diffusion models in real-time applications such creative tools, accessibility services, and mobile environments. This could result accelerating the production of synthetic media, including deepfakes and misleading content. Nonetheless, the capabilities of Tortoise and Hare Guidance remain bounded by those of the underlying diffusion model, introducing a limited impact to the quality of such synthetic media.
Acknowledgments and Disclosure of Funding
We sincerely thank Byeongju Woo, Chanyong Lee, and Eunjin Koh for their constructive discussions and support. We also appreciate Minseo Kim and Minkyu Song for providing insightful feedback. This work was supported by the Agency For Defense Development Grant Funded by the Korean Government (9 19445701.).
10

---

References
[1] Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram Voleti, Adam Letts, et Stable video diffusion: Scaling latent video diffusion models to large datasets, arXiv preprint ar511:231 2023.
[2] Arwen Bradley and Preetum Nakkiran. Classifier-fre free guidance is a mredictor-comeett arXiv preprint arXiv:2408,0900 2024.
[3] Angela Castillo, Jonas Kohler, Juan C Pérez, Juan Pablo Pérez, Albert Pumarola, Bernard Ghanem, Pablo Arbelácz, and Ali Thabet. Adaptive guidance: Training-free accelcration of conditional diffusion models. arXiv preprint arXiv:2312.12487 2023.
Pengtao Chen, Mingzhu Shen, Peng Ye, Jianjian Cao, Chongjun Tu, Christos-Savvar Bouganis, Yiren Zhao, and Tao Chen. A-DiT: A Training free Acceleration Method Tailored Diffusion Transformers. arXiv preprint 4rX.v:2406.011255 2024.
[5] Ho Kei Cheng, Masato Ishii, Akio Hayakawa, Takashi Shibuya, Alexander Schwing, and Yuki Mit- sufuji. Taming multimodal joint training for high-quality vidco-to-audio synthesis. arXiv preprint arXiv:2412.15322,, 2024.
[6] Jinyoung Choi, Junoh Kang, and Bohyung Han. Enhanced diffusion sampling via extrapolation with multiple ode solutions. arXiv preprint (tt.::5000008555 2025.
[7] Hyungjin Chung. Jeongsol Kim, Geon Yeong Park, Hyelin Nam, and Jong Chul Ye. CFG++: Manifold- constrained classifier free guidance for diffusion models. arXiv preprint arXiv:2406.0888 080000, 2024.
[8] Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. Advances in neural information processing systems, 1188880-8944 2021.
[9] Bradley Efron. Tweedic's 's formula and selection bias. Journal of the American Statistical Association, 106 1614, 2011.
[10] Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, Dustin Podell, Tim Dockhom, 7ion English, Kyle Lacey, Alex Goodwin, Yannik Marek, and Robin Rombach. Scaling Rectified Flow Transformers for High- Resolution Image Synthesis. arXiv preprint urXiv:2403.088 03200. 2024.
[11] C. W. Gear and D. R. Wells. Multirate linear multistep methods. BIT, 2444):484-502, December 1984. ISSN 1572-9125, doi: 10. 10.10006019949000 URL http:://t//...//// doi org/ 10. 1
[12] Pareesa Ameneh Golnari. Zhewei Yao, and Yuxiong He. Selective guidance: Are all the denoising steps of guided diffusion important? arXiv preprint arXiv:2.05.009..0 2023.
[13] Sylvain Gugger, Lysandre Debut, Thomas Wolf, Philipp Schmid, Zachary Mueller, Sourab Mangrulkar, Mare Sun, and Benjamin Bossan. Accelerate: Training and inference scale made simple, efficient and adaptable. https://gittiii wwt.g/tww..mm/ggggggggaa///////e wwtp://.w.....acccttatceeeee/eee 2022.
[14] Ersst Hairer, Syvert P. Nørsett, and Gerhard Wanner. Solving Ordinary Differential Equations Nonstiff Problems. volume of Springer Series in Computational Mathematics. Springer- Verlag, Berlin, Heidelberg. edition, 1993. ISBN 978-3-540-56670-0, 978-3-540-78862-11
doi: 10. 3017779778--5--788--
[15] Moayed Haji-Al Ali, Guha Balakrishnan, and Vicente Ordonez. Elasticdiffusion: Training- free arbitrary size image generation through global-local content separation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6603 6612, 2024.
[16] Jack Hessel, Ari Holtzman, Maxwell Forbes, Ronan Le Bras, and Yejin Choi. Clipscore: A reference-free evaluation metric for image captioning. arXiv preprint arXiv:2104.08711 18, 2021.
[17] Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by two time-scale update rule converge to a local nash equilibrium. Advances neural information processing systems, 30, 2017.
[18] Jonathan Ho and Tim Salimans. Classifier-frre diffusion guidance. arXiv preprint arXiv:2207.12595 2022.
[19] Jonathan Ho, Ajay Jain, and Pieter Abbeel, Denoising diffusion probabilistic models. Advances in neural information processing systems, 13:6540-6851, 2020.

---

[20]
Jonathan Ho, Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi. and David J Fleet. Video diffusion models. Advances Neural Information Processing Systems, 35:8633-8646, 2022.
[21] Wenyi Hong, Ming Ding, Wendi Zheng, Xinghan Liu, and Jie Tang. Cogvideo: Large-scale pretraining for text-to-video generation via transformers. arXiv preprint arXiv:2205. 15868, 2022.
[22] Sadeep Jayasumana, Srikumar Ramalingam, Andreas Veit, Daniel Glasner, Ayan Chakrabarti, and Sanjiv Kumar. Rethinking fid: Towards a better evaluation metric for image generation. In Proceedings of the IEEE/CVF Conference Computer Vision and Puttern Recognition, pages 9307-9315, 2024.
[23] Steffen Jung and Margret Keuper. Internalized biases in fréchet inception distance. In NeurlPS 2021 Workshop on Distribution Shifts: Connecting Methods and Applications, 2021.
[24] Kevin Kilgour, Mauricio Zuluaga, Dominik Roblek, and Matthew Sharifi. Fréchet audio distance: reference- free metric for evaluating music enhancement algorithms. In Proc. Interspeeck 20/9, pages 2350-2354, 2019.
[25] Chris Dongjoo Kim, Byeongchang Kim, Hyunmin Lee, and Gunhee Kim. Audiocaps: Generating captions for audios in the wild. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume (Long and Short Papers), pages 119-132, 2019.
[26] Tuomas Kynkääänneemi, Miik Aittala, Tero Karras, Samuli Laine, Timo Aila, and Jaakko Lehtinen. Applying guidance in limited interval improves sample and distribution quality in diffusion models. arXiv preprint arXiv:2404.07720 2024.
[27] Tsung- Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C. Lawrence Zätnick. Microsoft coco: Common objects in context. In Computer vision-ECCV 2014: 13th European conference, zurich, Switzerland, September 62, 2014, proceedings, part 13, pages 740-755. Springer, 2014.
[28] Haohe Liu, Yi Yuan, Xubo Liu, Xinhao Mci, Qiuqiang Kong, Qiao Tian, Yuping Wang. Wenwu Wang. Yuxuan Wang, and Mark D Plumbley. Audioldm 2: Learning holistic audio generation with self-supervised pretraining. IEEE/ACM Transactions on Audio, Speech, and Language Processing. 32:2871-288 2024.
[29] Luping Liu, Yi Ren, Zhijie Lin, and Zhou Zhao. Pseudo numerical methods for diffusion models on manifolds. arXiv preprint arXiv:2202.0977 2022.
[30] Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu. DPM-solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps. Advances in Neural Information Processing Systems, 35:5775-5787, 2022.
[31] Zhengyao Ly, Chenyang Si, Junhao Song, Zhenyu Yang, Yu Qiao, Ziwei Liu, and Kwan- Yee K Wong. Fastereache: Training-free video diffusion model acceleration with high quality. arXiv preprint arXiv:2410. 193555,5,, 2024.
[32] Xinyin Ma, Gongfan Fang, Michael Bi Mi, and Xinchao Wang. Learning-to-car cacca Accelerating diffussion transformer via layer caching. Advances in Neural Information Processing Systems, 001333222223333000 2024.
[33] Xinyin Ma, Gongfan Fang, and Xinchao Wang. Deepeache: Accelerating diffusion models for free. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 15762-15772, 2024.
[34] Chenlin Meng, Robin Rombach, Ruiqi Gao, Diederik Kingma, Stefano Ermon, Jonathan Ho, and Tim Salimans. On distillation of guided diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pages 14297-14306, 2023.
[35] Gaurav Parmar, Richard Zhang, and Jun-Yan Yan 7hu. On Aliased Resizing and Surprising Subtleties in GAN Evaluation. In CVPR, 2022.
[36] A Paszke. PyTorch: An Imperative Style, High- Performance Deep Learning Library. arXiv preprint arXiv:1912.07700 2019.
[37] Sayak Paul. soco-30-val 1000-al--0114 2014: 30k randomly sampled tion split. https::/huuginngace co /ttaa//asess/saaapaaulu////eeeee 30 vval2014 2023. Dataset on Hugging Face. Accessed 18 Apr 2025.
pairs from the COCO 2014 valida-
12

---

[38] Alec Radford, Jong Wook Kim, Chris Hallacy. Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pages 8748-8763. PmLR, 2021.
[39] John R Rice. Split Runge- Kutta methods for simultaneous equations. Journal of Research of the National Institute of Standards and Technology, 60, 1960.
[40] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684- 10695, 2022.
[41] Ludan Ruan, Yiyang Ma, Huan Yang. Huiguo He, Bei Liu, Jianlong Fu, Nicholas Jing Yuan, Qin Jin, and Baining Guo. Mm-diffusion: Learning multi-modal diffusion models for joint audio and vidco generation. In Proceedings the the IEEE/CVF Conference Computer Vision and Pottern Recognition, pages 10219-10228, 2023.
[42] Seyedmorteza Sadat, Jakob Buhmann, Derek Bradley, Otmar Hilliges, and Romann M Weber. CADS: Unleashing the diversity of diffusion models through muddtion-anncalee sampling. arXiv preprint 1xXiv:2310113444 2023.
[43] Tim Salimans Jonathan Ho. Progressive distillation for fast sampling diffusion models. arXiv preprint arXiv:2202.000.. 12 2022.
[44] Adrian Sandu. Multirate time integration: an overview. https://cnls... gww/ttmm/23/////// tim2023, pdf) Aug 2023. Presentation at the 2023 Los Alamos Workshop on Time Integration for Multiphysics. Accessed 30 Apr 2025.
[45] Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. arXiv preprint axiv:2010.02500,, 2020.
[46] Yang Song. Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-hased generative modeling through stochastic differential equations. arXiv preprint arXiv:2011. 2020.
[47] Stability AL Stable Diffusion 3.5 Large. https://huggingface co/stabilityai/
stable-diffustie www.ddiifffffnn.o 3.5 large 2024. Model on Hugging Face. Accessed 25 Apr 2025.
[48] Patrick von Platen, Suraj Patil, Anton Lozhkov Pedro Cuenca, Nathan I Lambert, Kashif Rasul, Mishig Davaadorj, Dhruv Nair, Sayak Paul, William Berman, Yiyi Xu, Steven Liu, and Thomas Wolf. Diffusers: 2022.
State-of-the-art diffusion models. battp:://githue www.tthh.......utteeeeee http:///uggiggggeeeeeeeeaeeaeeee
[49] Jinginn Wang. Dan Zhang, and Feng Luo. Unified Directly Demoising for Both Variance Preserving and Variance Exploding Diffusion Models. arXiv preprint 2rX 2024.
[50] Yusong Wu, Ke Chen, Tianyu Zhang, Yuchen Hui, Taylor Berg- Kirkpatrick, and Shlomo Dubnov. Large- scale contrastive language-audin pretraining with feature fusion and keyword-to-caption augmentation. In ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Prucessing (ICASSP). pages 1-5. IEEE, 2023.
[51] Jiazheng Xu, Xiao Liu, Yuchen Wu, Yuxuan Tong, Qinkai Li, Ming Ding, Tang, and Yuxiao Dong. Imagereward: I learning and evaluating human preferences for text-to-image generation. Advances in Neural Information Processing Systems, 36:15903- 15593-159355 2023.
[52] Zangwei Zheng, Xiangyu Peng, Tianji Yang, Chenhui Shen, Shenggui Li, Hongxin Liu, Yukun Zhou, Tianyi Li, and Yang You. Open-sora: Democratizing efficient video production for all. arXiv preprint arXiv:2412.20404 2024.
13

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
The claims made in the abstract and introduction are stated in Section and
Section
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
We discuss the limitation of our work in Section 5
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
14

---

We specify the assumptions in the preliminaries (Section 3) and the beginning of each algorithm. We also provide complete derivations of our formulas.
Justification:
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proof's can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification:
We provide full descriptions of the introduced algorithms.
Guidelines:
The answer NA means that the paper does not include experiments. If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
15

---

Answer: [Yes]
Justification:
We publicly release our code.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips. cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
/
wwt.Suubmisiioooiiooooooooio for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data,
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We specify which model, dataset, and hyperparameters are used in the experi- ments in Section 4
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: We provide 2-sigma error ranges for figures in Section 3 include results from repeated experiments.
Our main table
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
16

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We provide machine specification in Section
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't 't make it into the paper).
9. Code of ethics
Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Question:
Answer:
[Yes]
Justification:
The research conducted in the paper conform with the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
We discuss broader impacts of our research in Section
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no socictal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
17

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of MI..
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
The paper does not introduce any new models or datasets.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Lieenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: We mention the licenses of the used assets in the appendix.
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
18

---

If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: The paper does not release any new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/mmoel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question:
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
19

---

Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that if the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality the research, declaration is not required.
Answer: [NA]
Justification: The core method development in this research does not involve LLMs any important, original, or non-standard components.
Guidelines:
The answer NA
that the method development in this research does involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy for what should or should be described.
20

---

A
u-prediction models
Recent models such Stable Diffusion 3.5 [47 directly infer u, the velocity field of the reverse diffusion process. The diffusion ODE is then defined as
dc _t (0(x)),
2T ~ N(0,1)
(17)
By the definition of CFG 18). we have
(x())) = 80 (are) + w (Ue(2)) - 02 (3+)) = De (2re) + (w 1) Ade(w))
(18)
where (de(=1) == (x())) 02 (zz). Substituting Eq. 18 into Eq. 17 yields the following:
zi (z() + (w 1) (z)
(19)
We split this diffusion ODE into multirate system of ODEs similar to Section 3.1
d . (x
+ x
(w 1) ( a!
(20)
dt
dt
Both equations retain the form of Eq. 17 so that existing solvers as the Euler method can be applied to each equation without modification. Furthermore, Algorithm | could be utilized unchanged since it is agnostic the form of equation the type of the diffusion model solver.
B Proof for approximation error bound analysis
We provide a proof for error accumulation presented in Section 3.3 bounds could be found in Section II. 3. of [14).
More rigourous analysis of error
Theorem 1. Assume the local integration error of an ODE using a solver of order p and timestep size At is given by:
St-At At - Tt-A: = c. (At) + (((at)++2)
(21)
for sufficiently small At. Then the error of using the same solver repeatedly for m steps is given by ât-mAt Zi-mat
m( + (((At)++))
(22)
Proof. We use mathematical induction. (Base step) For m Eq. 22 reduces the assumption.
(Inductive step) Assume the error of using the solver m times is given by Eq. 22 We proceed to the next iteration obtain f (2-(m11)A- Let (1+(m+ be the exact solution given by solving the
ODE from mAI to (m + 1) using Att The error in Eq. 22 is transported to the next timestep as
2--(m+))Ar - (1=-(m+))a = (I + O(A+)) (&c-mar - FL-mar)
(23)
c- ccmm(At)pt+ + ((Att)))))
(24)
On the other hand, the local error of the next iteration is also given by Eq. 21
f Pe-(m+1)Ar Zz- (m+++ = (At)p+1 + ((ltt)
(25)
The error of using solver m + times is thus
(m I)At 2t-(m+ = c- (m 1) ((()))+)))
(26)
Therefore the error of using the ODE solver times is given by Eq. 22 for all positive integer m.
C More details for Richardson Extrapolation
We specify further details about the computation of the coarse timestep grid C. We calculate ||= - [2) and |
H(1) - _ H(2) by solving both the tortoise and hare equations on the fine-
grained timestep grid using Algorithm 3 In particular, for each denoising step 4, we first find (.())
by using the diffusion model solver once from 4 to 4, Then we find (2) by using the diffusion
model solver twice, from t, to (t, + f221)/2 and from (t, + f/41)/2 to f.41- We use 2 for the next denoising step to ensure that we follow the reference trajectory of CFG [8] Together with Algorithm 2 obtain the coarse timestep grid C specified in Table 4
21

---

Algorithm 3 Richardson Extrapolation
Require: 2T ~ ~N(00,
Initial noise
Require:
w w20
A Guidance scale
Require: {LibosiNNtt TN =0
Fine- grneigr timestep grid
zy FT
2: 2T
3: for arri=00 00 N - do
4:
ée t éB (0) 22 c)
5:
Eg t co + +rt, +
6:
De. & -zz
7:
Fra - S Eor li, li+1)
T(I)
D Sta1 of the tortoise
(1)
8:
Fiom - Solver( (w - 1) D bis 411)
H(L)
2141 of the hare
9:
Im [4: + 1
Midpoint current and next timesteps
I T(2) Solver 22 te tm)
10:
H(2)
11:
Fm t Solver
t1 (w - 1). tm)
T(2)
H(2)
12:
& to ttm
T(2)
H(2)
13:
Fmm .0)
14:
A + &
Ez
T(2)
Solver( T(2)
tm EEcr lm l441)
t of the tortoise
H(2) Solver( H(2) (w- 1) Ddcs tmm t4+1)
a/2)
of the hare
Faa Pras H(1)
Tortoise of next step
H(1) H(2) 18: F Seee
Hare of next step
end for
20: return T(L) T(2)
1441 2241
Table 4: Obtained coarse timestep grid for different P values. Our choice is marked in blue For brevity, only indices of the timesteps are shown. Note that only ini is actually used in the final algorithm.
Lilt E C}
Stable Diffusion 1.5 with DDIM
0.9
(0, 2, 3, 4, 5, 6, 8, 10, 12, 14, 16, 18, 20, 22, 26, 28, 30, 34, 35, 36, 37, 39,
40, 42, 43, 44, 45, 46, 47, 48, 49)
1.0
2,,1,,,3,4553 9, 11, 13, 15, 19, 19, 21, 25, 27, 29, 33, 37, 38, 39, 40, 41,
42, 42,43,44, 44, 45, 46, 47, 49.
1.1
1,, 6,8, 8, 10, 12, 14, 17, 20, 26, 28, 30, 32, 34, 36, 38, 39, 40, 41, 42, 43, 44,
45, 46, 47, 48, 49)
1.2
(0, 1, 5, 7, 7,9, 9, 11, 14, 17, 20, 23, 26, 29, 31, 37, 39, 41, 43, 44, 45, 46, 47,
48, 49)
1.3
(0, 2, 8, 10, 13, 16. 19, 22, 25, 28, 31, 34, 36, 38, 40, 42, 44, 45, 46, 47, 48, 49)
Stable Diffusion 3.5 Large with Fuler method
0.9
{0. 2, 3, 5, 7, 10, 13, 16, 18, 22. 23, 24, 26)
1.0
[0, 1, 6, 12, 15, 18, 20, 22, 24, 24, 26)
1.1
[0, 1, 2, 6, 13, 17, 20, 22, 23. 24, 25, 27)
1.2
1, 11, 15, 19, 21, 23, 25, 27)
1.3
[0. 1, 6, 10, 15. 19, 22, 24, 26)
22

---

Table 5: Obtained coarse timestep grid for different w values. Our choice is marked in blue. For brevity, only indices of the timesteps are shown. Note that only ihi is actually used in the final algorithm. The results show that while bigger results in denser C, the overall trend is consistent.
Variant
{iltz E C}
Stable Diffusion 1.5 with DDIM
w 6.5, p = 0.93 {0, I, 2, 3, 4, 6, 8, 10, 12, 14, 17, 20, 23, 26, 28, 30, 32, 34, 36, 38, 39, 40,
41, 43, 44, 45, 46, 47, 48, 49, 50)
w = 6 1.1
(0, 1, 1,1,2,34,,6.8 6. 10, 13, 16, 19, 22, 25, 28, 31, 33, 35, 37, 39, 41, 44,
47,
w -
10,1,2,3,4,5, 8, 10, 12, 10,
28, 30, 34, 38, 39, 40,
41, 43, 45,
49, 50)
w 8 1.1
44, 22, 26, 28, 30, 32, 34, 36, 50)
37,38, 39, 41,
8.5, = 1.22
p= [0, 1,2.3,2, 6, 8, 12, 14, 17, 20, 23, 26, 28, 30, 32, 34, 36, 38, 39, 40,
41, 42, 43, 44, 45, 46, 47, 48, 49, 50)
Table 6: Obtained coarse timestep grid with fewer sample trajectories. Our original choice is marked in blue For brevity, only indices of the timesteps are shown. Note that only < in is actually used in the final algorithm. The results show that C' can be computed accurately with only 1,000 sample trajectories.
Batch size
loU
filtz C}
Stable Diffusion 1.5 with DDIM 30k
[0. 1, 3, 4. 8, 10, 12. 14, 17, 20, 26. 28, 30, 34,
36, 38.
46, 47, 48, 49, 59, 4.2133.......00012.444....0..,,
44, 41,444444
Ik (trial 1)
100%
28, 30, 32,
36, 38, 39, 40,
47, 48, 49, 50] 49,4
1k (trial 2)
100%
3,4,,,
28, 30, 34,
36, 38, 39, 40, 40,4444,,,,,,,,444,4555
50]
Ik (trial 3)
100%
(0, 1,2.3, 3, 10, 14, 17, 26, 28, 30, 32, 34,
36, 38, 39, 40, 41, 43, 44, 46, 47, 48, 49, 50)
1k (trial 8)
57.5%
3,, 5, 11, 13, 16, 19, 22, 25. 28, 30, 32, 34, 36, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50)
1k (trial 30)
100%
{0, 3, 6, 8, 10. 12, 14, 20, 23, 26, 28, 30, 32, 34, 36, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50)
D Sensitivity of C
Guidance weight w The coarse timestep grid C' depends on guidance weight w, since the error bounds computed by Algorithm depend on w. As increases. both the hare term and its approxi- mation eттоr grow, resulting in a denser coarse grid C.. Table C evaluated under different w values. While a bigger guidance scale results in denser C, the overall trend is consistent; one can obtain the same C' by adjusting
While we computed C' with sample trajectories on 30.000 prompts from the COCO 2014 dataset, possible to compute C' using fewer sample trajectories. Table shows C computed on batch of 1,000 prompts, compared to C' computed on 30,000 prompts. The results closely matched our original, large-scale estimate. Compared the original estimate, the 1,000 sample estimate demonstrated 95.25% IoU (i.c., Jaccard index) in average over 30 trials.
Batch size
23

---

Table 7: Comparison of methods in terms of distributional similarity and prompt fidelity. Our method
marked in blue whereas vanilla CFG is marked in gray The parameters THG correspond to
(p, b, mn). The best results are highlighted.
Distributional similarity
Prompt fidelity
Method
N
NFE 4
FID +
CMMD
CS f
IR f
Stable Diffusion 1.5 with DDIM
CFG [18]
50
100
14.133
0.58948
26.295
0.14764
Selective Guidance 12
50
12.895
0,56052
25.602
-0.06691
Guidance Interval 26
50
70
14.555
0.62227
26.153
0.09658
CFG-Cache [31
50
14.422
0,59759
26.179
0.09705
THG (1.1, 1.1, 38)
50
70
14.232
0.59354
26.197
0.11576
CFG 18
35
70
13.342
0.57232
26.321
0.12694
Selective Guidance 12
35
49
12.299
0,54550
25.623
-0.09007
Guidance Interval 26
35
49
14,490
0.61178
26.140
0.05919
CFG-Cache [31
35
49
13.773
0,58162
26.120
0.05246
THG (1.7, 1.1, 30)
35
49
13.468
0.57637
26.265
0.09202
CFG [18]
20
13.366
0,56159
26.370
0,10090
Selective Guidance [1]
20
30
12.839
0.54740
25.900
-0.04452
Guidance Interval [66
20
14.937
0.61333
26.244
0.02827
CFG-Cache [31]
20
30
13.675
0.56815
26.211
0.04604
THG (3.5, 1.1, 17)
20
30
13.345
0.56719
26.278
0.07212
E
More comparisons with CFG variants
across wider range of NFEs. Note that we used different number of fine-graine fineggraieed timesteps to control the NFE of CFG. The results demonstrate that our approach achieves superior image quality with the same NFE budget. While Selective Guidance [12] shows lower FID and CMMD values, they generate images with low prompt fidelity and degraded details indicated by lower CS and IR values noted in Appendix G Aside from this phenomenon, THG achieves superior overall performance.
Table 7 extends our comparison to include additonal baselines 12 26
Comparison with vanilla CFG Although vanilla CFG with 70 NFE yields a FID of 13.342 (better than THG's 14.232 with NFE), its PSNR drops substantially to 19.42 dB compared 24. dB for THG. This aligns with observations in (6), where reducing N sometimes lowers FID. Moreover, 35 23 show that FID may not reliably reflect perceptual quality when the image structures diverge, so we focus on comparisons at the same N.
F
More details for metrics
FID, FAD Fréchet Inception Distance 17 (FID) is a ubiquitously used metric for developing and adopting image generative models. It measures the distance between real and generated images in deep feature space to capture relevant features of the two distributions (35] Therefore, lower FID value indicates realistic generation. To measure FID, first uses an Inception V3 feature extractor model to compute features from real and generated images. Under the assumption that the resulting feature sets follows multidimensionall Gaussian distribution, the distance of the two distributions ( E,) and (( Mg) is given by the Fréchet distance
FD ||, 1ayl tr(E,, + E.g 2(E,,,)))))
(27)
Similarly, Fréchet Audio Distance (24 (FAD) measures the distance between real and generated audio by using a VGGish embedding model to extract features from audio clips.
is recently proposed alternative for FID. It uses CLIP 38 embeddings and the Maximum Mean Discrepancy (MMD) distance instead of Inception V3 features and the Fréchet distance. CLIP is trained on 400 million images with corresponding text descriptions and therefore
CMMD CMMD 22

---

Table 8: Ablation study for the guidance scale w with CFG
[8
Method
NFE
FID +
CMMD +
CS
IR 1
Stable Diffusion 1.5 with DDIM
W 2.5
100
8,438
0.56672
25.153
-0.28577
3.5
100
9.143
0.54192
25.687
-0.09190
3 = 4.5
100
10.644
0.54764
25.935
0.00670
5.5
100
12.030
0,56171
26.110
0.07195
W 6.5
100
13.222
0,57673
26.225
0.11582
7.5
100
14.133
0.58948
26.295
0.14764
8.5
100
14.902
0.60343
26.369
0.17431
Figure 5: Generated images using w = 2.5 for prompts "A group of zebras grazing in the grass.", "A yellow commuter train traveling past some houses.", "A couple of men standing on a field playing baseball.", and "Zoo scene of children at zoo near giraffes, attempting to pet or feed them." from the COCO 2014 dataset.
much more suitable for capturing rich and diverse content. MMD does not make any assumptions about the underlying distributions, unlike the Fréchet distance which assumes multidimensional Gaussian distributions. By combining CLIP and MMD, CMMD avoids the drawbacks of FID.
CLIP Score, CLAP Score CLIP Score [16] uses CLIP (38] to assess image-caption compatibility, CLIP learns multimodal embedding space by jointly traning image encoder and text encoder, while the cosine similarity matching image and text embeddings are maximized. Leveraging this design, CLIP score is defined by the cosine similarity of the image embedding E, and text prompt embedding Ec as
CLIPScore(L, (I, C) max(100 cos(E,, Ec),
(28)
We report the average CLIP score over the generated image set. Similarly, CLAP Score uses the CLAP model to assess audio-caption compatibility.
text-to-image image human preference reward model, effectively encoding human preferences by traning actual human feedback. Experiments shows that ImageReward aligns with human ranking better than zero-shot FID and CLIP Score. Also, ImageReward values have larger quantile value range than that of CLIP Score, demonstrating that can well distinguish the quality of images from each other. We report the average ImageReward value the generated image set.
ImwgeReward ImageReward 51 is a general-
G Tradeoff of distributional similarity and prompt fidelity
Tables I and demonstrate tradeoff between distributional similarity metrics (FID, CMMD) and prompt fidelity metrics (CS, IR). When the prompt fidelity metrics improve so that each image matches better with the given prompt, the distributional similarity metrics worsen that the distribution of the images is further from that of real images.
We further investigate this phenomenon by conducting additional ablation study for the guidance scale using Stable Diffusion 1.5 and CFG. Table shows how the metries change as changed. The minimum FID is achieved 2.5 and the minimum CMMD is achieved at 3.5. However,
25

---

they suffer from low CS and IR. Generated images using w = 2.5 are visualized in Fig. 5 showing degraded details or insufficient text alignment. This suggests that lower FID or CMMD does not always indicate better generation quality. While these distributional similarity metrics measure both image plausibility and diversity, they can possibly fail report high-quality details of the images with lower values.
Since the global structure of each image is determined by the initial few steps of the reverse diffusion process [26] 3]. the images generated by the methods in Table T have mostl shared global structures and differ on delicate details. Given that, we suggest that the suman-perceives quality of generated samples could be better explained by the prompt fidelity metrics compared to the distributional similarity metrics. Our results in Table with slightly higher FID or CMMD therefore do not indicate a significant degradation of generation quality.
H
Licenses
Stable Diffusion 1.5 weights released under the CreativeML. Open RAIL-M license (v1.0; https://githhl: com CompVis/ Settipiiis/stableeeeee stable-d www.ddffsiooooooloooo wawaaaaiinninnnni
Stable Diffusion 3.5 Large weights released under the Stability AI Community Licence v3 (research commercial use for organizations or individuals with < USD 1 M annual hwvemuc:https:://tabbill
ai/license)
- FID-cleen-FDD implementation by Parmar et al., released under the MIT License (v1.0;
https://github httcchuuu.oomeeee www.cccmccaaccc http//aaaarmarr/
LLICENEE
CMMD - PyTorch implementation of CLIP Maximum Mean Discrepancy by Sayak
Paul, released under the Apache License 2.0 (v2.0; https://httt:::/thubb com cooom/ayaapaul/
CLIP Score - TorchMetrics' CLIPScore module released under the Apache License
2.0 (v2.0;
Lightning-AA
ImageReward - model and evaluation code released under the Apache License 2.0 (v2.0;
https://github. www.wcmmm....../ THUDM/ Imaggleearrd/bb00//mm/a///CCCCC Xu et al., 2023)
MS COCO 2014:
Annotations released under the Creative Commons Attribution 4.0 International license (CC BY 4.0; httts::/ccreativeee
org/liiiiic httg:/liceenseeby/44.. 0/)
UUnderlying images governed by Flickr Terms of Use; users must comply with Flickr's rules when reusing or redistributing any COCO images.
AudioLDM 2
weights released under the Creative Commons Attribu- Saa.aammmmeeria--aaaliiilii 4.0 International License (nttps://github com/
- released under the MIT License (v1.0; https:// com/ adjjaa/aachooopp//l//////aa/assr
implementation of Frechet Audio Distance by Hao Hao Tan, released under
the MIT License (v1.0; httts:t/tt:tthhii com hatm//uugd//frrceeetttt weaegg/dde//reeeet---atteee audio---auuu---e wwwi--iistiidddddee...ee blob/main//ICENNS:
SLAP model and evaluation code released under the Creative Commons CCO 1.0 Universal License; public domain dedication
Comm/LAION-AI/CAA
blob/main/LICENS..
I More qualitative results
Figure shows more qualitative results for Stable Diffusion 1.5. Figures anddd show more qualitative results for Stable Diffusion 3.5 Large.
26

---

Prompt: Two horses are frolicking as spectators take pictures.
Prompt: a male with a purple jacket on skies posing for picture
(a) CFG (baseline) NFE- 100
(b) CFG-Cache w/o FFT NFE 70
(c) CFG-Cache NFE 70
(d) Ours
NFE 70
Figure 6: Comparison of visual results for prompts from the COCO 2014 dataset using Stable Diffusion 1.5.
Prompt: A woman and a man are playing the nintendo wii video game system
Prompt: A cat sitting window sill near basket.
(a) CFG (baseline) NFE 56
(b) CFG-Cache w/o FFT NFE 38
(c) CFG-Cache NFE 38
(d) Ours
NFE 38
Figure 7: Comparison of visual results for prompts from the COCO 2014 dataset using Stable Diffusion 3.5 Large
27

---

Prompt: red fire hydrant set up a grassy clearing.
Prompt: A stop sign and one way sign are in front of large building
(a) CFG (baseline) NFE- 56
(b) CFG-Cache w/o FFT NFE= =
(c) CFG-Cacha NFE 38
(d) Ours
NFE 38
Figure 8: Comparison of visual results for prompts from the COCO 2014 dataset using Stable
Diffusion 3.5 Large.
28


Reviews:
[{"review_id": "1iOFGMS25C", "reviewer": "Reviewer_KWZf", "summary": "The authors propose Tortoise and Hare guidance,  a training free framework for reducing the number of function evaluations during guided sampling in diffusion model. This is achieved by splitting the evaluations into a branch that is sensitive to change in time step (noise estimate) and a second branch which is relatively less sensitive to change in time step (guidance term). The signal at each time step is then calculated as a sum of the noise estimate (tortoise branch) and the guidance term (hare branch). The noise estimated is updated on a finer time scale whereas the guidance term is only update on a coarse time grid. The qualitative performance is shown to be close to standard diffusion and comparable quantitative performance is reported with respect to relevant baselines.", "questions": "1. What is the result if $x_t^H$ is only updates at the coarse time steps and its approximation for intermediate fine time steps not calculated ? In particular, instead of calculating $x_s^H$ for $s<t$, would it be possible to use $x_t^H$ for every $s$ between $x_t$ and $x_{t+1}$.   \n  \n2. The authors mainly mention the use of DDIM and Euler method for evaluation. Is this approach also applicable to higher order methods for step reduction, and if not, what are the major considerations?", "limitations": "An adequate treatment of the limitations of the approach has been provided. In particular the authors acknowledge that the approach is only tested on first order solvers and on limited evaluation datasets.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 4, "originality": 4, "strengths_and_weaknesses": "## Strengths\n1. **Writing quality**: The paper is well written with attention to detail. The core motivation for the approach and the central idea of the paper is presented in a manner that is easy to understand and appreciate. \n\n2. **Simple and elegant**: The proposed solution is relatively straightforward and explained in detail aiding in the ease of reproduction.  \n\n3. **Theoretically Grounded**: The motivation for the multi rate approach is theoretically grounded and error bound analysis has been provided to support the choice and design of certain hyper-parametes .  \n\n4. **Adaptive guidance scale**: This is an interesting trick to help maintain quality of samples depending on the resolution of coarse grid.  \n\n5. **Generality**: The proposed approach is a fundamental change to denoising process, allowing it to be used in any kind of diffusion model. Additionally, since it is a training free approach, it can be swapped into existing inference pipelines easily. \n\n## Weaknesses\n1. **Limited Qualitative results**: The main paper shows only 2 qualitative result for the difference in quality between the proposed approach and other baselines. Although the appendix includes some more. Demonstrating this on a wide variety of examples would be beneficial to understand and appreciate the effectiveness of the approach.  \n\n2. **Effect of schedule**: No ablation or analysis has been provided to indicate the effect of the timestep schedule during inference. Are the error bounds and performance guarantees the same for all time step schedules? Some discussion about this would be insightful   \n\n3. **Calculation of boosting term**: The authors ablate this term as well as motivate the need for it. But since the sensitivity changes across time steps, should the $b$ term be time dependent? Some discussion on this would be helpful.  \n\n4. **Evaluations**: Although the authors acknowledge this in the limitation section, the approach has been evaluated only on the COCO dataset. Since the algorithm is general enough, checking the performance on a wide variety of diffusion models would be beneficial. Although not a strict requirement, this could potentially also be evaluate on other modalities like video and audio.   \n\n5. **Coarse grid calculation**: The coarse grid is calculated based on the error bounds. Is this calculation independent of starting noise and the guidance weight?", "was_revised": true, "final_justification": "The authors rebuttal address most of the concerns raised, and the included results help strengthen the claims of the paper.", "rebuttal": "Dear reviewer KWZf,\n\nThank you for your valuable feedback and detailed comments. We appreciate your recognition of the theoritically grounded motivation, the straightforward implementation, and the generality of our method. We address your concerns and questions in the response below.\n\n> 1. **Limited Qualitative results**: The main paper shows only 2 qualitative result for the difference in quality between the proposed approach and other baselines. Although the appendix includes some more. Demonstrating this on a wide variety of examples would be beneficial to understand and appreciate the effectiveness of the approach.\n\n[A1] We thank the reviewer for this constructive feedback. We agree that including more qualitative examples in the main paper is crucial for demonstrating how well our approach works and how it can be used in other situations. We will gladly add more qualitative results for the main paper in the final version with increased page limits.\n\n---\n\n> 2. **Effect of schedule**: No ablation or analysis has been provided to indicate the effect of the timestep schedule during inference. Are the error bounds and performance guarantees the same for all time step schedules? Some discussion about this would be insightful.\n\n[A2] Indeed, it is possible to use a different timestep schedule for some models. [1] suggested that the 'trailing' strategy should be used to decide the timestep schedule for DDIM, while we have used the original 'leading' strategy in our paper. We tested THG with the fine timestep grid decided by the 'trailing' strategy. The results show THG is still effective under this new timestep schedule.\n\n| Variant   | NFE | FID ↓   | CMMD ↓   | CS ↑    | IR ↑     |\n|---|---|---|---|---|---|\n| CFG (baseline) | 100 | 14.746  | 0.60033  | 26.302  | 0.14847  |\n| | | | | | |\n| CFG-Cache w/o FFT | 70  | 14.923  | **0.60296**  | 26.137  | 0.08263  |\n| CFG-Cache | 70  | 15.019  | 0.60725  | 26.169  | 0.09253  |\n| THG (Ours) | 70  | **14.834**  | 0.60307  | **26.176**  | **0.11392**  |\n\n---\n\n> 3. **Calculation of boosting term**: The authors ablate this term as well as motivate the need for it. But since the sensitivity changes across time steps, should the $b$ term be time dependent? Some discussion on this would be helpful.\n\n[A3] Thank you for your insightful suggestion. We performed an additional ablation study on SD1.5 with two time-dependent $b$ scenarios: (1) $b$ linearly grows from 1.0 at $t = T$, to 1.2 at $t = 0$. (2) $b$ linearly falls from 1.2 at $t = T$, to 1.0 at $t = 0$. The results indicate that there is no appreciable difference in generation quality between time-dependent $b$ and constant $b$. Our preliminary results were inconclusive, but we appreciate the suggestion and intend to explore it in future work by conducting a sensitivity-driven search for improved $b$ schedules.\n\n| Variant | NFE |  FID ↓   | CMMD ↓   | CS ↑    | IR ↑     |\n|---|----|---|---|---|---|\n| $b: 1.0 \\mapsto 1.2$ | 70 | **14.047** | **0.59008** | 26.185  | 0.11211  |\n| $b: 1.2 \\mapsto 1.0$ | 70 | 14.413 | 0.59568 | **26.201**  | **0.11938**  |\n| $b: 1.1 \\mapsto 1.1$ | 70 | 14.232 | 0.59354 | 26.197 | 0.11576 |\n\n---\n\n> 4. **Evaluations**: Although the authors acknowledge this in the limitation section, the approach has been evaluated only on the COCO dataset. Since the algorithm is general enough, checking the performance on a wide variety of diffusion models would be beneficial. Although not a strict requirement, this could potentially also be evaluate on other modalities like video and audio.\n\n[A4] We appreciate the reviewer for this constructive suggestion. Although exclusive use of the COCO dataset remains the standard for benchmarking T2I diffusion models [2-4], we agree that validating across modalities is important. Consequently, we performed preliminary audio generation experiments with AudioLDM2 [5]. We used the validation set of AudioCaps [6] as the audio-text pair dataset and measured the performance with FAD [7] and CLAP Score [8], each corresponding to FID and CLIP Score. (The three parameters of THG corresponds to $(\\rho, b, t_{hi})$.)\n\n| Variant | NFE | FAD ↓ | CLAP Score ↑ |\n|---|---|---|---|\n| CFG (baseline) | 100 | 2.596 | 0.2409 |\n| | | | | | |\n| CFG-Cache w/o FFT | 72 | 2.777 | 0.2312 |\n| THG (0.9, 1.15, 41) | 72 | **2.732** | **0.2362** |\n| | | | | | |\n| CFG-Cache w/o FFT | 70 | 2.901 | 0.2251 |\n| THG (0.9, 1.15, 39) | 70 | **2.764** | **0.2342** |\n| | | | | | |\n| CFG-Cache w/o FFT | 68 | 3.503 | 0.1972 |\n| THG (0.95, 1.15, 38) | 68 | **2.803** | **0.2328** |\n\nWe will include these results in the appendix. We believe that this expanded evaluation further demonstrates the generality and robustness of our approach.\n\n---\n\n> 5. **Coarse grid calculation**: The coarse grid is calculated based on the error bounds. Is this calculation independent of starting noise and the guidance weight?\n\n[A5] We thank the reviewer for raising this important point. The error bounds are computed over the COCO validation set by sampling independent initial noise per prompt. Therefore, they characterize the full noise manifold and are independent to any single noise.\n\nFor the guidance weight (or guidance scale) $\\omega$, we have treated it as a fixed model parameter in our experiments. We expect that as $\\omega$ increases, both the hare term and its approximation error grow, resulting in a denser coarse grid. To substantiate this, we evaluated the coarse timestep grid $C$ under several $\\omega$ values.\n\n| Variant | Steps |\n|---|---|\n| $\\omega=6.5, \\rho=0.93$   | 0, 1, 2, 3, 4, 5, 6, 8, 10, 12, 14, 17, 20, 23, 26, 28, 30, 32, 34, 36, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50 |\n| $\\omega=6.5, \\rho=1.1$   | 0, 1, 2, 3, 4, 6, 8, 10, 13, 16, 19, 22, 25, 28, 31, 33, 35, 37, 39, 41, 43, 44, 45, 46, 47, 48, 49, 50 |\n| $\\omega=7.5, \\rho=1.1$   | 0, 1, 2, 3, 4, 5, 6, 8, 10, 12, 14, 17, 20, 23, 26, 28, 30, 32, 34, 36, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50  |\n| $\\omega=8.5, \\rho=1.1$   | 0, 1, 2, 3, 4, 5, 6, 7, 8, 10, 12, 14, 16, 18, 20, 22, 24, 26, 28, 30, 32, 34, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50 |\n| $\\omega=8.5, \\rho=1.22$   | 0, 1, 2, 3, 4, 5, 6, 8, 10, 12, 14, 17, 20, 23, 26, 28, 30, 32, 34, 36, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50   |\n\nWhile a bigger guidance scale results in a denser $C$, the overall trend is consistent; one can obtain the same $C$ by adjusting $\\rho$.\n\n---\n\n> 1. What is the result if $x_t^H$ is only updates at the coarse time steps and its approximation for intermediate fine time steps not calculated ? In particular, instead of calculating $x_s^H$ for $s<t$, would it be possible to use $x_t^H$ for every $s$ between $x_t$ and $x_{t+1}$.\n\n[A6] As you suggested, updating $x_t^H$ only at the coarse time steps and reusing its most recent value for intermediate fine steps correspond to an alternative multirate integration method [9]. We implemented this alternative scheme, where $x_t^H$ remains constant for $t\\notin C$, and report the results. Our findings indicate that while this approach offers negligible computational savings, it yields a modest decrease in approximation fidelity compared to our fully interpolated method.\n\n| Variant  | NFE | FID ↓   | CMMD ↓   | CS ↑    | IR ↑     |\n|-----|----|-----|------|-----|-----|\n| Your suggestion | 70  | 14.351  | 0.59390  | 26.168  | 0.10067  |\n| Ours | 70  | **14.232**  | **0.59354**  | **26.197**  | **0.11576**  |\n\n---\n\n> 2. The authors mainly mention the use of DDIM and Euler method for evaluation. Is this approach also applicable to higher order methods for step reduction, and if not, what are the major considerations?\n\n[A7] Thanks for your constructive feedback. In addition to DDIM and the Euler method, we evaluated our approach with second-order solvers, namely DPM‑Solver‑2 and 2nd-order linear multistep method. Our results show that our method is not restricted to first-order solvers. THG preserves sample quality more effectively while providing comparable efficiency gains across different integrator orders.\n\n| Variant | NFE | FID ↓    | CMMD ↓   | CS ↑     | IR ↑     |\n|-----|-----|-----|-----|------|------|\n| DPM-Solver-2  | 100 | 13.255   | 0.60379  | 26.254   | 0.16148  |\n| + CFG-cache w/o FFT  | 70  | 13.387   | **0.60665**  | 26.107   | 0.10513  |\n| + CFG-cache   | 70  | 13.468   | 0.60880  | 26.147   | 0.11474  |\n| + THG  | 70  | **12.909**   | 0.60868  | **26.205**   | **0.14926**  |\n| | | | | | |\n| 2nd-order Linear Multistep Method  | 100 | 13.540   | 0.60653  | 26.260   | 0.15966  |\n| + CFG-cache w/o FFT | 70  | **13.686**   | **0.60844**  | 26.107   | 0.09881  |\n| + CFG-cache | 70  | 13.798   | 0.61142  | 26.144   | 0.10805  |\n| + THG | 70  | **13.686**   | 0.61094  | **26.204**   | **0.15184**  |\n\n**References**\n\n[1] Lin et al., \"Common Diffusion Noise Schedules and Sample Steps are Flawed,\" arXiv preprint arXiv:2305.08891, 2023.\n\n[2] Chung et al., \"CFG++: Manifold-constrained Classifier Free Guidance for Diffusion Models,\" ICLR 2025.\n\n[3] Podell et al., \"SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis,\" arXiv preprint arXiv:2307.01952, 2023.\n\n[4] Esser et al., \"Scaling Rectified Flow Transformers for High-Resolution Image Synthesis,\" arXiv preprint arXiv:2403.03206, 2024.\n\n[5] Liu et al., \"AudioLDM 2: Learning Holistic Audio Generation with Self-supervised Pretraining,\" arXiv preprint arXiv:2308.05734, 2023.\n\n[6] Kim et al., \"Audiocaps: Generating captions for audios in the wild,\" NAACL-HLT 2019.\n\n[7] Kilgour et al., \"Fréchet Audio Distance: A Metric for Evaluating Music Enhancement Algorithms,\" arXiv preprint arXiv:1812.08466, 2018.\n\n[8] Elizalde et al., \"CLAP: Learning Audio Concepts From Natural Language Supervision\", arXiv preprint arXiv:2206.04769, 2022.\n\n[9] Adrian Sandu, \"Multirate time integration: an overview\", Aug 2023. Presentation at the 2023 Los Alamos Workshop on Time Integration for Multiphysics."}, {"review_id": "MPZZS1J3ON", "reviewer": "Reviewer_M4bX", "summary": "The paper presents a method called \"Tortoise and Hare Guidance\" which accelerates the diffusion sampling under CFG with a coarser timestep grid. It took a principled approach by analyzing the ODEs and developed a method based on theoretical analysis. The proposed method gave a 30% reduction in NFE while maintaining sample quality with SD on Coco dataset.", "questions": "See the (-) points in the strength and weakness section. It also contains questions", "limitations": "n/a", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "(+) The paper took a first-principled numerical-analysis-inspired approach to improving the CFG schedule. The novelty is good in my opinion.\n\n(+) The paper is training free. It can be plugged into existing trained models with minimal changes. \n\n(+) The experimental results are relatively strong in my opinion for the method it tested and the dataset it tested on\n\n(+) The writing and the flow of the paper is good\n\n(-) Only SD series of models are tested, and it only works for 1st order solvers?\n\n(-) Only tested on a limited datasets such as COCO. Not sure how well it generalizes to other datasets / models, and how sensitive the hyperparameters are across datasets", "was_revised": false, "final_justification": "", "rebuttal": "Dear reviewer M4bX,\n\nThank you for your valuable feedback and comments. We are grateful for your appreciation of our numerical-analysis-inspired approach on the denoising process. We hereby answer your concerns and questions. \n\n> (-) Only SD series of models are tested, and it only works for 1st order solvers?\n\n[A1] Thank you for sharing your concerns. In addition to DDIM and the Euler method, we evaluated our approach with second-order solvers, namely DPM‑Solver‑2 and 2nd-order linear multistep method. Our results show that our method is not restricted to first-order solvers, and that THG preserves sample quality more effectively while providing comparable efficiency gains across different integrator orders. Furthermore, experiments on audio diffusion models in [A2] demonstrate that our approach extends beyond the SD model family.\n\n| Variant                                       | NFE | FID ↓    | CMMD ↓   | CS ↑     | IR ↑     |\n|-----------------------------------------------|-----|----------|----------|----------|----------|\n| DPM-Solver-2                                  | 100 | 13.255   | 0.60379  | 26.254   | 0.16148  |\n| + CFG-cache w/o FFT                           | 70  | 13.387   | **0.60665**  | 26.107   | 0.10513  |\n| + CFG-cache                                   | 70  | 13.468   | 0.60880  | 26.147   | 0.11474  |\n| + THG                                         | 70  | **12.909**   | 0.60868  | **26.205**   | **0.14926**  |\n| | | | | | |\n| 2nd-order Linear Multistep Method             | 100 | 13.540   | 0.60653  | 26.260   | 0.15966  |\n| + CFG-cache w/o FFT                           | 70  | **13.686**   | **0.60844**  | 26.107   | 0.09881  |\n| + CFG-cache                                   | 70  | 13.798   | 0.61142  | 26.144   | 0.10805  |\n| + THG                                         | 70  | **13.686**   | 0.61094  | **26.204**   | **0.15184**  |\n\n---\n\n> (-) Only tested on a limited datasets such as COCO. Not sure how well it generalizes to other datasets / models, and how sensitive the hyperparameters are across datasets\n\n[A2] Thank you for raising an important point. Exclusive use of the COCO dataset is still the standard for benchmarking T2I diffusion models [1-3] and ensures fair comparison with previous work. Nevertheless, we agree that validating across several benchmarks is important. Consequently, we conducted preliminary audio generation experiments with AudioLDM2 [4]. We used the validation set of AudioCaps [5] as the audio-text pair dataset and measured the performance with FAD [6] and CLAP Score [7], each corresponding to FID and CLIP Score. (The three parameters of THG corresponds to $(\\rho, b, t_{hi})$.)\n\n| Variant | NFE | FAD ↓ | CLAP Score ↑ |\n|---|---|---|---|\n| CFG (baseline) | 100 | 2.596 | 0.2409 |\n| | | | |\n| CFG-Cache w/o FFT | 72 | 2.777 | 0.2312 |\n| THG (0.9, 1.15, 41) | 72 | **2.732** | **0.2362** |\n| | | | |\n| CFG-Cache w/o FFT | 70 | 2.901 | 0.2251 |\n| THG (0.9, 1.15, 39) | 70 | **2.764** | **0.2342** |\n| | | | |\n| CFG-Cache w/o FFT | 68 | 3.503 | 0.1972 |\n| THG (0.95, 1.15, 38) | 68 | **2.803** | **0.2328** |\n\nWe will present these results in the appendix. While we adjusted the hyperparameters to obtain the same NFE, similar values can be used across different models and datasets. The consistent superior performance of our method underscores its broad applicability.\n\n**References**\n\n[1] Chung et al., \"CFG++: Manifold-constrained Classifier Free Guidance for Diffusion Models,\" ICLR 2025.\n\n[2] Podell et al., \"SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis,\" arXiv preprint arXiv:2307.01952, 2023.\n\n[3] Esser et al., \"Scaling Rectified Flow Transformers for High-Resolution Image Synthesis,\" arXiv preprint arXiv:2403.03206, 2024.\n\n[4] Liu et al., \"AudioLDM 2: Learning Holistic Audio Generation with Self-supervised Pretraining,\" arXiv preprint arXiv:2308.05734, 2023.\n\n[5] Kim et al., \"Audiocaps: Generating captions for audios in the wild,\" NAACL-HLT 2019.\n\n[6] Kilgour et al., \"Fréchet Audio Distance: A Metric for Evaluating Music Enhancement Algorithms,\" arXiv preprint arXiv:1812.08466, 2018.\n\n[7] Elizalde et al., \"CLAP: Learning Audio Concepts From Natural Language Supervision\", arXiv preprint arXiv:2206.04769, 2022."}, {"review_id": "m86NEdAATP", "reviewer": "Reviewer_huS5", "summary": "This paper introduces Tortoise and Hare Guidance (THG), a training-free method to accelerate CFG sampling by reformulating it as a multirate system of ODEs. The key insight is that the noise estimate and additional guidance term have different sensitivities to numerical error. This allows the guidance term to be computed on a coarser timestep grid while maintaining the noise estimate on a fine grid. The method includes an error-bound analysis to determine appropriate coarse grid spacing and achieves up to 30% reduction in NFEs while preserving generation quality on text-to-image generation.", "questions": "Algorithm 3 appears to compute the coarse grid for each example individually, which seems correct from an accuracy perspective. However, I'm curious about the variance in coarse time grids across different initial noise samples and text prompts. How much do these grids differ? For practical efficiency, would it be possible to compute coarse grids in a batch-wise manner across multiple text prompts?\n\n\nAs I found this work interesting, If the authors could provide sufficiently convincing explanations or additional experiments that address the experimental concerns I raised, I would be willing to increase my score.", "limitations": "yes.", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strength**\n\nThe ODE formulation of CFG and decomposition of CFG into separate tortoise and hare equations are quite novel. Especially, the decomposition of CFG based on their different sensitivities seems well-supported by the empirical evidence shown in Figure 2. The idea of treating these components differently is intuitive and their algorithms are mathematically sound. The error-bound analysis makes their approach more grounded approach.\n\n**Weakness**\n1. If I understand correctly, Algorithm 3 is used to obtain the coarse grid before running Algorithm 1. This preprocessing step likely requires additional NFEs that should be included when comparing computational efficiency. It would be more fair to account for these extra NFEs in the overall NFE comparison.\n\n2. The paper only evaluates performance at specific NFE values. It would be valuable to see performance curves across a wider range of NFE values, particularly at more limited NFE settings, to better understand when and how much the method helps.\n\n3. The experiments are limited to first-order solvers (DDIM, Euler). Since higher-order solvers like DPM-Solver are commonly used in practice, demonstrating the effectiveness on these methods would strengthen the paper's impact.\n\n4. Given that four different evaluation metrics are used (FID, CMMD, CS, IR), it would be helpful to include brief descriptions of these metrics in the experimental section or appendix for readers who may not be familiar with all of them.\n\n\nminor:\nThe notation for the difference estimation could be clearer to avoid confusion, as the current notation might be interpreted as a product of delta and epsilon.", "was_revised": true, "final_justification": "While the experimental validation appears somewhat limited, the paper proposes a novel approach and the rebuttal has largely addressed my concerns.", "rebuttal": "Dear reviewer huS5,\n\nThank you for the constructive feedback and insightful questions. Your comments on the novelty and soundness of THG are encouraging. Below, we address your concerns.\n\n> 1. If I understand correctly, Algorithm 3 is used to obtain the coarse grid before running Algorithm 1. This preprocessing step likely requires additional NFEs that should be included when comparing computational efficiency. It would be more fair to account for these extra NFEs in the overall NFE comparison.\n\n[A1] We sincerely thank the reviewer for pointing out the potential ambiguity in our description of Algorithm 3 and apologize for any confusion this may have caused. In the final manuscript, we will clarify that:\n\n1. **One-time preprocessing** : Algorithm 3 is executed *once* offline to compute that coarse grid $C$, incurring an $O(1)$ cost that is negligible compared to the $O(N)$ cost of processing $N$ requests.\n2. **Reusable grid** : Once computed, $C$ can be reused for all subsequent inferences without any additional NFEs as long as the model and the domain is not changed. \n3. **Flexible sampling**: It is also possible to compute $C$ using fewer sample runs. To demonstrate this, we estimated $C$ using a batch of 1,000 samples and found that it closely matched our original, large-scale estimate with 30,000 samples as the following. Compared to the original estimate, the 1,000 sample estimate demonstrated 95.25\\% IoU (i.e., Jaccard index) in average over 30 trials. \n\n| Samples     | IoU | Steps |\n|-------------|-----|----------------------------------------------------------------------------------------------------------------------------------------------|\n| 30k (original) | - | 0, 1, 2, 3, 4, 5, 6, 8, 10, 12, 14, 17, 20, 23, 26, 28, 30, 32, 34, 36, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50 |\n| | | |\n| 1k (trial 1)      | 100\\% | 0, 1, 2, 3, 4, 5, 6, 8, 10, 12, 14, 17, 20, 23, 26, 28, 30, 32, 34, 36, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50 |\n| 1k (trial 2)      | 100\\% | 0, 1, 2, 3, 4, 5, 6, 8, 10, 12, 14, 17, 20, 23, 26, 28, 30, 32, 34, 36, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50 |\n| 1k (trial 3)      | 100\\% | 0, 1, 2, 3, 4, 5, 6, 8, 10, 12, 14, 17, 20, 23, 26, 28, 30, 32, 34, 36, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50 |\n| ... | ... | ... |\n| 1k (trial 8)      | 57.5\\% |0, 1, 2, 3, 4, 5, 7, 9, 11, 13, 16, 19, 22, 25, 28, 30, 32, 34, 36, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50 |\n| ... | ... | ... |\n| 1k (trial 30)      | 100\\% | 0, 1, 2, 3, 4, 5, 6, 8, 10, 12, 14, 17, 20, 23, 26, 28, 30, 32, 34, 36, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50 |\n\n---\n\n> 2. The paper only evaluates performance at specific NFE values. It would be valuable to see performance curves across a wider range of NFE values, particularly at more limited NFE settings, to better understand when and how much the method helps.\n\n[A2] Thank you for the constructive advice. Below, we report our method’s performance evaluated at various NFE values. Since the design of CFG-Cache requires a minimum NFE of 68, for fair comparison we report results with NFE ≥ 68. In the final manuscript, we will include comprehensive performance curves plotting key quality metrics (e.g., FID, CLIP score) across a wide spectrum of more limited NFE settings. (The three parameters of THG corresponds to $(\\rho, b, t_{hi})$.)\n\n| Variant                  | NFE | FID ↓   | CMMD ↓   | CS ↑    | IR ↑     |\n|--------------------------|-----|---------|----------|---------|----------|\n| CFG baseline             | 100 | 14.133  | 0.58948  | 26.295  | 0.14764  |\n| | | | | | |\n| CFG-Cache w/o FFT        | 74  |  **14.145**  | 0.59163  | 26.238  | 0.11815  |\n| CFG-Cache                | 74  | 14.246  | 0.59497  | **26.272**  | **0.12979**  |\n| THG (1.0, 1.1, 39)       | 74  | 14.156  | **0.59068**  | 26.213  | 0.12260  |\n| | | | | | |\n| CFG-Cache w/o FFT        | 72  | **14.190** | **0.59282** | 26.206 | 0.10623 |\n| CFG-Cache                | 72  | 14.326 | 0.59604 | **26.240** | 0.11758 |\n| THG (1.05, 1.1, 39)      | 72  | 14.220 | **0.59282** | 26.213 | **0.11901** | \n| | | | | | |\n| CFG-Cache w/o FFT        | 70  | 14.282  | **0.59354**  | 26.138  | 0.08697  |\n| CFG-Cache                | 70  | 14.422  | 0.59759  | 26.179  | 0.09705  |\n| THG (1.1, 1.1, 38)       | 70  | **14.232**  | **0.59354**  | **26.197**  | **0.11576**  |\n| | | | | | |\n| CFG-Cache w/o FFT        | 68  | 14.418  | 0.60427  | 25.843  | –0.02967 |\n| CFG-Cache                | 68  | 14.678  | 0.61190  | 25.874  | –0.02859 |\n| THG (1.2, 1.1, 36)       | 68  | **14.303**  | **0.59258**  | **26.187**  | **0.10465**  |\n\n---\n\n> 3. The experiments are limited to first-order solvers (DDIM, Euler). Since higher-order solvers like DPM-Solver are commonly used in practice, demonstrating the effectiveness on these methods would strengthen the paper's impact.\n\n[A3] We appreciate the reviewer for highlighting this point. We evaluated our approach with second-order solvers, namely DPM‑Solver‑2 and 2nd-order linear multistep method. Empirical results confirm that our method retains sample fidelity more effectively while offering similar efficiency improvements across different solver orders.\n\n| Variant                                       | NFE | FID ↓    | CMMD ↓   | CS ↑     | IR ↑     |\n|-----------------------------------------------|-----|----------|----------|----------|----------|\n| DPM-Solver-2                                  | 100 | 13.255   | 0.60379  | 26.254   | 0.16148  |\n| + CFG-cache w/o FFT                           | 70  | 13.387   | **0.60665**  | 26.107   | 0.10513  |\n| + CFG-cache                                   | 70  | 13.468   | 0.60880  | 26.147   | 0.11474  |\n| + THG                                         | 70  | **12.909**   | 0.60868  | **26.205**   | **0.14926**  |\n| | | | | | |\n| 2nd-order Linear Multistep Method             | 100 | 13.540   | 0.60653  | 26.260   | 0.15966  |\n| + CFG-cache w/o FFT                           | 70  | **13.686**   | **0.60844**  | 26.107   | 0.09881  |\n| + CFG-cache                                   | 70  | 13.798   | 0.61142  | 26.144   | 0.10805  |\n| + THG                                         | 70  | **13.686**   | 0.61094  | **26.204**   | **0.15184**  |\n\n---\n\n> 4. Given that four different evaluation metrics are used (FID, CMMD, CS, IR), it would be helpful to include brief descriptions of these metrics in the experimental section or appendix for readers who may not be familiar with all of them.\n\n[A4] We thank the reviewer for pointing this out. To assist readers unfamiliar with these metrics, we will add concise definitions of FID, CMMD, CS, and IR in the final manuscript. For example: \n\n- **Metric**: To evaluate the methods, we use the following metrics: FID (Fréchet Inception Distance) compares the Inception‑v3 feature distributions of real and generated images; CMMD (CLIP Maximum Mean Discrepancy) quantifies the discrepancy between real and generated image distributions in the CLIP embedding space; CS (CLIP Score) is the average cosine similarity between each generated image and its corresponding text prompt in the joint CLIP embedding space; IR (ImageReward) denotes the mean reward assigned by a pretrained image‑text reward model to each generated sample.\n\n---\n\n> minor: The notation for the difference estimation could be clearer to avoid confusion, as the current notation might be interpreted as a product of delta and epsilon.\n\n[A5] Thank you for your suggestion. While we have followed previous work [1] for the notation of $\\delta \\hat{\\epsilon}\\_c$, we will use $\\Delta \\hat{\\epsilon}\\_c$ or $\\hat{\\epsilon}\\_{\\delta}$ in the final manuscript to make the paper more understandable.\n\n---\n\n> Algorithm 3 appears to compute the coarse grid for each example individually, which seems correct from an accuracy perspective. However, I'm curious about the variance in coarse time grids across different initial noise samples and text prompts. How much do these grids differ? For practical efficiency, would it be possible to compute coarse grids in a batch-wise manner across multiple text prompts?\n\n[A6] Thank you for this insightful question. It is certainly possible to compute the coarse grid in a batch-wise manner, and that is how we compute $C$ once on the dataset and utilize it afterwards in the paper. As shown in the table of [A1], we can estimate $C$ using a batch of 1,000 samples. This is accurate enough compared with the estimate on 30,000 samples, since the approximation error has low variance across different initial noise draws and text prompts (Fig. 3 of our paper). We will clarify that we compute $C$ in a batch-wise manner and use it for all samples in our final manuscript.\n\n**References**\n\n[1] Chung et al., \"CFG++: Manifold-constrained Classifier Free Guidance for Diffusion Models,\" arXiv preprint arXiv:2406.08070, 2024."}, {"review_id": "xovwOJMVVl", "reviewer": "Reviewer_FF6M", "summary": "This paper introduces Tortoise and Hare Guidance (THG), a training-free sampling acceleration method for diffusion model. THG reformulates classifier-free guidance (CFG) by utilizing the asymmetric sensitivity of the noise estimate (tortoise branch) and the additional guidance term (hare branch) to numerical error, reducing the number of function evaluations (NFE) by up to 30% with small loss in sample quality. THG achieves good performance on Stable Diffusion 1.5 and 3.5 Large.", "questions": "- Though choosing the coarse time schedule based on error bound analysis is convincing, it takes considerable efforts to estimate the error constants. How is the performance gain of this carefully designed schedule? Could a naive choice also provide good performance? A naive choice of the coarse time schedule can also make the use of higher-order solvers reasonable.\n- It could be better to include the latency comparison with existing method to show the superiority of the proposed method.", "limitations": "Yes.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths:\n- The paper is well-organized and the story is interesting. The proposed method is built upon multirate formulation with error bound analyzed for choosing proper coarse time schedule, which is novel.\n- THG achieves considerable acceleration without training, making it applicable to existing diffusion pipelines.\n- Experiments on Stable Diffusion models demonstrate competitive performance in terms of FID, CLIP Score, and ImageReward compared to the baseline.\n\nWeaknesses:\n- This paper only includes CFG-cache for comparison. However, there are a series of works [1, 2, 3], which are also mentioned in Section 2, focusing on accelerating CFG sampling but are not quantitatively compared. \n- Gradually increasing the guidance scale by $b$ is kind of wired, as it breaks the fair comparison with baseline where the guidance scale is fixed.\n- The experiment could be conducted more thoroughly. Currently, each model only has one choice of NFE. Including more choices like 20 or 10 steps could further enrich the results and demonstrate the superiority of THG.\n\nReferences\n\n[1] Angela Castillo, Jonas Kohler, Juan C Pérez, Juan Pablo Pérez, Albert Pumarola, Bernard Ghanem, Pablo Arbeláez, and Ali Thabet. Adaptive guidance: Training-free acceleration of conditional diffusion models. arXiv preprint arXiv:2312.12487, 2023.\n\n[2] Pareesa Ameneh Golnari, Zhewei Yao, and Yuxiong He. Selective guidance: Are all the denoising steps of guided diffusion important? arXiv preprint arXiv:2305.09847, 2023.\n\n[3] T. Kynkäänniemi, M. Aittala, T. Karras, S. Laine, T. Aila, and J. Lehtinen. Applying guidance in a limited interval improves sample and distribution quality in diffusion models. In Proc. NeurIPS, 2024.", "was_revised": true, "final_justification": "Most of my concerns are addressed by these new quantative results. I decide to increase my rating.", "rebuttal": "Dear reviewer FF6M,\n\nThank you for your valuable feedback and constructive comments. We appreciate your attention to the novelty and applicability of THG. We hereby address your concerns and questions.\n\n> * This paper only includes CFG-cache for comparison. However, there are a series of works [1, 2, 3], which are also mentioned in Section 2, focusing on accelerating CFG sampling but are not quantitatively compared.\n\n[A1] We sincerely thank the reviewer for this suggestion. In response, we extended our experimental comparison to include the acceleration techniques proposed in [2, 3]. This demonstrates that our approach achieves superior image quality with the same NFE budget. While [2] shows lower FID and CMMD values, they generate images with low prompt fidelity and degraded details indicated by lower CS and IR values. Please refer to Appendix B of the paper for further discussion of this phenomenon.\n\nOn the other hand, we noticed that [1] is not applicable to our scenario. As observed in [4], the core assumption of monotonically increasing cosine similarity does not hold on most open-source diffusion model checkpoints. \n\n(The three parameters of THG corresponds to $(\\rho, b, t_{hi})$.)\n\n| Variant                              | NFE | FID ↓   | CMMD ↓   | CS ↑     | IR ↑      |\n|--------------------------------------|-----|---------|----------|----------|-----------|\n| Selective Guidance [2]               | 74  | **13.394**  | **0.57148**  | 25.824   | 0.00972   |\n| Guidance Interval $(i>8, \\omega=16)$ [3]  | 74  | 14.762  | 0.63085  | **26.261**   | **0.13308**   |\n| THG (1.0, 1.1, 39)                   | 74  | 14.156  | 0.59068  | 26.213   | 0.12260   |\n| | | | | | |\n| Selective Guidance [2]               | 70  | **12.895**  | **0.56052**  | 25.602   | -0.06691  |\n| Guidance Interval $(i>8, \\omega=16)$ [3]  | 70  | 14.555  | 0.62227  | 26.153   | 0.09658   |\n| THG (1.1, 1.1, 38)                   | 70  | 14.232  | 0.59354  | **26.197**   | **0.11576**   |\n| | | | | | |\n| Selective Guidance [2]               | 66  | **12.180**  | **0.55062**  | 25.299   | -0.17810  |\n| Guidance Interval $(i>8, \\omega=16)$ [3]  | 66  | 14.152  | 0.60939  | 25.998   | 0.04062   |\n| THG (1.3, 1.1, 35)                   | 66  | 14.243  | 0.59437  | **26.163**   | **0.09979**   |\n\n---\n\n> * Gradually increasing the guidance scale by $b$ is kind of weird, as it breaks the fair comparison with baseline where the guidance scale is fixed.\n\n[A2] We thank the reviewer for this perceptive comment. We agree that using $b>1$ can appear to compromise a fair comparison with a fixed‑scale baseline. To address this, our ablation study (Tab. 2 of our paper) includes the $b=1$ case, which yields comparable CS and IR alongside superior FID and CMMD relative to other settings. Notably, even with a constant guidance scale $(b=1)$, our method continues to deliver strong performance, underscoring its effectiveness without any adjustment to the guidance weight.\n\n| Method                         | NFE ↓ | FID ↓   | CMMD ↓   | CS ↑    | IR ↑     |\n|---------------------|-------|---------|----------|---------|----------|\n| CFG (baseline)      | 100   | 14.057  | 0.58885  | 26.294  | 0.14765  |\n| | | | | | |\n| CFG-Cache w/o FFT   | 70    | 14.240  | 0.59187  | 26.141  | 0.08757  |\n| CFG-Cache           | 70    | 14.367  | 0.59556  | 26.180  | 0.09735  |\n| THG ($b=1.0$)       | 70    | **13.811**  | **0.58364**  | 26.137  | 0.09395  |\n| THG ($b=1.1$)       | 70    | 14.232  | 0.59354  | **26.197**  | **0.11576**  |\n\n\n---\n\n> * The experiment could be conducted more thoroughly. Currently, each model only has one choice of NFE. Including more choices like 20 or 10 steps could further enrich the results and demonstrate the superiority of THG.\n\n[A3] We sincerely thank the reviewer for this valuable advice. We evaluated our method across multiple NFE settings to provide a more comprehensive comparison. Since the design of CFG-Cache requires a minimum NFE of 68, for fair comparison we report results with NFE ≥ 68. THG continues to give superior results, standing out in low NFE values. We will include additional results with more limited NFE settings in the final manuscript to clearly demonstrate the robustness and superiority of our approach.\n\n| Variant                  | NFE | FID ↓   | CMMD ↓   | CS ↑    | IR ↑     |\n|--------------------------|-----|---------|----------|---------|----------|\n| CFG baseline             | 100 | 14.133  | 0.58948  | 26.295  | 0.14764  |\n| | | | | | |\n| CFG-Cache w/o FFT        | 74  |  **14.145**  | 0.59163  | 26.238  | 0.11815  |\n| CFG-Cache                | 74  | 14.246  | 0.59497  | **26.272**  | **0.12979**  |\n| THG (1.0, 1.1, 39)       | 74  | 14.156  | **0.59068**  | 26.213  | 0.12260  |\n| | | | | | |\n| CFG-Cache w/o FFT        | 72  | **14.190** | **0.59282** | 26.206 | 0.10623 |\n| CFG-Cache                | 72  | 14.326 | 0.59604 | **26.240** | 0.11758 |\n| THG (1.05, 1.1, 39)      | 72  | 14.220 | **0.59282** | 26.213 | **0.11901** | \n| | | | | | |\n| CFG-Cache w/o FFT        | 70  | 14.282  | **0.59354**  | 26.138  | 0.08697  |\n| CFG-Cache                | 70  | 14.422  | 0.59759  | 26.179  | 0.09705  |\n| THG (1.1, 1.1, 38)       | 70  | **14.232**  | **0.59354**  | **26.197**  | **0.11576**  |\n| | | | | | |\n| CFG-Cache w/o FFT        | 68  | 14.418  | 0.60427  | 25.843  | –0.02967 |\n| CFG-Cache                | 68  | 14.678  | 0.61190  | 25.874  | –0.02859 |\n| THG (1.2, 1.1, 36)       | 68  | **14.303**  | **0.59258**  | **26.187**  | **0.10465**  |\n\n---\n\n> * Though choosing the coarse time schedule based on error bound analysis is convincing, it takes considerable efforts to estimate the error constants. How is the performance gain of this carefully designed schedule? Could a naive choice also provide good performance? A naive choice of the coarse time schedule can also make the use of higher-order solvers reasonable.\n\n[A4] Thank you for your constructive suggestion. We evaluated THG with a couple of naive choices of the coarse grid $C$, namely a random one and an evenly spaced one. The results show the superiority of a carefully designed $C$.\n\n| Variant                | NFE | FID ↓   | CMMD ↓   | CS ↑    | IR ↑     |\n|------------------------|-----|---------|----------|---------|----------|\n| Random                 | 70  | 14.662  | 0.59890  | 26.167  | 0.07561  |\n| Evenly spaced          | 70  | 14.300  | **0.59068**  | 26.101  | 0.09282  |\n| $\\rho=1.1$             | 70  | **14.232**  | 0.59354  | **26.197**  | **0.11576**  |\n\n---\n\n> * It could be better to include the latency comparison with existing method to show the superiority of the proposed method.\n\n[A5] We thank the reviewer for this suggestion. We measured end‑to‑end inference latency of image generation on a single NVIDIA A100 GPU using bfloat16 precision. As expected, latency scales almost linearly with NFE since neural network evaluation dominates runtime. These results confirm that THG delivers improved image fidelity without sacrificing inference speed.\n\n| Variant                  | NFE | Latency (ms) |\n|--------------------------|-----|------------|\n| CFG baseline             | 100 |  746 |\n| | | | | | |\n| CFG-Cache w/o FFT        | 74  |  564 |\n| CFG-Cache                | 74  |  565 |\n| THG (1.0, 1.1, 39)       | 74  |  564 |\n| | | | | | |\n| CFG-Cache w/o FFT        | 72  |  550 |\n| CFG-Cache                | 72  |  551 |\n| THG (1.05, 1.1, 39)      | 72  |  550 |\n| | | | | | |\n| CFG-Cache w/o FFT        | 70  |  535 |\n| CFG-Cache                | 70  |  538 |\n| THG (1.1, 1.1, 38)       | 70  |  537 |\n| | | | | | |\n| CFG-Cache w/o FFT        | 68  |  522 |\n| CFG-Cache                | 68  |  522 |\n| THG (1.2, 1.1, 36)       | 68  |  521 |\n\n**References**\n\n[1] Castillo et al., \"Adaptive guidance: Training-free acceleration of conditional diffusion models,\" arXiv preprint arXiv:2312.12487, 2023.\n\n[2] Golnari et al., \"Selective guidance: Are all the denoising steps of guided diffusion important?,\" arXiv preprint arXiv:2305.09847, 2023.\n\n[3] Kynkäänniemi et al., \"Applying guidance in a limited interval improves sample and distribution quality in diffusion models,\" NeurIPS 2024.\n\n[4] Zhang et al., \"How Much To Guide: Revisiting Adaptive Guidance in Classifier-Free Guidance Text-to-Vision Diffusion Models,\" arXiv preprint arXiv:2506.08351, 2025."}]

Author rebuttal:

Meta-review: This paper introduces Tortoise and Hare Guidance (THG) which accelerates diffusion inference by splitting the diffusion ODE into two branches. Reviewers are satisfied with the writing and mathematical soundness of this paper but raised issues about insufficient quantitative results and ablation studies. Most of the concerns are addressed by further experimental results provided during rebuttal. In the end, the reviewers all recommend accept. With these results added into the revised version of the current submission, I think this work will meet the quality threshold of NeurIPS conference and hence recommend acceptance.
Decision: Accept (poster)