Error as Signal: Stiffness-Aware Diffusion Sampling via Embedded Runge-Kutta Guidance
Authors: Inho Kong
Sojin Lee
Youngjoon Hong
Hyunwoo J. Kim
Venue: ICLR 2026 Poster (2026)
Keywords: Diffusion models
Classifier-free guidance
Primary area: generative models

Abstract: Classifier-Free Guidance (CFG) has established the foundation for guidance mechanisms in diffusion models, showing that well-designed guidance proxies significantly improve conditional generation and sample quality. Autoguidance (AG) has extended this idea, but it relies on an auxiliary network and leaves solver-induced errors unaddressed. In stiff regions, the ODE trajectory changes sharply, where local truncation error (LTE) becomes a critical factor that deteriorates sample quality. Our key observation is that these errors align with the dominant eigenvector, motivating us to leverage the solver-induced error as a guidance signal. We propose **E**mbedded **R**unge–**K**utta **Guid**ance (ERK-Guid), which exploits detected stiffness to reduce LTE and stabilize sampling. We theoretically and empirically analyze stiffness and eigenvector estimators with solver errors to motivate the design of ERK-Guid. Our experiments on both synthetic datasets and the popular benchmark dataset, ImageNet, demonstrate that ERK-Guid consistently outperforms state-of-the-art methods. Code is available at https://github.com/mlvlab/ERK-Guid.

Published as a conference paper at ICLR 2026
ERROR AS SIGNAL: STIFFNESS-AWARE DIFFUSION SAMPLING VIA EMBEDDED RUNGE-KUTTA GUID- ANCE
Inho Kong"", Sojin Lee"r, Youngjoon Hong Hyunwoo J. Kim Korea University 2KAIST SSoull National University *Korea Institute for Advanced Study
inh212@korea.ac. kr {sojin. lee, hyurwoo jyurwwojkiinjkaistt.oom
hongyjesnu.ac.krt
ABSTRACT
Classifier-Free Guidance (CFG) has established the foundation for guidance mechanisms in diffusion models, showing that well-designed guidance proxies significantly improve conditional generation and sample quality. Autoguidance (AG) has extended this idea, but relies on an auxiliary network and leaves solver-induced errors unaddressed. In stiff regions, the ODE trajectory changes sharply, where local truncation error (LTE) becomes critical factor that dete- riorates sample quality. Our key observation is that these errors align with the dominant eigenvector, motivating us to leverage the solver-induced error as a guidance signal. We propose Embedded Runge-Kutta Guidance (ERK-Guid), which exploits detected stiffines to reduce LTE and stabilize sampling. We theo- retically and empirically analyze stiffness and eigenvector estimators with solver errors to motivate the design of ERK-Guid. Our experiments on both synthetic datasets and the popular benchmark dataset, ImageNet, demonstrate that ERK- Guid consistently outperforms state-of-the-art methods. Code is available at https:// Ehub. com m/lmlllvv ab/ -
INTRODUCTION
Generative models Kingma & Welling (2014); Rezende & Mohamed (2015): Heusel et al. (2017)
Lipman al. (2023) Ho al. 2020); Song al. Goodffelo al. (2020) aim approxi-
mate complex data distributions and generate new samples, enabling a wide range of applications in
image synthesis Brock
(2019) Rombach et (2022): ; Zhang et al. (2023); Kawar
(2023);
Park et al. al. 2025), editing Brooks et al. (2023). ), and video generation Gupta et al. (2024); Park Park al. (2024a). Among them, diffusion models Ho et al. (2020); [Song et al.) (202Tb): Karras et al. (2022) have emerged as a dominant paradigm, achieving strong performance across diverse gen- eration tasks Guppa et al. (2024): Karras et al. (2024a); Lee et al. (2024); Ko et al. (2024). They define forward process that gradually perturbs data into Gaussian noise, while neural network is trained to predict the score function of each noisy distribution. This score estimate parameterizes the teerse-time dynamics, allowing data to be reconstructed through iterative denoising. Sampling is commonly formulated as solving an ordinary differential equation (ODE) or stochastic differential equation (SDE), where the drift is defined by the learned network Karras et al.) (2022). [Song et al. (202Ta); | Lu et al. (2022). As a result, the quality of generated samples depends not only on model accuracy but also on the numerical solver used approximate the reverse dynamics, which can significantly influence fidelity and stability.
Guidance mechanisms emerged improve both sampling fidelity and perceptual quality by intro- ducing suitable proxies for steering the sampling trajectory. The de facto standard, Classifier- Free Guidance (CFG) Ho & Salimans (2022). combines unconditional and conditional predictions to
Equal contribution. Corresponning authors.

---

Published as a conference paper at ICLR 2026
strengthen alignment and enhance image quality. Predictto-Corrector Guidance (PCG) Bradley & Nakkiran 2024) further refines this view by interpreting CFG as a predictor predictor-correcto corrector update
that extrapolates between these predictions. Autoguidance (AG) Karras et al. 20224 follows similar principle by contrasting outputs from models of different capacities, using their discrepan- cies to identify regions where model-induced emros significant. However, subsequent meth- ods Kynkäänniemi et al. (2024); Sadat et al. (2024); Zheng & Lan| (2024); ZZhoo et al. (2025) rely solely on such model-based differences. overlooking the numerical errors arising from the solver it- self as potential guidance signals. We observe that, in stiff regions of the diffusion ODE, the solver's s local truncation error (LTE) aligns with the dominant eigenvector of the drift's 's Jacobian, revealing a numerically grounded proxy distinct from model-space signals and motivating our approach.
In this work, we propose Embedded Kunge-Kutta, Guidance (ERK-Guid), a novel approach that mitigates solver-induced local truncation error (LTE) during diffusion sampling by estimating the dominant eigenvector in stiff regions. In diffusion ODEs, stiffness arises when drift directions change rapidly, and we observe that the resulting LTE consistently aligns with the dominant eigen- vector under such conditions. To exploit this property, we introduce two cost-free estimators derived from the Embedded Runge-Kutee Kutta (ERK) formulation: ERK solution difference (between Heun and Euler solutions) and ERK drift difference (between their corresponding drifts). The ratio of their norms serves stiffness estimator identify regions of high stiffness, while the ERK drift difference further provides an eigenvector estimator to approximate the dominant eigenvector di- rection. ERK-Guid transforms theoretical insights on stiffness and dominant eigenvector alignment into a practical guidance mechanism. Ultimately, ERK-Guid provides a stable, cost-free proxy that effectively reduces solver-induced errors by applying guidance along this estimated eigenvector di- rection.
Our contributions are summarized follows:
We introduce the Embedded Runge- ance method that leverages solver errors as informative signals for diffusion sampling.
Runge-Kutta Kutta Guidance (ERK-Guid) stiffness- aware guid-
We propose cost-free estimators for stiffness detection and dominant eigenvector estima- tion, derived from ERK solution and drift differences to determine the guidance direction.
We design a stabilized guidance scheme that bridges theoretical insights with practical robustness, ensuring robustness without additional network evaluations.
We demonstrate that ERK- provides an orthogonal guidance signal, integrates into Kunge-Kutta-ben solvers as a plug-and-play module, and improves upon strong base- lines.
2 RELATED WORKS
ODE solvers diffusion models. A major research trend in diffusion models has been to accelerate sampling by improving ODE solvers that approximate the underlying probability flow dynamics more efficiently. Early work such as DDIM Song et al. (2021a) reinterprets the stochastic sampling process of DDPM Ho et al. (2020) as deterministic ODE trajectory, enabling significantly fewer sampling steps without retraining. Building on this view, PNDM [Lu et al. (2022) introduces a pseudo-numerical multistep method that generalizes DDIM beyond first-order updates. Subsequent studies further leverage the structure of diffusion dynamics. DEIS Zhang & Chen (2023) employs
exponential integrators reduce discretization error, DPM-Solver Lu et al. (2022) derives high- order solvers with coefficients designed to minimize local truncation error, and UniPC Zhao et al. (2023) unifies predictor predictor-corrector corrector schemes under single framework.
More recently. DPM-Solver-v3 Zheng et al. (2023) incorporates empirical model statistics into solver parameterization to jointly address discretization and model approximation errors. Bespoke solver approaches develop customized solver designs for fixed pre-trained model, encompassing methods that optimize time-step schedules Xue et al.] (2024) and solver parameters |Wang et al. (025): ; Shaul et al. (2024). In contrast to these approaches, which redesign or replace the ODE solver, ERK-Guid operates in a fundamentally different regime. We keep the solver fixed and in- 's own error: the discrepancy between low- and high-order solver updates. By using this discrepancy as a directional correction, ERK-Gut provides solver-aware guidance without modifying the solver's numerical structure.
stead leverage the solver's
2

---

Published as a conference paper at ICLR 2026
Adaptive guidance computation. In diffusion models, Classifier-Fre Freeeffee-FFeeeeeee Guidance (CFG) Ho & Sal- imans 2022) has become the de facto standard for improving fidelity and condition alignment by contrasting conditional and unconditional denoisers. Despite its success, CFG often suffers from overshoot, loss of diversity, and entangled fidelity-diversity trade-offs, limiting its flexibility across noise levels. Autoguidance (AG)|Karra et el. (2024)) improves robustness by replacing the uncon- ditional branch with a weaker model, correcting model-induced errors without sacrificing variation. Beyond these canonical approaches, several adaptive guidance strategies have been explored. Guid- ance Interval KKykkäännemi et al. (2024) activates guidance only at mid-range noise levels. Other works Sadat al. (2024): Zheng & Lan (2024) mitigate oversaturation under strong guidance, and Zheng
DyDiT Zhao et al. (2025) dynamically adjusts model capacity across timesteps. These advances demonstrate how CFG has shaped a broad family of model-based guidance mechanisms. contrast, ERK- Guid employs solver-driven proxy derived from ERK discrepancies, yielding an orthogonal guidance signal yet complementary to model-based guidance.
3 PRELIMINARIES
Denoising Diffusion Models. Denoising diffusion models Ho et al. (2020); Song et al. (2021b) 1)) Karras al. (2022) generate samples by simulating the reverse-time dynamics a predehned stochastic differential equation (SDE). The SDE gradually transforms the data distribution Pdata into a perturbed distribution p(x: or.. Following EDM2 Karras et al. (2024b), the perturbed dis- tribution is defined as the convolution of Pdata with Gaussian noise Kynkaänniemi et al. (2024), i.e.,
p(x; 0) = Piana (x) * (xx; 0, I2), where o E [0, m
The reverse-time SDE can be equivalently reformulated as ordinary differential equation (ODE) Song et al. (2021b), leading to a deterministic sampling process Xo ~ Pdata that solves the following initial value problem:
dx,
f Jma
dx do
do = f(xxoo) = -oVx, logp(xx:o) Xo = Xana
+
do,
(1)
where f(x;; σ)) denotes the drift function of the ODE, and X, refers to the trajectory of the sample as a function of the noise level σ.. The drift function is typically approximated by the learned model fo(x;; o), which is trained via score-macching objectives Song et al. (2021b). Note that the initial state Xo... can be approximately sampled from (O, o when 0max is sufficiently large.
In practice, the ODE cannot be solved analytically; numerical solvers employed. To enable numerical integration, EDM2 Karras et al. (2024b) discretizes the interval into a sequence of noise levels (oo...., where N is the total number of integration steps. Note that [o,, 0. denotes the i-th integration interval, with dg mma and ON = 0. A numerical solver is then applied to each interval to approximate the following integration:
X
Xo: +
f(xo:o)do.
The Buler method Hairer et al. (1993). widely used first-order solver, provides the following update and local truncation error (LTE):
Euler
Xo,, li((xo,
(3)
Lager
-oo411 - x = (hh2).
Fuler
(4)
where h o. 0:+1 > refers the step size. To reduce the local truncation error, higher-order solvers are commonly used. Heun's method Hairer et al. (1993), based on the trapezoidal rule Hairer et al. (1993), introduces correction to the Euler estimate and effectively incorporates implicit inte- gration. Its update and LTE are given as follows:
x oalll
Xo,
2 (f(x, (1xo,,o) o.) + fx
Ealer o141));
(5)
Heun LTE
X6.41 -
X
Houn = Ohh3)
(6)
This second-order Runge Kutta method serves as the default solver throughout work.
3

---

Published as a conference paper at ICLR 2026
Embedded Runge-Kutta pair. In Heun's method, the Euler solution is computed first and then corrected. Thus, we obtain two solutions of different orders (Euler of order 1 and Heun of order 2). This structure is referred to embedded Runge-Kutt Kungee--ttaa pair, and their solution difference A := Heeun Euler is commonly used as a proxy for the LITE Hairer et al. (993). In this paper, we refer Xai+1 di+t
to this difference A as the ERK solution difference. We also define the ERK drift difference as the difference of the drift evaluated at the two solutions, A == f 0s+1) Fuler, 0++1)-
Stiffness. Stiffness refers to the presence of both fast and slow dynamics within ODE sys- tem Hairer& & Wanner (1996). It is commonly encountered in physical simulations such as fluid dynamics. To ensure stability, numerical solvers must reduce their step sizes, leading to increased function evaluations and higher computational cost. To handle this, previous adaptive solvers Pet- zold (1983); [Shampine & Gear (1979) detect stiffness and dynamically adjust their integration be- havior by refining step sizes or switching to implicit solvers. Classically, stiffness is quantified by spectral properties of the Jacobian of the drift, such the ratio of the largest and smallest eigenvalue magnitudes or, more simply, the maximum eigenvalue in magnitude Hairer & Wanner [996):
[xx 0) == V xa fxxoi ()
Pstift (Xa, 0) max Ma(](xx (x))
(8)
where Xa (J) denotes the k-th eigenvalue of the matrix J. We denote by Vstift (Xa, o) a unit dominant eigenvector associated with Pstift
(atit((Xx000 (xa; 0).
4
METHOD
We start with theoretical analysis and empirical evidence to provide the insight that, in stiff ODEs, both the local truncation error (LTE) and the embedded Runge- Kutta (ERK) solution difference are aligned with the Jacobian's 's dominant eigenvector (Section 4.1) Based on this observation, we intro- duce cost-free estimators for stiffness and the dominant eigenvector (Section 4.2). and incorporate them into our guidance scheme, ERK-Guid (Section
4.1 ALIGNMENT OF LTE AND ERK SOLUTION DIFFERENCES IN STIFF ODES
Theoretical Insight. We assume that the score-based vector field of the diffusion model x, is well approximated by its local linearization around the current state Xo, when the step size h a; 0i+1 > is sufficiently small:
dx, = f(xoio) R f(xo.:o,) + J(xo,;0;) -xo,). do
(9)
Let Jo. and for denote J(xo,; o;) and f(xo; respectively. From Eq. the Jacobian is given by Jo. = -o, V2 log ())x;())lx. Assuming -a log p(x.; is Cm-smooth, its Hessian Jo. is symmetric and thus admits the following eigendecomposition:
Ja. = VAVT, s.t. V = J, Ja = AAVK:
Vk:
(10)
where V is the orthogonal matrix whose columns are the eigenvectors va of Ja., A is the diagonal matrix of the corresponding eigenvalues Λu. and / is the identity matrix. Therefore, the single-step Euler update can be decomposed along the eigenbasis as:
Fuler X
Xo: -hfo. = (For. v&) Vk:
(11)
where (1--) refers inner product. Similarly, Heun's update step is given by
Heun 0ee1
h Xx. = (fo, + sEnlory) ) 2
2/2 (fo. ff. + Jo, Euler -xo))
h
(12)
2
= -h Σ 1 $2x) (for, vu) Vk.
(13)
where fEuler Euler,
: x++1) and 2x -hdb. The exact update can be approximated as follows:
Xa,11 No. R hh Σ ((xx) (So,, vi) vas
(14)
4

---

Figure 1: Toy 2D example of stiffness and local drift variations. (a) Visualization of stiffness across the 2D plane, colored by magnitude. We sample non-stiff region (lines AB and CD) and stiff region (lines EF and GH), (b, e) In the non-stiff region, the drift projected onto the local eigen- basis exhibits minimal variation along both axes, indicating stable dynamics. (d, e) Conversely, in the stiff region, the projected drift shows pronounced variation along the dominant cigenvector axis (d), while remaining relatively parallel along the subdominant axis (c). Table gle step in the stiff region, both the local truncation error (LTE) and the ERK solution difference are Published as a conference paper at ICLR 2026
pig ection 3X55
projected drftt drecsos
Table I: Projection of lo- cal truncation error (LTE) and ERK solution differ- ence onto eigenvector axes.
104
(b) line AB
(c) line CD
ITF.
FRK solution diff
s
7.22e-45
4.03c-04
GH
5.78c-06
6.18c-05
Ratio 12.5x
6.5x
(a) Stiffness
(d) line EF
(c) line GH
that for a sin-
heavily amplified along the dominant axis (EF) due these drift variations. This strong alignment validates the ERK difference as practical proxy for the severe LTE in our proposed ERK-Guid.
where the function (0=) = 1 for + O and (0)) = 1. See Appendix A. for the derivation.
Thus, by subtracting Eq. 13 from Eq. 14 and Eq. TT from Eq. [3 respectively, the local truncation error of Heun's 's method and the ERK solution difference can be written as:
LTE Heun
X
Heun x01+1
h Σ (fx)) va) Vk
(15)
x
Houn
x
Euler
h Σ
(2x)fo, v&) Vk
(16)
k
where a(2) g--l 1 = for and (00) = O. Figure 6a visualizes the behavior of a(2).
As | = |h increases, the weights 1 2 and a<2x)) associated with each eigenvector component also grow in magnitude, so that contributions from directions with large |Axl come to dominate. Consequently, both the local truncation error (LTE) and the ERK solution difference tend to align with the dominant eigenvector corresponding to the largest eigenvalue magnitude, specifically in stiff regions, i.e., regions with high stiffness. Motivated by this observation, we estimate both stiffness and the dominant eigenvector from the ERK solution difference during sampling (Section 4.2). and when stiffness is sufficiently high, we apply guidance with Eq. 15| (Section 4.3).
Toy 2D Experiments. To better understand the connection between ODE dynamics and stiffiness, we construct a two-dimensional toy system with an analytically defined ground-truth drift, followed by Autoguidance Karras et al. (024a)). Experimental details, including the computation of eigen- vectors, eigenvalues, and LTE, are provided in Appendix B Figure aa) visualizes the degree of stiffness across the 2D plane. To examine the local drift behavior, we sample a non-stiff region (lines AB and CD) and a stiff region (lines EF and GH). For each region, we compute the Jacobian its center and project the drift field onto its eigenvectors (eigenbasis). as shown panels (b)-(e). In the non-stiff region (panels (b) and (c)), the projected drift exhibits minimal variation along both axes, indicating locally stable dynamics that numerical solvers can easily approximate. Conversely, in the stiff region, the drift shows pronounced variations along the dominant eigenvector axis (panel (d), line EF), while remaining relatively parallel along the subdominant axis (panel (e), line GH). Consequently, this variation causes ODE solvers to incur significant local truncation errors (LTE) predominantly along the dominant eigenvector direction. To verify this, we evaluate a single-step LTE using the center of the stiff region as the initial point. As presented in Table I the LTE projected onto the dominant eigenvector direction (line EF, 7.22c-05) is approximately 12.5x larger than that along the subdominant direction (line GH, 5.78e-06).
5

---

Published as a conference paper at ICLR 2026
Coare Siiterty
Siil
CoaRse amiaeit 6.
6.0
-24
0931
025-00 093-15
075-100
(4) LTE
(b) ERK solution difference
(c) ERK drift difference across stiffines quantiles
Figure 2: Eigenvector alignment across stiffness. (a) Cosine similarity between the dominant eigenvector and the local truncation error (LTE) increases with stiffness. (b) The ERK solution difference exhibits similar trend to LTE, suggesting it serve as reliable proxy for the LTE direction in high stiffinss regions. (c) Our ERK-Guid consistently achieves higher cosine similarity with the eigenvector, highlighting its strong alignment in stiff regions. CFG and Autoguidance exhibit weaker or mixed alignment with the dominant eigenvector in stiff regions, supporting the complementarity of our method.
To mitigate the severe LTE occurring predominantly along the dominant eigenvector, we need practical proxy to estimate this error direction. For this, we introduce the ERK solution difference, which measures the gap between a higher-order and lower-order solver starting from the same state. Because higher-order solver applies much larger corrections along the dominant axis where the drift varies drastically, this difference vector naturally aligns with that axis. Table confirmm this intuition: the ERK solution difference along the dominant direction (line EF, 4.03e-04) is roughly 6.5x larger than that along the subdominant direction (line GH, 6. 1e- This motivates our core utilizing the ERK difference to suppress severe errors in stiff regions.
idea:
For broader analysis, we measure the cosine similarity the dominant eigenvector with both the LTE and the ERK solution difference across different stiffness levels. As shown in Figure 2 (a), the alignment between the LTE and the dominant cigenvector steadily increases with stiffness. This demonstrates that the cigenvector serves Additionally,, Figure (b) illustrates that the ERK solution difference also strongly aligns with the dominant eigenvector when stiffness is high. This crucial insight motivates novel guidance strat- egy, ERK-Guid which leverages the dominant eigenvector estimated from the embedded Runge- Kutta (ERK) pair as its guiding signal.
reliable proxy for the LTE direction in stiff regions.
We evaluate ERK-Guid against two widely used baselines: CFG Ho & Salimans (002)) (condi- tional-unconditional score difference) and Autoguidance Karras al. (222a) (main-weak model difference). Figure 2 (c) shows that ERK-Guid consistently maintains strong alignment with the dominant eigenvector across all stiffiness quantiles. Notably, the performance gap widens in the high- stiffness bin (Q75-10 100), where CFG and Autoguidance exhibit weak or mixed alignment. These re- sults suggest that ERK- Guid provides an orthogonal guidance signal that effectively complements, rather than overlaps with, existing methods.
4.2 STIFFNESS AND DOMINANT EIGENVECTOR ESTIMATOR
The key intuition from the previous section is that, once stiffness is high, the dominant eigenvector provides reliable proxy for reducing local truncation error (LTE), In practice. however, direct access to the Jacobian is infeasible, making stiffness estimation challenging. A common alternative is to use Jacobian-vector product (JVP) based power iterations, which are supported in frameworks such as PyTorch but remain prohibitively expensive for diffusion sampling, where each step requires costly network evaluations. To overcome this, propose cost-free estimators of stiffness and the dominant eigenvector, exploiting the ERK solution/drift difference without additional evaluations.
Let x x
Heun and Euler denote the states at a; computed by Heun's update and the intermediate Fuler step from a,-1, respectively. We define the stiffness estimator as
Euler o..) || f (xHoun, X
Heun, as) - f(x Euler; oz) ll2
Pstitf (x
(17)
x Heuu
x xB
Buler
Euler corresponds exactly to the ERK solution difference, while f (xHHeu,, oi) Here, x Xx.
f [x Buler; .;) represents the ERK drift difference.

---

Published as a conference paper at ICLR 2026
Proposition 1 Let Jo: be the Jacobian matrix of the drift function fxx; σ)) evaluated at x Assume that f Ox) has a locally Lipschitz. Jacobian near xHeun, If the ERK solution difference nonzero vector, sufficiently small, and aligned with the eigenvector associated with the dominant eigenvalue X of Jar in magnitude, such that
Houn
A := =xHeun Euler
(x
Heun
Euler )ll |A|lx Heun -xEaler
EEller ll2 + O(1)A>1))
Xx,
then the magnitude of the dominant eigenvalue, 1 admits the approximation
[ ((xxHoun,o oz) (fae)-((xp) Bu ;;) + 0(114712).
[ =
x Xx,
Heun
Buler
Proposition establishes that the proposed stiffness estimator accurately recovers the true stiffness under the assumption. We provide the proof of Proposition in Appendix Since our estima- tor relies the alignment between the ERK solution difference and the dominant eigenvector, it reliable only when the estimated stiffness is sufficiently high. This requirement is explicitly in- corporated into guidance design (see Section 4.3). Notably, the stiffness estimator requires additional network evaluations: Heum Euler and f fxx az) are already obtained during the Heun correction, while f Heun, fxxe.un ;o;) is reused in the subsequent Huen step.
As shown in Section 4.1 the ERK solution difference provides useful proxy for the dominant eigenvector in stiff regions. To improve robustness, we define the dominant eigenvector estimator as normalized ERK drift difference:
f(x (eHeeun, σi) (xxerler, T..)
stiff Sx
Heun (x,,
Fuler ,o:) = I [1f(*Heun, Heun; oi) fx (xxBaler o.)) o,
(18)
Under local lincarization, this difference approximates the Jacobian applied the ERK solution difference:
fxxt Heun o.)) - - Euler, o.) Jo. (xH;
Heun, o.) (xa,
Heun
Euler)
Xx:
which corresponds to single-step JVP power iteration. In the eigenbasis of Jop. components with larger eigenvalues are amplified, effectively suppressing subdominant directions and naturally steer- ing the estimate toward the dominant eigenvector. We demonstrate this in Section 5.1
4.3 EMBEDDED KUNGE--KUTTA GUIDANCE
We now refine Eq. 15 into practical guidance scheme. Leveraging cost-free estimates of stiffness and the dominant eigenvector, we construct a stabilized variant that ensures consistent updates across stiffness levels. The modified formulation adaptively scales the correction according the estimated stiffness and aligns the update with the dominant local dynamics.
Using the proposed estimators, denote fot
Heen f(x or)) Po. Pouft (xHoun x 7;
Falle
or.),
and V., := Vaciif Vacir((XHHum (xHemm, X update as:
Euler ..).. With step size == o, -0++1 > O. we define the ERK-Guid
Heun
Heun h 2² (fthum vo.) Va.: 00+1
(19)
where 3 = (A., mean) is binary indicator that activates the guidance when the estimated stiffness exceeds the prescribed threshold Weon- and Wstif h Po. adaptively scales the guidance mag- nitude according to the estimated stiffness. Here, Wsaat hyperparameter that scales the overall guidance strength. Importantly, this update be equivalently written in the form of conventional guidance:
Heun = d011
Heun - hy (f pEuler) Xx
fo:
(20)
where 7 8. (S Hean Se. fllem pRaier
and &Buler =
f fx
Euler; o.)) This form reveals that ERK-Guid operates
as a guidance term that extrapolates between two model predictions, analogous to conventional guidance mechanisms, while grounding the guidance direction in estimator of the local truncation

---

Published as a conference paper at ICLR 2026
Contre
Offerezze
Powe Nethod
Suffness
(a) Stiffness Estimator
(b) Eigenvector Estimator
(a) 16 Steps
(b) 32 Steps
Figure 3: Accuracy of proposed estimators. (a) Our estimated stiffness values highly corre- late with the JVP-based one. (b) ERK drift dif- ference (blue) maintains higher alignment with the dominant eigenvector than the ERK solution difference (orange), especially at high stiffness,
Figure 4: Grid search of hyperparameters. Quantitative trends of FID and FD-DINOV2 as Weon varies. Each curve corresponds to dif- ferent value of Wstiff with points indicating in- creasing Wcon. shown for (a) 16-step and (b) 32- step sampling.
Table 2: Quantitative results on ImageNet-S11. Rows with wsift 0 indicate guidance, while nonzero Wsuff correspond to ERK-Guid.
Quantitative results of guidance compatibility on ImageNet-512. ERK-Guid is combined with established guidance methods to enhance sampling quality and robustness under various guidance configurations.
Table 3:
Matep
Msan
HD-DINDv2
HDD
Precision
Recall 1
18
0.0
90.1
2.58
0.631
0.672
244
1.0
85.2
2.56
0.635
0.673
247
Pspp
Method
FD DINOV2 1
FID
Precision:
Recall
15
1.5
83.7
2.60
0.635
0.675
249
0.612
0.707
2.5
84,9
3.03
0.625
0.667
241
16
0.0
97.4
2.79
0.629
0.652
238
0.75
88.9
2.68
0.659
244
Baik-Gaid
133.9
0.593
215
0.0
161.2
7.06
0.146
0.614
183
Autoguidance
822
0.652
0.641
229
0.5
136.9
4.91
0.545
0.598
201
+ERK-Guul Gald
752
1.90
0.009
0.648
236
error. Derivations from Eq. 15 to Eq. 19 and from Eq.
19 to Eq. 20
provided in Appendix A.3
and Appendix A.4 respectively.
Compared to the exact LTE expression, our final formulation incorporates confidence gate , to restrict guidance to sufficiently stiff regions, as analyzed in Figure 4 It also includes a stiffness- dependent scaling controlled by the hyperparameter Wstiff, which adjusts the overall correction strength, with no guidance applied when Waiff 0. In addition, we replace a(2) with a quadratic form 2², which prevents excessive amplification under inaccurate estimates while preserving smooth behavior near zero. The choice of modulation is examined through ablations in Appendix B.3 and Figure 6b The complete procedure is summarized in Algorithm
Computation cost. ERK-Guid incurs no additional network evaluations: all required quantities are already computed during the Heun update. Thus, unlike CFG or Autoguidance, our approach im- poses no extra evaluation overhead and relies solely on the discrepancy between two solver orders.
EXPERIMENTS
We evaluate ERK-Gui Reid on both synthetic and real-world datasets. On synthetic data, we validate our stiffness estimator against Jacobian Vector Product (JVP) references and compare ERK solution and drift differences for eigenvector estimation Section 5.1 In Section 5.2 we present quantitative results real-world datasets, comparing our method against unguided sampling. In Section 5.3 we examine ERK-Guid's compatibility with existing guidance methods and its plug-and-play adapt- ability across different solvers to demonstrate its versatility. Furthermore,
compare ERK-Guid against classical stiffness-wwar aware adaptive step-size control (Appendix [B.6). which reduces the step size when high stiffness is detected, well peddctor-cooreecte sampling (Appendix B.7], po- sitioning our method against established numerical strategies. Detailed experimental settings and computational overhead analysis provided in Appendix B.2
8

---

Published as a conference paper at ICLR 2026
Table 4: Quantitative results of plug-and-play adaptation of ERK-Guid to solver methods on ImageNet-64 and FFHQ-64.
Dataset
ImageNet 64 (FIDL)
FETI 64 (FID.)
Heun Hairer al.
1995
89.63
37.65
16.46
142.4
57.21
29.54
ERK-GURd
85.19
35.93
13.85
132.8
54.72
23.36
DPM-Solver La et ERK-GGid
2022
44.83
12.42
6.84
83.17
22.84
9.46
31.59
10.57
6.54
49.0
10.42
4.64
DEIS, 7hang & Chen 2023
12.57
6.84
5.34
12.25
7.59
5.56
ERK-GGod
9.56
6.25
4.89
9.96
6.14
4.46
5.1 ACCURACY OF THE ESTIMATORS
In Figure 3 we validate the accuracy of our estimators. FigureBja) illustrates that the stiffness esti- mator shows strong correlation with the JVP reference, increasing consistently with the reference values. Also, Figure bb) demonstrates that the eigenvector estimator based ERK drift differ- ences exhibits higher alignment with the dominant eigenvector than the ERK solution difference, particularly in stiff regions. These results confirm the reliability of our estimators for identifying the dominant eigenvector direction as guidance.
5.2 EFFECTIVENESS OF THE GUIDANCE
Table summarizes quantitative results on ImageNet 512x512 with EDM2 and the Heun sampler. We take W'tifff = as the baseline without guidance. As the guidance scale increases, FD-DINOv2 consistently decreases and reaches its best at Wssiff = 2.0, yielding 82.8 compared to the baseline 90.1. Importantly, this fidelity gain is achieved while keeping FID competitive and consistently improving Precision, Recall, and Inception Score, indicating that update strengthens fidelity without sacrificing diversity or alignment. The advantage becomes more pronounced under fewer sampling steps, where local truncation errors dominate. With 16 steps, FD-DINOv2 improves from 97.4 to 88.9 and FID from 2.79 to 2.68, accompanied by gains in Precision and Inception Score. At 8 steps, the effect is even stronger: FD-DINOV2 drops from 161.2 to 136.9, FID from 7.06 to 4.91, with substantial boosts in Precision, Recall, and Inception Score. Overall, these results demonstrate that ERK-Gul effectively mitigates error accumulation in stiff regions, delivering consistent im- provements across settings and providing particular advantages in low-step regimes, all without any additional training or model evaluations. Moreover, Figure provides grid-search analysis over Wsulf and threshold Weons illustrating robust trends across hyperparameter choices and confirming that -uid maintains stable improvements over a wide range of settings. Qualitative results across varying Wsitff and wron are provided in Appendix E
5.3 GUIDANCE COMPATIBILITY AND SOLVER PLUG-AND-PLAY SOLVER ADAPTATION
In this section, we highlight key properties of ERK-Guid: (i) its compatibility with existing model-based guidance methods, and (ii) its and-play adaptability to various solvers.
Guidanee compatibility. We examine whether ERK- can be combined with existing guidance schemes. Diffusion sampling errors arise from two sources: model error and solver error (LTE). Un- der the predictor-corrector view Bradley & Nakkiran (2024), guidance methods act correctors. ERK-Guid targets local truncation error (LTE), whereas CFG and Autoguidance primarily address model error arising from imperfect score estimates. Motivated by this complementarity, we com- bine ERK-Guid with CFG and Autoguidance in Table 3] showing that our correction consistently strengthens model-based guidance and generalizes well other guidance methods. Additional de- tails are provided in Appendix B.2
Flug-and-plas adaptation. Similar to other guidance methods, ERK-Guid can applied to various solvers plug-and-pla, correction module. To demonstrate its effectiveness and broad applicabil- ity, we evaluate ERK-Guid -uid on higher-order solvers, including Heun's 's method Hairer et al. (199)), DPM-Solver et al. (2022). and DEIS Zhang & Chen (2023) ImageNet Deng al. (2009) and FFHQ et al. (2019) at 64 x 64 resolution. Table demonstrates that combining ERK-Guid with solver methods consistently improves performance across all NFEs both datasets. These

---

Published as a conference paper at ICLR 2026
DPM-Solver
DPM-Solver + ERK-Guid (Ours)
A hyper-realistic Iron Man suit with crisp metallic reflections and glowing are reactor dark hangar.
A massive dragon breathing, stream of bright fire in a dark canyon, glowing scales and sharp wing edges, cinematic lighting.
Figure 5: Qualitative comparison on PixArt-a Chen et al. (2023). We perform text-to-image generation to compare DPM-Solver with our ERK-Guid. As shown in the blue zoomed-in regions, ERK-Guid captures fine semantic details more accurately.
results highlight the robust plug-and-play capability of ERK-Guid and its effectiveness even when paired with higher-order ODE solvers. Additional details are provided Appendix BB5 and Ta- ble 99) Moreover, Figure demonstrates that ERK-Guid delivers strong qualitative improvements on PixArt-a Chen (2023), built upon a Diffusion Transformer (DiT) Peebles & Xie (2023) backbone, further highlighting its architectural generalization capability.
6 CONCLUSION
In this work, we proposed ERK-Guid, a stiffness-aware diffusion sampling framework that exploits Embedded Runge-Kutte solver discrepancies as informative guidance signals. Motivated by the alignment between local truncation emors and dominant eigenvectors in stiff regions, we introduce cost-free estimators for stiffness detection and eigenvector estimation based on ERK solution and drift differences, without additional network evaluations. Building on these estimators, a stabilized guidance scheme that provides an orthogonal correction and integrates seamlessly as plug-and-play module into model-error-haal based guidance schemes and Runge- Moinge-Kutta--e Kutta-based solvers without modifying their core updates. Extensive experiments demonstrate that ERK-Guid consis- tently improves sample quality and efficiency comparisons against adaptive step-size control and predictor-correctoe samplers. Overall, ERK-Guid establishes a principled and practical framework for stiffness-aware guidance in diffusion models, opening new directions that bridge numerical analysis and generative modeling.
design
strong baselines, and further show favorable
10

---

Published as a conference paper at ICLR 2026
REPRODUCIBILITY STATEMENT
describe our pipeline design, and the O algorithmic details and full pseudocode. Appendix B documents experimental settings and hyperparameters. We provide the official implementation at https://github. http://aoioootttttt//t///g/aa/a com m/ htt.bb/vlbbb//b/mmammmmmmmmbmmbl wwtpl/laabb//euuu/ee G
In Section 4.3
ETHICS STATEMENT
Our method operates at the sampling stage of pretrained diffusion models and does not introduce new training data or modify model parameters. As such, it inherits the potential risks the un- derlying models, including biased, harmful, inappropriate outputs depending the conditioning input. Users and practitioners should ensure appropriate monitoring and alignment measures when applying ERK-Guid.
ACKNOWLEDGEMENT
This research was supported by the ASTRA Project through the National Research Foundation (NRF) funded by the Ministry of Science and ICT (No. RS-2024-00439619, 50%) and (No. RS- 2024-00440063, 50%).
REFERENCES
Arwen Bradley and Preetum Nakkiran. Classifier-free guidance is predictor-corrector In Advances in Neural Information Processing Systems, NeurIPS Workshop. 2024.
Andrew Brock, Jeff Donahue, and Karen Simonyan. Large scale GAN training for high fidelity natural image synthesis. In International Conference on Learning Representations, ICLR, 2019.
Tim Brooks, Aleksander Holynski, and Alexei A Efros. Instructpix2pix: Learning to follow image editing instructions. In Conference on Computer Vision and Pattern Recognition, CVPR, 2023.
Junsong Chen, Jincheng Yu, Chongjian Ge, Lewei Yao, Enze Xie, Yue Wu, Zhongdao Wang, James Kwok, Ping Luo, Huchuan Lu, et al. Pixart-a: Fast training of diffusion transformer for photore- alistic text-to-image synthesis. In International Conference on Learning Representations, ICLR, 2023.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In Conference on Computer Vision and Pattern Recognition, CVPR, 2009.
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial networks. In Communications of the ACM, 2020.
Agrim Gupta, Lijun Yu, Kihyuk Sohn, Xiuye Gu, Meera Hahn, Fei-Fei Li, Irfan Essa, Lu Jiang, and José Lezama. Photorealistic video generation with diffusion models. In European Conference on Computer Vision, ECCV, 2024.
Ernst Hairer and Gerhard Wanner. Solving Ordinary Differential Equations II: Stiff and Differential- Algebraic Problems, volume 14 of Springer Series in Computational Mathematics. Springer, 2nd revised edition, 1996. ISBN 978-3-540-60452-5.
Ernst Hairer, Syvert P. Nørsett, and Gerhard Wanner. Solving Ordinary Differential Equations I: Nonstiff Problems, volume of Springer Series in Computational Mathematics. Springer, 2nd edition, 1993. ISBN 978-3-540-5667000.
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by two time-scale update rule converge local nash equilibrium. In Advances in Neural Information Processing Systems, NeurlPS, 2017.
Jonathan Ho and Tim Salimans. Classifier-fie free diffusion guidance. In Advances in Neural Informa- fion Processing Systems, NeurIPS Workshop, 2022.

---

Published as a conference paper at ICLR 2026
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. In Advances in Neural Information Processing Systems, NeurIPS, 2020.
Tero Karras, Samuli Laine, and Timo Aila. A style-based generator architecture for generative adversarial networks. Conference on Computer Vision and Puttern Recognition, CVPR, 2019.
Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion- based generative models. In Advances in Neural Information Processing Systems, NeurIPS, 2022.
Tero Karras, Miika Aittala, Tuomas Kynkäänniemi, Jaakko Lehtinen, Timo Aila, and Samuli Laine. Guiding diffusion model with bad version of itself. In Advances in Neural Information Pro- cessing Systems, NeurlPS, 2024a.
Tero Karras, Mila Aittala, Jaakko Lehtinen, Janne Hellsten, Timo Aila, and Samuli Laine. Analyz- ing and improving the training dynamics of diffusion models. In Conference on Computer Vision and Pattern Recognition, CVPR, 2024b.
Bahjat Kawar, Shiran Zada, Oran Lang, Omer Tov, Huiwen Chang, Tali Dekel, Inbar Mosseri, and Michal Irani. Imagic: Text-based real image editing with diffusion models. In Conference on Computer Vision and Pattern Recognition, CVPR, 2023.
Diederik P Kingma and Max Welling. Auto-encoding variational bayes. In International Conference on Learning Representations, ICLR, 2014.
Juyeon Ko, Inho Kong, Dogyun Park, and Hyunwoo J Kim. Stochastic conditional diffusion models for robust semantic image synthesis. In International Conference Machine Learning, ICML, 2024.
Tuomas Kynkäänniemi, Tero Karras, Samuli Laine, Jaakko Lehtinen, and Timo Aila. Improved precision and recall metric for assessing generative models. In Advances in Neural Information Processing Systems, NeurIPS, 2019.
Tuomas Kynkäänniemi, Miik Aittala, Tero Karras, Samuli Laine, Timo Aila, and Jaakko Lehtinen. Applying guidance in a limited interval improves sample and distribution quality in diffusion models. In Advances in Neural Information Processing Systems, NeurIPS, 2024.
Sojin Lee, Dogyun Park, Inho Kong, and Hyunwoo J Kim. Diffusion prior- based amortized varia- tional inference for noisy inverse problems. In European Conference on Computer Vision, ECCV, 2024.
Yaron Lipman, Ricky TQ Chen, Heli Ben-Hamu, Maximilian Nickel, and Matt Le. Flow matching for generative modeling. In International Conference on Learning Representations, ICLR, 2023.
Luping Liu, Yi Ren, Zhijie Lin, and Zhou Zhao. Pseudo numerical methods for diffusion models on manifolds. In International Conference on Learning Representations, ICLR, 2022.
Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu. Dpm-solver: A fast ode solver for diffusion probabilistic model sampling in around 10 steps. In Advances in Neural Information Processing Systems, NewrIPS, 2022.
Dogyun Park, Sihyeon Kim, Sojin Lee, and Hyunwoo Kim. Ddmi: Domain-agnostic latent diffu- sion models for synthesizing high-quality implicit neural representations. In International Con- ference Learning Representations, ICLR, 2024a.
Dogyun Park, Sojin Lee, Sihyeon Kim, Taehoon I Lee, Youngjoon Hong, and Hyunwoo J Kim. Con- stant acceleration flow. In Advances in Neural Information Processing Systems, NeurlPS, 2024b.
Dogyun Park, Taehoon Lee, Minseok Joo, and Hyunwoo Kim. Blockwise flow matching: Improv- ing flow matching models for efficient high high-quality generation. Advances in Neural Information Processing Systems, NeurIPS. 2025.
William Peebles and Saining Xie. Scalable diffusion models with transformers. In International Conference on Computer Vision, ICCV, 2023.
12

---

Published as a conference paper at ICLR 2026
Linda Petzold. Automatic selection of methods for solving stiff and nonstiff systems of ordinary differential equations. SIAM journal on scientific and statistical computing. 4(1):136-148, 1983.
Danilo Rezende and Shakir Mohamed. Variational inference with normalizing flows. Interna- tional Conference on Learning Representations, ICLR, 2015.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image synthesis with latent diffusion models. In Conference on Computer Vision and Paitern Recognition, CVPR, 2022.
Seyedmorteza Sadat, Otmar Hilliges, and Romann M Weber. Eliminating oversaturation and ar- tifacts of high guidance scales in diffusion models. International Conference on Learning Representations, ICLR, 2024.
Tim Salimans, Ian Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen. Improved techniques for training gans. In Advances in Neural Information Processing Systems, NewriPS, 2016.
Lawrence F Shampine and Charles William Gear. A user's view of solving stiff ordinary differential equations. SIAM review, 21(1):1-17, 1979.
Neta Shaul, Juan Perez, Ricky TQ Chen, Ali Thabet, Albert Pumarola, and Yaron Lipman. Bespoke solvers for generative flow models. In International Conference on Learning Representations, ICLR, 2024.
Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. In Interna- tional Conference on Learning Representations, ICLR, 2021a.
Yang Song. Jascha Sohl-Dickstein, Diederik Kingma, Abhishek Kumar. Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In Interna- tional Conference on Learning Representations, ICLR, 2021b.
George Stein, Jesse Cresswell, Rasa Hosseinzadeh, Yi Sui, Brendan Ross, Valentin Villecroze, Zhaoyan Liu, Anthony L Caterini, Eric Taylor, and Gabriel Ganee. Exposing flaws of generative model evaluation metrics and their unfair treatment diffusion models. In Advances in Neural Information Processing Systems, NeurIPS, 2023.
Shuai Wang, Zexian Li, Tianhui Song, Xubin Li, Tiezheng Ge, Bo Zheng, Limin Wang, et al. Differentiable solver search for fast diffusion sampling. International Conference on Machine Learming, ICML, 2025.
Shuchen Xue, Zhaoqiang Liu, Fei Chen, Shifeng Zhang, Tianyang Hu, Enze Xie, and Zhenguo Li. Accelerating diffusion sampling with optimized time steps. In Conference on Computer Vision and Puttern Recognition, CVPR, 2024.
Lvmin Zhang, Anyi Rao, and Mancesh Agrawala. Adding conditional control to text-to-image diffusion models. In Conference on Computer Vision and Pattern Recognition, CVPR, 2023.
Qinsheng Zhang and Yongxin Chen. Fast sampling of diffusion models with exponential integrator. In International Conference on Learning Representations. ICLR, 2023.
Wangbo Zhao, Yizeng Han, Jiasheng Tang, Kai Wang, Yibing Song, Gao Huang, Fan Wang, and Yang You. Dynamic diffusion transformer. In International Conference on Learning Representa- tions, ICLR, 2025.
Wenliang Zhao, Lujia Bai, Yongming Rao, Jie Zhou, and Jiwen Lu. Unipe: unified predictor- corrector framework for fast sampling of diffusion models. In Advances in Neural Information Processing Systems, NeurlPS, 2023.
Candi Zheng and Yuan Lan. Characteristic guidance: Non-linear correction for diffusion model large guidance scale. In International Conference on Machine Learning, ICML, 2024.
Kaiwen Zheng, Cheng Lu, Jianfei Chen, and Jun Zhu. Dpm-solver-v3: Improved diffusion ode solver with empirical model statistics. In Advances in Neural Information Processing Systems, NewrIPS, 2023.
13

---

Published as a conference paper at ICLR 2026
A DERIVATION
A.1 DERIVATION OF THE EXACT ONE STEP INCREMENT
We provide detailed derivation of Eq. 14 i.c., the exact one-step increment under the local lin- carization. Recall that around X the score-based vector field is approximated as
dx Sr; + Jx,(Xx Xa;),
(21)
do
where we denote Jo: J (Xo: σ3) and for f(xozi This yields linear ODE of the form dy
do Jo, y for y(o.) = 0,
(22)
where we introduced the shifted variable y(o) x Xo, Let 0; 0++1 denote the step size. Our goal is to compute y(o, - h) X Xo..
Assuming -a log p(x:: a) is -m-smooth, its Hessian Ja, is symmetric and thus admits the following cigendecomposition:
Jo. VAV
VVV=I, Jo VAVK.
Wk
(23)
where is the orthogonal matrix whose columns are the eigenvectors Va of Ja,, the diagonal matrix of the corresponding cigenvalues Ag, and I is the identity matrix.
Let us define the state projected onto the cigenbasis space:
u()) V (T))
g V Soc
9k - g- (So. Vk).
(24)
where (5 refers to inner product, and ex is the k-th standard basis vector.
Since V is constant on this interval (by the local linearization assumption):
du de (V u(o)) = VT dy do
(25)
do do
= (Jo + fa.)
(26)
= =VT(VAV yy V for
(27)
=(VTV)A (V 8) g
(28)
= -Au+9
(29)
Therefore, the system decouples into independent scalar ODEs because A is diagonal.
Writing the k-th coordinate explicitly,
duk
duu + gk.
uk(o,) 0.
(30)
do
has the closed-form solution:
un(o) -o: oz)) gh
(31)
where the function ((z) s--1 for + and ((0) = 1.
Let z = AR(0r-1 σ:) = -hi Since = Vu, we obtain
XT.+1 Xx. =y(o, h) Vulo; h)
(32)
--hV Σ ((2x) gE
(33)
h Σ ((z)) gb Ver
(34)
= h Σ ((zx) (fo., vk) Vk.
(35)
which is the desired expression in Eq. 14
14

---

Published as a conference paper at ICLR 2026
A.2 DERIVATION OF PROPOSITION
I
In Section 4.2 of the main paper, we introduce a cost-free stiffness estimator that exploits the ERK difference without additional evaluations. We provide the proof of Proposition [ which shows that the proposed estimator can approximate the magnitude of the dominant eigenvalue.
Proposition 1 Jo: be the Jucobian matrix of the drift function f o)) evaluated at Heun Assume that O) has a locally Lipschitz Jacobian near Heun, If the ERK solution difference A == x - XEuler is nonzero vector, sufficiently small, and aligned with the eigenvector associated with the dominant eigenvalue of Jo, in magnitude, such that
[|Jo((xx Heun Euler )l = [ Heun - -x;; -x,; Buler 1l2 + O(1A4)))
then the magnitude of the dominant eigenvalue, [ admits the approximation
[ Heum, o.) - (x Buler, ;o)))2
O(1IA711))
Heun xEuler
Proof 1 Under the assumptions of the proposition, a firrtt-order Taylor expansion of f(x;; 0) about Heun gives
f fxx0 Buler, o.) = =fxxt
Heun; o.) Jo, (xx (x - -x -xElerr + e,,
(36)
where the remainder (1IA*1)) is bounded due the locally Lipschitz Jacobian.
Rearranging Eq. 36 to isolate the remainder e yields
Ja, Hx - Xx. Eulery (f(x Heun, σ,) f(x o.)) = =e
(37)
Taking the vector norm on both sides gives
|| (x
Heun
X5,
Euler)
(f(x Hemm, σ;) - f(x o;)) = |el12-
(38)
Applying the reverse triangle inequality |||a||2 - |lb|12 I ||a bll2 Eq. 38
we obtain
|| (x
Heun
x m.
Buler
I|l
f Heuu, σi) - f fxx,,
Euler, o1)1la 1 $|ee|2 01 12)
(39)
which directly implies
|| (x
Heun
Euler l|| [x Houn, o,) - f(x
Euler, : + (1144)
(40)
Applying the alignment assumption to Eq. 40 yields
| Heun
Euler
ll [x Heun, oi) f (xxeleer, 0s)ll2 + +0(A)*)))
(41)
a,
Since |eHu x
Buler 2|2 > O, dividing both sides by this norm produces the desired approximation
Heun,
os) Euler; owlle
+ (11A41122
(42)
Heun Euler 12
A.3 DERIVATION OF THE ERK-GUID FROM EQ. 15
We present a detailed derivation showing how the ERK-Guid (Eq. 19) can be obtained from the local truncation error of Heun's method (Eq. T5}. Let X₃ and V denote the dominant eigenvalue and its corresponding eigenvector of the Jacobian Js,, respectively. As discussed in Section 4.1 when the dominant eigenvector V1 governs local dynamics, the LTE of Heun's 's method dominated along the direction of V1:
Heun Xo.11
x X
Heun ~ -ha(zi)(fon v1) Vi
(43)
where := -hA Rearranging terms gives
X ss x
Heun - (f(z))(foo) v1) V1-
(44)
15

---

Published as a conference paper at ICLR 2026
The Taylor expansion of a(2) at yields
a(z1) = 2 + O
(45)
6
and substituting this Taylor approximation into Eq. 44 gives
Heun X 30441
zî (for, V1) V,
(46)
We interpret this additive term as a guidance correction and introduce a tunable scale Wain as follows
& Heun 04+
=X 0041
Heun h x² (fo:, v1) V., 2 == Wariff hA1
(47)
where constant absorbed into Wain
In practice, the sampling proceeds sequentially, the initial point for the current step is from the previous step's s Heun update. We denote the drift evaluated at this initial point as HHon Further- more, since neither the exact dominant eigenvector V1 the eigenvalue X1 is available, replace them with our cost-free estimators Va, and Po; evaluated at this state. This yields the practical ERK-Guid update:
Heun 7
Heun x0111
x² (ftt Vo.) Vo.. :: : hpo.-
(48)
Since the derivation assumes operation within stiff regions, we introduce binary indicator , that activates the guidance only when the estimated stiffness exceeds a threshold, i.e., Po. > Weon-
& Heun
Heun - hB3 3² x? (fHeun, Vo.) Voi z:= W:tiff hpo.-
(49)
A.4 RESEMBLANCE BETWEEN OUR ERK- -UID AND OTHER GUIDANCES
We provide the alternative (but equivalent) formulation of our proposed method common form of guidance schemes. Eq. the ERK-Guid updates the Heun prediction as
Houn
HeuD
h 2² (ftteum Vo.) Vo.
(50)
where h o, 0141. B (āx S J. Watiff po. Substituting the definition of the eigen- vector estimator vo. (Eq. 18 into the above ERK-Guid update yields
h 132² 2 (fHeem (o.)
Heun Doll
Heun
S
Heuu
Eullrr
(51)
Xaii1
fanler] [|
By grouping the scalar coefficients into adaptive scaling factor
3 22 (Heun vo.)
7xx
Houn
xo,
Fuler .oi) = f Buler
(52)
the ERK-Gui update can be rewritten as
Houn 7:-1
X
Houn
h (xx
Houn
x
Fuler
f
Houn
fa.
Fuler (53)
The above equation reveals that our method admits an interpretation guidance term with an adaptive scaling factor 7.
B EXPERIMENTAL DETAILS
B. 2D TOY EXPERIMENT
In Section 4.1 the main paper, we analyze synthetic 2D distribution with analytically known
scores study how stiffness influences drift geometry and local truncation error. Figure illustrates
16

---

Published as a conference paper at ICLR 2026
drift behavior in non-stiff and stiff' regions, Table I presents the norms of the LTE and ERK solu- tion difference when projected onto the dominant and subdominant eigenvector axes, and Figure presents quantitative alignment results.
2D Visualization. The target distribution Pasu (x)c) forms a tree-like structure in the 2D plane, with probability density concentrated along the branch centerlines. Figure [ (a) visualizes the degree of stiffness across the 2D plane at the 29th sampling step (out of 32). To explicitly compare the local dynamics between non-stiff and stiff regions, we plot the drift field projected onto the local eigenbasis for both the non-stiff' region (lines AB and CD) and the stiff region (lines EF and GH).
In the non-stiff region the main trunk (panels (b) and (c), lines AB and CD), the density varies smoothly. Thus, the projected drift exhibits minimal variation and consistently points toward the high-density interior, forming nearly parallel patterns. Conversely, the stiff region is centered on the low-density gap between two diverging branches. Panel (d) (line EF) traverses this gap between two sub-branches. Because points E and F are located close to distinct high- branches, the vectors in the gap split into two groups, pointing toward either E or F. Panel (e) (line GH) aligns with the branch leading toward the central junction. Since point G is closer to the high-density core branch than point H, the vectors flow uniformly from H toward G. Note that lines EF and GH correspond to the dominant and subdominant eigenvector axes, respectively: this drift behavior naturally illustrates why drift variations pronounced along the dominant axis, where the larger associated eigenvalue reflects higher sensitivity in the vector field across the density gap.
For Table [ we evaluate single-step local truncation error (LTE) and the ERK solution difference using the center of the stiff region as the initial point. The LTE is evaluated by comparing a single- step Heun solution with an approximate ground-truth solution obtained by subdividing the step into 100 finer substeps. The ERK solution difference is computed by comparing the Heun and Euler methods starting from the same initial point. Both the LTE and the ERK solution difference vectors are then projected onto the eigenvectors of the Jacobian evaluated at the starting point, effectively decomposing them along the dominant (EF) and subdominant (GH) axes.
Eigenvector Alignment. We explicitly construct the Jacobian by applying Jacobian- prod- uets (JVPs) to one-hot basis vectors, and compute its eigenvalues and eigenvectors directly. Since eigenvectors are -angguuug, we orient them to point in the same half-space as the drift at each state, following Eq. [5] Sampling is performed without any additional guidance and always with the ground-truth score function. Because the exact ground-truth solution is not available, we approxi- mate it by subdividing each original step into 100 smaller substeps with a much finer step size. The local truncation error (LTE) is defined relative to the Heun method under this reference. Figures 2 (a) and (b) are obtained by partitioning into stiffness bins and plotting the median cosine similarity within each bin.
B.2 MAIN EXPERIMENTS
Experimental setup. We conduct experiments on ImageNet (ILSVRC2012) Deng et al. (2009) at resolutions 512x512 and 64x64, well as on the FFHQ Karras al. (2019) 64x64. We use the pre-trained EDM Karras et al. (2022) and EDM2 used as the base solver, and other solvers are incorporated through our plug-and-play module. Evaluation metrics follow prior work: fidelity via FD-DINOV2 Stein et al. (202)). FID Heusel et al. (2177). and Precision Kynkaänniemi et al. (019); diversity via Recall Kynkaänniemi et al. (2019); and condition alignment using Inception Score (IS) Salimans et al. (0016). For all quantitative evaluations, we generate 50k samples per setting.
et al. (2024a) models. Heun's 's method
Implementation details. We estimate the dominant eigenvector and the stiffness using JVP-based power iteration with a random initialization and 300 iterations per timestep. Figure 7 visualizes the per-timestep convergence and indicates that 300 iterations are sufficient. As in the toy setup, we fix the eigenvector orientation via Eq. 55 that points toward the local drift; Figure3]bb reports the median cosine similarity under this convention. For Table 2 we use the confidence threshold
Weon = 0.5. For Table 3 we use Weon = 0.5. and set Wetiff = 1.0 and 0.75 for sampling with 32 and 16 steps, respectively. For Table 4 we use Weon 0.05, and Wainf is specified in Table 5 In
Figure 5 we conduct 15 sampling steps for text-to-image image generation on PixArt-a Chen et al. (2023) which adopts Diffusion Transformer (DiTT Peebles & Xie (203)) architecture.
17

---

Published as a conference paper at ICLR 2026
1.2
y=1
FD-DINOV2
03
FID
(a) The behavior of function y a(x) and daa. (b) Ablation on the scaling function
Figure 6: Choice of ablation functions. (a) Functional comparison of a(2) and its quadratic ap- proximation. (b) Comparison of a(z), |z], and z², under the FID-FD-DINOv2 trade-off, where the quadratic form 2² achieves the most favorable balance between stability and fidelity.
Computational cost. All main experiments were run on 8x NVIDIA RTX 3090 GPUs. In Table and Table 7 we evaluate the wall-clock time and memory overhead on the ImageNet 512x512 dataset using single RTX 3090 GPU with batch size I, comparing ERK-Guid against Heun's 's method. FRK E-id incurs only slight increase in wall-clock time, and memory consumption remains identical to that of Heun's 's method.
Table 5: Values of the guidance magnitude hyperparameter Watift
Dataset NFEs
ImageNet x4
FFHQ 64x64
10
6
8
Heun
0.75
1.25
1.0
0.5
1.25
1.5
DPM-Solver
1.25
1.25
1.25
1.25
1.25
1.25
DEIS
2.5
2.5
2.5
3.5
3.5
3.5
B.3 CHOICE SCALING FUNCTION
We were concerned about excessive amplification from the original factor ax2), whose exponential growth makes highly sensitive to estimation errors. To mitigate this issue, we consider replacing a(2) with lower-growth alternatives derived from its local behavior around = 0. Specifically, we use the lowest-order nontrivial term its Taylor expansion, yielding a quadratic scaling =², as well even more conservative alternative, [2], which is symmetric and linear in the magnitude of (See Figure 6a).
In Figure 6b we present the corresponding ablation results in terms of the FID-FD-DINOv2 trade- off. Compared to the original a²2), both alternatives improve stability by suppressing excessive growth. Among them, z² consistently achieves lower FD-DINOv2 comparable better FID, providing the best trade-off between fidelity and stability. In contrast, a²)) often degrades FD- DINOv2 due amplification-indueed instability, while |=| remains stable but yields more limited gains. These results indicate that the quadratic form z² strikes favorable balance by preserving the local behavior of (2z) while avoiding detrimental exponential growth.
B.4 GUIDANCE INTEGRATION WITH CFG AND ACTOGUIDANCE
we introduce guidance compatibility of ERK-Guid and ERK-Proj. We define the standard model-bascl guidance term, which applies to methods such as CFG and Autoguidance, follows
In the main paper Section 5.3
Smain (Xa,i σı) - f gaidinn (Xo,5 oi).
(54)
To incorporate ERK-Guid,
we simply replace the original drift of with the guided drift as follows
f (Xo.i os) f (Xozi (x) + (w 1)g,
(55)
18

---

Published as a conference paper at ICLR 2026
Table 6: Wall-clock time (seconds per im- age) on a single RTX 3090 GPU.
Table 7: Memory consumption for generating single image.
Method
Avg
Min
Max
Method
Avg (MB)
Hcun
2.777
2.775
2.782
Heun
1906.82
ERK-Guid (Ours)
2.794
2.785
2.811
ERK-Guid (Ours)
1906.82
Table
Quantitative results of ERK-proj.
Nstep
Method
ID-DINOV2
Precision
Recall
Autoguidance
50.4
1.36
0.691
0.628
262
-ER--Guid
+ERK-Proj
44,9
1.36
0.710
0.605
274
where is the scaling hyperparameter. Then, compute the eigenvector estimator and stiffness estimator as follows:
Heun; σi) - f Euler, σi) X
(56)
Heun; oi) f (x Ao;
Euler; σi)
X
Heun, σ.) 1 Euler, Oi) 112
P
(57)
Euler Heem X
Finally, we constitute our ERK-Gui as follows
HHeun 01+1
Heun
hB 2 (for v v
(58)
a4+1
where B = Lpp >we.e and Wsitf P..-
Lightweight Extension of Guidance Integration. We additionally introduce ERK-Proj, which interpolates between model-error-based and LTE-based correction signals, aiming to reduce both errors simultaneously. With g in Eq. 54 ERK-Proj is defined as follows
c Watair
(59)
7 + (1 11) (g. Vo.) Va.
(60)
(xx,) o.) f σ2) + (w 1)g,
(61)
where 7 adjusts the guidance scaling by the stiffness estimator to interpolate the two guidance sig- nals. In T we report quantitative results demonstrating the effect ERK-Proj when combined with Autoguidance.
Table 9: Guidance configuration for Heun, DPM-Solver, and DEIS.
Solver
Pair of states
h
8
Heun
Xay x
Euler
0i 0i+1
{0, 1]
DPM-Solver (2S)
No,tō, Xo,
0i+1
{0, 1}
DEIS
X Xo.-1
0;-1 0j
10,
B.5 PLUG-AND-PPLYY MODULE FOR ADVANCED SOLVERS
In the main paper Section 5.3
we present additional experimental results that confirm the effective-
ness of method combined with various solvers.
x solver
(x.ver(x,,:0,.1
(62)
0.+1
S solver o4+1
X
solver h.322 (f Vo.) Vo,
(63)
DPM-Solver (2S) Lu et al. (2022) computes an intermediate state during its two-stage update. We denote this intermediate state Xo,+5, and construct the pair {xo,+5. Xo, DEIS Zhang & Chen (2023) computes its update using previous states in a multi-step formulation. We use the most
recent previous state X construct the pair (Xa,, Xx;-1).
19

---

Published as a conference paper at ICLR 2026
Table 10: Evaluation of adaptive step-size control under different stifflness thresholds. (A)
ERK-Guid and (B) Heun do not adopt adaptive step-size. (C-G) adapt step-size using thresholds 0.5, 1,2,5, 10.
Adaptive step-size
Threshold
NFE (Avg.)
FD-DINOv21
FID 4
(A) ERK-Guid (Ours)
63
86.2
2.56
(B) Heun
X
63
90.1
2.58
(C)
T 0.5
90.7
88.9
2.57
(D)
1.0
85.5
89.4
2.57
(E)
2.0
79.5
90.0
2.58
(F)
5.0
67.6
90.1
2.58
(G)
10.0
64.5
90.1
2.58
Table 11: Quantitative results of predictor-corrector and ERK-Guid (Ours).
Method
Stochasticity
NFE
FD-DINOv2
FID
0.05
64
91.5
2.65
2.66
Predictor- Corector
Frrrector-Corector
/
0.15
64
98.9
2.90
Song et al. 2021h
X
0.01
90.6
2.61
X 0.02
87.4
2,73
x 0.03
64
88.6
3.33
ERK Guid (Ours)
X
83.7
2.60
B.6 COMPARISON WITH ADAPTIVE STEP SIZE
In Table TO) we analyze our proposed method with classical adoption of stiffness during diffusion sampling. For the experimental setting, we utilize a pre-trained EDM2 network on ImagNet-512. We use the same EDM discretization and halve the step size whenever the stiffness estimator exceeds thresholds of 0.5, 1, 2, 5, or 10. Although small threshold (e.g.. T = 0.5) produced modest gains in FID and FD-DINOv2, it requires 1.44x more NFEs than ERK-Guid, making it substantially less efficient. Our method still achieved the best performance and efficiency compared to the baseline with various adaptive step-size.
B.7 COMPARISON WITH THE PREDICTOR-CORRECTOR SAMPLER
We evaluate the predictor- corrector (PC) sampler from Song et al 2021b by applying its corrector step after each Heun update. Since the corrector in [Sng et al. (2021b) is designed for the reverse SDE rather than an ODE, we also consider deterministic variant obtained by removing its noise term. We vary the hyperparameter r and measure FID and FD-DINOv2 under a comparable NFE. In Table ERK-Guid achieves strong performance compared to both variants of the PC sampler. The stochastic PC sampler consistently degrades performance across both metrics. In contrast, the deterministic PC sampler of Song et al. (2021b) exhibits clear trade-off: the correction scale increases, FD-DINOv2 increases while FII) decreases.
C ALGORITHM
In the main paper Section 4.3 we introduce our ERK-Guid framework. To provide a clearer step- by-step overview, outline the full procedure in Algorithm I Note that the first sampling step does not admit this guidance mechanism, as it requires an ERK pair from the previous iteration. Therefore, we simply skip guidance this initial step, which is practically justified since stiffness at initialization is typically very small (i.e., 3 s 0).
D LLM USAGE
We only used large language model as writing assistant to refine phrasing, grammar, and clarity. It was not used for technical content, experiments, analyses, results.
20

---

Published as a conference paper at ICLR 2026
Algorithm Sampling procedure with ERK-Guid
1: procedure OuRS(fo ot), for Wsifff Weens E)
2: sample Xg (0, IfI) E #xxxx
3: for - to N do
4:
fi < fx(
5:
4- 0; 0i+1
6:
if of then
7:
Af - Si f
8:
A < X; x
Euler
A ||
9:
v1 + ||T A
10:
3, (p
Weon) Wsitft hp
11:
gi 3 x-
(fi v.) v;
12:
else
13:
gi + 0
14:
end if
15:
Fuler c- X; - lift
16:
if N then
17:
eluler t fo 1 0i+1)
[+11
18:
Heun
1-1 f- x; h ( f
19:
x f- i+1
Ieeu hgi
buffer Buler
20:
4-1 Js+1
21:
else
22:
X x++
23:
end if
end for
25:
return XN
end procedure
Absolute Error to Final (semilog-y)
10-1
10
10
10-10
10-13
10-16
median error
25%
10-19.
75%
50
100
150
200
250
300
iteration k
Figure 7: Convergence of stiffness estimation under JVP power iteration. The plot shows the absolute error between the estimated stiffness at iteration k and the final converged value. The solid line denotes the median across all seeds and timesteps, while the dashed lines indicate the 25th and 75th percentiles. Errors decrease rapidly and stabilize, demonstrating reliable convergence of the iteration procedure.
21

---

Published as a conference paper at ICLR 2026
W stiff
0.0
0.5
1.0
1.5
0.2
Decrease Thre
0.3
W con
0.5
1.0
Baseline
Increase Guidance Strength
Figure 8: Qualitative results of ERK-Guid across adjusting guidance scale.
E QUALITATIVE RESULTS
In this section, present qualitative results of ERK-Guid ImageNet 512x512. Figures and show that applying ERK-Guid enhances image fidelity when sufficient guidance is applied. These results demonstrate that method effectively mitigates solver-induced emror during conditional updates along the sampling trajectory.
22

---

Published as a conference paper at ICLR 2026
W stiif
0.0
0.5
1.0
1.5
0.2
Decrease Threshold
0.3
W con
0.5
1.0
Baseline
Increase Guidance Strength
Figure 9: Qualitative results of ERK-Guid across guidance scales.
23


Reviews:
[{"review_id": "m3sMEC92Fo", "reviewer": "Reviewer_qmxA", "summary": "In this paper, the authors propose ERK-Guid, a new sampling algorithm for diffusion ODE samplers that uses the embedded Euler/Heun pair to estimate a local stiffness scalar and approximate the dominant eigenvector direction via the ERK drift difference in order to cancel the local truncation errors. The authors demonstrate the effectiveness of their method on toy examples and ImageNet-512.", "questions": "1. How did the authors determine the hyperparameters $\\omega_{conf}$ and $\\omega_{stiff}$?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 1, "contribution": 2, "strengths": "1. The proposed method is “cost-free” ,i.e. no additional network evaluation is needed.\n2. ERK-Guid is complementary to CFG and autoguidance.\n3. The sampling recipe is clean and easy to follow.", "weaknesses": "1. The paper positions itself in the same line of work as CFG and autoguidance. However, the proposed algorithm resembles more of an advanced solver with a corrector rather than a guidance sampling method. The first two pages of the paper read extremely disconnected to the rest of the paper.\n2. Following up on the previous point, for an advanced solver paper, the authors fail to compare their algorithm with other strong solvers like DPM-Solver-v3, UniPC and DEIS, which the authors cite but did not provide any empirical comparison to. Moreover, the authors also cite a classic remedy, adaptive step-size, for stiffness (Petzold (1983); Shampine & Gear (1979)), which they also did not compare in their experiments.\n3. Only toy examples and one dataset (Imagenet-512) are examined in their experiments.\n4. ERK-Proj, which outperforms the main algorithm ERK-Guid in one of the major experiments, is only introduced on the second to the last paragraph of the main paper with very minimal details in it. There is also very minimal theoretical backing to this algorithm.\n5. The theoretical analysis for the entire Section 4.3 is very vague and mostly heuristic. However, it constitutes the algorithms that work in practice.\n6. The difference among images in Figure 7 and 8 is extremely subtle and mostly not visible.\n7. No wallclock time or memory overhead is compared, which can be a big factor affecting the practicality of the algorithm.\n\nMinor: \n    (i) Line 51-52 seems to be missing citations?\n    (ii) Line 209 “Let denote”\n    (iii) Table 2 caption: “InageNet”", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "8VQqCin1xv", "reviewer": "Reviewer_2Zyk", "summary": "This paper presents ERK-Guid, a guidance mechanism for diffusion models aimed at reducing the local truncation error (LTE) of ODE solvers, with explicit consideration of the stiffness of ODE trajectories.\nThe authors provide a mathematical analysis of LTE and embedded Runge-Kutta pairs, leveraging the Jacobian of the score function.\nBased on this analysis, they propose cost-free estimators for ODE stiffness and the direction in which LTE is dominant, and demonstrate how to guide the Heun sampler to achieve smaller LTE.\nThrough experiments on a toy dataset, mathematical observations are confirmed. Further experiments using EDM2 on ImageNet examine design choices for the cost-free estimators and demonstrate the effectiveness of ERK-Guid.", "questions": "Regarding Weakness 4: Do you have any results or insights regarding the comparison of your method with advanced ODE solvers for diffusion models?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The theoretical analysis in Section 4 and the toy example provide clear and insightful explanations of the proposed method.\n2. The proposed method is supported by solid mathematical justification and ablation studies.\n3. The flow of the paper after the introduction is clear and easy to follow, with well-defined mathematical formulations and illustrative figures.\n4. The method can be combined with other guidance techniques such as CFG and Autoguidance, suggesting broad applicability.\n5. The experimental results demonstrate the effectiveness of ERK-Guid.", "weaknesses": "1. The introduction is somewhat difficult to follow, especially regarding the relationship between stiffness and other guidance methods such as CFG and Autoguidance, since the motivation for the proposed method and those for CFG and Autoguidance seem different.\n2. It is unclear whether comparing ERK drift and the dominant eigenvector in Figure 2(c) with CFG and Autoguidance is meaningful, since these methods are not designed to minimize ODE solver step error.\n3. The choice of the hyperparameters w_con and w_stiff is important for performance, as shown in Table 2. Hyperparameter search appears necessary, which may be considered a weakness.\n4. While the paper focuses on guidance mechanisms, a more thorough comparison with advanced ODE solvers for diffusion models, such as those mentioned in the RELATED WORKS section or other models such as GENIE [Dockhorn et al., 2022], would be valuable.\n5. Minor comment: Typo in the caption of Table 2 (\"InageNet\" should be \"ImageNet\").\n\n[Dockhorn et al., 2022] Tim Dockhorn, et al. \"Genie: Higher-order denoising diffusion solvers.\" Advances in Neural Information Processing Systems 35 (2022): 30150-30166.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "pjG8zZgeda", "reviewer": "Reviewer_4VrJ", "summary": "Based on the situation that LTE in stiff regions aligns with the dominant eigenvector of the drift function’s Jacobian, this paper proposes Embedded Runge–Kutta Guidance is a new guidance method for diffusion models. The experiments validate ERK-Guid's effectiveness and can combine well with existing guidance methods.", "questions": "- See weaknesses.\n- Typo should also be considered. For example, the first row of Table 2 and Table 3 has the same content but different formats.\n- Please make sure to use vector graphics.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- The paper rigorously derives the alignment between LTE/dominant eigenvectors via local linearization of the drift field, and proves the approximation accuracy of the stiffness estimator \n- This paper designs Comprehensive experiments. Evaluations cover synthetic data and ImageNet with metrics including fidelity, diversity, and alignment. This paper also combines ERK-Guid with existing guidance methods and these methods don't conflict with each other.\n- ERK-Guid improves quality in low-step regimes, which means it has the potential to do acceleration in other works.", "weaknesses": "- Experiments are limited to EDM2 and ImageNet. It remains unproven whether ERK-Guid works for other architectures or non-image tasks. Stiffness varies across tasks, so generalization needs more validation.\n- The performance improvement is not significant, and it is hard to determine whether it is superior to the existing methods. The most crucial point is that even after combining ERK-Guid with other guiding methods, degradation occurred on some datasets, which requires thorough argumentation.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "3f9yw5sc1M", "reviewer": "Reviewer_Rzwt", "summary": "Classifier-Free Guidance (CFG) has established the foundation for guidance mechanisms in diffusion models, showing that well-designed guidance proxies significantly improve conditional generation and sample quality. Autoguidance (AG) has extended this idea, but it relies on an auxiliary network and leaves solver-induced errors unaddressed. In stiff regions, the ODE trajectory changes sharply, where local truncation error (LTE) becomes a critical factor that deteriorates sample quality. This paper's key observation is that these errors align with the dominant eigenvector, motivating it to target the solver-induced error as a guidance signal. This paper proposes \\textbf{E}mbedded \\textbf{R}unge–\\textbf{K}utta based \\textbf{Guid}ance (ERK-Guid), which exploits detected stiffness to reduce LTE and stabilize sampling. This paper theoretically and empirically analyzes stiffness and eigenvector estimators with solver errors to motivate the design of ERK-Guid. This paper's experiments on both synthetic datasets and the popular benchmark dataset ImageNet demonstrate that ERK-Guid consistently outperforms state-of-the-art methods.", "questions": "In practice, extrapolation acceleration is more widely used in sampling. Could this method apply to linear multistep solvers (Adams–Bashforth solver)?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1. Unlike Autoguidance (AG) which relies on an auxiliary network and fails to address solver-induced errors, ERK-Guid avoids dependence on auxiliary networks and specifically targets solver-induced errors, overcoming key drawbacks of prior guidance mechanisms.\n\n2. Leverages a key observation that solver-induced errors align with the dominant eigenvector, innovatively using such errors as a guidance signal—providing a novel direction for optimizing diffusion model sampling.\n\n3. Exploits detected stiffness in ODE trajectories (where trajectories change sharply) to reduce local truncation error (LTE) and stabilize sampling, directly mitigating a critical factor that degrades sample quality in stiff regions.", "weaknesses": "1. Lack of visualization comparison.\n\n2. More experiments are needed. e.g. t2i, t2v.\n\n3. Missing related works with bespoke solver[1, 2, 3], which also searches the optimal solver parameters (linear multisteps solver, RK solver) of a pretrained diffusion model.\n\n[1] Xue, Shuchen, et al. \"Accelerating diffusion sampling with optimized time steps.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024.\n\n[2] Wang, Shuai, et al. \"Differentiable Solver Search for Fast Diffusion Sampling.\"  International Conference on Machine Learning (ICML) 2025. \n\n[3] Shaul, Neta, et al. \"Bespoke solvers for generative flow models.\" arXiv preprint arXiv:2310.19075 (2023).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)