Dual-Solver: A Generalized ODE Solver for Diffusion Models with Dual Prediction
Authors: Soochul Park
Yeon Ju Lee
Venue: ICLR 2026 Poster (2026)
Keywords: Generative Models
Diffusion Models
Fast Sampling
ODE Solver
Primary area: generative models
TL;DR: This work generalizes multistep samplers via parameterizing the prediction type, integration domain, and second-order residual term, and introduces classification-based parameter learning.

Abstract: Diffusion models achieve state-of-the-art image quality.
However, sampling is costly at inference time because it requires a large number of function evaluations (NFEs).
To reduce NFEs, classical ODE numerical methods have been adopted.
Yet, the choice of prediction type and integration domain leads to different sampling behaviors.
To address these issues, we introduce Dual-Solver, which generalizes multistep samplers through learnable parameters that continuously (i) interpolate among prediction types, (ii) select the integration domain, and (iii) adjust the residual terms.
It retains the standard predictor-corrector structure while preserving second-order local accuracy.
These parameters are learned via a classification-based objective using a frozen pretrained classifier (e.g., MobileNet or CLIP).
For ImageNet class-conditional generation (DiT, GM-DiT) and text-to-image generation (SANA, PixArt-$\alpha$), Dual-Solver improves FID and CLIP scores in the low-NFE regime ($3 \le$ NFE $\le 9$) across backbones.

Published as a conference paper at ICLR 2026
DUAL-SOLVER: A GENERALIZED ODE SOLVER FOR DIFFUSION MODELS WITH DUAL PREDICTION
Soochul Park
Yeon Ju Lee
SteAl MODULABS
Korea University MODULABS
scpark200 gma com
jaeyeennju@@kooee..ccc
ABSTRACT
Diffusion models achieve state-of-the-art image quality. However, sampling is costly at inference time because requires a large number of function evalu- ations (NFEs). To reduce NFEs, classical ODE numerical methods have been adopted. Yet, the choice of prediction type and integration domain leads to dif- ferent sampling behaviors. To address these issues, we introduce Dual-Solver, which generalizes multistep samplers through learnable parameters that contin- uously (i) interpolate among prediction types, (ii) select the integration domain, and (ii) adjust the residual terms. It retains the standard predictor-corrector corrector struc- ture while preserving second-order local accuracy. These parameters are learned via classification-bar
objective using frozen pretrained classifier (c.g.. Mo- bileNet or CL.IP). For ImageNet class-conditional generation (DiT, GM-DiT) and text-to- toxt-to-image generation (SANA, PixArt-a)), Dual-Solver improves FID and CLIP scores in the low-NFE regime (3 S NFE S 9) across backbones.
INTRODUCTION
Generative modeling aims to learn a data distribution and generate new samples that resemble real data. Classic approaches include autoregressive models that factorize likelihoods over pixels or tokens (Van den Oord et al.) 2016; [Salimans et al.] (017), variational auto-encoders that optimize evidence lower bound (Kingma & Welling 2013 Vahdat & Kautz 2020), flow-based models that construct exact, invertible density maps Dinh et al., 2014; Kingma & Dhariwal 2018), and generative adversarial networks that learn via adversarial training between generator and discriminator (Goodfellow et al.) 2020 Arjovsky et al.) 2017). Diffusion models have emerged as a powerful class of generative models. seminal work, SShl-Dicksttinn et al. (2015) introduced diffusion probabilistic modeling with a forward noising process and a learned reverse process trained by minimizing Kl--divergence objective. Subsequent reformulations (Ho et al.) 2020) have streamlined training and inference with simple denoising objectives (e.g.. noise or data prediction), Icading to statc-of-the-arr fidelity and robust scaling. Today, diffusion models drive progress across multiple modalities, including images (Dhariwal & Nichol 2021; Rombach et al. 2022), audio (Kong et al. 2020 Liu et al. 2023). and video (Ho et al. 2022 Kong et al. 2024).
Diffusion models generate samples through iterative denoising steps, making the inference cost scale with the number of function evaluations (NFEs). Leveraging the probabilityfloo formulation, which casts sampling as an ordinary differential equation (Song et al.) 2021b), a number of recent works have proposed ODE-based acceleration methods. Along one axis, classical Kutta (Runge 1895) and multistep Adams 1883) provide off-the-shelf accuracy-NFF. NFF trade-offs for a given evaluation budget
ODE methods singlestep
Baasffohshoorth (Bashlorth
[Butcher 2016). Along second axis, diffusionndedicated solvers exploit the structure of the
denoising dynamics: they approximate noise or data predictions with low-order Taylor expansions or Lagrange interpolation and derive closed-form updates (Lu et al.) 2022a b Qinsheng & Chen
2023 Zhao et al., 2023 Xue et al. 2024). Lastly, learned solvers optimize the timestep schedule
and other sampling- related parameters (Zhou et al. 2024; Shaul et al. 2023 2024 Wang et al.
2025). Because these parameters depend on the backbone and dataset, such solvers are typically
tailored specific backbones and settings (e.g., chosen NFE and guidance scale). Training also

---

Published as a conference paper at ICLR 2026
(a) DPM-Solver++ Lu al. 2022b
(b) Dual-Solver (Ours)
(c) BNS-
(d) DS-Solver
Shaul et al. 2024
(Wan al. 2025
Figure 1: Sampling results. SANA (iie et al. 2024 NFE-3, CFG=4.5. See Fig. 18 for further results.
incurs substantial preparation overhead, it requires many teacher trajectories final samples generated using high-NFE sampling. However, compared the classical methods and dedicated solvers discussed above, learned solvers deliver substantially better sample quality and remain an active area research. We introduce Dual-Solver, a learned solver for diffusion models with three types of learnable parameters:
prediction parameter that interpolates among noise, velocity, and data predictions (Sec.33.1)
domain change parameter that interpolates between logarithmic and linear domains (Sec.3.) 3.2);
a residual parameter that adjusts the residual term while preserving second-order accu- racy (Sec. 3.3
We further propose classification-bast based learning strategy that yields high-fidelity images even in the low-NFE regime (Sec. (.2).. Unlike regression- based learning, which typically requires many target samples generated at high NFEs, approach requires no target samples. Solver parame-
ters are learned using either pretrained image classifiers (He et al. 2016) Dosovitskiy et al. 2020
Howard et al. (017) zero-shot classifier (Radford et al.] 2021) For 3 I NFE < Dual Soll-r outperforms prior state-of-the- solvers (Sec. 6).
2
PRELIMINARIES
2.1 DIFFUSION MODELS
Training process. Diffusion models (Ho et al. 2020 Song et al. 2021b) are trained as follows. A clean sample æn is drawn from the data distribution, and noise is drawn independently from the standard Gaussian distribution. They are then linearly combined with signal rate a₁ and noise rate σe
D2 = O4 Do
0x 6,
where 2g ~ Pdattu 6 ~~ A0, I), STST...
(1)
Here, a and o, are set by a predefined schedule. Common choices include variance- variance-preser preserving (Sohl-Dickstein al. 2015 Ho et al. 2020); variance-exploding (VE), with 0, = 1, o, 2 (Song & Ermon 2019 (2000]; and optimal transport (OT), with 0₁ =
(VP), with of to?
t, d, = T - (Lipman et al. (2022). The backbone trained to take a noisy sample æ; and the time t as input and to predict one of the following: the noise e, the clean sample 2p (Ho et al. 2020), or di 6 (Lipman et al. 2022). By convention, the backbone parameters are
das the velocity in dt 2o + da. denoted and the predictions are written eg (noise), 2p (data), and my (velocity).
Sampling process. The dynamics corresponding to Eq. can be expressed as the following stochastic differential equation (SDE) (Song al. 2021b):
fi = dlog a dt g. dt
do? d an
dx. = ft x1 dt + 9i dwa,
o,
(2)
dt
2

---

Published as a conference paper at ICLR 2026
where wy is standard Wiener process. A probability-ffoxy ODE that shares the marginal distributions as the SDE at every time has been proposed (Song et al. 2021b):
dx di
1 Se at - 2 gi V log de (xre, where Ve log (r(())
= IElE xt)]
Eg (2(,,)) t)
Ox
Given this ODE, can perform sampling using classical numerical methods beyond Euler, such as singlestep Runge-Kutta (Runge 1895) and multistep Adams-Bashtin schemes (Bashforth & Adams [883). Moreover, several works split the right-hand side into linear and nonlinear parts, and evaluate the nonlinear term using finiteddifference approximations (Lu et al. 2022a b Zhao et al.
2023 or via Lagrange interpolation (Qinsheng & Chen 2023
Xue al. 2024
2.2 PREDICTION TYPES
Diffusion backbones are typically trained with noise prediction (Ho ct al. 2020 Karras al. 2022 Peebles Xie 2023), whereas flow-matching backbones are trained with velocity prediction (Lip-
man et al. 2022 Chen al. 2023 Xie al. 2024). However, noise, velocity, and data predictions
can be converted into one another as follows.
ze ( t)
Dt
Jz Co (x).
v) = da xo t) + do dt
ce(x,,
(4)
dt
Using these relationships, different prediction type be used sampling time from that used during training. For diffusion models, many early samplers use noise prediction (Ho et al. 2020 Song et al.] 2021a Liu et al. 2022: Qinsheng & Chen 2023 Lu et al. 2022a).. More recently, data prediction has also been widely adopted to improve sample quality (Lu et al. 2022b Xie et al. 2024 Wang et al. 2025). contrast, flow-matching backbones typically use velocity prediction at sampling time, as in training (Lipman et al. [2022] Chen et al. [2025] Xie et al. 20224 Shaul et al.,
2024]. Table [ summarizes the sampling equations both differential and integral form for each prediction type, derived from Eq. and Eq. 4
Discretization discrepancy. We ask a simple question: do the prediction types yield the same update? In tinuous time, yes; in discrete time, no. Fig. - illus- trates this discrepancy in two dimensions. For example, applying the Euler method in the X-domain (half log- SNR) to noise prediction yields x
Noise Velocity Data
04
velocity
0t: OCe,
Et;t1
data
d1
w
noise
Tt. Ade, cg(xz,,ta) tz]]. where Ad,, := Ae -Àr,- Applying the same method data prediction and rewriting the result
data using Eq, 4 gives fill "1,+1 [(1 + AA;;) - Bt:
ax;
28
e AA or, Adr, Eg (at 4:)]. Since e -
Ad,,
O22 wgg
A +.... they differ at order (((AA,,)2)) )). This natu- rally raises the question of which update preferable in practice.
Figure 2: Euler updates for noise, ve- locity, and data predictions.
Table 1: Differential and integral forms for noise, velocity, and data predictions. (aş: signal rate, oz noise rate, A : log(a,/o,,)
Differential form
Integral form [+11
da
Noise
d d
dAs co(ar, t) df
dà cs(#r, 1) dt
dt dt
da:
plat:
Velocity
xo(x do. co(
Bs,4s Ct, Jer
do: 20(2,, t) + di color t) dt
dt
da. dt
M441
Data
dloge x, or (x t)
xt. + Jeeti
ar da a dt xo(2),
oc,
3

---

Published as a conference paper at ICLR 2026
3 DUAL-SOLVER
3.1 DUAL PREDICTION WITH PARAMETER Y
We propose a dual prediction scheme that uses both 2y and Ea together. We call dual because it treats 2a and separately, unlike velocity prediction, which bundles them into vg. Moreover, we introduce the following integral formulation parameterized by y, which interpolates between the integral forms of noise, velocity, and data prediction.
Integral form of dual prediction.
pfit
( Deo(x
2444 Ax, + B
Cxos(ze,) tt +
d
d dl ( for 2 0,
A (ot11//1:) B =0- C di ((.00)))
(5)
where
A = (ar44) " B = ann C=
d di (attt))
d D (ora?) for < O.
dt
has the following properties:
When ? the integral reduces the noise-prediction form (Table I
When O, the integral reduces to the velocity form (Table I
When 7 1, the integral reduces to the data-prediction form (Table
derive the differential form of dual prediction and, by integrating, obtain the integral form given above; the full derivation is provided in Appendix [. Another role of y is to control how much Ft contributes at increasing 7 decreases the coefficient A multiplying Ft whereas decreasing y increases A.
From Eq. 3
3.2 LOG-LINEAR DOMAIN CHANGE WITH PARAMETER T
Domain change. Let L (0, I be C2 diffeomorphism onto an interval I C R. To apply a change of variables to the integrals Eq. we define the following:
(u()) L(a or v(t) ((,'"") for 7 0,
(6)
((()) L(a, ((a)t)), v(t) Lo, a,") for 7 < 0.
By the chain rule, d8
-.-- -.- dr. and similarly for v. Thus, we obtain du di
mutt dL (u) du ag du frutt
gott dL dL)
D1441 Axx, + B
cv) de
du
we
Here, u, u(t;), Ui+1 u(tiu1) and vi v(t), ui41 vl++1). and B defined in Eq. and (u) == wa (xa (u)); u (2)) and ea = (0( (w)) v0 (v)).
Log-linear transform. Previous works (Dockhorn et al. 2022, Qinsheng & Chen 2023
Zhou
al. 2024) use the linear transform L(y) with noise prediction. Because du a I. n) the
integrand carries no weighting factor, making it straightforward to develop approximations such as Tavlor expansions and Lagrange interpolation. By contrast, other works (Lu et al. 2022a b Zhao et al. 2023 Xue et al. 2024) use a logarithmic transform L(y) log Because T. L
e". the integrand carres an exponential weight. A closed- sored-ffrmmooo approximation can be obtained via an ex- ponential integrator (Hochbruck & Ostermann 2010) or by using Lagrange interpolation. Motivated by these works, we propose a log-linear transform parameterized by a scalar T.
L(y;T) log(1 + Tyy
0.
(
This transform is invertible, with inverse L (u; (eTH - 1)/T; its weighting factor is du L (i; 7) e Consequently, it has the following properties:
As 0+: T(y:T) y, de 1--1(w;; T) When = 1: M(::T) = log(1 + v).
-. -.- e.
du
We apply the log-lineat transform to Eq. allowing separate parameters т, and t, for the u- and u-integrals, respectively. A comparison of domain-change functions is presented in Sec. 6.2.3
4

---

Published as a conference paper at ICLR 2026
3.3 APPROXIMATION WITH PARAMETER
On the interval [ug, 15+1]. we approximate Ty in two ways: (i) the first-order apprommatton we (z(u) (u,); and (īi) the second-order forward-difference approximation æs xz
(4;++ x Au;
Azo(u,] (u
wz). Here, (0((u)) = =0(("r-((u)) u (u:)), Aro (u:) == ag (uc+1) - 2 (w:), and Au; W1+1
ug. We also introduce K ((Au;; k,) = Ka, (An,)² an ((AAu,))) term, to allow additional flexibility in the residual term while preserving local accuracy. Ku is real scalar parameter that controls the
magnitude of the residual term. Applying the same approximations to Eg yields the following first-
order predictor pstd and second-order corrector 2nd corr.
pred. Ft,
Axr, + (oo + K K(Au; K.)) + Ea(oa\AL= + K K(Au;; km))
(9)
2nd-com l141
Axt, + B 200 (1. AL (u,)
(AL- + KK kw)
(10)
co(u:) AL (v:) + (Ea( (AL-1(u:) + K(Au; k.))
2
I, 1. and A and B are as defined in Eq. 5 When evaluating (u,), with Ot Tt., and already known, it can be obtained without writing 1.-1 explicitly. For example, for 7 0, we have L (u,) or. (see Eq. 6
Here,
= (w.)
Detailed derivations of the predictor and corrector are provided in Appendix B.2 and the corre- sponding local-accuracy theorem and its proof are given in Appendix
IMPLEMENTATION DETAILS
Dual-Solver performs sampling using medictor-correctto scheme (Butcher 2016) based on the equations developed in the previous section. We detail the sampling scheme in Sec. 4.1 and then present all learnable parameters in Sec. 4.2
4.1 SAMPLING SCHIME
Alg. I describes the sampling procedure of Dual-Solver. Sampling requires backbone that provides both Eg and eø: when only one prediction is available, the other can be obtained via Eq. 4 Given M steps, we use timesteps L with ln = and draw the initial noise Tto ~ (0, 7). We also use list l to store previous evaluations. Empirically, first-order predictor with a second-order corrector performs best, as shown in Sec. 6.22 At step i, the first-order predictor takes the current state æt, and the model evaluations (xx 44), c0( (.) to produce provisional sample 2' The second-order corrector then combines the evaluations at l; with fresh evaluations at 4 i.e., (asa (a'44. 1)+,), en (x' (i41) } to yield the corrected sample Zt;; At the final step i = M 1. the corrector not applied.
4.2 LEARNABLE PARAMETERS
prod prodd ..pred prod For each i-th step, the parameter sets are prod = (rp Tu,i Te,i Muus , ,Pugi and de = (ncorr n. --oor coore meoor) }. Thus, each step uses 2 x 5 = 10 parameters, except for the last step (1= M 1 1), which does not use the corrector and therefore has only parameters. Fig. shoows the learned parameters for the NFE=5 setting using DiT (Peebles & Xie| Xie) (023) backbone. Assuming backbones) and a first-order predictor with a second-order corrector (Sec. 4.1). o. greed and øooer determine the coefficients for an update that combines the current state Ft, and two model evaluations, co : t,) and 6o (x) k (+41) 1), to produce Σ;,-,- This may seem heavy, but Sec. 6.2.2 shows is necessary.
noise-prediction backbone (the same reasoning applies data- and velocity-predicte
In addition these parameters, we also learn the evaluation times {t. | (with to and tm fixed),
where M denotes the number of steps. Following prior work (Tong et al. 2025 Wang et al. 2025
we employ unnormalized step variables (AG), M4 and apply a softmax over = 0, -,..., M
to obtain normalized step sizes {At, ),M- M-1 (nonnegative and summing to one). The timesteps are
obtained via cumulative sum: to (lar- 4o) Ei-1 Ale, i= =, M
5

---

Published as a conference paper at ICLR 2026
Algorithm Dual-Solver predictor corrector crerector-correctoo
Algorithm 2 Hard-label classification for param-
sampling (Sec. 4.1)
eter learning (Sec. 5.2)
Require:
Diffusion backbone with dual predic- tion (arg, co) timesteps {4 } initial noise
Require:
Diffusion backbone with parameters 0, VAE decoder 'D, pretrained classifier C, solver S with parameters Φ, label dataset y learning rate
Pto empty list l. parameters
1: Evaluate {ang(I to); co (Zto: (o)) and
add to
1: while not converged do
2: for 1ori=00 1=
M do
2: Sample æt ~ _N(
initial noise
Tho_,, f Predictor (2, E; preed)
3:
3:
Sample y
class label
4:
i-=== - then break
4:
zo t -S(TT ;y, 0,
> sampling decoding
5: Evaluate (10 : (tt1),
iy + (D(xg)
c0(2/ 1i+1)) and add to
p C(ău)
b class probabilities
6: Ih.-1 Corrector p!oT)
7:
L + CrossEntropy(p.UU
> losss
7: end for
8:
Φ + -- VoL
b parameter update
8:
return 2/4
end while
SOLVER PARAMETER LEARNING
In this section, we review existing regression-basees parameter learning methods, identify their lim- itations, and introduce a classification-bsed approach. Fig. 4 provides schematic overview of all these methods. We apply them to the proposed Dual-Solver and report FID results for each method in Sec. 6.2.5
5.1 REGRESSION- SAADEESSOOO--SEED PARAMETER LEARNING
In regression- based learning, solver with trainable parameters is referred to student solver, while an existing fixed solver is referred to as a teacher solver. The student is then trained to imi- tate the behavior of the teacher running
high NFE. Most prior works adopt trajectory regres- sion (Shaul et al.] 2023 Zhou et al. 2024 Wang et al. 2025). which compares the trajectories generated by the teacher and the student, or sample regression (Shaul al., 2024). which compares only the final samples. Since comparisons in the trajectory sample space often show mismatch with visual perceptual quality, feature regression has been proposed (Tong et al.) 2025), in which the discrepancy is measured in feature space using metrics such LPIPS (Zhang et al. 2018). However, all of these methods require teacher solver and incur substantial overhead in preparing supervision targets, and they tend to perform poorly in the very low NFE regime (e.g., NFE S 5; see Table
5.2 CLASSIEICATION- PARAMETER LEARNING
Beyond feature regression, we consider labt- classification, where we apply a cross-entropy loss between the classifier outputs (probabilities) of the student and the teacher. This approach yields improved results in the low-NFE regime (Table 5). but it still requires teacher solver to generate the target probabilities.
+2
Figure 3: Learned parameters. Values of { Tux Toi Kags Koy across sampling steps, learned on DiT (Peebles & Xie 2023) at NFE = 5. See Figs. 13 and 14 for further results.

---

Published as a conference paper at ICLR 2026
Student solser Slary 19.
Hard-label clusificatio (propesed, Alg
Step
M
Classifier
probahilniess
latent
image
class label
Trajectory regression
Sample regression
Feature regression
Soft-label clussification
u02
latent un
image (4
Step
Step L-1
Decoder
Classifier
probabilities p!
Teacher solver (xrr; v.
Figure 4: Solver parameter learning methods. schematically illustrates trajectory, sample, and feature regression, as well as soft- and hard-label classification methods.
To remove this dependency, we further propose hard-label classification, which compares the stu- dent's classifier outputs (probabilities) against the input class label using a cross-entropy loss, with- out any teacher solver. The overall training procedure is described in Alg. 2 The sample æo gener- ated by the solver S is passed through the decoder and the classifier to obtain class probabilities p. We then compute cross-entropy loss between p and the class label y ~ Y, and update the solver parameters ø for all time steps via backpropagation. For text-to-ime -magg tasks, the class labels are replaced with text prompts, which can be obtained from datasets such MSCOCO 2014 (Lin et al. 2014) or MJHQ-30K eL. 2024), and the cross-entropy loss is replaced with CLIP loss (Radford al.
202T
Unlike regression to teacher targets, this method focuses on whether the generated samples lie on the correct side of the classifier's decision boundary, enabling less restrictive training. A potential issue is a mismatch between the distribution learned by the classifier and the ground-truth data distribution, but this can be mitigated by selecting an appropriate classifier. In Sec. 6.2.4 we examine the relationship between the classifier's 's accuracy and the resulting FID scores.
EXPERIMENTS
We benchmark Dual-Solver against two families of baselines:
Dedicated solvers: DDIM (Song et al. 2021a). DPM-Solver++ (Lu et al.
2022b).
Learned solvers: BNS-Solver (Shaul et al. 2024). DS-Solver (Wang et al. 2025
We select backbones that span diffusion and flow matching, covering both ImageNet (Deng et al.
2009) conditional generation and text-to-image tasks:
DiT-X1//-256x2556
(Peebles & Xie
2023y: diffusion, ImageNet.
PixArt-a XI-22-12
(Chen et al.
2023
diffusion, text-to-image.
GM-DiT 256x256
(Chen et al.
2025
flow matching, ImageNet.
SANA 600M-512px
(Xie et al.
2024 : flow matching, text-to-image.
Solver implementations are taken from official sources or reimplemented by us. The backbones can
be run via the diffusers library (von Platen al. 2022). Further details are provided in Appendix D
6.1 MAIN QUANTITATIVE RESULTS
We evaluate quantitative performance using FID (Heusel et al. 2017) and CLIP score (Radford 202I). For DiT and GM-DiT, FID computed on 50k images uniformly sampled across the 1,000 ImageNet (Deng et al. 2009) classes. For SANA and PixArt-a, FID and CLIP are com- puted on the MSCOCO 2014 (Lin al. 2014) validation set (30k image-caption pairs). The CLIP score is computed as the cosine similarity between text and image features. As described in Sec. 5 we train Dual-Solver with a classification-based objective. For DiT and GM-DIT, MobileNetV3 Large (Howard et al.) 2019 is used, and for SANA and PixArt-a, CLIP(RNI01) is used. Fig. 5 shows the measured FID and CLIP scores. Across all evaluated NFEs, Dual-Solver outperforms competing solvers both FID and CLIP for DIT, SANA, and PixArt-a: For GM-DiT, Sual-Solver underperforms at NFE 7-9; however, when trained with trajectory regression- based objective, surpasses the baselines at NFE 8 and 9 (Table

---

Published as a conference paper at ICLR 2026
300
DC
DCC Sohee
Kapa
Stans
(a) DIT, FID L (Table
(b) PixArt-a, FID & (Table
(c) PixArt-a, CLIP 1 (Table
Dual-shherr
DeekSShver
-3
Dual solver
04.22
0443
(d) GM-DiT, FID J. (Table
(c) SANA, FID (Table
(1) SANA, CLIP (Table
Figure 5: Main quantitative results. FID and CLIP score; evaluated on 50k (DiT/GM-DiT) and 30k (SANA/PixArt-a) samples; CFG: DiT=1.5, GM-DiT=1.4, SANA=4.5, PixArt-a=3.5.
6.2
ABLATION STUDY
6.2.1
PREDICTOR-CORRECTOR CONFIGURATIONS
Table 2:
Ablation of predictor-
We ablate the predictor-corrector configurations: p1 (first-order predictor only), plc2 (first-order predictor + second-order corrector), p2 (second-order predictor only), and p2c2 (second-order predictor + second-order corrector). The equations for the first-order predictor and second-order corrector are given by Eqs. |ndd [T]] respec- tively, and the equation for the second-order predictor provided in Eq. [2] As shown in Table [ p c2 yields the best performance across NFE = 3, 5, 7, and 9.
corrector configurations DiT.
Cross-entropy loss (1)
NFE
5
p1
0.667
0.225
0.183
0.175
plc2
0,574
0.197
0.178
0.173
p2
1.023
0.253
0.222
0.181
p2c2
5.009
0.317
0.203
0.191
6.2.2
PARAMETER SETTINGS FOR (7, kt)
We ablate the parameterization by fixing selected val- ues or sharing parameters to reduce the degrees of free- dom. Setting y to 1, 0, or -1 recovers data, velocity, and noise prediction, respectively; setting T yields removes the
Table 3: Ablation of the (7, 1c) parameter settings on DIT.
Cross-entropy loss (L)
the loglp transform; and setting K second-order residual term. We also tie T. = Tμ and 6u K₂ share parameters across the integrations for œy and ég. As shown in Table leaving all param- eters free yields the best performance at NFE and 5. Configurationss that fix or share certain parameters occasionally perform slightly better NFE and 9.
NFE
3
5
7
9
7 fixed
0.816
0.223
0.182
0.176
0 fixed
0.600
0.202
0.183
0.180
fixed
7,871
7.676
0.238
0.196
= fixed
0.601
0.217
0.175
0.178
K fixed
0.944
0.256
0,202
0.190
r. x. shared
0.667
0.221
0.177
0.169
all learnable
0,574
0.197
0.178
0.173
6.2.3 DOMAIN-CHANGE FUNCTION CHOICE
As candidates for the domain-change function, we consider Type 1, which linearly interpolates between the linear and logarithmic transforms: L(y: T) = (1 - T)y + T log y. and Type 2, defined in Eq. 8 In Table 4 Type 2 achieves notice- ably better results at NFE 5, 7, and 9. According to Eq [ the argument y of the function can become very close to zero; in this regime, the log function diverges, whereas log Ip is numerically stable near zero, and we attribute the improved performance to this stability.
Table 4: Ablation of Domain
Change Functions on DIT.
FID (L)
NFE
3
7
9
Type
23.38
4.09
3.56
3.64
Type
24.91
3.52
2.75
2.67
8

---

Published as a conference paper at ICLR 2026
6.2.4
CLASSIFICATION MODEL SELECTION
A natural question for classification-based learning is which classifier to use. To an- swer this, we evaluate 20 pretrained clas- sifiers from Vision (maintainers & contributors 2016). Fig. 6 shows FID (on T0k samples) versus accuracy for each model, ordered by rank. The curve exhibits clear V-shape: accuracy decreases, FID improves, but beyond ranks 14-16 the FID degrades sharply. This suggests that neither very high nor very low classifier accuracy is optimal: moderate level is most beneficial for FID. A detailed anal- ysis linking this pattern to precision and recall (Kynkäänniemi et al. 2019) is pro- vided in Appendix
20
19
Legend
18
17
15
16
9.14-1.3 LPPS
RID rank best). NFE
14
1.3:11 OF JFLPPS
13
>-11 GFLOPS
12
11
10
rank
10011223114111611118929
Classiler accuracy rank (1 bus))
Figure 6: Classifier Accuracy vs. FID on GM-DiT.
6.2.5 PARAMETER LEARNING METHODS
Table reports the results of applying the regression- and classification-basee based parameter learning methods discussed in Sec. 5 to Dual- Solver. Feature regression implemented us- ing LPIPS (Zhang et al. 2018). and it gener- ally outperforms trajectory- and sample-space re- gression. The improvement is particularly pro- nounced at NFE which classifier (AlexNet (Krizhevsky et al. 2012). VGG (Simonyan & Zisserman (0044), or SqueezeNet (landola al., (0T6)) is used tract features, the results vary significantly, under- scoring the importance of classifier choice in fea- ture space. Classification-ben based methods further improve performance over feature regression NFE and 5. In particular, the method that uses hard labels achieves the best results across all NFEs. We attribute this to the choice of an appropriate classifier, as discussed in Sec. 6.2.4
Table 5: Comparison of parameter learning methods on DIT.
FID (J)
NFF
3
5
7
9
Regression- learning (Scc.
5.1
and 5. Depending on
Sample
107.13
11.71
4.60
2.99
Trajectory
100.89
11.59
3.66
2.84
Feature (AlexNet)
47.75
7.24
3.42
2.91
Feature (VGG)
41.58
5.48
3.23
2.88
Feature (SqueezeNet)
44.00
7.07
3.31
2.79
Classification- learning (Sec. 5.2
Soft-label Hard-label
25.13
4,90
3.37
3.01
24.91
3.52
2.75
2.67
6.3 PARAMETER INTERPOLATION ACROSS NFES
From the learned parameters of Dual-Solver (Ap- pendix F). we observe that their overall shapes main similar across different NFEs. Motivated by this, test the robustness of the learned parameters by applying them to different NFEs. To obtain the parameters for an unseen NFE, we interpolate be- tween the parameters of its two neighboring NFEs. The detailed formulas are provided Appendix G Fig. 7 shows the results. Interp. (3,5), (5,7), (7,9) denotes that we interpolate between the parameters learned at NFEs (3.5), (5,7), and (7.9) to obtain the parameters for the intermediate NFEs 4, 6, and 8, re- spectively. The other interpolation schemes are de- fined analogously. Although these interpolated pa- rameters do not match the performance of parame- ters directly optimized for each NFE, the gaps are modest, and the resulting FID scores still outper- form those of other solvers.
Dual Salver
DS-Sclver
10
ByS-Solver
DOM
(5,7),
Interp. (3,6), (6,9)
Interp. (3,9)
Steps
Figure 7:
FID results for parameter interpo- lation across NEEs with DIT.

---

Published as a conference paper at ICLR 2026
(a) + Lu al. 2022b
(b) Dual-Solver (Ours)
(c) BNS- Solver
(d) DS-Solver
Shaul et al.
2024
(Wag al. 2025
Figure 8: Sampling results. PixArt-a results.
Chen et al. 2023 NFE=5, CFG=3.5. See Fig. 19 for further
(a) DPM-Solver++ +
(Lu et al.
2022b)
(b) Dual- Hual-Solver (Ours)
BNS-Solver Shaul et al. 2024
(d) DS-Solver Wang el al. 2025
Figure 9: Sampling results. GM-DiT Chen al. 2025 NFE=3, CFG=1.4. See Fig. 17 for further results.
7 CONCLUSIONS AND LIMITATIONS
This paper introduces Dual-Solver, predictor-correttee sampler that achieves second-order numer- ical accuracy. It features per-step learnable parameters (7, x) that govern the prediction types, the integration domain, and the second-order residual adjustment. All solver parameters are optimized end-to-end with classification-bass. objective using a pretrained image classifier. Across diverse diffusion and flow matching backbones, experiments show substantial improvements over compet- ing solvers in the low-NFE regime NFE < 9), as measured by FID and CLIP score. Limitations include the absence of unconditional backbones and lack of analysis beyond second-order accu- racy; both are left for future work.
ACKNOWLEDGEMENTS
We would like to thank Juhee Lee, Jiyub Shin, and Kyungdo Min for their helpful discussions and careful review of the manuscript. This research partially supported by MODULABS. This work was also supported by the Basic Science Research Program through the National Research Foundation of Korea (NRF), funded by the Ministry of Education (Grant No. RS-2025-25424642)
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Martin Arjovsky, Soumith Chintala, and Léon Bottou. Wasserstein generative adversarial networks. In International conference on machine learning, pp. 214-223. PMLR, 2017.
Francis Bashforth and John Couch Adams. An Attempt to Test the Theories of Capillary Action by Comparing the Theoretical and Measured Forms of Drops of Fluid: With Explanation of the Method of Integration Employed in Constructing the Tables. Cambridge University Press, Cambridge, 1883.
John Charles Butcher. Numerical methods for ordinary differential equations. John Wiley & Sons, 2016.
Hansheng Chen, Kai Zhang, Hao Tan, Zexiang Xu, Fujun Luan, Leonidas Guibas, Gordon Wet- zstein, and Sai Bi. Gaussian mixture flow matching models. arXiv preprint arXiv:2504.05304, 2025.
Junsong Chen, Jincheng Yu, Chongjian Ge, Lewei Yao, Enze Xie, Yue Wu, Zhongdao Wang, James Kwok, Ping Luo, Huchuan Lu, al. Pixart-o: Fast training of diffusion transformer for photore- alistic text-to-image synthesis. arXiv preprint arXhv:2310:00422
2023.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. ImageNet: A large-scale hier- archical image database. In 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248-255. IEEE, 2009.
Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. Advances in neural information processing systems, 34:8780-8788 2021.
Laurent Dinh, David Krueger, and Yoshua Bengio. Nice: Non-linear independent components esti- mation. arXiv preprint arXiv:1410.8516, 2014.
Tim Dockhorn, Arash Vahdat, and Karsten Kreis. Genie: Higher-order denoising diffusion solvers. Advances in Neural Information Processing Systems, 35:30150-30166, 2022.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010. 1929, 2020.
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial networks. Communications of the ACM, 63(11):139-144 2020.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770-778, 2016.
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. GANs trained by two time-scale update rule converge to a local Nash equilibrium. In Is- abelle Guyon, Ulrike von Luxburg, Samy Bengio, Hanna M. Wallach, Rob Fergus, N. Vishwanathan, and Roman Garnett (eds.), Advances in Neural Information Processing Systems, volume 30, pp.
Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. In NewrIPS 2021 Workshop on Deep Generative Models and Downstream Applications, 2021.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems (NeurIPS), 33:6840-668 6851, 2020.
Jonathan Ho, Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi, and David J Fleet. Video diffusion models. arXiv preprint arXiv:2204.03458, 2022.
Marlis Hochbruck and Alexander Ostermann. Exponential integrators. Acta Numerica, 19:209-286. 2010.
11

---

Published as a conference paper at ICLR 2026
Andrew Howard, Mark Sandler, Grace Chu, Liang- Chig- Chen, Bo Chen, Mingxing Tan, Weijun Wang, Yukun Zhu, Ruoming Pang, Vijay Vasudevan, et al. Searching for mobilenetv3. In Pro- ceedings of the IEEE/CVF international conference computer vision, pp. 1314-1324, 2019.
Andrew Howard, Menglong Zhu, Bo Chen, Dmitry Kalenichenko, Weijun Wang, Tobias Weyand, Marco Andrectto, and Hartwig Adam. Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv: (:X.::/7740048/1/ 2017.
Forrest N landola, Song Han, Matthew W Moskewicz, Khalid Ashraf, William J Dally, and Kurt Keutzer. Squeezenet: Alexnet-level accuracy with 50x fewer parameters and; 0.5 mb model size. arXiv preprint arXiv: arXiv: /602.07360 2016.
Gabriel Ilaarco, Mitchell Wortsman, Ross Wightman, Cade Gordon, Nicholas Carlini, Rohan Taori, Achal Dave, Vaishaal Shankar, Hongseok Namkoong, John Miller, Hannaneh Hajishirzi, Ali Farhadi, and Ludwig Schmidt. Openclip. July 2021. URL https: dei. 1ttp://222..1911..2211111
021377.5143773
Tero Karras, Miik Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion- based generative models. Advances in neural information processing systems, 35:26565-26577, 2022.
Diederik Kingma and Max Welling. arXiv:1312.61 14, 2013.
Auto-encoding variational bayes.
arXiv preprint
Durk P Kingma and Prafulla Dhariwal. Glow: Generative flow with invertible Ixl convolutions. Advances in neural information processing systems, 31, 2018.
Weijie Kong, Qi Tian, Zijian Zhang, Rox Min, Zuozhuo Dai, Jin Zhou, Jiangfeng Xiong, Xin Li, Bo Wu, Jianwei Zhang, et al. Hunyuanvideo: A systematic framework for large video generative models. arXiv preprint arXiv:2412.03600 2024.
Zhifeng Kong, Wei Ping, Jiaji Huang, Kexin Zhao, and Bryan Catanzaro. Diffwave: A versatile diffusion model for audio synthesis. arXiv preprint N.Xi:22009.0966: 2020.
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convo- lutional neural networks. Advances in neural information processing systems, 25, 2012.
Tuomas Kynkäänniemi, Tero Karras, Samuli Laine, Jaakko Lehtinen, and Timo Aila. Improved precision and recall metric for assessing generative models. Advances in neural information processing systems, 32, 2019.
Daiqing Li, Aleks Kamko, Ehsan Akhgari, Ali Sabet, Linmiao Xu, and Suhail Doshi. Playground v2. 5: Three insights towards enhancing aesthetic quality in text-to-image generation. arXiv preprint ar WxXi:2242.112444 2024.
Tsung-Yi Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects context. In Computer Eision-ECCV 2014: 13th European Conference, Zurich, Swirzerland, September 6-12, 2014, Pro- ceedings, Part V 13, pp. 740-755. Springer, 2014.
Yaron Lipman, Ricky TQ Chen, Heli Ben-Hamu, Maximilian Nickel, and Matt Le. Flow matching for generative modeling. arXiv preprint 27X10:2210.02747, 2022.
Haohe Liu, Zehua Chen, Yi Yuan, Xinhao Mei, Xubo Liu, Danilo Mandic, Wenwu Wang, and Mark D Plumbley. Audioldm: Text-to-audio generation with latent diffusion models. arXiv preprint arXiv:2301. 2023.
Luping Liu, Yi Ren, Zhijie Lin, and Zhou Zhao. Pseudo numerical methods for diffusion models on manifolds. arXiv preprint 20ii:2202.09778 2022.
Ilya Loshchilov and Frank Hutter. Sgdr: Stochastic gradient descent with warm restarts. arXiv preprint arXiv: /608. 03988, 2016.
12

---

Published as a conference paper at ICLR 2026
Iyy Loshchilov and Frank Hutter. Decoupled weight decay regularization. arXiv preprint arXiv:1711.0S101, 2017.
Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu. Dolesolver: A fast ode solver for diffusion probabilistic model sampling in around 10 steps. Advances in Neural Information Processing Systems, 35:5775-5787, 2022a.
Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu. Dpm-solver++: Fast solver for guided sampling of diffusion probabilistic models. arXiv preprint arXiv:221..00095 2022b.
Torch Vision maintainers and contributors. Torchvision: Pytorch's computer vision library. https:
wttp http./c..//tttroc//iiii....oo 2016.
William Peebles and Saining Xie. Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 4195-4205, 2023.
Zhang Qinsheng and Yongxin Chen. Fast sampling of diffusion models with exponential integrator. ICLR, 2023.
Alec Radford, Jong Wook Kim, Chris Hallaey, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pp- 8748-8763, PMLR, 2021.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image synthesis with latent diffuusio models. In Proceedings of the IEEE/CVF confer- ence on computer vision and pattern recognition, pp. 10695, 2022.
Carl Runge. Ueber die numerische auflösung differeemiallleeccunngen Mathematische Annalen, 46:167-178, 1895. URL/atp:: http::////eemmm.oooood/oo///////
Tim Salimans, Andrej Karpathy, Xi Chen, and Diederik P Kingma. Pixelenn++: Improving the pixelcnn with discretized logistic mixture likelihood and other modifications. arXiv preprint arXiv:1701.055: 2017.
Neta Shaul, Juan Perez, Ricky TQ Chen, Ali Thabet, Albert Pumarola, and Yaron Lipman. Bespoke solvers for generative flow models. arXiv preprint arXiv:2310.19075 2023.
Neta Shaul, Uriel Singer, Ricky TQ Chen, Matthew Le, Ali Thabet, Albert Pumarola, and Yaron Lipman. Bespoke non-stationary solvers for fast sampling of diffusion and flow models. arXiv preprint arXiv:2403.01329, 2024.
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv: 1559, 2014.
Jascha Sohl- bohl-Dickstten, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In International Conference on Machine Learn- ing, pp. 2256-2265. PMLR, 2015.
Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. In Interna- fional Conference on Learning Representations, 2021a. a.
Yang Song and Stefano Ermon. Generative modeling by estimating gradients of the data distribution. Advances in neural information processing systems, 32, 2019.
Yang Song and Stefano Enmon. Improved techniques for training score-based generative models. Advances in Neural Information Processing Systems, 33, 2020.
Yang Song. Jascha Sohl-Dickstein Diederik P. Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Scsee generative modeling through stochastic differential equations. In Interna- fional Conference on Learning Representations, 2021b.
13

---

Published as a conference paper at ICLR 2026
Vinh Tong, Trung- Mung--Dng Hoang, Anji Liu, Guy Van den Broeck, and Mathias Niepert. Learning to discretize denoising diffusion odes. In International Conference on Learning Representations (ICLR), 2025.
Arash Vahdat and Jan Kautz. Nvae: A deep hierarchical variational autoencoder. Advances in neural information processing systems, 33:19677-1967, 2020.
Aaron Van den Oord, Nal Kalchbrenner, Lasse Espeholt, Oriol Vinyals, Alex Graves, et al. Con- ditional image generation with pixelenn decoders. Advances neural information processing systems, 29, 2016.
Patrick Platen, Suraj Patil, Anton Lozhkov, Pedro Cuenca, Nathan Lambert, Kashif Rasul, Mishig Davaadorj. Dhruv Nair, Sayak Paul, William Berman, Yiyi Xu, Steven Liu, and Thomas Wolf. Diffusers: State-of-the-art diffusion models. http:: /// / htggtthuub... wwt.c//ggg...cceeeeeeececeooeeee diffusers 2022.
Shuai Wang, Zexian Li, Tianhui Song, Xubin Li, Tiezheng Ge, Bo Zheng, Limin Wang, et al. Dif- ferentiable solver search for fast diffusion sampling. arXiv preprint arXiv:2505.2/114, 2025.
Enze Xie, Junsong Chen, Junyu Chen, Han Cai, Haotian Tang, Yujun Lin, Zhekai Zhang, Muyang Li, Ligeng Zhu, Yao Lu, et al. Sana: Efficient high-resolution image synthesis with linear diffusion transformers. arXiv preprint arXiv:2410. 10629, 2024.
Shuchen Xue, Mingyang Yi, Weijian I Luo, Shifeng Zhang, Jiacheng Sun, Zhenguo Li, and Zhi-Ming Ma. Sa-solver: Stochastic adams solver for fast sampling of diffusion models. Advances in Neural Information Processing Systems, 36, 2024.
Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEFE conference on computer vision and paitern recognition, pp. 586-595, 2018.
We Zhao, Li Bai, Yong Rao, Jian Zhou, and Jun Lu. Unipe: A unified predictor-correcto framework for fast sampling of diffusion models. arXiv preprint arXiv:2302.04453 2023.
Zhenyu Zhou, Defang Chen, Can Wang, and Chun Chen. Fast ode-based sampling for diffusion models in around 5 steps. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 7777-7786, 2024.
14

---

Published as a conference paper at ICLR 2026
CONTENTS
Introduction
2
Preliminaries
Diffusion Models
2.2
Prediction Types
3
Dual-Solver
3.1
Dual Prediction with Parameter
3.2
Log-Linear Domain Change with Parameter 7
3.3
Approximation with Parameter
implementation Details
4.1
Sampling Scheme
Learnable Parameters
5
Solver Parameter Learning
5.1
Regression-based Parameter Learning
5.2
Classfiiationnbusedd Parameter Learning
6
Experiments
6.1
Main Quantitative Results
6.2
Ablation Study
6.3
Parameter Interpolation across NFES
Conclusions and Limitations
10
LLM Usage
17
Derivations
17
I
Derivation Dual Prediction
17
B.2 Derivation of Sampling Equations
18
Local Truncation Error
19
Local Truncation Error of First-Order Predictor
19
Local Truncation Error of Second-Order Predictto
20
Local Truncation Firor of Second-Order Connectoo
21
D Experimental Details
23
D.I Setup
23
D.2 Quantitative Results
23
15

---

Published as a conference paper at ICLR 2026
Classifier Accuracy Sample Quality
25
Learned Parameters
27
Details of parameter interpolation across NFEs
16

---

Published as a conference paper at ICLR 2026
A
LLM USAGE
We disclose that Large Language Models (LLMs) were used as assistive tools for:
verifying the consistency of mathematical derivations, assisting literature search and surfacing related work,
helping with formal writing (style, grammar, typos),
drafting figure and table scripts.
All scientific ideas, methods, and results originate from the authors, who take full responsibility for the content of this paper.
B DERIVATIONS
B.1 DERIVATION OF DUAL PREDICTION
Differential Form of Dual Prediction. in Eq. 5 is given by
The differential form corresponding to the integral form
dæ:
dolog at
dlog o
Brr + an
20 1) + oz
c0 1).
(11)
dt
dt
dt
To derive this form, we start from the probability-flow ODE Eq.
dæ,
1
dl Fe
2 D6 ve log ((xx),
dl doog 04
d o -f
<o(a))
where ft
of
V logg(z)))
dt
dl
0r
Substituting fr, o². and V log ((x;) yields:
dæ,
doog at dt v. +
d dlog at
of en ca(2))
0,
dt
20: dt.
dt
d dloga at
dlogor or
dlog an dt dt
ae + az
co t).
dt
Introducing an arbitrary 3 E R. we can rewrite:
dæ, Sxx + ddlog 0t B x B)22)
dlog or
d log 04
+ oi
(0(
dt
dt
dt dt
Bax + (dlog oz
(dlogo:
d at
(f dt
B) (o, meeo 1)) + oi
co( ()
dt dt
dlogo az - 3 ao - f) + of dt
d log az
B.0 + on dt
co(
Thus, we obtain the differential form of dual prediction. is straightforward to verify that choosing log Ou, recovers the noise-prediction form,
log a, recovers the data-prediction form,
and B = O recovers the velocity- form in Table
Integral Form of Dual Prediction. We next apply the variation-of-constan method Eq. to obtain the following integral representation:
L1,41 = =exp(
Bu Bud xt. +
exxo (rall Ba du)
|as /dogaa as
dlog o.
Br) xs + o. ds
3,) ds.
ds
17

---

Published as a conference paper at ICLR 2026
We reparameterize 3 in terms of new variable y R follows:
dlog de di 7
7 2 0,
8(7)
dlog ar dt = d
7 0.
Then we obtain the 7-interpolated integral form of dual prediction (Eq. 5
B.2 DERIVATION OF SAMPLING EQUATIONS
B.2.1 DERIVATION OF FIRST-ORDER PREDICTOR
First, we take a first-order approximation æz(()) and eœ(u) in Eq. 7
Bt,t1 Amt, + B ro (o( AL (u:) + co(u;) co( (1s)
Here, we write no (u;) == xo definitions for ea(2)) and AL A are analogous.
u u (u:),, and (us)(us) = L- (u L (u;); the
Next, while preserving first-order accuracy, we incorporate the K ((Au:Ka)) Ol(Au,))) and K (Aeg Ke) = (((Ae) which yields the first-order predictor of Dual-Solver (Eq. 9
B.2.2 DERIVATION SECOND-ORDER PREDICTOR
First, we approximate ro(u) and (œ(u) at uy and v; by a second-order backward- nackward-difference expan- sion.
A
putts
(u us) dl. dI.- du
Ft.- Axi, + B Al (u,)
Au--1
Jus
du
(E
po4t
dL dL- du
(w- - v.)
Ar
dv
24
Using
1
uit1
dL (I) du
(u u.)
Au Ju.
du
1
dL (v-
Aus O((Au:)))
2r (u)
du
144
(AL (AL-1(u)) + + +(((Au)))
2r
where (w)
Au;-1 and
(v)
Ar
we obtain
Au.
Ae,
+ B B Amo(ui-1) (u)+
Dt.t Ax,,
2r("))
AEØ(U:-1) (AT- O((Av:)))
ca(m)AL-
(v:)
2r
Next, while preserving second-order accuracy, incorporate the K(Au;; k..) O((Au))) and ((An;; ky) = O(Au,)?) residual terms, which yields the following second-order predictor of Dual-Solver.
AZo(ui-1) (AL (us) + K(Au;: kw)) 2x((a)
2nd-pred. Axı, Et41
+ B x0( AL
(12)
Acp(Ui-1) (0( (v)
+ co( AL
(LL-1(e)) + K(Ar;s kx))
18

---

Published as a conference paper at ICLR 2026
B.2.3 DERIVATION OF BECOND-ORDER CORRECTOR
First, we approximate 2e (u) and (b( at u; and e; by second-order forward-difference expansion.
Axo(us)
Au J.m (u us) dL (L_((a) du +
B4+t 2 Ax B xo(u,)AL (u:) +
du
Aco(U:)
S (v 0x)
dL (u)
co(o))AL-1(0 +
du
Aus
dv
Using
Mi11
dL (u) du
(u - u.)
Aug
du
dl dL-11()
Aus + O((Au:)))
2
du
(AL (A,))(u) + (((u)))))
obtain
B oo Axo(u,)
Bta11 Aw,,
(AL (w:) (((A.))))
+T. + (E (AT. (v:) + (((An))))
2
Next, while preserving second-order accuracy, incorporate the K K(Au;; k..) O (((Au))) and
K (Ang Ke) (((Av))) residual terms, which yields the second-ordes corrector Dull-Solver
(Eq.
10
C LOCAL TRUNCATION ERROR
C.I LOCAL TRUNCATION ERROR OF FIRST-ORDER PREDICTOR
Theorem C.1. Assume that ma(u) and u)) are C³ on [us, uital and [ui, 0.41). respectively. Let Ist-pred. denote the first-order predictor defined Eas denote the exact update in equation and let t4,11 in equation Then we have
Ist-pred
((Au,) + (Av;)2).
6.+1
Proof. From Eq.
7 the exact update can be written as
Acc + B Bh + Io]
where Ly (1 d1. (1))
du Dg (u) du and La = 1 dr. dr)
(u( do.
du
Eq. defines the first-order predictor
Ist-pred. Ax + B I' + [ x F6,41
with I, (044))) (AL-'(w:) + K(Au; k.)) and Ir (u();) (AL-1(v)) K(Aun
The accuracy be obtained by taking Taylor expansion of T, and estimating the order of T, P' Expansion of L
dL- dL- x (u)du
In du
(o ( + O(Au,))))
19

---

Published as a conference paper at ICLR 2026
Order estimate of Ia I'
I - = =((u)))) (oo( K (Auu; Ku).
Since zo(u,) = (1)) and K (Aus N.)) = O((Au)))). it follows that xo(u,) K K(Au; Nu) O ((Au)))).
Therefore,
Iu =
The bound - 11 (((s))))) can be derived in the same way.
Conclusion
exact 2 1,41
lst-pred.
BB( - 1' + (1. 11)
1441
=B(O((Au:)) + O((An))))
=O((Au;) (Av))3).
C.2 LOCAL TRUNCATION ERROR OF SECOND-ORDER PREDICTOR
Theorem C.2. Assume that so(u)) and ea (v) are C² on [ui- (4--1,05-1) and [w:-1, 0s+1]. respectively. pred denote the second-order predictor Let aeee denote the exact update in equation and let tt defined in equation 12 Then we have
exaee b,41
2 b141
2nd-pred.
|| ((Au)) (A
Proof. From Eq.
7 the exact update is given by
T ti+1
exact Axt.
+ B [Lu + Io]
where Ia
f
dL dL"'(w) (u)
du Wg (u) du and I. = Suit
dL-'(() (v)
(σ(v) dv.
du
Eq. 12 defines the second-order predictor
2nd 2nd pred.
Axi, + B I' + [.].
with 1 = x (u:) AL (u:) + Amp Den(w--)) 2r!"0
(AL-)(u) + K (Auu kow)) and 1 = E0(2)) AL AL))))+
2r!"))
( (o:) + K(Ar;; k.)).
The accuracy can be obtained by taking a Taylor expansion of I, and estimating the order of 12 - -
Expansion of T
Ia = furt putt= dL- xo(u)) du
du
xx(4)) AL (w:) + f muEl d
(zp(u) - xe(u)) du
du
raits (uitt(u (u-ui) u.) dL dL (u) du + ((d4)))))
xx)(u:) AL (w: + 2/b(4))
du
=mo(us) AT. AT. (14) + ab((us) Au AI. AI. O(Au:)))
=00(4:) AL- (w:) + 2r (u) AZD ( AL + (
20

---

Published as a conference paper at ICLR 2026
Order estimate of Ia I'd Iu r' = (((u;))))
Amp((:-) K (Aus; Ku))
2r: (1)
Since Aws (u:i1= O(Au;) and K ((Aui O((Au;))); follows that
Ax K(Au;i K.N) = O((Au))))
Therefore,
Iu -,
The bound I, - 1 = ((Au;)))) can be derived in the same way.
Conclusion
exact T t241
2 t.41
2nd-pred. =
Blu II,) + (Tu - [.) =B(o((Au:))) + O((Av:)))
=O((A))) + (A11)))).
C.3 LOCAL TRUNCATION ERROR OF RECOND-ORDER CORRECTOR
Theorem C.3. Assume that mg(u) and en cg(u) are C2 on [us, uct1] and [Vi, 0e+1), respectively. Let 2nd- denote the second-order corrector ponnt denote the exact update in equation 7 and let ai,41
defined in equation TO Then have
2nd-corz
((Am)))) + (Av:)
Proof. From Eq.
7 the exact update is expressed as
Axt;
+ B [L + Lo]
where Ly f
dL (1) (u) du and L. f dI (v) du
De
Eo(U) dc.
du
Eq. definns the second-order corrector
2nd-corr. Ax,, + B[I, + I.].
x t:
where T/ (o(()) AT. (u;) + Ama(ue) Sealw))
(AI.-1(u;) + K(Aug and V = =E0 <)())) AT. Av))
(AL (N,) + K(Au;: k.))) k,)).
2
The accuracy can be obtained by taking Taylor expansion of J, and estimating the order of J, I'--
Expansion of Iu
piit1 dIL-I(u)
Ig Ju
xo(u)du
du
ri+i du) (zo(l) - =u:() du
= =t.
du
=xo(u)AL (v.) + x'blu)) (ui (w-w)) u.) dl dL (u)
du + O((Au)3)
du
1
= =x (24) +
+ ((Au))
2
1
- =xo(u,AL -(4)) + Aoo A (w,)) + O(Au,)))
2
21

---

Published as a conference paper at ICLR 2026
Order estimate of Ia I'
1
1' = (A(m))))) Ammo(us (us) K K(Au;; Ku).
2
Since Aas(u) (u;) = (A);) and K (Auu;Kt)) = O((Au;))) }), it follows that Ama(u:) K(Au); (AAu,))).
Therefore,
Iu - =(AA)))
The bound I, - =o((=) O((us))))
can be derived the same way.
Conclusion
exnct 1i+1
2nd-enrr.
B[( 12,) (I, [,)] =B(o((Au:))) + +((Av))))]
=O((Au,)² + +(vv))²).
22

---

Published as a conference paper at ICLR 2026
D EXPERIMENTAL DETAILS
D.1 SETUP
Environment details. We run all experiments
single NVIDIA RTX 6000 pro (Driver 575.57.08) under Ubuntu 24.04 with Python 3.11.13, PyTorch 2.8.0, and CUDA 12.9.
Backbone Details. We evaluate DiT-XT/2 (Peebles Xie 2023), GM-DiT (Chen et al.) 2025). SANA (Xie al. 2024), and PixArt-a (Chen et al. (023). All models are obtained via the diffusers library (von Platen et al. 2022) pipelines and run in evaluation mode with b6loat16. The model identifiers
DīT: facebook/D1T-Xi-22233
GM-DiT: haw.m//nk//mmmmooo..omm.eeteeee.
SANA: SAAAA//T..ccaccarrrreeoo///////a
PixArt-a: Pttppp/ppataaaappp//aaaa--------
Solver Details. The solvers used in our experiments are the diffusion-dedicated DDIM (Song et al. 202 the second-orde multistep DPM-Solver++ (Lu et al. 2022b). and the learned solvers (Shaul et al. 2024) and DS-Solver (Wang et al. 2025). We use DDIM the first- order counterpart of
(as proposed in Lu et al. (2022a)). Implementations of DPM- Solver++ and DS-Solver taken from their official GitHub repositorie and BNS-Solver is implemented according to the paper.
Learning Details. We train with AdamW (Loshchilov & Hutter 2017 using 3 (0.9, 0.999), The
= 1e-8, 8, and weight decay 0.01. The learning rate decays from 2 TO to 1 1 10 over 20k steps cosine annealing (Loshchilov & Hutter (016). The batch size is fixed to 10 for all experiments. For regression-based learning, the teacher trajectory is 200-step DDIM (Song et al. 2021a) method
on 1k samples.
Sampling Details. We evaluate all backbones
NFE grid of 3, 4, 6, 7,89. For classifier- free guidance (CFG; Ho & Salimans] 2021), we use backhone-specifi. fixed scales: DiT 1.5, GM-DiT 1.4, SANA 4.5, and PixArt-a 3.5. We compute FID using the publicly released ImageNet training-set set statistics (Dhariwal & Nichol 2021) for ImageNet, and MSCOCO 2014 validation-set statistics for text-to-image (SANA, PixArt-a). CLIP scores are computed with the official RN101 variant (Radford et al. 2021 We generate 50k images for ImageNet and 30k images for text-to- image evaluation.
D.2 QUANTITATIVE RESULTS
Table DiT
Peebles & Xie
2023
FID ({) vs. NFE. ImageNet generation, 50k samples.
Method
3
4
5
6
7
8
9
DDIM (Song et al. 2021a)
89.33
56.33
32.91
20.06
13.64
9.55
7.42
DPM-Solver++ Lu et al.
2022b)
88,46
47.64
22.19
11.49
7.06
5.19
4.43
BNS-Solver Shaul et al. 2024
103.26
38.20
14.53
6.37
4.25
3.37
3.05
DS DS-Solver(W)) Wang et al.
2025
67.31
17.31
7.66
5.46
3.79
3.08
3.02
Dual-Solver (Ours)
24.91
6.05
3.52
3.13
2.75
2.60
2.67
ver
23

---

Published as a conference paper at ICLR 2026
Table 7: GM-DiT Chen al.
2025 : FID (4) vs. NFE. ImageNet generation, 50k samples.
Method
4
5
6
8
9
DDIM (Song et al. 2021a)
70.15
35.98
19.70
12.55
9.03
7.01
5.78
DPM-Solvert+ Lu et al.
2022b
63.24
19.53
7.85
4.74
3.65
3.15
2.89
BNS SNS-SolverrShaul et al. 2024)
57.88
26.64
10.31
5.02
3.43
2.66
2.44
DS-Solver Wang et al.
2025)
34.15
11.60
5.64
3.49
2.70
2.48
2.41
Dual-Solvers (Ours)
45.53
14.43
7.49
3.75
2.77
2.44
2.32
Dual-Solver-C (Ours)
6.81
3.76
3.09
2.97
2.77
2.70
2.60
R = trajectory regression- based;; C classification-based.
Table 8: PixArt-a (Chen et al.
2023 FID (L) and CLIP score (1) vs. NFE. Text-to-image on
MSCOCO 2014 (Lin al. 2014 with 30k samples.
Method
7
FID (L)
DDIM (Song al. 2021a
71.37
45.21
35.12
30.92
28.60
27.39
26.58
(Lu ct al. 2022b
76.01
41.77
31.48
27.83
26.56
25.82
25,48
BNS-Solver (Shaul et al. 2024
125.65
66.94
41.31
32.55
28.55
25.18
24.15
DS Solver (Wang et 2025
118.01
66.09
43.62
29.74
28.25
26.57
25.22
Dual-Solver (Ours)
66.61
31.61
24.68
22.39
22.51
21.96
22.01
CLIP (RN101, 1)
DDIM (Song al. 2021a
0.4469
0.4670
0.4739
0.4764
0.4763
0.4778
0.4779
DPM-Solver (Lu al.
2022b
0.4422
0.4676
0.4746
0.4763
0.4768
0.4771
0.4771
BNS-Solver (Shaul et al. 2024
0,4320
0.4582
0.4694
0.4733
0.4746
0.4755
0.4757
DS-Solwe Solver (Wang et al.
2025
0,4303
0.4568
0.4692
0.4748
0.4754
0,4762
0.4764
Qual-Solvee (Ours)
0.4499
0.4732
0.4784
0.4803
0.4806
0.4814
0.4815
Table 9: SANA (Xie al.
FID (L) and CLIP score (RN101, 1) NFE. Text-to-image 2014) with 30k samples.
2024):
MSCOCO 2014 (Lin et al.
Method
3
7
8
FID (L)
DDIM (Song al.
2021a
45.05
27.72
23.93
23.06
22.99
22.96
22.97
DPM DPM-Solver++ (Lu
2022b
45.33
26.12
22.56
22.48
22.79
22.90
23.11
BNS Solver (Shuul et al.
2024
48.16
26.37
21.04
20.66
20.79
21.13
21.64
DS-Solver (Wang et al.
2025
48.65
29.15
21.66
20.65
21.43
21.80
22.27
Dual-Solved (Ours)
21.79
19,40
18.81
18.52
19.57
19.43
19.77
CLIP (t)
DDIM (Song et al. 2021a
0.4656
0.4782
0.4813
0.4821
0.4824
0.4824
0.4824
DPM- + (Lu
2022b
0.4652
0.4779
0.4813
0.4821
0.4822
0.4820
0.4820
BNS Solver (Shaul et
2024
0.4651
0.4772
0.4800
0.4807
0.4808
0.4809
0.4808
DS Solver (Wang et al.
2025
0.4635
0.4765
0.4808
0.4816
0.4818
0.4818
0.4816
Dual Solver (Ours)
0.4795
0.4801
0.4821
0.4837
0.4843
0.4844
0.4849
24

---

Published as a conference paper at ICLR 2026
E CLASSIFIER ACCURACY VS. SAMPLE QUALITY
As examined in Sec. 6.2 we study the relationship between classifier accuracy and FID. In this section, Table I0 provides detailed numerical results, and Fig. I0 analyzes the trend from the per- spectives of precision and recall (Kynkäänniemi et al. 2019). Table rerrrts the pretrained weights from Torch Vision (maintainers & contributors ( and the results of training Dual-Solver with each set of weights. Using a GM-DiT (Chen et 2005)) backbone, we present FID, precision, and recall at NFE= 3 and 9. The best value for each metric is highlighted in bold.
Table 10: ImageNet per-classifier metrics. Top-5 accuracy. FID, precision/recall at NFE=3 and 9, and GFLOPs; FID is measured on 10k samples after training Dual-Solver for each classifier.
Weighe
Acc-5Acc. (%)
F
F1D69
Precisiene)
Precitonn29
Recalle/3
Recalle
GELOPS
VST 2...8.14.eeceen. Meleeat. IHOGEHEEL SMAG E28 V2
98,044
10:71
6:72
0.8376
0,3963
0.7420
0.7311
1816-72
Beglett_. .V. 200 ght IMAGEEET J.N.NAG,..NNE
07.844
12.99
0.8995
0.7123
6.7316
127.52
Begitet_V. M.F.Meiightt -IAAGIEETT 2ER.AAA
07.244
0.9007
0.7392
6.7467
ConvNox:__aso www.hhh..CCCEEETTTT INACENET1R.01
0.7227
11.53
0.7393
9.63
0.559
0.7455
Swrtn92T,je (10977_..6111_11 w.w.i.....11111 TMAGGMETT1L.1
06.132
9.7%
5.36
01858
0.00%
0.2490
0.7410
5.94
Suiin Ke ght.. INAGEXTTTI V1
95.776
9.65
5.39
(1889
0.0055
0.2463
0.7435
4.49
Boaleet.Y.522 wtw..YY.22........ Meiglits. LVACCSETTIL.V
85.5 05.340
9.55
5.08
05530
0.5433
B.7464
0.7455
32.28
Segliet_V.100 aariggraa.... INAGUNETII 11
03.048
9.54
5.10
01530
0.9074
0.7438
0.7414
5.47
Segllet_X. x.3 220 www.ttt.t........... -MAGENETT V3
94.576
9.50
5.01
0.564
0.9065
0.7468
0.7436
3.18
Sestltt13) 152.Neiight... wtt.gghtt........ 2NGGEXSTIK..)
94.046
9.89
3.25
08837
0.749
11:51
Restlet123 Neighta. -.A.ahh.-.AAAEESEE111 KXAGENETTK
03.545
0.59
05529
0.7459
0.7454
7.50
Begllt y 600ME Weieht1. JM8GGEEE
03.136
9.40
08332
0.7508
0.7457
Tellet JP'_ Aaaagn..ee 2a1p._e--1000 ggh
Eegliet.Y www.lt.w....oooooooooo.o.o......
01.718
0.7470
50,550
9.60
4.55
01853
03438
0,7410
Hons Masiiellett.0 V22 INASENUTIK,Y
90.286
9.52
3.68
0.8441
0.9118
0.477
0.7331
0.30
Shustlediet. V2 w.W.ignts. IMAGINETLR
83.316
11.57
3.51
0.836
0.5017
0.7388
0.7436
0.14
Slexlet_keightt TNA.ENETT1K.9)
79,066
11.20
6.23
0.500)
0.9163
0.2855
0,7187
0.71
10
Procition NIPE (NFE-2) ronk be500
10
Accuracy
(a) Accuracy vs. Precision NFE=3 3
(b) Accuracy vs. Recall NFE=
Precision (NFE-9) rank (best)
Pocall INFE- rank (11
10 10112221141100111100
Accuracy rank (1
besti
Accuracy rank 12 - best)
(c) Accuracy vs. Precision @ NFE 9
(d) Accuracy vs. Recall @ NFE= 9
Figure 10: Accuracy vs. precision/recall. We select 20 Torch classifiers sorted by accuracy, learn the Dual-Solver for each, and report precision/recal at NFE=3 and 9 on 10k samples.
Classifier accuracy precision and recall. Fig. 10 (a) and (b) plot precision and recall versus classifier accuracy at NFE= 3, respectively. The results show little relationship between accuracy
http:////...miio/mee..m..mm..... h.mmm
25

---

Published as a conference paper at ICLR 2026
and precision, while accuracy versus recall follows the V-shape seen in Fig. 6 In other words, neither very high nor very low accuracy helps recall; a moderate level yields higher recall. The pattern differs NFE= 9. Fig. (c) and (d) plot precision and recall versus classifier accuracy NFE= 9. Unlike the NFE= 3 case, precision exhibits strong negative correlation with accuracy, and accuracy appears largely unrelated recall.
OpenCLIP accuracy vs. FID. Table reports the FID evaluation used to select the CLIP model for learning Dual-Solver on the text-to-image task. All weights are from OpenCLIP (Ilhareo al. 2021] and are available from the official repository
F Using the SANA (Xie et al. 2024) backbone we trained the Dual-Solver for 20k steps. At NFE and 6, we generated 30k samples with MSCOCO 2014 (Lin et al., 2014) prompts and measured FID on the evaluation set. Based chose the RNI01 weights-Which achieved an FID 18.52 at NFE = 6-as the model for learning the Dual-Solver in the main text-to-image experiment, Notably, this result also indicates that models with somewhat lower classification accuracy can yield lower FID.
these results,
Table 11: OpenCLIP per-classifier metries. MSCOCO accuracy, FID at NFE- = 3 and 6, and GFLOPs; FID is measured on 30k samples after learning Dual-Solver for each classifier.
Weights
FID@3
FID@6
ViT quickgelu,
63.76
23.98
23.28
1054.05
coca ViT-1-14, macece finetuned laien2b #13b b90k
60.28
23.09
21.22
214.52
EVA02-E-14, leion2b_sdb.tl15k
21.12
1007.93
convnext.xxlarpe 1aion2b.e330bb22...aug
58.34
21.05
20.86
800.88
Var-B-16-SigLIe-25- webli
57.24
21.03
19.87
$7.84
mVA02=-1143336, marged2b a6b b61
56,05
23.15
23.28
167.50
VIT-L-14, Lsionaeethheiieel33.3..
55.13
23.46
22.81
175.33
senvnext.base.ww
20.97
49.38
sonvnext.base...222 htt.s/aaaaaatttteaaaa...a..aaaa.
$1.42
20.69
20.26
175.33
91T-B-16-plua-2400 laion40Cm.c32
ViT-B-32, 1aion2b.e16
47.68
23.75
23.49
14.78
ViT ViT-B-32-que 233-quckkgeu,, metaclip.fullee
46.62
22.46
21.42
14.78
RN50x16, openai
45.38
22.49
21.36
33.34
ViT-B-32, 1aion400m_c31
43.27
22.40
21.70
14.78
RN101, openai
40.25
21.78
18.52
25.50
ViT B 16, commenpooi...tett..lll.
37,30
23.50
23.54
41.09
ViT-B-16, commenроol
ViT-B-32, wttpn/poo...ttx._al22....
14.52
22.39
22,55
14.78
VIT-R-32,
commonpaol_a_al1 pm_133m_44
21.31
coca.ViT
32, meccoo_Iieecuneeleeooobb........
0.60
23.76
23.14
33.34
https:://githbb...om/ http:///oo//onoooaaaaiiaia/////o
26

---

Published as a conference paper at ICLR 2026
F LEARNED PARAMETERS
In Figs. 11 12 13 14 we plot the Dual-Solver parameters learned via classification-base learning (Sec. 5.2), as defined Sec. 4.2 For the DiT, GM-DiT, SANA, and PixArt-a backbones, we set CFG to I.5, 1.4, 4.5, and 3.5, respectively, and train for 20k steps. We plot results at NFE = 3, 5, 7, and for DiT and GM-DiT, and at NFE = 5, and 6 for SANA and PixArt-a. Separate parameter sets are learned for the predictor and the corrector, and they are shown in different colors. Within the same backbone, the parameter curves are similar even across different NFEs. We conjecture that this arises from the backbone's s intrinsic trajectory.
Red.
0-
slep
AlMp
seep
step
(a) NFE 33
Pec.
D-
1
0
3lap
AlMp
(b) NFE =5
Prc.
DaT.
$20p
SAUp
$200
stop
(c) NFE
Indd
ONT
$200
stop
1280
300p
step
(d) NFE
Figure 11: Learned parameters. (7, Tas Tay Kun Ken for DiT (Peebles & Xie 2023
27

---

Published as a conference paper at ICLR 2026
2020
slep
Maa
(a) NFE =3
2-
$100
200
(b) NFE
-2
(c) NFE =7
-1
2-
(d) NFE =
Figure 12: Learned parameters, (7/ Ta, Tus Kus Ki for GM-DIT (Chen et al. 2025
28

---

Published as a conference paper at ICLR 2026
Mlep
slep
Mlap
MMQ
(a) NFE
2-
JA00
$100
200
(b) NFE
-2
slep
(c) NFE 5
-1
2
(d) NFE =
Figure 13: Learned parameters. (7, Tu: To, Kuy Kop for SANA (Xie et al. 2024
29

---

Published as a conference paper at ICLR 2026
202
slep
Meap
(a) NFE
7
2-
$100
20p
St2p
(b) NFE
-2
slep
(c) NFE 5
(d) NFE =
Figure 14: Learned parameters. (7, Tus Tux Kuu K.p for PixArt-ax (Chen et al. 2023
30

---

Published as a conference paper at ICLR 2026
G DETAILS OF PARAMETER INTERPOLATION ACROSS NFES
In this section, we describe how to interpolate the parameters discussed in Sec. 6.3 so that they can be used at other NFEs. The parameters of Dual-Solver,
e Pred pred, preed p.eeed l.pred
K ki 7 coir, coir corr corr corr T Fi
Kyy
given arrays whose length equals the NFE. (The corrector parameters have length NFE-1, and we match the length to NFE by repeating the last element.) For example, y (1g pred ..... .....
pred 7N To interpolate these parameters, we consider the following linear interpo-
lation scheme for generic array.
Definition G.1 (Linear interpolation). Let f(M) (56 ..... ---1 be array of length M. The linearly interpolated array Interp(((M) N) of length N is defined as follows. First, set
(M (M- - 1),
je [ta].
a; t, Ji.
and for each O..... N define
Interp( N) = - o.) f/ as f
Definition G.2 (Averaged linear interpolation). Let M < < L. three NFEs, and let f(M) E RM and f(L) E R be the corresponding arrays. We first obtain their linearly interpolated versions of length N,
f(() Interp( (());N) N)- We then use the relative position of N between M and L as weights and define
f(a) Interp ([(M;
N
N M
WM
w1 = L L-M M
L - M
and, for each i O.....NII..
N
Interp of ((D)) p((); N) wM F(M) [i] + wT [()
Using this procedure, we obtain the array for an intermediate NFE from the two arrays at neighboring NFEs, and apply the same construction to every parameter in ø. Examples of interpolated parameters are shown in Fig. 15] Specifically, we obtain the parameters for NFE 4, 6, and 8 by interpolating those learned NFE (3, 5), (5, 7), and (7, 9), respectively.
31

---

Published as a conference paper at ICLR 2026
drp
dxp
dtp
(a) NFE =
Om
200
st0p
229
NFB 4, interpolated from (3,5)
0100
NFE
Mapp
Map
(d) NFE 6, interpolated from (5,
(e) NFB
dxp
400
(f) NFE 8, interpolated from (7. 9)
(g) NFE
Figure 15: Interpolated parameters. {7, Tus
Tas Tun Kus Ko for DiT (Peebles & Xie 2023
32

---

Published as a conference paper at ICLR 2026
(a) DPM-Stlver++ Lu et al.
2022b
(b) Dual-Solver (Ours)
(c) BNS-Solver Shaul et al. 2024
(d) DS-Solver (Wang et al. 2025
Figure 16: Additional sampling results. D/T-XL/2 256x256 (NFE-4, CFG=1.5)
33

---

Published as a conference paper at ICLR 2026
(a) DPM-Stlver++ (Lu et al
2022bb
(b) Dual-Solver (Ours)
(c) BNS- (Shaul et al. 2024
(d) DS-Solver WWng et al. 2025
Figure 17: Additional sampling results. GM-DiT 256 (NFE=3, CFG=1.4)
34

---

Published as a conference paper at ICLR 2026
Go den autumn park of falling leaves, graceful girl playing violin, flowing satin dress, impressionist brush strokes
Towering neon-lit cyberpunk skyl ine, armored samurai with glowing katana, chrome textures, cinematic digital art
Romoto desert oasis at sunrise, powerful white stallion galloping, glossy mane and muscies, hyperreal photo realism
Vibrant tropical beach sunset, pirate ship anchored offshore, weathered wood huil, playful cartoon illustration
Overgrown jungle temple ruins, spotted leopard stalking prey, sleek fur patterns, painterly realism concept art
(a) OPM-Solver++ (Lu et al. 2022b)
(b) Dual-Solver (Ours)
(c) BNS-Solver Solver Shaul et al. 2024)
(d) DS-Solver
(Wang et al. 2025
Figure 18: Additional sampling results. SANA Xie et al.
2024 NFE=3, CFG=4.5.
35

---

Published as a conference paper at ICLR 2026
in misty emerald forest clearing, a majestic golden wolf with silky glowing fur, painted in luminous oil style
Towering neon-lit cyberpunk sky? ine, armored samurai with glowing katana, chrome textures, cinematic digital art
Vast desert boneath starry cosmos, colossal sphinx carved in sandstone, rough surface, surreal dreamlike painting
Sleek futuristic laboratory interior, humanoid robot motion, polished steel surfaces, sci-fi blueprint style
Storm-drenched battlefield ruins, giant mech rising from fire, rusted armor plates, gritty photorealistic concept art
(a) -oM-Sotve+++ (Lu et al. 2022b)
(b) Dual-Solver (Ours)
(c) BNS-Solver Solver Shaul et al. 2024)
(d) DS-Solver
(Wang et al. 2025
Figure 19: Additional sampling results. PixArt-a Chen al. 2023), NFE=5, CFG=3.5.
36


Reviews:
[{"review_id": "23SfScO2Ev", "reviewer": "Reviewer_TpbM", "summary": "The paper proposes Dual-Solver, a learned predictor–corrector sampler for diffusion/flow-matching models that targets the low-NFE regime. It introduces three per-step learnable knobs: γ (interpolates prediction type across noise/data/velocity), τ (a log–linear domain transform that interpolates between λ- and ρ-domains), and κ (a residual term that preserves second-order local accuracy). The method keeps second-order local truncation error, learns the timestep schedule, and trains all solver parameters via a classification-based objective that uses a frozen classifier.", "questions": "Does the method require separate training runs for different NFE targets?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "1. Unified, principled parameterization. γ cleanly bridges noise/data/velocity predictions; τ smoothly interpolates λ↔ρ domains; κ adjusts second-order residuals\n\n2. Strong low-NFE results across backbones. Consistent FID/CLIP gains at ≤6 steps on DiT, SANA, PixArt-α, and GM-DiT.\n\n3. Classification-based training removes the need for high-NFE teacher samples, reducing preparation cost and directly optimizing a perceptual proxy.", "weaknesses": "1. Training introduces ten learnable parameters per step (in addition to a learned schedule), which could increase brittleness or overfitting to specific backbones and guidance settings. Also, does the method require separate training runs for different NFE targets?\n\n2. Because CE/CLIP is computed with a frozen classifier, the optimization might bias samples toward ‘classifier-friendly’ patterns. How do the authors mitigate proxy-objective leakage and ensure alignment with the intended generative quality?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "X4ZSIOzyLm", "reviewer": "Reviewer_g7wy", "summary": "This paper introduces Dual-Solver, which accelerates the sampling of diffusion models using learnable parameters $\\gamma$, $\\tau$, and $\\kappa$. This framework incorporates and extends many existing multi-step and learning-based solvers, and abandons high-NFE (Number of Function Evaluations) teacher trajectories; instead, it updates the learnable parameters via classification loss or CLIP loss. The paper reports experimental results on ImageNet class-conditional and text-to-image diffusion models, effectively demonstrating the sampling performance of Dual-Solver under low NFE.", "questions": "1. How does Dual-Solver perform compared to current advanced learning-based solvers (e.g., EPD-Solver[1], AMED-Solver[2])?\n2. The paper proposes that using a \"weak classifier\" with lower accuracy can alleviate the problem of reduced sample diversity, but the results in Table 7 do not seem to clearly demonstrate the effectiveness of this strategy. Could you provide further explanation?\n3. The paper learns parameters for each step but does not use ablation studies to quantify the effect of globally shared parameterization. Given that shared parameters can effectively reduce the number of learned parameters, could the authors conduct more detailed comparative experiments?\n\n[1] Zhu B, Wang R, Zhao T, et al. Distilling Parallel Gradients for Fast ODE Solvers of Diffusion Models[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. 2025: 19557-19566.\n\n[2] Zhou Z, Chen D, Wang C, et al. Fast ode-based sampling for diffusion models in around 5 steps[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024: 7777-7786.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. It constructs a generalized ODE solver based on a \"predictor-corrector\" structure. Through explicit interpolation of $\\gamma$ and domain transformation via $\\tau$, it unifies noise/data/velocity prediction methods within a single framework.\n2. It proposes a solver parameter update method based on classification loss or CLIP loss, abandoning the parameter update learning strategy that relies on teacher trajectories. This reduces computational overhead and improves performance under low NFE.\n3. The paper features rigorous mathematical derivations, covering the mathematical foundations of dual prediction and log-linear domain mapping. It also provides a complete proof of the second-order local truncation error.", "weaknesses": "1. The method is derived based on second-order accuracy, but it does not discuss potential issues when extending to higher-order schemes, which limits the further performance improvement of Dual-Solver.\n2. It lacks comparative results with current advanced learning-based solvers, such as EPD-Solver[1] and AMED-Solver[2].\n3. The parameter update of Dual-Solver relies on the cross-entropy loss of a classifier or CLIP loss, which means the parameter update of Dual-Solver will fail for unguided conditions.\n\n[1] Zhu B, Wang R, Zhao T, et al. Distilling Parallel Gradients for Fast ODE Solvers of Diffusion Models[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. 2025: 19557-19566.\n\n[2] Zhou Z, Chen D, Wang C, et al. Fast ode-based sampling for diffusion models in around 5 steps[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024: 7777-7786.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "6ByyPKwJw6", "reviewer": "Reviewer_qooq", "summary": "This paper proposes Dual-Solver, a learnable, generalized ODE sampler that unifies noise/data/velocity predictions via dual prediction, and combines a log–linear domain transform with a second-order predictor–corrector and learnable residuals for stable few-step sampling.\n\nIt learns per-step parameters and the timestep schedule end-to-end using discriminative objectives (e.g., classifier/CLIP), yielding large quality gains at very low NFE across diverse backbones and tasks.", "questions": "1. About the training process, my current understanding is that after sampling M-1 steps, we directly optimize the corresponding 10*(M-1) groups of parameters with a set of labels or captions? Once these parameters are trained, the parameters can be reused for this backbone? Can the author provide a more specific training process?\n\n2. Why classification objectives beat regression is not deeply unpacked. The paper shows a V-shaped accuracy–FID trend and argues “reaching the right decision region” is enough, but it stops short of a deeper mechanism: when/why classification consistently wins across backbones/NFE, and under what conditions it might overfit decision boundaries.\n\n3. Consistency vs. transfer across backbones/settings.\nIt observes that learned parameter curves look similar across NFEs for a given backbone, yet recommends training per backbone/setting. Readers may wonder about the practical transfer value of those similarities (warm-starts, NFE interpolation, shared parameterizations), which isn’t explored methodologically.\n\n4. Fig2 is not mentioned in the article. And I feel that the Sec 4 and 5 are somewhat abrupt logically, it seems subsection of Sec2? \n\n\nOverall, my main concern is 1. Please clarify the approach for obtaining parameters for all steps. If my concern addressed, I would be willing to raise my score.", "limitations": "", "rating": 4, "confidence": 5, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "1. The proposed method is novel and solid theoretical. This work analyzes currently popular semi-linear samplers and different prediction targets and proposes a more general PC-based sampler. Contributions include:\n    - unifies different prediction according to a dual-prediction parameterization. \n    - uses a learned change of variables in semi-linear family for integration\n    - second order residual adjustment for p1c2\n\n2.  Experimental sufficient. This work provides strong, multi-backbone results on FID. And there's enough ablations over predictor/corrector order and parameter freedoms\n\n3. Limitations are acknowledged. \n\nOverall, this work proposed a well-motivated and clearly articulated solvers, especially for few-step sampling.", "weaknesses": "1. Personally, the article writing is somewhat difficult to follow up on. The overall structure is somewhat confusing to me.\n\n2. The description of the parameter training part is insufficient. Especially the training process. \n\n3. The discussion of details for certain algorithms lacks intuitive understanding.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "XHAx62X6mj", "reviewer": "Reviewer_8gbp", "summary": "Diffusion models deliver state-of-the-art image quality. However, sampling is costly at inference time because it requires many model evaluations (number of function evaluations, NFEs). To reduce NFEs, classical ODE multistep methods have been adopted. Yet differences in the choice of prediction type (noise/data/velocity) and integration domain (half log-SNR/noise-to-signal ratio) lead to different outcomes. This paper introduces Dual-Solver, which generalizes multistep samplers by introducing learnable parameters that continuously (i) interpolate among prediction types, (ii) select the integration domain, and (iii) adjust the residual terms. It maintains the traditional predictor-corrector structure and guarantees second-order local accuracy. These parameters are learned with a classification-based objective using a frozen pretrained classifier (e.g., ViT or CLIP). On ImageNet class-conditional generation (DiT, GM-DiT) and text-to-image (SANA, PixArtα), Dual-Solver consistently improves FID and CLIP scores in the low-NFE regime (NFE ≤ 6) across backbones.", "questions": "1. For a single prediction model, does Dual-Solver need to first transform the prediction into eps and $x_0$?\n\n2. Is dual prediction necessary? Given that a predefined noise scheduler has deterministic prediction transforms( eps to v, v to eps, v to x0), I have doubts regarding the arguments for dual prediction. Although other works [1, 2] use a single prediction, no discriminator is employed for learning.\n\n[1] Shuai Wang, Zexian Li, Tianhui Song, Xubin Li, Tiezheng Ge, Bo Zheng, Limin Wang, et al. Differentiable solver search for fast diffusion sampling. arXiv preprint arXiv:2505.21114, 2025.\n\n[2] Neta Shaul, Uriel Singer, Ricky TQ Chen, Matthew Le, Ali Thabet, Albert Pumarola, and Yaron Lipman. Bespoke non-stationary solvers for fast sampling of diffusion and flow models. arXiv preprint arXiv:2403.01329, 2024", "limitations": "", "rating": 6, "confidence": 3, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "1. Generalizes multistep samplers by introducing learnable parameters that continuously interpolate among different prediction types (noise/data/velocity), select the integration domain, and adjust residual terms, addressing the issue of inconsistent outcomes caused by differences in prediction type and integration domain choices.\n2. Structural compatibility and guaranteed accuracy: Maintains the traditional predictor-corrector structure (ensuring compatibility with existing multistep method frameworks) while guaranteeing second-order local accuracy, balancing structural familiarity and precision.\n3. Learns the key parameters via a classification-based objective, leveraging frozen pretrained classifiers (e.g., ViT, CLIP) — this avoids the need for heavy independent training and capitalizes on existing well-performing models.", "weaknesses": "see Questions", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)