PFDiff: Training-Free Acceleration of Diffusion Models Combining Past and Future Scores
Authors: Guangyi Wang
Yuren Cai
lijiang Li
Wei Peng
Song-Zhi Su
Venue: ICLR 2025 Poster (2025)
Keywords: diffusion models
accelerated sampling
training-free sampler
orthogonal sampling method
Primary area: generative models
TL;DR: We propose a new training-free fast sampler for accelerated sampling of diffusion models, which is orthogonal to existing fast solvers.

Abstract: Diffusion Probabilistic Models (DPMs) have shown remarkable potential in image generation, but their sampling efficiency is hindered by the need for numerous denoising steps. Most existing solutions accelerate the sampling process by proposing fast ODE solvers. However, the inevitable discretization errors of the ODE solvers are significantly magnified when the number of function evaluations (NFE) is fewer. In this work, we propose PFDiff, a novel training-free and orthogonal timestep-skipping strategy, which enables existing fast ODE solvers to operate with fewer NFE. Specifically, PFDiff initially utilizes score replacement from past time steps to predict a springboard. Subsequently, it employs this ``springboard" along with foresight updates inspired by Nesterov momentum to rapidly update current intermediate states. This approach effectively reduces unnecessary NFE while correcting for discretization errors inherent in first-order ODE solvers. Experimental results demonstrate that PFDiff exhibits flexible applicability across various pre-trained DPMs, particularly excelling in conditional DPMs and surpassing previous state-of-the-art training-free methods. For instance, using DDIM as a baseline, we achieved 16.46 FID (4 NFE) compared to 138.81 FID with DDIM on ImageNet 64x64 with classifier guidance, and 13.06 FID (10 NFE) on Stable Diffusion with 7.5 guidance scale. Code is available at https://github.com/onefly123/PFDiff.

Published as a conference paper at ICLR 2025
PFDIFF: TRAINING-FREE ACCELERATION OF DIF- FUSION MODELS COMBINING PAST AND FUTURE SCORES
Guangyi Wang! Yuren Cai 1, Lijiang Li Wei Peng', Songzhi Su
School of Informatics, Xiamen University.
Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University, 361005, P.R. China. 3 Department of Psychiatry and Behavioral Sciences, Stanford University. [wangguangy1, cai yuren, lilliiijfie ailijjanggeeee @stu. xmu, edu. en, wepeng@stanfors edu, s5z0 xmu. edu. on
ABSTRACT
Diffusion Probabilistic Models (DPMs) have shown remarkable potential in im- age generation, but their sampling efficiency is hindered by the need for numer- ous denoising steps. Most existing solutions accelerate the sampling process by proposing fast ODE solvers. However, the inevitable discretization errors of the ODE solvers are significantly magnified when the number of function evaluations (NFE) is fewer. In this work, we propose PFDiff, novel training- free and or- thogonal timestep-skippinne operate with fewer NFE. Specifically, PFDiff initially utilizes score replacement from past time steps to predict a "springbound". Subsequently, it employs this "springboard" along with foresight updates inspired by Nesterov momentum rapidly update current intermediate states. This approach effectively reduces un- necessary NFE while correcting for discretization errors inherent first-order ODE solvers. Experimental results demonstrate that PFDiff exhibits flexible ap- plicability across various pre-trained DPMs, particularly excelling in conditional DPMs and surpassing previous state-of-the-art training-free methods. For in- stance, using DDIM as a baseline, we achieved 16.46 FID (4 NFE) compared to 138.81 FID with DDIM on ImageNet 64x64 with classifier guidance, and 13.06 FID (10 NFE) on Stable Diffusion with 7.5 guidance scale. Code is available at gi Attps: / giggitthhe thub www.:a/.ooocoeeeeeeeeeeeeeeeeeee wwtp:///nwn///////22000
strategy, which enables existing fast ODE solvers to
INTRODUCTION
In recent years. Diffusion Probabilistic Models (DPMs) (Sohl-Dickstein al. 2015 Ho al. Song et al. 2020b) have demonstrated exceptional modeling capabilities across various domains in-
2020
cluding image generation (Dhariwal Nichol 2021 Peebles Xiel 2023 Karras et al.
2024).
video generation (Dehghani et al.] 2023). text-to-image generation (Rombach et al. 2022 (ct al., 2023), speech synthesis (Song et al., 2022), and text-to-3D generation (Poole et al.
Betker
2022
Lin et al.) 2023). ). They have become a key driving force advancing deep generative models. DPMs
initiate with a forward process that introduces noise onto images, followed by utilizing a neural net- work to learn backward process that incrementally removes noise, thereby generating images (Ho let al. 2020; Song et al. 2020b). Compared to other generative methods such as Generative Adver- Sarial Networks (GANs) (Goodfellow al. 2014) and Variational Autoencoders (VAEs) (Kingmal & Welling 2013), DPMs not only possess a simpler optimization target but also are capable of pro- ducing higher quality samples (Dhariwal & Nichol (22TT). However, the generation of high-quality samples via DPMs requires hundreds or thousands of denoising steps, significantly lowering their sampling efficiency and becoming a major barrier their widespread application.
"Corresponding Author.

---

Published as a conference paper at ICLR 2025
Existing techniques for rapid sampling in DPMs primarily fall into two categories. First, training- based methods (Salimans & Ho. 2022 Liu et al. 2022b; Song et al. 2023 Yin et al. 2024), which significantly compress sampling steps, even ac achieving single-step sampling However, this com- pression often comes with a considerable additional training cost, and these methods are challenging to apply to large pre- traitre models. Second, training-free samplers (Song et al. 2020a] Lu al.,
2022a Bao et al., 2022b Liu et al. 2022a; ei ct al., 2023 Zheng et al. 2023] Ma et al. 2024] Wimbauer et al. 2024: Zhao
2023 Xue al., 2023). which typically employ implicit or ana-
lytical solutions to Stochastic Differential Equations /ODEyOrdinary Differential Equations (ODE) for lower-error sampling processes. For instance, Lu al. (Lu et al.) 2022a b), by analyzing the semi-linear structure of the ODE solvers for DPMs, have sought to analytically derive optimally the solutions for DPMs' ODE solvers. These training-free sampling strategies can often be used in plug-and-play fashion, compatible with existing pre-trained DPMs. However, when the NFE is below 10, the discretization error of these training-free methods will be significantly amplified, leading to convergence issues (Lu et al. 2022a b), which can still be time-consuming.
To further enhance the sampling speed of DPMs, we have analyzed the potential for improvement in existing training-free accelerated methods. Initially, we observed a high similarity in the model's outputs for the existing ODE solvers when time step size At is not extremely large, as illustrated in Fig. Ta This observation led us utilize the scores that have been computed from past time steps to approximate current scores, thereby predicting a "springhoand". Furthermore, due to the similarities between the sampling process of DPMs and Stochastic Gradient Descent (SGD) (Rob- bins& & Monro, 1951) as noted in Remark [ we incorporated a foresight update mechanism using Nesterov momentum (Nesterov) 1983), known for accelerating SGD training. Specifically,, we first predict future scores using the "springboard" to reduce errors, as shown in Fig. [b] Then, we further replace the current scores with the future scores facilitate larger update step size At, as shown in Fig. Ic
Motivated by these insights, we propose PFDiff, a timestep-skipping sampling algorithm that rapidly updates the current intermediate state combining past and future scores. Notably, PFDiff is training- free and orthogonal to existing DPMs sampling algorithms, providing a new orthogonal axis for DPMs sampling. Furthermore, prove that PFDiff, despite utilizing fewer NFE, corrects for er- rors in the sampling trajectories of first-order ODE solvers, as visualized in Fig. 1c This ensures that improving sampling speed does not compromise sampling quality; it only reduces unnecessary NFE in existing ODE solvers. To validate the orthogonality and effectiveness of PFDiff, exten- sive experiments were conducted on both unconditional (Ho et al. 2020; Song al. 2020b and conditional (Dhariwal& & Nichol [02T: Rombach et al., 2022) pre-trained DPMs. The results of the visualization experiment are depicted in Appendix [.9 The results indicate that PFDiff sig- nificantly enhances the sampling performance of existing ODE solvers. Particularly in conditional DPMs, PFDiff, using only DDIM as the baseline, surpasses the previous state-of-the-ar training-free sampling algorithms.
BACKGROUND
2.1 DIFFUSION SDES
Diffusion Probabilistic Models (DPMs) (Sohl-Dickstein et al. 2015 Ho et al. 2020 Song et al., 2020b) aim to generate D-dimensional random variables 2g E R" that follow data distribution q(zo). Taking Denoising Diffusion Probabilistic Models (DDPM) (Ho et al. 2020) these models introduce noise to the data distribution through a forward process defined over discrete time steps, gradually transforming it into a standard Gaussian distribution FT (0, I). The forward process's 's latent variables { { are defined as follows:
example,
q(xx| (D) = -M(ra [, Tf
where ar is a scalar function related to the time step t, with af + +oï = 1. In the model's 's reverse pro- cess, DDPM utilizes neural network model po(x2-1 (x)) to approximate the transition probability q(12-1 Ie. Øo).
(0 (x4) -M (o( (1))
(2)
2

---

Published as a conference paper at ICLR 2025
Noisc
UDD Rogactonie
Data
10 DOP
EPU Shvered
MC Santt
Buffi
MT Trom Scnt)
gemdurt Paa Sond
((Fs)
20
Sepikgrniee wice Fores mopll Cecinesoin
(a) score Changes in SDE/ODE Solvers
(b) Future score More Reliable than "Springboard"
(c) Comparison of Sampling
Trajectories
Figure I: (a) The trend of the MSE of the noise network output c 1) over time step size At, where 17 comes from og in Eq. (6) Solid lines: ODE solvers, dashed lines: SDE solvers. (b) MSE of the status separately updated using "springboard" Seeen. and future score co co (z tith), relative
the sampling process with 1000 NFE, is given by: [ - ( 11 (c) Comparison
of partial sampling trajectories between PFDiff-1 and first-order ODE solver, where the update directions are guided by the tangent direction of the sampling trajectories.
where og(() (t) is defined as a scalar function related the time step t. By sampling from a standard Gaussian distribution and utilizing the trained neural network, samples following the data distribu-
tion pe (20) Po
22) can be generated.
Furthermore, Song et al. (2020b) introduced SDE to model DPMs over continuous time steps, where the forward process is defined
dr, = f())xxd + g(f)dw:, zo ~ q(xo),
(3)
where we represents standard Wiener process, and f and are scalar functions of the time step
It's noteworthy that the forward process in Eq. ①1 is a discrete form of Eq. (3). where f(1)
doogg and (4) do? 2 logoa of. Song et al. (220b)) further demonstrated that there exists an df equivalent reverse process from time step 7 to for the forward process in Eq. (3)
dit
dr. [f(()=; g² Vr log (xx)) di g())drx, or ((TT);
(4)
where vo denotes a standard Wiener process. In this reverse process, the only unknown is the score function Va log qe (z+), which can be approximated through neural networks.
2.2 DIFFUSION ODES
In DPMs based on SDE, the discretization of the sampling process often requires a significant num- ber of time steps to converge, such as the T 1000 time steps used in DDPM (Ho et al. 2020 This requirement primarily stems from the randomness introduced at each time step by the SDE. To achieve a more efficient sampling process, [Song et al. (2020b) utilized the Fokker-Planck equa- tion (kksendal & Øksendal, 2003) to derive a probability flow ODE related to the SDE, which possesses the same marginal distribution at any given time t as the SDE. Specifically, the reverse process ODE derived from Eq. (3) can be expressed as:
dx:- = (()) - 29 (t) log (1x) dt, BT ~ q(xT).
(5)
Unlike SDE, ODE avoids the introduction of randomness, thereby allowing convergence to the data distribution in fewer time steps. [Song et al. (2020b) employed a high-order RK45 ODE solver (Dor- mand & Prince 1980), achieving sample quality comparable
SDE 1000 NFE with only 60
NFE. Furthermore, research such as DDIM (Song et al. 2020a) and DPM-Solver (Lu et al. (202aa) explored discrete ODE forms capable of converging in fewer NFE. For DDIM, it breaks the Markov chain constraint on the basis of DDPM, deriving a new sampling formula expressed as follows:
Ft
V aico(Ie.))
xt-1 = Var-1
+ at-1 - Feco t) + oic,
(6)
Va
3

---

Published as a conference paper at ICLR 2025
where "y (1 (1-a--1)(11- a-)//aaa///aa///aaa and ar corresponds to aj in Eq. (1). When = 1, Eq. (6) becomes a form of DDPM; when 7 = O. it degenerates into an ODE, the form adopted by DDIM, which can obtain high-quality samples in fewer time steps.
Remark 1. In this paper, we regard the score dē,, the noise network output eg (Z t), and the score function v log (x44)) expressing equivalent concepts. This is because Song et al. (2020b) demonstrated that cg( - V.V, log (e Moreover, we have discovered that any jurst-order solver of DPMs can be parameterized 1t-1 It J,dT, fez. Taking DDIM (Song al. 2020ah
as an example, where ãt- - - - 7t 0--1 at-1 vI- at-1 dē. Ea Ea t). and
at
& = O. This indicates the similarity between SGD and the sampling process of DPMs, discovery also implicitly suggested in the research of Xue et al. (2023) and Wang al. (2024)
METHOD
3.1 SOLVING FOR REVERSE PROCESS DIFFUSION ODES
By substituting co(0,, t) os v log (1,) (Song al. 2020b Eq. (5 can be rewritten as:
dr s(co(21, 21, = f(t)xz + 20
g²(()
co (z t), 2r ((T)).
dt
Given initial value ær, we define the time steps {4,) progressively decrease from to T to bT = O. Let t.o = FT be the initial value. Using T steps iteration, we compute the sequence (Tx, ) to obtain the solution of this ODE. By integrating both sides of Eq. (7 we can obtain the exact solution of this sampling ODE:
84 Si + [(' s(co(oe,)), f), t)dt.
(8)
For any p-order ODE solver, Eq. can be discretely represented as:
-t 100, 1t, 1411 = 8ea-1 Σ Mc) fn), fn) AE, i [L, [l,.., T7. (9)
Here, Q= (f { t.) stores the set of p scores computed over the intervals f and t,, where in 1i-1 Ep t. and At int1 i, denote the time step size. Particularly, when p == Q, co ( 4-11). The function is any p-rreer ODE solver that updates the current state Tt._, from time point 4 to t. using the scores stored in Q. The function h represents the way in which different p-order ODE solvers handle the function and its specific form depends on the solver's design. For example, in the DPM-Solver (Lu et al., (022a)), an exponential integrator is used to transform into h in order to eliminate linear terms. In the case of first-order Euler-Maruyee Maruyama solver (Kloeden et al. 1992). it serves as an identity mapping of s.
When using the ODE solver defined in Eq. (9) for sampling, the choice of T 1000 leads to significant inefficiencies in DPMs. The study on DDIM (Song et al. 2020a) first revealed that by constructing new forward sub-state sequence of length M + 1 (M < T), ) from a subsequence of time steps [O..... I] and reversing this sub-state sequence, it possible to converge to the data distribution in fewer time steps. However, as illustrated in Fig. Ial for ODE solvers, as the time step size At f f,-1 increases, the score direction changes slowly initially, but undergoes abrupt changes At - T. This phenomenon indicates that under minimal NFE (i.e., maximal time step size At) conditions, the discretization error in Eq. g9) is significantly amplified. Consequently, existing ODE solvers, when sampling under minimal NFE, must sacrifice sampling quality to gain speed, making it an extremely challenging task to reduce NEE to below 10 (Lu et al. 2022a b). Given this, aim to develop efficient timestep-skipping sampling algorithm, which reduces NEE while correcting discretization errors, thereby ensuring that sampling quality is not compromised, and may even be improved.
4

---

Published as a conference paper at ICLR 2025
3.2 SAMPLING GUIDED BY PAST SCORES
As illustrated in Fig. Ta when the time size A/ (i.e., 4 44- is not not excessively large, the MSE of the noise network, defined as T-Ar E ||co (zt, - co co( + At)]l is remarkably similar. This phenomenon is especially pronounced in ODE-based sampling algorithms, such as DDIM (Song et al., 2020a) and DPM-Solver (Lu et al., 2022a). This observation suggests that there are many unnecessary time steps in ODE-based sampling methods during the complete sampling process (e.g., when 7 = 1000), which is one of the reasons these methods can generate samples in fewer steps. Based on this, we propose replacing the noise network of the current timestep with the output from previous timestep reduce unnecessary NFE without compromising the quality of the final generated samples. Specifically, for any p-order ODE solver Φ, the sampling process from It.-, to 2t, can be reformulated according Eq. (9) as follows:
ob{ ((t Sh 11,
(10)
Then, we store the noise network output in a buffer for use in the next timestep, as follows:
buller
(ea 11-1,, b1-1,4).
(11)
where 4:-1 and f; represent the intervals over which the set of p scores are computed. For the sampling process from Lt, to St.,., we directly use the noise network output saved in the buffer from the previous timestep to replace the current timestep's ' noise network, thereby updating the intermediate states to the next timestep (i.e., the "springboard" Itz ,). detailed below:
Brig @QQ Te, t4. t.41), where Q = @=(((oo0z (()) -
(12)
By using this approach, we can reduce unnecessary NFE, thereby accelerating the sampling process.
Remark 2. Notably, when the time step size At is very large (NFE<10), the similarity between
past and current scores decreases sharply. making "springboard" 2t,,, unreliable in Eq. (72). Therefore, in Sec. 3.3 we use Øt.+, solety to predict foresight update direction future score) to reduce errors caused by the replacement, as shown Fig. Th Fig. [c Both past and future scores are complementary and indispensable, as demonstrated by the ablation study in Sec. 4.3
3.3 SAMPLING GUIDED BY FUTURE SCORES
As stated in Remark D considering the similarities between the sampling process of DPMs and SGD, and inspired by Nesterov momentum (Nesterov) 1983). we introduce a foorrsight update direc- tion (i.e., future score) to assist the current intermediate state in achieving more efficient leapfrog updates. Notably, employing future scores is more reliable than directly using the "springboard", as discussed Remark 2 Specifically, during the sampling process from ét, to atuea. we consider using future scores (corresponding to time point 44+1) 1) to replace the current scores (corresponding to (2). Continuing from Eq. (12). we estimate the future score using the "springboard" Iti1 and update the buffer follows:
buffer
(feo(Tr,, (m)) (4421442)
(13)
Subsequently, leveraging the concept of foresight updates, we predict further future intermediate state 2t,42 using the current intermediate state Tr, along with the future score corresponding to time point 1, as shown below:
i olQ, Se, ti t442), where
(x), (142)
(14)
Furthermore, analyze how to correct the enrors of the first-order ODE solvers in the discretized
Eq. using future scores. Let so (2 t) == ((ea(x), t), ze, t), we further analyze the term from Eq.
(m) g that may cause errors, ftz so(2e, t)dt. Assuming that $9 (2,, E [t:-1, ts] exists and is
5

---

Published as a conference paper at ICLR 2025
((colBe.. Se.A),
Select of "prannhoord
Future Score
Mc Ba 4-)
Soore Q
"Pasi" relative round cof Po fish)
(o f1-(p-2-1))
Buffer
Status update
Temporury status update
Scores propagation
1, 1,2,... Butfer access order
Illustration single iteration update of PFDiff-k: combined with any first-order ODE solver Φb Given specific values of k and h (k < 3 (h < k)), PFDiff first uses the past score Q stored in the Buffer from the previous iteration to replace the current score, updating to the "springboard" Ft.aa ; then the future score is calculated using the "springboard"; finally, the future score is used to replace the current score, completing a full update iteration. The future score will also be passed to the next iteration the "past" score for the next round of updates.
Figure 2:
continuous, applying Taylor's expansion at / = r, we derive:
(1) (Er, (t - (r) Rt((t) dt
so (Fe, ()
n!
4-1
(m) (Ir:
(t (r) dt r)"
n!
(t: r) n+1
(ti-1 r)
(15)
$8 (x,,,
(n + 1)!
m=0
(te -) n+1 (t--1
80 (Ir, (1: - 14-1)+
8
(Irr r).
(n+
7=1
"higher derivative
Proposition 3.1. For any given DPM first-order ODE solver, the absolute values of the coefficients for higher-order derivative terms in Eq. (15) are smaller when using the future time point r score compared to the current time point ti-1 score, as follows (Proof in Appendix B.2):
(t+
(t:-1 =)
(t: t n
where E E (t)
(16)
n!
n!
Proposition 3.1 indicates that neglecting higher-order derivative terms has less impact when sam- pling with future scores, correcting for the discretization errors inherent in first-order ODE solvers. However, higher-order ODE solvers approximate higher-order derivative terms by estimating the noise network's output multiple times (Luu et al., [002a]] b) Zheng et al. 2023). Future scores and higher-order ODE solvers reduce the discretization derivative terms in two parallel manners, complicating the error analysis when both methods are used simultaneously. Therefore, when using higher-order ODE solvers a baseline, the sam- pling process is accelerated by only using past scores. It is only necessary to modify Eq. Tr @(Q, 2 f441, f442 while keeping Q constant.
caused by neglecting higher-order
(14)
3.4 PFDIFF: SAMPLING GUIDED BY PAST AND FUTURE SCORES
Combining Sec. 3.2 and Sec. 3.3 the "springboard" Tt,,. obtained through Eq. 12 is used to update the buffer Q in Eq. 13 this way, we achieve our proposed efficient timestep-skipping algorithm, which we name PFffff. Notably, during the iteration from intermediate state It, to 21, we only perform a single batch computation (NFE = p) of the noise network in Eq. (13). Further- more, propose that in single iteration process, Tt,12 in Eq. (14) can be modified to Z Tt

---

Published as a conference paper at ICLR 2025
achieving k-step skip to sample more distant future intermediate states. Also, when & * 1, the buffer Q from Eq. (13) has various computational origins. This can be accomplished by modify- ing "springboard" zt in Eq. (12) to Zt.,.. which represents h (h I k) different springboard selections. We collectively refer to this multi-step skipping and different "springboard" selection strategy as PFDiff-k J. (h S k). The algorithmic process is illustrated in Fig. [ and Algorithm Additionally, through the comparison of sampling trajectories between PFDiff-1 I and a first-order ODE sampler, as shown in Fig. cases its capability to correct the sampling trajectory of the first-order ODE sampler while reducing the NFE. Meanwhile, we observed that PFDiff completes two updates with just one score compu- tation (1 NFE), which is equivalent to achieving an update process of second-order ODE solver with 2 NFE. This effectiveness is derived from PFDiff's cnfformation-ffficiia utilizes both past and future scores that are complementary and indispensable. The convergence of PFDiff's sampling outcomes to the data distribution consistent with solver e relies on the Mean Value Theorem, detailed in Appendix B.3 Finally, it is important to emphasize that although PFDiff is orthogonal to an arbitrary ODE solver, PFDiff can also be viewed as an independent ODE solver, depending the perspective.
I with further details provided in Appendix [
Ic PFDiff-1 show-
update process, which
3.5 ANALYSIS OF EFFECTIVENESS BASED ON THE SHAPE OF THE TRAJECTORY
In Proposition 3.1 we theoretically analyze how PFDiff corrects the error of the first-order ODE solver to achieve efficient sampling. In this section, we explain the effectiveness of PFDiff from the perspective of the trajectory's s geometric shape. Previous studies have explored the sampling trajec-
tories of diffusion models (Sabour et al. 2024 Zhou et al. 2024 Chen et al. 2024 Zhou et al. (202)) pointed out that the sampling trajectories of DPMs li in low-dimensional subspace bedded in a high-dimensional space, and the trajectory shapes closely resemble a straight line. This finding supports the strategy of using past scores to replace the current score in PFDiff reliable. Moreover, [Chen et al.] (2024) further noted that the sampling trajectories exhibit a "boomerang" shape, meaning the curvature of the sampling trajectory starts small, then increases, and finally de- creases. Based on this observation, we can analyze that the first-order ODE solver, which samples along the tangent direction, leads larger discretization errors in regions of the trajectory with large curvature. On the other hand, PFDiff uses future scores to predict the future update direc- tion, thereby correcting the discretization errors introduced by sampling along the tangent direction. In Fig. [c] we vividly demonstrate the sampling correction process of PFDiff for first-order ODE solvers, thereby validating the effectiveness of PFDiff.
Algorithm PFDiff-
Require: initial value æT, NFE N, model cg. any p-order solver & 1: Define time steps M with M 2N 1p
2: Eto 2T
3: Q butter
(fEo(EE,) &n) ) to
>> Initialize buffer
4: Ht, = =(Q, R2oz la, l1)
5: for i + to
2 do
6:
if (i-1) mod 2 2 then
7:
Sess = =QQ 4, 1411)
b Updating guided by past scores
8:
Q buffer
(Eo(TT {n)) tu t4t2
> Update buffer (overwrite)
9:
if p then
10:
Tt, = s(Q, Tr,, Li, (++2)
> Anticipatory updating guided by future scores
else if p > 1 then
12:
Tt = sQ2, T.... 41 (441,442)
The higher-order solver uses only past scores
end if
14:
end if
end for
16:
return Te
7

---

Published as a conference paper at ICLR 2025
4 EXPERIMENTS
In this section, validate the effectiveness of PFDiff orthogonal and training- sampler through a series of extensive experiments (the results of the visualization experiment are shown in Figs. 1 of Appendix (.9). This sampler can be integrated with any order of ODE solvers, thereby significantly enhancing the sampling efficiency of various types of pre-trained DPMs. To systematically showcase the performance of PFDiff, we categorize the pre-trained DPMs into two main types: conditional and unconditional. Unconditional DPMs are further subdivided into discrete and continuous, while conditional DPMs are subdivided into classifier guidance and classifier-free guidance. In choosing ODE solvers, we utilized the widely recognized first-order DDIM (Song al., 2020a), Analytic-DDIM (Bao et al., [002b)) and the higher-order DPM-Solver (Lu et al., 2022a) as baselines. For each experiment, we use the Fréchet Inception Distance (FID)) (Heusel ct al. )) as the primary evaluation metric, and provide the experimental results of the Inception Score (ISf) (Salimans al. 2016) in the Appendix D.7 reference. Lastly, apart from the ablation studies on parameters le and discussed in Sec. [.3] we showcase the optimal results of PFDiff- k h (where k = 2, and h k) across six configurations as a performance demonstration of PFDiff. As described in Appendix [ this does not increase the computational burden in practical applications. All experiments
conducted
NVIDIA RTX 3090 GPU.
4.1 UNCONDITIONAL SAMPLING
For unconditional DPMs, we selected discrete DDPM (Ho et al. 2020) and DDIM (Song et al.) 2020a), as well as pre-trained models from continuous ScoreSDE (Song et al., 2020b), to assess the effectiveness of PFDiff. For these pre-trained models, all experiments sampled 50k samples to compute evaluation metrics.
For unconditional discrete DPMs, first select first-order ODE solvers DDIM (Song ct al. 2020a)
and Analytic-DDIM (Bao et al. 2022b) as baselines, while implementing SDE-based DDPM (Ho)
al. 2020) and Analytic-DDPM (Bao et al. 2022b) methods for comparison, where 1 1.0 is
from Og in Eq. (6) We conduct experiments on the CIFAR1O (rizhevsky et al. 2009) and CelebA 64x64 (Liu et al. 2013)) datasets using the quadratic time steps employed by DDIM. By varying the NFE from 6 to 20, the evaluation metric FID. is shown in Figs. aaland 3b Additionally, experiments with uniform time steps conducted the CelebA 64x64, L-UN-
256x256 (Yu al., 2015). and LSUN-church 256x256 Yu et al. 2015) datasets, with more results available in Appendix D.2 Our experimental results demonstrate that PFDiff, based on pre-trained models of discrete unconditional DPMs, significantly improves the sampling efficiency of DDIM and Analytic-DDIM, samplers across multiple datasets. For instance, on the CIFARIO dataset, PFDiff combined with DDIM achieves FID of 4.10 with only 15 NFE, comparable to DDIM's performance of 4.04 FID with 1000 NFE. This something other time-step skipping algorithms (Bao et al. 2022b Ma al., 2024) that sacrifice sampling quality for speed cannot achieve. Furthermore, in Appendix D.2 by varying 77 from 1.0 to 0.0 in Eq. (6) to control the scale of noise introduced by SDE, we observe that 17 decreases (reducing noise introduction), the performance of PFDiff gradually improves. This is consistent with the trend shown in Fig. [Ta where reducing noise introduction leads to an improvement in the similarity of the model's outputs.
For unconditional continuous DPMs, we choose the DPM-Solver-1, -2 and -3 (Lu et al. 2022a) as the baseline to verify the effectiveness of PFDiff an orthogonal timestep-skipping algorithm on the first and higher-order ODE solvers. We conducted experiments on the CIFARIO (Krizhevsky al. 2009) using quadratic time steps, varying the NFE. The experimental results using FID,, as the evaluation metric are shown in Fig. 3c More experimental details can be found in Appendix D.3 We observe that PFDiff consistently improves the sampling performance over the baseline with fewer NFE settings, particularly in cases where higher-order ODE solvers fail to converge with
a small NFE (below 10) (Lu et al. 2022a).
4.2 CONDITIONAL SAMPLING
For conditional DPMs, we selected the pre-trained models of the widely recognized classifier guid- ance paradigm, ADM-G (Dhariwal & Nichol| 2021). and the classifier-free guidance paradigm, Stable-Diffusion (Rombach et al. 2022), validate the effectiveness of PFDiff. We employed
8

---

Published as a conference paper at ICLR 2025
DONBTE
40
DOM
40
DOM
DOM Pawytt DOMPRRS
Sww.Satreneee 200 1000 Sidwe
24
30
26
20
14
(a) CIFARIO (Discrete)
(b) CelchA 64x64 (Discrete)
(c) CIFAR1O (Continuous)
Figure 3: Unconditional sampling results. We report the FID4 for different methods by varying the
number of function evaluations (NFE), evaluated on 50k samples.
201
20
2.30
0
30
30
MFC
(a) ImageNet 64x64 (Classifier Guidance, s=1.0)
(b) MS-COC02014
(c) MS-COC02014 "Stable-Diffussee
(Stable-Diife DifffusDDD
(Classifier-Free Guidance, s=1.5)
(Classifier- Guidance, s=7.5)
Figure 4: Conditional sampling results. We report the FID for different methods by varying the NFE, Evaluated: ImageNet 64x64 with 50k, others with 10k samples. *AutDDiffusion (Li et al.
2023
requires additional search costs. We borrow the results reported in DPM-Solver-v3 (Zheng
et al. 2023 directly.
uniform time steps setting and the DDIM (Song et al., 2020a) ODE solver as a baseline across all datasets. Evaluation metrics were computed by sampling 50k samples on the ImageNet 64x64 (Deng et al., 2009) dataset for ADM-G and 10k samples on other datasets, including ImageNet 256x256 in ADM-G and MS-COC02014 (Lin et al. 2014 in Stable-Diffusion.
For conditional DPMs employing the classifier guidance paradigm, we conducted experiments on the ImageNet 64x64 dataset with a guidance scale (s) set to 1.0. For comparison, we implemented DPM-Solver-2 and -3 (Lu et al. 2022a),, and DPM Solver++(2M) (Lu al., 2022b), which hibit the best performance on conditional DPMs. Additionally, we introduced the AutoDiffusion method (Li et al.
2023) using DDIM baseline for comparison, noting that this method incurs
additional search costs. We compared FID., scores by varying the NFE as depicted Fig. 4a with corresponding visual comparisons shown in Fig. 7b We observed that PFDiff reduced the FID from 138.81 with 4 NFE in DDIM to 16.46, achieving an 88. 14% improvement in quality. The visual results Fig. 76 further demonstrate that, at the same NFE setting, PFDiff achieves higher- quality sampling. Furthermore, we evaluated PFDiff's s sampling performance based on DDIM on the large-scale ImageNet 256x256 dataset. Detailed results are provided in Appendix D.4
For conditional, classifier-free free guidance paradigms of DPMs, we employed the sd-vl 1- check- point and computed the FIDJ, scores on the validation set MS-COC02014 (Lin et al. 2014). We conducted experiments with guidance scale (s) set to 7.5 and 1.5. For comparison, we implemented (Lu et al. 2022b) methods. At 7.5, we introduced the ofte-offthe-ar method reported in DPM-Solver-v3 (Zheng et al. 2023) for comparison, along with PPM-Solver+++MM (Lu et al. 2022b), UniPC (Zhao et al. 2023) ) and PPM-Solver-v3(2M) The FID,, metrics by varying the NFE presented in Figs. 4b and 4c with additional visual results illustrated in Fig. 7a We observed that PFDiff, solely based on DDIM, achieved state-of-the-ar results during the sampling process of Stable-Diffusion, thus demonstrat-
DPM-Solver-2 and -3 (Lu et al. 2022a), and DPM-Solver++

---

Published as a conference paper at ICLR 2025
Table 1: Sample quality measured by FIDJ on the MS-COC02014 dataset (Lin al. 2014). using Stable-Diffusion (Rombach et al. (2022) pre-trained model with guidance scale of 7.5, varying the number of function evaluations (NFE). Evaluated with 10k samples.
NFE
Method
4
6
8
10
15
20
DPM-Solver-1 (Lu ct al.
2022a
35,48
20.33
17.46
16.78
16.08
15.95
PFDiff
29.02
15.47
13.26
13.06
13.57
13.97
DPM-Solver-2
(u ct al.
2022a
184.21
157.95
148.67
135.81
92.62
40.47
PFDiff
147.20
106.24
57.07
31.66
17.87
14.13
ing the efficacy of PFDiff. Further experimental details can be found in Appendix D.5 Additionally, to further validate the orthogonality PFDiff, we conducted experiments on the original (single- step) DPM-Solver-1 and -2, comparing the performance with and without the PFDiff, using the Stable-Diffusion pre-trained model, as shown Tab. [ The experimental results demonstrate that PFDiff effectively enhances the performance of DPM-Solver across different orders.
4.3 ABLATION STUDY
We conducted ablation experiments on the six different algorithm configurations of PFDiff men- tioned in Appendix C with & 1, (h k)). Specifically, evaluated the FID scores the unconditional and conditional pre-trained DPMs (Ho et al. 2020 Dhariwal & Nichol 2021) Detailed experimental setups and results can be found in Appendix D.6.1 The experimental sults indicate that for various pre-trained DPMs, the choice of parameters A and h is not critical, as most combinations of k and h within PFDiff can enhance the sampling efficiency over the base- line. Moreover, with k = 2 and h = 1 fixed, PFDiff-2 I can always improve the baseline's 's sampling quality within the range of 4~200 NFE. For even better sampling quality, one can sample a small sub- set of examples (e.g., 5k) to compute evaluation metrics or directly conduct visual analysis, easily identifying the most effective and combinations. Furthermore, in Appendix [D..1] we propose an automatic search strategy with almost no additional cost, which can more rapidly obtain more competitive combinations of l and h based on truncation error.
To validate the effectiveness of PFDiff, key factor is its informmttion-effcciine update process, which uses both past and future scores that are complementary and indispensable in jointly guiding first- order ODE solvers. We employ DDIM (Song et al. 2020a) the baseline, removing past and future scores separately. Moreover, we introduce methods (Ma et al. 2024) that cache part of past scores for comparison. As shown in Appendix D.6.2, experimental results indicate that only past (including cache) or only future scores can slightly improve sampling performance, but their combination (i.e., the complete PFDiff) significantly enhances the performance of first-order ODE solvers, especially with very few NFE (<10). Additionally, we provide experimental results on inference time in Ap- pendix [.6.2] revealing that methods (Ma al., (2244) that cache part of past scores not only incur additional inference costs but also exhibit relatively weak acceleration effects with few NFE (<10). However, PFDiff and the used baseline have consistent inference times and exhibit significantly accelerated effects, further validating its effectiveness.
5 CONCLUSION
In this paper, based on the recognition that the ODE solvers of DPMs exhibit significant similarity in model outputs when the time step size is not excessively large, and with the aid of a foresight update mechanism, we propose PFDiff, a novel method that leverages past and future scores to rapidly update the current intermediate state. This approach effectively reduces the unnecessary number of function evaluations (NFE) in the ODE solvers and significantly corrects the errors of first-order ODE solvers during the sampling process. Extensive experiments demonstrate the orthogonality and effectiveness of PFDiff on both unconditional and conditional pre- pre-trained DPMs, especially in conditional pre-trained DPMs where PFDiff outperforms previous state-of-the art training free sampling methods.
10

---

Published as a conference paper at ICLR 2025
ETHICS STATEMENT
DPMs, like GANs and VAEs, may utilized deep generative models for generating fake and ma- licious content. The proposed PFDifT can accelerate the generation of DPMs, which may facilitate the rapid creation of such content, thereby posing potential negative impact on society.
REPRODUCIBILITY STATEMENT
Our code is based the official implementations of DDIM (Song et al. 2020a). DPM-Solver (Lu
et al. 2022a), and Analytic-DPM, (Bao et al. 2022b). We utilized unconditional checkpoints from DDPM (Ho et al. 2020), DDIM (Song al. 2020a), and ScoreSDE (Song al. (020b), well
as conditional checkpoints from AMD-G (Dhariwal & Nichol 2021) and Stable- StableDDiffusion (Rom- bach al. 2022). Detailed experimental settings and algorithm implementations are described in
Appendices and D
REFERENCES
Fan Bao, Chongxuan Li, Jiacheng Sun, Jun Zhu, and Bo Zhang. Estimating the optimal covariance with imperfect mean in diffusion probabilistic models. arXiv preprint arXiv:2206. 07309, 2022a.
Fan Bao, Chongxuan Li, Jun Zhu, and Bo Zhang. Analytic-dpm: an analytic estimate of the optimal reverse variance in diffusion probabilistic models. arXiv preprint arXiv:2201 (000:2001.0603,, 2022b.
James Betker, Gabriel Goh, Li Jing, Tim Brooks, Jianfeng Wang, Linjie Li, Long Ouyang, Juntang Zhuang, Joyce Lee, Yufei Guo, al. Improving image generation with better captions. Computer Science. https://cdn. openai. com/papers/dull-e33 pdf, 2(3):8, 2023.
Defang Chen, Zhenyu Zhou, Can Wang, Chunhua Shen, and Siwei Lyu. On the trajectory regularity of bde-based diffusion sampling. Forty-first International Conference on Machine Learning. 2024.
Elliott Ward Cheney, EW Cheney, and W Cheney, Analysis for applied mathematics, volume 1. Springer, 2001.
Mostafa Dehghani, Basil Mustafa, Josip Djolonga, Jonathan Heek, Matthias Minderer, Mathilde Caron, Andreas Steiner, Joan Puigcerver, Robert Geirhos, Ibrahim Alabdulmohsin, al. Patch n' pacck: Navit, a vision transformer for any aspect ratio and resolution. In Proceedings of the 37th International Conference Neural Information Processing Systems, pp. 2252-2274, 2023.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hi- crarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pp. 248-255. Icee, 2009.
Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. Advances in neural information processing systems, 117888887744 2021.
John R Dormand and Peter J Prince. A family of embedded runge-kutta formulae. Journal of computational and applied mathematics, 6(1):19-26, 1980.
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farles Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. Advances in neural information processing systems, 27, 2014.
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to local nash equilibrium. Advances in neural information processing systems, 30, 2017.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840-6855 2020.
Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion- based generative models. Advances in neural information processing systems, 14-26565-265777 2022.
11

---

Published as a conference paper at ICLR 2025
Tero Karras, Miika Aittala, Jaakko Lehtinen, Janne Hellsten, Timo Aila, and Samuli Laine. Analyz- ing and improving the training dynamics of diffusion models. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattemn Recognition, pp. 24174-2418, 2024.
Diederik Kingma and Max Welling.
Auto-encoding variational bayes.
arXiv preprint
arXiv: 14, 2013.
Peter E Kloeden, Eckhard Platen, Peter Kloeden, and Eckhard Platen. Stochastic differential equations. Springer, 1992.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
Lijiang Li, Huixia Li, Xiawu Zheng, Jie Wu, Xuefeng Xiao, Rui Wang, Min Zheng, Xin Pan, Fei Chao, and Rongrong Autodiffusion: Training-free optimization of time steps and architec- tures for automated diffusion model acceleration. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 7105-7114, 2023.
Chen-Hsuan Lin, Jun Gao, Luming Tang, Towaki Takikawa, Xiaohui Zeng, Xun Huang, Karsten Kreis, Sanja Fidler, Ming-Yu Liu, and Tsung-Yi Lin. Magic3d: High-resolution text-to-3d con- tent creation. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 300-309, 2023.
Tsung-Y Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects context. In Computer Vision-ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13, pp. 740-755. Springer, 2014.
Luping Liu, Yi Ren, Zhijie Lin, and Zhou Zhao. Pseudo numerical methods for diffusion models manifolds. arXiv preprint arXiv:2202.09778 2022a.
Xingchao Liu, Chengyue Gong, and Qiang Liu. Flow straight and fast: Learning generate and transfer data with rectified flow. arXiv preprint arXiv:2209.0300 2022b.
Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. In Proceedings of the IEEE international conference on computer vision, pp. 3730-3738, 2015.
Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu. Dpm-solver: a fast ode solver for diffusion probabilistic model sampling in around 10 steps. In Proceedings of the 36th International Conference on Neural Information Processing Systems, pp. 5775-5787, 2022a.
Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu. Dpm-solver++: Fast solver for guided sampling of diffusion probabilistic models. arXiv preprint arX1v:2211.01095, 2022b.
Xinyin Ma, Gongfan Fang, and Xinehao Wang. Deepeache: Accelerating diffusion models for free.
In Proceedings of the IEFF/CVF Conference on Computer Vision and Pattern Recognition, pp- 15762-15772, 2024.
Yurii Nesterov. A method of solving convex programming problem with convergence rate (1/k**
2). Doklady Akademii Nauk SSSR, 269(3):543, 1983.
Bernt Øksendal and Bernt Øksendal. Stochastic differential equations. Springer, 2003.
William Peebles and Saining Xie. Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 4195-4205. 2023.
Ben Poole, Ajay Jain, Jonathan T Barron, and Ben Mildenhall. Dreamfusion: Text-to-3d using 2d diffusion. arXiv preprint arXiv:2209. 14988, 2022.
Herbert Robbins and Sutton Monro. A stochastic approximation method. The annals of mathemati- cal statistics, pp. 400-407, 1951.
12

---

Published as a conference paper at ICLR 2025
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF confer- on computer vision and pattern recognition, pp. 10684-10695, 2022.
Amirmojtaba Sabour, Sanja Fidler, and Karsten Kreis. Align your steps: Optimizing sampling schedules in diffusion models. arXiv preprini arXiv:2404. 2024.
Tim Salimans and Jonathan Ho. Progressive distillation for fast sampling of diffusion models. arXiv preprint arXiv:2202.00512. 2022.
Tim Salimans, Ian Goodfellow, Wojeiech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen. Improved techniques for training gans. Advances in neural information processing systems, 29, 2016.
Jascha Sohl- Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In International conference on machine learn- ing. pp. 2256-2265. PMLR, 2015.
Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02500. 2020a.
Kaitao Song, Yichong Leng, Xu Tan, Yicheng Zou, Tao Qin, and Dongsheng Li. Transcormer: Transformer for sentence scoring with sliding language modeling. Advances in Neural Informa- fion Processing Systems, 35:1 155111600-1174, 2022.
Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. arXiv preprint arXiv:2011.13456, 2020b.
Yang Song. Prafulla Dhariwal, Mark Chen, and Ilya Sutskever. Consistency models. arXiv preprint arXiv:2303.01469, 20233
Kai Wang. Zhaopan Xu, Yukun Zhou, Zelin Zang, Trevor Darrell, Zhuang Liu, and Yang You. Neural network diffusion. arXiv preprint 14X14:4402.13144, 2024.
Felix Wimbauer, Bichen Wu, Edgar Schoenfeld, Xiaoliang Dai, Hou, Zijian He, Artsiom Sanakoyeu, Peizhao Zhang, Sam Tsai, Jonas Kohler, et al. Cache me if you can: Accelerat- ing diffusion models through block caching. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6211-6220, 2024.
Shuchen Xue, Mingyang Yi, Weijian Luo, Shifeng Zhang, Jiacheng Sun, Zhenguo Li, and Zhi-Ming Ma. Sa-solver: stochastic adams solver for fast sampling of diffusion models. In Proceedings of the 37th International Conference Neural Information Processing Systems, pp. 77632-77674, 2023.
Tianwei Yin, Michaël Gharbi, Richard Zhang, Eli Shechtman, Fredo Durand, William Freeman, and Taesung Park. One-step diffusion with distribution matching distillation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 661 6613-6623, 2024.
Fisher Yu, Ari Seff, Yinda Zhang, Shuran Song, Thomas Funkhouser, and Jianxiong Xiao. Lsun: Construction of large-scale image dataset using deep learning with humans in the loop. arXiv preprint arXiv: 150.00336, 2015.
Wenliang Zhao, Lujia Bai, Yongming Rao, Jie Zhou, and Jíwen Lu. Unipe: a unified predictor- corrector framework for fast sampling of diffusion models. Proceedings of the 37th Interna- tional Conference on Neural Information Processing Systems, pp. 49842-49869, 2023.
Kaiwen Zheng, Cheng Lu, Jianfei Chen, and Jun Zhu. Dpm-solver-v3: improved diffusion ode solver with empirical model statistics. In Proceedings of the 37th International Conference on Neural Information Processing Systems, pp. 55502-55542, 2023.
Zhenyu Zhou, Defang Chen, Can Wang, and Chun Chen. Fast ode-based sampling for diffusion models in around 5 steps. In Proceedings of the IEEE/CVF Conference on Computer Vision and Patlern Recognition, pp. 7777-7786, 2024.
13

---

Published as a conference paper at ICLR 2025
A RELATED WORK
While the solvers for Diffusion Probabilistic Models (DPMs) are categorized into two types, SDE and ODE, most current accelerated sampling techniques are based on ODE solvers due to the obser- vation that the stochastic noise introduced by SDE solvers hampers rapid convergence. ODE-based
solvers are further divided into training-based methods (Salimans & Ho 2022
Liu et al., 2022b
Song et al.
2023 Yin et al.
2024) and training-free -ereinn-g-eeeeee samplers (Song et al. 2020a
Lu et al. 2022a bj
Bao et al.
2022b aj Liu et al. [2022a] Li et al., [023] Zheng et al.) 2023 Ma et al. 2024, Wim-
bauer ct al. 2024; Zhao et al. 2023] Xue et al. 2023). Training-based methods can notably reduce
the number of sampling steps required for DPMs. An example of such method is the knowledge
distillation algorithm proposed by [ong et al. (2023), which achieves one-step sampling for DPMs.
that of GANs (Goodfellow al.) 2014) and VAEs (Kingma) & Welling, 2013). However, these methods often entail significant additional costs for distillation training. This requirement poses challenge when applying them large pre-trained DPM models. Therefore, our work primarily focuses on training-free, ODE-based accelerated sampling strategies.
This sampling speed is comparable
Training-free free accelerated sampling techniques based on ODE can generally be applied in plug-and- play manner, adapting to existing pre-trained DPMs. These methods can be categorized based on the order of the ODE solver- that is, the NFE required per sampling iteration-inu first-order (Song
et al. 2020a;
Bao et al.
2022b Liu et al. 2022a) and higher-order (Lu et al. 2022a Zheng
al.
2023 Zhao al.
2023] Dormand & Prince 1980). Typically, higher-order ODE solvers
tend to sample at a faster rate but may fail to converge when the NFE is low (below 10), sometimes performing even worse than first-order ODE solvers. In addition, there are orthogonal techniques for accelerated sampling. For instance, [i al. (2023) build upon existing ODE solvers and use search algorithms to find optimal sampling sub-sequence and model structures to further speed up the sampling process: Ma et al. (2024) and Wimbauer et al. (2024) observe that the low-level features of noise networks at adjacent time steps exhibit similarities, and they use caching techniques to substitute some of the network's low-level features, thereby further reducing the number of required time steps.
The algorithm propose belongs to the class of training-free and orthogonal accelerated sampling techniques, capable of further accelerating the sampling process on the basis of existing first-order and higher-order ODE solvers. Compared to the aforementioned orthogonal sampling techniques, even though the skipping strategy proposed by Ma et al. (2024) and Wimbbuer et al. (202)) effec- tively accelerates the sampling process, may do so at the cost of reduced sampling quality, making it challenging to reduce the NFE below 50. Although Li et al.] (2023) can identify more optimal subsampling sequences and model structures, this implies higher search costs. In contrast, our pro- posed orthogonal sampling algorithm is more efficient in skipping time steps. First, our skipping strategy does not require extensive search costs. Second, we can correct the sampling trajectory of first-order ODE solvers while reducing the number of sampling steps required by existing ODE solvers, achieving more efficient accelcrated sampling.
B PROOF OF CONVERGENCE AND ERROR CORRECTION FOR PFDIFF
In this section, first prove that neglecting higher-order derivative terms has smaller impact on the first-ODE solvers when using future scores (i.e., Proposition 3.1). Subsequently, we prove the convergence of PFDiff based on the Mean Value Theorem for Integrals.
B. ASSUMPTIONS
For h we make the following assumptions:
Assumption B.1. The higher-order derivatives s!n)) (Ix,T) (as a function ofr), as defined in Eq. 15,. where E [f:-1,t]] tæ] and n 2 1, exist and are continuous.
Assumption B.2. When the time step size At= =t; (k-kk)) is not excessively large, the output estimates of the noise network based on the p-order ODE solver at different time steps are approxi- p-1 mately the same, that is. (fcolōz,,fu) ( te, t(th) (fco(0r, (t) {n)) tt- (kt(k=) ts).
14

---

Published as a conference paper at ICLR 2025
Assumption B.3. For the integral from time step (1+(++l) St: flititt) s(co(ar, t), x2, t)dt,
there exist intermediate time steps la L. E ((+41)) such that St: flit0tt) s(Ea( v2, Ie, ljdl s(eg(Its) fa); 3tzs (2) (1++0++1) 14) ih(ca(ae, 1.) Fe. (2) (1++(1+1) - 14) holds, where the definition of the function h remains consistent with Sec. 3.7
The first assumption ensures the application of Taylor expansion in Eq. 15 The second assumption based on the observation in Fig. Ia that when At is not excessively large, the MSE the noise network remains almost unchanged actoss different time steps. The last one is based on the Mean Value Theorem for Integrals, which states that if (z) is a continuous real-valued function on a closed interval [a, b). then there exists least one point E [a, b] such that S (x))dx f(c) (a-(bbaa) a) holds.
Remark 3. It is important to note that the Mean Value Theorem for Integrals originally applies to real-valued functions and does not directly apply vector-valued functions (Cheney et al. (007). However, the study by Zhou et al.) (2024), which uses Principal Component Analysis (PCA) on the trajectories the ODE solvers for DPMs, demonstrates that these trajectories almost lie in a dimensional plane. The finding ensures the applicability of the Mean Value Theorem for Integrals in Assumption B.3
B.2 PROOF OF PROPOSITION 3.1
In this section, we prove that Eq. 16 holds, where E (1i-1,4; and n 2 2. First, given e E (t)-,,4), we have:
f:2 E| + [t,-1 El = lt, tz-1/.
(B.1)
Next, we analyze the Eq. 16 based on the parity of n.
When is even (n 2 2): We derive:
(t; - 2) (f--1 - 8)nl
1/ - El"
M1 8
n!
n!
1/4 n!
14-1 8 1/m
max
(B.2)
nn!
|t4
t--1
n
(t: t )
n1!
n!!
Here, the second "<" holds because: due to E E (4)-1,4) and Eq. 14, - 4-11 and |4-1 < 14, 4-1/1 thus validating the second "<"
B.
we have 1t, e <
When n is odd (n 2 3): Since e E (4--1,4), if - & >>> 0. then 4--1 - 8 if - < 0, then 4-1 - 0. Therefore, we obtain:
(ta =)" (t,-1 =)
I - E
[/: El
(B.3)
n!
n!
Let a |4 - E/. 6 1 E/. and = |/: - ti-1l: we have a, e > 0 and e >> a, b. Next, using
mathematical induction, we prove b" < c,, where n 2 3 and a + (Eq. B.I..
When n = 3, we have:
c (a b)3 a + +aab 3ab² + b3 > a +
(B.4)
which holds.
When (k: 3, N), suppose then t + 6t * holds.
When = k + 1, we have:
a b+1 b++1
a + b. b <b
a² bb. ba = (a + + b. ct < ctt) (B.5)
which holds.
Thus, a 6 < c" holds, where and
+ b = c. Furthermore, obtain:
144 e + 1/-1 el"
11 4-11"
(4, 4-11""
(B.6)
n!
n!
n!
In conclusion, by combining Eq.
B.2 Eq.
B.3
and Eq. (B.6 have proven Eq. (6).
15

---

Published as a conference paper at ICLR 2025
B.3 PROOF OF CONVERGENCE FOR PFDIFF
A Assumption B.2 ensures the convergence of PFDift-k h using past scores. Starting from Eq. we consider iteration process of p-order ODE solver from 3.; 3 where h is the springbourd" choice determined by Jr This iterative process can be expressed as:
Nith
Besnh = 3z, +
s(co(1e, 2e, t)dt.
(B.7)
Discretizing Eq. (B77 yields:
h(co ( &n), (fn+1 11-41-1n f.n)
==0
(B.8)
ith-1
St, +
Σ
; : (n) (tn+1 bn),
where the function h represents the different solution methodologies applied by various p-order ODE solvers to the function 8, consistent with Sec.
3.1
To accelerate sampler convergence namely guiding the sampling of the
and reduce unnecessary NFE, we adopt Assumption B.2
current intermediate state by utilizing past score information. Specifically, we approximate that (few(Tz,, (= ti, 1eth (f, (m)) n=o 44- (k-))))))), where k represents the num- ber of steps skipped in one iteration by PFDiff-k Eq. (B.8) can be further rewritten as:
23 Bt. +
Σ
b(co(Et, st), Fen (tn+1 - tn)
n= (M-B-1)
(B.9)
= = ( (n) : (4-(-- Tre tis tstn);
where ò is any p-order ODE solver. Eq. B.9. demonstrates that under Assumption B.2 PFDiff- k-h utilizes past scores to replace current scores, converging to the same data distribution as that of any p-order ODE solver é. However, noted in Remark 2 the time step size At is very large (NFE<10), making "springbeard" ot,1, unreliable in Eq. (B.9). Therefore, we only use ε,,, predict foresight update direction (i.e., the future score). "The introduction of the future score can reduce the impact of neglecting higher-order derivative terms, thus correcting the discretization errors of the first-order ODE solvers.
Convergence of PFDiff-k.h using future scores. As described in Sec. 3.3 higher-order ODE
solvers and future scores reduce the discretization error caused by neglecting higher-orde derivative terms in two parallel manners. Therefore, PFDiff combines higher-order ODE solver using only past scores, with convergence guarantees based on Assumption B.2 Next, we consider an iteration process of a first-order ODE solver from हt, to Et;-(41))> which can be expressed
= -t, + f ((alla11)
S(Ea(IE, t), x1, t)dt
J: z ãr, + (((ee(Te, (1), Ze, ti) :)) (1++(x-)) = :(coo (it,, , 14), 5r,, be ((+(++)))
(B.10)
where the second line is obtained by discretizing the first line with existing first-order ODE solver, and the definitions of á and are consistent with Sec. 3.1 it well known that the "z²" in Eq. (0.10p introduces discretization errors. We have revised Eq. B.10) based on Assumption B.3
as follows:
Ttstaen) = =ū,, / ( s(Ea(Zr, t), xe. t)dt
It, + s tz), ( (1++(8+1) - tt)
(B.11)
-r, + hh(eel(Te, to), Tr, to) t2) 42) =fcol tw), te, (1)()))))
16

---

Published as a conference paper at ICLR 2025
Eq. (B.11) indicates that there is optimal time point i, E [t,, (k++))))) corresponding to the optimal score 6p (it, 1.)) that can correct the discretization error of Eq. (0).0). Furthermore, when the time step size At (++(K+1) 4 very large (for example, NFE<T0), using the score at the current time point E leads to non-convergence of the sampling process. This implies that the sampling trajectory of DPMs is not a straight line (if it were a straight line, a larger sampling step size could be used). Therefore, the optimal time point is not achieved at the endpoints, i.e,, t, # to and f, * (k+(k+))= and we adjust that to falls within the interval (fa, (+4 Additionally, approximate the optimal score, introduce the foresight update mechanism of the Nesterov momentum (Nesterov, 1983), and guide the current intermediate state sampling with future score information. In other words, we replace eg(#e, (a) with eo (F (4th), follows:
11 = (TT to), It, ti (++(++1))
(B.12)
@0o 1eth), Bees tis (k+1)))
where k and h are determined by the selected PFDifT-k h. According to the definition of PFDiff- k h, lith also lies within the interval (42; ((++)) For six different versions of PFDiff-k: h defined in Appendix C we believe the optimal ta within the interval (t,, (x+(1))) has been approximated, thereby completing the convergence proof of using future scores. Finally, we note that PFDiff using future scores to replace current scores is approximation of the optimal score. Together with this section and Proposition [3.]] (future scores have less impact at neglecting higher-order derivative terms), jointly verify that future scores can more effectively guide first-order ODE solver in sampling.
C ALGORITHMS OF PFDIFFS
As described in Sec. 3.4 during a single iteration, we can leverage the foresight update mech- distant future. Specifically,
anism to skip to Φ(Q, Lt, t4, (k+1)) to achieve k-step skip. We refer to this method PFDiff-k. Additionally, when k + 1, the computation of the buffer Q, originating from Eq. (13), presents different selection choices. We modify Eq. 12) to Tteth +1Q, 51, bi (4th), h S l denote different "springboard"
modify Eq. (14) to Rt R
Algorithm 2
Require: initial value TT, NFE N, model eg. any p-order solver c, skip type
1: Define time steps {t, MM with M 3.V 2p
2: Tto + FT
3: Q baffer
((eo(Eg,)) ( lo. 11
Initialize buffer
4: Hc, =0(2 01g, to, t1)
5: for i 110
do
6: if (i - 1) mod 3 = then
7:
if h then
PFDiff-2.1 > Updating guided by past scores D Update buffer (overwrite) PFDiff-2
8:
Thus ( Dr, to t++1)
Q buffer
(5ol3 (n)) p0-1 let (411,
9:
else h then
11:
Tt buffer
( Tr,, (i11)2)
Updating guided by past scores Update buffer (overwrite)
12:
(1ol (n) (t)) |--) 14221443
13:
end
14:
if p 1 then
15:
Tt, = s(Q Fr,, , li, 4++3)
D Anticipatory updating guided by future scores
else if p > 1 then
17:
It..s (Q, T a tiths t443)
The higher-order solver uses only past scores
end
19:
end if
end for
21:
return Tt
17

---

Published as a conference paper at ICLR 2025
Algorithm 3 PFDiff-3
Require: initial value æT, NFE N, model cg. any p-order solver o, skip type h
1: Define time steps MM with M 4N -
2: Tto 2T
3: Q
baffer
((co(Es,, En) B to.t1
b Initialize buffer
4: Zt, = =Q(Q, Et tu, t1)
5: for + 1
do
6:
if (i-1) mod 4 = then
7:
Bes41 = "Q to t(th)
Updating guided by past scores
8:
buffer
(fol {n) ditn lots)
Update buffer (overwrite)
9:
if p 1 then
10:
Tie. = pQQ Bra tip (444)
Anticipatory updating guided by future scores
else p then
12:
It. (Q, Tt, tn li li44)
The higher-order solver uses only past scores
end if
14:
end if
end for
16:
return ZTM
choices with the parameter h. This strategy of multi-step skips and varying "springboard" choices is
collectively termed as PFDiff- (h k). Consequently, based on modifications to parameters
and h in Eq. 12 and Eq. 14 Eq.
13 is updated to Q (foo(E,,)))) (n)] lith (4(++)))))))
butter
and Eq. 11b updated Q buffer
{ ( (.))),)))))) I 4 (4((--41);+1))
When & = 1, since h < k, then h = 1, and PFDiff-k is the same as PFDiff-1, as shown in 2, h can be either
forming Algorithms PFDiff-2 and PFDiff-2 2, as shown in Algorithm 2 Furthermore, when k = 3, this forms three different this study, we utilize the optimal results from the
Algorithm [ in Sec. 3..4 When
versions of PFDiff-3, as shown in Algorithm six configurations of PFDifT-k_h (k 2, (h k)) to demonstrate the performance of PFDiff. As described in Appendix B.3 this essentially an approximation of the optimal time point t.- approximately search for the optimal t,. It
Through these six different algorithm configurations, important to note that despite using six different algorithm configurations, this does not increase the computational burden in practical applications. This because, by visual analysis of small number of generated images or computing specific evaluation metrics, one can effectively select the algorithm configuration with the best performance. Moreover,
without any selection, with k 2 and h = fixed, PFDiff-2.1 can always improve the baseline's 's sampling quality within the range of 4~20 NFEs, as shown in the ablation study results in Sec. 4.3
D ADDITIONAL EXPERIMENT RESULTS
In this section, we provide further supplements to the experiments on both unconditional and con- ditional pre-trained Diffusion Probabilistic Models (DPMs) as mentioned Sec. 4 Through these additional supplementary experiments, we more fully validate the effectiveness of PFDiff as an or- thogonal and training-fit sampler. Unless otherwise stated, the selection of pre-trained DPMs, choice of baselines, algorithm configurations, GPU utilization, and other related aspects in this sec- tion are consistent with those described in Sec.
D.1 LICENSE
In this section, we list the used datasets, codes, and their licenses in Table
18

---

Published as a conference paper at ICLR 2025
Table The used datasets, codes, and their licenses.
Name
URL
License
CIFARIO Krizhevsky et al. 2009
Cs. Moronto.edu
CelebA 64x64 Liu a.
mmlla..e.cuuk.eddu
BUUN-eerrom
y1.10
LSUN-Church Yuu ct al. 2015
y1.10
ImageNet (Deng ct all 2009
image-neta net.org
MS-COCO2014 (Lin
2014
cocodalasel.co
CC BY 4.0
ScoreSDE (Sog
202
ruthub.om/yang-some
Apache-2.0
DDIM (Song el al.
2020
gthub.com//rronggnn
MIT
Analytic-DPM DPM [ao et al.
2022b
eithub.com baotff
DPM-Solver (Lu et al. 2022a)
Cttuu..com/LuChere THU
MIT
DPM-Solver++ (Lu
2022b)
gtthub.com/Lut Ceng IHU
MIT
Guided-Diffision (Dhanwal Nichol 2021)
gthhb.com/openal
MIT
Stable-Diffusion (Rombach ct al.
2022
github.co//oopp
CreativeMI, Open RAIL-M
Table 3: Sample quality measured by FIDI the CIFARIO (Krizhevsky al. 2009), CelebA
64x64 (Liu et al. 2015), LSUN-bedroom 256x256 (Yu et al. 2015), and LSUN-church 256x256 Yuu
al. 2015) datasets using unconditional discrete-time time DPMs, varying the number of function eval- uations (NFE). Evaluated on 50k samples. PFDiff uses DDIM (Song al.
2020a and Analytic- Analytic-
DDIM (Bao et al. 2022b) as baselines and introduces DDPM (Ho et 2020)
DDPM (Bao et al. 2022b) with 7 1.0 from Eq. (6) for comparison.
NFE
PFDiff Method
8
10
12
15
20
CIFARIO (discrete- time model (Ho et 2020), quadratic time steps)
DDPM( 1.0) (Ho al. 2020
108.05
71.47
52.87
41.18
32.98
25.55
18.34
Analytic-DDPM [Bao al. 2022h)
65.81
56.37
44.09
34.95
29.96
23.26
17.32
Analytic-DDIM [ao et al. 20226
106.86
24.02
14.21
10.09
8.80
7.25
6.17
DDIM (Song et al. 2020al
65.70
29.68
18.45
13.66
11.01
8.80
7.04
Analytic-DDIM
289.84
23.24
7.03
4.51
3.91
3.75
3.65
DDIM
22.38
9.84
5.64
4.57
4.39
4.10
3.68
CclebA 64x64 (discrete-time model
(Song ct al.
2020a),
quadratic time steps)
DDPM(7 = 1.0) (Ho ct al.
2020
59.38
43.63
34.12
28.21
24.40
20.19
15.85
Analytic-DDDD DDPM [Bao et al
2022b
32.10
39.78
32.29
26.96
23.03
19.36
15.67
Analytic- DDIM [Bao et
20226)
69.75
16.60
11.84
9.37
7.95
6.92
5.84
DDIM (Song al. 2020a
37.76
20.99
14.10
10.86
9.01
7.67
6.50
Analytic-DDIM
360.21
28.24
5.66
4.90
4.62
4.55
4.55
DDIM
13.29
7.53
5.06
4.71
4.60
4.70
4.68
CelebA 64x64 (discrete-ting model (Song al.
2020
uniform time steps)
DDPM( 1.0) (Ho al.
2020
65.39
49.52
41.65
36.68
33.45
30.27
26.76
Analytic-DDPM [Boo
2022b
102.45
42.43
34.36
33.85
30.38
28.90
25.89
Analytic-DDIM [Bao et
20226
90.44
24.85
16.45
16.67
15.11
15.00
13.40
X
DDIM (Song et al. 2020a)
44.36
29.12
23.19
20.50
18.43
16.71
14.76
Analytic-DDIM
308.58
56.04
10.98
8.97
6.39
5.19
DDIM
51.87
12.79
8.82
8.93
7.70
6.44
5.66
256x256 (discrete-ting time model (Ho et al.
2020),
uniform time steps)
DDIM (Song et al. 2020a
115.63 140.40
47.40
26.73
19.26
15.23
11.68
9.26
DDIM
18.72
11.50
9.28
8.36
7.76
7.14
LSUN-church 256x256 (discrete- time model (Ho
2020),
uniform time steps)
DDIM (Sng al. 2020ai
121.95
50.02
30.04
22.04
17.66
14.58
12.49
DDIM
72.86
18.30
14.34
13.27
12.05
11.77
11.12
19

---

Published as a conference paper at ICLR 2025
D.2 ADDITIONAL RESULTS FOR UNCONDITIONAL DISCRETE TIME SAMPLING
In this section, we report on experiments with unconditional, discrete DPMs on the CI- FARIO (Krizhevsky et al. 2009) and CelebA 64x64 (Liu et al. 2013) datasets using quadratic time steps. The FIDI, scores for the PFDiff algorithm are reported for changes in the number of func- tion evaluations (NFE) from 4 to 20. Additionally, we present FID scores on the CelebA 64x64 (Liu
et at. 2015). LSUN--edroom 256x256 Yuu et al. 2015), and LSUN-church 256x256 YYu al. 2015)
datasets, utilizing uniform time steps. The experimental results are summarized in Table 3 Results indicate that using DDIM (Song et al. 2020a) as the baseline, our method (PFDiff) nearly achieved significant performance improvements across all datasets and NFE settings. Notably, PFDiff facil- itates rapid convergence of pre-trained DPMs to the data distribution with NFE settings below 10, validating its effectiveness on discrete pre-trained DPMs and the first-order ODE solver DDIM. is important to note that the CIFARIO and CelebA 64x64 datasets, have included the FIDD scores of Analytic-DDIM DDIM (Bao et al., (022b), which serves as another baseline. Analytic-DDIN modifies the variance in DDIM and introduces some random noise. With NFE lower than 10, the presence of minimal random noise amplifies the error introduced by the score information approximation in PFDiff, reducing its error correction capability compared to the Analytic- DDIM sampler. Thus, in fewer-step sampling (NFE<10) 10F, using DDIM as the baseline is more effective than using Analytic- DDIM, which requires recalculating the optimal variance introducing additional computational overhead. In other experiments with pre-trained DPMs, we validate the efficacy the PFDiff algorithm by combining it with the overall superior performance of the DDIM solver.
different pre-trained DPMs, thereby
Furthermore, to validate the motivation proposed Sec. 3.2 based on Fig. aa -that at not exces- sively large time step size At, ODE-based solver shows considerable simi limilarity in the noise net- work outputs-we compare it with the SDE-based solver DDPM (Ho et al., 2020). Even at smaller At, the mean squared error (MSE) of the noise outputs from DDPM remains high, suggesting that the effectiveness of PFDiff may be limited when based on SDE solvers. Further, we adjusted the parameter Eq. (6) (which controls the amount of noise introduced in DDPM) from 1.0 to 0.0 (at
Table 4: Sample quality measured by FID4 on the CIFARIO (Krizhevsky al. 2009) and CelebA 64x64 (Liu et al., 2013) using unconditional discrete-time DPMs with and without our method (PFDiff), varying the number of function evaluations (NFE) and 17 from Eq. 66. Evaluated on 50k samples.
NFE
Method
15
20
CIFARIO (discrete-time model (Ho al.
2020).
quadratic time steps)
DDPM(1 1.0)
(Ho et al. 2020
108.05
71.47
52.87
41.18
32.98
25.59
18.34
+PFDiff (Ours)
475.47
432.24
344.96
332.41
285.88
158.90
28.05
DDPM(1 0.5)
Song
2020a
71.08
34.32
22.37
16.63
13.37
10.75
8.38
+PFDiff (Ours)
432.50
349.09
311.62
167.65
59.93
23.17
10.61
DDPM(1 0.2)
Song al.
2020a
66.33
30.26
18.94
14.01
11.25
9.00
7.18
+PFDiff (Ours)
316.15
189.02
18.55
7.73
5.70
4.53
4.00
DDIM() 0.0) (Song al. +PFDiff (Ours)
2020a
65.70
29.68
18.45
13.66
11.01
8.80
7.04
22.38
9.48
5.64
4.57
4.39
4.10
3.68
CelebA 64x64 (discretc-time model (Song
22000a,,
quadratic time steps)
DDPM(n 1.0) (Ho et al. +PFDiff (Ours)
2020
59.38
43.63
34.12
28.21
24.40
20.19
15.85
433.25
439.19
415.41
317.43
324.58
326.50
171.41
DDPM(1 0.5) Song al. +PFDiff (Ours)
2020a
40.58
23.72
16.74
13.15
11.27
9.36
7.73
435.27
417.58
314.63
310.10
252.19
69.31
19.23
DDPM(1 0.2) Song al. +PFDiff (Ours)
2020al
38.20
21.35
14.55
11.22
9.47
7.99
6.71
394.03
319.02
45.15
12.71
7.85
5.10
4.96
DDIM 0.0) (Song et al.
2020al
37.76
20.99
14.10
10.86
9.01
7.67
6.50
+PFDiff (Ours)
13,29
7.53
5.06
4.71
4.60
4.70
4.68
20

---

Published as a conference paper at ICLR 2025
7 0.0, the SDE-based DDPM degenerates into the ODE-based DDIM (Song al. 2020a)). As shown in Fig. la as 7) decreases, the MSE of the noise network outputs gradually decreases at the same time step size At, indicating that reducing noise introduction can enhance the effectiveness of PFDiff. To verify this motivation, we utilized quadratie time steps on CIFAR1O and CelebA 64x64 datasets and controlled the amount of noise introduced by adjusting 7/ to demonstrate that PFDiff can leverage the temporal redundancy present in ODE solvers to boost its performance. The exper- imental results, as shown in Table 4 illustrate that with the reduction of r from 1.0 (SDE) to 0.0 (ODE), PFDiff's s sampling performance significantly improves at fewer time steps (NFE$20). The experiment results regarding FID variations with NFE as presented in Table align with the trends of MSE of noise network outputs with changes in time step size AL as depicted in Fig. la This reaffirms the motivation we proposed Sec. 3.2
D.3 ADDITIONAL RESULTS FOR UNCONDITIONAL CONTINUOUS- SAMPLING
In this section, we supplement the specific FIDI, scores for the unconditional, continuous pre-trained DPMs models with first-order and higher-order ODE solvers, DPM-Solver- and -3, (Lu et al, 2022a) baselines, shown in 5 For all experiments in this section, we conducted tests on the CIFAR 10 dataset (Krizhevsky et al. 2009), using the checkpoint checkpoints intoinnt__ 8.pth under configuration provided by ScoreSDE (Song ct al. 2020b). For the hyperparameter method of 3PM-Solver (Lu et al. 2022a), we adopted singlestep fixed; to maintain consistency with the discrete time model in Appendix D.2 the parameter skip was set to lemme_quadratte (i.e., quadratic time steps). Unless otherwise specified, we used the parameter settings recommended by DPM-Solver. The results in Table show that by using the PFDiff method described in Sec. and4fddn taking DPM-Solver as the baseline, were able to further enhance sampling performance on the basis of first-order and higher-order ODE solvers. Particularly, in the 12 NFE range. PFDiff significantly improved the convergence issues of higher-order ODE solvers under fewer NFEs. For instance, at 9 NFE, PFDiff reduced the FID of DPM-Solver-3 from 233.56 to 5.67, improving the sampling quality by 97.57%. These results validate the effectiveness of using PFDiff with first-order or higher-order ODE solvers as the baseline.
the vp/ci far lacfffrrarraoooollllllllllleloee 1_w_____p___pp________
Table 5: Sample quality measured by FID of different orders of DPM-Solver (Lu et al. 2022a) on the CIFARIO (Krizhevsky et al. 2009) using unconditional continuous-ting DPMs with and without our method (PFDiff), varying the number of function evaluations (NFE). Evaluated on 50k samples.
NFE
Method
order
6
10
20
CIFARIO (continuous-time model (Song al.
2020b
quadratic time steps)
DPM-Solver-1 (Lu et al.
2022a)
1
40.55
23.86
15.57
11.64
9.64
7.23
6.06
+PFDiff (Ours)
1
113.74
11.41
5.90
4.23
3.92
3.73
3.75
DPM-Solver-2 (Lu et al.
2022a
2
298.79
106.05
41.79
14.43
6.75
4.24
3.91
+PFDiff (Ours)
2
85.22
16.30
9.67
6.64
5.74
5.12
4.78
9
21
DPM-Solver-3 [Lu et al.
2022a
3
382.51
233.56
44.82
7.98
3.63
+FFDiff (Ours)
3
103.22
5.67
5.72
5.62
5.24
D.4 ADDITIONAL RESULTS FOR CLASSIFIER GUIDANCE
In this section, we provide the specific FID scores for pre-trained DPMs the conditional, clas-
sifier guidance paradigm on the ImageNet 64x64 (Deng et al. 2009) and ImageNet 256x256
datasets (Deng et al., 2009), as shown in Table 6 We now describe the experimental setup in detail. For the pre- trained models, we used the ADM-G (Dhariwal & Nichol 2021 pro- vided 64x64 di fiffiin.. pt and 64x54 64x54_ccaasseae wwwssifiarr.eeeee pt for the ImageNet 64x64 dataset, and 25 2x6x256 di ffufffuiioo on. pt and 1116x266_classee fier fier.fet.tt... .pt for the ImageNet 256x256
21

---

Published as a conference paper at ICLR 2025
Table 6: Sample quality measured by FID. on the ImageNet 64x64 (Deng et al. 2009) and ImageNet 256x256 (Deng et al. 2009), using ADM-G (Dhariwal & Nichol 2021) model with guidance scales (s) of 1.0 and 2.0, varying the number of function evaluations (NFE). Evaluated: ImageNet 64x64 with 50k, ImageNet 256x256 with 10k samples. "We directly borrowed the results reported by AutoDiffusion (Li et al. 2023), and AutoDiffusion requires additional search costs. ") represents missing data in the original paper.
NFE
Method
Step
4
6
8
10
15
20
ImageNet 64x64 (pixel DPMs model Dhariwal & Nichol
2021
uniform time steps, 1.0)
DDIM (Song et al.
2020a
138.81
23.58
12.54
8.93
5.52
4.45
APM-Solver-22
Lu al.
2022a
327.08 07
292.66
264.97
236.80
166.52
120.29
DPM- 2
Lu ct al.
2022a
M
48.64
21.08
12.45
8.86
5,57
4.46
DPM-Solver-3
Lu al.
2022a
S
383.71
376.86
380.51
378.32
339.34
280.12
3PM--olver-3
(Lu ct
Z0224
M
54.01
24.76
13.17
8.85
5.48
4.41
2PM (Lu
2022b
44.15
20.44
12.53 \
8.95
5.53
4.33
AutoDiffusion
al.
2023)
17.86
11.17
6,24
4.92
3.93
DDIM+PFDiff (Ours)
16.46
8.20
6.22
5.19
4.20
3.83
ImageNet 256x256 (pixel DPMs model
Dhariwal & Nichol
2021
uniform time steps,
2.0)
DDIM (Song et al.
2020a
S 51.79
23.48
16.33 12.22
12.93
9.89
9.05
DDIM-PFDitf (Ours)
37.81
18.15
10.33
8.59
8.08
dataset. All experiments were conducted with uniform time steps and used DDIM as the base- line (Song et al. 2020a). We implemented the second-orde and third-order methods from DPM- Solver (Lu et al. 2022a for comparison and explored the method hyperparameter provided by DPM-Solver for both s singlester (corresponding to "S" in Table 6) and multister (cor- responding to "M" in Table 6) Additionally, implemented the best-performing method from DPM-Solver++ (Lu et al., (002b)), multi-step 42M))))ver++((MM,, as a comparative measure. Fur- thermore, we also introduced the anperior-performing AutoDiffusion (Li et al. (023) method as a comparison. "We directly borrowed the results reported in the original paper, emphasizing that although AutoDiffusion does not require additional training, it incurs additional search costs. "1" represents missing data in the original paper. The specific experimental results of the configurations mentioned are shown in Table 6 The results demonstrate that PFDiff, using DDIM as the base- line on the ImageNet 64x64 dataset, significantly enhances the sampling efficiency of DDIM and surpasses previous optimal training-fret -free sampling methods. Particularly. in cases where NFE 10, PFDiff improved the sampling quality of DDIM by 41 41.88%~88. 14%. Moreover, the large Ima- geNet 256x256 dataset, PFDiff demonstrates a consistent performance improvement over the DDIM baseline, similar the improvements observed the ImageNet 64x64 dataset.
D.5 ADDITIONAL RESULTS FOR CLASSIFIER GUIDANCE
In this section, we supplemented the specific FIDI, scores for the
(Rombach et al,
2022) (conditional, classifier-free guidance paradigm) setting with guidance scale (s) of 7.5 and
1.5. Specifically, for the pre-trained model, we conducted experiments using the sd 11-11-4-.. ckpt checkpoint provided by Stable-Diffusion. All experiments used the MS-COC02014 (Lin et al. 2014) validation set to calculate FID4 scores, with uniform time steps. PFDiff employs the DDIM (ong al.] (2020a) method as the baseline. Initially, under the recommended configuration by Stable-Diffusion, we implemented DPM- 2 and -3 as comparative methods, and set the method hyperparameters provided by DPM-Solver to mu 1 t. issepp (corresponding to "M" in Table 77). Additionally, we introduced previous state-of-the-an training-fres free methods, including DPM-Solver++(2M) (Lu et al. 2022b), UniPC (Zhao et al., 2023). and DPM-Solver v3(2M) (Zheng et al. 2023) for comparison. The experimental results are shown in Table 7 !We borrow the results reported in OPM-Solver-w (Zheng et al.) (023) directly. The results indicate that tf-tte--rtttt
7.5
on Stable-Diffusions PFDiff, using only DDIM as a baseline, surpasses the previous training-free sampling methods in terms of sampling quality in fewer steps (NFE<<20, Particularly, NFF=10, PFDiff achieved 13.06 FID, nearly converging to the data distribution, which is
22

---

Published as a conference paper at ICLR 2025
Table 7: Sample quality measured by FIDJ on the validation set MS-COC02014 (Lin et al, 2014) using Stable-Diffusion model (Rombach et al. 2022) with guidance scales (s) of 7.5 and
1.5, varying the number of function evaluations (NFE). Evaluated on 10k samples. We borrow the results reported in DPM-Solver-v3 (Zheng et al. 2023 directly.
NFE
Method
Step
5
15
MS-COC02014 (latent DPMs model RRombach et al.
2022
uniform time steps, 7.5)
DDIM (Song al. 2020a
S
23.92
20.33
17.46
16.78
16.08
15.95
DPM-Solver-2 [Lu
2022a
M
18.97
17.37
16.29
15.99
14.32
14.38
DPM-Solver-3 (Lu ct al
2022a
M
18.89
17.34
16.25
16.11
14.10
13.44
'DPM-Solvert+ (Lu et
2022b
M
18.87
17.44
16.40
15.93
15.84
15.72
'UniPC (Zhao et al. 2023)
M 18.77
17.32
16.20
16.15
16.06
15.94
'DPM-Solver-v3(MM [Zhnng al.
2023
M
18.83
16.41
15.41
15.32
15.30
15.23
DDIM+PFDiff (Ours)
S 18.31
15.47
13.26
13.06
13.57
13.97
MS-COC02014 (latent DPMs model Rombach et al.
2022
uniform time steps,
1.5)
DDIM (Song et al. 2020a
S
70.36
54.32
37.54
29.41
20.54
18.17
DPM-Solver-2 [u et al.
2022a
M
37.47
27.79
19.65
18.39
17.27
16.85
DPM-Solver-3 (Lu et
M
35.90
25.88
18.26
19.10
17.21
16.67
DPM (MM [L
2022b
M
36.58
26.78
18.92
20.26
18.61
17.78
DDIM+PFDifT (Ours)
S
24.31
20.99
18.09
17.00
16.03
15.57
14.25% improvement the previous state-of-the-art method JPM-Solver-v3 20 NFE, which had a 15.23 FID. Furthermore, to further validate the effectiveness of PFDiff on Stable-Diffusiin, conducted experiments using the s 1.5 setting with the same experimental configuration as 8 = 7.5. For the comparative methods, we only experimented with the multi-step versions of DPM- Solver-2 and -3 and (2M), which had faster convergence at fewer NFE under the 7.5 setting. As for UniPC and corresponding experimental results mental results show that PFDiff, using DDIM as the baseline under the s = 1.5 setting, demonstrated consistent performance improvements as seen the s = 7.5 setting, as shown in Table
since DPM-Solver-v3 did not provide 1.5, we did not list their comparative results. The experi-
D.6 ADDITIONAL ABLATION STUDY RESULTS
D.6.1 ADDITIONAL RESULTS FOR PFDIFF HYPERPARAMETER STUDY
In this section, we extensively investigate the impact of the hyperparameters and h on the perfor- mance of the PFDiff algorithm, supplementing with the results of ablation experiments and experi- mental setups. Specifically, for the unconditional DPMs, we conducted experiments on the CIFARIO dataset (Krizhevsky et al. 2009) using quadratic time steps, based pre-trained unconditional dis- crete DDPM (Ho et al. 2020). For the conditional DPMs, we used uniform time steps in classifier guidance ADM-G (Dhariwal & Nichol 2021) pre-trained DPMs, setting the guidance scale (s) to 1.0 for experiments on the ImageNet 64x64 dataset (Deng et al. 2009). All experiments were con- ducted using the DDIM (Song al. (020a) algorithm base and PFDiff-k:. h configurations (k = 2, (h < k)) were tested in six different algorithm configurations. The FIDI, scores are presented in Table S by varying the number of function evaluations (NFE) and the sample number used to compute the evaluation metrics.
We first analyze the impact of the hyperparameters ic and using 50k samples to compute the FID scores, which is common method for evaluating the performance of sampling algorithms. The experimental results demonstrate that, under various combinations of k and h, PFDiff is able to enhance the sampling performance of the DDIM baseline in most cases across different types of pre-trained DPMs. Particularly when setting & = 2 and h = 1, PFDiff-2.1 can always improve the sampling performance of the DDIM baseline within the range of 4~20 NFE. Furthermore, an exciting discovery regarding the further optimization of algorithm performance: Searching with just 1/10 of the data provides consistent results compared searches using the full 50k samples,
have
23

---

Published as a conference paper at ICLR 2025
Table Ablation study of the impact of and in CIFARIO Krizhevsky 2009) and ImageNet 64x64 (Deng et al. 2009) datasets using DDPM (Ho et al., 2020) and ADM-G (Dhariwal Nichol 202T) models. We report the FID4 and MSEJ, varying the number of function evaluations (NFE) and the number of samples for evaluating algorithm performance.
NFE
Samples
Method
15
20
CIFARIO (unconditional DPMs model
(Ho
2020
quadratic time steps)
DDIM (Song al. 2020a)
65.70
29.68
18.45
13.66
11.01
8.80
7.04
+PFDiff-1
124.73
19.45
5.78
4.95
4.63
4.25
4.14
+PFDiff-2.1
59.61
9.84
7.01
6.31
5.58
5.18
4.78
50k
+PFDiff-2.2
167.12
53.22
8.43
4.95
4.41
4.10
3.78
(FID)
+PFDiff-3.1
22.38
13.40
9,40
7.70
6.73
6.03
5.05
+PF-Dif--322
129.18
19.35
5.64
4.57
4.39
4.19
4.08
+PFDDff-3.3
205.87
76.62
20.84
5.71
4.73
4.41
3.68
DDIM (Song
2020a
69.79
34.20
22.84
17.39
15.56
12.87
11.62
+PFDifi-T
127.82
23.96
10.35
9.73
9.29
9.09
8.74
+PFDiff-2
63.59
14.34
11,40
11.08
9.23
10.03
9.38
+PFDiff-2 2
170.58
57.74
12.94
9.86
10.08
9.02
8.44
(FID)
+PFDitf-3 1
26.85
17.93
13.95
12.37
9.59
10.89
9.65
+PFDiff-3.2
132.45
23.80
10.11
9.58
9.16
9.09
8.69
+PFDiff-3_3
208.80
80.13
24.84
10.73
11.16
9.14
8.34
DDIM (Song
2020a)
0.1009
0.0608
0.0414
0.0314
0.0255
0.0199
0.0152
+PFDifi-1
0.0542
0.0217
0.0131
0.0100
0.0089
0.0082
0.0081
+PFDiff-2.1
0.0277
0.0137
0.0110
0,0104
0.0098
0.0093
0.0088
+PFDiff-2.2
0.1001
0.0468
0.0277
0.0184
0.0145
0.0122
0.0105
(MSE)
+PFDiff-3_1
0.0218
0.0167
0.0146
0.0130
0.0120
0.0107
0.0098
+PFDiff-3.2
0.0614
0.0228
0.0133
0.0101
0.0089
0.0083
0.0082
+PFDifT-3_3
0.1790
0.0820
0.0444
0.0299
0.0224
0.0165
0.0126
ImageNet 64x64 (conditional DPMs model (Dhariwal & Nichol
2021
uniform time steps,
1.0)
DDIM (Song 2020a)
138.81
23.58
12.54
8.93
6.74
5.52
4.45
+
26.86
11.39
7.47
5.83
5.16
4.76
4.39
+PFDiff-2.1
17.14
8.94
6.38
5,46
5,46
4.30
3,83
+PFDiff-2.2
23.66
9.93
6.86
5.72
5.17
4.49
3.94
(FID)
+PFDiff-3.1
16.74
9.43
7.19
5.86
5.07
4.69
4.44
+PFDiff-3.2
16.46
8.20
6.22
5.19
4.62
4.20
4.28
+PFDifT-3.3
23.06
9.73
6.92
5.55
5.21
4.47
4.49
DDIM (Song et al.
2020a
146.03
29.61
19.11
15.13
13.15
11.65
10.81
+PFDitt-
32.82
17.80
13.61
12.16
11.20
10.99
10.82
+PFDiff-2.1
23.70
14.81
12.38
11.82
11.53
10.77
10.24
+PFDiff-2.2
30.10
16.35
13.09
11.80
11.68
10.67
10.56
(FID)
+PFDiff-3.1
23.09
15.78
12.09
11.71
11.00
10.77
+PFDiff-3
22.54
14.23
12.24
11.27
11.16
10.47
10.48
+PFDitf-3.3
29.45
16.12
13.25
11.90
11.29
10.68
10.66
significantly reducing the cost of hyperparameter searching for k: and h. Specifically, for the same NFE, the optimal combinations of and h based on FID scores are consistent for both 5k and 50k samples. For instance, when NFE=6. the best FID values for both Sk and 50k samples are achieved with k = 2 and h. 1. For the six combinations used in this study with & < 3(h < k)), only a total of 30k samples are required search the optimal & and combination for each NFE less than the cost of evaluating algorithm performance normally with 50k samples. Additionally, in practical applications where only a small number samples are needed for visual analysis, we minimize training resources and rapidly identify the optimal & and h combination. In summary, the hyperparameters and h do not impede the practical application of PFDiff in accelerating the sampling of DPMs.
even
Furthermore, we propose an automatic search strategy with almost no additional cost based on truncation errors for selecting and h. Specifically, in Sec. 3.5 discuss how PFDiff can correct
24

---

Published as a conference paper at ICLR 2025
0.10
NFE-10
NFE-10
NFE-10
NFE-2U
NFE-20
0.06
NFE-20
Truncationn ettor
0.00
0.000
0.000
0.04
0.00
1110
0.00
000
200
060
600
100
000
200
(a) DDIM
(h) DDIM + PFDiff-1
(c) DDIM + PFDiff (A S 3)
Figure 5: The trend (mean and standard deviation (std)) of accumulated truncation error over time step t on the CIFARIO Krizhevsky et al. 2009) dataset, relative to DDIM (Song et al. 2020a) with 1000 NFE, varying the number of function evaluations (NFE) (6, 10, 20).
the discretization error of the baseline solver in regions with large curvature along the sampling trajectory, which means that PFDiff accumulates less truncation error. To validate this conclusion, we conducted experiments on the CIFARIO dataset, where we sampled 256 samples using DDIM, PFDiff-1, and PFDiff (k: I 3) with varying NFE values. The truncation errors were calculated using MSE relative to DDIM with 1000 NFE, and the results are shown in Fig. [ The results demonstrate that PFDiff significantly corrects DDIM's truncation errors in regions with large curvature of the sampling trajectory (i.e., regions where truncation errors increase rapidly). Based on these findings, propose automatic search strategy: by varying the hyperparameters & and h. we "warm up" with only 256 samples respectively. calculate the average truncation error using MSE, and use this value guide the selection of ie and h. On the CIFARIO dataset, by varying NEE, k, and h, we sampled 256 samples and computed the average truncation errors relative to DDIM with 1000 NFE. The results are presented in Tab. g Based on Tab. 8 we select the values of & and h corresponding to the minimum truncation error, and further refer to the corresponding FID values found in Tab. g which are {22.38, 9.84, 7.01, 4.95, 4.63/4.39, 4.25, 4.14) for NFE E (4, 6, 8, 10, 12, 15, 20). This set of FID values is comparable to the optimal FID, indicating that determining i and h based on truncation error is reasonable. Notably, in Tab. X we "warmed up" only 256 samples to compute the truncation error, so the additional cost introduced by the automatic search strategy is negligible. The 256 samples are sufficient to capture the dataset's statistical properties because the shapes of sampling trajectories are highly similar (Chen al. 2024).
D.6.2 ABLATION STUDY OF PAST AND FUTURE SCORES
To further investigate the impact of scores from the past or future on first-order ODE solvers on the rapid updating of current intermediate states, this section supplements related ablation study results and their settings. Specifically, we first use PFDiff with the first-order ODE solver DDIM as a base- line, removing past and future scores separately based on the discrete-time pre-trained models (Ho) jtt al. al. 2015) datasets, we alter the number of function evaluations (NFE) to compute the FID met- ric. Additionally, we introduce a method from previous literature (Ma et al. (024) that accelerates sampling by caching part of past scores for comparison. Specifically, we configure the hyperpa- rameters based on the DeepCache (Ma et al. 2024) codebase, setting cache interval to 2 and branch to with all other settings remaining unchanged. As shown Table the experimental results indicate that using only past scores or only future scores can slightly improve the first-order ODE solvers sampling performance. However, their combined use (i.e., the complete PFDiff) sig- nificantly enhances first-order ODE solvers sampling performance, especially with very few steps (NFE<10), a phenomenon particularly evident. These results further validate the efficiency of the PFDiff algorithm when NFE<10, benefiting from its information-effiiies update process, which utilizes past and future (complementing each other) scores to jointly guide the current intermediate state.
2020 Song et al., (020a).. On the CIFAR10 (Krizhevsky et al.] 2009) and CelebA 64x64 (Liu
Furthermore, to verify whether the update process of the PFDiff algorithm increases additional in- ference time, we employed the same experimental settings in Table Ø and provided specific
25

---

Published as a conference paper at ICLR 2025
Table 9: Ablation study of the impact of the past and future scores on PFDiff, using DDIM (Song et al. 2020a) as the baseline, in CIFARIO (Krizhevsky et al. 2009) and CelebA 64x64 (Liu et al. 2015 datasets using discrete-time models (Ho et al. 2020] Song al., 2020a). We report the FID., varying the number of function evaluations (NFE), Evaluated on 50k samples.
NFE
+PFDiff
Method
12
CIFARTO (discrete-time model
(Ho
2020).
quadratic time steps)
DDIM (Song al.
2020al
65.70
29.68
18.45
13.66
11.01
8.80
7.04
+Cache (Ma et al.
2024
49.02
24.04
15.23
11.31
9,40
7.25
6.25
+Past
52.81
27.47
17.87
13.64
10.79
8.20
7.02
+Future
66.06
25.39
11.93
8.06
6,04
4.17
+Past Future
22.38
9.84
5.64
4.57
4.39
4.10
3.68
CelebA 64x64 (discrete-time model
(ong et
2020a
uniform time steps)
DDIM (Song
2020a
44.36
29.12
23.19
20.50
18.43
16.13
14.76
+Cache (Ma 2024
33,86
25.95
22.29
19.83
18.28
16.05
14.45
+Past
28.45
21.56
18.65
17.03
15.89
14.05
12.73
+Future
39.85
16.30
14.40
12.79
12.13
9.73
+Past & Future
51.87
12.79
8.82
8.93
7.70
6.44
5.66
inference time comparison under different NEE. The inference time. (second, meantist) required per 1K samples on single NVIDIA 3090 GPU is shown in Table TO The experimental results reveal that PFDiff+DDIM has consistent inference times with DDIM alone under the same NFE, indicating that the PFDiff algorithm does not add extra inference time. Additionally, methods (Ma ict al. 2024) that cache part of past scores not only incur additional inference costs but also exhibit relatively weak acceleration effects with small number of steps (NFE<10). These results collec- tively demonstrate that the PFDiff algorithm can significantly enhance sampling quality without any increase in inference time, further proving its effectiveness.
Table 10: Inference timel (second, mean required per 1k samples single NVIDIA 3090 GPU, varying the number function evaluations (NFE). We additionally present the inference time with only past only future scores, the same NFE. Moreover, introduce methods (Ma al. 2024 Wimbauer et al. 2024) that cache part of past scores for comparison.
NFE
+PFDiff
Method
4
10
16
20
CIFAR10 (discrete-time model (Ho al.
2020),
quadratic time steps)
DDIM (Song et al.
2020a)
6.1440.010
9.81+0
13.58++0.090
15.90++0.081
+Cache (a et al.
2024
6.3110.150
13.55110.019
19.42-10.09
24.07-10.185
+Past
6.17.10.29
9.88110040
13.66.10.257
15.81
+Future
6.16++0.036
9.77
13.73+0.345
15.67=00096
+Past Future
6.10::0.006
9.741:0036
13.48+t0.220
15.79+0.036
CelebA 64x64 (discrete time model (Song al.
2020a), uniform time steps)
DDIM (Song et al.
2020a)
13.65 166
27.29
40.55:100.118
49.43-10.497
+Cache (Ma et al.
2024
19.82+10.131
45.60
71.70=10.66
89,45
+Past
13.67+00.55
26.88-00.14
40.24+00.11
49.82++0.081
+Future
13.61+0.304
26.38+0..07
39.95-t0.440
49.05-00.543
+Past Future
13.21+0.000
26.41+-0.042
40.26-40.186
49.38++0.257
26

---

Published as a conference paper at ICLR 2025
D.7 INCEPTION SCORE EXPERIMENTAL RESULTS
To evaluate the effectiveness of the PFDiff algorithm and the widely used Fréchet Inception Dis- tance (FID4) metric (Heusel et al. 2017) in the sampling process Diffusion Probabilistic Models (DPMs), we have also incorporated the Inception Score (IS1) metric (Salimans et al.) 2016) for both unconditional and conditional
DPMs. Specifically, for the unconditional discrete-time pre-trained DPMs DDPM (Ho al. 2020), maintained the experimental configurations de- scribed in Table Appendix [D2] and added IS scores for the CIFARIO dataset (Krizhevsky et al. 2009). For the unconditional continuous-time pre-trained DPMs ScoreSDE (Song et al.) 2020b), the experimental configurations are consistent with Table S in Appendix [33] and scores for the CIFAR10 dataset were also added. For the conditional classifier guidance paradigm of pre- traitra DPMs ADM-G (Dhariwal & Nichol 2021). the experimental setup aligned with Table Appendix
D.4 including IS scores for the ImageNet 64x64 and ImageNet 256x datasets (Deng al. 2009).
Considering that the computation of IS scores relies on features extracted using Incept inneptiovv scores for ImagmeaeeNet datasets was not
pre-trained on the ImageNet dataset, calculating feasible, hence no IS scores were provided for the classifier-free guidance paradigm of Stable- Diffusion (Rombach al. 2022). The experimental results are presented Table IT comparison between the FIDI, metrics in Tables 3 and 6 and the ISt metrics in Table [1 shows that both IS and FID metrics exhibit similar trends under the same experimental settings, i.e., as the number of function evaluations (NFE) changes, lower FID scores correspond to higher IS scores. Further, Figs. 7a and 7b. along with the visualization experiments in Appendix D.9 demonstrate that lower FID
Table 11: Sample quality measured by IST on CIFARIO (Krizhevsky et al. 2009) ImageNet 64x64 (Deng et al. 2009) and ImageNet 256x256 (Dngg al. 2009) using DDPM (Ho et al. 2020). ScoreSDE (Song et al. 2020b) and ADM-G (Dhariwal Nichol 2021) models, varying the number of function evaluations (NFE). Evaluated: ImageNet 256x256 with 10k, others with 50k samples. "We directly borrowed the results reported by AutoDiffusion (Li et al. 2023). and AutoDiffusion requires additional search costs. "i" represents missing data in the original paper and DPM-Solver-2 (Lu et al. 2022a implementation.
NFE
+PFDiff Method
6
8
10
15
20
CIFARIO (discrete-time model (Ho et al.
2020),
quadratic time steps)
DDPM() = 1.0) (Ho al.
2020
4.32
5.66
6.55
7.08
7.91
8.25
Analytic-DDPM [Boo
2022b
5.76
6.29
6.93
7.42
8.07
8.33
Analytic-DDIM [ao el
20226)
4.46
7.47
8.11
8.43
8.72
8.89
DDIM (Song et al 2020a
5.68
7.21
7.92
8.26
8.62
8.81
Analytic-DDIM
8.78
9.43
9.61
9.35
DDIM
7.79
9.29
9.62
9.43
9.29
9.29
CIFAR1O (continuous- model (ong et al.
2020b). quadratic time steps)
DPM-Solver-l (Lu et 2022a
8.30
8.85
8,98
9.43
DPM-Solver-2
LLu ét al. 2022a
1.70
5.29
7.94
9.09
9.74
DPM-Solver-l
4.29
9.25
9.76
9,86
9.85
DPM-Solver-2
6.96
8,58
8.75
9.26
9.69
ImageNet 64x64 (pixel DPMs model (Dhariwal & Nichol
2021).
uniform time steps,
1.0)
DDIM (Song al. 2020a)
7.02
31.13
40.51
46.06
54.37
59.09
DPM-Salver-Z(Mull
(Lu et al.
2022a
19.03
33.75
44.65
$1.79
62.18
67.69
x
DPM-Solver=3(Mulii (Lu el al.
2022a
17.46
29.80
41.86
50.90
62.68
68.44
DPM-Solver++(M) (Lu al.
2022b
20.72
34.22
43.62
50.02
60.00
65.66
"AutoDiftusion (Li 2023
34.88
43.37
57.85
64.03
68.05
DDIM
35.67
50.14
58.42
59.78
64.54
69.09
ImageNet 256x256 (pixel DPMs model (Dhariwal & Nichol
2021).
uniform time steps, 8 = 2.0)
DDIM (Sog et al. 2020a)
37.72
95.90
122.13
144.13
165.91
179.27
DDIM
55.90
122.56
158.57
169.72
183.07
192.70
27

---

Published as a conference paper at ICLR 2025
DDMM
0.14
NFE.
Har''s 2nd
NF PelPass
S
00122
MF5-8. Reie
NFE-8, apl
0.254
SPM--Shhar33
(Meetiv to MF)
MFE-32.
NFF.
8,000
KFF 90, Rave
0.154
0.000
0.00
0.071
0.00
0300
200
400
502
000
400
600
1030
31
Decoa Gimareo Pack
(a) Score Changes ODE Solvers Based EDM
(b) Future Score More Reliable than "Springboard"
Figure 6: (a) The trend of the MSE of the noise network output c0 () over time step size At in the EDM framework (Karras et 2022). (b) MSE relative to the sampling process with 1000 NFE: "Springboard", ||
35 44h
Future score, ||co (ittnn (eriitttsssse 144h) co (āgt (4th)12
scores and higher IS scores correlate with higher image quality and richer details generated by the PFDiff sampling algorithm. These results further confirm the effectiveness of the PFDiff algorithm and the FID metric in evaluating the performance of sampling algorithms.
D.8 EXPERIMENTAL SETUP AND ADDITIONAL RESULTS CLARIFYING THE MOTIVATION
In this section, provide detailed explanation of the experimental in Fig. la and Fig. Additionally, we further extend the experiments from Fig. aaland Fig. 1b as shown in Fig. 6
1b
For Fig.
la
first store the noise network output all time steps, with 1000 NFE:
1= We then compute the mean the MSE the noise network output over the time interval At. For instance, when A/ 2, compute the following mean: ||so [ 40) (co(Z),, ti)
1909
- 6 6a (oooo (g99)]l 2 By varying the value of At from to 999, are able to compute the mean of the MSE of the noise network output, respectively, and ulti- mately obtain Fig. Ta Notably, in Fig. Ta the curves for DDPM (Ho et al. 2020) and DDIM (Song 2020a) are derived from the pre-trained model of DDPM (Ho et al. 2020p: the curves for DPM- (Lu et al.] 2022a) and DPM-Solver++ (Lu et al., 2022b) are obtained from the pre- trained model ScoreSDE (Song et al., 2020b). Additionally, Fig. 6al we present experimental results from a more advanced diffusion model framework the EDM (Karras et al. 2022) pre- trained model, including DDIM, DPM--olver, and Heun's 2nd (Karras al. (022) solvers. For the three different architecture pre- tre-trained models, when the time step size At not excessively large, the noise network outputs exhibit remarkable similarity. This validates that the method we propose in Sec. 3.3 using past scores to guide sampling, is reliable.
<o( 222 "?,....,. [
For Fig. during the sampling process from Lt, to Ct 1)> based on the PFDiff, we first compute the "springboard" Ztven. and then further obtain the future score e co( 4.+t). Next, we compute the MSE of the status ot. which is updated using the "springboard" and future score, respec- tively. The MSE is calculated relative to the sampling process with 1000 NFE, resulting in Fig. [b] Moreover, We also directly evaluate the MSE of the "springboard" and the future score different 4+th moments, relative to the sampling with 1000 NFE, as shown in Fig. 6b) Notably, both Fig. lb and Fig. [bb exhibit the same trend, demonstrating that the future gradient is more reliable than the "Springboard".
D.9 VISUALIZE STUDY RESULTS
To demonstrate the effectiveness of PFDiff. present the visual sampling results the CI-
FARI0 (Krizhevsky et al., 2009), CelebA 64x64 (Liu et al. 2013), LSUN-bedron 256x256 YYu al. 2015), LSUN-church 256x256 (Yu al. 2015). ImageNet 64x64 (Deng et al. 2009). Ima-
28

---

Published as a conference paper at ICLR 2025
geNet 256x256 (Deng al., 2009), and MS MS-COCO2014 (Lin al. 2014) datasets in Figs. L3
These results illustrate that PFDiff, using different orders of ODE solvers as a baseline, is capa- ble of generating samples of higher quality and richer detail on both unconditional and conditional pre-trained Diffusion Prohabilistic Models (DPMs).
Text Prompts: Winter night with snow -covered rooftops and soft yellow lights. (Left)
A Corgi running towards me in Times Square. (Right)
NFE = 6. FID 20.33
NFE 10. FID 16.78
NFE 250. FID 15.86
Stable- Diffusion
NFE=6, 11.475.47
NFE
FID 13.06
NFE 1000 FID 15.82
Stable- Diffle-Diffuiin
Ours
(a) Results from Stable-Diffusion on MS-COC02014 (Classifier-Fres Guidance, 8 7.5)
NFE-4
NFE-8
NFE-10
NFE=20
NFE-100
Guided-Diffusion
NFE-4
NFE-8
NFE-10
NFE-20
NFE=250
Guided Diffusion
(b) Results Guided Diffusion on ImageNet 64x64 (Classifier Guidance, s 1.0)
Figure 7: Sampling by conditional pre-trained DPMs (Rombach et al. 2022 Dhariwal & Nichol
2021 using DDIM (Song et al. 2020a) and our method PFDiff (dashed box) with DDIM as baseline, varying the number of function evaluations (NFE).
29

---

Published as a conference paper at ICLR 2025
NFE
10
NFE
NFE
NFE
FID 65.70
FID 18.45
FID 13.66
FID
37.76
FID 14.10
FID 10.86
DDIM (Song et al.
2020a
106.86
FID 14.21
10.09
FID
69.75
11.84
9.37
(b) Analytic-DDID DDIM
Bao et al.
2022b
22.38
FID= 5.64
FID) 4.57
FID = 13.29
FID = 5.06
FID=
(c) DDIM+PFDiff (Ours)
Figure 8: Random samples by DDIM (Song et al. 2020a), Analytic-DDIM (Bao et al. 2022b), and PFDiff (baseline: DDIM) with 4, 8, and 10 number of function evaluations (NFE), using the same random seed, quadratic time steps, and pre-trained discrete-time DPMs (Ho et al.,
2020 Song al., (Liu et al.
2020ab on CIFARTO (rizhevsky et al. 2009) (left) and CelebA 64x64
2015 (right).
5
NFE
=5
NFE
FID 71.02
FID 19.26
FID 73.43
FID = 22.04
DDIM Song et al.
2020a
FID = 25.32
FID = 9,28 (b) DDIM+PFDif (Ours)
FID = 26.69
FID 13.27
Figure 9: Random samples by DDIM (Song et al. 2020a) and PFDif (baseline: DDIM) with 5 and number of function evaluations (NFE), using the same random seed, uniform time steps, and pre-trained discrete-time DPMs (Ho al. 2020) on LSUN-bedroom 256x256 (Yu et al. 2015 (left) and LSUN-church 256x256
Yuu et al. 2015) (right).
30

---

Published as a conference paper at ICLR 2025
NFE
NFE 12
FID = 23.86
=ID=
DPM-SolverT
(Lu et al. 2002a
FID = 11.41
FID = 3.92
(b) DPM-Solver-1 1 +PFDif (Ours)
FID 106.05
FID 6.75
(c) DPM-Solver> Lu et al. 2022a
16.30
FID 5.74
P+DFi (Ours)
FID 382.51
FID 44,82
DPM-Solver-3 al. 2022a
FID 103.22
FID 5572
DPM-Solver:++PDDif (Ours)
Figure 10: Random samples by DPM-Solver-1, -2, and -3 (L et al. 2022a) with and without our method (PFDiff) with and number of function evaluations (NFE), using the same ran- dom seed, quadratic time steps, and pre-trained continuous-time DPMs (Song et al. 2020h) on CIFARIO Krizhevsky et al. 2009).
31

---

Published as a conference paper at ICLR 2025
NFE
NFE
FID = 138.81
FID 12.54
(a) DDIM (ong et al. 2020a
FID 48.64
FID 12.45
(b) DPM-Solver-2 Lu al.
2022a
= 44.15 (c) DPM-Solver++(2M) [Lu al. 2022b
FID 12.53
FID 16.46
FID 6.22
(d) DDIM+PFDif (Ours)
Figure 11: Random samples by DDIM (Song et al. 2020a). DPM-Solver-2 (Lu et al. 2022a). DPM- Solver++((M) (Lu al. 2022b), and PFDiff (baseline: DDIM) with 4 and number of func- tion evaluations (NFE), using the same random seed, uniform time steps, and pre-trained Guided- Diffusion DDariwal Nichol 2021) on ImageNet 64x64 (Deng al. 2009) with a guidance scale of 1.0.
NFE 4
NFE 8
FID=51.79 51.79
FID=
(a) DDIM (Song et al. 2020a
FID=37.90
FID= 12.22
(b) DDIM+PFDif (Ours)
Figure 12: Random samples by DDIM Song al. 2020a) and PFDiff (baseline: DDIM) with and 8 number of function evaluations (NFE), using the same random seed, uniform time steps, and pre-trained Guided- Diffusion (Dhariwal & Nichol 2021 on ImageNet 256x256 (Deng et al. 2009 with guidance scale of 2.0.
32

---

Published as a conference paper at ICLR 2025
Text Prompts (listed from left to right): A large bird is standing the water by some rocks.
candy covered cup cake sitting on top white plate.
People wine tasting with table of wine bottles and glasses wine. bathtub sits
tiled floor near sink that has ornate mirrors over while greenery grows on the
other side of the tub.
A kitchen and dining area in house with open floor plan that looks out over the landscape from
large set of windows.
DDIM 13.23
DPM- Solvertt
(FID 18.87)
DDIM-PFDIR
(FID 18.31)
NFE=5
(FID 16.78)
DDIM
DDIM-PFDIR DPM-Solver
(FID 15.93)
(FID - 13.06)
NFE = 10
Figure 13: Random samples by DDIM Song al. 2020a), D2M- Lu al. 2022b
and PFDiff (baseline: DDIM) with 5 and 10 number of function evaluations (NFE), using the same random seed, uniform time steps, and pre-trained guidance scale of 7.5. Text prompts are random sample from the MS-COC02014 (Lin et al. 2014) validation set.
Diffu--iff (Rombach al.) 2022) with
33


Reviews:
[{"review_id": "HLUlOQFJd0", "reviewer": "Reviewer_sz2Y", "summary": "This paper introduces a novel methodology to accelerate diffusion model sampling. The core concept involves reusing past score predictions to generate a preliminary estimate (springboard) for the next step. Then, future score prediction is obtained from this springboard. By leveraging this future score prediction, the method enables step skipping, directly calculating the point two steps ahead from the current position. This approach offers practical advantages as it is orthogonal to existing advanced samplers and does not require additional training. Extensive experiments demonstrate its effectiveness in significantly accelerating diffusion model sampling when integrated with various state-of-the-art samplers.", "questions": "1. In the Stable diffusion experiment, why was the proposed methodology not applied to DPM-Solver? If the results were presented in the paper, please provide a reference.\n2. In Equation 14, is it correct to plug the $n$ points obtained from the $\\Delta t$ interval ODE solver into the $2\\Delta t$ interval ODE solver? Do I understand it correctly?\n3. Are the MSE scales of the future gradient and the springboard directly comparable? Would the author(s) think that using the MSE of the image updated with the future gradient instead of the future gradient in Figure 2(b) provides a more meaningful comparison?\n4. Is the mention of Nesterov momentum solely due to the similarity in form between the proposed springboard prediction method and Nesterov momentum? Have any properties of Nesterov momentum, such as improved convergence speed, been leveraged in the theoretical analysis or practical implementation of the proposed method?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "1. Extensive experiments conducted with diverse models and baselines underscore both the superiority and generality of the proposed methodology. Comprehensive results reveal a substantial improvement in the efficiency of diffusion sampling.\n2. By approaching diffusion model acceleration through time-skipping, the authors introduce a technique that is orthogonal to existing advanced samplers. This characteristic, coupled with its training-free nature, enhances its practical applicability.\n3. Despite its simplicity and ease of implementation, the methodology presented in the paper yields significant benefits.", "weaknesses": "1. The multi-step solver's exclusion of future gradients, a core component of the proposed methodology, undermines the claimed orthogonality. Additionally, the absence of experimental results (Stable diffusion) integrating the method with the DPM-Solver series raises doubts about its performance enhancement potential and the extent of its orthogonality when applied to multi-step solvers.\n2.  While the methodology is presented as an orthogonal wrapper for arbitrary ODE solvers, its classification as a standalone ODE solver is also plausible, depending on the perspective.\n3. The use of \"gradient guidance\" in the title and text is potentially misleading. In the context of diffusion models, this term is typically associated with guiding the sampling process using external model gradients (e.g. classifier guidance). For better clarity, using terms like \"score\" or \"predicted noise\" would be more appropriate.\n4. The direct comparison between the future gradient and the springboard in Figure 2(b) is questionable. Given their different scales, a direct MSE comparison might not be the most accurate approach to assess their relative reliability.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "wHqyRp1jm6", "reviewer": "Reviewer_BQ25", "summary": "The PFDiff paper introduces a novel, training-free, and orthogonal timestep-skipping mechanism to improve existing ODE solvers used in Diffusion Probabilistic Models. The proposed approach helps to reach solutions with fewer NFE, with the aid of springboard along with foresight updates. This addresses a significant challenge in reducing computational cost while keeping high sample quality. Furthermore,  PFDiff improves the efficiency and quality of diffusion model samplig.", "questions": "Please see the weaknesses section.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "1). The paper is well written and easy to understand\n\n2). The given illustrations, and provided Algorithms further helps understanding the paper.\n\n3). The paper identifies a limitation of DPMs, which is their sampling efficiency is low as they often require multiple number of denoising steps. Existing methods tend to amplify discretization errors when NFE is below 10, often leading to convergence issues. The proposed approach, named PFDiff, is a training-free and orthogonal timestep-skipping algorithm that helps mitigating these errors while operating with fewer NFEs\n\n4). PFDiff employed the potential for improvements in existing training free accelerated methods, and the sequence of observations that led to the development of PFDiff is remarkable. \n\n5). The proposed sampler can be integrated to any order of ODE solvers regardless of the type.", "weaknesses": "I believe this is a good paper as it provide valuable insights while providing solid reasoning, but I have some questions regarding the scalability, as well as about k and h values of the proposed method.\n\n1). I would like to know how will PFDiff maintain quality across different types of diffusion models other than those mentioned in paper? \n\n2). As the algorithm's construction is based on gradients, I would like to know what happens if gradients show a dispersion. How this kind of a scenario is handled? Also is there a possibility of accumulating errors in the proposed approach?\n\n3). A more ablation on the parameters k and h will further enhance the paper. For instance, is it possible to further increase the value of k? At that kind of instance, how would PFDiff work?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "OKmQxbcvJQ", "reviewer": "Reviewer_7KLd", "summary": "This paper introduces a sampling method to accelerate the first-order ODE solvers by utilizing the past gradient ($\\epsilon$) and future gradient, leading to FID improvements in the NFE regime between 4 and 20. Also, the authors accelerate the higher-order ODE solver by using the past gradients. Empirical results show that the sampling method works well for continuous and discrete diffusion models under unconditional and conditional settings.", "questions": "have the authors also tried adding the future gradient step to higher-order ODE solvers?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1. Unlike other generic ODE solvers (e.g., Heun 2nd-order solver, DPM-solver), this paper proposes the reuse of the network output from previous time steps to accelerate the sampling based on the observation of the output similarity between two consecutive time steps.\n2. Experiments on various diffusion models (continuous, discrete, conditional, unconditional) verify that PFdiff-1 outperforms other ODE solvers in the low NFE regime (4-20)", "weaknesses": "1. The proposed method lacks theoretical support and is mainly motivated by the output similarity observation shown in Figure 2a. How reliable is this similarity? How much does this observation vary in different diffusion frameworks? Please provide more details of experiments in Figures 2a and 2b, you can put the details in into appendix. I suggest the authors explore the connection between the sampling trajectory and the proposed sampling method. I think the curvature of the trajectory can explain the reuse of the gradient and your methods. Refer to [1] and [2] for details of trajectory shape.\n\n2. the overall writing is problematic and significantly affects the readability of this paper. I list some below:\n- the definition of Q is not clear, in line 222, plug in n=0 does not give $x_{t_{i-1}}$. Please rethink the expression of Q since it is used throughout the paper.\n- if the proposal of using future gradients is based on Proposition 3.1, why not put the proposition at the beginning of section 3.3?\n- function s() is not defined in eq 7 and eq 8\n- function h() is not defined in eq 9\n- line 348, notations l and h are undefined\n- In Figure 2b, treating the samples derived from 1000NFE as the ground truth is not rigorous.\n\n3. The authors claim that PFDiff is effective and orthogonal to existing ODE solvers, please provide the FID results of PFDiff in the regime NFE>20 to support the claim.\n\n4. in Figure 4, some FID results of PFDiff are missing (NFE=4 and NFE>12). In Figure 5, some FID results of PFDiff are missing (NFE>10)\n\n5. in Figure 4b, why PFDiff is worse than the baseline Analytic-DDIM when NFE=6? A similar outlier in Figure 4a\n\n6. in Figure 5, the results of DPM-solver+PFDiff are missing.  \n\n7. I encourage the authors to also compare the FID of PFDiff with [2] \n\n\nothers:\n1. line 52, the last two papers are published in 2023, not 2024, please cite papers correctly\n2. I suggest the authors move Figure 1 to the appendix to leave space for the main content.\n\n\n[1] Sabour, Amirmojtaba, Sanja Fidler, and Karsten Kreis. \"Align your steps: Optimizing sampling schedules in diffusion models.\" ICML. 2024\n\n[2] Zhou, Zhenyu, et al. \"Fast ode-based sampling for diffusion models in around 5 steps.\" CVPR. 2024", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)