ParaSolver: A Hierarchical Parallel Integral Solver for Diffusion Models
Authors: Jianrong Lu
Zhiyu Zhu
Junhui Hou
Venue: ICLR 2025 Poster (2025)
Keywords: Diffusion Models;
Primary area: generative models
TL;DR: A parallel sampling algorithm for accelerating inference of diffusion models

Abstract: This paper explores the challenge of accelerating the sequential inference process of Diffusion Probabilistic Models (DPMs). We tackle this critical issue from a dynamic systems perspective, in which the inherent sequential nature is transformed into a parallel sampling process. Specifically, we propose a unified framework that generalizes the sequential sampling process of DPMs as solving a system of banded nonlinear equations. Under this generic framework, we reveal that the Jacobian of the banded nonlinear equations system possesses a unit-diagonal structure, enabling further approximation for acceleration. Moreover, we theoretically propose an effective initialization approach for parallel sampling methods. Finally, we construct \textit{ParaSolver}, a hierarchical parallel sampling technique that enhances sampling speed without compromising quality. Extensive experiments show that ParaSolver achieves up to \textbf{12.1× speedup} in terms of wall-clock time. The source code is publicly available at https://github.com/Jianrong-Lu/ParaSolver.git.

Published as a conference paper at ICLR 2025
PARASOLVER: A HIERARCHICAL PARALLEL INTE- GRAL SOLVER FOR DIFFUSION MODELS
Jianrong Lu, Zhiyu Zhu, and Junhui Hou*
"Department of Computer Science, City University of Hong Kong, Hong Kong SAR irong. alvin@ com, chiyuzha2-cemy cityu. edu, hk, jh. hwuhoouucccccooo ci tyu. tyy.. edu. hk
ABSTRACT
This paper explores the challenge of accelerating the sequential inference pro- cess of Diffusion Probabilistic Models (DPMs). We tackle this critical issue from a dynamic systems perspective, in which the inherent sequential nature is propose uni-
transformed into a parallel sampling process. Specifically, fied framework that generalizes the sequential sampling process of DPMs as solving system of banded nonlinear equations. Under this generic frame- work, we reveal that the Jacobian of the banded nonlinear equations system possesses unit-diagonal structure, enabling further approximation for acceler- ation. Moreover, we theoretically propose an effective initialization approach for parallel sampling methods. Finally, we construct ParaSolver, hierarchical parallel sampling techmique that enhances sampling speed without compromis- ing quality. Extensive experiments show that ParaSolver achieves up to 12.1x speedup in terms of wall-clock time. The source code is publicly available at https:/ thtbbb....uuu.buuooo//// com
INTRODUCTION
Over the past few years, the landscape of generative modeling has been significantly reshaped by the ascent of diffusion probabilistic models (DPMs) (Ho et al., 2020; Song et al., b). These models have emerged as a pivotal methodology for diverse applications (Yang et al., 2024; Liu et al., 2023a; Lu et al., 2024; Chung et al., 2023; Lu et al., 2023; You al., 2025; Liu et al., 2025), spanning from high-quality image/video synthesis (Rombach et al., 2022: Blattmann et al., 2023) to molec- ular generation (Wu et al., 2024; Nguyen et al., 2023). In essence, diffusion models hinge noise reduction process that is represented mathematically as an ordinary or stochastic differential equation (ODE/SDE). The equation systematically removes noise from initial normal distribu- tion, transforming it into vivid sample that conforms to the intended real data distribution, thereby facilitating high-quality generative modeling. To produce high-quality samples, nevertheless, DPMs typically necessitate a multitude of sequential noise reduction iterations involving extensive evalua- tions of large neural networks, leading to a notably slow sampling speed.
Researchers have put forth range of methods (Gong et al., 2024; Zheng al., 2023; Gonzalez et al., 2024; Geng et al., 2024: Liu et al., 2023b; Luo et al., 2023) to improve sampling speed. One strategy involves developing faster SDE/ODE solvers like DDIM (Song et al., a) and DPMSolver (Lu et al., 2022) based on mathematical principles to expedite the sampling process. However, these approaches often require a reduction the number of denoising steps, which impacts the quality of the samples. Another category is to distill the ODE trajectory of pre-trained diffusion model into another neural network that enables short-step sampling, exemplified by work such as (Salimans & Ho; Song et al., 2023). An additional enhancement for this kind of method is straighten the convoluted ODE trajectory of DPMs into a straight line using the Rectified Flow (Lipman et al.; Liu et al.). Nevertheless, it is common for these approaches to result in decline in both image quality and diversity.
"The first two authors contributed to this work equally. Corresponding authors: Zhiyu Zhu and Junhui Hou. This project supported in part by the NSFC Excellent Young Scientists Fund 62422118 18, and in part by the Hong Kong RGC under Grants 11219422, 11219324 and 12181211

---

Published as a conference paper at ICLR 2025
To circumvent the aforementioned questions and expedite the sampling process. Shih et al. (2024b) introduced a parallelization technique that simultancously denoises multiple steps through Picard iteration. Subsequently. Tang et al. (2024) further redefined the parallel sampling paradigm of dif- fusion models, framing it as the solution of a series of triangular nonlinear equations via fixed- point iteration (FP). These methodologies present trio of distinct benefits compared to existing approaches: (1) training-fiee frre and compatible with existing fast sampling methods; (2) yielding sam- ples with comparable quality sequential sampling; (3) resulting in a notable decrease sampling steps, thereby greatly enhancing sampling efficiency.
To further extend the capabilities of the parallel methods and enable new sampling paradigms, we propose ParaSolver, unified framework that generalizes previous approaches through the lens of nonlinear equations (NEs). Specifically. we formulate the sampling process of DPMs as a system banded structure. This framework leads to parallel sampling
of NEs with a computation in hierarchical way, thus improving the efficiency of parallel sampling methods in situations with constrained computing resources. Moreover, any current sequential or parallel sampling algorithms for DPMs can be seamlessly integrated into our framework.
In summary, this paper encompasses the following theoretical and practical contributions:
we propose a novel parallel sampling algorithm for diffusion models named ParaSolver, via partitioning the sequential inference process into a system of non-linear equations;
we accelerate and simplify the updating process via the unique structure of the Jacobian matrix of the system, which has a unit-diagonal structure with compute-intensive gradients in the sub-diagonal;
we conduct extensive experiments, showcasing that our ParaSolver can achieve 2x ~ 12x speedup in terms of wall-clock time, establishing new record in this field.
2
RELATED WORK
Parallel sampling that enables faster sampling without quality degradation opens up a promising av- enue for faster sampling of DPMs. It concurrently denoises multiple sample points on the ODE/SDE trajectories of DPMs by iteratively refining them from an array of initial guesses. To the best of our knowledge, the recent studies Shih al. (2024a) and Tang et al. (2024) are the only research en- deavors concentrating on parallel sampling methods for DPMs. ParaDiGMS (Shih et al., 2024a) represents the pioneering parallel sampling approach. treats the discretized sample points along the ODE trajectories as sequence of fixed points. These points are progressively enhanced through iterations using the established fixed-poimt theorem and et al., 2024) extends the fixed-poin iteration approach by defining the sampling process of DPMs as solving triangular nonlinear equations, with all discretized sample points acting as unknown vari- ables. In contrast to ParaTAA, nonlinear equations. This leads to a hierarchical parallel method, which totally different from existing methods parallelizing all points in the ODE trajectories. Moreover, this banded structure, characterized by its sparsity, facilitates significantly more efficient computations compared to the compute-intensin intensiie triangular structure. Additionally, ParaTAA can be seamlessly incorporated into our framework. We have also identified intriguing research areas dedicated to expediting diffusion models. These include topics like model or sample partitioning (Ma et al., 2024b;a; Wang et al., 2024; Li et al., 2024), trajectory stitching (Pan et al., 2024), nested diffusion (Elata et al., 2024), minimum denoising step prediction (Yu & Barad, 2024), and theoretical assurances (Gupta et al., 2024; Chen et al., 2024). Our strategy revolves around parallelizing the sequential sampling process of DPMs, which complementary to theirs.
theorem. ParaTAA (Tang
frame the sampling procedure of DPMs as set of banded
While these methods show promising benefits, several drawbacks constrain their parallel efficiency. First, parallelizing all discretized sample points simultaneously becomes compute-intensive when handling large models, leading to a rapid degradation in parallel efficiency. Second, the final clean sample relies all samples from previous timesteps. As result, errors from these prior sample points all accumulate in the final clean samples. This, in turn, necessitates additional iterations to correct these accumulated errors, further restricting parallel efficiency.
2

---

Published as a conference paper at ICLR 2025
3 PRELIMINARY
Diffusion Models. Denote by X, E R BWC the noised variable on the diffusion trajectory, t E [0, T] the scalar indicating the time-stamp, [(1) the drift coefficient function, and g(() the diffusion coef- ficient function. Then, the forward diffusion process can be formulated as the following stochastic differential equation (SDE), which projects the clean image XT to a random noise Xo (we define the clean image at time T and the noise at time 0),
dX, [(L)X,dl + g())d,
where d'W indicates the standard Wiener process. Then, to generate the corresponding clean latent from the easily sampled random noise, we have reverse the forward SDE in Eq. (l), resulting in the following reverse SDE formulations
dX,- = [f(t)x, Vt) log (X))) dt + g(t) dw,
(2)
where Vx, ((X;) can be approximated by score function Sz(+), parameterized by neural network with learnable weights of 0; (o(,,,t) denotes the drift function for the reverse diffusion process; o, represents the corresponding coefficient of diffusion counterpart. Let $(t, 8, X,) repre- sent an integral result of X, by Eq. (2) over a time interval from to t, with an initial value X,:
(tt s, X,) - x, + !.
xX,
(x(x r)dr + a.dW.
Consequently, the analytical solution of Eq. (2) at time & can be expressed as X, (c, 0, Xo), Xo N((O I),
(4)
where (0, I) denotes the standard Gaussian distribution.
Parallel Sampling Algorithms. For array of ODE/SDE trajectories for DPMs
[X,, f = 0,.... 77, existing parallel sampling algorithms (Shih et al., 2024a; Tang et al., 2024) establish the following system of non-linear equations to reformulate the integral-based formulation
of the diffusion model:
X H (X,
x,-)) 0,
(5)
where is the number of utilizing previous states at timestamp 1; H4(0) denotes a solver for estimating results in timestamp t with acknowledging previous states, i.e., X,, .X,ii
The sampling methods utilize an iterative refinement manner to gradually adjust an estimation trajectory (x,, f 0,..,,7" .T-- Each state from the trajectory (X,, 1. = 0,... T) is first initial-
ized with coarse, even noise value, denoted as {x , 0, ... .T. .T. Denote by X, the vector,
NOT [T],".. JT. Then, for the k-th parallel iteration (k [0, KJ), Newton-Raphson method (Kelley, 2003) updates the variables by the following scheme
(k+1) (k) o.r
g*
O:T" (
(6)
O.T
where 2,²0)
x((²)
(k)
H,
(5)
(())) indicates residual term to be optimized; and G+
(J(K) indicates the inverse of Jacobian matrix j(A)
2Xa.7
4 PROPOSED METHOD
Parallel solvers transform the sequential integral process of the diffusion model, represented in Eq. (4), into the view of non-linear equations articulated in Eq. (6). However, the typically large number of timestamps complicates the simultaneous solving of all nodes, particularly with limited number of GPU/CPU cores. To mitigate this issue, we propose a hybrid approach that integrates the fully sequential inference of classical diffusion models with comprchensive parallel methodolo- gies. Specifically, we reformulate the partitioning of non-linear equations using a banded structure, which leverages the strengths of both parallel and sequential techniques, as discussed in Sec. 4.1. Furthermore, based on this formulation, we analyze strategies to simplify and accelerate the solving process with corresponding steps for initialization and stopping criterion in Sec. 4.2.
'To adapt the parallel sampling algorithm, we discretize the continuous trajectory [0. T] into several inter- vals [0, 1,.... , T] by a step size of
3

---

Published as a conference paper at ICLR 2025
4.1 HIERARCHICAL SAMPLING VIA BANDED NON-LINEAR EQUATIONS
We start with the re-investigation of the integral formulation of diffusion process Eq. (4) in the following partitioned manner,
Xt (ln++1, X,,,,, n (0, ,....,NN11 -N
where the whole temporal region of [0, 7 partitioned into N time sub-intervals, with N & T and 0 to < 11 < .. T Inspired by current parallel algorithms, e.g.. Song et al. (2021), such a sequential problem can be solved in a parallel manner. Specifically, such a series of cascaded functions can be treated as a system of NEs with a banded structure. Denote by (X,,. n
0, 0, N a set of points exactly on the diffusion trajectory, X 0, .N the points to be optimized approaching the trajectory. A cascade of local integral formulations as Eq. (7) can be reformulated as the solutions of following Banded NEs.
Definition (Banded NEs). in Eq. (7) as
We define the system of banded NEs for the sequential sampling process
X - x =0, X ((nn+1, tn x.,) 0, (0.1, ,N 1}.
(8)
Moreover, we also have theoretical analysis that the solution of Eq. (8) is the unique and unbiased estimation of the sequential sampling results, as shown in Prop.
Proposition 1 (Unbiased Estimation). The set of banded NEs in Eq. (8) possesses unique solution, which is also an unbiased estimation of the sequential diffusion results.
Proof. See Appendix A.
By the aforementioned, we reformulate the sequential sampling process of the diffusion model into two hierarchical phases, i.e., globally solving the banded NEs of Eq. (8) and locally calculating the integral process ((.). Each of them can be solved parallel or sequential by existing methods, which distinguishes the proposed method compared to other sequential or parallel sampling methods. Since solving the global non-linear equations is the primary bottleneck for the acceleration of diffusion sampling, we will mainly discuss the methods to further simplify and accelerate solving Eq. (8).
Novelty Claim. Due to the aforementioned hierarchical sampling manner, the system of NEs in our proposed method has smaller-scal NEs and variables than the existing parallel sampling al- gorithms and features notably sparse banded structure. Consequently, hierarchical sampling can conserve numerous computational resources than the existing methods with dense triangular struc- ture. Moreover, we want to note that our framework encapsulates the existing parallel and sequential methods. The banded NEs with N 1 and N T are precisely aligned with the sequential sam- pling method Eq. (4) and the parallel sampling approach in Eq. (5), respectively.
4.2 SOLVING THE SYSTEM OF BANDED NES
We investigate the Newton-Raphson method (Kelley, 2003) solve the aforementioned NEs as Eq. (8), as
X
(k-1) (k) battx
(k) (J
R
(9)
where R!!.
(R) (k)
o( ty. (k) indicates the residual term based on our previously for-
fatt
mulated NEs; [(t)
represents the inverse of Jacobian matrix as (R(()
(k) toity"
(k)
(ostt
Proposition 2 (Smoothness Analysis). b(tn+1, X.,)) satisfies 1..t,,11.1
8x,, 0t Ctn - 1) x 1, where O o < 1; ar and a are the noise schedule for the forward J0tn11
process (ot, Xey, ot., from clean image Xax noisy image at la- This small upper bound
leads to sufficiently smooth 4. implying Re,, is smooth enough.
We discretize the continuous trajectory [0, T) into several intervals ffo,t1,".. ,N by a step size of not
less than
4

---

Published as a conference paper at ICLR 2025
Proof. See Appendix B.
Applicability analysis. Proposition 2 suggests that the residual for the nonlinear system in Defini- tion has a sufficiently smooth landscape, enabling Newton's method for root-finding to converge rapidly without getting stuck easily.
Although, through such manner, seem be solving the problem efficiently in parallel manner, the calculation the reverse Jacobian matrix is computationally complex, even unsolvable.
Inspired by the classical optimization theory in NEs (Wolfe, 1959; Broyden, 1965; Anderson, 1965), approximate the reverse Jacobian of the NEs system and execute Jacobian-like update based on this estimation. Our initial crucial insight is that we can precisely calculate the diagonal component of the Jacobian by taking into account the specific structure of the banded NEs system. The resid-
(()
ual term of the banded NEs system comprises two components: the part foitw behaves as linear
functions of itself, while the remaining part of 4 ti LN, to tN-11 ((k
tg:tt generally represents
nonlinear functions of (R) toity because it involves the neural network Sp
(g:ty)
These semi-
linear NEs are ignorant by existing method (Tang et al., 2024) as estimates the entire triangular part of the Jacobian matrix, which causes estimated errors in the linear part.
We note that for the semi-linear banded NEs system, the reverse Jacobian at h parallel iteration can be exactly formulated as a unit lower block banded matrix:
I, oe ox
if i
(
Bij
if if=
(10)
otherwise
row i and column j. Based on this insight, we can obtain an exact parallel sampling iteration for our banded NEs system.
where Bij is block matrix within the unit lower block banded matrix (Tostt
Proposition 3 (Exact Parallel Sampling). Given a set of initial values (X nn= 0, ,N}, the exact parallel recurrence for our semi-linear banded NEs system in Definition parallel iteration k is
(k+1) = - = X10 ~ ~N(O.I). (k+1) = (ttn+1, tnn tn +
60(1, 1stn 1nt.. x!!i
(1+1)
x())
(b)
(11)
fnt1
ex!
tn
Proof. See Appendix C.
For the nonlinear part, i.e., the Jacobian term (t x(1) ax
in practice, is expensive to compute.
Following numerous studies Jacobian-free backpropagation (Fung et al., 2022; Geng et al.; Liu et al., 2024; Knoll & Keyes, 2004; Poole et al.), we find that substituting the Jacobian with the identity matrix cannot impact the iteration process. This yields the following effective recurrence:
X (k+1) {(1) X ~N(,
(+D (ttn-1, (k) + (k+1) x!!) t.
(8)
(12)
Reorganizing Eq. (12) leads to the formula of its general term
(k+1)
x())
to X ~ N(O,I),
to
x
((++1) Σ 11
Σ
(K)
(13)
Proposition 4 (Convergence Analysis). The update rule in Eq. (13) has descent direction that is not contradictory to the exact update rule in Eq. (11) for the residual (k) Its convergence speed falls between linear and quadratic convergence.
5

---

Published as a conference paper at ICLR 2025
Proof. See Appendix D.
Stopping Criterion. We need to establish a stopping criterion for our parallel sampling to prevent any decline in sample quality. Setting a sufficiently low tolerance ensures that the outcome of parallel sampling closely resembles that of the sequential sampling procedure. Specifically, denote by ó the tolerance. We follow Shih et al. (2024a) to define the stopping criterion as
| & 8'
(14)
where is the dimensional of sample X,,; 111p is Frobenius norm. As demonstrated in Shih al. (2024a), it has been established that this relaxed stopping criterion can guarantee that the samples of X.K are obtained from a distribution that exhibits minimal total variation distance from the fx
DDPM model distribution.
Initialization. The parallel iteration in Eq. (13) commences with an array of initial values (X(1) n 0,...,. ,N.. Initiating the iterations with values that closely approximate the solu- tions (X, = 0.... ,N will lead to swift convergence. Hence, initialization also stands as a crucial component for ensuring parallel efficiency. To review the banded NEs system in Defini- tion 1, we find that the initializations should satisfy the condition X Φf tn. x (0)) with
(0) IO. However, executing such sampling procedure, whether in parallel or sequentially to
across N + time points lo. ... ,,,, which may be large, usually requires a significant amount of time.
Given this, our core insight is to utilize the principle of DPMs to fast construct set of more precise initial values that conform to the Definition 1. particular, the condition X,,.1 = ((1x+1, In, x... can be estimated by the reverse process using coarse timesteps from bn to tn-1:
X _ XX,,, X:x),
(15)
where q is the transition probability distribution from Xt,, to XI... for the reverse process, which becomes traceable when conditioned on the clean sample Xxx (we define the clean image time
IN = T and the noise at time I = 0). According to the forward process X,,, ~ N No,,, Xtx Tt,, I) where and are the noise schedule, the clean sample parallel iteration can be estimated by the neural network Sa( (t,,,., tx) at time tx:
X (6)
(k) .t,) ot, S6 (xx
X 110 (t.,)
(16)
at
Once obtaining the estimated clean sample Xix- we can then approximate all the remaining noisy samples via the reverse process negligible costs. To initialize the diffusion trajectory, we can perform a few steps of the reverse process as preconditioning steps, with minimal expense. Denote by M the number of the preconditioning steps. Then, after M steps of the reverse process, we can approximate the clean sample by Eq. (16) using x (fM-1). Formally, we define the set of initialized values follows:
x
X 1 g (0) (0)
if < n M ( (tM-)), if <<< h.11
(17)
x 1m
where X., N(O. I); X,, is approximated by (00) (tM-1) time tw- using Eq. (16); for M = 0, we follow Shih et al. (2024a) to initialize all the samples with random noises.
Sliding Window. ParaSovler requires maintaining N + 1 parallel denoised points {X (k)
L.
o,... .N) throughout time, which can be excessively large feed into GPU memory when dealing with a sizable N. To tackle this, we use the sliding window technique from Shih et al. (2024a). This
technique conducts parallel iteration merely on subset of points (x(()
0,. ,P- p--1 1} within
a window of size p. where 1 & p & N. This window size can be adjusted to adhere to the limitations
of the GPU memory. The window is then promptly advanced once the states of the current timesteps within it converge.
Algorithm details the complete process of the proposed ParaSolver. After preparing an array of initial guesses (Xx(0) 0,... .) via random noises or few preconditioning steps at negligible

---

Published as a conference paper at ICLR 2025
Algorithm 1: ParaSolver: a hierarchical parallel sampling method for diffusion models
Input Diffusion model Sş, subinterval number preconditioning steps M, tolerance batch window size p. sample dimension
Output : A sample.
Initialize (8 ..... pp) by Eq. (17)
// Initialize with a few sampling steps
0,0, E [0, KJ. and n 10,
while do
((t441,44; ()), ViE fn.... , n
I/ Solve each subproblems concurrently.
A (1 X(, - x Vie (n...., n --
Calculate drifts.
x( x!11 Σ i (k) Vie (n....,,. ..t..npppp ,n+p-1)
Update previous states via Eq. (13)
argmint, t2 X (k+1) unsatisfying Eq. (14), Vi (n.... , 1-1 // The sliding stride
Ohtain & (to+p-1) with the score from Line 4 bntp-1 by Eg. (16) xx
// Predict clean sample. // Initialize new
x(t+1) q( (xx... (1+1) 11 x00 (I++p-1)), Vi E + p. n + 1).
X144
points.
10
n n +
k +l, p p- min(p,
Return: ((K)
IN
costs (Line 1), ParaSolver initiates the parallel sampling loop at Line 2 in which a batch of samples [X[[ = 0, 0,... P 1} within a sliding window undergo parallel denoising. Line 4 executes sampling for all the subproblems coinstantaneously in which each subproblem can be solved by existing parallel or sequential sampling methods, starting from the preceding states (x
0,... p 1}. The drifts are computed in Line subsequent subproblems sampling, and Line updates the previous states from the prior iteration. Line 7 checks the variation between new states and the current states and then determines the extent which the window can slide forward. Line initializes new points outside the current window according to the sliding stride.
EXPERIMENT
We evaluate ParaSolver across various figh-dimensional image generation models such as latent- space diffusion model StableDiffussin-v22 (Rombach et al., 2022) and pixel-space diffusion model LSUN Church (Yu et al., 2015). The results of these experiments demonstrate that ParaSolver enhances the efficiency of sequential sampling methods by approximately 2 12 times, all while maintaining consistent sample quality as measured by metrics like FID score or CLIP score.
5.1 EXPERIMENT SETTINGS
Evaluation Metrics. We explore five widely used metrics for our proposed PararSolver: the number of function evaluations (NFE), the iteration number (iters), wall-clock time, FID score (Heusel al., 2017), and CLIP score (Hessel et al., 2021).
Datasets and Models. As per ParaDiGMS Shih et al. (2024a), we analyze our ParaSolver across latent-space and pixel-space models. For latent-space models, we leverage the StableDiffusion-22 model model and evaluate FID and CLIP scores on random selection of 5000 samples from the ImageNet- 1k dataset. . The StableDiffusion--22 model generates images at a resolution of 768 by 768 pixels, utilizing diffusion model that operates within a latent space dimension of 4 by 96 by 96. For pixel-space models, we employ models pretrained on the LSUN Church dataset This LSUN Church pre-trained model operates in the pixel space with resolution of 256 by 256. We compute the FID score on 5000 random samples of LSUN Church datasets. We assess the performance of all methods on 8 NVIDIA RTX 3090 GPUs, each equipped with 24268 MB of memory.
Algorithms. We apply our approach to accelerate the performance of the state-of-the-ar sequential sampling methods: DDPM (Ho et al., 2020), DDIM (Song et al., a), and DPMSolver (Lu et al., 2022). We evaluate the parallel efficiency of our ParaSolver against ParaDiGMS in accelerating the aforementioned three sequential methods. ParaDiGMS is implemented based on the the Diffusers library, and the other algorithms are all accessible in the widely-used library Diffusers (von Platen

---

Published as a conference paper at ICLR 2025
Table 1: Quantitative comparisons of different methods Stable Diffusion-v2 over 5000 random samples from the ImageNet-1k dataset, with classifier guidance w = 7.5. The visual comparisons are shown in Figs. 3, 4, and in Section in Appendix. The best results are highlighted in bold. "I" (resp. "I") means the larger (resp, smaller), the better.
Stable Diffusion-v2
Steps
Method
Iters 1 NFE
CLIP1
FID
Time (s) Speedup1
DDPM
1000
1000
25.6
55.9
128.0
1.0x
1000
DDPM + ParaDiGMS
65
2024
25.6
55.6
32.8
3.9x
DDPM + ParaSolver
32
1065
25.6
55.3
15.9
8.1x
DDIM
50
50
25.6
57.2
6.3
1.0x
50
DDIM ParaDiGMS
21
132
25.6
56.9
3.3
1.9x
DDIM ParaSolver
13
83
25.6
56.9
1.9
3.3x
DDIM
25
25
25.4
62.9
3.4
1.0x
25
DDIM ParaDiGMS
18
53
25.4
62.8
2.9
1.2x
DDIM ParaSolver
49
25.4
61.9
1.2
2.8x
DPMSolver
50
25.6
57.2
6.3
1.0x
50
DPMSolver + ParaDiGMS
25
132
25.6
57.2
3.3
1.8x
DPMSolver + ParaSolver
15
96
25.6
57.1
2.2
3.1x
DPMSolver
25
25
25.4
62.3
3.2
1.0x
25 DPMSolver + ParaSolver
DPMSolver + ParaDiGMS
15
81
25.4
62.3
2.2
1.5x
11
58
25.4
62.1
1.5
2.1x
et al., 2022); hence we utilize them directly. However, we exclude the comparison with another parallel method (Tang et al., 2024) as it has not been integrated into the Diffusers library yet.
Hyperparameter Settings. Following the previous settings in (Shih et al., 2024a; Tang et al., 2024), we apply our ParaSolver and ParaDiGMS to DDPM with 1000 sequential sampling steps. For DDIM and DPMSolvvr, we consider two settings: 25 and sequential sampling steps, as they are commonly utilized and capable of producing samples of similar quality to DDPM. Besides, We tune the best tolerance for ParaDiGMS via grid search on (0.001, 0.005, 0.01, 0.5, 0.1). We find for StableDiffusion-v2 ParaDiGMS for DDPM, DDIM, and DPMSolver need to set the tolerance as 0.5, 0.01, and 0.01 to achieve the similar sample quality as the corresponding sequential method, respectively. For LSUN Church model, the tolerance of ParaDiGMS should be 0.5 and 0.001 for DDPM and DDIM. Moreover, for each setting, we sweep all the window size to find the best speedup for ParaDiGMS. For ParaSolver StableDiiftsionn-22 model, set the tolerance 0.55, 0.05, and 0.05 for ParaSolver on DDPM, DDIM, and DPMSolver respectively. For LSUN Church model, the tolerances of ParaSolver are set 0.55 and 0.005 for DDPM and DDIM, respectively.
5.2 EXPERIMENTAL RESULTS
Latent-space diffusion model. Table I shows the experimental results DDPM, DDIM, DPM- Solver, and their parallel variants when combined with ParaSolver and ParaDiGMS. respectively. Initially, ParaSolver can expedite the performance of DDPM, DDIM, and DPMSolver without com- promising FID and CLIP scores. The FID and CLIP scores are marginally superior to the baseline values, think the employed tolerances might be relatively tight, thereby enhancing the overall outputs. Futhermore. ParaSolver achieves the best speedup, up to 8.1x in terms of the wall-clock time. This is a new record for existing related parallel methods. Parallelization sacrifices com- putation for speed, resulting in higher NEEs compared to the sequential methods. Nevertheless, when contrasted with ParaDiGMS, our ParaSolver notably reduces NFEs, even requiring mere 1065 NFEs for 1000 steps in DDPM. ParaSolver significantly diminishes the iteration count required for sequential methods by a factor ranging from 2.5 times to 31 times.
8

---

Published as a conference paper at ICLR 2025
Pixel-space diffu- sion model. Table shows the exper- imental outcomes regarding DDPM and DDIM alongside their parallel adaptations with ParaSolver and ParaDiGMS on the pretrained LSUN Church model. Simi- lar to the latent space results, ParaSolver consistently surpasses existing techniques, achieving speedups ranging from 2.2 12.1 times while upholding sample quality comparable to that of the sequential methods. Producing sample without any loss in quality within 4.1 seconds instead of 50.0 seconds can notably enhance the interactive experience for various generative scenarios like the human-in-the-loop applications (Bhattacharya et al., 2023).
Table Quantitative comparisons of different methods on LSUN Church over 5000 samples. DPMSolver is not integrated with the LSUN model in the Diffusers library; thus we exclude it.
LSUN Church
Steps
Method
Iters
NFE
FID
Time (s) Speedup
DDPM
1000
1000
12.7
50.0
1.0x
1000
DDPM + ParaDiGMS
65
2082
12.7
10.8
4.6x
DDPM ParaSolver
42
1079
12.7
4.1
I
12.1x
DDIM
50
50
15.5
1.8
1.0x
50
DDIM + ParaDiGMS
23
202
15.8
1.5
1.2x
DDIM + ParaSolver
14
73
15.7
0.8
2.2x
DDIM
25
25
15.6
1.1
1.0x
25
DDIM + ParaDiGMS
15
96
15.7
0.8
1.4x
DDIM + ParaSolver
10
44
15.7
0.5
2.2x
The effect of M. Ta- ble 3 investigates the impact of the precon- ditioning steps M. It evident that a trade- off exists between M and the acceleration achieved, with the op- timal M identified at M 10. This is because increasing the number of precondi- tioning steps leads to a more accurate ini- tialization but also re- quires additional time. As we implement the preconditioning steps through a sequential sampling approach, the efficiency of parallel processes is affected by large M. We believe that by executing the preconditioning steps in parallel, we can enhance the precision of initialization, facilitating a with higher M.
Table 3: The effect of preconditioning steps M using LSUN Church over 5000 random samples. Sect. H in Appendix shows the visual comparisons.
LSUN Church
Steps
Method
M
Iters]
NFE
FID]
Time (s)
Speedupt
DDPM
1000
1000
12.7
50.0
1.0x
DDPM + ParaSolver 0
1036
12.8
4.4
11.3x
DDPM + ParuSolver 1
54
1017
12.8
4.3
11.6x
1000
DDPM
ParaSolver 10
42
1066
12.7
4.1
12.1x
DDPM + ParaSolver
15
1078
12.7
4.2
11.9x
faster acceleration
The effect of N. Table 4 examines the influence of N. The most favorable wall-clock time is ob- served at N = 15. At N 1, ParaSolver reverts to a sequential sampling approach. However, for N 5 and N 10, the wall-clock time exceeds the sequential sampling because the current implementation for solving the subproblems is through the sequential sampling. With each paral- lel iteration involving the sequential sampling, the parallel efficiency is significantly compromised. Substituting the sequential sampling with parallel techniques could substantially enhance the effi- ciency. We believe that by leveraging optimized implementations, the wall-clock times Table can be further refined. For N = 30, ParaSolver aligns with existing parallel methods with our ini- tialization method, necessitating the computation of the entire triangular part as specified in Eq, (5). This demanding computational task results in extended wall-clock times. We note that to tune the window size, the wall-clock time in the Table can be further improved. Moreover, parisons presented in Section G in the Appendix indicate that employing a larger value of N may result in increased errors on the refining samples during parallel iteration. This occurs because the ultimate clean samples are connected to all samples from earlier time steps, causing their errors dur-
visual com-

---

Published as a conference paper at ICLR 2025
Table 4: The effect the number of subintervals N using LSUN Church over 5000 random sam- ples. Our ParaSolver is employed to expedite DDIM with 30 sequential steps. We set M 10 and fix the window size p = 30. The visual comparisons are shown in Section Appendix.
N =
N W==
N 10
15 FIDI time (s)
N
N 30
Method
FIDI time (s)
FID time (s)
FID] time (s)
FID] time (s)
Ours
15.4
1.2
15.3
1.8
15.3
1.4
15.2
0.9
15.2
1.5
The effect of the number of GPUs using LSUN Church. Our ParaSolver is employed to expedite DDPM with 1000 sequential steps.
Table 5:
GPU
2 GPUs
4 GPUs
GPUs
Method
Time (s)l Speedupt Time (s) Speedupt
Time (s) Speedupt
Time (s) Speedupt
DDPM 50.0
1.0x
50.0
1.0x
50.0
1.0x
50.0
1.0x
ParaDiGMS37.0
1.4x
23.3
2.1x
15.2
3.3x
10.8 I
4.6x
Ours 19.7
2.5x
13.0
3.8x
9.9
5.1x
4.1
|
12.1x
ing updation to all accumulate in the final clean samples. Refining these errors requires additional parallel iterations, which can reduce the parallel efficiency.
The effect of the number of GPUs. In Table 5, the analysis focuses on how the number of GPUs affects performance. When ample computing resources are available, the paral- lel efficiency of our method can be greatly released than ParaDiGMS, resulting in a speedup of 12.1 times. Addi- tionally, even scenarios with limited computing power, such utilizing only single RTX 3090 GPU, the speedup can reach 2.5 times, whereas ParaDiGMS shows only mod- est improvement. This is attributed to our sampling process, which only needs to handle a sparse banded NEs system, in contrast to ParaDiGMS, which requires full computation over the dense triangular NES system.
Table 6: The effect of the toler- ance 5 on FID using LSUN Church over 5000 random samples. Our Para- Solver is employed to DDIM with 30 sequential steps with FID 15.5.
M 8 0.001|0.0030.0100.000
0.1
FID 15.3
15.6 (5.6|21.440.6(10044
FID 14.8
15.3
20.7
50.3
The effect of the tolerance 5. Table 6, we demonstrate how tolerance impacts FID under different preconditioning steps M. is evident that ParaSolver can uphold a similar
1I 14.3 15.2 16.6
18.2
23.2
or better FID to the baseline when 0.001, respectively. Furthermore, a higher M leads to a better FID even under a looser tolerance.
0.005 and
The effect of the window size In Figure 1, we examine how the window size p affects the speedup of 1000 DDPM over the LSUN Church Model. The peak speedup is ob- served at p 36 rather than the maximal p. This because the parallel efficiency is inundated by the significant com- putational burden associated with larger window sizes. In situations with restricted computational resources, it is cru- cial to select an appropriate window size for optimal per- formance.
Pookka Counte
Waloo Side 51
Figure 1: The effect of p
CONCLUSION
In this paper, we conceptualized the process of sequential sampling in DPMs solving system of banded NEs leading to general parallel sampling approach Leveraging the semi-linear structure of the Jacobian matrix within this NEs system, we derived more precise parallel sampling iteration. Building upon this iteration, we introduced a novel hierarchical parallel sampling technique called ParaSolver. ParaSolver is computationally efficient and seamlessly compatible with existing parallel and sequential methodologies. Extensive experiments demonstrate that our ParaSolver can achieve speedup of up to 12x in terms of wall-clock time without degrading the sample quality.
10

---

Published as a conference paper at ICLR 2025
ETHICS & REPRODUCIBILITY STATEMENTS
The proposed sampling algorithm is designed to accelerate the pre-trained models. We have read and adhere to the Code of Ethies of ICLR 2025. Thus, no further additional information on human subjects and potentially harmful insights is involved. Moreover, set random seed of 1 during the experiment, which ensures reproducibility. We will make the source code publicly available on Github.
REFERENCES
Donald G Anderson. Iterative procedures for nonlinear integral equations. Journal of the ACM (JACM), 12(4):547-560, 1965.
Stefan Banach. Sur les opérations dans les ensembles abstraits et leur application aux équations intégrales. Fundamenta Mathematicae, 3:133-181, 1922.
Mangolika Bhattacharya, Mihai Penica, Eoin O'Connell, Mark Southern, and Martin Hayes. Human-in-loop: review of smart manufacturing deployments. Systems, 111::33, 2023.
Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram Voleti, Adam Letts, et al. Stable video diffusion: Scaling latent video diffusion models to large datasets, arXiv preprint arXiv:231..15127 2023.
Charles G Broyden. A class of methods for solving nonlinear simultancous equations. Mathematics of computation, 19(92):577-593 1965.
Haoxuan Chen, Yinuo Ren, Lexing Ying, and Grant M Rotskoff. Accelerating diffusion models with parallel sampling: Inference sub-linear time complexity. arXiv preprint arXiv:2405. 15986, 2024.
Hyungjin Chung, Jeongsol Kim, Sehui Kim, and Jong Chul Ye. Parallel diffusion models of operator and image for blind inverse problems. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6059-6069, 2023.
Noam Elata, Bahjat Kawar, Tomer Michaeli, and Michael Elad. Nested diffusion processes for anytime image generation. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 5018-5027, 2024.
Samy Wu Fung, Howard Heaton, Qiuwei Ii, Daniel McKenzie, Stanley Osher, and Wotao Yin. Jfb: Jacobian-fres backpropagation for implicit networks. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pp. 6648-6656, 2022.
Zhengyang Geng, Meng-Hao Guo, Hongxu Chen, Xia Li, Ke Wei, and Zhouchen Lin. Is attention better than matrix decomposition? In International Conference Learning Representations.
Zhengyang Geng, Ashwini Pokle, and Zico Kolter. One-step diffusion distillation via deep equi- librium models. Advances in Neural Information Processing Systems, 36, 2024.
Mingming Gong, Shaoan Xie, Wei Wei, Matthias Grundmann, Kayhan Batmanghelich, Tingho Hou, et al. Semi-implicit denoising diffusion models (siddms). Advances in Neural Information Pro- cessing Systems, 36, 2024.
Martin Gonzalez, Nelson Fernandez Pinto, Thuy Tran, Hatem Hajri, Nader Masmoudi, et al. Seeds: Exponential sde solvers for fast high-quality sampling from diffusion models. Advances in Neural Information Processing Systems, 36, 2024.
Shivam Gupta, Linda Cai, and Sitan Chen. Faster diffusion-based sampling with randomized mid- points: Sequential and parallel. arXiv preprint arXiv:2406.000 00924, 2024.
Jack Hessel, Ari Holzzman, Maxwell Forbes, Ronan Le Bras, and Yejin Choi. Clipscore: A reference-free evaluation metric for image captioning. arXiv preprint urXiv:2104.08711 2021.
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium. Advances in neural information processing systems, 30, 2017.

---

Published as a conference paper at ICLR 2025
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 1166840-65511 2020.
Carl T Kelley. Solving nonlinear equations with Newton's method. SIAM, 2003.
Dana A Knoll and David E Keyes. Jacobian-free newton-krylov methods: a survey of approaches and applications, Journal of Computational Physics, 193(2):55-3377, 2004.
Muyang Li, Tianle Cai, Jiaxin Cao, Qinsheng Zhang, Han Cai, Junjie Bai, Yangqing Jia, Kai Li, and Song Han. Distrifusion: Distributed parallel inference for high-resolution diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp- 7183-7193, 2024.
Yaron Lipman, Ricky TQ Chen, Heli Ben-Hamu, Maximilian Nickel, and Matthew Le. Flow match- ing for generative modeling. In The Eleventh International Conference on Learning Representa- fions.
Kendong Liu, Zhiyu Zhu, Chuanhao Li, Hui Liu, Huanqiang Zeng, and Junhui Hou. Prefpaint: Aligning image inpainting diffusion model with human preference. Advances in Neural Informa- tion Processing Systems, 37:30554-30589, 2025.
Linghai Liu, Shuaicheng Tong, and Lisa Zhao. Training implicit networks for image deblurring using jacobian-free backpropagation. arXiv preprini arXiv:2402.02065. 2024.
Xingchao Liu, Chengyue Gong, al. Flow straight and fast: Learning to generate and transfer data with rectified flow. In The Eleventh International Conference on Learning Representations.
Xingchao Liu, Xiwen Zhang, Jianzhu Ma, Jian Peng, et al. Instaflow: One step enough for high-quality diffusion-based text-to-image generation. In The Twelfth International Conference on Learning Representations, 2023a.
Xingchao Liu, Xiwen Zhang, Jianzhu Ma, Jian Peng, et al. Instaflow: One step is enough for high-quality diffusion-based text-to-image generation. In The Twelfth International Conference on Learning Representations, 2023b
Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu. Dpm-solver: A fast ode solver for diffusion probabilistic model sampling in around 10 steps. Advances in Neural Information Processing Systems, 35:5775-5787, 2022.
Jianrong Lu, Lulu Xue, Wei Wan, Minghui Li, Leo Yu Zhang, and Shengqing Hu. Preserving privacy of input features across all stages of collaborative learning. In 2023.
Jannong Lu, Shengshan Hu, Wei Wan, Minghui Li, Leo Yu Zhang, Lulu Xue, Haohan Wang, and Hai Jin. Depriving the survival space of adversaries against poisoned gradients in federated learn- ing. IEEE Transactions on Information Forensics and Security, 2024.
Simian Luo, Yiqin Tan, Longbo Huang, Jian Li, and Hang Zhao. Latent consistency models: Synthe- sizing high-resolution images with few-step inference. arXiv preprint arXiv:2310.04377. 2023.
Xinyin Ma, Gongfan Fang, Michael Bi Mi, and Xinchao Wang. Learning-to-cache: Accelerating
diffusion transformer via layer caching. arXiv preprint arXiv:2406.01733, 2024a.
Xinyin Ma, Gongfan Fang, and Xinchao Wang. Deepcache: Accelerating diffusion models for free. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp- 15762-15772, 2024b.
Loe X Nguyen, Pyae Sone Aung, Huy Le, Seong-Bae Park, and Choong Seon Hong. A new chap- ter for medical image generation: the stable diffusion method. In 2023 International Conference Information Networking (ICOIN), pp. 483-486. IEEE, 2023.
Zizheng Pan, Bohan Zhuang, De-An Huang, Weili Nie, Zhiding Yu, Chaowei Xiao, Jianfei Cai, and Anima Anandkumar. T-stitch: Accelerating sampling in pre-trained diffusion models with trajectory stitching. arXiv preprint arXiv:2402. 14167, 2024.
12

---

Published as a conference paper at ICLR 2025
Ben Poole, Ajay Jain, Jonathan T Barron, and Ben Mildenhall. Dreamfusion: Text-to-3d using 2d diffusion. The Eleventh International Conference on Learning Representations.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF confer- ence on computer vision and pattern recognition, pp. 10684-10695, 2022.
Tim Salimans and Jonathan Ho. Progressive distillation for fast sampling of diffusion models. In International Conference Learning Representations.
Andy Shih, Suneel Belkhale, Stefano Ermon, Dorsa Sadigh, and Nima Anari. Parallel sampling of diffusion models. Advances in Neural Information Processing Systems, 36, 2024a.
Andy Shih, Suneel Belkhale, Stefano Ermon, Dorsa Sadigh, and Nima Anari. Parallel sampling of diffusion models. Advances in Neural Information Processing Systems, 36, 2024b.
Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. In Interna- tional Conference on Learning Representations, a.
Yang Song, Jascha Sohl-Dickstein, Diederik Kingma, Abhishek Kumar, Stefano Ermon, and Ben
Poole. Score-based generative modeling through stochastic differential equations. In Interna- tional Conference on Learning Representations,
Yang Song, Chenlin Meng, Renjie Liao, and Stefano Ermon. Accelerating feedforward computation via parallel nonlinear equation solving. In International Conference Machine Learning, pp- 9791-9800. PMLR, 2021.
Yang Song, Prafulla Dhariwal, Mark Chen, and Ilya Sutskever. Consistency models. International Conference Machine Learning, 2023.
Zhiwei Tang, Jiasheng Tang, Hao Luo, Fan Wang, and Tsung-Hui Chang. Accelerating parallel sampling diffusion models. In Forty-first International Conference on Machine Learning, 2024.
Patrick von Platen, Suraj Patil, Anton Lozhkov, Pedro Cuenca, Nathan Lambert, Kashif Ra- sul, Mishig Davaadorj, Dhruv Nair, Sayak Paul, William Berman, Yiyi Xu, Steven Liu, and Thomas Wolf. Diffusers: State-of-the-ar diffusion models. https:/ /ttt:///gthble com/ muggingfaces di fuseers, 2022.
Jiannan Wang, Jiarui Fang, Aoyu Li, and PengCheng Yang. Pipefusion: Displaced patch pipeline parallelism for inference of diffusion transformer models. arXiv preprint arXiv:2405. 200:22405.1440, 2024.
Philip Wolfe. The secant method for simultancous nonlinear equations. Communications of the ACM, 2212):12-13, 1959.
Junde Wu, Rao Fu, Huihui Fang. Yu Zhang, Yehui Yang, Haoyi Xiong, Huiying Liu, and Yanwu Xu. Medsegdiff: Medical image segmentation with diffusion probabilistic model. In Medical Imaging with Deep Learning, pp. 1623-1639. PMLR, 2024.
Run Yang, Yuling Yang, Fan Zhou, and Qiang Sun. Directional diffusion models for graph repre- sentation learning. Advances in Neural Information Processing Systems, 36, 2024.
Meng You, Zhiyu Zhu, Hui Liu, and Junhui Hou. Nys-solver: Video diffusion model as zero-shot novel view synthesizer. International Conference on Learning Representations, 2025.
Fisher Yu, Ari Seff, Yinda Zhang, Shuran Song, Thomas Funkhouser, and Jianxiong Xiao. Lsun: Construction of large-scale image dataset using deep learning with humans in the loop. arXiv preprint 1 03365, 2015.
Jean Yu and Haim Barad. Step saver: Predicting minimum denoising steps for diffusion model image generation. arXiv preprint arXiv:2408 02054, 2024.
Hongkai Zheng, Weili Nie, Arash Vahdat, Kamyar Azizzadenesheli, and Anima Anandkumar. Fast sampling diffusion models via operator learning. In International conference on machine learn- ing, pp. 42390-42402 PMLR, 2023.
13

---

Published as a conference paper at ICLR 2025
CONTENTS
Proof of Proposition
14
Proof of Proposition
15
Proof of Proposition
16
D Proof of Proposition
17
E Proof of Lemma
19
F Visual Comparison for Different Methods
22
Visual Comparison for Different Subintervals Number N
22
Visual Comparison for Different Preconditioning steps M
22
A PROOF OF PROPOSITION
Firstly, straightforward to confirm that the analytical expression p(fn+1, tn. Y.,) possesses unique solution. Now, based on the principle of proof by contradiction, suppose that there exist two distinct solutions A '.. Aiy and B B,,.".. B.y so they satisfy:
A +ftt+ti,ns As,)) Bint = *1(n++1, tny B:n))
(18)
Assume by induction that for 0 & nnn N. Ar, B,.,, so this yields
At 6(bnt1,bn Ar..) 11In+1,bn Bi..))
(19)
Thus, the sets of solutions A A.. Aty and B B...... By are the identical.
Secondly, it is easy to confirm that the solutions unbiased estimation of the sequential sampling results. In particular, according to Eq. (4) and Eq. (3), the the sequential sampling results can be expressed as
14

---

Published as a conference paper at ICLR 2025
in =x0
p(X,, S o,dW
I'
(t,n+1,0,X))
titl
X + Σ
(0 T) + Σ
o,
i=
1=0
(20)
n
Netz
X
(0(X; r)
0,dW
1-0-1
i-0 4
Σ
ptt++1, t4, X1,)
Therefore, we have
E (1 =
Σ E x + Σ
r)]dT
Σ
d,d
(21)
For our method, we have
E[A = (l bn Ar,)]
Mal1
Elp(Ar, + S o,dW
dall
E[A +
(22)
1411 Blp(Ar,T))dt +
Σ f4t
E[A,, +
o,dW
Recall that At, and Xo follow the distribution. Therefore, according Eq. (21) and Eq. (22), it is easy to see that E| Ar E[X.,,] because of the linear nature of expectation.
B PROOF OF PROPOSITION 2
Before proceeding with the proof, we need state several assumptions and definitions. Following the Banach fixed-point theorem Banach (1922), we first make the following assumptions. Assumption 1.
The denoising network So (X,,,)) 1) is continuous and differentiable. Holding i. fixed and letting 0 the optimal model 0², the mapping operator Sga (:,L) is contraction, iee, there exists a constant y L, such that:
||Sga (U,1) Sga (V,0) < V NWC
(23)
Remark: It is sensible consider the optimal denoising network Sow (X,, as contraction map- ping, as it can convert data points from a large space into a single dense data space that adheres to a standard Gaussian distribution, thereby effectively compressing the data points in the larger space.
For simplicity of expression, we denote Gn 24 Xi.) and dτ. log ai. (one half of the 2x.
log-SNR), Recall that define Xt, and XLy as the random noise and clean image respectively. Be- sides the forward process p( (X,,,, [xy) N (ar,, X.y: Jt, I) where a and a are the noise schedule. Then it is easy to know that Xo, is strictly decreasing function of t.
Following Proposition 3.1 in Lu al. (2022), we can obtain the exact solution of diffusion SDEs, i.e.
(t,, X.)
01 "x -
> Sox (Xx.))dd
/ o(A)dW.
(24)
as
15

---

Published as a conference paper at ICLR 2025
Now conduct the proof. Recall that we have the residual function:
Re, 1,41 (In++1, b X
(25)
Since Xt. is linear for Ri,,- is sufficiently smooth. We need to affirm whether the nonlinear part (((n+1, lug Xx..) sufficiently smooth. It is well known that if the normal of Jacobian matrix of (la++1, In, xx,) is bounded by small number, then the function is smooth enough. In light of this, we seek to find the upper bound of the norm of the Jacobian matrix Gn
do.
2Xin
According to the definition of gradient, we get:
[ tn Xt, + h) ((n++1, Eas Xx,))
lim h-
(26)
|h|l
The equation holds with being on the principle direction of singular value decomposition of g. Here we donote such subset of h as h'. Then, we have
[ tni X.,, + h²) D(nn+1, lns X.n>Il
G = lim h-=00
(27)
h'll
Utilizing Eq. (24) yields:
+ (tn++i, tni X + h²) D(tn+1, X.,.) )1l
0t h' atbut den
dent
e e>1 (X,, + h A) Spx (Xx, A)]c1
Qta
Aenti
0ta+1
h'
+ atati
e- Syp (Xx h A) Spa (X2, 2)
Otn
den
(28)
Aenni
0ta+1
h'
+ at H|1bi|| Nen
ed
Otn
0ia++1
+ ot, 11 7 h
((
Aln 1)
Oth.
atat1 9tn
Oftut1
+ ot, 1 h'|((
log
otn 1).
Ott,
Taking Eq. (28) into Eq. (27), we obtain the upper bound of the gradient as follows:
at Log ae atg
| &
at1t1
Olaes
Ple
1)
(29)
at
x 0ta41 Otuti 7(
ala Onn
1).
(30)
at,
The relationship between the Jacobian norm and the noise schedule parameters a and clearly illustrated in Eq. (29). Given that a and a in DPMs are strictly monotonically decreasing and increasing with respect to I, respectively, we can conclude that the ratios 0ta+1 and dia are nearly dte. Tt
equal to 1, as their rates of change with respect to the time step are minimal. As a result, |G,l is constrained to small value close to indicating highly smooth function Φ bay Xt,)) which implies that the residual R, is adequately smooth.
C PROOF OF PROPOSITION 3
According to Definition 1, the specific form the residual term for our banded NEs system as follows:
(0) X 4(tn+1,tn
(k)
x
(k+1)
ta
-11
(31)
16

---

Published as a conference paper at ICLR 2025
Using this specific form, we can express the Newton update (J R
in a particular manner
do (R)
(K) X
X
to
X X(K
$(11, 10: x (K1)
04 (x) ((
x (k)
(b09
2 (1, X/11
ex
00(tN, & (8)
x *(tN. -N- 1, X
I
2Nix-1
(32)
Rearranging Eq. (32) by multiplying through by the Jacobian and taking into the Eq. (9), we derive fundamental recurrence used for our parallel iteration at iteration
to x110 Xto ~(O.II) X (k+1)
((++1) $(Int1, ln x(1)) +
20 1n,
(x(++)) {((1))
(33)
ex(6)
D PROOF OF PROPOSITION 4
Assumption 2. The residual Reght is Li-Liscchizz continuous with respect to input X
(X..."". NtN-.), satisfying:
[R
Ri (V)Il L-|UU VII, V ((N4))HWC
(34)
Assumption 3. Suppose that the Hessian matrix HX) of the residual Righ with respect to input X is bounded, that is, there is constant L2 such that
[X(X)]| < L2, VX B(N+))XHWC
(35)
Remark: Proposition demonstrates that the residual RegHt sufficiently and possesses a very small upper bound on the Jacobian of the nonlinear part D(tn, tn-1, Xt,,). suggesting that L1 and L2 small real numbers. This insight will aid in analyzing the convergence behavior of the proposed update rule Eq. (11).
For the sake of simplicity, we denote Gn 2404,, Lstn Xind R(X) Regtt (X). Therefore, 2x.,, according to the Jacobian in Eq. (32), the Jacobian matrix for R(X) has the following convenient form:
Go
J
(36)
GN I
Since we use identity matrix to substitute In we thus have an approximated matrix for J, denoted as,
I
-
I
J =
(37)
17

---

Published as a conference paper at ICLR 2025
Proof of descent direction. Now, we first prove that the approximated Jacobian matrix I is descent direction for R(X), which is not contradictory to the descent direction of the true Jacobian matrix J. Formally, to complete the proof, suffices to show their inner product satisfies that:
(J.J) 0, vX (N
(38)
According to the relation between the trace and Frobenius inner product of the matrix, we have
(J,J) tr(JT J)
I+ 9o
-0
-I
I 1 9
=tr
-
I g NN
- G
(39)
N
N-1
Σ (r(1) Σ (r(9n)
(4) (N + 1)HWC HWC
N
|
Here, the incquation (a) uses the Lemma 1. According to Proposition 2, we know |G]] is upper bounded by a number close to 1. Therefore, Eq. (40) becomes
(J.J) > 1)HWC- NVHWC 0
(40)
Proof of convergence speed. In this part, will prove that the convergence speed of the update rule in Eq. (13) falls between linear and quadratic convergence. According to Taylor's 's theorem, any function that has a continuous second derivative can be represented by an expansion about a point that is close to root of this function. In view of this, we suppose a root of the residual function RXX) is A. Then the expansion of R(A) on point X (k) at parallel iteration A: is:
1 (A (A- - x() H())A - (()),
R(A) R(X(1) JT (*)
(41)
2
where the Lagrange form of the Taylor series expansion remainder denoted as R in which & is in between (A) and A. And H.)) is the Hessian matrix with respect to (k), Since A is the root,
Eq. (41) becomes:
= (() (A x()) R.
(42)
Multiply both sides by the inverse of JT and rearranging gives:
[JT]-'R(X (k)) A (k) -ITT--R
(43)
Remembering that the update rule in Eq. (12) uses J to approximate J. yielding:
x (k+1) (4) -
(44)
x [ ( + {15T]-1 - --13((((())
Combining Eq. (44) with Eq. (43), we find:
= A X (k+1) + ([57]-1- I --()
(45)
18

---

Published as a conference paper at ICLR 2025
Denote by g(h) A x(1), Eq. (45) becomes:
((++1)
J- -- ( (H(E)(
(46)
Taking the norm of both sides gives:
[e(l++ 1
- -J1 |
-- H(
1
1 1 --1
R ((X(()) (8))
( H(E)
+
2
[ J - 3-1
(R(A) R(X
1 [J | LLe (2) HH())
(47)
11 1J 1 8-1
(1)
1
[J
2
L1
1
(R)
L2
(1)
2
2
Mi
M2
here (b) holds since is the root of R (x); (c) relies on Assumption 2; (d) is derived from the properties of matrix transpose and norm, as well as Assumption 3.
Since the identity matrix approximation provides descent direction and the residual function is smooth enough, it indicates that convergence possible, implying that M1 and My are relatively small. Therefore, the above Eq. (47) describes the rate at which the sequence ( (g(k) (k) converges to the root A. Since the inequality contains both linear and quadratic terms, can conclude that the convergence rate of the sequence X (k) is at least linear but influenced by the quadratic term. This means that the convergence rate of the sequence X faster than linear but slower than quadratic. The exact order depends on the specific values of M1 and M2:
If Mj is small and M2 non-zero, then the quadratic term will dominate, and the sequence will converge at a quadratic rate.
If Mj is non-zero and M2 is small zero, then the linear term will dominate, and the sequence will converge at linear rate.
Since this study mainly concentrates parallel sampling for DPMs, offering specific values for M and detailed convergence speeds falls outside our current scope. We provide this initial insight for general understanding and will explore the specific relationship between M, and M2 in future work.
E PROOF OF LEMMA 1
Lemma
For any matrix E R", the relationship between the trace
tr(A) and the Frobenius norm A|| satisfies the following inequation:
[r((A) & | Vn
Proof:
To show the relationship |tr(A) & | F TT, we can start with the definitions of both
concepts.
Trace of a matrix A:
tr(A) =
ag
i=1
where dg the diagonal elements of the matrix A.
19

---

Published as a conference paper at ICLR 2025
2. Frobenius Norm of a matrix A:
|||l
azg
To show the relationship [tr(A)] < ||]]| vn, we can use the Cauchy-Schwarz inequality.
Using the Cauchy-Schward inequality in the context of sums, we have:
Σ
aui
Σ
1
Σ |asa
Step Simplify the Terms.
The first term simplifies to
===n
Thus, rewrite the inequality
Σ lail
<
aa 2
The Frobenius norm can be expressed as:
11
n
|A
2
Σ
agg
includes the sum of the squares of elements the matrix A, hence:
AA
aus 2
Combining these results, we have:
2
Σ Jaal
<
Taking the square root on both sides gives:
laul A
20

---

Published as a conference paper at ICLR 2025
Thus, conclude:
[tr(A]] =
0g
laxl & A
This completes the proof:
[tr(A) &
&|A||
F. Vn
21

---

Published as a conference paper at ICLR 2025
F VISUAL COMPARISON FOR DIFFERENT METHODS
This section shows the visual comparisons of when our ParaSolver and ParaDiGMS are applied to speed up DDPM, DDIM, and DPMSolver on Stable Diffusion v2. The results are shown in Figure 2, Figure 3, and Figure We can see that our ParaSolver significantly outperforms the competitors, with faster speed to generate better image.
(a) iteration
(b) 3 iteration
(c) iteration
(d) 7 iteration
The intermediate generated images for expediting DDPM with 100 sequential steps on Stable Diffusion Model v2. The images in the first row are produced by DDPM. The images in second row are created by ParaDiGMS. The images in the third row are generated by ParaSolver. For our ParaSolver, we set the number of subintervals as 100 and the preconditioning steps as
Figure 2:
G VISUAL. COMPARISON FOR DIFFERENT SUBINTERVALS NUMBER N
This section presents visual comparisons illustrating the application of our ParaSolver to enhance the speed of DDIM and DPMSolver Stable Diffusion v2 when using different N. The outcomes are depicted in Figure 5, Figure 6. It is clear that our ParaSolver performs better with smaller values of N, leading to enhanced output quality. For instance, Figure at N 20. the image of the cute corgi at iteration 3 appears natural without any noise, whereas at N = 100, some noise is noticeable this stage. This observation supports our claim that system of nonlinear equations with N 100 variables is more prone to accumulating errors during refinement compared to system with only N 20 variables.
H
VISUAL COMPARISON FOR DIFFERENT PRECONDITIONING STEPS M
This portion delves into the impact of the preconditioning steps M on the visual outcomes produced by ParaSolver, which is utilized accelerate DDIM with 100 sequential steps across the Stable
22

---

Published as a conference paper at ICLR 2025
(a) iteration
(b) 3 iteration
(c) iteration
(d) 7 iteration
Figure 3: The intermediate generated images for expediting DDIM with 100 sequential steps on Stable Diffusion Model v2. The images in the first row are produced by DDIM. The images in the second row are created by ParaDiGMS. The images in the third are generated by ParaSolver. For our ParaSolver, we set the number of subintervals as 40 and the preconditioning steps as 2.
23

---

Published as a conference paper at ICLR 2025
(a) iteration
(b) 3 iteration
(c) iteration
(d) 7 iteration
Figure The intermediate generated images for expediting DPMSolver with 100 sequential steps on Stable Diffusion Model v2. The images in the first row are produced by DPMSolver. The images in the second row are created by ParaDiGMS. The images in the third row are generated by ParaSolver. For our ParaSolver, we set the number of subintervals as 40 and the preconditioning steps as 2.
24

---

Published as a conference paper at ICLR 2025
(a) iteration
(b) 3 iteration
(c) iteration
(d) 7 iteration
Figure 5: The effect of the number of the subintevals N on the generated images when expediting DDIM with 100 sequential steps over Stable Diffusion Model v2. The images in the first row produced by ParaSolver with N = 20. The images in the second row are created by ParaSolver with N 40. The images in the third row are generated by ParaSolver with N 60. The images in the fourth row are created by ParaSolver with N = 80. The images in the fifth row are generated by ParaSolver with N 100. For our ParaSolver, we set the preconditioning steps as
25

---

Published as a conference paper at ICLR 2025
(a) iteration
(b) 3 iteration
(c) iteration
(d) 7 iteration
Figure 6: The effect of the number of the subintevals N' on the generated images when expediting DPMSolver with 100 sequential steps over Stable Diffusion Model v2. The images the first row are produced by ParaSolver with N = 20. The images in the second row are created by ParaSolver with N 40. The images in the third row are generated by ParaSolver with N 60. The images in the fourth row are created by ParaSolver with N 80. The images in the fifth row are generated by ParaSolver with N 100. For our ParaSolver, we set the preconditioning steps as
26

---

Published as a conference paper at ICLR 2025
Diffusion Model. The outcomes depicted in Figure 7. An intuitive observation emerges: elevat- ing the number of preconditioning steps notably boosts the sample quality. Notably, even with just one parallel iteration, the resulting image appears impressive.
12
11
7
0
2
4
(a) iteration
(b) 3 iteration
(c) iteration
(d) 7 iteration
Figure 7: The effect of the preconditioning steps M on the generated images when expediting DDIM with 100 sequential steps over Stable Diffusion Model v2. The images in the first row are produced by ParaSolver with M 0. The images in the second row are created by ParaSolver with M 1. The images the third row are generated by ParaSolver with M 5. The images in the fourth row are created by ParaSolver with M 10. The images in the fifth row are generated by ParaSolver with 15. For our ParaSolver, we set N 40.
27


Reviews:
[{"review_id": "wZPZpftdAZ", "reviewer": "Reviewer_totX", "summary": "The authors present an approach to accelerating the inference of diffusion probabilistic models (DPMs). They transform the problem of sequential sampling of DPMs into one of solving banded nonlinear equations. The Jacobian of the nonlinear system, required by Newton's method for rootfinding (aka Newton-Raphson) is unit block-lower-banded (1 on the diagonal, bands below), allowing for efficient parallel solution through a simple recurrence relation. The authors also present an initialization procedure that accelerates convergence. Finally, they combine this framework with a sliding window technique to conduct parallel iterations only a subset of the points. The combined approach is then evaluated on StableDiffusion-v2 and the LSUN Church pixel-space diffusion model, and demonstrates large speedups on inference without a loss in visual quality.", "questions": "1. See the Weaknesses section above. These must be addressed.\n\n2. The language in the paper hinders the presentation occasionally. For instance, the second paragraph of the related work section (Section 2) was challenging to read, primarily due to strange use of passive voice. There are similar issues throughout the paper. I suggest reframing to active voice wherever possible to improve clarity.\n\n3. Section 4.2, below equation (9): What is the \"reverse of Jacobian matrix\"? Do the authors mean the inverse? \n\n4. The authors separately explore tolerance and speedup in the results. I'd like to know which tolerance leads to the best speedup without compromising visual results. The authors should add a new graph with this extra information.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "There has been a surge of recent interest in fast parallel sampling of diffusion models. The state of the art for parallel sampling, to the best of my knowledge, appears to use Picard iterations to solve the nonlinear system of equations. The authors of this work make a few important contributions, all of which serve to accelerate convergence: (1) they use Newton's rootfinding method, which converges quadratically to the root for smooth enough functions; (2) they leverage the banded structure of the Jacobian to accelerate their solver; (3) they come up with a good initialization for Newton so it in fact converges; (4) they batch their parallel sampling and denoising so that it only happens within a sliding window.", "weaknesses": "1. Newton's method for rootfinding converges rapidly only if the function one is rootfinding on is sufficiently smooth. The authors should discuss the smoothness properties of the nonlinear system and how it impacts the convergence of the Newton solver, and also comment on the theoretical guarantees and limitations of their approach in this context.\n\n2. If the nonlinear residual for the nonlinear system has a complicated landscape, Newton can easily get stuck. The state of the art in optimization is to either use trust-region Newton methods or use quasi-Newton. The authors skirt around this issue altogether and count on their results and experiments to drive their point home. It would be useful to see the loss landscape as a function of, say, two of the most \"important\" unknowns (determined for instance by PCA) or the eigenvalues of the kernel matrix of the neural tangent kernel to determine if Newton is the right choice for this problem. Alternatively, if the authors could justify why these failure modes don't occur in PDMs, that would also suffice.\n\n3. How are Equation 12 and 13 justified? If the Jacobian term in the paragraph below Equation 11 is expensive to compute, why not approximate it? Newton's convergence rate requires at least an estimate for the Jacobian. Using the identity matrix instead effectively reverts Newton to a first-order method. Did the authors experiment with alternatives? Please provide theoretical/empirical justification for using the identity matrix approximation and discuss any experiments you conducted with alternatives.\n\n4. Rootfinding can be inherently unstable. Did the authors investigate other alternatives, such as optimization-based methods? Why did the authors choose one over another?\n\n5. This is minor, but I would've picked a less generic name for the paper. \"ParaSolver\" could imply a large number of things, but this is mainly a Newton-based parallel solver for PDMs. Consider a name change.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "SCCxEgbEzG", "reviewer": "Reviewer_7Ggk", "summary": "The authors present an interesting extension of previous work for inference in DPMs. The general idea is to formulate the solution to the ODE or SDE not as a sequential integration, but instead look at it as solving a set of nonlinear equations, done either via fix-point iteration or utilizing root finding algorithms. While this class of approaches does not improve the computational effort per se, it can lead to reduced wall-clock time by using less evaluation points compared to what is necessary when sequentially integrating the differential equation.\n\nThe paper proposes a unified framework that encompasses previous approaches as extreme cases. This results in a set of banded nonlinear equations. One key insight of the authors is to realize and proof that the banded system posses a unique and unbiased solution. They then further utilize the Newton method of root finding to accelerate the fix-point iterations. For this, one needs to calculate the Jacobian matrix. This, in general, is computationally prohibitive. An approximation scheme is proposed, where only the diagonal of the Jacobian is used, and the off-diagonal terms are set to unity. This results in an only modest increase in function evaluations over a sequential solution, indicating in addition to the reduced wall-clock time only a small increase in computational cost.\n\nThe achieved scores are on par with previous methods. A sizeable speed up in terms of wall-clock time is achieved leading to a better user experience. This is done without an massive increase in computational cost.", "questions": "I have only a few minor comments:\n- Please improve Figure 1 by using higher resolution, adding axis labels, and using a consistent font and style with the other figures in the paper.\n- Figure 5: There is, to me, no discernible difference between the images for different N. Can the authors comment on this? Why do we see such a clear difference for DDPM, but not for DDIM? It would be good if the authors either (1) provide a quantitative analysis of the differences between results for different N, if they exist, or (2) explain why DDIM results are less sensitive to N compared to DDPM.\n- In the Table 1 & 2, the results are ordered as DDPM, DDIM, DPMSolver, whereas in the figures the order is DDIM, DPMSolver, DDPM. I would appreciate some reordering to make it consistent.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "I believe this paper is generally well written and makes an relevant contribution to the field.\n\nAll claims are well supported by experiments, and the analyses appear sound.", "weaknesses": "- I believe the differences to the established methods utilizing fixed-point iterations for DPMs and their advances such as utilizing the Anderson acceleration used in previous work could be made clearer. It is currently not clearly mentioned that ParaTAA utilizes an conceptually similar idea. Albeit of course the approaches are different, they share common ideas which do not become clear without reading the literature carefully. I would encourage the authors to authors to rework the related work section and mention the differences to the other works more clearly.\n- It would be interesting to see by how much the number of necessary iterations to reach the threshold decreases by utilizing the Newton method. I recommend the authors include an ablation study showing how the number of iterations and convergence are affected by (1) using the Newton method vs. fixed-point iteration, and (2) approximating vs. fully computing the Jacobian, on a toy problem.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "5xjfWyBznv", "reviewer": "Reviewer_eRWg", "summary": "This paper proposed a framework that generalizes the sequential sampling process of diffusion models as solving a system of banded nonlinear equations. Applying the Newton-Raphson method to solve the nonlinear equations then yields a corresponding parallel sampling algorithm for diffusion models. By utilizing the unit-diagonal structure of the banded nonlinear equations' Jacobian matrices, the authors further simplified the updating rules of the parallel algorithm. Extensive numerical experiments were also conducted to show that the ParaSolver algorithm proposed in this paper can indeed accelerate the inference time of diffusion models compared to existing implementations based on parallel sampling.", "questions": "1. The reviewer's main question about the design of the ParaSolver algorithm is the claim in lines 244-247 of the paper. Specifically, the authors proposed to approximate the Jacobian term $\\frac{\\partial}{\\partial \\hat{X}^{(k)}_{t_n}}\\Phi(t_{n+1}, t_n, \\hat{X}^{(k)}_{t_n})$ with the identity matrix in the original update rule (11). Could the authors discuss which specific parts in the cited papers on Jacobian-free backpropagation (lines 245-246) actually used similar techniques? Furthermore, would it be possible for the authors to provide some mathematical intuitions on why the identity matrix should work here? Is it possible to derive some error bounds via numerical analysis?\n\nReferences:\n\n[1] Tang, Z., Tang, J., Luo, H., Wang, F. and Chang, T.H., 2024, January. Accelerating parallel sampling of diffusion models. In Forty-first International Conference on Machine Learning.\n\n[2] Shih, A., Belkhale, S., Ermon, S., Sadigh, D. and Anari, N., 2024. Parallel sampling of diffusion models. Advances in Neural Information Processing Systems, 36.\n\n[3] Luhman, E. and Luhman, T., 2021. Knowledge distillation in iterative generative models for improved sampling speed. arXiv preprint arXiv:2101.02388.\n\n[4] Meng, C., Rombach, R., Gao, R., Kingma, D., Ermon, S., Ho, J. and Salimans, T., 2023. On distillation of guided diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 14297-14306).\n\n[5] Salimans, T. and Ho, J., 2022. Progressive distillation for fast sampling of diffusion models. arXiv preprint arXiv:2202.00512.\n\n\n[6] Xu, Y., Deng, M., Cheng, X., Tian, Y., Liu, Z. and Jaakkola, T., 2023. Restart sampling for improving generative processes. Advances in Neural Information Processing Systems, 36, pp.76806-76838.\n\n[7] Song, Y., Dhariwal, P., Chen, M. and Sutskever, I., 2023. Consistency models. arXiv preprint arXiv:2303.01469.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "1. This paper has provided a complete literature review of related work on accelerating diffusion models via parallel sampling. Also, both the theoretical and algorithmic results in this paper are presented in a relatively clear way to follow. \n\n2. A complete set of large-scale numerical experiments on the Imagenet and LSUN-Church datasets are included to justify the acceleration achieved by the proposed ParaSolver algorithm.", "weaknesses": "1. The authors mentioned in lines 366-367 that the ParaTAA algorithm proposed in [1] needs to be implemented for comparison as it has yet to be integrated into the Diffusers library. However, given that there are only a few empirical works on combining parallel sampling with diffusion models, the reviewer thinks it would be essential for the authors to implement ParaTAA and use it as one extra baseline. Moreover, it might also be necessary for the authors to compare ParaSolver with approaches that accelerate diffusion models from other aspects, such as knowledge distillation [3-5], restart sampling [6], and self-consistency [7]. Furthermore, the authors should consider releasing the code used for implementing the ParaSolver algorithm. \n\n2. There are some minor issues regarding the presentation of the paper. For instance, the phrase \"to fast construct a set of more precise initial values that conform to the Definition 1\" in lines 296-297 doesn't seem quite right. It can be possibly rephrased as \"to construct a set of more precise initial values that conform to the Definition 1 quickly\". Moreover, the authors might also consider adding a few figures to illustrate the ParaSolver algorithm more vividly, just as what has been done in previous work [2].", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)