Accelerating Guided Diffusion Sampling with Splitting Numerical Methods
Authors: Suttisak Wizadwongsa
Supasorn Suwajanakorn
Venue: ICLR 2023 poster (2023)
Keywords: Splitting Numerical Methods
Guided Diffusion Models

Abstract: Guided diffusion is a technique for conditioning the output of a diffusion model at sampling time without retraining the network for each specific task. However, one drawback of diffusion models, whether they are guided or unguided, is their slow sampling process. 
Recent techniques can accelerate unguided sampling by applying high-order numerical methods to the sampling process when viewed as differential equations. On the contrary, we discover that the same techniques do not work for guided sampling, and little has been explored about its acceleration. This paper explores the culprit of this problem and provides a solution based on operator splitting methods, motivated by our key finding that classical high-order numerical methods are unsuitable for the conditional function. Our proposed method can re-utilize the high-order methods for guided sampling and can generate images with the same quality as a 250-step DDIM baseline using 32-58% less sampling time on ImageNet256. 
We also demonstrate usage on a wide variety of conditional generation tasks, such as text-to-image generation, colorization, inpainting, and super-resolution.

Published as a conference paper at ICLR 2023
ACCELERATING GUIDED DIFFUSION SAMPLING WITH SPLITTING NUMERICAL METHODS
Suttisak Wizadwongsa, Supasorn Suwajanakorn VISTEC, Thailand
{suttisak.w s19, supasorn.) @ @istec.accth
ABSTRACT
Guided diffusion is technique for conditioning the output of diffusion model at sampling time without retraining the network for each specific task. However, one drawback diffusion models, whether they are guided or unguided, is their slow sampling process. Recent techniques can accelerate unguided sampling by applying high-order numerical methods to the sampling process when viewed as differential equations. On the contrary, we discover that the same techniques do not work for guided sampling, and little has been explored about its accelera- tion. This paper explores the culprit of this problem and provides solution based on operator splitting methods, motivated by our key finding that classical high- order numerical methods are unsuitable for the conditional function. Our pro- posed method can re-utilize the high-order methods for guided sampling and can generate images with the same quality 250-step DDIM baseline using 32-58% less sampling time on ImageNet256. We also demonstrate usage on a wide vari- ety of conditional generation tasks, such as -o-aag- generation, colorization, inpainting, and super-resolution.
INTRODUCTION
A family of generative models known diffusion models has recently gained a lot of attention with state-of-the-ar image generation quality (Dhariwal & Nichol 202I). Guided diffusion is an approach for controlling the output of trained diffusion model for conditional generation tasks without retraining its network. By engineering task-specific conditional function and modifying only the sampling procedure, guided diffusion models can be used in variety of applications, such as class-conditional image generation (Dhariwal Nichol 2021 Kawar et al. 2022). ) text-to-image
generation (Nichol et al. 2022), image-to-image translation (Zhao 2022 inpainting (Chung
et al. 2022a). colorization Song et al. 2020b), image composition (Sasaii et al. 2021), adversarial
purification (Wang al. 2022 Wu et al. 2022) and super-resolution (Choi al.
2021
One common drawback both guided and regular "unguided" diffusion models their slow sam- pling processes, usually requiring hundreds of iterations produce single image. Recent speed- up attempts include improving the noise schedule (Nichol & Dhariwal, 2021 Watson et al.) 2021), redefining the diffusion process be non-Markovian, thereby allowing deterministic sampling process [onn et al. (2020a). network distillation that teaches a student model to simulate multiple sampling steps of teacher model [Salimans& & Ho (2022); Luhman & Luhman (202T). among oth- ers. Song et al. (2020a) show how each sampling step can be expressed as a first-order numerical step of an ordinary differential equation (ODE). Similarly, [Song et al. (2020b) express the sam- pling of score-based model solving a stochastic differential equation (SDE). By regarding the sampling process as an ODE/SDE, many high-order numerical methods have been suggested, such as Liu et al. (2022). Zhang & Chen (2022). and Zhang al. (2022) with impressive results guided diffusion models. However, when applied to guided diffusion models, these methods produce surprisingly poor results (see Figure [ -iveneen& a few number of steps, those high-order numerical methods actually perform worse than Tow-order methods.
Guided sampling differs from the unguided one by the addition of the gradients of the conditional function to its sampling equation. The observed performance decline thus suggests that classical high-order methods may not be suitable for the conditional function and, consequently, the guided

---

Published as a conference paper at ICLR 2023
Number of steps
16
32
64
128
256
DDIM
PLMS4
STSP4
(Ours)
Figure 1: Generated samples of classifier sisssiie--uuieedd diffusion model trained on ImageNer256 using 8-256 sampling steps from different sampling methods. Our technique, STSP4, produces high- quality results in fewer number of steps.
sampling equation whole. Our paper tests this hypothesis and presents approach to acceler- ating guided diffusion sampling. The key idea to use an operator splitting method to split the less well-behaved conditional function term from the standard diffusion term and solve them separately. This approach not only allows re-utilizing the successful high-order methods on the diffusion term but also provides us with options combine different specialized methods for each mize performance. Note that splitting methods have also been explored by Dockhorn et al. (2022) solve unguided diffusion SDEs, but our work focuses on accelerating guided diffussinn ODEs.
maxi-
Our design process includes comparing different splitting methods and numerical methods for each split term. When tested on ImageNet, our approach achieves the same level of image quality as DDIM baseline while reducing the sampling time by approximately 32-58%. Compared with other sampling methods using the same sampling time, our approach provides better image quality as measured by LPIPS, FID, and Perception/Recall. With only minimal modifications to the sampling equation, we also show successful acceleration on various conditional generation tasks.
BACKGROUND
This section provides a high-level summary of the thcoretical foundation of diffusion models as well numerical methods that have been used for diffusion models. Here we briefly explain few that contribute to our method.
2.1 DIFFUSION MODELS
Assuming that zo random variable from the data distribution we wish to reproduce, diffusion models define a sequence of Gaussian noise degradation of zo as random variables æ 22, ..., IT, where It ((11 81 3,I) and Be E [0, 1] are parameters that control the noise levels. With property of Gaussian distribution, we can express 2, directly as function of ao and noise e (00, I) by x = Vararo + v1 dtt,, where a = II (1 B.). By picking sufficiently large (e.g., 1,000) and an appropriate set of (3, we can assume xr is a standard Gaussian distribution.
The main idea of diffusion model generation is to sample Gaussian noise 2T and use it to reversely sample ZT ØT-2, until we obtain Zo. which belongs to our data distribution.
Ho et al. (2020) propose Denoising Diffusion Probabilistic Model (DDPM) and explain how to employ a neural network c0 t) to predict the noise e that is used to compute Et. To train the network, sample training image Zu, and compute I; using the above relationship. Then, we optimize our network 6o to minimize the difference between the predicted and real noise, i.c.,
||e (x 2)
2

---

Published as a conference paper at ICLR 2023
Song et al. (2220a) introduce Denoising Diffusion Implicit Model (DDIM), which uses the network Eg to deterministically obtain 2t-1 given Tt. The DDIM generative process can be written as
at-1 ā
(at v1 a,co(z,,))) + / V1-
(1)
This formulation could be used to skip many sampling steps and boost sampling speed. To turn this into ODE, we rewrite Equation Tas:
At
Pt
+
(1
Öt- At
Ot ā
col t),
A4-A!
va
which is now equivalent to numerical step in solving an ODE. To derive the corresponding ODE,
we can re-parameterize o; v1 - āe/ /a,, z(t) = x// 0 and Eg (ū) = c t), yielding
a(t At) =(t) (0r-At (T). By letting (Oi-At or) - the ODE becomes:
dē
E (T).
(3)
do
Note that this change variables is equivalent an exponential integrator technique described in both Zhang & Chen (2022) and [u et al. (2022). Since I₁ and F(t) have the same value at = O, our work can focus on solving æ(t) rather than It. Many numerical methods can be applied the ODE Equation to accelerate diffusion sampling. We next discuss some of them that are relevant.
2.2 NUMERICAL METHODS
Euler's Method the most basic numerical method. A forward Euler step is given by ēn+1 = In + A (Z,). When the forward Euler step is applied to the ODE Equation we obtain the DDIM formulation (Song et al., 2020a).
Heun's Method, also known as the trapezoid rule improved Euler, is given by: En+1 Σn + Ar (ei + e2), where = To (T,, and 62 To (T., + Aree). This method splits Euler's method into two steps to improve accuracy, Many papers have used this method on diffusion models, including Algorithm in Karras et al. (022) and DPM-Solver-2 in Lu et al. (2022). This method is also the simplest case Predictor-Corrector methods used Song et al. (2020b).
Runge-Kutta Methods represent class of numerical methods that integrate information from mul- tiple hidden steps and provide high accuracy results. Heun's 's method also belongs to a family of 2nd order Runge-Kute Kutta methods (RK2), The most well-known variant is the -rder Runge-Kutee Kutta method (RK4), which written as follows:
Ag
Aa In
(ān),
e2 Ea
In
+
El
E3 =
2 =
e4 = En (Jn + Aoee),
2
Fn + Ag
T Int1
(c1 202 + 2es ca).
(4)
This method has been tested on diffusion models in Liu (2022) and Salimans Ho 2022 but
it has not been used as the main proposed method in any paper.
Linear Multi-Step Method, similar to the Runge- Kutta methods, aims to combine information from several steps. However, rather than evaluating new hidden steps, this method uses the previous steps estimate the new step. The order formulation is the same as Euler's method. The 2nd order formulation is given by
Ao (3ea -e),
Tnt1 Fr. +
(5)
while the tth ooddr formulation is given by
Aa (55eg 59e1 + 37ez - -
Rntl = R, +
(6)
24
where é = E (#n-A)) These formulations are designed for a constant Ao in each step. However, experiments and previous work that uses this method (e.g., Liu al. (2022): Zhang & Chen
3

---

Published as a conference paper at ICLR 2023
2022 still show good results when this assumption is not strictly satisfied, i.., when Ao is not constant. We will refer to these formulations as PLMS (Pseudo Linear Multi-Step) for the rest of the paper, like in Liu et al. (022). A similar linear multi-step method for non-constant Ao can also be derived using a technique used in Zhang & Chen (2022), which we detail in Appendix B This non-constant version
improve upon PLMS slightly, but it is not flexible because have to re-derive the update rule every time the schedule changes.
3 SPLITTING METHODS FOR GUIDED DIFFUSION MODELS
This section introduces our technique that uses splitting numerical methods to accelerate guided diffusion sampling. We first focus investigation
mlassifier-guidee diffusion models for class- conditional generation and later demonstrate how this technique can be used for other conditional generation tasks in Section 4.3 Like any guided diffusion models, classifier-guider models (Dhari- wal & Nichol (02T) share the same training objective with regular unguided models with no mod- ifications the training procedure; but the sampling process is guided by an additional gradient signal from an external classifier to generate class-specific output images. Specifically, the sampling process is given by
ē= co(x) vI V.
(ogp)() V. 21-1 vā--1
ÜGÉ
/ 1 ---1 (7)
where Po (c((r))) is a classifier model trained to output the probability of xa belonging to class As discussed in the previous section, we can rewrite this formulation "guided ODE":
dæ = ( (z).
(8)
do
where fo(2) - log (p(c)))). We refer to Sa as the conditional function, which can be substituted with other functions for different tasks. After obtaining the ODE form, any numerical solver mentioned earlier be readily applied to accelerate the sampling process. However, observe that classical high-order numerical methods (e.g.. PLMS4, RK4) fail to accelerate this task (see Figure D and even perform worse than the bascline DDIM.
We hypothesize that the two terms in the guided ODE may have different numerical behaviors with the conditional term being less suitable classical high-order methods. We speculate that the difference could be partly attributed to how they are computed: (T) is computed through back- propagation, whereas Eo (z) is computed directly by evaluating network. One possible solution to handle terms with different behaviors is the so-called operator splitting method, which divides the problem into two subproblems:
dy do (v) do
de
=
We call these the diffusion and condition subproblems, respectively. This method allows separating the tond--t-approximat fz)) from {z(y) and solving them separately in each time step. Impor- tantly, this helps reintroduce the effective use of high-order methods on the diffusion subproblem as well as provides us with options to combine different specialized methods to maximize performance. We explore two most famous first- and second-order splitting techniques for our task:
3.1 HE-TROTTER SPLITTING (LTSP)
Our first example is the simple first-order Lie-Trotte Trotter splitting method (Trotter 1959 which presses the splitting as
dy (v): do dz - (2),
v(on) = En. 2(5n) = v(ant1).
E [ 1 Fn)
(10)
o E 10.-1.0m
(11)
do
with the solution this step being 01 = 2( Note that o, is a decreasing sequence. Here Equation i the same Equation 3 which can be solved using any high-order numerical method,
4

---

dz Published as a conference paper at ICLR 2023
Algorithm 2: Strang Splitting (STSP)
Algorithm 1: Trottrr Splitting (LTSP)
sample To (0, 0 max I):
sample En ~ (0, o max I):
for n E {0. .... N 1] do 2n+1 = Tn [p--1
for {0. ..... N- 1 do
Vs(Ta);
Vn+1 = PLMS [Zn nn. 0n+1, Eg); In+ ya+1 (0n+1 on)) V (Bn+11);
2
Un+1 = PLMS (2++1, PLMS(2n+11 On 1nt1: Eo):
Fn+1 Da+1 (0--1 00)) 2
(yn+1);
end Result: IN
end Result: N
e.g., PLMS. For Equation
11 we can use a forward Euler step:
2n+1 An Vo (Z.))
(12)
This is equivalent to single iteration of standard gradient descent with learning rate Ao. This splitting scheme is summarized by Algorithm T each subproblem in Section 4.1
We investigate different numerical methods for
3.2 STRANG SPLITTING (STSP)
Strang splitting (or Gtran--aacchul (Strang 1968) is one of the most famous and widely used operator splitting methods. This second-order splitting works follows:
fz),,
z(Oa) Inn
o
1 fn + ont1),
(13)
do
dy &o (v),
=2 u(ox)=z (2
v(on))
o E 1n
(14)
do
dž
(lon+ontt)) = y(on+1),
1 a Øntl. ((on + +
do (B),
(15)
Instead of solving each subproblem for full step length, solve the condition subproblem for half a step before and after solving the diffusion subproblem for full step. In theory, we can swap the order of operations without affecting convergence, but it is practically cheaper to compute the condition term twice rather than the diffusion term twice because fo is typically a smaller network compared &g. The Strange splitting algorithm shown Algorithm 2 This method can be shown to have better accuracy than the Lie-Trotter method, as proven in Appendix N Although it requires evaluating the condition term twice per step in exchange for improved image quality. We assess this trade-off in the experiment section.
4 EXPERIMENTS
Extending on our observation that classical high-order methods failed on guided sampling, we con- ducted series of experiments to investigate this problem and evaluate our solution. Section 4.1 uses a simple splitting method (first-order LTSP) to study the effects that high-order methods have on each subproblem, leading to our key finding that only the conditional subproblem is less suited to classical high-order methods. This section also determines the best combination of numerical meth- ods for the two subproblems under LTSP splitting. Section 4.2 explores improvements from using higher-order splitting method and compares our best scheme to previous work. Finally, Section 4.3 applies our approach a variety of conditional generation tasks with minimal changes.
For our comparison, we use pre-trained state-of- of-the-art diffusion models and classifiers from Dhari wal & Nichol (202T), which were trained on the ImageNet dataset (Russakovsky et al., 2015) with 1,000 total sampling steps. We treat full-path samples from classifier-guidee DDIM at 1 1,000 steps as reference solutions. Then, the performance of each configuration is measured by the image similarity between its generated samples using fewer steps and the reference DDIM samples, both starting from the same initial noise map. Given the same sampling time, we expect configurations with better performance to better match the full DDIM. We measure image similarity using Learned Perceptual Image Patch Similarity (LPIPS) (Zhang et al. 2018) (lower is better) and measure sam- pling time on single NVIDIA RTX 3090 and 24-core AMD Threadripper 3960x.
5

---

Published as a conference paper at ICLR 2023
Eulor (DOM))
3x10
PLM54
5x
PLMSS M555
10-1
1882.
[LM53]
15-1
(RK4.
PLMS3]
Edder (DOM)
[PLMSL PLMSS]
3-30
IPUMSL PLMSZI
00
IPLMSL PLMS4]
[PLMSL RK4]
10-7
10-7
5.0
12.5
7.5
15.0
175
22.5
Sampling
(a) Varying the method for the diffusion subproblem (b) Varying the method for the condition subproblem
Figure 2: Comparison of different combinations of numerical methods under LTSP splitting for guided diffusion sampling. We plot LPIPS against the sampling time. [A, B] denotes the use of method A in the diffusion subproblem and method B in the condition subproblem. The red dotted lines indicate a reference DDIM score obtained from 250 sampling steps, which produce images visually close to those from 1,000 steps.
4.1 FINDING A SUITABLE NUMERICAL METHOD FOR EACH SUBPROBLEM
To study the effects of different numerical methods on each subproblem of the guided ODE (Equa- the simplest Trotter splitting, which itself requires no additional network evalu- ations. This controlled experiment has two setups: a) we fix the numerical method for the condition subproblem (Equation 1T to first-order PLMS1 (Fuler's method) and vary the numerical method for the diffusion subproblem (Equation TO). and conversely b) we fix the method for the diffusion subproblem and vary the method for the condition subproblem. The numerical method options are Euler's 's method (PLMSI), Heun's method (RK2), 4th order KongeeKKuta' 's method (RK4), and 2d order pseudo linear multi-step (PLMS2/PLMS4) We report LPIPS vs. sampling time of various numerical combinations on diffusion model trained on ImageNet 256x 256 in Figure 2 The red dotted lines indicate reference DDIM score obtained from 250 sampling steps, common choice that produces good samples that are perceptually close to those from a full 1,000-step DDIM (Dhariwal Nichol 202T Nichol & Dhariwal 202T).
tion 8).
Given a long sampling time, non-split PLMS4 performs better than the DDIM baseline. However, when the sampling time is reduced, the image quality of PLMS4 rapidly decreases and becomes much worse than that of DDIM, especially under 15 seconds in Figure 2 When we split the ODE and solve both subproblems using first-order PLMS1 (Euler), the performance is close to that of DDIM, which is also considered first-order but without any splitting. This helps verify that merely splitting the ODE does not significantly alter the sampling speed.
In the setup a), when RK2 and RK4 used for the diffusion subproblem, they also perform worse than the DDIM baseline. This slowdown is caused by the additional evaluations of the network by these methods, which outweigh the improvement gained in each longer diffusion step. Note that if we instead measure the image quality with respect to the number of diffusion steps, RK2 and RK4 can outperform other methods (Appendix D): however, this is not our metric of interest. On the other hand, PLMS2 and PLMS4, which require no additional network evaluations, are about 80% faster than DDIM and can achieve the same LPIPS score the DDIM that uses 250 sampling steps in 20-26 fewer steps. Importantly, when the sampling time is reduced, their performance does not degrade rapidly like the non-split PLMS4 and remains at the same level as DDIM.
In the setup b) where we vary the numerical method for the condition subproblem, the result re- veals an interesting contrast-none of the methods beats DDIM and some even make the sampling diverged [PLMSI, RK4]. These findings suggest that the gradients of conditional functions are less "compatible" with classical high-order methods, especially when used with a small number of steps. This phenomenon may be related to the "stiffness" condition of ODEs, which discuss further in Section For the remainder of our experiments, we will use the combination [PLMS4, PLMSI] for the diffusion and condition subproblems, respectively.

---

Published as a conference paper at ICLR 2023
Buler (DDIM)
PLMS4
Sampling time within
RG4
(LMS))
sec.
10 sec.
15 sec.
20 sec.
10:1
1P8C
DDIM
0.116
0.062
0.043
0.033
PLMS4
0.278
0.141
0.057
0.026
RK2
0.193
0.059
0.036
0.028
RK4
0.216
0.054
0.039
0.028
LTSP4
0.121
0.058
0.037
0.028
10a
15.0
20.0
STSP4
0.079
0.035
0.022
0.013
Saimpling tinnc (soc.)
Figure 3: Comparison of different numerical methods for guided diffusion sampling.
Table 1: Average LPIPS when the sampling time is limited to be under 5 20 seconds.
4.2 IMPROVED SPLITTING METHOD
This experiment investigates improvements from using a high-order splitring method, specifically the Strang splitting method, with the numerical combination [PLMS4, PL.S1] and compares our methods to previous work. Note that besides DDIM Dhariwal & Nichol (202T) D no previous work is specifically designed for accelerating guided sampling, thus the baselines in this comparison are only adaptations of the core numerical methods used in those papers, And to our knowledge, no prior guided-diffusion work uses splitting numerical methods. Non-split numerical method baselines are PLMS4, which is used in Liu et al. (2022), RK2, which is used in Karras al. (2022): ; Lu et al.
(202), and higher-order RK4. We report the LPIPS scores of these methods with respect to the sampling time in Figure 3 and Table
Without any splitting, PLMS4, RK2 and RK4 show significantly poorer image quality when used with short sampling times < 10 seconds. The best performer is Strang splitting (STSP4), which can reach the same quality as 250-step DDIM while using 32-58% less sampling time. STSP4 also obtains the highest LPIPS scores for sample times of 5, 10, 15, and 20 seconds. More statistical details and comparison with other split combinations are in Appendix F
In addition, we perform a quantitative evaluation for class-conditional generation by sampling 50,000 images based on uniformly chosen class conditions with a small number of sampling steps and evaluating the Fenchel Inception Distance (FID) Heusel et al. (2017) (lower better) and the improved precision/recal| Kynkäänniemi et al. 2019) (higher is better) against the ImageNet test set 128, 256, and 512 resolutions. Following (Dhariwal Nichol| 202T), we use a 25-step DDIM as a baseline, which already produces visually reasonable results. As PLMS and LTSP require the same number of network evaluations as the DDIM, they are used also with 25 steps. For STSP with a slower evaluation time, it is only allowed 20 steps, which is the highest number of steps such that its sampling time is within that of the baseline 25-step DDIM. Here LTSP2 and STSP2 are Lie- vieTTrotter and Strang splitting methods with the combination [PLMS2, PLMS1J. In Table 2 results for three different ImageNet resolutions and the average sampling time per image in seconds.
we report the
Our STSP4 performs best all measurements except Recall ImageNet512. On ImageNet512 12, PLMS4 has the highest Recall score but a poor FID of 16, indicating that the generated images have good distribution coverage but may poorly represent the real distribution. On ImageNet256, STSP4 can yield 4.49 FID in 20 steps, compared to 4.59 FID in 250 steps originally reported in the paper (Dhariwal & Nichol 2021). Our STSP4 is about 9.4x faster when tested on the same machine.
4.3 SPLITTING METHODS IN OTHER TASKS
Besides class-conditional generation, approach can also accelerate any conditional image gen- cration as long as the gradient of the conditional function can be defined. We test our approach on four tasks: text-to-image generation, image inpainting, colorization, and super-resolution.
Text-to-image generation: We use a pre-trained text-to-image Sisoo-Difusiio (Letts et al. 2021) based on Crowson 2202I), which substitutes the classifier output with the dot product of the image and caption encodings from CLIP (Radford et al.) (021). For more related experiments on Stable- Diffusion (Rombach et al. 2022), please refer to Appendix I. M
7

---

Published as a conference paper at ICLR 2023
Method
Steps
Time
FID
Pree
Rec
Method
Steps
Time
FID
Prec
Rec
ImageNet256
ImageNet1288
DDIM
25
1.99
5.47
0,80
0,47
DDIM
25
0.55
6.69
0.78
0.49
PLMS4
25
2.05
4.71
0.82
0.49
PLMS2
25
0.57
5.71
0.80
0.51
STSP4
20
1.95
4.49
0.83
0.50
PLMS4
25
0.57
4.97
0.80
0.53
ADM-G
250
20.9*
4.59
0.82
0.50
LTSP2
25
0.55
5.14
0.81
0.51
LTSP4
25
0.55
3.85
0.81
0.54
ImageNet512
STSP2
0.54
5.33
0.80
0.52
DDIM
25
5.56
9.07
0.81
0.42
STSP4
20
0.54
3.78
0.81
0.54
PLMS4
25
5.78
16.00
0.75
0.51
ADM-G
250
5.59
2.97
0.78
0.59
STSP4
20
5.13
8.24
0.83
0.45
ADM-G
250
56.2*
7.72
0.87
0.42
Table 2: Comparison of different numerical methods using few steps on guided diffusion sampling. Our methods and the best scores highlighted in bold. We provide the reported scores from Dhariwal & Nichol (202T) using 250 sampling steps, referred to as ADM-G in their paper. *ADM- G's sampling times are measured using our machine.
"A beautiful painting singular lighthouse, shining light tumultuous of blood, trending artstation."
"A beautiful painting starty night, sunflower sea, trending artstation."
Full DDIM (1,000 steps)
DDIM (45 steps)
PLMS4 (45 steps)
LTSP4 (45 steps)
STSP4 (30 steps)
(approximately using the same sampling time)
Figure 4: Text-to-image generation using different sampling methods.
Image inpainting & colorization: For these two tasks, we follow the techniques proposed in Song et al. (2020b) and Chung al.) (2022a), which improves the conditional functions of both tasks with "manifold constraints." We use the same diffusion model Dhariwal & Nichol 2021 trained on
ImageNet as our earlier Experiments 4.1 4.2
Super-resolution: We follow the formulation from ILVR (Choi et al. 2021 combined with the manifold contraints Chung et al. (202aa), and also use our earlier ImageNet diffusion model.
Figure compares our techniques, LTSP4 and STSP4, with the DDIM baseline and PLMS4 on text-to-image generation. Each result produced using fixed sampling time of about 26 seconds. STSP4, which uses 30 diffusion steps compared to 45 in the other methods, produces more realistic results with color contrast that more similar the full DDIM references'. Figure 5 shows that our STSP4 produces more convincing results than the DDIM baseline with fewer artifacts on the other three tasks while using the same second sampling time. Implementation details, quantitative evaluations, and more results are in Appendix K
DISCUSSION
Our findings show that when the sampling ODE consists multiple terms from different networks, their numerical behaviors can be different and treating them separately can be more optimal. Another promising direction to improve the behavior of the gradient of the conditional function classifier itself and study whether related properties such adversarial robustness or gradient smoothness induce the desirable temporal smoothness in the sampling ODE. However, it is not yet clear what specific characteristics of the behavior play important role. This challenge may be related
8

---

Published as a conference paper at ICLR 2023
Original
Input
STSP4 (Ours)
DDIM
Inpainting
Colorization
8xS SxSuper- Resolution
Figure 5: -uifed-diffusion results of our STSP4 and DDIM inpainting, colorization, and super- resolution. Both methods were limited to use approximately the same sampling time.
condition called "stiffness" in solving ODEs Ernst & Gerhard (2010), which lacks clear definition but describes the situation where explicit numerical methods, such as RK or PLMS, require a very small step size even in regions with smooth curvature.
propose a classifier-ff free model that can perform conditional generation without classifier while remaining generative model. This model can utilize high-order methods as no classifier is involved, but it requires evalu- ating the classifier-free network twice per step, which typically more expensive than evaluating normal diffusion model and classifier. It is important to note that our accelerating technique and classifier-free models are not mutually exclusive, and one can still apply a conditional function and our splitting technique to guide classifier-fres model in a direction it has not been trained for.
As an alternative to the classifier-guide model, Ho & Salimans 2021
While our paper only focuses on ODEs derived from the deterministic sampling of DDIM, one can convert SDE-based diffusion models to ODEs (Karras et al.) (022) and still use our technique. More broadly, we can accelerate any diffusion model that can expressed as differential equation with summation of two terms. When these terms behave differently, the benefit from splitting can be substantial. Nevertheless, our findings are based on common, existing models and a schedule from Dhariwal & Nichol (2021). Further investigation into the impact of the schedule or different types and architectures diffusion models is still required.
6 CONCLUSION
In this paper, we investigate the failure to accelerate guided diffusion sampling of classical high- order numerical methods and propose solution based splitting numerical methods. We found that the gradients of conditional functions are less suitable classical high-order numerical meth- ods and design a technique based on Strang splitting and combination of forth- and first-order numerical methods. Our method achieves better LPIPS and FID scores than previous work given the same sampling time and is 32-58% faster than a 250-step DDIM baseline. Our technique can successfully accelerate a variety of tasks, such as text-to-imee -ittt----maee generation, inpainting, colorization, and super-resolution.

---

Published as a conference paper at ICLR 2023
REFERENCES
Jooyoung Choi, Sungwon Kim, Yonghyun Jeong, Youngjune Gwon, and Sungroh Yoon. ILVR: Conditioning method for denoising diffusion probabilistic models. In 2021 IEEE/CVE Interna- tional Conference on Computer Vision (ICCV), pp. 14347-14356. IEEE, 2021.
Hyungjin Chung, Byeongsu Sim, Dohoon Ryu, and Jong Chul Ye. Improving diffusion models for inverse problems using manifold constraints. In Advances in Neural Information Processing Systems, 2022a.
Hyungjin Chung, Byeongsu Sim, and Jong Chul Ye. Dlone-Co--DDffuse-eeeee Accelerating conditional diffusion models for inverse problems through stochastic contraction. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 12413-12422, 2022b.
2021. Katherine Crowson. CLIP guided diffusion 512x512, secondary model method. https://
twitter
Prafulla Dhariwal and Alexander Nichol. Diffusion models beat GANs on image synthesis. Ad- vances in Neural Information Processing Systems, 34:8780-8777 8794, 2021.
Tim Dockhorn, Arash Vahdat, and Karsten Kreis. Score-based generative modeling with critically- damped langevin diffusion. In International Conference on Learning Representations (ICLR), 2022.
Hairer Ernst and Wanner Gerhard. Solving Ordinary Differential Equations. 2: Stiff and Differential Algebraic Problems, volume 2. Springer, 2010.
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. GANs trained by a two time-scale update rule converge to a local nash equilibrium. Advances in neural information processing systems, 30, 2017.
Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. In NearlPS 202/ Workshop on Deep Generative Models and Downstream Applications, 2021.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. In Pro- ceedings of the 34th International Conference on Neural Information Processing Systems, pp- 6840-6851, 2020.
Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion- based generative models. In NeurlPS 2022 Workshop on Deep Generative Models and Down- stream Applications, 2022.
Bahjat Kawar, Roy Ganz, and Michael Elad. Enhancing diffusion- baaod image synthesis with robust classifier guidance. arXiv preprint wrXiv:2208.08666 2022.
Tuomas Kynkäänniemi, Tero Karras, Samuli Laine, Jaakko Lehtinen, and Timo Aila. Improved precision and recall metric for assessing generative models. Advances Neural Information Processing Systems, 32, 2019.
Adam Letts, Chris Scalf, Alex Spirin, Tom Mason, Chris Allen, Max Ingham, Mike Howles, Nate Baer, and David Sin. Disco diffusion. https://gittub.com com/ aatoomaalmbba alembics/
disco-diffusion 2021.
Wei Li, Xue Xu, Xinyan Xiao, Jiachen Liu, Hu Yang, Guohao Li. Zhanpeng Wang, Zhifan Feng, Qiaoqiao She, Yajuan Lyu, et al. Upainting: Unified text-to-image diffusion generation with cross-modal guidance. arXiv preprint arXiv:2210. 16031, 2022.
Luping Liu, Yi Ren, Zhijie Lin, and Zhou Zhao. Pseudo numerical methods for diffusion models on manifolds. In International Conference on Learning Representationss 2022.
Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu. DPM-Solver: A fast ode solver for diffusion probabilistic model sampling in around 10 steps. In Advances in Neural Information Processing Systems, 2022.
10

---

Published as a conference paper at ICLR 2023
Eric Luhman and Troy Luhman. Knowledge distillation in iterative generative models for improved sampling speed. arXiv preprint arXiv:2101.02:: 023388 2021.
Chenlin Meng, Yang Song. Jiaming Song, Jiajun Wu, Jun- Yan Zhu, and Stefano Ermon. SDEdit:
Guided image synthesis and editing with stochastic differential equations. In International Con- ference on Learning Representations, 2021.
Alexander Quinn Nichol and Prafulla Dhariwal. Improved denoising diffusson probabilistic models. In International Conference on Machine Learning, pp. 3162-8171, PMLR, 2021.
Alexander Quinn Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob Megrew, Ilya Sutskever, and Mark Chen. GLIDE: Towards photorealistic image generation and editing with text-guided diffusion models. In International Conference Machine Learning, pp- 16784-16804. PMLR, 2022.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, al. Learning transferable visual models from natural language supervision. In International Conference on Machine Learning. pp. 8748-8763. PMLR, 2021.
Robin Rombach, Andreas Blattmann, Dominík Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Con- ference Computer Vision and Pullern Recognition, pp. 10684-10695, 2022.
Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfir Aberman. DreamBooth: Fine tuning text-to-image diffusion models for subject-driven generation. arXiv preprint arxiv:2208. 12242, 2022.
Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma. Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al. Imagenet large scale visual recognition challenge. International journal of computer vision, 115(3):211-25) 2015.
Tim Salimans and Jonathan Ho. Progressive distillation for fast sampling of diffusion models. In International Conference on Learning Representations, 2022.
Hiroshi Sasaki, Chris G Willcocks, and Toby P Breckon. Unit-DDPM: Unpaired image translation
with denoising diffusion probabilistic models. arXiv preprint arXiv:2104.05338, 2021.
Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. In Interna-
tional Conference on Learning Representations, 2020a.
Yang Song, Jascha Sohl-Dickstein, Diederik Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In Interna- tional Conference on Learning Representattons, 2020b.
Gilbert Strang. On the construction and comparison of difference schemes. SIAM journal on nu- merical analysis, 5(3):506-517, 1968.
Hale F Trotter. On the product of semi-groups of operators. Proceedings of the American Mathe- matical Society, 10(4):545-551, 1959.
Jinyi Wang, Zhaoyang Lyu, Dahua Lin, Bo Dai, and Hongfei Fu. Guided diffusion model for adversarial purification. arXiv preprint arXiv:2205. 149.999
14969, 2022.
Daniel Watson, Jonathan Ho, Mohammad Norouzi, and William Chan. Learning efficiently sam- ple from diffusion probabilistic models. arXiv preprint arXiv:2 /06i::2/06.00002... 2021.
Quanlin Wu, Hang Ye, and Yuntian Gu. Guided diffusion model for adversarial purification from random noise. arXiv preprint arXiv:2206. 10875, 2022.
Qinsheng Zhang and Yongxin Chen. Fast sampling of diffusion models with exponential integrator. In NewrlPS 2022 Workshop on Score-Based Methods, 2022.
Qinsheng Zhang, Molei Tao, and Yongxin Chen. gDDIM: Generalized denoising diffusion implicit models. arXiv preprint 22Xiv:2206005664 2022.

---

Published as a conference paper at ICLR 2023
Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness deep features as a perceptual metric. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 586-595, 2018.
Min Zhao, Fan Bao, Chongxuan Li, and Jun Zhu. EGSDE: Unpaired image-to-image translation via energy-guided stochastic differential equations. In Advances in Neural Information Processing Systems, 2022.
12


Reviews:
[{"review_id": "jl45xQlGQaU", "reviewer": "Reviewer_CQRF", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper focuses on accelerating guided diffusion sampling, where the backward ODE to be solved consists of two parts: the first one from the diffusion contribution and the second one from the classifier contribution. It is empirically found that non-splitting high-order ODE solvers does not work well when the number of time steps is small. To address the above issue, the authors consider applying splitting methods (LTSP 1959 and Strange splitting 1968) for solving the backward ODE, where the two parts in the ODE are treated separately and sequentially.  ", "summary_of_the_review": "The paper tried out splitting methods for solving the backward ODE in guided diffusion sampling and obtained better performance. No theoretical justification is provided to motivate the splitting methods over non-splitting methods.  ", "strength_and_weaknesses": "Strength: \n1. The paper, for the first time, applies existing splitting methods (LTSP 1959 and Strange splitting 1968) for solving the backward ODE in guided diffusion sampling and obtain promising sampling results, which I think is interesting and novel.\n2. The literature over existing high order methods seems up-to-date.   \n\nWeaknesses: \nIt is unclear from a theoretical point of view why non-splitting high-order methods do not work well. There may exist other high-order methods that work which yet to be discovered. \n\n\n\n \n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "7Z784BF4Bd", "reviewer": "Reviewer_PHsB", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper considers the problem of accelerating the sampling process of the classifier-guided diffusion models. The classifier-guided diffusion consists of two components: the conditional function (gradient of classifier) and the standard diffusion term. The authors observe that high-order numerical solvers can lead to degraded performance when sampling with the conditional function. To alleviate the issue while speed up the sampling, they opt to use higher-order method for standard diffusion term and forward Euler method for conditional function. They combine the two interleaved procedures through operator splitting methods. Experimentally, the proposed sampling method outperforms DDIM in speed and sample quality.", "summary_of_the_review": "The reviewer leans toward rejection due to the lack of justification of the proposed method, and the seemingly low practical value of the considered (classifier-guidance) problem.", "strength_and_weaknesses": "### Strong points\n\n- The paper applies the operator splitting method to the classifier-guided generation. They study various combinations of solvers of the individual parts, and the best combination outperforms the baseline DDIM.\n\n- The proposed method consistently improved over the baselines across image resolutions and tasks.\n\n\n\n### Weak points\n\n- The current dominating method is classifier-free generation, such as stable diffusion. The problem of classifier-guided diffusion models seems to have lower practical value. As the authors mention in Section 5, classifier-free involves more expensive function evaluation per step. However, they allow higher-order solvers for all components. Could the authors compare the FID/sampling time of the two methods, in order to demonstrate the utility of classifier-guided generation. (I understand it could be hard to complete the experiments in the short rebuttal time, but it really boosts the utility of classifier-guided generation if it has certain advantages.)\n\n- The \"stiffness\" reasoning of the condition function is vague and insufficient. The phenomenon is the major point in the paper, and it would be very helpful to dive deeper into this problem: why does the condition function perform better when paired with a simpler ODE solver in practice? It's also a bit counter-intuitive when the function is stiff for higher-order solvers but not for simpler ones. Could the authors give some illustrative examples? The review has one plausible hypothesis: the gradient of classifier (condition function) has large variance or changes rapidly across different $\\sigma$. Hence, it's harmful to combine the evaluations along the ODE trajectory.\n\n- The evaluation protocol in Section 4.1 is a bit problematic. The generated samples are compared against DDIM w/ 1000 steps. It assumes that the samples generated by DDIM w/ 1000 steps are true samples. As we see in hindsight, there are many samplers better than DDIM, and they could generate better samples while incurring high LPIPS. Also, in this setup, the goal is to purely decrease the ODE simulation error (there is nothing to do with network estimation error). It's counter-intuitive that higher-order methods do not work out under such a metric.\n\n\n### Writing suggestions\n\nBelow are some points that could potentially make the paper more readable and consistent:\n\n- Section 3.1 & 3.2: the $\\sigma$_s is increasing over time. But normally, they should decrease over time during sampling. \n- Section 4, page 5, \"with 1000 total steps\" - do you mean \"with 1000 total sampling steps\"? \n- Section 4.1: It would be helpful to change PLMS1 to Euler's method.\n\n\n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "5QullzSEOR", "reviewer": "Reviewer_3tU3", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes an acceleration method of guided diffusion sampling based on splitting numerical methods. Based on the finding that the high-order numerical methods are unsuitable for the conditional function, it develops a method based on Strang splitting and a combination of fourth and first-order numerical methods. Experimental results show that the proposed can accelerate the guided diffusion sampling.", "summary_of_the_review": "Although the experimental results are good, I expect the authors could provide more theoretical analysis about the proposed method.", "strength_and_weaknesses": "Strength:\n\nThe paper proposes a simple method that accelerates the guided diffusion sampling. The proposed method is efficient and can be applied to other problems, e.g., super-resolution, colorization, for acceleration. Experimental analysis shows the effectiveness of the proposed method.\n\nWeaknesses:\n\nI mainly concern the limited theoretical analysis about the proposed method.\n\nThe paper claims that the high-order numerical methods are unsuitable for the conditional function. However, it does not explain this finding clearly. It would be better to provide a theoretical analysis of this finding.\n\nThe paper claims that the Strange splitting algorithm can be proved to have better accuracy. However, there is no theoretical analysis.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "3Ci9uyv0lk", "reviewer": "Reviewer_sMbW", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper analyzes classifier-guided sampling of diffusion models and observes that applying higher-order methods for accelerated sampling from the model does not work well in the guidance scenario. The root cause for that is that the additional guidance term defined by the classifier makes the generative ODE harder to solve. Explicit higher-order integrators do not seem to be suitable for the resulting stiff classifier-guided ODE. Based on that observation, the work proposes to split the ODE into two separate terms, the diffusion model's score term and the classifier term. These are then solved separately with different integrators and afterwards combined, using numerical splitting methods. The paper shows improved performance of classifier-guided diffusion model sampling over some selected baselines when using a limited number of synthesis steps.", "summary_of_the_review": "In summary, this paper proposes some interesting and sensible ideas to improve classifier-guided diffusion model sampling. However, the paper is lacking on multiple fronts (see weaknesses above). (a) Baseline comparisons are insufficient. (b) The proposed splitting methods lack theoretical discussion and are only very superficially discussed in the paper. (c) The significance is somewhat limited because only classifier guidance, but not classifier-free guidance is covered.\n\nTherefore, overall I do not think that this paper is ready for publication in its current form. Most importantly, I would suggest the authors to include more thorough experiments and a more in-depth analysis and discussion of the proposed splitting integration methods.", "strength_and_weaknesses": "**Strengths:**\n\n- The idea to split the diffusion model's main score and classifier term in the generative ODE into separate parts, and solving them separately, is novel and certainly seems like a good and sensible idea.\n\n- The analyses that are performed in Figure 2 are interesting and insightful. They nicely show that the additional classifier term is the problem that makes higher-order methods break down in the classifier-guidance setting.\n\n- The experimental results on classifier-guided diffusion model sampling support the value of the proposed method.\n\n**Weaknesses:**\n\n- Most state-of-the-art conditional diffusion models these days rely on classifier-free guidance for conditional sampling. Unfortunately, the proposed methodology only applies to classifier guidance. This reduces the significance of the proposed method.\n\n- The baseline comparisons are insufficient. Comparisons to the recent state-of-the-art solvers DEIS [1] and DPM solver [2] are missing. Furthermore, for the experiments on inpainting, colorization and super-resolution, only qualitative comparisons are presented. Also, only DDIM is considered there for comparison.\n\n- The paper lacks a proper theoretical analyses of the proposed methods. Convergence, order and stability of the Lie-Trotter Splitting and Strang Splitting techniques are nowhere discussed. It is insufficient to just write sentences like \"This method can be proved to have better accuracy\". Moreover, I believe the deep learning community is not deeply familiar with these splitting schemes, so a much more thorough discussion and analysis is needed.\n\n- The statement that \"no prior diffusion work uses splitting numerical methods\" is incorrect. Critically-damped Langevin Diffusion [3] uses Strang Splitting to develop a sampler for their setting (see their Section 3.3). This work is not cited or discussed at all.\n\n\n[1] Zhang and Chen, Fast Sampling of Diffusion Models with Exponential Integrator, 2022\n\n[2] Lu et al., DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps, 2022\n\n[3] Dockhorn et al., Score-Based Generative Modeling with Critically-Damped Langevin Diffusion, 2021", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Decision: Accept: poster