BELM: Bidirectional Explicit Linear Multi-step Sampler for Exact Inversion in Diffusion Models
Authors: Fangyikang Wang
Hubery Yin
Yue-Jiang Dong
Huminhao Zhu
Chao Zhang
Hanbin Zhao
Hui Qian
Chen Li
Venue: NeurIPS 2024 poster (2024)
Keywords: diffusion model; exact inversion; ODE sampler
Primary area: diffusion_based_models
TL;DR: We propose a novel generic eacxt inversion samplers of diffusion models and derive the variant with optimal accuracy .

Abstract: The inversion of diffusion model sampling, which aims to find the corresponding initial noise of a sample, plays a critical role in various tasks.
Recently, several heuristic exact inversion samplers have been proposed to address the inexact inversion issue in a training-free manner. 
However, the theoretical properties of these heuristic samplers remain unknown and they often exhibit mediocre sampling quality.
In this paper, we introduce a generic formulation, \emph{Bidirectional Explicit Linear Multi-step} (BELM) samplers, of the exact inversion samplers, which includes all previously proposed heuristic exact inversion samplers as special cases.
The BELM formulation is derived from the variable-stepsize-variable-formula linear multi-step method via integrating a bidirectional explicit constraint. We highlight this bidirectional explicit constraint is the key of mathematically exact inversion.
We systematically investigate the Local Truncation Error (LTE) within the BELM framework and show that the existing heuristic designs of exact inversion samplers yield sub-optimal LTE.
Consequently, we propose the Optimal BELM (O-BELM) sampler through the LTE minimization approach.
We conduct additional analysis to substantiate the theoretical stability and global convergence property of the proposed optimal sampler.
Comprehensive experiments demonstrate our O-BELM sampler establishes the exact inversion property while achieving high-quality sampling.
Additional experiments in image editing and image interpolation highlight the extensive  potential of applying O-BELM in varying applications.

BELM: Bidirectional Explicit Linear Multi-step Sampler for Exact Inversion in Diffusion Models
Fangyikang Wang!* Hubery Yin
Yuejiang Dong
Huminhao Zhu' Chen Li
Chao Zhang't Hanbin Zhao' Hui Qian'
'Zhejiang University Weehat, Tencent Inc. Tsinghua University
(wangfangyikang zhuhuminhao zczju ju, zhaohanbin, qianauu @zju. edu, Clubery, chasell lillllllaa 11)@@inceettt com dongyj 21 210malls.... tsinghua.edis edu. cn
Abstract
The inversion of diffusion model sampling, which aims to find the corresponding initial noise of sample, plays critical role in various tasks. Recently, several heuristic exact inversion samplers have been proposed to address the inexact inversion issue in training-free manner. However, the theoretical properties these heuristic samplers remain unknown and they often exhibit mediocre sampling quality. In this paper, introduce generic formulation, Bidirectional Explicit Linear Multi-step (BELM) samplers, the exact inversion samplers, which includes all previously proposed heuristic exact inversion samplers as special cases. The BELM formulation is derived from the sitiabbl-tttsize-varaal---- linear multi-step method via integrating bidirectional explicit constraint. We highlight this bidirectional explicit constraint is the key of mathematically exact inversion. We systematically investigate the Local Truncation Error (LTE) within the BELM framework and show that the existing heuristic designs of exact inversion samplers yield sub-optimal LTE. Consequently, we propose the Optimal BELM (O- BELM) sampler through the LTE minimization approach. We conduct additional analysis to substantiate the theoretical stability and global convergence property of the proposed optimal sampler. Comprehensive experiments demonstrate our O-BELM sampler establishes the exact inversion property while achieving high- quality sampling. Additional experiments in image editing and image interpolation highlight the extensive potential of applying O-BELM in varying applications.
Introduction
The emerging diffusion models (DMs) [52 20, 55 (6]. generating samples of data distribution from
initial noise by learning reverse diffusion process, have been proven to be an effective technique
for modeling data distribution, especially generating high-quality images [44] [0 50
46
48 21
The diffusion process along with its sampling processes in DMs can be delineated as the forward and
corresponding backward stochastic differential equations (SDE) [56] M. Furthermore, the sampling deterministic diffusion ordinary differential equation (ODE)
process also be represented [3). which is also called Probability Flow ODE (PF-ODE) in some papers. Notably, the backward SDE and diffusion ODE share the same marginal distribution(56).
The inversion of the diffusion sampling, which aims to elucidate the correspondences between samples and initial noise, plays critical role in various tasks of DMs. The diffusion inversion has
"qqual contribution. This work done when Fangyikang Wang was intern at WeChat. Corresponding author.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

DDIM inversion
BELM inversion
D -
T;-1
11+1
EB
11
1,
EB (
Eg (ii
DDIM
BELM
Figure 1: Schematic description of DDIM (left) and BELM (right). DDIM uses x; and Ex i) calculate X;-1 based on a linear relation between X;, X and Eœ xx,,) i) (represented by the blue line). However, DDIM inversion uses x;-1 and 6 linear relation represented by the red line. This mismatch leads to the inexact inversion of DDIM. In contrast, BELM seeks to establish a linear relation between x, X,, X,+1 and ex i) (represented by the green line). BELM and its inversion are derived from this unitary relation, which facilitates the exact inversion. Specifically, BELM uses the linear combination of x,, X,+1 and ( i) to calculate x;-1, and the BELM inversion uses the linear combination of x;-1, X; and E (x; to calculate X;+1. The bidirectional explicit constraint means this linear relation does not include the derivatives at the bidirectional endpoint, that is, c0(x,,1,? 1) and E0(X,+1, 1).
1) to calculate x; based on a different
a variety of downstream applications, including image editing 18 57]. image interpolation [53], inpainting 77), and super-resolution [67]. Several studies [3] 30 7 have endeavored to tackle the inversion task within the context of SDE-based diffusion sampling. However, these works have not been able to achieve a mathematically exact inversion due to the inherent stochasticity of SDE.
In contrast, the diffusion ODE naturally gives out correspondence between samples and noise. The famous DDIM [53] and its inversion are formulated by considering first-order explicit Euler discretization to the diffusion ODE. However, as noted in the work of [18] the DDIM inversion introduces an inconsistency problem due to the schematic mismatch between DDIM and its inversion (see Figure [. Encoding from Zo to IT using DDIM inversion and then decoding using DDIM often leads to inexact reconstructions of the original samples (see Figure (4. To enable exact inversion, the 42) introduces intensive training for itcrative optimization but still falls short of achieving mathematically exact inversion.
work of null-text inversion
Recently, several heuristic exact inversion samplers have been proposed address this inexact inversion issue in training- free manner [63] [1]. These samplers enable the mathematically exact inversion without the need for additional training and are thus compatible with pre-trained models. Taking inspiration from affine coupling layers in normalizing flows [1] 12), EDICT [63 intuitively introduces an auxiliary diffusion state and performs alternating mixture updates on the primal and auxiliary diffusion states. Later, BDIA [71] employs symmetric bidirectional integration structure to achieve exact inversion intuitively. However, these hcuristic exact inversion samplers often compromise the sampling quality due their intuitive formula design. They may also introduce undesirable extra computational overhead or non-robust hyperparameters.
In this paper, we develop a generic formula for the general exact inversion samplers, termed as Bidirectional Explicit Linear Multi-step (BELM) samplers. We demonstrate that all previously proposed heuristic exact inversion samplers are, in fact, special instances of BELM samplers. The concept of BELM originates from the observation of the mismatch between DDIM formula and its inversion formula. BELM is formulated by establishing an unifying relationship. from which both BELM and its inversion are derived. More specifically, the unifying relationship of BELM is constructed in a muriable-teessize-varidd wat---vaaaibbb-mulllll (VSVF) linear multi-step manner, supplemented with an additional bidirectional explicit constraint to facilitate exact inversion.
We systematically investigate the Local Truncation Error (LTE) within the BELM framework and show that the existing heuristic designs of exact inversion samplers yield sub-optimal LTE. Consequently, we employed a LTE minimization approach to design the formula of the optimal case within BELM, which we refer to as O-BELM. The formula for O-BELM dynamically adjusts in accordance with the timesteps, thereby ensuring minimized local error and consequently yielding the highest possible sampling accuracy. Furthermore, we demonstrate that O-BELM possesses the desirable property of zero-stability, which makes O-BELM robust to initial values. It also has the beneficial property of

---

global convergence, which prevents O-BELM from diverging during sampling. To the best of our knowledge, O-BELM is the first theoretically guaranteed exact inversion diffusion sampler.
We perform an image reconstruction experiment on the COCO dataset to validate that our O-BELM indeed achieves exact inversion, thereby enabling it to precisely recover complex image features. Furthermore, experiments involving both unconditional and conditional image generation demonstrate that O-BELM can ensure high sampling quality. Additional experiments in downstream tasks such as image editing and image interpolation highlight the extensive application potential of O-BELM.
2 Preliminaries
2.1 Diffusion Models and Diffusion SDEs
Suppose that we have a d-dimensional random variable x(0) E R following an unknown target distribution go (aro)- Diffusion Models (DMs) define forward process (x(())ejoo7) with T >
starting with x(0), such that the distribution of xt) conditioned on x(0) satisfies
gejo
(1)
where a()) C((0,7] RR+ have bounded derivatives, and denote them oz and o for simplicity. The choice for a₁ and o, is referred to as the noise schedule of a DM. According to
33 29 38 with some assumption on a(-) and a--), the forward process can be modeled as linear SDE which is also called OrnsteinUUhlebbed process:
dx(() (())x(t)dt++ g(()dB,
(2)
where B, the standard d-dimensional Brownian Motion (BM), f(t) dlogg a and g² dt
do? 2o au of. Under some regularity conditions, the above forward SDE have a reverse SDE di
from time H O, which starts from x(t) [
(x( = (ft))(()) g² (1) x(e) log (((()), t)] dt + g(t)dB,
33)
where B, is the reverse-time Brownian motion and ((((t), f) is the single-time marginal distribution
of the forward process. In practice, DMs (0 (56) use eg (x(()),, t) to estimate -o(() Vx(1) log ((x(t), and the parameter 0 is optimized by the following objective:
f arg min E, {A.Eurg [1))o( V., log fa ))
(4)
2.2 Diffusion ODE and DDIM
It is noted that the reverse SDE 3 has associated probability flow ODE (also called diffusion ODE), which a deterministic process that shares the same single-time marginal distribution [56]
dx(t) = [(t)x()) 2 (t) log ((x(t) t)
dt.
(5)
Upon substituting the f() and g(t) into Eq. (5) obtain the following first-order form:
x(t) Eo (x(t),
at (1))
(6)
az
The famous DDIM sampler [53] can be obtained by applying the explicit Euler method to Eq. (6)
X,
0:-1
(a-11-
0i-1
a; oi
Eo (x; i).
a;
The inversion of DDIM is obtained by applying the explicit Euler method in the reverse of Eq. (
aj X; 0;-1
as
X4-1 + oi
0 E E i 1).
(8)
a:-1
3

---

2.3 Intuitive Exact Inversion Samplers of Diffusion Models
In practice, we observe an inconsistency issue with the DDIM inversion (8). Consider a sample X using DDIM inversion, we obtain the corresponding noise X and then use DDIM to reconstruct xi- The reconstructed xj would exhibit significant inconsistency with the original sample X Recently, two exact inversion samplers, EDICT and BDIA, have been heuristically proposed to address this inconsistency issue in a training-frve manner.
EDICT sampler Taking inspiration from affine coupling layers in normalizing flows 12 the recent work [63] proposed EDICT to enforce exact diffusion inversion. The basic idea lies in introducing an auxiliary diffusion state Yt to be coupled with x Denoting a; and
b; 02-1 di- oi the formulation of EDICT writes:
inter 0;8; + bieo(y,, x
Yi inter o,y, (1t)(x(t) .i),
X;-1 px. inter + (1 p)y!
Yi-1 py!nter (1 p)x;-1
where p 1) is the mixing coefficient. The details of EDICT inversion defers to Appendix A.1
BDIA sampler BDIA sampler [7]] utilizes a symmetric bidirectional integration structure to achieve exact inversion. BDIA reformulate the expression of DDIM
7 to DDIM
x. DDIM + A (i + - 1|xPDIM) and the expression of DDIM inversion
8 to be x = x
(i- ( I|x.))M). BDIA intuitively leverage [[1 - (x,+1 x,) + 7 (i + i + 1|x;)] to
approximate the increment from 2+++1 to 2; and A (i + 1 (x.) as the increment from dy to #:-1 Thus, the updating rule of BDIA writes:
[(1 (x+11 x:) 7 (i + + + A A(i+i-1|x;
(10)
(ncrement(xx.4) +
increment (x. -
The comprehensive formulation of BDIA and inversion can be found in Appendix A.2
However, the theoretical properties these heuristic samplers remain unknown and they often exhibit compromised sampling quality. To the best of our knowledge, there is systematic approach to derive a diffusion sampler that simultaneously possesses the exact diffusion inversion property and maintains high sampling quality.
The Generic Bidirectional Explicit Linear Multi-step (BELM) Samplers
In this section, we first model the diffusion sampling process as a well-posed initial value problem to facilitate subsequent analysis. By the rethinking of DDIM inversion, propose the generic Bidirectional Explicit Linear Multi-step (BELM) samplers in a satplesstessiie-vaaaal---fooe (VSVF) manner. We further illustrate that EDICT and BDIA are, in fact, special instances of the BELM framework.
The diffusion sampling problem as an IVP By denoting x(t) x(t) (t) = ae and
(x(()), o.) = εe (x(t), t), the deterministic sampling process of DMs (6) can seen as an special reverse-time diffusion initial value problem (IVP) 188 p.310 p.3]:
dx(() Eu (x(t), o.) dō,,
(11)
where x(T)= = /(T)/ar. A fundamental question before any further analysis whether given diffusion IVP I admits any solution and, if so, whether this solution is unique. Firstly, need to establish some regularity assumptions on our diffusion sampling problem (6).
Assumption 1. Ex(x, t) is continuous w.r.t. t and Lipschitz continuous w.F.f. x with the Lipschitz constant Tago which implies [) (x22, 1)]l2 S T.ep ll X2l12-

---

The Assumption I is a common assumption of the noise predictor Eg(x. t) in the DMs literature [54]. Under the condition of Assumption[] we can confirm the diffusion IVP TT] is well-posed by a direct application of the existence and uniqueness theorem in the IVP theory 3 p. 23].
Proposition 1. Under Assumption
0 there exists a unique solution to the diffusion IVP
In this paper, x(-) denote the continuous solution, and x; denote numerical approximations.
Rethinking on DDIM inversion As shown in Figure I DDIM 7 and its inversion (8) are derived based on different linear relationships. We highlight that this mismatch results in the inexact inversion of DDIM. Building on this observation, natural idea is to construct the DDIM inversion based on the same linear relationships as the DDIM eliminate this mismatch. Regrettably, DDIM is constructed on a relationship between X, X and Ee (x;,,,)) a) (utilizes x,, and Ee ( i) to compute x,-1). which DDIM inversion cannot leverage to directly calculate X;, as En (x,, 1) is also unknown in the DDIM inversion case. This relation is explicit for DDIM but implicit for DDIM inversion. It should be noted
that implicit equations must be solved using iterative methods such as Newton's method [58 p. 19].
which are time- cons-connuming and can introduce numerical error in the context of DMs 23 39
To address this issue, we establish new relationship between adjacent states and derivatives, which can be explicitly computed in both directions. Subsequently, we formulate both the sampler and its inversion based this singular linear relationship achieve exact inversion. This is the fundamental concept of BELM samplers.
Bidirectional Explicit Linear Multi-step (BELM) samplers In attempt establish linear relationship between X,, X;-1, Ea (x:, i), and e ( 1) that can be explicitly computed bidi- rectionally, we must exclude both (x;,, i) and (x 1). However, this exclusion results in a relationship that lacks sufficient information. Consequently, it becomes imperative to take more states into account. This prompts us to explore the concept of the linear multi-step (LM) method [3] P.111] as a means to derive a linear relationship between adjacent states and the derivatives of the diffusion IVP. However, the commonly used noise schedule of DMs would lead to a non-equidistant N. So, instead of the classical LM methods with fixed stepsize, we shall seaelll-ttesize--ariiiee farmuale-fooouula (VSVF) manner (8, which use dynamic mul-
series of {ā:). i ....N. consider in the tistep formulae w.r.t. different stepsizes. Let to < t ... = tq + T be a grid in [to, to T h; o. 0, N... ho og and h max h4, the k-step VSVF LM methods w.r.t. Eq. will calculate x,-1 at the points J,-1 with the following difference equation:
k
X Σ 1=1
OL 11114
+
b h:-1+
Eo Eo(xx:1-1) 5,-1+)).
(12)
1=0
where the coefficient of updates and stepsizes are all dependent on Throughout this paper, any reference LM will, by default, imply VSVF LM unless explicitly stated otherwise. If 61,0 = for all i in Eq. (12). the method is called explicit, since the formula can directly compute X;-1. Clearly, the LM (1) have reversed formula which is also k-step LM as follows (assume ai j 0),
Oi.i
bi.j. -4-1+ E0(S)-14),
X:
+
(13)
ai.k
0g
If the reversed VSVFM explicit, i.e. byR O for all i, we call the origin LM 12 to be backward explicit. Now we can define a k-step LM to be bidirectional explicit when it is explicit as well backward explicit. We call the LM samplers abide by the bidirectional explicit constraint as the Bidirectional Explicit Linear Multi-step (BELM) samplers, which have the general form:
X
Σ
X 8 + Σ buj h4-145 (X 0+-14))-
(14)
We highlight this bidirectional explicit constraint is key to mathematically exact diffusion inversion:
Proposition 2. Any BELM method (14 with a, # has the exact inversion property.
5

---

Table 1: Theoretical properties comparison of different samplers.
Theoretical properties
exact inversion
local error
zero-stable
global convergence
DDIM 53
(o,hi")
EDICT 63
(yai-iht)) (ar (h+41))))
unclear unclear
unclear unclear
BDIA
O-BELM (Ours)
(a:(h: 1h44))
As an instance, setting k = 2 in Eq.
14 yields the 2-step BELM diffusion sampler:
(15) For detailed information on the 3-step BELM diffusion sampler, the general k-step case, and their optimal design, readers referred to Appendix A.4 and A.5 In the main body of this paper, we will default mean 2-step case unless explicitly stated.
x x 0,,1 (x,,
BDIA and EDICT as special case of BELM We find that, although developed from heuristic ideas, both BDIA and EDICT are special cases within the BELM framework. That is, their exact inversion property is inherited from the fact that they are fundamentally instances of BELM samplers.
Remark 1. EDICT p and BDIA TO are both special cases within the BELM framework.
The detailed mathematical derivation for Remark can be found in Appendices A.7 and A.8
The Optimal-BELM (O-BELM) Sampler
In this section, we systematically investigate the Local Truncation Error (LTE) within the BELM framework and show that the existing heuristic designs of exact inversion samplers yield sub-optimal LTE. Consequently, we introduce Optimal-Bai -ELM (O-BELM), which utilizes a more refined dynamic formula developed through the LTE minimization approach. Additional analysis is conducted to substantiate the theoretical stability and global convergence property of O-BELM.
4.1 Analysis on Local Truncation Error
The Local Truncation Error (LTE) quantifies the error introduced in a step update. Specifically, computes the difference between the numerical solution and its underlying true solution, assuming perfect knowledge of the true solution at the previous states.
Definition 1. The LTE of BELM (55) on X; at each step i is defined as
7 = x(1x - 2,,28 - a;,1%; -
(16)
Under Assumption 2 (details in Appendix A.3).
utilize the Taylor expansion to investigate the
LTE of BELM (15 as follows:
Proposition 3. Under Assumption 2 the LTE of the BELM 15 gives general form as follows:
-c; -0 + 64 (8(1)-1), 01 + C1,3 V Eo (0:1-));;:1)) + + ((h: : + hi4 1)
(17)
where C1,1 = 1 1-:;11 0g,
6222 a:1 04,2 (h: h441) bi and c1,3
04,1 h
2 (h; + 1++11112 b,, N.
In the task of DMs, our primary concern the LTE on x,-1 rather than x;-1. We denote the LTE on x; e, It is clear that e, 04-17%- We investigate the LTE of existing samplers as follows:

---

Corollary 1. Under Assumption 2 the LTE e; of DDIM sampler 7 is O (oi-1h,)); The LIE C₁ of BDIA sampler
110 is O a5-1(h; h(+1)²) for any fixed y E [0, 1); The LTE c₁
of EDICT sampler
9 is (vas-1hi) for any constant E (0,1).
4.2 Optimal BELM Sampler via LTE Minimization
We then demonstrate that, through meticulous design of formulae, of LTE within the BELM framework compared to existing sub-optimal instances. Specifically, we utilize an LTE minimization approach, inspired by the design of renowned LM methods such as the
achieve higher order
methods (2] or the Adams-Moos Moonn-Moulton methods 43 40
Proposition 4. Under Assumption 2
the LTE Tr of BELM diffussion sampler
15
can be
accurate up to O ( (he + het1))) when formulae are designed as a,,
h2
042
bi, F441
hi hothhe
h1-1
When this is satisfied, obviously, the LTE e, x,-1 is O (a:-1(h; h(+1)*) Substituting the designed formulas into (15), we derive the Optimal-BELL (O-BELM) sampler:
h2 0:-1
h h2 ai-1
he(hs + hitt)
0;-180(x(,1) (18)
X;-1 h R1 01+1
X
h2+1 as
h+t1
The inversion of O-BELM diffuusin sampler
18
writes:
h 0++1
h
h
1441(1e + hit1)
X;+1
h? Q4-1
X;-1
h2
a
x;
hy
034180(x;,1)
(19)
4.3 Further Theoretical Analysis on O-BELM
Here, we further demonstrate that the O-BELM not only surpasses in terms of local accuracy but also excels stability and global convergence properties.
As is clear from 15p these, the initial one is given by the initial condition, but the others, have to computed by other means, say, by using DDIM. At any rate, the starting values will contain numerical errors and it crucial ensure that perturbations of the initial values not lead to an error explosion in the subsequent steps. This concept is encapsulated in numerical analysis as zero-stability.
we need starting values before we can apply method to the diffusion IVP. Of
Definition 2. The LM (12) is said to be zero-stable if there exists constant K such that, for any two sequences { and {2:} that have been generated by the same formulae but different starting values NN XN -....... &N -++ 1+1 and NN; AN ... ZN -N-k+1. respectively, we have
2 S K max [|| - z | nN
|N- - ZN -N-k+1ll>, (20)
for all i, and as tends to
We also want to ensure that a method will gradually converge to the underlying truth as the stepsizes decrease, concept that aligns with the global convergence property.
Definition The LM 12 is globally convergent if for every solution x(1) of
lim max ||x;
x(t+) =0
(21)
h-10 DSiSN
when initial error ΣØ=N
N-1+ (|| - x(t,))l + h11 - Ex(x(t)), tends to zero.

---

Original Images
Edited Images by O-BELM
"a angry
*an gid!
makeug"
"Picasso portralt
"a man"
"... with glasses'
without beard"
with joker mask
happy man'
with hal"
"a car'
"a police car'
"a vintage car'
"a cabriolet car"
"a fire fighttng car"
"a car school bus"
'a farm"
*an airoraft on...
Kock panguins
lighthuse on
"the Iltanic on
"a waterfall in the mountains'
'impressionist paiting of
"van Gogh. paiting of
"wateroolor
paiting of ...
Figure 2: Examples of editing results using O-BELM on both synthesized and real images. We showcase the diverse editing capabilities of O-BELM across range of tasks, including human face modifications, content change, entity addition and global style transfer. The exact inversion property of O-BELM enables large-scale image alterations while preserving auxiliary details (background in first row, hairstyle in second row, traffic sign in third row, tree and crop in fourth row, composition in last row). Its stability and accuracy further ensure the high quality of the resulting images.
We affirm that our O-BELM sampler possesses the nice zero-stable property as well as the global convergence property.
Proposition 5. The O-BELM sampler 18)
(a) zero-stable and (b) globally convergent.
5
Experiments
In this section, we conduct experiments to verify that O-BELM achieves the exact inversion property while maintaining high-quality sampling ability. We further demonstrate the extensive potential of applying the O-BELM sampler in various applications, such as image editing and image interpolation (deferred to Appendix C.3). All the pre-trained models utilized are listed in Appendix C.5
5.1 Image Reconstruction
We adopt the experimental setting from 63 to demonstrate the exact diffusion inversion property of
O-BELM using 10k images in the MS-COCO-2014 validation set [35] Given an image, inverted
latents are caleulated and used to reconstruct the image using SD-1.5. Mean-square error (MSE)
calculated on pixels normalized to [-1, and averaged across 10k images. The autoencoder (AE)

---

Table Comparison of different samplers on MSE reconstruction loss on COCO-14.
MSE loss of reconstruction
DDIM
AE
EDICT
BDIA
O-BELM
10 steps
0.026
0.004
0.004
0.004
0.004
20 steps
0.016
0.004
0.004
0.004
0.004
50 steps
0.008
0.004
0.004
0.004
0.004
100 steps
0.007
0.004
0.004
0.004
0.004
Table 3: Comparison of different samplers on FID score( J) for the task of unconditional generation.
CIFARTO (32 x 32)
CelebA-HO (256 x 256)
DDIM
EDICT
BDIA
O-BELM
DDIM
EDICT
BDIA
O-BELM
10 steps
17.45
87.11
12.27
10.98
27.13
57.82
27.41
19.13
20 steps
10.60
38.84
7.27
7.17
16.33
39.24
16.18
11.54
50 steps
6.96
10.24
5.77
5.24
10.77
16.72
10.65
10.41
100 steps
5.72
5.31
5.07
4.18
10.19
12.24
10.30
10.17
reconstruction error in the SD pipeline serves as lower bound. From Table we observe that, regardless of the stepsize, O-BELM and its sub-optimal siblings BDIA and EDICT consistently achieve the lowest MSE, signifying their exact inversion the latent level. contrast, DDIM tends to suffer from inconsistency. More visual reconstruction examples can be found in Appendix
5.2 Unconditional Image Generation
In this section, conduct unconditional image generation task validate the high-quality sampling ability of O-BELM. Utilizing a pre-trained model, we generate 50k artificial images over specific number of steps and compute the corresponding Fréchet Inception Distance (FID) score with the real data. Specifically, Fréchet Inception Distance (FID) [19] calculates the Fréchet distance between the real data and the generated data. A lower FID implies more realistic generated data. Table summarizes the computed FID scores for the CIFARIO and CelebA-HQ datasets. It is evident that O-BELM consistently outperforms other exact inversion samplers in terms of sampling quality. This experimental result corroborates the error analysis presented in Table [] The parameters y for BDIA and p for EDICT are determined through grid search. Details can be found in Appendix C.2
5.3 Conditional Image Generation
We further evaluate these samplers under conditional image generation tasks. We employ the StableDiffusion V1.5 and V2-base models to generate 30k images of resolution 512x512, based on text prompts from the COCO-14 validation set. All methods utilize the same seed and the same text prompts set. As evident from Table 4 O-BELM also exhibits superior sampling quality the context of conditional image generation. We ensure fair comparison by selecting appropriate guidance weights and hyperparameters, details of which can be found Appendix
Original lmages
DDIM
EDICT
BDIA
O-BELM
"a river"
"a castle by river"
"a dog"
"a deg dalmatian"
Figure 3: Comparison of editing results from different samplers under 50 steps. DDIM leads to
inconsistencies (highlighted by the red rectangle), and the EDICT and BDIA samplers may introduce unrealistically low-quality sections (highlighted by the yellow rectangle). Our O-BELM sampler ensures consistency and demonstrates high-quality results.

---

Table Comparison of different samplers on FID score( 4) for the task of text-to-image generation with pretrained stable diffusion models.
SD-1.5 (512 x 512)
SD-2.0-base (512 x 512)
DDIM
EDICT
BDIA
O-BELM
DDIM
EDICT
BDIA
O-BELM
10 steps
21.44
85.77
23,96
18.19
20.40
75.14
22.00
17.01
20 steps
19.45
27.17
20.39
17.92
18.57
24.15
18.72
16.53
50 steps
18.93
21.30
19.38
17.96
17.82
19.76
17.98
16.52
100 steps
18.83
21.13
19.21
18.19
17.64
19.49
17.86
16.75
5.4 Training-free Image Editing
In this section, we present the results of the O-BELM sampler in an image editing task shown in Figure and compare the editing effects of different samplers in Figure We demonstrate that the exact inversion property of O-BELM ensures the preservation of image features that we do not wish to edit. Furthermore, we illustrate how the high accuracy and stability properties of O-BELM contribute the high quality of the edited image.
We emphasis that the goal of experiments here is not going to use our O-BELM sampler alone to achieve commercial-grade level image editing. It's quite unfair for graining-freee exact sampler methods to compete with commercial -rrade image editing pipelines involving donain-specific training [25 68 attention modification [18 451, testing-time time finetuning 622 [4 6 complex control 32 In fact, our O-BELM
73 real-data inversion alignment (75 or input text refinement
sampler is orthogonal to these image editing techniques, using better exact inversion sampler like
O-BELM in the commercial-gree grade image editing pipeline remains a promising future work.
Conclusions
We tackle the inexact inversion issue of DMs in a training-free manner. We introduce the generic Bidirectional Explicit Linear Multi-step (BELM) framework based on linear multi-step observation, which encompasses existing heuristic exact inversion samplers as special cases. Furthermore, we devise Local Truncation Error (LTE) minimization approach construct the Aptimal-BELM, (O-BELM) within the BELM framework, which achieves higher order of local error. We provide theoretical guarantee of global stability and convergence for O-BELM and conduct various ex- periments to demonstrate that O-BELM not only accomplishes exact inversion but also maintains high-quality sampling capability. Please refer to further discussion and limitations in appendix D The code repository can be found at htthttp::/ / h//ggittuub... http://tttuuu.cmiittiitittttteet
7 Acknowledgments
This work was supported in part by National Natural Science Foundation of China under Grant 62206248 and National Natural Science Foundation of China under Grant 62402430. We would like to thank all the reviewers for their constructive comments. Fangyikang Wang wishes to express gratitude to Pengze Zhang from ByteDance, as well as Yiling Zhang and Yinan Li from Zhejiang University, for their insightful discussions on the experiments.
References
[1] Brian DO Anderson. Reverse-time diffusion equation models. Stochastic Processes and their Applications, 12(3):313-326, 1982.
[2] Francis Bashforth and John Couch Adams. An allempt test the theories of capillary action by comparing the theoretical and measured forms of drops of fluid. University Press, 1883.
[3] John Charles Butcher. Numerical methods for ordinary differential equations. John Wiley & Sons, 2016.
[4] Zhichao Chen, Haoxuan Li, Fangyikang Wang, Odin Zhang, Hu Xu, Xiaoyu Jiang, Zhihuan Song, and Eric H Wang. Rethinking the diffusion models for numerical tabular data imputation from the perspective of wasserstein gradient flow. arXiv preprint arXiv:2406.1551 15762, 2024.
10

---

[5] Hansam Cho, Jonghyun Lee, Seoung Bum Kim, Tae-Hyun Oh, and Yonghyun Jeong, Noise map guidance: Inversion with spatial context for real image editing. arXiv preprint arXiv:2402.04622 2024.
[6] Jooyoung Choi, Yunjey Choi, Yunji Kim, Junho Kim, and Sungroh Yoon. Custom-edit: Text- guided image editing with customized diffusion models. arXiv preprint arXiv:2305.15779 2023.
[7] Hyungjin Chung, Byeongsu Sim, Dohoon Ryu, and Jong Chul Ye. Improving diffusion models for inverse problems using manifold constraints. Advances in Neural Information Processing Systems, 35:25683-25696 2022.
[8] M Crouzeix and Lisbona. The convergence of pariable-stepsize pariable-formuia multistep methods. SIAM journal on numerical analysis, 21(3):512-553 1984.
[9] Germund Dahlquist. Convergence and stability in the numerical integration of ordinary differ- ential equations. Mathematica Scandinavica, pages 33-53, 1956.
[10] Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. In Advances in Neural Information Processing Systems, volume 34, pages 8780-8794, 2021.
[11] Laurent Dinh, David Krueger, and Yoshua Bengio. Nice: Non-linear independent components estimation. arXiv preprint arXiv:1410.8516, 2014.
[12] Laurent Dinh, Jascha Sohl-Dickstein, and Samy Bengio. Density estimation using real nvp. arXiv preprint arXiv: /605. 088003 2016.
[13] Jiahua Dong, Wenqi Liang, Hongliu Li, Duzhen Zhang, Meng Cao, Henghui Ding, Salman Khan, and Fahad Khan. How to continually adapt text-to-image diffusion models for flexible customization? Advances in Neural Information Processing Systems, 2024.
[14] Qian Feng, Hanbin Zhao, Chao Zhang, Jiahua Dong, Henghui Ding, Yu-Gang Jiang, and Hui Qian. Pectp: Parameter-efficieett cross-task prompts for incremental vision transformer. arXiv preprint arXiv:2407.03873, 2024.
[15] Qian Feng, Dawei Zhou, Hanbin Zhao, Chao Zhang, and Hui Qian. Lw2g: Learning whether to grow for prompt-based continual learning. arXiv preprint arXiv:2409. 18860, 2024.
[16] Daniel Garibi, Or Patashník, Andrey Voynov, Hadar Averbuch-Elor and Daniel Cohen-Or. Renoise: Real image inversion through iterative noising. arXiv preprint arXiv:2403. 14602, 2024.
[17] Ligong Han, Song Wen, Qi Chen, Zhixing Zhang, Kunpeng Song, Mengwei Ren, Ruijiang Gao, Anastasis Stathopoulos, Xiaoxiao He, Yuxiao Chen, al. Proxedit: Improving tuning-free real image editing with proximal guidance. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 4291-4301, 2024.
[18] Amir Hertz, Ron Mokady, Jay Tenenbaum, Kfir Aberman, Yael Pritch, and Daniel Cohen-or. Prompt-to- Prompt-to-prompt image editing with cross-attention control. In International Conference on Learning Representations, 2023.
[19] Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by two time-scale update rule converge a local nash equilibrium. Advances in Neural Information Processing Systems (NeurIPS), 2017.
[20] Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems, 33:6840 6851, 2020.
[21] Jonathan Ho, Chitwan Saharia, William Chan, David Fleet, Mohammad Norouzi. and Tim Salimans. Cascaded diffusson models for high fidelity image generation. Journal of Machine Learning Research, 23(47):1 2022.
[22] Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. arXiv preprint 2rX1v:220..12598, 2022.

---

[23] Seongmin Hong. Kyeonghyun Lee, Suh Yoon Jeon, Hyewon Bae, and Se Young Chun. On exact inversion of dpm-solvers. arXiv preprint arXiv:2311.18387, 2023.
[24] Jiancheng Huang, Yifan Liu, Jin Qin, and Shifeng Chen. Kv inversion: Kv embeddings learning for coxticonditi real image action editing. In Chinese Conference on Pattern Recognition and Computer Vision (PRCV), pages 172-184. Springer, 2023.
[25] Nisha Huang, Yuxin Zhang, Fan Tang, Chongyang Ma, Haibin Huang, Weiming Dong, and Changsheng Xu. Diffstyler: Controllable dual diffusion for text-driven image stylization. IEEE Transactions on Neural Networks and Learning Systems, 2024.
[26] Inbar Subberman-SSiegelglass Vladimir Kulikov, and Tomer Michaeli. An edit friendly ddpm noise space: Inversion and manipulations. arXiv preprint arXiv:2304.061400 2023.
[27] The WathW Inc. Matlab version: 9.13.0 (r2022b), 2022.
[28] Xuan Ju, Ailing Zeng, Yuxuan Bian, Shaoteng Liu, and Qiang Xu. Direct inversion: Boosting diffusion-based editing with 3 lines of code. arXiv preprint arXiv:2310.01500 2023.
[29] Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion-based generative models. arXiv preprint arXiv:2206.008: 00364, 2022.
[30] Bahjat Kawar, Michael Elad, Stefano Ermon, and Jiaming Song. Denoising diffusion restoration models. Advances in Neural Information Processing Systems, 0--23533-23600 2022.
[31] Bahjat Kawar, Gregory Vaksman, and Michael Elad. Snips: Solving noisy inverse problems stochasticallyy Advances in Neural Information Processing Systems, 34:21757-21769, 2021.
[32] Sunwoo Kim, Wooseok Jang, Hyunsu Kim, Junho Kim, Yunjey Choi, Seungryong Kim, and Gayeong Lee. User-friendly image editing with minimal text input: Leveraging captioning and 2023.
injection techniques. arXiv preprint arX6vv2336.022
[33] Diederik Kingma, Tim Salimans, Ben Poole, and Jonathan Ho. Variational diffusion models. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and Wortman Vaughan, editors, Advances in Neural Information Processing Systems, volume 34, pages 21696-21707, Curran Associates, Inc., 2021.
[34] Liangchen Li and Jiajun He. Bidirectional consistency models. arXiv preprint arX1v:2403.18035, 2024.
[35] Tsung-Y Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In Computer Vision-ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Purt 13, pages 740-755. Springer, 2014.
[36] Yaron Lipman, Ricky TQ Chen, Heli Ben-Hamu, Maximilian Nickel, and Matt Le. Flow matching for generative modeling. arXiv preprint arXiv:2210.0274: 2022.
[37] Zhen Liu, Yao Feng, Michael J. Black, Derek Nowrouzezahrai, Liam Paull, and Weiyang Liu. Meshdiffusion: Score-hased generative 3d mesh modeling. International Conference on Learning Representations, 2023.
[38] Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu. Dpm-solver: fast ode solver for diffusion probabilistic model sampling in around 10 steps. Advances in Neural Information Processing Systems, 35:5775-5787, 2022.
[39] Barak Meiri, Dvir Samuel, Nir Darshan, Gal Chechik, Shai Avidan, and Rami Ben-Ari. Fixed- point inversion for text-to-image diffusion models. arXiv preprint arXiv:2312.125444 2023.
[40] William Edmund Milne. Numerical integration of ordinary differential equations. The American Mathematical Monthly, 33(9):455-46) 1926.
[41] Daiki Miyake, Akihiro lohara, Yu Saito, and Toshiyuki Tanaka. Negative-prompt inversion: Fast image inversion for editing with text-guided diffusion models. arXiv preprint arXiv:2305. 16807, 2023.
12

---

[42] Ron Mokady, Amir Hertz, Kfir Aberman, Yael Pritch, and Daniel Cohen- Or. Null-text inversion for editing real images using guided diffusion models. In Proceedings of the IEEE/CVP Conference Computer Vision and Pattern Recognition, pages 6038-6043 6048, 2023.
[43] Forest Ray Moulton. New methods exterior bullistics. University of Chicago Press, 1926.
[44] Alexander Quinn Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob Megrew, Ilya Sutskever, and Mark Chen. GLIDE: Towards photorealistic image generation and editing with text-guided diffusion models. In Proceedings of the 39th International Conference on Machine Learning, volume 162, pages 16784-16804, 2022.
[45] Gaurav Parmar, Krishna Kumar Singh, Richard Zhang, Yijun Li, Jingwan Lu, and Jun- Yan Zhu. Zero-shot image-to-image translation. In ACM SIGGRAPH 2023 Conference Proceedings, pages 1-11, 2023.
[46] Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hicrarchical text-conditional image generation with clip latents. arXiv preprint arXiv:2204.06125 2022.
[47] Hareesh Ravi, Sachin Kelkar, Midhun Harikumar, and Ajinkya Kale. Preditor: Text guided image editing with difffusion prior. arXiv preprint arXiv:2302..7979, 2023.
[48] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10684-10695, 2022.
[49] Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In Medical image computing and computer-assisee intervention-MICCA. 2015: 1 18th international conference, Munich, Germany, October 5-9, 2015, proceedings. part III 18, pages 234-241. Springer, 2015.
[50] Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, Jonathan Ho, David J Fleet, and Mohammad Norouzi. Photorealistic text-to-image diffusion models with deep language understanding. In Advances in Neural Information Processing Systems, volume 35, pages 36479-36494, 2022.
[51] Ken Shoemake. Animating rotation with quaternion curves. In Proceedings of the 12th annual conference on Computer graphics and interactive techniques, pages 245-254, 1985.
[52] Jascha Sohl-DDckstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsuper- vised learning using nonequilibrium thermodynamics. International Conference on Machine Learning, pages 2256-2266 2266.2265.2622 PMLR, 2015.
[53] Jiaming Song. Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. In International Conference on Learning Representations, 2021.
[54] Yang Song, Prafulla Dhariwal, Mark Chen, and Ilya Sutskever. Consistency models. In Proceedings of the 40th International Conference on Machine Learning, ICML'23. JMLR.org, 2023.
[55] Yang Song and Stefano Ermon. Generative modeling by estimating gradients of the data distribution. Advances neural information processing systems, 32, 2019.
[56) Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-hased generative modeling through stochastic differential equations. arXiv preprint arXiv:201 I. 13456, 2020.
[57] Xuan Su, Jiaming Song, Chenlin Meng, and Stefano Ermon. Dual diffusion implicit bridges for image-to-image translation. arXiv preprint arXiv:2203.08388 2022.
[58] Endre Süli and David F Mayers. An introduction to numerical analysis. Cambridge university press, 2003.
13

---

[59] Gan Sun, Wenqi Liang, Jiahua Dong, Jun Li, Zhengming Ding, and Yang Cong. Create your world: Lifelong text-to-image diffusion. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(9):6454-6477 2024.
[60] Jiahang Tu, Hao Fu, Fengyu Yang, Hanbin Zhao, Chao Zhang, and Hui Qian. Textooucher: Fine-grained text-to-touch generation. arXiv preprint axXiv:2409.05427, 2024.
[61] Jiahang Tu, Wei Ji. Hanbin Zhao, Chao Zhang, Roger Zimmermann, and Hui Qian. Driveditfit: Fine-tuning diffusion transformers for autonomous driving. arXiv preprint arXiv:2407.16661, 2024.
[62] Dani Valevski, Matan Kalman, Yossi Matias, and Yaniv Leviathan. Unitune: Text-driven image editing by fine tuning image generation model on single image. arXiv preprint 1..0072 2(3):5, 2022.
[63] Bram Wallace, Akash Gokul, and Nikhil Naik. Edict: Exact diffusion inversion via coupled transformations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 22532-22541, 2023.
[64] Boyang Wang, Fengyu Yang, Xihang Yu, Chao Zhang, and Hanbin Zhao. Apisr: Anime production inspired real-world anime super-resolution. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 25574-255555 25584, 2024.
[65] Fangyikang Wang, Huminhao Zhu, Chao Zhang, Hanbin Zhao, and Hui Qian. Gad-pvi: general accelerated dynamic-weight particle-base variational inference framework. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 15466-15473, 2024.
[66] Hao Wang, Jiajun Fan, Zhichao Chen, Haoxuan Li, Weiming Liu, Tianqiao Liu, Quanyu Dai, Yichao Wang, Zhenhua Dong, and Ruiming Tang. Optimal transport for treatment effect estimation. Advances in Neural Information Processing Systems, 36:1-15, 2024.
[67] Xintao Wang, Ke Yu, Shixiang Wu, Jinjin Gu, Yihao Liu, Chao Dong, Yu Qiao, and Chen Change Loy. Esrgan: Enhanced super-resolution generative adversarial networks. In Proceed- ings of the European conference on computer vision (ECCV) workshops, pages 0-0, 2018.
[68] Zhizhong Wang, Lei Zhao, and Wei Xing. Stylediffusion: Controllable disentangled style transfer via diffusion models. In Proceedings of the IEEE/CVF International Conference Computer Vision. pages 7677-7689, 2023.
[69] Andre Wibisono, Ashia C Wilson, and MichaelIJordan. A variational perspective on accelerated methods in optimization. proceedings of the National Academy of Sciences, E13(4)):E7351- E7358, 2016
[70] Duzhen Zhang, Yahan Yu, Jiahua Dong, Chenxing Li, Dan Su, Chenhui Chu, and Dong Yu. MM- LLMs: Recent advances in MultiModal large language models. In Findings of the Association for Computational Linguistics ACL 2024, pages 12401-12430. August 2024.
[71] Guoqiang Zhang, Jonathan P Lewis, and W Bastiaan Kleijn. Exact diffusion inversion via bi-directional integration approximation. arXiv preprint arXiv:2307. 10829, 2023.
[72] Jiaxin Zhang, Kamalika Das, and Sricharan Kumar. On the robustness of diffusion inversion image manipulation. In ICLR 2023 Workshop on Trustworthy and Reliable Large-Scale Machine Learning Models, 2023.
[73] Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In Proceedings of the IEEE/CVF International Conference Computer Vision, pages 3836-3847, 2023.
[74] Pengze Zhang, Hubery Yin, Chen Li, and Xiaohua Xie. Tackling the singularities at the endpoints of time intervals in diffusion models. arXiv preprint 083388, 2024.
14

---

[75] Yuechen Zhang, Jinbo Xing, Eric Lo, and Jiaya Jia. Real-world image variation by aligning diffusion inversion chain. In A. Oh, Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine, editors, Advances in Neural Information Processing Systems, volume 36, pages 30641-30661. Curran Associates, Inc., 2023.
[76] Huminhao Zhu, Fangyikang Wang, Chao Zhang, Hanbin Zhao, and Hui Qian. Neural sinkhorn gradient flow. arXiv preprint 4rX1v:2401.14669, 2024.
15

---

Appendix
Contents
Formulations
17
A.I
Detail Formulation of EDICT
17
Detail Formulation of BDIA
17
A.3
Continuity Assumption and Other Mathematical Remarks
17
A.4
Detailed Formulation of 3-step BELM
18
A.5
Detailed Formulation of k-step BELM
18
A.6
Definitions of Consistency
20
A.7
BDIA Sub-Optimal Special Case of BELM
................................
20
A.8
EDICT as a Sub-Optimal Special Case of BELM
20
A.9
Order of Accuracy
21
A. 10 Further Theoretical Properties of DDIM
21
A.II Pseudocode for O-BELM Sampling Process
22
Proofs
22
Proof of Proposition
22
B.2
Proof of Proposition
22
Proof of Proposition
23
B.4
Proof of Corollary
23
Proof of Proposition a) and Proposition
25
B.6 Proof of Proposition b) and Proposition
27
Experiments Details and Extra Results
27
Image Reconstruction
27
Image Generation Results
28
Image Interpolation
................................
29
4 Image Editing
30
Pretrained Models
30
D
Discussions
31
D.I
Hyperparameters of BDIA and EDICT
D.2
The Different Definition on LTE
32
D.3
Time Complexity and Memory Complexity
32
D.4
Other Inversion Techniques
33
D.5
Broader (Social) Impacts
35
D.6
Limitations
35
16

---

A Formulations
A.1 Detail Formulation of EDICT
sequential inversion and rearrangement of EDICT 9 yields the EDICT inversion:
y: inter (y:-1 (1 1)/p,
x; inter =(xi-1- - (1 - -0ly. p)y
(22)
Yi (yinter b, (x/nter, .i) /ois
(xinter - b,E0(ye..)) 2)) /aq.
A.2 Detail Formulation of BDIA
BDIA sampler [1 utilizes bi-directional integration achieve exact inversion, also introduc-
ing an additional hyperparameter. Reformulate the expression of DDIM 7 to be DDDM XPDIM
A (i + i - 1|xPDIM) and the expression of DDIM inversion
8 to be xDDIM xXPIMM
(i 1 i|x MDMM that is,
A (i 1i-i-11 1/x,)
0:-1
x; +
0
Q4-1
ai
(23)
A (i - 10-1--0 1 i(--))
-1
x:-1 +
0i
ai
0i-1
E0(Ji-1,8-11)
0:--
0;-1
The updating rule of BDIA write:
x,-1 =X++1 [1 (x x.) + 7 +i 1|x;]]
A
inerement(x::11 11 xx;]
increment(x; 1)
=xi+1 (1 7) (x1+1 - xx) -7 (
[att1 - x; +
[k+1
0++1
0/+1 0i
Eg( i)
aii
as
0i-1 -1 x + 0;-1
0i-1 oi
EO(x;1
04
ai
=7X++1
[1-1
as+1
X;
0t-1
0:-1
0s
-
[0141
as+1 o:)
)]
Ee Ex((x;,) 2).
a;
ai
as
a;
(24)
By rearranging the BDIA
(2). the inversion of BDIA
Xi+1
-x,-1/7 + (1
1/7)
x;
+
+ + 1(x,) - (1/7) +i- -1)1xx
ai+l
ai+1 oi EO(
X4 + 1
X; +
-
Xi + 0++1
0i-1 (a:-1
x; +
0:-1
01i11 Oi
εo(x;;1)
ai
ai++1
1 10;-1
(0:t1-
a+tl
1 ( 0:-1 0,
) E (x 1).
x
X
x;
+
aj
ai
(25)
A.3 Continuity Assumption and Other Mathematical Remarks
Continuity Assumption Much of our Local Truncation Error (LTE) analysis such as Proposition D [ is built on the Taylor expansion, which requires that the noise predictor satisfies the necessary continuity conditions. Therefore, we establish the following continuity assumption:
Assumption 2. Denote Eo(o,) = Eo (x(1), ot), assume Eo(ot) is continuous w.r.t. ō::
Eo( E C C (R,
(26)
17

---

This assumption can be met by selecting differentiable activation design in the noise predictor U-Net 49
Variable of IVP Here, we further wish to clarify that the notation éω (x()),))) Eo (x(t), t) presented in Section Biss well-defined. This is because there exists bijective relationship between o and L, and (() simply a scaled version of x()).
Singularity Issue Assumption 1 we do not consider the singularity points at and because these points can lead to unusual performance of the noise predictor as discussed in [74). In fact, our numerical method is minimally affected by these singularity points, thus making Assumption reasonable.
A.4 Detailed Formulation of 3-step BELM
For 3-step BELM, we got five coefficients in the formulation:
x 0,,3X142 + 0,,2X:41 + 0x,1%; + 11224:1160(K4411011 + (,,1,,((X, ō.).
(27)
Follow the idea of Proposition The local truncation error of the 3-step BELM diffusion sampler
T; can be accurate up to the fifth order of step sizes Ti O (h: + h4t1 + hot2)) by setting
coefficients as the following linear system
1
0
a111
h;
h4 + ho+1
h4 + h + h442
h4
hs+1
a22
h
3 (h: + hi+1)²
1
(ha hit1 + hat2)
h?
h (h: + h5+1)
01,3
I (h: + (1441)
(he + hi41 + h242)
thas (h: + hit1) )2
bi,1
24 hg
24 (h; + hx41)*
24 (h4 + h441 + hs42)*
h
hi+1 + hx+1)
3
bi,2
(28)
There no linear dependence between any two equations in (28). Through calculation by hands or equation-solving tools like Matlab (27), the linear system above yields the unique solution provided below, which can be verified by readers.
((h,the41) (3h2)
2h, 21 hhahhehhheaa 2h,4, 2 k/-, 2h 1 hess
h (Nuer +hx42)²
0
(h3(-A2 12 12 2h. ahehe the 12 22. -h. A A1 1
0;
hy ki
A hht))
0;,3
b
(Cha +h/ (h: (t
by
(Ph +h(22)
(ha [(h, thes thssa))
Ny h
(29)
A.5
Detailed Formulation of k-step BELM
For general k-step BELM, we got 2k coefficients in the formulation:
aij X; -
big hi-144 E8(- 01-14)).
(30)
18

---

Following the derivation of 2-step case, we first applying the Taylor's expansion to x;-1+f and -x; -1:x: 11--+++--;-113 1+1))
Xi
143
big
hi-14; Eo Ex(xi- (i-11)
j=1
j=1
2k-2
By 1) (x(ti-1), (x(t(-1))0-11)
0i1
81-1
+
h++m
(1)!
j=1
m==
1-1
k-1
2k-2
J-1
+
hishi
hi+m
E 1111) (x(ti-1),
(I - 1)!
=
mm=
(2k-1))
+0
hi+m
(31)
mm
k
2k 2
+
Oi
Σ
hi+m-1
Ep
(1-1)
(8(1i-1),
j=1
1=1
j=l
mm=l
1-1)
2k
R-1
biihit
Σ
hi4m-1
Eg
(1-1)
((ti-1))ō,-11
((1
1)!)
1=1
1=1
m=1
(2R-1)
+0
h4+m
m==
Thus, the optimal coefficient can be computed by:
a;,1
A 12x
0j,k
(32)
b
21
bi,k 1_ 2h-1
where A = [A] |
A2], and
f;
h4 + hit1
i hits
(h: + 11+1)*
listj)
A
=
(33)
2k
(2k 24 (h, ho4 32
2
(EK= hits
2k-2
22
(2k 2)
0
h;
hi+1
h 2-2
1?
h441 (hy + hit1)
h 2
2
k 2-2 1i+s
j
2
A2
2 1
} hea (he + hs41)²
h
k-2
j=0 Mitt
2k-3
(2k 3)
h
2k-3
-TR- bis (h+ hi+1) 2k-3
(2
33T
1
-5-0 hits)
(34)
19

---

A.6 Definitions of Consistency
Consistency The consistency property refers to the ability of the method to accurately represent the IVP equation it's trying to solve. More specifically. method is said to be consistent if, as the step size approaches zero, the difference between the numerical method and the exact differential equation also approaches zero.
Definition 4. The LM method
12 is consistent if for every functionn y C. [to, to + T]
lim f
(35)
A.7
BDIA Sub-Optimal Special Case of BELM
the updating rule BDIA write:
0:-1 0r++1 + 01-1
0s-1 -7 -0000011-10 a++1
X;-1 +0x++ +
EØ (x1, i).
asi
ag
(36)
With the same alpha, scaled sigma and stepsize schedule the BELM, the BDIA update an equivalent bidirectional explicit linear multi-step form with an easy rearrangement,
(36) have
- = 012 x;41 + 01,1 X; b,,1 h4 Ex(X, ox),
(37)
where
04+1
04,1 = 0++1
0441 h441
0,2 7
b -1
(38)
Q4-1
a;11
a4-1 h4
Thus we find that BDIA is indeed a special case of our BELM framework.
A.8 EDICT as a Sub-Optimal Special Case of BELM
In this section, we will demonstrate that sequence of (x,), {y:). (y:nter), and (xinter), where i E [N... 1], generated by EDICT (9), indeed corresponds to sequence of zj. where E [4N.... 1]. produced by a special BELM. The EDICT updates as follows:
x
inter
0:-1
X;
0i-1
ai-1
Oi
en (y:;
y
vittr
0:1
yi + 01:1
a1-1
Eo (x(x(1)inter .))
(39)
ai
ai
X;-1 = px inter
+ - p)y
Yi-1 pyinter + x)
x xiter transfer x; to 241, Yv to 24t
to Z4l and y
to 241-3-
0;-1
0:11
Z1l-2
Ztl
0:-1
ai oi
E0 E0(zN-1,1),
a₁
ZØ-3
0:-1
Ztl + 01-1
04-1
0i
E0 (0(2
(40)
ai
ai
241-4 p4l 2 + (1 - 22)
Z4r-5 = pZs! 3 + (1 p)
We set alpha schedule to be
Ct4i = 0;; 0: = ai Q41- 10;0;-1,
041- = 0;-1-
(41)
Then we set sigma schedule to be
1 / Vai-i
Voi
T 04 Ty-1 o 04-2 =
0.
+
0
01-3 01-1-
(42)
2
Va;
20

---

Thus the scaled sigma writes
0i-1
0:-1
σu 01--
0412
O1l-3
(43)
O4
0i-1
And the stepsize schedule will be har = 0, h4r
(
0z 0i-1
1
0i
)
02-1
har-2
h4t-3 =
(44)
a2 0:
2
04
a;-1)
With easy substitution, the EDICT update multi-step form:
(40) have an equivalent bidirectional explicit linear
2--1 = d322 35+1 + 4₁₁ 2, + +b,11 hy Eo(Z,,
(45)
where the coefficients take the following picce-wise function form:
0++1
.j 41
0,j 4l
Vas,
P,J = 41
0,j 4/
p.j -
P. =
0j22 =
V . -
03
0, = -4l - 2
b
-2
a4-1
. 41 -
Vag
V0z
0, =
2. 41 -
1,j 41 3
(46)
Despite the formulation of (9) being subject cyclic variations, the variable aj,2 consistently remains non-zero, thereby satisfying the conditions of the BELM framework. Consequently, EDICT can indeed be considered special case within our BELM framework.
A.9 Order of Accuracy
In this section, we further explore the order of accuracy of DDIM, EDICT, BDIA, and our proposed O-BELM. Our findings indicate that O-BELM achieves the superior order of accuracy among these methods. Intuitively, the order of accuracy provides insight into which functional class of the IVP can be accurately approximated by given method.
The method 14 is said to have order of accuracy if is the largest positive integer such that there exist constants K and h² such that for all i,
Definition 5.
| < Khttl, for 0 h h
(47)
The BELM diffusion sampler (15) is with second-order accuracy: The DDIM diffusion sampler ( is with first-order accuracy; The EDICT diffusion sampler ( is with zero-order accuracy; The BDIA diffusion sampler (10) is with first-orrr accuracy.
Proposition 6.
Proof. This proposition can be directly inferred from the Definition in conjunction with Proposition and
Remark Though the order of accuracy of BDIA is the same as DDIM be 1, its step size in local error of BDIA is about twice that of DDIM. This theoretical result confirms the experimental observation that the sampling quality of BDIA sometimes inferior that of DDIM.
A.10 Further Theoretical Properties of DDIM
We have also conducted an analysis of global stability and convergence for DDIM. It is apparent that the success of DDIM fundamentally stems from its nice theoretical property. Our O-BELM preserves these excellent theoretical properties of DDIM and maintains high-quality sampling performance.
Proposition The DDIM diffusion sampler 77) is (a) zero-stable and (b) globally convergent.
21

---

A.11 Pseudocode for O-BELM Sampling Process
To more effectively elucidate the implementation of O-BELM, we provide the pseudocode for O- BELM in Algorithm[] Upon examination, it is apparent that the implementation O-BELM requires little modifications compared to DDIM, thus facilitating its easy portability to pretrained models.
Algorithm 1 O-BELM sampling process
1: Input: pretrained noise predictor Eø, number of timesteps N, noise schedule {a+} and {or)
x =.
2: Sample XN NN Ttx
3: x_listaapped((xx)
4: for i = N, 1,..., do
5: i N then
6: Calculate h4, 0,,1, 042 and b according (53).
7: X;-1 = list-] + a,,2 list[-2] + bx, list[-1], i)
8:
else
X1-1 si-Lx x list[-1t] + 0i-1 a. a.1o,) 0.) ED (xst(-ss[[- list[-11, i).
9:
a,
10: end if
11:
x list..ppeed((x-1)
12:
end for
13: Output: x list
B Proofs
B.1 Proof of Proposition
Proof. We demonstrate the exact inversion property of BELM (14) by initially establishing that its local reconstruction error is zero. Assuming that we have already obtained {x:-141/ we compute
8 in accordance with [4 as follows:
k
8
Oij X; 1-1
big hi-145 Eo 01
(48)
and we will (8i k-1
to reconstruct & according follows:
k-1
Oi.j 84-
bi.j aiı
11
8,-1
+
h4 E0( 01-1-))-
(49)
ai.k
Oj,b
1=0
The local reconstruction error, defined as the difference between X;-1+ and X;- can be caleu- lated and is found to be zero. Furthermore, global exact inversion can be inferred from local exact inversion through the application of Mathematical Induction (MI).
B.2 Proof of Proposition
Proof. The Local Truncation Error (LTE) of the BELM diffusion sampler 15) can be computed by substituting x(t.), x(t+41), and Ex(x(t)), o,) I6) with their corresponding Taylor expansions at
22

---

0 as follows:
" =x((i-1) - 01,1 x(14) 01.² x((++1) bú,1 h4 (o(x(t,),
En (x;(i-1)0;--1)) (14)
-x((i-1) dil x(t;-1)
11
V V Eo (x(t; 1); 0i-1) (1:)² O (1:²²)
2!
Eg (0 (l4 1++1)+ +
412 x(1(-1) +
11
Vo._ Eo (x(ti-1) ,
(h4 + h1t1)² + ((hs h(t2)²)
(50)
11 h4 En (x(1;-) x(-1) V Eo (x(()-1), (h+) + 0 (1133
1!
(1 01,1 01.2) X 01.2)x((13)) (11=1))
+ [-as ai, his - 0;,2 (hs + hi+1) ) - bi,1 hs] Eo (((1-11,0;;-
+ aill 122 04. (11 + h1+1)² - bi, 12?] Vo. Eo (8(ti-1), ōi-1)
2
2
+ O (14: + l1s+1)))
B.3 Proof of Proposition
Proof. In the 17 of Proposition 3 we have three degrees of freedom: 01 0i,2, and b. in the LTE of BELM Therefore, the highest order that T; can achieve three, under the condition that:
a,,1 a, = 0,
011144 a1,2 (h4 ho+1) b h4 = 0,
(51)
01, hī 0.2 (hr + ho41)² bú, h3
2
whose matrix form writes
1
[a
hi
(he hat1)
h;
01,2
(52)
1 (h4 + h(+1)²
h
b
There is no linear dependence between any two equations in (5I). Through a straightforward calculation, the linear system above yields the unique solution provided below, which can be verified by readers.
122
h?
h4 + hi+1
04,1
1++
h
04,2
h2
bi
h44
(53)
B.4 Proof of Corollary 1
B.4.1 LTE of DDIM
Proposition 8. The LTE e, DDIM sampler is O (at-2h;?) 3).
23

---

Proof. By applying the Taylor expansion and substitute into the DDIM formulation calculate the local error of DDIM on x, following.
we can
e, -x((i-1)
0i-1 x(t:)
02-1
01-1
0i
Ey (8(t:),
Oi
ai
=x(1-11
a:-1
0i
x(t:-1)
Eo 13,
(h4) 0 (h:
(54)
ai
a:-1
11
- (-h+) a;11 (Ea
(x(t)-1), +0 (h.))
=0 (04-1h;?))
B.4.2 LTE of BDIA
Proposition 9. The LTE e; of BDIA sampler 10 O ( + h² for any fixed
7 [].
Proof. By applying the Taylor expansion and substitute into the BDIA formulation 1O calculate the local error of BDIA on x, as following.
we can
e, =x(t,-1) (x((++1)
a;-1
- ai+1
x(ti)
ai
ai
0:-1
a:-1
0; -7
01+1
ai+1
Ji
Ey (x(ti), o.)
Oii
ai
x(ti-1)
Eo (x(t;-11 0i-1
=x(t,-1) - 7as+1
+
(h: + h(t1)+ +
ai-1
11
V Va._, Eo
(x(t:- 1), o(-1)
(h: + h1+1)2 + O (hs + hett)*)
2!
x(ti-1) + Eg (x((i-))) (hi)
0:-1
ai+1
ai
(55)
a1-1
11
+
Ep
(x(t,
.T.-1))
(hi) + (1x3)
2!
ai+1 03 )]
0t-1
|so (8(4)-
0
0j -7
0it1
0i
ai
aa Ep (x(4i-1),
(h:) + + (h:2)
1!
-(- 04-- h
h2 +
Vo...o (8(ti-1), (h:²).
2
For fixed 7, the term Eg (x(t(-1), 11- cannot be eliminated for every This due to the fact that the second-order term 1 1 -0,-1h; h is dynamic with respect i. Consequently, the second-order local error will persist in the BDIA.
B.4.3 LTE of EDICT
Proposition 10. The LTE e, of EDICT sampler p is (Vai-ih)) for any constant p E (0, 1).
To prove the Proposition 10 need first establish order estimate lemma:
Lemma 1. The term Va: vai-1 have order O (h+)
24

---

Proof. Recall that we define h; to be o, - In order to figure out the relation of Vai v0
w.r.t. o; 0i-1, we first use 7 to represent va:
V1 a
2
σ²
a
(0² + 1)a =1
(56)
72 +1 Va (o +1)
0
then discover that dvā = - (0² + 1)
ōdō Cdo. This implies that Vaz Va-1 and
0; 0--1 of the same order, which is h,
Now we can start to prove Proposition
T0
Proof. Since larger emrors absorb smaller ones, the 4l and 4/ terms in 446 introduce errors in the zeroth order of the Taylor expansion. This is where the main error occurs. Both of these updates
introduce error of yai- Vai-1 on X;, which means that the error x; is vaz (Va; va-1)
/as
Therefore, according Lemma the error e; is of the order O (Varh.).
Remark 3. Please note that we have only established error bound for EDICT based the perspective of the linear multiplication method. There may be a tighter bound of EDICT constants when viewed from the perspective of an interactive mixing system.
B.5 Proof of Proposition 5la) and Proposition 7(a)
Assumption ō₁ is strictly concave w.r.t i.
o w.r. is composition of o, w.r. a; and a, w.r.t. i. strictly convex. Thus Assumption 3 can be achieved by choosing schedule of to be strictly convex w.r.t.
which is non-inereasing and
Lemma 2. There exist a real constant C which is independent of a, and o such that for every we have |o,,1| C, |0,,2| S C and [br.a < C in (5)).
We will use an variabb-stpsine-varbblffoommm analogy of the root condition of Dahlquist 9 to prove the zero-stability (15).
Theorem 1. 18 (3.10)) Define the root matrix of LM LM at step to be R.,
0i.1
0g,k
R,
(57)
If all coefficients be bounded and there exists a regular matrix H such that for all
[ R $1,
(58)
then the L2 is zero-stable.
Finally, start to give the proof of Proposition 5 a) under the Assumption 3
25

---

Proposition 11. The O-BELM diffusion sampler 18) is zero-stable.
Proof. the root matrix of 15 writs
A b
R
N
(59)
The Assumption can reach to 0i+1 + 0:-1
20,, thus h4t1 < hs < 0. Then denote
h2 < 1. by setting H as following
max; 1
(60)
0
1-1
then calculate that
R
H
R
8 1
(61)
|1441 12 + |hs1?
21
max
11+11 12
117
where we can compute that
12
11
12
+
h
2
1
11
+
2
2
14+1
h2
12
11
2
h
(62)
h?
2
2
h
h
h?
1,
and obviously
2
1.
(63)
11
Consequently, we have the conclusion that for all i, the requirement of Thus due to Theorem The BELM diffusion sampler (15 is zero-stable.
S is satisfied.
Remark 4. Here, we present very strong proof of Proposition T7 under Assumption 3
demonstrating that the iterative mapping of BELM constitutes contraction mapping at each step i. However, is important to note that in practical applications, even if Asssumptioo 3 not met at some step i, resulting in R; not being contractive sometimes, global stability may still be achieved.
The proof for Proposition a) writes:
Proof. As DDIM can be seen as an explicit Fuler method the diffusion IVP, following the same reasoning of B.5 the root matrix of DDIM is R, I. Obviously, DDIM is zero-stable. D
26

---

B.6 Proof of Proposition S b) and Proposition (b)
To analyse the global convergence property of a method, we first need to analyse the consistency property of a method. Please look the definition of consistency in Appendix A.6 We first establish the consistency of DDIM and BELM by the following theorem.
Theorem 8 (2.5.1)) If a method have an order of accuracy and all its coefficients is bounded by constant, then it is consistent.
Lemma The BELM diffusion sampler 15 is consistent.
Proof. This lemma is direct result of Lemma Proposition 6 and Theorem
Lemma 4. The DDIM diffusion sampler is consistent.
Proof. In common choice of noise schedule, and 03-1 og is bounded. Thus this lemma direct result of Theorem 2
After we establish the consistency of DDIM and BELM, we can prove their global convergence by famous sufficiency of conditions for convergence.
Theorem 3.
p.342 (Theorem 406D)) A linear multistep method is convergent if consistent and zero-stable.
With the help of Theorem 3 we can reach to Proposition (b) b by Lemma 4 and Proposition aa); and reach to Proposition b) by Lemma Jand Proposition
C Experiments Details and Extra Results
In these image tasks, only apply 2-step O-BELM, it has been demonstrated that higher-order numerical methods can lead to strong oscillations stiff spaces such as images (58) p.343]. However, the application of higher-order O-BELM in other domains of Diffusion Models (DMs) continues hold promise.
For the sake of open accessibility, the dataset used in this paper is publicly available the internet. We have included codes, accompanied by corresponding instructions, the supplementary materials and plan to make them accessible GitHub. However, interwoven with our proprictary business code, and we are the process of decoupling the codebase. As soon as this task completed, will make the codes available GitHub.
Stable Diffusion-relatee code is intricately
C.1 Image Reconstruction
presents the reconstruction results from several example images under 50 steps. It is evident that DDIM reconstructs images with non-negligible distortions compared to the original images, marked by the red rectangle in Figure 4 Our findings suggest that the exact inversion samplers (EDICT, BDIA, and O-BELM) indeed achieve exact inversion at the latent level, thereby achieving the lower bound of the reconstruction error of AE in latent diffusion models. Although the encoding and decoding processes of AE introduce some reconstruction error, these errors do not result in any detectable inconsistencies in the image perceived by the human eye. It's also important note that exact inversion requires the storage of two intermediates for precise reconstruction. This feasible in downstream tasks such image editing.
Figure 4
We have also conducted an additional experiment to assess the reconstruction error in the latent space of O-BELM and other baseline methods shown in Figure
27

---

Table Comparison of different samplers on MSE reconstruction loss on latent space on COCO-14.
MSE loss of reconstruction latents
DDIM
EDICT
BDIA
O-BELM
10 steps
0.414
0.000
0.OIX)
0.000
20 steps
0.243
0.000
0.000
0.000
50 steps
0.063
0.000
0.000
0.000
steps
0.041
0.000
0.000
0.000
Original
Reconstruction
DDIM
O-BELM
EDICT
BDIA
AE
house"
Recon 0.012
0.003
MSE
0.003
"A cupmake'
Recon MSE 0.011
Rezon MSE 0.001
Reeonn MSE
Rezan MSE
Recon MSE 0.001
'A cal"
MSE MSE
"A dog"
Recon MSE 0.004
Recon MSE 0.001
Recon MSE 0.001
Recon MSE 0.001
Recon MSE 0.001
"Aatyy"
Recon MSE 0.007
Recon MSE 0.002
Recon MSE 0.002
Reson MSE 0.002
Recon MSE 0.002
Figure 4: Results of image reconstruction and MSE error using DDIM and exact inversion samplers under 50 steps. The red rectangle point out the inconsistent part in the reconstructed images of DDIM.
C.2 Image Generation Results
hyperparreeeerr choosing for EDICT and BDIA For EDICT and BDIA, each has an additional hyperparameter (ry and respectively) whose optimal values are sensitive to the task at hand. To ascertain their appropriate hyperparameters for CIFAR- 10 and CelehA-HQ, we executed grid search in the 10-step scenario, as depicted in Table 6 and Table 7 These values were then fixed when performing cases with more steps. We evaluated ? in BDIA from 0 to with grid increment of 0.1, and assessed p in EDICT from 0.90 to 0.97 with a grid increment of 0.01, adhering to their suggested hyperparameter intervals, For the text-guided generation task using COCO-14 captions, we employed the values recommended in their respective papers.
Guidance Weight in Conditional Generation The 30k prompts randomly selected from the COCO datasct 35) as the test set. For the text-guided generation task, utilize a fassifier--free
28

---

Table 6: Comparison of FID score( 4) of BDIA method for the task of CIFARIO/CCelbb--HO generation with different choice of in the 10-step scenario.
The choice of
BDIA
0.5
CIFARIO
17.41
12.27
15.60
23.62
33.39
43.93
54.93
66.32
78.33
92.16
106.37
ColebA-HQ
52.56
61.19
76.31
Table 7: Comparison of FID score( 4) of EDICT method for the task of CIFARIOOCelebA-HO generation with different choice of p in the 10-step scenario.
The choice of p
EDICT
0.90
0.91
0.92
0.93
0.94
0.95
0.96
0.97
CIFARIO: 10 steps
149.00
142.81
135.05
127.52
119.57
110.50
99.86
87.11
CelebA-HIQ: 10 steps
82.16
78.09
74.10
70.18
66.61
63.16
60.43
57.82
technique 2 which requires guidance weight. For BDIA, select guidance weight of 4.0 and for EDICT, we choose 3.0, as recommended their respective papers. For DDIM, we perform a grid search in the 20-step scenario, as shown in Table g and determine the optimal guidance weight be 5.5. This value is then fixed for other scenarios as well as for the O-BELM sampler.
Table 8: Comparison of FII) score( J) of DDIM method for the task of text-guided generation with different choice of guidance weight.
The choice of guidance weight
DDIM
2.5
3.0
3.5
4.0
4.5
5.0
5.5
6.0
6.5
7.0
7.5
COCO-14
26.03
22.66
20.90
20.14
19.66
19.47
19.45
19.47
19.61
19.81
19,97
Examples O-BELM We present unconditionally generated samples of O-BELM sampler in Figure 5(a) (CIFARIO. 32 and Fib)re5(()) (CelebA-HQ, 256 256). Furthermore, display text-guided generated samples Figure utilizing the pretrained SD-1.5 model 48 (512x512) with captions from the COCO-14 dataset 335. The guidance weight for our O-BELM has been set at 5.5 to align with the choice of DDIM.
C.3 Image Interpolation
Image interpolation refers to the process of morphing between two images by interpolating between their corresponding latent vectors in the latent space, usually expecting to achieve a smooth transition between these images.
The diffusion ODE (5) establishes a correspondence between latent noise and samples, which can also be perceived as coding for the samples. Given that O-BELM can more effectively simulate the diffusion ODE while preserving the one-to-one relationship of the coding, we believe that the exact inversion of O-BELM can intrinsically provide more rational correspondence, This, turn, facilitates superior interpolation effects.
We follow the experiment setting in 53 to generate interpolations on line, which randomly sample (1)
two initial values (0) and zy from the standard Gaussian A (0, 1). interpolate them with spherical linear interpolation [51], then use the BELM to obtain zo samples. The spherical linear interpolation
(a) is calculated by o
sin((1 - a))) sin(O)
sin sin(a0) sin( X
X7 +
(64)
where 0 arccos
We demonstrate the interpolation results of various models including CelebA-HQ (a), Butterflies (b), Emoji (c) and Anime (d) in Figure
29

---

as our Figure 5: (a) uncurated CIFAR10 samples with BELM, steps 100 (b) uncurated CelebA-HQ samples with BELM, steps = 100
C.4 Image Editing
We adhere to the experimental setup of 63). initially introducing inversion noise to the images while preserving 20 percent of the steps during the inversion process. We utilize new prompts reconstruct and edit the images. The guidance weight is consistently set at 3.0 for all instances.
C.4.1 ControlNet-Basee Image Editing
We evaluated O-BELM and baseline algorithms on ControlNet-based image editing tasks, which included canny-based and depth-map-bass editing as illustrated in Figure 8 The editing hyperpa- rameters chosen the same as our original paper. The ControlNet hyperparameters were kept at their default values, consistent across all methods. We set the number of steps to 100. The canny images were obtained using the Canny function from the opencv pythhnn library, and the depth- map model used was Intel/dpt-large htt.lo/..gggggggggo/ggggggooeoen diffusion-vl-S model http:://waggggggeece//mmm////a/s base model.
co
large). We use stable-
C.4.2 Style Transfer
We evaluated O-BELM and baseline algorithms on style transfer tasks using the style transfer sub-dataset of the PIE-Bench dataset (https://paperswithhe)) illustrated in Figure 9 The editing hyperparameters were selected to match those our orig- inal paper. We use stableddiffusion-2-baa model http:///hhggigggaaaa..cesaaaaaaa
com/ com/dataset/pie-beet as
Srable-diffusti 2) as our base model.
C.5 Pretrained Models
All of the pretrained models used in our research are open-sourced and available online as follows:
CIFARIO generation : ddpm eereeeeee_a______e_e_eeeeeeeeeee saaeeeeaaeeeeeeeeeeeeeeeeeeeeeee ci
CelebA-HQ generation and interpolation dudpm-ema--cllebbag2233
Text-to-Imaae generatioo stable-diffussion-i115,, stable-diffusion-
30

---

the exact bowl of prepared food including rice, meat and broccoli.
modern day bathroom with a unusual sink
A motorcycle parked on the pavement near building.
Center city street itt up by a series light displays.
close up photo shaggy car
flying in the ocean with the sun in the background.
Three men walking the middle of the road
aerial view a planes window of clouds and a sunset
There woman standing city street.
cute kitten is sitting table.
two trains go side by street
A yellow and brown bird perched leaf on top of grass covered field.
Half of white cake coconuts
woaring sunglasses and black
sitting a window sill next window.
Figure 6: Prompts and generated images by O-BELM COCO-14 dataset using SD-1.5 with 100 steps.
Butterflies interpolation ddinn-butterflies-122
Emoji interpolation 4doo--FEnnj//liggndFaaaee4 http://wwggggiileecoooooo/////np
Anime interpolation dddpm-ema-amiie-2256
The scheduler setting For these pre-trained diffusion models, we adopt the noise scheduler outlined in their respective configurations and apply it consistently across all our experiments. As experiments do not involve the training or fine-tuning of diffusion models, there is no requirement to develop a new scheduler setting.
D Diseussions
D.1 Hyperparameters of BDIA and EDICT
Notice that, the intuitive exact inversion samplers achieve exact diffusion inversion cost of introducing an additional hyperparameter. comparing to DDIM, including both BDIA (TO) (with additional hyperparameter 7) and EDICT ( (with additional hyperparameter p). We point out that the need for additional hyperparameters would hinder the widespread application inversion samplers. The sampling quality of the previous exact inversion samplers is highly inrobust to the additional hyperparameter. EDICT recommend to choose E [0.9, 0.97] as EDICT would result in inconvergence when p 0.9.
we observe that the use of different hyperparameters within the recom- mended interval could potentially result in divergence. In Table and Table note that the Frechet
As depicted in Figure TO
31

---

(a) Interpolation Results on CelebA-HQ
(b) Interpolation Results on Butterfiies
(c) Interpolation Results Emoji
Interpolation Results on Anime
Figure 7: Interpolation samples of various models using O-BELM with 100 steps.
Inception Distance (FID) fluctuates significantly with respect these unstable hyperparameters. Furthermore, the optimal hyperparameters vary across different datasets and steps.
D.2 The Different Definition on LTE
We would like draw readers' attention the fact that the term Local Truncation Error (LTE) used this paper might differ from its usage in some other mathematical papers [58, p.317(12.24)). Specifically, what is referred to Ti /b; in this paper is often called LTE in other contexts, implying that their definition of LTE includes an additional division by a stepsize. However, in the context of variable-stepsize-aaie waterre-vrraabe-fforeee
(VSVF), our definition proves be more convenient and is more commonly adopted in papers dealing with VSVF 8 (2.1)).
D.3 Time Complexity and Memory Complexity
Time complexity Regarding the sampling task of diffusion models, the time cost bottleneck is the access to the noise network εø (x,, t). The number of accesses to (, i) is also referred to as NEE (the number of function evaluations). For the same value of N, observed that O-BELM, DDIM, and BDIA all require an NEE equal to for a single sampling chain. However, EDICT doubles this requirement to 2N.
Experimentally, we've conducted additional tests to compare the average time cost of different methods across sampling, editing, and reconstruction tasks. The results show that O-BELM does
32

---

Original Images
Control Conditioning
(Ours)
ControlNett Canny-basne
Inversion
X
Eddting
ControlNet Depth-map--ased Editing
Inversion
X
Figure 8: Comparison of ControlNet- based editing results of different samplers. DDIM leads to inconsistencies (red rectangle), and the EDICT and BDIA samplers introduce low-quality sections (yellow rectangle). Our O-BELM sampler ensures consistency and demonstrates high-quality results, in such large scale editing and still preserve features from original images (face in the first example and clothing in the second example).
Original Images
O-BELM (Ours)
DDIM
EDICT
BDIA
painting
plxel of)
[watercolor painting of]
Figure 9: Comparison of Style Transfer results of different samplers the PIE Benchmark. DDIM leads to inconsistencies (red rectangle), and the EDICT and BDIA samplers introduce low-quality sections (yellow rectangle). Our O-BELM sampler ensures structure preservation and high-qualij style transfer, thus show the robustness and effectiveness of O-BELM sampler.
not incur any additional computational overhead compared to DDIM across all these tasks. Detailed information about these experiments be found in Table
Memory complexity During the sampling process of diffusion models, typically the entire chain of the process is maintained. Both BDIA and O-BELM do not require additional memory beyond the previous sampling path. However, due the auxiliary states, the memory requirements of EDICT need to be doubled.
All experiments were conducted single V100 GPU and an Intel Xeon Platinum 8255C CPU. The sampling of 30k images using SD models under 100 steps took approximately 24 hours. The sampling of 50k images using pre-trained CIFAR10 model under 100 steps took around 4 hours. Meanwhile, the sampling of 50k images using a pre-trained CelebA-HQ model under 100 steps required about 40 hours.
D.4 Other Inversion Techniques
We observe that the field of diffusion inversion is rapidly evolving. Recently, several works related to diffusion inversion have been proposed.
For instance, the study by 26 suggests altering the prior distribution, as opposed to using Gaussian noise, for more convenient inversion. However, this approach requires training new models, rendering it incompatible with existing pretrained models.
33

---

0.97
0.96
0.95
= 0.94
0.93
EDICT
red-naired
origin:
BDIA
7= 1.0
== 0.98
7 - 0.96
? 0.94
Y= 0.92
Figure 10: Image editing example for EDICT and BDIA with different hyperparameters, carried out over 200 steps. We observe that even within the interval advised in the original paper, the editing result may still diverge.
Time costs of Different Tasks (50 steps)
Image Generation (s) Image Editing (s) Image Reconstruction (s)
DDIM EDICT
6.67
13.30
13.20
12.67
25.77
25,72
BDIA
6.59
13.37
13.28
O-BELM(Ours)
6.53
13.22
13.20
Table 9: Comparison of time costs for different methods on the PIE Benchmark using the SD-2b model, tested single NVIDIA Tesla V100. The results indicate that O-BELM does not incur additional computational time costs compared to DDIM across Generation, Editing, and Reconstruc- tion tasks. We assessed the time costs of O-BELM and baseline algorithms using the PIE-Bench dataset bottps://paperswithhee
com/ dataset/pi pettaee/pieebbech),, which included tasks such as image generation, image editing, and image reconstruction. The number of steps was set to 50. We employed the stable-diftusion22-ass model (https: (https://hun htt.///hggggga.....o// htt.gaf....o..aaaaaaaaeeeooooo wo..tttbiittaaiiii stable-diffusion:
our base model and conducted tests on single NVIDIA VIO0 chip and an Intel Xeon Platinum 8255C CPU.
The research conducted by (72 16 advocates for the training of a model-dependent bias corrector for precise inversion. Despite this, fails achieve mathematically exact inversion.
The work of (39 23 proposes the use an implicit method in inversion to align with the sampling. However, this approach is time-consuming and residual optimization errors persist.
And, the study by 34 suggests training reverse one-step consistency model. However, its experi- mental performance also demonstrates reconstruction inconsistency.
We understand that there several techniques proposed to address the inexact inversion issue of DDIM within the context of classífier-free-eex--eeee guided image editing. These include NMG [], DirectingInv (28), ProxEdit [17), NPT [41 and NT [42]. We point out that the proposed O-BELM and these techniques should not be considered as comparative algorithms for following reasons.
These methods are orthogonal. O-BELM modifies the discretization formula achieve exact inversion, while these techniques adjust the classifier-fite--uiine guiddnce mechanism. They address this problem from different directions.
They be used together in the classifier- mussifier-free-teer free-text-guidee image editing. Take DirectingIny as an instance, its inversion is just DDIM inversion and its forward process encompasses state-interacting DDIM forward processes with different prompts. We can substitute the DDIM inversion/forward DirectingInv to be O-BELM inversion/forwarr and get O-BELL
Their working scenarios differ. The O-BELM is built on the general diffusion IVP, and can guarantee exact inversion and minimized error under tasks based on diffusion ODE (PF-ODE), O-BELM can always converge to underlying IVP solution as demonstrated by
34

---

Proposition 5 This means that the BELM framework is compatible with wide variety of diffusion- based tasks, irrespective the data type (images or words), the task type (editing interpolation), the guidance method (unconditional, classifier-free, classifier-basett or adjoint ODE-based), or the network structure (whether includes an attention layer or not). On the contrary, these techniques are developed specific for maassffer-frreetext--eee editing task.
image
D.5 Broader (Social) Impacts
The development of accurate and stable exact inversion diffusion sampling like O-BELM for DMs. as discussed in this paper, holds sigmificant potential for several domains, including machine learning. healthcare, environmental modeling, and economics.
However, while this research holds great potential for positive impacts, it is also important consider potential negative societal impacts. The enhanced ability of an accurate and stable exact diffusion sampler could potentially be misused. For instance, it could be exploited to creating deepfakes, leading to misinformation. It may also raise privacy concerns, as more detailed and source data can be decoded from the intermediates using O-BELM. In healthcare, if not properly regulated, the use of synthetic patient data could lead to ethical issues. Therefore, it is crucial to ensure that the findings of this research are applied ethically and responsibly, with necessary safeguards in place to prevent misuse and protect privacy.
D.6 Limitations
This paper does explore the integration of high-accuracy exact inversion samplers such O- BELM with more powerful image editing pipelines. Additionally, the application of high-accuracy exact inversion samplers like O-BELM to tasks beyond image processing remains uninvestigated in this work. The concept of employing bidirectional explicit constraints to ensure exact inversion when applied to accelerated DM-solvers remains unexplored. There also lot downstream tasks of DMs that a exact inversion samplers like O-BELM can apply will be also interesting apply the exact inversion ODE sampler in variational inference or flow matching [36 76].
[64] [1l 660 4 66 [3] 59
770
15
[4]
69
35

---

NeurIPS Paper Checklist
The checklist is designed encourage best practices for responsible machine learning research, addressing issues of reproducibility. transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and precede the (optional) supplemental material. The checklist does NOT count towards the page limit.
Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:
You should answer [Yes], [No]. or [NA].
[NA] means either that the question Not Applicable for that particular paper or the relevant information is Not Available.
Please provide short (1-2 sentence) justification right after your answer (even for NA).
The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include (after eventual revisions) with the final version of your paper, and its final version will published with the paper.
The reviewers of your paper will be asked use the checklist as one of the factors their evaluation. While "[Yes] is generally preferable to "[No]" ", it is perfectly acceptable to answer "[No] providedd proper justification is given (e.g.. "error bars are not reported because it would be too computationally expensive" or "we were unable to find the license for the dataset we used"). In general, answering binary way, we
"[No] or "[NA] is not grounds for rejection. While the questions are phrased acknowledge that the true answer is often more nuanced, please just use your best judgment and write justification to elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] to a question, in the justification please point to the section(s) where related material for the question can be found.
IMPORTANT, please:
Delete this instruction block, but keep the section heading "NeurIPS paper checklist",
Keep the checklist subsection headings, questions/answers and guidelines below.
Do not modify the questions and only use the provided macros for your answers.
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Our main claims do reflect the theory and experiment contents of the paper.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize other settings.
is fine include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
We discuss about the limitations of the work in Appendix D.4.
36

---

Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, c.g., if the approach was only tested few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
Justification: For every theoretical result, we give the needed assumptions and proofs.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: Our dataset, parameters and code will be uploaded to ensure reproducibility.
Guidelines:
The answer NA means that the paper does not include experiments.
37

---

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution
specific model and empirical evaluation, may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, described in supplemental material?
Question:
Answer: [Yes]
Justification: The dataset utilized in this paper is publicly accessible on the internet. We have included codes, along with corresponding instructions, in the supplementary materials and intend to make them available on GitHub. However, our Stable Diffusion-related code is deeply intertwined with our proprietary business code, and we are currently working on decoupling the codebase. As soon as this process complete, we will make them available GitHuu.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.co cc/
for more details.
While encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https //nips.
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
38

---

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Answer: [Yes]
Justification:
We provide the details of experiments in Appendix C.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, or as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [NA]
Justification: This paper does not raise any issues related to statistical significance.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by enror bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall
with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma enror bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric eтror bars that would yield results that are out of range (e.g. negative eтror rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: We do provide sufficient information on the computer resources in Appendix D.3.
39

---

Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental rums well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code Of Ethies
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer: [Yes]
Justification: This paper do conform with the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. if there is a special consid- eration due to laws or regulations their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive socictal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: We talk about the potential societal impacts Appendix D.5.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (c.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Question: Does the paper describe safeguards that have been put in place for responsible release of data models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
40

---

Answer:
[NA]
Justification:
We do not release anything that carries a high risk of misuse.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging. and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g.. code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: All such assets are appropriately credited, and the license and terms of use are explicitly mentioned and duly respected.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website). the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the bataasess.ppprsstttt paperswi theode. com/datasets package should be provided. For popular datasets,
has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. If this information is available online, the authors are encouraged reach out to the asset's 's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and the documentation provided alongside the assets?
Answer: [NA]
Justification:
We do not introduce new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/model as part of their
submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
41

---

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification:
This paper do not involve Human participants.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country or institution) were obtained?
Question:
Answer: [NA]
Justification: This paper not involve Human participants.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
42


Reviews:
[{"review_id": "7XpQhJTKed", "reviewer": "Reviewer_QGyL", "summary": "This manuscript introduces a generic formulation named ``Bidirectional Explicit Linear Multi-step'' (BELM) samplers for exact inversion of diffusion sampling. In contrast to DDIM Inversion, BELM inversion establishes the relationship between $x_{i-1}$, $x_i$, $x_{i+1}$, and $\\epsilon_\\theta(x_i, i)$. The authors prove that BELM is the generic version of EDICT and BDIA.", "questions": "Please see the weakness part.", "limitations": "Yes.", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1. The authors propose a new variable-stepsize-variable-formula (VSVF) linear multi-step scheme for exact inversion.\n2. The paper investigates the Local Truncation Error (LTE) within the BELM framework.\n3. O-BELM is designed by minimizing LTE to ensure minimized local error, and the experiments on COCO validate the exact inversion property of O-BELM.", "weaknesses": "1. How about the computation cost and latency of 2-step O-BELM compared with BDIA and EDICT?\n2. Lack of more applications such as ControlNet-based Image Editing exemplars, and more failure cases is better for analyzing the limitation of BELM.\n3. The local error of O-BELM seems higher than BDIA. Besides, for BDIA, $\\gamma$ is tuned for different quality and effect, when $\\gamma=1$, BDIA seems zero-stable and global convergence.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your valuable feedback, we will answer your questions one by one regarding these weaknesses/questions.\n\n> **Answer to Weaknesses 1:** ''How about the computation cost and latency of 2-step O-BELM compared with BDIA and EDICT?''\n\n\n- Theoretically, the computation cost bottleneck of diffusion sampling is the number of accesses to the noise network, $\\boldsymbol{\\varepsilon}_\\theta(\\mathbf{x}_i,i)$, which is also referred to as the Number of Function Evaluations (NFE).  In each iteration, O-BELM only accesses the noise network once, as demonstrated in equation 18. Therefore, for a number of steps equal to $N$, O-BELM requires an NFE equal to $N$, which is the same as DDIM and BDIA. However, EDICT doubles this requirement to $2N$. \nMoreover, O-BELM only requires the additional storage of a state, which is a relatively small extra memory cost compared to methods that need to store the entire chain, such as NMG [1] or P2P [2]. And as there is no parallelism in diffusion sampling, the latency is equivalent to the computational time cost.\n- Experimentally, we've conducted additional tests to compare the average time cost of different methods across sampling, editing, and reconstruction tasks. The results show that O-BELM does not incur any additional computational overhead compared to DDIM across all these tasks. Detailed information about these experiments can be found in the `Table 2 in attached PDF` and `Author Rebuttal`.\n\n| Method              |  Time-cost (s) of Image Sampling | Time-cost (s) of Image Editing  |Time-cost (s) of Image Reconstruction  |\n| :---------------- | :------ | :---- |:---- |\n| DDIM       |   6.67   | 13.30 | 13.20 |\n| EDICT      |   12.67   | 25.77 | 25.72 |\n| BDIA       |  6.59   | 13.37 |  13.28|\n| **O-BELM (Ours)** |  **6.53**   | **13.22** | **13.20** |\n\n*The details of the experiment can be found in the Global Rebuttal section.*\n\n> **Answer to Weaknesses 2:** ''Lack of more applications such as ControlNet-based Image Editing exemplars, and more failure cases is better for analyzing the limitation of BELM.''\n- **ControlNet-based Image Editing exemplars:** Despite we have conducted thorough theoretical analysis on the BELM framework and use experiments as validation to our theory. We agree that practical applications such as ControlNet-based Image Editing examples are beneficial for demonstrating the robustness of our O-BELM. Our BELM framework is built on the general diffusion IVP (equation 11), and can guarantee exact inversion (Proposition 2) and minimized error (Proposition 4) under all tasks based on diffusion ODE (PF-ODE). This means that the BELM framework is compatible with a wide variety of diffusion-based tasks, irrespective of the data type (such as image or language), the guidance method (unconditional, classifier-free, classifier-based, ControlNet-based, or adjoint ODE-based), or the network structure (whether it includes an attention layer or not). Following your valuable advice, we conducted experiments on ControlNet-based Image Editing using O-BELM and baseline methods, as shown in `Figure 1 in the Attached PDF`. The results demonstrate that O-BELM is effective in ControlNet-based Image Editing, preserving features from the original images while maintaining high-quality samples.\n- **failure cases:** Compared to EDICT and BDIA, one major advantage of O-BELM is its ability to prevent hard-to-tune hyperparameters and reduce the occurrence of failure cases, as demonstrated in Figure 8 of the original paper. However, we recognize the importance of analyzing failure cases and will include this in our revised version.\n\n> **Answer to Weaknesses 3:** ''The local error of O-BELM seems higher than BDIA. Besides, for BDIA, $\\gamma$ is tuned for different quality and effect, when $\\gamma=1$, BDIA seems zero-stable and global convergence.''\n\n- **The local error:** Our analysis, as outlined in Corollary 1 and Proposition 4, demonstrates that the asymptotic convergence rate of the local truncation error (LTE) for O-BELM is of order 3 with respect to the small variable $h$, whereas for BDIA it is of order 2. In mathematical terms, a larger order number signifies faster convergence to zero and smaller error [3]. Therefore, the local error of O-BELM is smaller than that of BDIA.\n\n- **zero-stable and global convergence:** The zero-stable property and global convergence property are rigorous and strict mathematical properties, serving as sufficient conditions for achieving stability and high accuracy in practical tasks. While BDIA performs well with some datasets for a certain number of steps when $\\gamma = 1$, there is no evidence to suggest that BDIA satisfies these stringent properties. In fact, in Figure 6 of BDIA's original paper [4], BDIA with $\\gamma = 1$ could lead to deformed edited images. In Table 5 of our original paper, BDIA with $\\gamma = 1$ could lead to low-quality sampling results on CIFAR-10 and CalebA-HQ datasets (FID $\\approx 100$).  Furthermore, there is no solid report indicating that the $\\gamma$ of BDIA has a clear high-level meaning.\n\n**References**\n\n[1] Cho, Hansam, et al. \"Noise map guidance: Inversion with spatial context for real image editing.\" arXiv:2402.04625 (2024).\n\n[2] Hertz, Amir, et al. \"Prompt-to-prompt image editing with cross attention control.\" arXiv preprint arXiv:2208.01626 (2022).\n\n[3] Rudin, Walter. Principles of mathematical analysis. Vol. 3. New York: McGraw-hill, 1964.\n\n[4] Zhang, Guoqiang, Jonathan P. Lewis, and W. Bastiaan Kleijn. \"Exact diffusion inversion via bi-directional integration approximation.\" arXiv preprint arXiv:2307.10829 (2023).", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ZRt3QmomQ5", "reviewer": "Reviewer_brf4", "summary": "The paper introduces the Bidirectional Explicit Linear Multi-step (BELM) sampler framework for exact inversion in diffusion models. The authors systematically investigate the Local Truncation Error (LTE) within the BELM framework and propose an optimal variant, O-BELM, which minimizes LTE for high sampling quality. Comprehensive experiments validate O-BELM's effectiveness in tasks like image reconstruction, editing, and interpolation.", "questions": "See weakness.", "limitations": "This paper does not explore the integration of high-accuracy exact inversion samplers such as O-BELM with more powerful image editing pipelines. Additionally, the application of high-accuracy exact inversion samplers like O-BELM to tasks beyond image processing remains uninvestigated in this work.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "Novel Framework: The BELM framework generalizes existing exact inversion samplers and introduces a bidirectional explicit constraint to ensure exact inversion.\n\nTheoretical Rigor: The paper provides a thorough theoretical analysis of the Local Truncation Error (LTE) and the stability and convergence properties of the proposed samplers.\n\nPractical Applications: Demonstrates the practical potential of O-BELM in various tasks such as image reconstruction, editing, and interpolation.", "weaknesses": "1.\tAs for downstream applications of diffusion inversion, style transfer [1] should also be included. I encourage the author could apply this method to some style transfer to show the robustness and effectiveness of this framework.\n2.\tMany punctuation marks at the end of the formulas are omitted by the authors. It is suggested to careful revise this article.\n3.\tThe comparative analysis of qualitative and quantitative results is deficient. It is suggested that the authors provide detail comparative analysis and conclusion in experiment section.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your valuable reviews, we will answer your questions one by one regarding these weaknesses/questions.\n\n> **Answer to Weaknesses 1:** ''As for downstream applications of diffusion inversion, style transfer [1] should also be included. I encourage the author could apply this method to some style transfer to show the robustness and effectiveness of this framework.''\n\nDespite we have conducted thorough theoretical analysis on the BELM framework and use experiments on downstream tasks as validation to our theory. We agree that more downstream applications of diffusion inversion, like style transfer, is beneficial. Therefore, we have evaluated the O-BELM and baseline methods on the style transfer sub-dataset in the PIE-Bench dataset [1].\n- The qualitative comparative analysis can be found in `Figure 2 in Attached PDF`. The results demonstrate that the O-BELM sampler ensures consistency and produces high-quality results in the style transfer task.\n- We also carry out a quantitative analysis on PIE-Bench style transfer sub-dataset. We employ Structure Distance [2] to evaluate whether the edited image preserves the structure of the original image during style transfer and use CLIP Similarity [3] to measure whether the edited image accurately reflects the meaning of target prompts. The following table shows that O-BELM achieves superior structure preservation according to the Structure Distance metric, validating that our O-BELM achieves the exact inversion property (as per Proposition 2 of the original paper). It is also demonstrated that O-BELM achieves better CLIP Similarity, validating that O-BELM reduces sampling error (as per Proposition 4 of the original paper).\n\n| Style Transfer Methods              | Structure Distance$\\times10^3$ ($\\downarrow$) | CLIP Similariy ($\\uparrow$) |\n| :---------------- | :------ | :---- |\n| DDIM       |   71.1   | 24.82 | \n| EDICT      |   19.5   | 24.39 | \n| BDIA       |  25.3   | 23.20 |  \n| **O-BELM (Ours)** |  **18.0**   | **25.09** | \n\n*The details of the experiment can be found in the Global Rebuttal section.*\n\n> **Answer to Weaknesses 2:** ''Many punctuation marks at the end of the formulas are omitted by the authors. It is suggested to careful revise this article.''\n\nThank you for your detailed review. We have conducted a thorough second revision and corrected all typographical errors. We assure you that all necessary punctuation marks will be included in the formulas in our revised version.\n\n\n> **Answer to Weaknesses 3:** ''The comparative analysis of qualitative and quantitative results is deficient. It is suggested that the authors provide detail comparative analysis and conclusion in experiment section.''\n\nOur theoretical investigation and comparison are thorough and precise. We not only establish the exact inversion property (Proposition 2 of the original paper), local error property (Proposition 4 of the original paper), stability property, and global convergence property (Proposition 5 of the original paper) of O-BELM, but also scrutinize the theoretical properties of other methods. This comprehensive analysis allows us to provide a rigorous theoretical comparison of different samplers in Table 1 of the original paper.\n\nWe understand that more comparative analysis of qualitative and quantitative results in downsteam task is still beneficial. Therefore, we not only conduct qualitative and quantitative comparative experiments in style-transfer tasks as mentioned above, but also carry out a thorough quantitative comparison of experiments on image editing tasks. We follow the experimental design of DirectingInv [1] to evaluate methods on the PIE-Bench image editing dataset using eight metrics covering four aspects:  structure distance [2], background preservation (PSNR, LPIPS [4], MSE, and SSIM [5] outside the annotated editing mask), edit prompt-image consistency (CLIPSIM [3] of the whole image and regions in the editing mask) and time cost. These results can be found in `Table 1 in the attached PDF` and `Table 2 in the attached PDF`. It is shown that O-BELM outperform baselines in this benchmark.\n\n**References**\n\n[1] Ju, Xuan, et al. \"Direct inversion: Boosting diffusion-based editing with 3 lines of code.\" arXiv preprint arXiv:2310.01506 (2023).\n\n[2] Tumanyan, Narek, et al. \"Splicing vit features for semantic appearance transfer.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022.\n\n[3] Wu, Chenfei, et al. \"Godiva: Generating open-domain videos from natural descriptions.\" arXiv preprint arXiv:2104.14806 (2021).\n\n[4] Zhang, Richard, et al. \"The unreasonable effectiveness of deep features as a perceptual metric.\" Proceedings of the IEEE conference on computer vision and pattern recognition. 2018.\n\n[5] Wang, Zhou, et al. \"Image quality assessment: from error visibility to structural similarity.\" IEEE transactions on image processing 13.4 (2004): 600-612.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "pmNsyxwN4r", "reviewer": "Reviewer_eBdS", "summary": "This paper introduces a novel method for inverting real images into a diffusion model. It presents Bidirectional Explicit Linear Multi-step (BELM) samplers aimed at minimizing the mismatch in DDIM inversion.", "questions": "Please see in weakness", "limitations": "N/A", "rating": 6, "confidence": 5, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "The proposed Bidirectional Explicit Linear Multi-step (BELM) samplers seems reasonable, and outputforms DDIM sampler.", "weaknesses": "1. The paper lacks discussion and comparison with several related works such as NMG, EDICT, DirectingInv, ProEdit, ReNoise, and others. These works also aim to address the limitations of DDIM inversion through various proposed solutions. It is essential for the authors to provide a thorough discussion and comparative analysis with these existing methods.\n\n2. The evaluation metrics used in the paper may not be sufficiently convincing. DirectingInv, for example, introduces several metrics specifically tailored for evaluating editing tasks, which are generally considered more reliable. Furthermore, DirectingInv also establishes a standard benchmark dataset for diffusion inversion tasks. It would be beneficial for the authors to conduct experiments using this benchmark dataset to ensure a comprehensive study and rigorous evaluation of their proposed method.\n\n3. In terms of reconstruction performance compared to AE, EDICT, and BDIA, O-BELM does not demonstrate any improvement.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your valuable reviews, we will answer your questions one by one regarding these weaknesses/questions.\n\n> **Weaknesses 1:** The paper lacks discussion and comparison with several related works such as NMG, EDICT, DirectingInv, ProEdit, ReNoise, and others. These works also aim to address the limitations of DDIM inversion through various proposed solutions. ...\n\nWe understand that there are several techniques proposed to address the inexact inversion issue of DDIM within the context of **classifier-free-text-guided image editing**. These include the methods you mentioned, such as NMG[1], DirectingInv[2], ProEdit[3], and ReNoise[4], as well as others like NPT [5] or NT [6]. We argue that the proposed O-BELM and these techniques should not be considered as comparative algorithms for following reasons.\n- **These methods are orthogonal.** O-BELM modifies the discretization formula to achieve exact inversion, while these techniques adjust the classifier-free-guidance mechanism. They address this problem from different directions.\n- **They can be used together in the classifier-free-text-guided image editing.** Take DirectingInv as an instance, its inversion is just DDIM inversion and its forward process encompasses two state-interacting DDIM forward processes with different prompts. We can substitute the DDIM inversion/forward in DirectingInv to be O-BELM inversion/forward and get O-BELM+DirectingInv. We conduct experiments of origin DDIM+DirectingInv and O-BELM+DirectingInv in `Table 1 in attached PDF`, showing that O-BELM can be integrated with DirectingInv to further enhance the performance.\n- **Their working scenarios differ.** The O-BELM is built on the general diffusion IVP (equation 11), and can guarantee exact inversion (Proposition 2) and minimized error (Proposition 4) under all tasks based on diffusion ODE (PF-ODE). O-BELM can always converge to underlying IVP solution as demonstrated by Proposition 5. This means that the BELM framework is compatible with a wide variety of diffusion-based tasks, irrespective of the data type (images or words), the task type (editing or interpolation), the guidance method (unconditional, classifier-free, classifier-based, or adjoint ODE-based), or the network structure (whether it includes an attention layer or not). On the contrary, these techniques are developed specific for classifier-free-text-guided image editing task.\n\nWe will add a paragraph to discussion these works.\n\n> **Weaknesses 2:** The evaluation metrics used in the paper may not be sufficiently convincing. DirectingInv, for example, introduces several metrics specifically tailored for evaluating editing tasks, which are generally considered more reliable. Furthermore, DirectingInv also establishes a standard benchmark dataset for diffusion inversion tasks. ...\n\nAlthough we have conducted thorough theoretical analysis on BELM framework and use experiments as validation to our theory, we agree that comprehensive evaluation metrics on standard benchmark dataset is beneficial. To illustrate the efficiency of our proposed O-BELM at downstream image editing task, we follow your valuable advice to compare O-BELM with baseline methods on the standard benchmark PIE-Bench established by DirectingInv [2]. We follow the experiment design of DirectingInv to evaluate in terms of eight metrics covering four aspects:  structure distance, background preservation (PSNR, LPIPS, MSE, and SSIM outside the annotated editing mask), edit prompt-image consistency (CLIPSIM of the whole image and regions in the editing mask) and inference time. These results can be found in `Table 1 in the attached PDF` and `Table 2 in the Author Rebuttal`. It is shown that O-BELM outperform baselines in this benchmark.\n\n\n> **Weaknesses 3:** In terms of reconstruction performance compared to AE, EDICT, and BDIA, O-BELM does not demonstrate any improvement.\n\nThis is because O-BELM, BDIA, and EDICT all fall within the BELM framework, and as Proposition 2 proves, any BELM method possesses the exact inversion property. However, since SD-1.5 is a latent diffusion model (LDM), errors will be inevitably introduced during the encoding/decoding process of the AutoEncoder (AE) component of SD model. It is a prevailing practice to take the reconstruction error of AE as a lower bound of LDM pixel-level reconstruction error[7]. O-BELM has already reached this lower bound.\n\nWe have conducted an additional experiment to assess the reconstruction error in the latent space of O-BELM and other baseline methods.\n| Method|  MSE on latents (10 steps) | MSE on latents (20 steps)    | MSE on latents (100 steps)  |\n| :--| :- | :-|:-|\n|DDIM|0.414|0.243|0.041|\n|EDICT|0.000|0.000|0.000|\n|BDIA|0.000|0.000|0.000|\n|**O-BELM (Ours)**|**0.000**|**0.000** |**0.000**|\n\nIt is demonstrated that O-BELM (along with BDIA and EDICT) achieves exact inversion at the latent level, resulting in zero reconstruction error. O-BELM outperforms BDIA and EDICT in terms of sampling accuracy due to reduced local error (as shown in Table 1 of the original paper), not in terms of reconstruction error.\n\n**References**\n\n[1] Cho, Hansam, et al. \"Noise map guidance: Inversion with spatial context for real image editing.\" arXiv:2402.04625 (2024).\n\n[2] Ju, Xuan, et al. \"Direct inversion: Boosting diffusion-based editing with 3 lines of code.\" arXiv:2310.01506 (2023).\n\n[3] Han, Ligong, et al. \"Proxedit: Improving tuning-free real image editing with proximal guidance.\" WACV 2024.\n\n[4] Garibi, Daniel, et al. \"ReNoise: Real Image Inversion Through Iterative Noising.\" arXiv:2403.14602 (2024).\n\n[5] Miyake, Daiki, et al. \"Negative-prompt inversion: Fast image inversion for editing with text-guided diffusion models.\" arXiv:2305.16807 (2023).\n\n[6] Mokady, Ron, et al. \"Null-text inversion for editing real images using guided diffusion models.\" CVPR 2023.\n\n[7] Wallace, Bram, Akash Gokul, and Nikhil Naik. \"Edict: Exact diffusion inversion via coupled transformations.\" CVPR 2023.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "CAHkast7vT", "reviewer": "Reviewer_HZYw", "summary": "This paper introduces a systematic framework, referred to as BELM, that is designed for the specific task of exact inversion of diffusion sampling. This framework encompasses several existing intuitive exact inversion samplers as its special cases. Subsequently, the authors derive an optimal variant within this framework via local truncation error minimization, named O-BELM, and investigate its theoretical properties, including zero-stability and convergence. Experimental results demonstrate that O-BELM can achieve exact inversion and high-quality sampling. The authors further explore its potential applications in downstream computer vision tasks.", "questions": "1 This paper examines the zero-stable property of the O-BELM algorithm and baseline samplers. However, I've noticed that there are other stability properties, such as Absolute-stability or Butcher-stability, for ODE solvers. I'm interested in knowing how the samplers discussed in this paper fare in regards to these stability terms.\n2 I'm curious to understand how adopting this new sampler might affect the marginal distribution of the diffusion models. Could you elaborate on the potential impacts this could have?", "limitations": "The authors have acknowledged the unexplored semi-linear structure as a limitation in the design of the BELM. \nAlso, the effects of utilizing O-BELM in advanced image editing like P2P remain unexplored, which is a limitation of O-BELM as well.\nAdditionally, the impact of the paper is primarily confined to the sampling sub-area within the diffusion-based model community.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1 This paper represents the first attempt to formalize the task of exact inversion of diffusion sampling as a rigorous mathematical problem. It highlights that bidirectional explicit property is a sufficient condition for a general sampler to achieve exact inversion. \n2 This paper introduces O-BELM as an efficient sampler intended for practical use. Additionally, the paper offers theoretical guarantees for the newly proposed sampler, albeit under mild assumptions.\n3 This paper derived the form of local truncation error for general linear multistep samplers within the context of diffusion models.\n4 A comprehensive range of experiments has been conducted to confirm that the O-BELM exhibits both exact inversion and high-quality sampling capabilities.\n5 This paper is well-articulated and clearly presented.", "weaknesses": "1 There is a typographical error in the caption of Table 3. \n2 The information regarding the scheduler setting used in the experiments has not been included.\n3 Some mathematical derivations in the paper are too succinct, making them difficult to follow. For instance, the transition from equation (30) to equation (31) is not straightforward as it involves a series of Taylor's expansions.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your appreciation of our work, we will answer your questions one by one regarding these weaknesses/questions.\n\n\n> **Weaknesses 1:**  ''There is a typographical error in the caption of Table 3.'' \n\nWe apologize for the oversight. We have now corrected the error and ensured that the caption accurately reflects the contents of the table. The caption of the Table 3 should be \"Comparison of different samplers on FID score( $\\downarrow$) for the task of unconditional generation.\" We have conducted a thorough second revision and fix all typographical errors. \n\n\n> **Weaknesses 2:** ''The information regarding the scheduler setting used in the experiments has not been included.'' \n\nWe understand the importance of this detail for the complete understanding and replication of our experiment. For these pretrained diffusion models, we utilize the noise scheduler delineated in their respective configurations, applying it uniformly across all our experiments. Given that our experiments do not necessitate the training or fine-tuning of diffusion models, there is no need to devise a new scheduler setting.\nWe have now added the necessary information about the scheduler setting in the Appendix C.6 section of the paper. \n\n\n> **Weaknesses 3:** ''Some mathematical derivations in the paper are too succinct, making them difficult to follow. For instance, the transition from equation (30) to equation (31) is not straightforward as it involves a series of Taylor's expansions.''\n\nWe acknowledge that some of the transitions, such as from equation (30) to equation (31), may not be easy to follow. We have provided more detailed explanation of these derivations to ensure that the logic and process of our mathematical derivations are clear and easy to follow.\nTo elucidate the derivation from Equation (30) to Equation (31), we have included an intermediate illustration as follows:\n$$\n\\begin{aligned} \n& \\sum\\_{j=1}^{k} a\\_{i,j} \\bar{x}\\_{i-1+j} +\\sum\\_{j=1}^{k-1}b\\_{i,j} h\\_{i-1+j}\\bar{\\boldsymbol{\\varepsilon}}\\_\\theta(\\bar{x}\\_{i-1+j},\\bar{\\sigma}\\_{i-1+j})\\\\\\\\\n=&\\sum\\_{j=1}^{k} a\\_{i,j} ( \\bar{x}\\_{i-1} + \\sum\\_{l=1}^{2k-2} \\frac{1}{(l)!}(\\sum\\_{m=0}^{j-1}h\\_{i+m})^{l}\\bar{\\boldsymbol{\\varepsilon}}^{(l-1)}\\_\\theta(\\bar{x}(t\\_{i-1}),\\bar{\\sigma}\\_{i-1}) ) \\\\\\\\\n&+\\sum\\_{j=1}^{k-1}b\\_{i,j} h\\_{i-1+j}( \\sum\\_{l=1}^{2k-2} \\frac{1}{(l-1)!}(\\sum\\_{m=0}^{j-1}h\\_{i+m})^{l-1}\\bar{\\boldsymbol{\\varepsilon}}^{(l-1)}\\_\\theta(\\bar{x}(t\\_{i-1}),\\bar{\\sigma}\\_{i-1}) )\\\\\\\\\n&+\\mathcal{O}(({\\sum\\_{m=0}^{k-1}h\\_{i+m}})^{(2k-1)})\\\\\\\\\n=& \\sum\\_{j=1}^{k} a\\_{i,j} \\bar{x}\\_{i-1} + \\sum\\_{l=1}^{2k-2}(\\frac{1}{(l!) }\\sum\\_{j=1}^{k}a\\_{i,j}( \\sum\\_{m=1}^{j}h\\_{i+m-1})^{l})\\bar{\\boldsymbol{\\varepsilon}}^{(l-1)}\\_\\theta(\\bar{x}(t\\_{i-1}),\\bar{\\sigma}\\_{i-1})\\\\\\\\\n&+\\sum\\_{l=1}^{2k-2}(\\frac{1}{((l-1)!) }\\sum\\_{j=1}^{k-1}b\\_{i,j}h\\_{i+j-1}( \\sum\\_{m=1}^{j}h\\_{i+m-1})^{l-1})\\bar{\\boldsymbol{\\varepsilon}}^{(l-1)}\\_\\theta(\\bar{x}(t\\_{i-1}),\\bar{\\sigma}\\_{i-1})\\\\\\\\\n&+\\mathcal{O}(({\\sum\\_{m=0}^{k-1}h\\_{i+m}})^{(2k-1)}) \n\\end{aligned}\n$$\nWe have also conducted a comprehensive review of our mathematical derivations, enhancing them with additional details to improve their clarity and ease of understanding.\n\n\n\n> **Questions 1:** ''This paper examines the zero-stable property of the O-BELM algorithm and baseline samplers. However, I've noticed that there are other stability properties, such as Absolute-stability or Butcher-stability, for ODE solvers. I'm interested in knowing how the samplers discussed in this paper fare in regards to these stability terms.'' \n\nZero-stability and Absolute-stability (abbreviated as A-stability) are the most commonly used stability conditions. The A-stability is quite similar to zero-stability, with the primary difference being that A-stability measures the discrete step-size case, while zero-stability measures the limiting step-size case. However, it has been demonstrated that A-stability is extremely demanding and no explicit linear multistep method can satisfy A-stability as shown by the Second Barrier Theorem of Dahlquist [1]. Given the heavy cost of using implicit methods in the diffusion model area, A-stability is not suitable for analysis in the diffusion sampler. \n\nOn the other hand, Butcher-stability (abbreviated as B-stability) is typically used to analyze the stability property of Runge-Kutta type ODE samplers. For these samplers, B-stability can derive zero-stability [2]. However, for non-RK type ODE samplers, the implications of the Butcher-stability condition remain unclear.\n\nTherefore, it is reasonable to select zero-stability as the criterion for assessing stability in the analysis of diffusion samplers.\n\n\n> **Questions 2:** ''I'm curious to understand how adopting this new sampler might affect the marginal distribution of the diffusion models. Could you elaborate on the potential impacts this could have?''\n\nThe marginal distribution of the proposed sampler will also conform to the marginal distribution implied by the diffusion ODE.\nIntuitively, the approximated $x_t$ obtained from the discretized sampler serves as a proxy for the underlying truth $\\mathbf{x}(t)$ of the diffusion initial value problem (IVP). \nTheoretically, our convergence analysis ensures that our approximated $\\mathbf{x}_t$ will indeed converge to this underlying truth. The DDIM and other deterministic diffusion samplers, such as the DPM-solver, are also constructed to simulate this diffusion IVP problem. Their marginal distributions are identical, when omitting discretization errors.\nExperimental results show that our O-BELM can generate high-quality samples, effectively demonstrating the capability of O-BELM to accurately model the underlying data distribution at time $T$.\n\n**References**\n\n[1] G. Dahlquist, A special stability problem for linear multistep methods, BIT 3, 27–43, 1963.\n\n[2] Theorem 357D, Butcher, John Charles. Numerical methods for ordinary differential equations. John Wiley & Sons, 2016.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
## To All Reviewers
We sincerely appreciate the time and effort you have dedicated to reviewing our paper! Your valuable feedbacks have been carefully considered, and we have provided point-to-point responses to your reviews in respective rebuttals. We remain open to any additional feedback you may have. If you feel it is appropriate based on our responses, we would be extremely grateful if you could consider raising our score.


## Attached PDF
In the attached PDF document, we have provided several figures and tables. The details of the experiments are listed as follows:

- **Figure 1 -- ControlNet-based Editing Results**
We evaluated O-BELM and baseline algorithms on ControlNet-based image editing tasks, which included canny-based and depth-map-based editing. The editing hyperparameters are chosen the same as our original paper. The ControlNet hyperparameters were kept at their default values, consistent across all methods. We set the number of steps to 100. The canny images were obtained using the Canny function from the opencv-python library, and the depth-map model used was Intel/dpt-large (https://huggingface.co/Intel/dpt-large). We use stable-diffusion-v1-5 model (https://huggingface.co/runwayml/stable-diffusion-v1-5) as our base model.


- **Figure 2 -- Style Transfer Results**
We evaluated O-BELM and baseline algorithms on style transfer tasks using the style transfer sub-dataset of the PIE-Bench dataset (https://paperswithcode.com/dataset/pie-bench). The editing hyperparameters were selected to match those in our original paper. We use stable-diffusion-2-base model (https://huggingface.co/stabilityai/stable-diffusion-2) as our base model. 


- **Table 1 -- Quantitative Evaluation in Image Editing on PIE Benchmark**
We evaluated O-BELM and baseline algorithms on image editing tasks using the PIE-Bench dataset (https://paperswithcode.com/dataset/pie-bench). The editing hyperparameters were selected to match those in our original paper. We use stable-diffusion-2-base model (https://huggingface.co/stabilityai/stable-diffusion-2) as our base model. We follow the experimental design of DirectingInv [1] to evaluate methods on the PIE-Bench image editing dataset using seven metrics covering three aspects:  structure distance [2], background preservation (PSNR, LPIPS [3], MSE, and SSIM [4] outside the annotated editing mask), edit prompt-image consistency (CLIPSIM [5] of the whole image and regions in the editing mask). We utilize the dino_vitb8 model (https://huggingface.co/facebook/dino-vitb8) and clip-vit-large-patch14 (https://huggingface.co/openai/clip-vit-large-patch14) for metric evaluations.


-  **Table 2 -- Time Costs Comparison on PIE Benchmark**
We assessed the time costs of O-BELM and baseline algorithms using the PIE-Bench dataset (https://paperswithcode.com/dataset/pie-bench), which included tasks such as image generation, image editing, and image reconstruction. The number of steps was set to 50. We employed the stable-diffusion-2-base model (https://huggingface.co/stabilityai/stable-diffusion-2) as our base model and conducted tests on a single NVIDIA V100 chip and an Intel Xeon Platinum 8255C CPU.




**Reference**

[1] Ju, Xuan, et al. "Direct inversion: Boosting diffusion-based editing with 3 lines of code." arXiv preprint arXiv:2310.01506 (2023).

[2] Tumanyan, Narek, et al. "Splicing vit features for semantic appearance transfer." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022.

[3] Zhang, Richard, et al. "The unreasonable effectiveness of deep features as a perceptual metric." Proceedings of the IEEE conference on computer vision and pattern recognition. 2018.

[4] Wang, Zhou, et al. "Image quality assessment: from error visibility to structural similarity." IEEE transactions on image processing 13.4 (2004): 600-612.

[5] Wu, Chenfei, et al. "Godiva: Generating open-domain videos from natural descriptions." arXiv preprint arXiv:2104.14806 (2021).

Meta-review: The paper proposes BELM, a generic formulation of exact inversion samplers. This includes existing exact inversion samplers and the analysis of the LTE allows to optimize the method. The reviewers appreciated the novelty of the framework, the theoretical analysis of the LTE as well as the practical potential of BELM. The rebuttal clarified some concerns (difference with existing classifier-free-text-guided image editing works) and provided additional experiments which were appreciated by the reviewers.

All the reviewers recommend acceptance, and I concur with this evaluation.
Decision: Accept (poster)