FreeInv: Free Lunch for Improving DDIM Inversion
Authors: Yuxiang Bao
Huijie Liu
xun gao
Huan Fu
Guoliang Kang
Venue: NeurIPS 2025 poster ()
Keywords: DDIM inversion
diffusion model
Primary area: deep_learning

Abstract: Naive DDIM inversion process usually suffers from a trajectory deviation issue, i.e., the latent trajectory during reconstruction deviates from the one during inversion. To alleviate this issue, previous methods either learn to mitigate the deviation or design cumbersome compensation strategy to reduce the mismatch error, exhibiting substantial time and computation cost. In this work, we present a nearly free-lunch method (named FreeInv) to address the issue more effectively and efficiently. 
In FreeInv, we randomly transform the latent representation and keep the transformation the same between the corresponding inversion and reconstruction time-step. 
It is motivated from a statistical perspective that an ensemble of DDIM inversion processes for multiple trajectories yields a smaller trajectory mismatch error on expectation.
Moreover, through theoretical analysis and empirical study, we show that FreeInv performs an efficient ensemble of multiple trajectories. FreeInv can be freely integrated into existing inversion-based image and video editing techniques. Especially for inverting video sequences, it brings more significant fidelity and efficiency improvements. Comprehensive quantitative and qualitative evaluation on PIE benchmark and DAVIS dataset shows that FreeInv remarkably outperforms conventional DDIM inversion, and is competitive among previous state-of-the-art inversion methods, with superior computation efficiency.

FreeInv: Free Lunch for Improving DDIM Inversion
Yuxiang Ba Huijie Liu'", Xun Gao", Huan Fu?, Guoliang Kang't Beihang University, HUJING Digital Media & Entertainment Group (bbao5802, liuhuijic6410, kgl.prml/ @gmail.com
Abstract
Naive DDIM inversion process usually suffers from trajectory deviation issue, i.e., the latent trajectory during reconstruction deviates from the one during inversion. To alleviate this issue, previous methods either learn to mitigate the deviation or de- sign a cumbersome compensation strategy to reduce the mismatch error, exhibiting substantial time and computation cost. In this work, present nearly free-lunch method (named FreeInv) to address the issue more effectively and efficiently. In FreeInv, randomly transform the latent representation and keep the transforma- tion the same between the corresponding inversion and reconstruction time-step. It is motivated from statistical perspective that ensemble of DDIM inversion processes for multiple trajectories yields a smaller trajectory mismatch error on expectation. Moreover, through theoretical analysis and empirical study, show that FreeInv performs an efficient ensemble of multiple trajectories. FreeInv can be freely integrated into existing inversion-basee image and video editing techniques. Especially for inverting video sequences, it brings more significant fidelity and efficiency improvements. Comprehensive quantitative and qualitative evaluation on PIE benchmark and DAVIS dataset shows that FreeInv remarkably outperforms conventional DDIM inversion, and is competitive among previous sfate-of-the-art inversion methods, with superior computation efficiency.
Introduction
The recent developments of large-scale text-guided diffusion models, e.g. Stable Diffusion 36 have fueled the rise of image and video editing. To ensure the editing results are faithful to the original input, these methods typically employ Denoising Diffusion Implicit Models (DDIM) inversion [38 techniques. The DDIM inversion process involves mapping an image back to its noisy latent representation, from which the original image is expected to be reconstructed with high fidelity.
However, the reconstruction process usually suffers from trajectory deviation issue, i.e., the latent trajectory of the reconstruction process deviates from that of the inversion process, which means the error of latent representations between inversion and reconstruction may be accumulated along the denoising steps. This is because the ideal DDIM inversion and reconstruction process is theoretically based on the local linear assumption, i.e., 6 (Ir) R <0(x++1), where co(-) denotes the noise predicted by neural network parameterized with 0. The assumption usually does not hold in practice. Thus, the error introduced in each step will be accumulated and lead to non-negligible deviation between the inversion and reconstruction trajectory, hampering the quality of reconstructed and edited results.
To mitigate the trajectory deviation, previous works focus on reducing the mismatch error of latent representations between inversion and reconstruction processes, i.e., reducing |cp (c.(2.) - eg (a(+++)) 1] in each step. Learning-baat based methods [5] 5 aim to minimize the mismatch error through back- propagating the gradients to the null-text embedding (see Fig. Ta)). Another group of methods [[7
'qqual contribution. 'Corresponding author, Code is available at https://githubb com/yuxiangbao/ FreeInv Project page is available htttp:://yuxiaaaaaeee github.ioo/Erelln..
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Irpui
U. DDI
UANN DIT
U-NeL
UANe
U.Nel FDIT
U-Miee DDT
Xy-1
Xg
Componsaae
Pc -21
(a) Null-text Inversion
(b) PnP Inversion
(c) Freelnv (Ours)
Figure I: Illustration of different ways to mitigate the trajectory deviation in DDIM inversion. The
small image above the networks denotes the input. (a) Null-text Inversion (25 reduces the mismatch eπror via optimizing a null-text embedding. (b) PnP Inversion [17] saves the reconstruction error of each step in memory and makes compensation during the reconstruction or editing process. (c) FreeInv improves the DDIM inversion by applying random transformation (e.g. rotation) to the input latent, with negligible time or memory costs.
50 15 6 memorize or store the errors generated in each time-step, and exploit them to compensate for the latent representation deviation in the reconstruction procedure (see Fig. (b)) Although these techniques improve the reconstruction fidelity, they introduce high computation (time and memory) costs. Especially when inverting videos with hundreds of frames, they are cumbersome and inefficient.
In this paper, propose method named FreeIny to deal with the trajectory deviation issue in nearly free-lunch manner. Specifically, from a statistical perspective, we find that an ensemble of the inversion and the reconstruction processes for multiple image samples yields smaller mismatch error. In detail, we average the predicted noise from multiple images for inversion and reconstruction and find the mismatch error can be suppressed. Further, based our theoretical analysis and empirical observations. we propose FreeInv, which is a simplified version of performing trajectory ensemble. As illustrated in Fig. [c), in FreeIny, randomly transform (e-g., rotate) the latent representation and keep the transformation (e.g., rotation angles) the same between the corresponding inversion and reconstruction step Thanks operational simplicity, FreeInv can be readily plugged into U-Net [13 36) and DiT (29 I 77 architectures. Extensive experiments on PIE benchmark [17]
demonstrate that FreeInv significantly outperforms the DDIM baseline, and achieves performance
comparable or superior to existing state-of-the tit-hhh-rttt art inversion approaches [25 45
46
50
17
15
9 47
For efficiency, our method introduces negligible costs compared the DDIM baseline and consumes much smaller time and memory than all previous inversion methods tailored for mitigating the trajectory deviation. Due to the efficient design, FreeInv well-suited for the inversion of video sequences. When combined with TokenFlow (10,, FreeInv exhibits superior reconstruction/editing fidelity and efficiency, compared to previous state-of-the-arr inversion method STEM-Inv [20].
In nutshell, our contribution is summarized as follows
From the statistical perspective, find that ensemble of trajectories for multiple images can effectively reduce the latent mismatch error between inversion and reconstruction processes, thereby improving reconstruction fidelity effectively.
We propose a method named FreeInv to perform an efficient ensemble of trajectories, i.e. we randomly transform (e.g., rotate) the latent representations and keep the transformation (e.g.. rotation angles) at each step the same between the inversion and reconstruction processes.
FreeIny is compatible with both U-Net and DiT architectures. Its efficient design enables it to be applied not only to image reconstruction but also to video sequences. Extensive experiments demonstrate that FrecIny achieves reconstruction performance on par with, or even exceeding, existing inversion methods, while offering significantly improved efficiency.
2
Related Works
Text guided image editing. Recently, text-guided diffusion models 26 offer significantly more powerful and flexible image editing capabilities compared to previous
37

---

methods [66 53 27
Text-guided image editing requires editing the image following the prompt while maintaining the main components of the original image. Imagic [9 and UniTune 43 achieve this goal through restrictive fine-tuning of the pretrained model. Blended diffusion
28
44
48
22
35
33
54 18
2 and GLIDE [26] utilize the provided mask to control the editing region, which is not user-friendly. To realize controllable and precise editing with prompt, Prompt-to- Prompt [12] introduces the feature and attention map from the DDIM reconstruction process into the editing process, achieving promising editing results.
DDIM inversion for image/video editing. In image/video editing tasks, DDIM inversion technique is widely adopted [12 42 10, 32
However, the editing results are always constrained by the reconstruction quality. Naive DDIM inversion usually suffers from trajectory
23
3
14
51 8).
deviation issue, leading to distorted reconstruction. A lot of works [25 (4] [1]
45
SD 17 46
have been proposed to alleviate this issue. Null-text Inversion (25 minimizes the reconstruction error
50 6 choose to save the error with extra memory occupation and make compensation during editing process. EDICT designs a parallel inversion and reconstruction process to realize accurate preservation. Although these works are well applied image inversion, but few of them are suited for processing video sequences for the increased computation burden. STEM Inversion (2) is designed to invert video sequences, but it still requires iterations to calculate a compact video representation. In comparison, FreeInv offers a free-lunch and more general alternative for both image and video inversion.
at each time-step through optimizing the null-text embedding. Other works 17
45]
3 Methodology
3.1 DDIM Inversion Revisiting
Recently, Song al. 38 proposed the Denoising Diffusion Implicit Model (DDIM), which serves as an efficient technique for diffusion model sampling, following the formula
Ft
I++1
Te
co (#141, 1),
Var+1
V
I at
1)t 04++1
Oz
where F2 denotes the latent at time-step f, and cg (::-) refers to the noise prediction network with parameter 0. Note that there are no stochastic terms in the formula, meaning that the sampling procedure is deterministic. Therefore, if we inverse the DDIM sampling process from / = to | T. where T denotes the total sampling steps, we can get the initial noisy latent of the original image. This inversion process can be formulated as
Et+1
+ 11t (Ie, t + 1).
(2)
Vor
If performing DDIM sampling on the inverted noisy latent can recover the original image ideally, may greatly benefit diffusion-based image/video editing tasks 9 17), which aims to modify part of the image while keeping the rest unchanged.
However, due to the discrete nature, slight error exists in each reconstruction time-step, resulting in flawed reconstruction. In order to quantitatively describe the reconstruction error, let us consider the adjacent time-step f. and + 1, where the latent is inverted from time-step / to + 1 and then goes back to The error can be formulated as
|
+ x, = Vann Iea (Ie41, ++
- 1)
(I,, t + 1)
where | refeers to calculating demeent-wise absolute value and we utilize x; and to represent the reconstructed latent and the inverted latent, respectively. Because a₂ the predefined in DDIM schedule that keeps unchanged. the reconstruction error is determined by the mismatch error (eg (IZ+1, + 1) eg (xie, / 1) The ideal DDIM inversion and reconstruction process assumes that (I++1, 1/411 eg (Iz, + 1). Such an error will be accumulated along the time-steps and become non-negligible. As a result, the trajectory of the reconstruction deviates from the one of the inversion process, thus hampering the fidelity of reconstruction results.
3

---

U-Nett DIT
U.Net:
U-
Rox (Ka, Pa)
(ot((s:11) (o)
Ro((ss.91)
Rot Mh-1)
U DI
USeN DIT
LANSI PARII DIT
Roc(K Vu)
Rot(st.
Rot(x Ror(xf-1.PPr-)
Rot vh--1)
Inversion Trajectory
Reconstruction Trajectory
Figure 2: Detailed illustration of FreeInv. We employ rotation :ot(:s)) as the transformation f(-) for example. During both the inversion and reconstruction phases, rotate the latent representation with the same angle 1/s at the 1-th time-step, where th randomly sampled.
3.2 Multi-Branch DDIM Inversion
As discussed in Sec. 3.1 the key of high-fidelity DDIM inversion is minimize the mismatch error in each time-step. Inspired by the ensemble techniques [11 in various computer vision tasks, propose to ensemble multiple trajectories to enhance reconstruction fidelity by constructing multi-branch DDIM inversion (MBDI) and reconstruction.
Specifically, when inverting one image, arbitrary number of different images are also sampled as auxiliary samples and follow the parallel inversion and reconstruction trajectory. In each time-step, instead of invertingrreconsttucinng each branch independently, we make ensemble of the noise predictions from all the branches to invert/reconstruet all the branches simultancously. Specifically, average all the noise predictions at each time-step, which is
6 (or, 1)
Σ Xeeo (z), 1),
where A 1d 22, - = I+ # In Eq. 44) ot refers to the latent of the i-th branch at time-step t. For reconstruction, the ensemble noise 60 (Te+1. 1) can be obtained in a similar way. Compared with Eq. [I] and (2). the latent in each branch is inverted and reconstructed with S. (Ix, / + 1) and -6,5 (Iz+1, t instead of Eg (Iz, + 1) and eø (Iz+1, t + 1) respectively, Note that this modification does not affect the deterministic nature of the procedure in the sense that the original image can still be reconstructed theoretically.
MBDI reduces mismatch error. With MBDI, the mismatch error between estimated noise in inversion and reconstruction is
1 (#++1, t E (x
[ea (#l+1, 1) (2],
(5)
N
According to the triangle inequality we get
N lea (zi41, (c44,,+11 (xt, (+
|e (Ie+1,
E (x1,
(6)
N
The right side of the inequality is the mean error of each independent branch. Given that the initial samples ai are independently drawn from the natural image distribution and the inversion procedure is deterministic, the right side of Eq. can be considered as an unbiased estimation for the expectation of mismatch error E (xooppzz)) {lea ( 1) 6g (æ+, + 1) []- Given that the reconstruction error proportional to the mismatch enror in Eq. (3), the reconstruction error of multi-branch is no larger than that of single branch in each step on expectation. Therefore, performing the ensemble of multiple trajectories may yield better reconstruction results compared to single branch.

---

3.3 Free-lunch DDIM Inversion
Though effective, the computation and memory cost of N-branch inversion framework high, and the cost is approximately N times more than the standard DDIM inversion. Thus, in Freeinv, we make two modifications improve the efficiency.
(1) One-time MC sampling each time-step. Different from deterministic À in MBDI in Eq. (4). we introduce a random variable = X5, X ~ Categorical (N= No.... N) (in the following, we omit superseript which denotes the time-step for simplicity). Then, we obtain & (It, E Aeso (at,t 1). The expectation of < (ar, + 1) over can be denoted as
1 Σ co (z)
Ex (I,, + 1)]
which means Ex/e,.x (z,, 1 1)] equals to performing MBDI. Therefore, the key is to estimate Ex (#r, 1)]- We utilize Monte Carlo (MC) sampling estimate Ex (Ie, & + 1)]. For efficiency reasons, we only perform one-time MC sampling at each inversion time-step, ie.,
N
0 (zt, 1 +) = Σ Sieo (1),++1),
(8)
where Å = .A]] is a one-hot vector sampled from the distribution of X. Thus, only one branch is randomly sampled at each time-step of inversion. Note that MC sampling is performed independently at each time-step in FreeInv, essentially distinguishing it from single-branch DDIM inversion which can be treated deterministically selecting one branch all time-steps. We empirically (Sec. 4.4) find that it performs comparably to mulli-time MC sampling and MBDI.
branch. Through one-time MC estimation, only branch is randomly selected at each time-step to estimate the noise instead of using all N branches, effectively reducing time consumption. However, the memory cost remains high, as it is still needed maintain the latent representations of all branches.
(2) Image transformation
To further improve efficiency, we replace the explicit multiple branches by applying transformations (e.g. rotation, flipping, patch-shufllnng, etc.) to the image/latent representation, thereby generating multiple augmented versions. Since FreeInv does not impose any spatial or semantic constraints on different branches, is reasonable to mimic mulli-branch image sampling through transformation, i.e. r fz(z),, where f-(-) refers to the transformation that implicitly represents the i-th branch. Then Eq. becomes:
N
0. (It, 1 +1 Σ
dico (ft (xx),
Overall, following Eq. 11 the inversion and reconstruction process of FreeInv can be formulated as
D++1
Ft
+
E (Ie, + 1),
(10)
Vau
Et
Ft-1
So (1441, +1).
(11)
Vai
Vai+1
In Fig. 2 we employ rotation operation as the transformation for example to illustrate the whole process. In detail, during the inversion process, for an image, we rotate the latent code I; each time-step with a randomly selected angle (i.e., 0, π 222 म, 3T/2) to predict the noise, implicitly formulating 4-branch DDIM inversion process. For the reconstruction process, apply similar operation. To ensure consistency, we keep the rotation angle the same between the inversion and reconstruction processes at each time-step.
In this way, the additional computational consumption is limited to transformation and memory required to store the transformation type (e.g. rotation angles), both of which are negligible compared to previous methods tailored for mitigating the trajectory deviation (see Sec. 4.2 for computation cost comparisons).
5

---

Table 1: Quantitative comparison: reconstruction. We quantitatively evaluate reconstruction faithfulness, as well as computation costs of existing inversion methods, including U-Net based and DiT based methods, on the PIE benchmark. FreeInv achieves competitive results with superior high efficiency. All the U-Net based methods use Stable Diffusion 1.5 and 50-step schedule except VI uses the Latent Consistency Model (I.CM) and 12-step schedule. All the DiT based methods adopt 25-step schedule.
Methods
PSNR
LPIPS
Reconstruction Accuracy
(x10"))
MSE (x10"))
SSIM
Time (Seconds)
Inversion Computation Costs
Memory (MB)
DDIM
25.04
NTI
25
26.72
5.46
3.13
0.79
148
11945
EDICT 551
27.21
5.12
2.88
0.80
12325
U-Net Based
13595
VI
50
27 86
5.45
3.77
0.80
13853
ReNoise g
26.61
6.52
3.19
0.79
6395
PI
12
27.12
5.13
2.91
0.79
7197
Ours
27.69
5.14
2.45
0,81
3031
14.92
46.19
52430
DIT Based
FLUX-RF-Solve
17
26.38
10.98
3.89
0.84
15
32430
OurS
29.24
4.25
1.64
0.90
32430
Table 2: Quantitative comparison: editing. With P2P as bascline, we quantitatively compare existing inversion methods, with regard to background preservation and description alignment of edited images.
Method
Strueture Distance (x10 4
Background
Preservation
Similarity Edited
Inversion
Editing
PSNR
LPIPS (x10 $44
MSE (x10 4
SSIM
Whole
DDIM
P2P
88
17.84
22.07
25.18
22.33
NTI
P2P
10.11
27.80
4.99
2.99
0.85
24.80
21.76
EDICT
45
P2P
3,84
29.79
3,70
2.04
0.87
23.09
20.32
15
P2P
11.64
25.96
6.16
3.93
25.60
22,61
VI
30
P2P
17.35
28.00
5.75
7.61
0.85
24.86
22.12
23.81
BELM
46
P2P
17.28
25.51
8.46
4.76
0,82
24.23
21.30
P2P
10.89
27.21
5.44
0.85
22.12
Ours
PIP
17.13
26.03
6.79
4.17
0.83
25.30
22.33
4 Experiments
We make both quantitative and qualitative comparison with statc-of-the-art inversion enhancing techniques, covering Null-Text Inversion (NTI) 251. EDICT 45), DDPM Inversion (DI) (15). Virtual Inversion (VI) (50), PnP Inversion (PI) 17). ReNoise 99, BELM [46] and STEM Inversion (20]
Moreover, conduct comprehensive experiments by plugging FreeIny into popular inversion-basee image/video editing approaches, including Prompt-to- -ro-rr (P2P) [12, MasaCtrl 4, PnP 42), and
TokenFlow [10). We also conduct ablation studies to provide more comprehensive understanding of our method.
4.1 Implementation Details
In experiments, unless otherwise stated, adopt Stable Diffusion [36 1.5 with 50-step DDIM schedule for U-Net based methods, and FLUX.1-des [I] (abbr. FLUX) with 25-step schedule for DiT based methods.
4.2 Image Reconstruction and Editing
Dataset. Following previous works [7 50), we employ the PIE-benchmark 17 and its officially released code to quantitatively evaluate image editing results from FreeInv and the compared methods. PlE-benchmark consists of 700 images resolution 512 x 512, the content of which from nature artificial generation. In the benchmark, each image is associated with source prompt, an editing prompt, and an editing mask indicating anticipated editing areas.
Evaluation Metrics. For the image reconstruction task, we employ PSNR, LPIPS 52 MSE, and SSIM 44 to evaluate the reconstruction quality. addition, we record the time cost and GPU memory usage to assess the computational efficiency. For the image editing task, we utilize structure

---

Ieput Image
Curs Recon
wil Frechivw
Freetev
Isput Imsage
Origami, bear
A photo of pink toy horse
MasaCtri
jumpa up
galloping horse
white cat
yellow car
Figure 3: Qualitative comparison. We integrate FreeInv into PnP 42]. MasaCtrl 4 and P2P [2] respectively. We compare the reconstruction and editing results w/ or w/o FreeInv.
distance 41 and background preservation metrics 17 to measure how well the layout and the
unedited regions are preserved. Furthermore, CLIP similarity 33 is adopted assess the alignment between the edited image and the target textual prompt.
Quantitative Evaluation. Quantitative results for image reconstruction provided in Tab. The results show that (i) Effectiveness: We observe that FreeInv and the other existing inversion methods superior
boost the reconstruction accuracy effectively, where FreeInv achieves competitive results. (ii) Generality: FreeInv can be seamlessly integrated with U-Net based or DiT based methods, benefiting from its ultra simple design and implementation. (iii) Efficiency: Unlike other methods that rely on complicated numerical solvers (e.g. BELM, EDICT), require gradient back-propagation (e.g. NTI, ReNoise), or need extra memory consumption (e.g. g. DI, VI, PT), Freelnv incurs negligible computational overhead, i.e., the computational cost of FreeInv is approximately equal to that of the DDIM baseline. Quantitative comparison for image editing presented in Tab, 2 where we adopt P2P as the baseline editing framework, and all the inversion approaches are integrated into it. With inversion techniques, the edited images show improved faithfulness the original content, with lower structure distance and better- better-preseer preserved backgrounds. Compared to existing methods, FreeIny achieves superior prompt-image alignment, as indicated by its high CLIP similarity.
Qualitative Comparison. In Fig. 3 we visual-
PLUX
ELUC--F Sober
PLUSA
ize the reconstruction results and the correspond- ing editing outcomes of PnP [42) MasaCtrl 44. and P2P [12] with or without FreeInv. Due to the poor preservation of naive DDIM inversion, reconstruction results without FreeInv often ex- hibit significant deviations from the original im- age. In contrast, FreeInv significantly improves reconstruction quality. This improvement in reconstruction fidelity further leads to better editing out- comes, such as restoring the distorted face of the teddy bear and harmonizing the color of the horse's legs. Besides U-Net based methods, we also evaluate the effectiveness of FreeIny on DiT-based approaches. Fig. 4 we present the reconstruction results of FLUX, -L-Sver-Sllver 47), and FLUUX+Freelny where the results of FLUX+Frrelnn demonstrate superior fidelity. More visualization results are shown in the appendix.
Figure 4: Visualization of the reconstructed images
of different approaches with FLUX.
Moreover, Fig. presents the editing results of P2P equipped with different diffusion inver- sion methods. P2P with naive DDIM inversion can hardly maintain the structure details of the input image. In contrast, EDICT, DI, VI, and BELM exhibit such a strong fidelity to the source image that it hinders their editing capa- bilities. Overall, the editing results from NTI, PI, and FreeInv demonstrate strong alignment
Table 3: Human evaluation. We conduct a user study on the preference of editing results w/o or w/ FreeInv. The details about the user study are provided in the appendix.
User Preference
PnP
MasaCtrl
P2P
w/o Freeinv
18.18
12.73
10.34
w/ FreeInv
81.82
87.27
89.66

---

Imput Image
P2P
NTI
EDICT
DI
VI
BELM
PI
Ours
white sitting chair with flowers
sitting flowers
round cake with orange frosting wooden plate
square cake with orange frosting wooden plate
A plate with steak on
plate with salmon on It
Figure Qualitative comparison. We select Prompt-to-Promy (P2P) as the baseline editing framework, and compare the editing results with different inversion approaches. The source and target prompts are provided below each row of the images.
with the target prompt, well as high faithfulness with respect to the input. More comparisons can be found in the appendix.
Human Evaluation. To further validate the effectiveness of FreeInv, we also conduct user study to calculate the user preference rate of the edited images with the editing instruction. The edited images generated with FreeInv achieve significantly higher user preference compared to those without FreeInv, as illustrated in Tab. 3 In the appendix, we additionally provide the human preference rate of the editing results with different inversion methods, as well as more implementation details about the user study.
4.3 Video Reconstruction and Editing
we use the videos from the DAVIS dataset [31] or downloaded from the Internet for evaluation. The video is captured for the first 120 frames, cropped to a resolution of 512 512 pixels. We measure the mean PSNR across all the frames to evaluate the reconstruction fidelity. Additionally, time and memory costs are reported to compare efficiency.
Dataset and Metrics Following previous works 10
Quantitative and Qualitative Comparison. TokenFlow (10 is adopted as the baseline method, which is representative for inversion-based video editing. To demonstrate the superiority of FreeInv, it is also compared with STEM Inversion (20), a state-of-the-ar method designed for video inversion. The quantitative and qualitative comparisons are presented in Fig. 6 More visualization results and videos that can be played are available in the appendix. Through Fig. 6 it can be seen that (i) DDIM inversion and reconstruction exhibit poor preservation of the original video contents. Consequently, in the pixar animation case, the eyes of the woman look strange, and in the black SUV case, the road appears dark at some regions. (ii) Although STEM Inversion makes a great improvement, there remains artifacts in the reconstruction with regard to some details, which are annotated with red boxes. Moreover, we notice the editing results with STEM-Inv are usually over sharpened, e.g., the cloud in the pixar animation FreeInv achieves the best reconstruction results regarding the highest PSNR value and visualization faithfulness, and brings negligible extra consumption (2MB GPU memory occupation). The enhanced reconstruction quality benefits editing, making editing results more natural and detailed.
and the trace the road in the black SUV case. (ii) In comparison,
4.4
Ablation Study
MC sampling ps. MBDI. We compare the reconstruction quality of MC sampling and MBDI. We adopt MBDI (N 4) as the baseline and compare it with one-time, two-time, and four-time

---

Source Prompt: woman running
Source Video
Target Prompt: pixar animation
Tok
Time Cost: 222
Memory Cocup:: 12341MBB
Mean PSNR: 53
STE
Memory Ocoup::
TokenFlow--urs
Time
PSNR:
Source Prompt: A SUV
Videa
Target Prompt: black SUV
TakonFlow
TokenFFow+STTEM
Time Cost: 580
Memory Ocoup:: 18543MB
Mean PSNR: 27.77
TokenFlow+Ours
Time Cost: 222
Memory Occup:: 12343MB
Mean PSNR: 29.24
Figure 6: Video comparison. We compare TokenFlow 10. TokenFlow+STEN 20 and Token- Flow+Ours with respect to the reconstruction results (on the left side of the dash-line), the editing outcome (on the right side of the dash-line), as well as time and memory costs (below the reconstruc- tion results).
MC sampling. The comparison is shown in Tab. 4 We observe that one-time sampling performs comparably to MC sampling with multiple times and MBDI.
Table 4: Ablation study on the number of MC sampling steps.
Sampling Times
PSNR
LPIPS (x10 3
MSE (x10 ")
SSIM 1
MBDI (4-branch)
28.14
4.94
2.30
0.81
1-time MC (Ours)
28.13
55.0
2.30
0.81
2-time MC
28.14
5,00
2.30
0.81
4-time MC
28.14
5.00
2.30
0.81
Transformation rs. Multiple Images. As discussed Sec. 3.3 instead sampling different images in multiple branches, we exploit different transformations to improve efficiency. For multi-branch DDIM inversion, we compare two variations. One is that each branch consists of distinct images, termed MB-1 in experiment, while the other is that each branch consists of the original image rotated with different angles, termed as MB-R. In the ablation, the branch number is set to 4. The comparison is listed in Tab. FreeInv performs comparable to MB-1 and MB-R, but outperforms the DDIM baseline by a large margin (27.64 .6. versus 25.04 in terms of PSNR).
Comparison between different types of transformations. We implement FreeInv with different types of transformations including random rotation, random horizontal/vertical flipping, random patch shuffling, random color jittering, as well as the combination of these transformations for comparison. The experiment is conducted on the PIE benchmark. The results in Tab. ( show that different types of transformations achieve comparable performance in improving the reconstruction faithfulness.

---

Table Comparison among a) MB-I: multi-branch inversion where each branch represents distinct images, b) MB-R: multi-branch inversion where each branch corresponds to one image rotated a certain angle, and c) Ours: single-branch inversion where random rotation is applied in each
time-step.
Methods
PSNR
LPIPS 4
MSE (x100))
SSIM
DDIM
25.04
9.14
4.43
0.77
MB-I
28.11
4.94
2.30
0.8T
MB-R
27.73
5.06
2.42
0.81
Ours
27.64
5.14
2.45
0.81
Table 6: Comparison between different types of transformations,
Methods
PSNR f
LPIPS (x10-3)4-
MSE (x (x100)) +
SSIM 1
flip
27.47
5.43
2.55
0.80
patch shuffle
27.61
5.18
2.55
0.80
color jitter
27.53
5,40
2.55
0.80
rotation 27.64
5.14
2.45
0.81
combination 27.69
5.14
2.45
0.81
Cross-attention Map Visualization To provide an intuitive understanding of the improvement brought by Freelny, we visualize the cross-attention map in Fig- The prompt is "a woman running", and we aggregate the cross-attention maps with respect to the word "woman" among all time-steps for each sample. We observe that FreeInv enables the model to focus more precisely on the region of "woman" compared to the DDIM baseline.
Source
Ours
Figure Visualization of cross-attention map diffusion U-Net.
Conclusion
In this paper, we find that ensemble of trajectories for multiple images effectively reduce the DDIM reconstruction error. Based on such finding, we propose a method named FreeIny to perform an efficient ensemble. FreeInv enhances DDIM inversion we randomly transform the latent representation, and keep the transformation at each time-step the same between the inversion and the reconstruction. FreeInv is compatible with both U-Net and DiT architectures. Thanks to its efficiency, FreeInv is applicable not only to image reconstruction but also video sequences. In both image and video reconstruction tasks, achieves reconstruction fidelity comparable to or better than existing methods, while demonstrating significantly improved efficiency.
free-lunch manner. detail,
10

---

Acknowledgments
This project is supported by National Natural Science Foundation of China under Grant 923701 14. This work is also supported by HUJING Digital Media & Entertainment Group through HUJING Digital & Entertainment Innovative Research Program.
References
[1] Flux. httts:///iithii
[2] Omri Avrahami, Ohad Fried, and Dani Lischinski. Blended latent diffusion. arXiv preprint arXiv:2206.02779, 2022.
[3] Yuxiang Bao, Di Qiu, Guoliang Kang, Baochang Zhang, Bo Jin, Kaiye Wang, and Pengfei Yan. Latentwarp: Consistent diffusion latents for zero-shot video-to-video translation. arXiv preprint arXiv:2311.00353 2023.
[4] Mingdeng Cao, Xintao Wang, Zhongang Qi, Ying Shan, Xiaohu Qie, and Yingiang Zheng, Masactrl: Tuning-free mutual self-attention control for consistent image synthesis and editing. In ICCV, 2023.
[5] Wenkai Dong, Song Xue, Xiaoyue Duan, and Shumin Han. Prompt tuning inversion for text-driven image editing using diffusion models. In ICCV, pages 7430-7440, 2023.
[6] Xiaoyue Duan, Shuhao Cui, Guoliang Kang, Baochang Zhang, Zhengcong Fei, Mingyuan Fan, and Junshi Huang. Tuning-free inversion-enhanced control for consistent image editing. In AAAI, volume 38, pages 1644-1652, 2024.
[7] Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling rectified flow transform- ers for high-resolution image synthesis. In ICML, 2024.
[8] Yutang Feng. Sicheng Gao, Yuxiang Bao, Xiaodi Wang, Shumin Han, Juan Zhang, Baochang Zhang, and Angela Yao. Wave: Warping ddim inversion features for zero-shot text-to-video editing. ECCV, 2024.
[9] Daniel Garibi, Or Patashník, Andrey Voynov, Hadar Averbuch-Elor, and Daniel Cohen-Or. Renoise: Real image inversion through iterative noising, In ECCV. Springer, 2024.
[10] Michal Geyer, Omer Bar-Tal, Shai Bagon, and Tali Dekel. Tokenflow: Consistent diffusion features for consistent video editing. In ICLR. 2024.
[11] Lars Kai Hansen and Peter Salamon. Neural network ensembles. IEEE TPAMI, 2002.
[12] Amir Hertz, Ron Mokady, Jay Tenenbaum, Kfir Aberman, Yael Pritch, and Daniel Cohen-Or. Prompt-to-prompt image editing with cross attention control. In ICLR, 2023.
[13] Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. In NeurIPS, volume 33, pages 6840-6851, 2020.
[14] Li Hu. Animate anyone: Consistent and controllable image-to-video synthesis for character animation. CVPR, pages $153-8163, 2024.
[15] Inbar Huberman HubermannSppegeelllas Vladimir Kulikov, and Tomer Michaeli. An edit friendly ddpm noise space: Inversion and manipulations. In CVPR, pages -2478, 2024.
[16] Phillip Isola, Jun Yan Zhu, Tinghui Zhou, and Alexei A. Efros. Image-to-image translation with conditional adversarial networks. In CVPR, pages 5967-5976, 2017.
[17] Xuan Ju, Ailing Zeng, Yuxuan Bian, Shaoteng Liu, and Qiang Xu. Pnp inversion: Boosting diffusion-based editing with 3 lines of code. In ICLR, 2024.
[18] Minguk Kang, Jun-Yan Zhu, Richard Zhang, Jaesik Park, Eli Shechtman, Sylvain Paris, and Taesung Park. Scaling up gans for text-to-image synthesis. In CVPR, pages 10124-10134, 2023.

---

[19] Bahjat Kawar, Shiran Zada, Oran Lang, Omer Tov, Huiwen Chang, Tali Dekel, Inbar Mosseri, and Michal Irani. Imagic: Text-based real image editing with diffusion models. In CVPR, pages 6007-6017, 2023.
[20] Maomao Li, Yu Li, Tianyu Yang, Yunfei Liu, Dongxu Yue, Zhihui Lin, and Dong Xu. video is worth 256 bases: Spatial-temporal editing. In CVPR, pages 7528-752 9228-5337, 2024.
inversion for zero-shot video
[21] Semmao Li, Joost van de Weijer, Taihang Hu, Fahad Shahbaz Khan, Qibin Hou, Yaxing Wang, and Jian Yang. Stylediffusion: Prompt-embeddiig inversion for text-based editing. arXiv preprint MXXv:22303.15649 2023.
[22] Jiadong Liang, Wenjie Pei, and Feng Lu. Cpgan: Content-parsing generative adversarial networks for text-to-image synthesis. In ECCV, pages 491-508. Springer, 2020.
[23] Shaoteng Liu, Yuechen Zhang, Wenbo Li, Zhe Lin, and Jiaya Jia. Video-p2p: Video editing with cross-attention control. In CVPR, 2024.
[24] Daiki Miyake, Akihiro Iohara, Yu Saito, and Toshiyuki Tanaka. Negative-prompt inversion: Fast image inversion for editing with text-guided diffusion models. arXiv preprint arXiv:2305.16807 2023.
[25] Ron Mokady, Amir Hertz, Kfir Aberman, Yael Pritch, and Daniel Cohen-Or. Null-text inversion for editing real images using guided diffusion models. In CVPR, 2023.
[26] Alexander Quinn Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob Megrew, Ilya Sutskever, and Mark Chen. Glide: Towards photorealistic image generation and editing with text-guided diffusion models. ICML, pages (6784-16804, PMLR, 2022.
[27] Taesung Park, Ming- Yu Liu, Ling-Chun Wang, and Jun- Yan Zhu. Semantic image synthesis with maatially-adaptiv normalization. In CVPR, pages 2337-2346, 2019.
[28] Or Patashnik, Zongze Wu, Eli Shechtman, Daniel Cohen-Or, and Dani Lischinski. Styleclip: Text-driven manipulation of stylegan imagery. In ICCV, pages 2085-2094, 2021.
[29] William Peebles and Saining Xie. Scalable diffusion models with transformers. In ICCV, pages 4195-4205, 2023.
[30] Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach. SDXL: Improving latent diffusion models for high-resolutiin image synthesis. In ICLR, 2024.
[31] Jordi Pont-Tuset, Federico Perazzi, Sergi Caelles, Pablo Arbeláez, Alex Sorkine- Korkine-Homung. and Lue Van Gool. The 2017 davis challenge video object segmentation. arXiv preprint arXiv: 1704v07704.00655 2017.
[32] Chenyang Qi Xiaodong Cun, Yong Zhang, Chenyang Lei, Xintao Wang, Ying Shan, and Qifeng Chen. Fatezero: Fusing attentions for zero-shot text-based video editing. In ICCV, pages 15932-15942, 2023.
[33] Tingting Qiao, Jing Zhang, Duanqing Xu, and Dacheng Tao. Mirrorgan: Learning text-to-image generation by redescription. In CVPR, pages 1505-1514, 2019.
[34] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In ICML., pages 8748-8763. PMLR, 2021.
[35] Scott Reed, Zeynep Akata, Xinchen Yan, Lajanugen Logeswaran, Bernt Schiele, and Honglak Lee. Generative adversarial text to image synthesis. In ICMI,, pages 1060-1069. PMLR, 2016.
[36] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer: High- resolution image synthesis with latent diffusion models. In CVPR, pages 10684-10695, 2022.
12

---

[37] Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily Denton, Seyed Kamyar Seyed Ghasemipour, Bureu Karagol Ayan, Sara Mahdavi, Rapha Gontijo Lopes, et al. Photorealistic text-to-image diffusion models with deep language understanding. arXiv preprint arXiv:2205. / 1487, 2022.
[38] Jiaming Song. Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. In ICLR, 2021.
[39] Yang Song and Stefano Ermon. Generative modeling by estimating gradients of the data distribution. In NeurlPS, volume 32, 2019.
[40] Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In ICLR, 2021.
[41] Narek Tumanyan, Omer Bar-Tal, Shai Bagon, and Tali Dekel. Splicing vit features for semantic appearance transfer. CVPR, pages 10748-10757, 10757, 2022.
[42] Narek Tumanyan, Michal Geyer, Shai Bagon, and Tali Dekel. Plug-and-play diffusion features for text-driven image-to-imee -mage translation. In CVPR, pages 1921-1930, 2023.
[43] Dani Valevski, Matan Kalman, Eyal Molad, Eyal Segalis, Yossi Matias, and Yaniv Leviathan. Unitune: Text-driven image editing by fine tuning a diffusion model on a single image. ACM TOG, 42(4):1-100 2023.
[44] Yael Vinker, Eliahu Horwitz, Nir Zabari, and Yedid Hoshen. Image shape manipulation from single augmented training sample. In ICCV, pages 13769-37778, 2021.
[45] Bram Wallace, Akash Gokul, and Nikhil Naik. Edict: Exact diffusion inversion via coupled transformations. In CVPR, pages 22532-22541, 2023.
[46] Fangyikang Wang, Hubery Yin, Yue-Jiang Dong, Huminhao Zhu, Chao Zhang, Hanbin Zhao, Hui Qian, and Chen Li. BELM: Bidirectional explicit linear multi-step sampler for exact inversion in diffusion models. In NeurlPS, 2024.
[47] Jiangshan Wang, Junfu Pu, Zhongang Qi, Jiayi Guo, Yue Ma, Nisha Huang, Yuxin Chen, Xiu Li, and Ying Shan. Taming rectified flow for inversion and editing. arXiv preprini arXiv:2411.0474 2024.
[48] Ting Chun Wang, Ming Yu Liu, Jun Yan Zhu, Andrew Tao, Jan Kautz, and Bryan Catanzaro. High-resolution image synthesis and semantic manipulation with conditional gans. In CVPR. 2018.
[49] Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Simoncelli. Image quality assessment: from enror visibility to structural similarity. IEEE TIP, 13(4):600-6) 612, 2004.
[50] Sihan Xu, Yidong Huang, Jiayi Pan, Ziqiao Ma, and Joyce Chai. Inversion-free image editing with natural language. In CVPR, 2024.
[51] Zhongcong Xu, Jianfeng Zhang, Jun Hao Liew, Hanshu Yan, Jia-Wei Liu, Chenxu Zhang, Jiashi Feng, and Mike Zheng Shou. Magicanimate: Temporally consistent human image animation using diffussonn model. In CVPR, pages 1481-1490, 2024.
[52] Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The un- reasonable effectiveness of deep features as a perceptual metric. In CVPR, pages 586-595, 2018.
[53] Jun Yan Zhu, Taesung Park, Phillip Isola, and Alexei A. Efros. Unpaired image-to-ing to-image translation using cycle-consistent adversarial networks. In ICCV, pages 2242-2222222222 2251, 2017.
[54] Minfeng Zhu, Pingbo Pan, Wei Chen, and Yi Yang. Dm-gan: Dynamic memory generative adversarial networks for text-to-image synthesis. In CVPR, pages 5802-5810, 2019.
13

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
The paper's contributions and scope reflected in the abstract and introduc-
tion.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
The limitation discussed in the appendix.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
14

---

Justification:
We provide complete assumption and detailed proof to support our theoretical
claims.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proof's can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification:
We disclose all the implementation details.
Guidelines:
The answer NA means that the paper does not include experiments. If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
15

---

Answer:
[No]
Justification:
We will release the code upon the acceptance of the paper.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.cm
cessiinnooccyyl for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
/ ccpppp.ccppulli//guiii wwwiicccguie//eeeeeeeeeis CoodeSubeiiieeee ssi wswinnolli onnolicy)))) for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
We provide complete implementation details in the Experiment section.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification: Error bars are not reported because it would be too computationally expensive.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, at least for the experiments that support main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation the standard error of the mean.
16

---

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of enrors is not verified.
For asymmetric distributions, the authors should be careful to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of exccution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
The computer resources are listed in the main table.
Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer: [Yes]
Justification: The research conducted in the paper conform with the NeurIPS Code Ethics in every respect.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make to preserve anonymity (e.g.. if there is special consid- cration due to laws or regulations their jurisdiction..
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
We discuss the societal impact in appendix.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
17

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of MI..
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
We do not release a new model, and the paper poses no such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Lieenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: They credited properly.
Guidelines:
The answer NA means that the paper does existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
18

---

If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
The paper does not release new assets.
Guidelines:
The answer NA means that the paper does release new assets.
Researchers should communicate the details of the dataset/code/mmodll as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: The main contribution of the paper does not involve crowdsourcing experi- ments.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, not include any information that would break anonymity (if applicable), such the institution conducting the review.
19

---

16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: The core method development in this research does not involve LLMs as any important, original, or non-standard components.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy buctps://neure wwwparippps..coooeeeeeeeee for what should or should be described.
20

---

Appendix
In the appendix, we provide more quantitative and qualitative results to facilitate a more compre- hensive understanding and evaluation of the proposed method. Moreover, supplement more visualization results comprising both image and video editing in web-page through the link httpss//yuxiangbao. wataaaaogggtbbbii.. github.io//reelee for better visualization. Further, we discuss the social im- pact and limitations of Freelnv.
A Image/Video Editing with FreeInv
As FreeInv may improve the reconstruction quality in free-lunch manner, it can be easily combined with existing image/video editing approaches [2] 42 10] to improve editing performance. As current editing methods typically rely spatial coherence (e.g., the self-attentio map in U-Net) as guidance, we make minor modifications FreeInv to make it better aligned with existing editing approaches, Le.. additionally apply inverse transformation to the predicted noise. For example, at certain time-step, we rotate the latent by an angle to predict the noise and then reversely rotate the noise with the same angle before adding the noise to the latent Jy- Note that such inverse transformation of noise is optional for reconstruction as the reconstruction quality is mainly determined by the closeness between inversion and reconstruction trajectories. We compare the reconstruction and editing results with and without applying inverse transformation on the predicted noise on PIE benchmark. The results presented in Tab. 7 and Fig. & show that the inverse transformation has minimal impact on the reconstruction results, but may benefit the structural faithfulness in editing.
Table Ablation study on the effect of inverse transformation applied on the predicted noise with respect to reconstruction quality the PIE benchmark.
Methods
PSNR LPIPS (x10 3)
MSE (x10 9)
SSIM f
DDIM Baseline
25.04
9.14
4.43
0.77
w/o inverse transformation
27.64
5.13
2.45
0.81
w/ inverse transformation
27.64
5.14
2.45
0.81
Input Image
Recon. w/o IF
Recon. w/ IF
Edit w/o IF
Edit w/ IF
A cat wooden chair
A dog on wooden chair
A colorful bird
A red bird
Figure 8: Visualization of the reconstruction and editing results with and without inverse transforma-
tion (IF). We adopt PnP as the editing method.
B Combination with NTI
To further demonstrate the free-lunch benefit, we supply extensive experiments by combining Freelny with the previous representative method NTT (5 further boost performance. The result provided
in Tab. g
21

---

Table 8: Ablation study on plugging FreeInv into NTI (25]
Methods
PSNR t
LPIPS (x10-3)4
MSE (x10 9)
SSIM
DDIM Bascline
25.04
9.14
4.43
0.77
NTI
26.74
5.46
3.13
0.79
FreeInv
27.69
5.14
2.45
0.81
NTI+FreeIny
28.15
4.89
2.31
0.81
C
Experiments on SDXL
Following previous works 25
45
15
50
in the literature, choose SD1.5 perform U-Net
based experiments, and FLUX to perform DiT based experiments. In Tab. 2 we further include SDXL 30 for comparison with U-Net-based architectures. FreeInv consistently delivers performance improvements, further demonstrating its effectiveness and generality.
Table 9: Ablation study plugging FreeInv into SDXL
30
Methods
PSNR f
LPIPS (
MSE (x10
SSIM 1
SDXL DDIM Baseline
24.78
12.3
6.02 3.15
0.75
SDXL FreeInv
26.68
5.57
0.79
D Human Evaluation
In Tab. 3 compare the editing results from PnP 42 MasaCtrl 4. and P2P [22 under the scenarios with or without FreeInv. We use fifteen images from the PIE benchmark, with each editing method applied five distinct images. During the survey, participants are shown the source image, the edited results with and without FreeInv, and the target prompt. They are then asked to choose the edited image that demonstrates higher textual alignment and better source preservation. Finally, receive 165 votes from participant pool. A screenshot of the survey interface is provided in Fig. 9
In similar way, we perform another study to evaluate the effectiveness of different inversion methods. Participants are presented with P2P editing results generated using each inversion method, and then asked to choose their preferred results. Finally, receive 130 votes from participant pool, and the result is provided in Tab. TO
Table 10: User study different inversion methods.
User Study
DDIM
NTI
EDICT
DI
Ours
preference (%)
4.6
12.3
8.5
10.8
7.7
26.2
30.0
E More Visualizations
E.1
Image Reconstruction
We additionally visualize reconstruction results in Fig. 10 to compare FreeInv with other state-of-the-arr inversion methods. The results further verify the effectiveness and generality of FreeInv indicated in Sec. 4.2
E.2 Image Editing
Comparison with Other Inversion Methods. We show more examples edited by P2P with previous state-of-the-art inversion approaches in Fig. 11 The visualization results further validate that Freelny significantly outperforms the DDIM baseline, and achieves performance comparable existing state-of-the-an inversion approaches.
22

---

Source image
Source Image
"A teddy bear releing his right hand
photo of bronze horse in museum.
Opt
Opt
Opc.
The screenshot of the user survey interface on the phone. The source image, the edited results with and without FreeInv, and the target prompt are presented to the participants.
Figure 9:
Plugging in Existing Image Editing Methods. Due operational simplicity, FreeInv can be readily plugged into existing inversion-based image editing frameworks. In Fig. 12 and Fig. [3] we present more editing results of PnP Ap and MasaCtrl [4] respectively. As shown in Fig. 12 FreeIny outperforms the baseline editing method remarkably.
E.3
Video Editing
We provide the video editing results through this link where compare the results of TokenFlow [10 baseline, TokenFlow with STEM-Inv (20), and TokenFlow with FreeInv. Besides, we provide the video reconstruction results of DDIM inversion, STEM-Inv and FreeInv. Through the visualization results, we observe that FreeInv exhibits superior reconstruction fidelity and editing effects, compared with DDIM inversion and STEM-Inv.
Social Impact and Limitation
Social Impact. We introduce free-lunch DDIM inversion-enhanced technique FreeInv in this work. FreeInv enables high-fidelity reconstruction, benefiting image and video editing accordingly. However, aware that it can be potentially abused by those malicious individuals groups
23

---

Input Image
DDIM
NII
EDICT
BELM
RF-Solver
SDD-Our
FLUX + Ours
cat sitting next to mirror
A woman in white dress sitting on chair with flowers
wearing pink
A woman with blue hair wearing shirt
sitting with
cup of coffes with drawing of tulip put on the wooden table
Figure 10: Qualitative comparison. Visualization of the reconstruction results in comparison with state-of-the- inversion methods.
distribute false information and cause confusion, which undoubtedly violates the intention of our research. We believe the misuse can be alleviated through developing AIGC detection algorithms and being supervised with regulations.
Limitation. While our proposed FreeInv improves the efficiency DDIM inversion significantly, there still remains room for improvement. One key challenge lies in balancing editability and reconstruction fidelity, which is common issue for inversion methods. In some cases, Freelny may overemphasize the preservation of source content, which can limit its editability. Another limitation lies in that FreeIny is an inversion-enhancee technique. The editing quality also relies on the editing framework itself which is integrated with FreeInv. Thus, better editing framework may yield better editing result. For specific scenarios, a suitable editing framework s should also be considered.
24

---

Input Image
Ours
P2P
NTI
EDICT
DI
VI
BELM
PI
white dress sitting chair with flowers
sitting chair with flowers
round with orange frasting wooden
square cake with orange fresting wooden plate
plate with steak on it
plate with salmon on it
cat sitting wooden chair
A dog sitting wooden chair
A cat sitting next to mirror
A silver cat sculpture sitting next to mirror
orange sitting top
sitting top
horse grass field
zebra grass field
woman wearing
storm- trooper wearing
woman sitting in living room
A watercolor painting of woman sitting living room
Figure 11:
Qualitative comparison. More comparison with state-of-the -oe-----thhh-tttt inversion methods.
P2P [12 with DDIM inversion serves as the baseline method, and all of the methods are plugged into it.
25

---

Image
shiny robot bear,
photorealis- image zebra
photorealis- Ni image of sparrow
polar bear
A waoden sculpture
shiny silver robot
photo on eagle
photorealis- image of grizzly bear
Figure 12: Qualitative comparison. More editing results of PnP 42
with and without FreeInv.
Inpul Image
MesaChi
MisaClii wi Durs
Input Image
MasaCrri
MaseCrrl vi Oura
lying
cat
on the ground
A dog Jumps
A photo of couple holding
their hands
glass of milk and Oneo biscuit
standing on rope
A fidy bear raising right
Two birds
stand
Figure 13: Qualitative comparison. More editing results of MasaCtrl 4
with and without FreeIny.
26


Reviews:
[{"review_id": "SyLFKunFLp", "reviewer": "Reviewer_aAd8", "summary": "This paper proposed to adopt extra transformation operation to mitigate the inconsistency issue in the inversion process of FMs and DMs.", "questions": "See the weaknesses, please.", "limitations": "yes.", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 1, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n\n-\tThe paper is well written and easy to follow.\n\nWeaknesses:\n\n-\t(My main concern) The theoretical derivation in the section 3 did not persuade me to believe that the extra rotation operation in the inversion would result in less reconstruction error. Based on the theory in BELM, the reconstruction inconsistency is introduced by the discrepency of the formulations of DDIM and its inversion. Based on that theory, I cannot understand why rotation will mitigate this. I think a deeper theoretical explanation of FreeInv is needed.\n\n-\tThe math of (5) and (6) cannot deduce to a final smaller inconsistency error of the inversion path. The average error across different branched should not matter to the inversion of the path. And the process of FreeInv does not really mimic the MBDI, cause it is not a branch, but an image rotating.", "was_revised": true, "final_justification": "The theory part is not good enough, however, its practical usage is recognized.", "rebuttal": "Thank you for the constructive comments. \n\n**Q1: The theoretical derivation in the section 3 did not persuade me to believe that the extra rotation operation in the inversion would result in less reconstruction error. Based on the theory in BELM, the reconstruction inconsistency is introduced by the discrepency of the formulations of DDIM and its inversion. Based on that theory, I cannot understand why rotation will mitigate this. I think a deeper theoretical explanation of FreeInv is needed.**\n\n**A1:** \n\n(1) From a high-level, our theoretical goals are consistent with BELM. Our derivation also originates from the discrepancy between DDIM and its inversion, which is termed as mismatch error formulated in Eq. (3).\n\n(2) To reduce the mismatch error, BELM interprets DDIM process as solving ordinary differential equations (ODE) and increases the error order by employing a linear multi-step ODE solver, which is a standard technique for improving ODE solution accuracy.\n\n(3) In contrast, we adopt a different approach from BELM. While we still use a first-order solver, we propose ensembling the DDIM trajectories across multiple images, as shown in Eq. (4). We further show in Eq. (6) that the multi-branch version of DDIM can reduce the mismatch error. In Sec. 3.3 we explain how we reduce the computational and memory costs through one-time MC sampling and image transformations.\n\n**Q2: The math of (5) and (6) cannot deduce to a final smaller inconsistency error of the inversion path. The average error across different branches should not matter to the inversion of the path. And the process of FreeInv does not really mimic the MBDI, cause it is not a branch, but an image rotating.**\n\n**A2:** \n\n(1) According to the question, we think the reviewer may misunderstand our operation in MBDI and FreeInv. As discussed in Line 137-138, the righthand of Eq. (6) approximates the expectation of mismatch error of original inversion path without ensemble (note that the $\\epsilon$ does not have a superscript $e$). The lefthand of Eq. (6) denotes the mismatch error of the inversion path with ensemble (note that the $\\epsilon$ has a superscript $e$). Thus, Eq. (6) actually shows that through multi-branch ensemble, the mismatch error of inversion path can be reduced on expectation.\n\n(2) The key characteristic of different branches in MBDI is that different branches correspond to different images. Thus, we make an approximation in FreeInv, i.e., using different transformations to generate multiple images from a single image. Different transformed images (e.g., images rotated with different angles) in FreeInv correspond to different images in different branches in MBDI. In Sec. 3.3, we have provided detailed theoretical connections between MBDI and our approximation. \n\n(3) In FreeInv, rotation is not the unique choice, and other types of image transformation also work, including flip, patch shuffle, etc., as stated in Line 162 of our paper and proved in our experiments. In Fig. 2 we just take rotation for example to illustrate our method."}, {"review_id": "oVsQMdEbQD", "reviewer": "Reviewer_ko1D", "summary": "This paper presents a lightweight method (FreeInv) for improving the quality of image and video inversion. FreeInv approximates a multi-trajectory ensemble by randomly transforming (e.g., rotating, transformation) the latent representation at each time step and keeping it consistent during inversion and reconstruction, significantly reducing reconstruction errors with little cost.", "questions": "1. The rot and flip don't disturb the original image, but wouldn't the patch-shuffle on the latent?\n2. Is it feasible to combine FreeInv with existing optimization-based methods like NTI to further boost performance, or would that negate the \"free-lunch\" benefit?\n3. Why is unet-based diffusion model is only on sd1.5? For example, Renoise have experimented on SDXL. It is feasible to experimented on SD3.\n\nOverall, I am positive about this manuscript. Some deeper theoretical and experimental analysis and discussion is necessary, however the manuscript mostly focuses on ‘how to do’ and ‘how well it works’, and lacks in ‘why it works’.", "limitations": "N/A", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\n[+] Easy to follow.\n[+] Nice performance\n\nMajor weakness:\n\n1. It is really amazing to achieves better results with such an easy method. However, the discussion on the theoretical part is still too weak, in that the theoretical nature of simulating multipath ensemble through spatial perturbations is underpinned by a lack of mathematical rigor.\n2. The given image or video visualization seems too simple. I'm wondering if there might be an upper limit. How does it perform in the reconstruction or editing of more complex images (e.g. more detailed images)?\n3. In some tasks (image editing), the expense is actually tolerable, and the effect is primary. This finding is interesting, but on its own, if it can't beat the strongest models now available (or by combining extant methods) or doesn't provide enough theoretical proofs, I think it may be difficult to inspire follow-up research.\n4. The proposed method outperforms some of the more costly methods, but these more costly methods may perform better in more complex image inversions, whereas simple yet effective methods are often difficult to scale up. I'd like to hear a discussion of that.", "was_revised": true, "final_justification": "The authors have addressed most of my concerns, and I am maintaining my score.", "rebuttal": "Thank you for the constructive comments. \n\n**Q1: It is really amazing to achieves better results with such an easy method. However, the discussion on the theoretical part is still too weak, in that the theoretical nature of simulating multipath ensemble through spatial perturbations is underpinned by a lack of mathematical rigor.**\n\n**A1:** \n\n(1) Note that at different time steps, we apply different transformations, e.g., at different time steps, the rotation angles are randomly sampled. Thus, from a high-level, FreeInv can be interpreted as an implicit ensemble of multiple paths which correspond to different transformations. \n\n(2) More rigorously, we can interpret the connection between multi-branch ensemble and applying image transformations from two aspects, as discussed in Sec. 3.3. Firstly, we mathematically show that multi-branch ensemble can be viewed as one-time MC sampling at each time step, which means at each time step, we randomly sample one branch from distinct images. Then, based on this understanding, we randomly transform an image into different augmented versions to form different images/branches as this operation yields more efficient implementations than performing real data sampling. Its principle is similar to using data augmentations to mimic real data sampling in general deep learning practice. \n\n(3) Eq. (6) for multi-branch ensembe does not impose restrictions on the sampled images for different paths. Thus, it still holds for different transformations of a single image.   \n\n(4) In our original paper, we conduct a detailed ablation study, summarized in Tab. 6. The results verify that FreeInv remains effective.\n\n**Q2: How does it perform in the reconstruction or editing of more complex images (e.g. more detailed images)**\n\n**A2:** \n\n(1) In Tab. 1, we conduct a convincing quantitative evaluation on PIE benchmark, which is a public benchmark covering a wide range of natural and artificial contents, following previous studies [14,46] in the area.\n\n(2) We have also showed some visualizations on complex or detailed images. For example, in the \"A yellow car\" case of Fig. 3, the camera captures a car speeding past, with the trees, buildings, and sculptures in the background slightly blurred due to the car's high speed. We consider this a complex scene due to the combination of fast motion and intricate background elements. Our method successfully reconstructs both the foreground car and the motion blur in the background with high fidelity, enabling precise image editing. We will add more visualizations on complex images in revision.\n\n**Q3: In some tasks (image editing), the expense is actually tolerable, and the effect is primary. This finding is interesting, but on its own, if it can't beat the strongest models now available (or by combining extant methods) or doesn't provide enough theoretical proofs, I think it may be difficult to inspire follow-up research.**\n\n**A3:** \n\n(1) No single previous method obtains state-of-the-art results across all the evaluation metrics. As demonstrated in Tab. 2, although EDICT exhibits strong ability for background preservation, it suffers from poor prompt alignment, as reflected by low CLIP similarity scores. On the contrast, DI excels in prompt following, but its background preservation is not well. Taking a full consideration of the background preservation and CLIP similarity, the performance of FreeInv is balanced and competitive among the previous methods.\n\n(2) The main merit of our method is the superior efficiency. Pevious methods (e.g., NTI, EDICT, BELM), whose reconstruction/editing quality are on par with our method, consumes much more computation load, either in memory or time. Some method (e.g., NTI) even needs additional training.\n\n(3) We believe the efficiency is important. As shown in our paper, high efficiency may benefit the editing of video which consists of many frames. Moreover, it may promote future research to apply diffusion models to other downstream tasks, e.g., classification, segmentation, etc.\n\n**Q4: The proposed method outperforms some of the more costly methods, but these more costly methods may perform better in more complex image inversions, whereas simple yet effective methods are often difficult to scale up. I'd like to hear a discussion of that.**\n\n**A4:** In our opinion, we think the simplicity of our method makes it more scalable and easier to adapt. As shown in our paper, our method can be seamlessly integrated with both U-Net-based models (e.g., SD1.5, SDXL) and DiT-based models (e.g., FLUX), supporting both image and video inversion. Moreover, it can be combined with various reconstruction/editing methods to improve the generation quality further, e.g., combining with the previous optimization-based method NTI to further enhance reconstruction fidelity. To the best of our knowledge, FreeInv is *the first method* to achieve such broad compatibility and flexibility across architectures and modalities.\n\n**Q5: The rot and flip don't disturb the original image, but wouldn't the patch-shuffle on the latent?**\n\n**A5:** \n\n(1) Actually, patch-shuffle spatially disrupts the original image, and we employ it to enhance the diversity of inversion trajectories, serving as a role similar to that of flipping and rotation.\n\n(2) FreeInv does not impose any spatial or semantic constraints on different branches. We think the diversity matters. As shown in Tab. 4, MBDI, formulated with distinct images, still brings performance improvements. By leveraging more diverse branches, MBDI attains superior results.\n\n**Q6: Is it feasible to combine FreeInv with existing optimization-based methods like NTI to further boost performance, or would that negate the \"free-lunch\" benefit?**\n\n**A6:** Yes, it is feasible to combine FreeInv with NTI to further boost reconstruction fidelity. \n\n|    Methods    | PSNR  | LPIPS($\\times 10^{-2}$) | MSE($\\times 10^{-3}$) | SSIM |\n| :-----------: | :---: | :------------------: | :----------------: | :--: |\n| DDIM Baseline | 25.04 |         9.14         |        4.43        | 0.77 |\n|      NTI      | 26.74 |         5.46         |        3.13        | 0.79 |\n|    FreeInv    | 27.69 |         5.14         |        2.45        | 0.81 |\n|  NTI+FreeInv  | 28.15 |         4.89         |        2.31        | 0.81 |\n\n**Q7: Why is unet-based diffusion model is only on sd1.5? For example, Renoise have experimented on SDXL. It is feasible to experimented on SD3**\n\n**A7:** \n\n(1) We choose SD1.5 to perform U-Net based experiments, since most of previous studies [22,41,12,46] in the literature utilize this model. Therefore, we also perform experiments with SD1.5 for a fair comparison.\n\n(2) SD3 is a DiT-based model whose architecture is similar to that of FLUX. However, FLUX demonstrates stronger generative capabilities and has become the preferred choice in DiT-related research. Therefore, in this paper, we adopt FLUX for all DiT-based experiments.\n\n(3) To make a more convincing conclusion and consider the limited time during the rebuttal phase, we further include SDXL for comparison with U-Net-based architectures. FreeInv consistently delivers performance improvements, further demonstrating its effectiveness and generalizability. \n\n|    Models    | PSNR  | LPIPS($\\times 10^{-2}$) | MSE($\\times 10^{-3}$) | SSIM |\n| :----------: | :---: | :------------------: | :----------------: | :--: |\n|     SDXL     | 24.78 |         12.3         |        6.02        | 0.75 |\n| SDXL+FreeInv | 26.68 |         5.57         |        3.15        | 0.79 |"}, {"review_id": "8PNBYgiWBJ", "reviewer": "Reviewer_WToP", "summary": "This paper presents a DDIM inversion method without optimization (training/fine-tuning), which uses latent representation transformation to obtain multiple add noise/de-noising trajectories to further reduce the mismatch error. The paper includes a theoretical analysis to make the proposed method more trackable. The paper provides some good visual results in image/video reconstruction and editing tasks.", "questions": "1. Why is there not much difference between these transformations in table 5? Is the combination an equal probability selection of flip, patch shuffle and rotation?\n2.  Why is the A white cat below figure 3 missing a leg even after using FreeInv?", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "**Strengths.**\n\n1. The paper has a clear motivation, and there are significant speed and storage advantages over methods that require optimizing additional embedding or storing errors in these processes.\n2.  The paper presents a theoretical analysis process that helps to understand the rationale and motivation of the approach.\n3.  The paper visualizes an intermediate cross- attention map to show a better word-to-pixel alignment than DDIM.\n4.  Based on some T2I models and Video generation models, the method visually shows good reconstruction and editing results.\n\n- - -\n\n**Weaknesses.**\n\n1.  The paper needs deeper statistical analysis, and the results in Tables 1-2 show that the overall performance of the method is not as good as emphasized in the paper in terms of reconstruction and editing.\n2.  L103-L104, the paper puts forward a strong dependence, but without any theoretical basis, it seems that the argumentations are somewhat subjective.\n3.  L118, ensemble techniques lack the necessary citations\n4.  Although the paper puts a lot of visualization results, as well as some pictures on the dataset ablation, but the effectiveness of the method is not convincing enough, if we can see some quantitative results on the video and in-depth analysis would be better.", "was_revised": true, "final_justification": "Most of my concerns have been addressed, and I am keeping my initial score.", "rebuttal": "Thank you for the constructive comments. \n\n**Q1: The paper needs deeper statistical analysis, and the results in Tables 1-2 show that the overall performance of the method is not as good as emphasized in the paper in terms of reconstruction and editing.**\n\n**A1:** \n\n(1) As demonstrated in Tab. 1-2, no single method consistently outperforms all others across all the metrics, highlighting the inherent trade-offs in current approaches. Taking a full consideration of the reconstruction and editing performance, FreeInv appears balanced and competitive among the previous methods.\n\n(2) The main advantage of FreeInv is its simplicity and efficiency. It can be seamlessly integrated with U-Net and DiT based methods for both image and video inversion, while introducing ignorable memory and computational cost.\n\n**Q2: L103-L104, the paper puts forward a strong dependence, but without any theoretical basis, it seems that the argumentations are somewhat subjective.**\n\n**A2:** Actually, this perspective is proposed by previous literature [7,14,22]. \n\n**Q3: L118, ensemble techniques lack the necessary citations**\n\n**A3:** Thank you for your suggestion. We will add citations in our final version of the paper.\n\n**Q4: Although the paper puts a lot of visualization results, as well as some pictures on the dataset ablation, but the effectiveness of the method is not convincing enough, if we can see some quantitative results on the video and in-depth analysis would be better.**\n\n**A4:** We have included quantitative results in our paper. For the video part, in Fig. 6, we report the time cost, memory occupation, and mean PSNR for each video sequences, listed below the corresponding frames. \n\n**Q5: Why is there not much difference between these transformations in table 5? Is the combination an equal probability selection of flip, patch shuffle and rotation?**\n\n**A5:** \n\n(1) As stated in Sec. 3.3 of our paper, we apply image transformations to form different branches. Empirically, we find different types of transformation do not yield too much difference. We think the reason is that whether to exploit a transformation is more important than which type of transformation to exploit.\n\n(2) Yes, the combination in Tab. 5 refers to an equal probability selection of flip, patch shuffle and rotation. This combined augmentation strategy yields the highest trajectory diversity, and thus obtains the best reconstruction performance observed in Tab. 5.\n\n**Q6: Why is the A white cat below figure 3 missing a leg even after using FreeInv?**\n\n**A6:** We think its left leg is positioned behind the right leg. Since the leg color is also white, the left leg is partially occluded and difficult to distinguish."}, {"review_id": "0v13KLjQle", "reviewer": "Reviewer_rX7b", "summary": "This paper introduces FreeInv, a method to mitigate inversion errors in DDIM. In standard DDIM inversion, cumulative errors degrade reconstruction and editing quality, and existing solutions often incur substantial computational and memory overhead. FreeInv creates a multi-branch ensemble effect by using one-time MC sampling combined with random image transformations, enabling efficient inversion and reconstruction. The authors demonstrate that FreeInv is adaptable to both image and video data, as well as compatible with various editing methods and model architectures.", "questions": "- In Figure 3 and Table 3, the comparisons are limited. In Figure 3, it would be valuable to include other advanced methods (e.g., NTI, PI) that can also be integrated with various editing frameworks. For the user study in Table 3, comparing against other inversion methods would better demonstrate the strengths of the proposed approach.\n\n- The setup for MB-I in Table 6 is unclear. I understand that MB-R uses four branches generated by rotating the input image. What constitutes “independently sampled images” in MB-I? Does this imply using additional distinct images for inversion? Clarification on this point would help readers better understand and interpret the results.\n\n- Have the authors tried using strong augmentations, such as color jitter or contrast adjustments? While these may risk semantic distortion and potentially degrade the ensemble effect, if semantic changes can be controlled, stronger augmentations could increase trajectory diversity and improve performance. It would be helpful to know whether the authors have explored this experimentally.\n\n*Minor Suggestions*\n- In Eq. (2), it appears that “t+1” should be replaced with “t,” since the inversion step goes from t to t+1, and the model $ \\epsilon_{\\theta}$ is applied at timestep t. Accordingly, all subsequent equations should be revised to reflect this correction.", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "**[Strengths]**\n- FreeInv achieves an ensemble effect by applying random transformations, avoiding substantial computational and memory overhead.\n- The method is applicable to both image and video data and is compatible with various editing methods and model architectures.\n\n**[Weaknesses]**\n- Quantitative performance is not superior to the comparison methods.\n- Qualitative results are similar to or, in some cases, slightly worse than those of the comparison methods.", "was_revised": true, "final_justification": "I acknowledge that the theoretical foundation of the proposed method is weak. However, its simplicity and competitive performance compared to existing approaches, without incurring additional computational costs, are its strengths. Additionally, the introduction of the multi-branch concept in the inversion process is a meaningful contribution. Therefore, I raise my score.", "rebuttal": "Thank you for the constructive comments. \n\n**Q1: Quantitative performance is not superior to the comparison methods.**\n\n**A1:** \n\n(1) As demonstrated in Tab. 1-2, no single method consistently outperforms all the others across all metrics, highlighting the inherent trade-offs in current approaches. Taking a full consideration of the reconstruction and editing performance, FreeInv appears balanced and competitive among the previous methods.\n\n(2) The main advantage of FreeInv is its simplicity and efficiency. Moreover, it can be seamlessly integrated with U-Net and DiT based methods for both image and video inversion, while introducing ignorable memory and computational cost.\n\n**Q2: Qualitative results are similar to or, in some cases, slightly worse than those of the comparison methods.**\n\n**A2:** \n\n(1) In this paper, we aim to show that **without additional computational and memory cost**, FreeInv can achieve competitive results compared with previous methods, such as NTI and PI. We present more visualization results in the supplementary material and project page, to demonstrate FreeInv's effectiveness. We hope the reviewer may refer to those results for more comprehensive evaluation of our method. \n\n(2) To further validate the qualitative results of FreeInv, we supplement a user study among different inversion methods in A3 to Q3. The results also verify the visualization quality of our method.\n\n**Q3: In Figure 3 and Table 3, the comparisons are limited. In Figure 3, it would be valuable to include other advanced methods (e.g., NTI, PI) that can also be integrated with various editing frameworks. For the user study in Table 3, comparing against other inversion methods would better demonstrate the strengths of the proposed approach.**\n\n**A3:** \n\n(1) Sorry for making you confused. We have two dimensions to verify the effectiveness of our method. One dimension is to show our method can be combined with various editing methods. In Fig.3, we show that FreeInv can be seamlessly combined with different inversion-based editing methods, including P2P, MasaCtrl, and PnP. The other dimension is to show our method is competitive against other types of inversion method when integrated within the same editing framework. In Fig. 4, we compare the editing results with different inversion methods (e.g., NTI, PI) , with P2P as the common editing method to ensure fairness. \n\n(2) Thank you for your suggestion, we make a more comprehensive user study that consists of the editing results with different inversion methods, where each participant is asked to choose the most visually appealing and prompt aligned editing results. Finally, we receive 130 votes from a participant pool.\n\n| User Study    | DDIM | NTI  | EDICT | DI   | VI   | PI   | Ours |\n| ------------- | ---- | ---- | ----- | ---- | ---- | ---- | ---- |\n| preference(%) | 4.6  | 12.3 | 8.5   | 10.8 | 7.7  | 26.2 | 30.0 |\n\n**Q4: The setup for MB-I in Table 6 is unclear. I understand that MB-R uses four branches generated by rotating the input image. What constitutes “independently sampled images” in MB-I? Does this imply using additional distinct images for inversion? Clarification on this point would help readers better understand and interpret the results.**\n\n**A4:** Yes, MB-I means using additional distinct images for inversion. We will clarify this point in our final version.\n\n**Q5: Have the authors tried using strong augmentations, such as color jitter or contrast adjustments? While these may risk semantic distortion and potentially degrade the ensemble effect, if semantic changes can be controlled, stronger augmentations could increase trajectory diversity and improve performance. It would be helpful to know whether the authors have explored this experimentally.**\n\n**A5:** \n\n(1) FreeInv does not impose any spatial or semantic constraints on different branches. As demonstrated in Tab. 6, MB-I constituted with distinct images can bring higher improvements compared to MB-R, for the reason that distinct images enrich the diversity of the inversion trajectories. \n\n(2) During the rebuttal phase, we add the experiment applying color jitter, where we alter the brightness, contrast, and brightness of the original image. The results show that jitter can also serves as an alternative. \n\n|    Methods    | PSNR  | LPIPS($\\times10^-2$) | MSE($\\times10^-3$) | SSIM |\n| :-----------: | :---: | :------------------: | :----------------: | :--: |\n|     DDIM      | 25.04 |         9.14         |        4.43        | 0.77 |\n|     flip      | 27.47 |         5.43         |        2.55        | 0.80 |\n| patch shuffle | 27.61 |         5.18         |        2.55        | 0.80 |\n|   rotation    | 27.64 |         5.14         |        2.45        | 0.81 |\n|    jitter     | 27.53 |         5.40         |        2.55        | 0.80 |\n\n**Q6: In Eq. (2), it appears that “t+1” should be replaced with “t,” since the inversion step goes from t to t+1, and the model is applied at timestep t. Accordingly, all subsequent equations should be revised to reflect this correction.**\n\n**A6:** In Eq. (2) we follow the officially released code of [8,38], and use $t+1$ rather than $t$."}]

Author rebuttal:

Meta-review: This paper presents a lightweight method, called FreeInv, for improving the quality of image and video inversion. FreeInv approximates a multi-trajectory ensemble by randomly transforming the latent representation at each time step and keeping it consistent during inversion and reconstruction, significantly reducing reconstruction errors with little cost. All reviewers acknowledge the simplicity and practical effectiveness of the proposed method. Their main concern is that the theoretical analysis of the method is not sufficiently rigorous. The authors’ response addressed most of the reviewers' concerns, resulting in a final set of ratings: 3 "Borderline Accept" and 1 "Borderline Reject". The reviewer who assigned the "Borderline Reject" rating (aAd8) noted they would be comfortable if the paper were accepted. After considering the paper, the reviews, and the rebuttal, the Area Chair recognizes that this paper provides a novel method for improving DDIM inversion, which is simple, clean, and effective. In all, the Area Chair believe that this work makes a valuable contribution to the field and meets the standards of NeurIPS. Accordingly, the Area Chair recommends acceptance. The authors are requested to revise the paper by addressing the reviewers’ comments in the camera-ready version.
Decision: Accept (poster)