Lightning-Fast Image Inversion and Editing for Text-to-Image Diffusion Models

## Abstract

Abstract Diffusion inversion is the problem of taking an image and a text prompt that describes it and finding a noise latent that would generate the exact same image.
Most current deterministic inversion techniques operate by approximately solving an implicit equation and may converge slowly or yield poor reconstructed images. We formulate the problem by finding the roots of an implicit equation and devlop a method to solve it efficiently. Our solution is based on Newton-Raphson (NR), a well-known technique in numerical analysis. We show that a vanilla application of NR is computationally infeasible while naively transforming it to a computationally tractable alternative tends to converge to out-of-distribution solutions, resulting in poor reconstruction and editing. We therefore derive an efficient guided formulation that fastly converges and provides high-quality reconstructions and editing. We showcase our method on real image editing with three popular open-sourced diffusion models: Stable Diffusion, SDXL-Turbo, and Flux with different deterministic schedulers. Our solution, Guided Newton-Raphson Inversion , inverts an image within 0.4 sec (on an A100 GPU) for few-step models (SDXL-Turbo and Flux.1),
opening the door for interactive image editing. We further show improved results in image interpolation and generation of rare objects.

## 1 Introduction

Text-to-image diffusion models (Rombach et al., ; Saharia et al., ; Ramesh et al., ; Balaji et al., ) can generate diverse and high-fidelity images based on user-provided text prompts.
These models are further used in several important tasks that require inversion, namely,
discovering an initial noise (seed) that, when subjected to a backward (denoising) diffusion process along with the prompt, generates the input image. Inversion is used in various tasks including image editing (Hertz et al., ), personalization (Gal et al., ; ), seed noise interpolation for semantic augmentation  (Samuel et al., ) and generating rare concepts (Samuel et al., ).

As inversion became a critical building block in various tasks, several inversion methods have been suggested. Denoising Diffusion Implicit Models (DDIM) (Song et al., ) introduced a deterministic and fast sampling technique for image generation with diffusion models. DDIM inversion transforms an image back into a latent noise representation by approximating the inversion equation. Although this approximation makes it very fast, it also introduces an approximation error (as explained in section 3), causing noticeable distortion artifacts in the reconstructed images. This is particularly noticeable in few-step diffusion models (Luo et al., ; Sauer et al., ; Esser et al., ), with large gaps between the diffusion time-steps, and where inference is achieved with only 2 – 4 steps. Several efforts have been made to address inconsistencies in DDIM Inversion, which result in poor reconstruction quality  (Mokady et al., ; Wallace et al., ; Pan et al., ).
For example, (Pan et al., ; Garibi et al., ) used fixed-point iterations to solve the DDIM-inversion equation. Another approach in (Hong et al., ) tackled the minimization of residual error in DDIM-inversion via gradient descent. Although these methods demonstrate improvements over previous approaches, their editing quality and computational speed are still limited.

In this paper, we frame the deterministic diffusion inversion problem as finding the roots of an implicit function. We propose a solution based on the Newton-Raphson (NR) numerical scheme (Burden et al., ) - a very fast and well-tested optimization method for root finding.
We further define a scalar NR scheme for this problem that can be computed efficiently.
However, NR has a main disadvantage in our context. When applied to our highly non-convex function, it tends to find roots that drift outside the distribution of latents that the diffusion model was trained on. To address this, we introduce a method to guide the NR iterations towards in-distribution roots. This is done by incorporating prior knowledge about the noise distribution added at each diffusion step. This approach enables rapid inversion while maintaining state-of-the-art reconstruction and editing quality.

We name our approach GNRI, for Guided Newton Raphson Inversion. GNRI converges in just a few iteration steps at each diffusion time step and achieves high-quality image inversions (in terms of reconstruction accuracy). In practice, 1-2 iterations are sufficient for convergence that yields significantly more accurate results than other inversion methods.
Importantly, GNRI requires no model training, fine-tuning, prompt optimization, or additional parameters, and is compatible with all pre-trained diffusion models and deterministic schedulers. We demonstrate its effectiveness for inverting different deterministic schedulers used with latent diffusion model (Stable Diffusion) (Rombach et al., ), few-step latent diffusion model (SDXL-turbo) (Sauer et al., ) and few-step flow matching model (Flux.1) (Black-Forest, ).
Figure 1 demonstrates the quality and speed of GNRI for iterative editing using few-step diffusion models. Using such models models that require 4 denoising steps, our approach can edit real images within 0.4 seconds. This allows users to edit images on the fly, using text-to-image models.

We conduct a comprehensive evaluation of GNRI. First, we directly assess the quality of inversions found with GNRI by measuring reconstruction errors compared to deterministic inversion approaches. Our method suppresses all methods with $\times 2$ to $\times 40$ speedup gain. We then demonstrate the benefit of GNRI in two downstream tasks (1) In Image editing, GNRI smoothly changes fine details in the image in a consistent and coherent way, whereas previous methods struggle to do so.
(2) In Seed Interpolation and Rare concept generation (Samuel et al., ) that require diffusion inversion.
In both of these tasks, GNRI yields more accurate seeds, resulting in superior generated images, both qualitatively and quantitatively.

## 2 Related work

Text-to-image diffusion models (Rombach et al., ; Saharia et al., ; Ramesh et al., ; Balaji et al., ; Esser et al., ) translate random samples (seeds) from a high-dimensional space, guided by a user-supplied text prompt, into corresponding images. DDIM (Song et al., ) is a widely used
deterministic scheduler, that demonstrates the inversion of an image to its latent noise seed. When
applied to the inversion of text-guided diffusion models, DDIM inversion suffers from low reconstruction accuracy that is reflected in further tasks, particularly when applied to few-step diffusion models that require only 3-4 denoising steps. This happens because DDIM inversion relies on a linear approximation, causing the propagation of errors that result in inaccurate image reconstruction.
Recent studies (Mokady et al., ; Wallace et al., ; Pan et al., ; Hong et al., ) address this limitation. Null-text inversion (Mokady et al., ) optimizes the embedding vector of an empty string. This ensures that the diffusion process calculated using DDIM inversion, aligns with the reverse diffusion process.
(Miyake et al., ) replace the null-text embedding with a prompt embedding instead. This enhances convergence time and reconstruction quality but results in inferior image editing performance.
In both (Mokady et al., ) and (Miyake et al., ), the optimized embedding must be stored, resulting in nearly 3 million additional parameters for each image (using 50 denoising steps of StableDiffusion  (Rombach et al., )).

EDICT (Wallace et al., ) introduced invertible neural network layers, specifically Affine Coupling Layers, to calculate both backward and forward diffusion paths.
While effective, it comes at the cost of prolonging inversion time.
BDIA (Zhang et al., ) introduced a novel approximation tailored for EDICT, enhancing its computational efficiency while maintaining the accuracy in diffusion inversion. However, it still demands considerably more time, approximately ten times longer than DDIM Inversion. TurboEdit (Wu et al., ) introduced an encoder-based iterative inversion technique for few-step diffusion models, enabling precise image inversion and disentangled image editing. DirectInv (Ju et al., ) separates the source and target diffusion branches for improved text-guided image editing, leading to better content preservation and edit fidelity.
AIDI (Pan et al., ) and ReNoise (Garibi et al., ) used a fixed-point iteration technique at each inversion step to find a better solution for the implicit function posed by DDIM equations and thereby achieving improved accuracy. ExactDPM (Hong et al., ), similar to  (Pan et al., ; Garibi et al., ) proposed gradient-based methods for finding effective solutions for the implicit function.

Alternative methods proposed in (Huberman-Spiegelglas et al., ; Brack et al., ; Deutch et al., ) use a stochastic DDPM inversion instead of a deterministic one. Although it can recover the exact image during reconstruction, the stochastic nature of these approaches usually causes the method to struggle with reconstructing fine details during editing. These methods require many steps for high-quality editing, making the process time-consuming. Additionally, they demand a larger memory footprint, as they need to store $T+1$ latents for each inverted image, restricting their use only to reconstruction and editing tasks. We also compare our approach with these methods in terms of editing quality and demonstrate that our method outperforms them all.

## 3 Preliminaries

We first establish the fundamentals of Denoising Diffusion Implicit Models (DDIMs).
In this model, a backward pass (denoising) is the process that generates an image from a seed noise. A forward pass is the process of adding noise gradually to an image until it becomes pure Gaussian noise. Inversion (Song et al., ) is similar to the forward pass but the goal is to end with a specific Gaussian noise that would generate the image if denoised.

Forward Pass in Diffusion Models.
Diffusion models (Rombach et al., ) learn to generate images through a systematic process of iteratively adding Gaussian noise to a latent data sample until the data distribution is mostly noise. The data distribution is subsequently gradually restored through a reverse diffusion process initiated with a random sample (noise seed) from a Gaussian distribution.
In more detail, the process of mapping a (latent) image to noise is a Markov chain that starts with $z_{0}$, and gradually adds noise to obtain latent variables $z_{1},z_{2},\dots,z_{T}$, following a distribution
$q(z_{1},z_{2},\ldots,z_{T}|z_{0})=\Pi^{T}_{t=1}{q(z_{t}|z_{t-1}})$,
where $\forall t:z_{t}\in\mathbb{R}^{d}$ with $d$
denoting the dimension of the space. Each step in this process is a Gaussian transition, that is, $q$ follows a Gaussian distribution

$$ $q(z_{t}|z_{t-1}):=\mathcal{N}(z_{t};\mu_{t}=\gamma_{t}z_{t-1},\Sigma_{t}=\beta _{t}I),$ (1) $$

parameterized by a schedule $(\beta_{0},\gamma_{0}),\ldots,(\beta_{T},\gamma_{T})\in(0,1)^{2}$. As discussed below, in DDIM, $\gamma_{t}=\sqrt{1-\beta_{t}}$ and in Euler scheduling $\beta_{t}=\gamma_{t}=1$ $\forall t$.

Deterministic schedule diffusion models.
It has been shown that one can “sample” in a deterministic way from the diffusion model, and this accelerates significantly the denoising process. Several deterministic schedulers have been proposed (Song et al., ; Lu et al., ; Karras et al., ), we describe here two popular ones, DDIM and Euler schedulers.

Denoising Diffusion Implicit Models (DDIM).
Sampling from diffusion models can be viewed as solving the corresponding diffusion Ordinary Differential Equations (ODEs) (Lu et al., ). DDIM (Song et al., ), a popular deterministic scheduler, proposed to denoise a latent
by

$$ $z_{t-1}=\sqrt{\frac{\alpha_{t-1}}{\alpha_{t}}}z_{t}-\sqrt{\alpha_{t-1}}\cdot \Delta\psi(\alpha_{t})\cdot\epsilon_{\theta}(z_{t},t,p),$ (2) $$

where $\alpha_{t}=1-\beta_{t}$, $\psi(\alpha)=\sqrt{\frac{1}{\alpha}-1}$, and
$\Delta\psi(\alpha_{t})=\psi(\alpha_{t})-\psi(\alpha_{t-1})$ and
$\epsilon_{\theta}(z_{t},t,p)$ is the output of a network that was trained to predict the noise to be removed.

Euler schedulers.
Euler schedulers follow a similar deterministic update rule

$$ $z_{t-1}=z_{t}+(\sigma_{t-1}-\sigma_{t})v_{\theta}(z_{t},t,p),$ (3) $$

here $\sigma_{t},\sigma_{t}-1$ are scheduling parameters and $v_{\theta}$ is the output of the network that was trained
to predict the velocity.

Diffusion inversion.
We now focus on inversion in the latent representation. We describe our approach first for DDIM, and extension to other schedulers is discussed below. Given an image representation $z_{0}$ and its corresponding text prompt $p$, we seek a noise seed $z_{T}$ that, when denoised, reconstructs the latent $z_{0}$. Several approaches were proposed for this task. DDIM inversion rewrites Eq. (2) as:

$$ $\displaystyle z_{t}$ $\displaystyle=$ $\displaystyle f(z_{t})$ (4) $\displaystyle f(z_{t})$ $\displaystyle:=$ $\displaystyle\sqrt{\frac{\alpha_{t}}{\alpha_{t-1}}}z_{t-1}+\sqrt{\alpha_{t}} \cdot\Delta\psi(\alpha_{t})\cdot\epsilon_{\theta}(z_{t},t,p).$ $$

DDIM inversion approximates this implicit equation in $z_{t}$ by replacing $z_{t}$ with $z_{t-1}$

$$ $f(z_{t})\approx\sqrt{\frac{\alpha_{t}}{\alpha_{t-1}}}z_{t-1}+\sqrt{\alpha_{t}} \cdot\Delta\psi(\alpha_{t})\cdot\epsilon_{\theta}({\color[rgb]{1,0,0}z_{t-1}}, t,p).$ (5) $$

The quality of the approximation depends on the difference
$z_{t}-z_{t-1}$ (a smaller difference would yield a small error) and on the sensitivity of $\epsilon_{\theta}$ to that $z_{t}$. See (Dhariwal & Nichol, ; Song et al., ) for details.
By applying Eq. (5) repeatedly for every denoising step $t$, one can invert an image latent $z_{0}$ to a latent $z_{T}$ in the seed space.

DDIM inversion is fast, but the approximation of Eq. (5) inherently introduces errors at each time step. As these errors accumulate, they cause the whole diffusion process to become inconsistent in the forward and the backward processes, leading to poor image reconstruction and editing (Mokady et al., ; Wallace et al., ; Pan et al., ). This is particularly noticeable in few-step and consistency models with a small number of denoising steps (typically 2-4 steps), where there’s a significant gap between $z_{t}$ and $z_{t-1}$ Garibi et al. ().

This inversion technique can also be applied to other deterministic schedulers. For instance, for Euler scheduler, one defines
$z_{t}=f(z_{t})$, $f(z_{t}):=z_{t-1}+\left(\sigma_{t}-\sigma_{t-1}\right)v_{\theta}(z_{t},t,p)$
,and this is approximated by $z_{t-1}+\left(\sigma_{t}-\sigma_{t-1}\right)v_{\theta}({\color[rgb]{1,0,0}z_{t
-1}},t,p)$.

Iterative inversion and optimization methods.
Several papers proposed to improve the approximation using iterative methods.
AIDI (Pan et al., ) and ReNoise (Garibi et al., ) proposed to directly solve Eq. (3) using fixed-point iterations (Burden, ), a widely-used method in numerical analysis for solving implicit functions.
In a related way, (Hong et al., ) solves a more precise inversion equation,
obtained by employing higher-order terms, using gradient descent.

Newton-Raphson Root finding. The Newton-Raphson method is a widely used numerical technique for finding roots of a real-valued function $\mathcal{F}$ (Burden et al., ). It is particularly effective for solving equations of the form $\mathcal{F}(z)=0$, when $\mathcal{F}:\mathbb{R}^{D}\rightarrow\mathbb{R}^{D}$ for an arbitrary dimension $D$. It provides fast convergence, typically quadratic, by requiring the evaluation of the function and the inversion of its Jacobian matrix. It has also been shown that when initialized near a local extremum, NR may converge to that point (oscillating around it) (Kaw et al., ). The NR scheme in general form, is given by

$$ $z^{k+1}_{t}=z^{k}_{t}-J(z^{k}_{t})^{-1}\leavevmode\nobreak\ \mathcal{F}(z^{k}_ {t}),$ (6) $$

where $J(z_{t}^{k})^{-1}$ presents the inverse of a Jacobian matrix $J\in\mathbb{R}^{D\times D}$ for $\mathcal{F}$ (all derivatives are w.r.t to $z$), and $k$ stands for the iteration number. The iteration starts with an initial guess, $z=z^{0}$.

Figure: Figure 2: Newton-Raphson Inversion iterates over an implicit function Eq. 3 using Eq. 10 scheme, at every time-step in the inversion path. Initialized with $z^{0}_{t}=z_{t-1}$ it converges within $\approx$ 2 iterations, to $z_{t}$. Each box denotes one inversion step; black circles correspond to intermediate latents in the denoising process; green circles correspond to intermediate Newton-Raphson iterations.
Refer to caption: extracted/5948112/figures/iclr25/method.png

## 4 Our method: Guided Newton Raphson Inversion

Existing inversion methods are either fast-but-inaccurate, like DDIM inversion, or more precise-but-slow, like (Pan et al., ) and (Hong et al., ).
This paper presents a method that achieves a balance of speed and improved precision compared to these existing approaches.

To achieve these features we introduce two key ideas. First, we frame inversion as a scalar root-finding problem that can be solved efficiently, and use the well-known Newton-Raphson root-finding method.
Second, we make the observation that an inversion step is equivalent to finding a specific noise vector, and we use a strong prior obtained from the distribution of that noise. This prior during the iterative solution guides Newton-Raphson towards finding a root that adheres to the correct distribution of latents. See Figure 2 for illustration.

### 4.1 Framing inversion as efficient root-finding

Diffusion inversion can be done by finding the fixed point of the function $f(z)$ in Eq. 3. This is equivalent to finding the zero-crossings or roots, $z_{t}$ of the residual function $r:\mathbb{R}^{D}\rightarrow\mathbb{R}^{D}$ defined as
$r(z_{t}):=z_{t}-f(z_{t})$.
One could now apply Newton-Raphson (NR) to find a root.
However, in our case, $z_{t}$ is a high dimensional latent($D\approx 16K$ in StableDiffusion (Rombach et al., )), making it computationally infeasible to compute the Jacobian (Eq. 6), explicitly and invert it.
To address this computational limitation, we apply a norm over $r(z_{t})$, $\hat{r}:\mathbb{R}^{d}\rightarrow\mathbb{R}^{+}_{0}$ yielding a multi-variable, scalar function, which has the same set of roots:

$$ $\displaystyle\hat{r}(z_{t})$ $\displaystyle:=$ $\displaystyle||z_{t}-f(z_{t})||_{1}\quad,$ (7) $$

and seek the roots $\hat{r}(z_{t})=0$ where $||\cdot||$ denotes a $L_{1}$ norm which simply sum over all absolute values of $\hat{r}$. By reducing the problem from $r$ to $\hat{r}$, the Jacobian becomes a vector and can be computed quickly (see derivation in Appendix A).
We call this method Newton-Raphson Inversion, NRI.

This method is efficient and converges quickly in practice. However, NRI in our case has a key limitation: Eq. (3), and thus $\hat{r}$, can have multiple solutions or local minima. The accelerated convergence of NR due to linear “extrapolation”, can cause it to ”jump” to solutions outside the latent variable distribution for the diffusion model. Indeed, in practice, we find that solving Eq. (7) for $\hat{r}(z_{t})=0$ often converges quickly to a solution that results with poor image reconstruction. We quantify and illustrate this phenomenon in section 5.1.
Next, we describe a remedy to this problem.

### 4.2 Guided Newton Raphson Inversion

The NR procedure discussed above may find roots that are far from the modes of the latent distribution. How can we guide NR to converge to useful roots among all possible ones?

Note that determining a $z_{t}$ given $z_{t-1}$, is equivalent to finding which noise term is added during the forward pass of the diffusion process. Luckily, the distribution of that noise is known by the design of the diffusion process and we can use it as prior information.
More precisely,
since each step in the diffusion process follows a Gaussian distribution $q(z_{t}|z_{t-1})$ (Eq. 1), we define a positive function that obtains a zero value at the maximum likelihood of the $q$ distribution by taking the negative log of the exponent part of the distribution (without the normalizing factor). This can be simplified to

$$ $G(z_{t})=\begin{cases}\frac{1}{\beta_{t}}||z_{t}-\mu_{t}||&\text{DDIM},\\ ||z_{t}-\mu_{t}||&\text{Euler}.\\ \end{cases}$ (8) $$

where $\mu_{t}$, $\beta_{t}$ were defined in Section 3, see Eq (1).
To guide NR to select an ”in-distribution” solution,
we define a new objective:

$$ $\mathcal{F}(z_{t}):=||z_{t}-f(z_{t})||_{1}+\lambda G(z_{t}).$ (9) $$

Here,
$\lambda>0$ is a hyperparameter weighting factor for the guidance term. See Appendix D for ablation of $\lambda$ values.

We then seek to drive $\mathcal{F}(z_{t})$ near zero. Applying the Newton Raphson iteration scheme for Eq. (9) then emerges as (see derivation in Appendix A):

$$ $\displaystyle z_{t}^{0}$ $\displaystyle=z_{t-1}$ $\displaystyle z_{t}^{k+1}(i)$ $\displaystyle=z_{t}^{k}(i)-\frac{1}{D}\leavevmode\nobreak\ \frac{{\mathcal{F}} (z_{t}^{k})}{g_{i}(z_{t}^{k})+\eta}.$ (10) $$

Here $g_{i}:=\frac{\partial\mathcal{F}(z_{t})}{\partial z_{t}(i)}$ is the partial derivative of $\mathcal{F}$ with respect to the variable (descriptor) $z_{t}(i)$. The index $i\in[0,1,...,D]$ indicates $z$’s components and $k$ stands for the iteration number, while $\eta$ is a small constant added for numerical stability. The function
$g$ can be computed efficiently using automatic differentiation engines. We initialize the process with $z$ from the previous diffusion time-step $z_{t-1}$.

We call our approach GNRI for Guided Newton Raphson Inversion.

## 5 Experiments

Here, we first provide motivation and analysis for the guidance term presented in Section 4.2. Then, we show evaluation results of GNRI on three main tasks: (1) Image inversion and reconstruction: Here, We assess the inversion fidelity by evaluating the quality of the reconstructed images. (2) Image Editing: We demonstrate the efficacy of our inversion scheme in image editing, highlighting its ability to facilitate real-time editing. (3) Rare Concept Generation: We illustrate how our method can be applied to improve the generation of rare concepts.

Compared Methods:
We compared our approach with the following methods. Standard DDIM Inversion (Song et al., ), SDEdit Meng et al. (), Null-text (Mokady et al., ) and EDICT (Wallace et al., ). Two fixed-point based methods: AIDI (Pan et al., ) and ReNoise (Garibi et al., ). Gradient-based method ExactDPM (Hong et al., ). Wu2024Turbo (Wu et al., ) and two DDPM inversion methods: EditFriendly Huberman-Spiegelglas et al. () and Gal2024Turbo (Deutch et al., ).

In all experiments, we used code and parameters provided by the respective authors.

Implementation details: To demonstrate the versatility of our approach, we conducted experiments on latent diffusion model SD2.1 Rombach et al. () and two few-step models: SDXL-turbo (Sauer et al., ) and the newly introduced flow matching Flux.1-schnell model (Black-Forest, ). Sampling steps were set to 50 for SD2.1 and 4 for SDXL-turbo and Flux.1. All methods ran until convergence. All methods were tested on a single A100 GPU for a fair comparison. PyTorch’s built-in gradient calculation was used for computing derivatives of Eq. (9).
Editing for all iterative methods was done using Prompt-to-Prompt (Hertz et al., ).

### 5.1 Guidance Term Motivation and Comparison to Other Numerical Schemes

To illustrate the motivation behind our guidance term,
we compare the reconstruction quality of non-guided NRI a ”vanilla” NR inversion method with two iterative numerical schemes: Fixed-point Iteration (FPI) and Gradient Descent (GD). We test the reconstruction of 5,000 image-caption pairs from COCO Lin et al. (), presenting three metrics in Fig. 3.

Fig 3a shows that FPI, GD and non-guided NRI, all converge to low residual values but the reconstruction values for NRI (green line) in Fig. 3b are notably lower. This suggests that solutions reached by NRI are in areas of the latent space that are not processed correctly by the diffusion model.

Figure: Figure 3: The effect of GNRI guiding term on NR inversion, and comparison to other iterative inversion methods. All results are averages computed with SDXL-Turbo applied to 5,000 COCO images. (a) Average residuals throughout optimization. NR-based methods are the fastest to converge. Gradient-descent was run with the largest learning rate that was stable but still is slowest. (b) Reconstruction quality (PSNR). Adding guidance (blue) to NRI (green) significantly improves the quality of the converged solution. (c) Likelihood of inferred noise. Without the guiding term, NRI (green) finds solutions that are substantially different from those found by other methods, which explains the low reconstruction quality.
Refer to caption: extracted/5948112/figures/iclr25/analysis.png

Figure: Figure 4: (Left) Reconstruction qualitative results: Comparing image inversion-reconstruction performance. While all baseline methods struggle to preserve the original image, GNRI successfully excels in accurately reconstructing it. (Right) Inversion Results: Mean reconstruction quality (y-axis, PSNR) and runtime (x-axis, seconds) on the COCO2017 validation set. Our method achieves high PSNR while reducing inversion-reconstruction time by a factor of $\times 2$ to $\times 40$ on SDXL-turbo and $\times 10$ to $\times 40$ on Flux.1, compared to other approaches.
Refer to caption: x1.png

Comparing the blue curve in Fig 3 to the green curve highlights the significant impact of adding a guidance term. It accelerates convergence
(Fig. 3a), improves
reconstruction (PSNR) (Fig 3b) and reaches a better likelihood (Fig. 3c).

### 5.2 Image Reconstruction

To evaluate the fidelity of our approach, we measure the PSNR of images reconstructed from seed inversions.
Specifically, we used the entire set of 5000 images from the MS-COCO-2017 validation dataset (Lin et al., ), along with their corresponding captions. For each image-caption pair, we first found the inverted latent and then used it to reconstruct the image using the same caption.
Given that the COCO dataset provides multiple captions for each image, we used the first listed caption as a conditioning prompt.
Figure 4 (left) demonstrates a qualitative comparison of reconstructed images.
Figure 4 (right) illustrates the PSNR of reconstructed images versus inversion time, highlighting the performance of our method compared to SoTA inversion techniques. The dashed black line represents the upper bound set by the Stable Diffusion VAE, showing that our method achieves the highest PSNR among SoTA methods and approaches the VAE upper bound, while also being the fastest. Additional qualitative comparisons for SD2.1 and further analysis are in Appendix G.

Figure: Figure 5: (Left) Qualitative results of image editing. GNRI edits images more naturally while preserving the structure of the original image. All baselines were executed until they reached convergence. (Right) Evaluation of editing performance: GNRI achieves superior CLIP and LPIPS scores, indicating better compliance with text prompts and higher structure preservation.
Refer to caption: x2.png

### 5.3 Real-Time Image Editing

Current state-of-the-art methods for editing images start by inverting a real image into latent space and then operating on the latent. As a result, the quality of editing depends strongly on the quality of inversion (Mokady et al., ).

We now evaluate the effect of inverting images with GNRI, compared to SoTA inversion baselines.
We show that GNRI outperforms all baselines in editing performance while requiring the shortest time. This opens the door for real-time editing capabilities.
We present below both qualitative and quantitative results.

Qualitative results.
Figure 5(left) gives a qualitative comparison between SoTA approaches and GNRI in the context of real image editing. GNRI excels in accurately editing with target prompts producing images with both high fidelity to the input image and adherence to the target prompt. The examples illustrate how alternative approaches may struggle to retain the original structure or tend to overlook crucial components specified in the target prompt.
As an example, in the first row of Figure 5(left), GNRI exclusively converts the cat into a dog without changing the text in the sign whereas other methods either fail to generate a proper dog or change the text in the sign. Results for SD2.1 in Appendix. G.

User Study.
We further evaluated editing quality using human raters. We followed the evaluation process of Mokady et al. () and asked human raters to rank edits made by the three leading methods: Ours, ReNoise (Garibi et al., ), and ExactDPM (Hong et al., ). Unfortunately, the code for TurboEdit (Wu et al., ) was not available at the time of our evaluation. 100 images were rated, 12 images were provided by the authors of (Mokady et al., ) and the rest were randomly selected from COCO (Lin et al., ) and PieBench (Ju et al., ) benchmarks. Three raters for each image were recruited through Amazon Mechanical Turk and were tasked to choose the image that better applies the requested text edit while preserving most of the original image. GNRI was preferred with 71.6% preference compared to ReNoise’s (Garibi et al., ) 15.9% and ExactDPM (Hong et al., ) 12.5%.

Quantitative results.
Following (Wallace et al., ; Huberman-Spiegelglas et al., ; Garibi et al., ), we evaluate the results using two
complementary metrics: LPIPS (Zhang et al., ) to quantify the extent of structure preservation (lower is better) and a CLIP-based score to quantify how well the generated images comply with the text prompt (higher is better). Metrics are averaged across 100 MS-COCO images. Figure 5(right) illustrates that editing with GNRI yields a superior CLIP and LPIPS score, demonstrating the ability to perform state-of-the-art real image editing with superior fidelity.
Following (Wu et al., ) we also evaluated our approach on the newly introduced PieBench (Ju et al., ) dataset. Table 1 shows that our method more effectively follows the text guidance while preserving the background, outperforming current SoTA techniques.
These evaluations further affirm the findings derived from the user study.

**Table 1: Image editing comparison using descriptive text in PIE-Bench (Ju et al., ) dataset with SDXL-turbo. The efficiency is measured in a single A100 GPU. Our method achieves the best background preservation and clip similarity while being significantly faster than other methods.**
| PIE-Bench | Background Preservation | CLIP Similarity | Efficiency |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
|  | PSNR | LPIPS | MSE | SSIM | Whole | Edited | Inverse | Forward |
|  | $\uparrow$ | $10^{3}\downarrow$ | $10^{4}\downarrow$ | $10^{2}\uparrow$ | $\uparrow$ | $\uparrow$ | (s) $\downarrow$ | (s) $\downarrow$ |
| DDIM | 18.59 | 177.96 | 184.69 | 66.86 | 23.62 | 21.20 | 0.454 | 0.445 |
| ReNoise (Garibi et al., ) | 27.11 | 49.25 | 31.23 | 72.3 | 23.98 | 21.26 | 5.41 | 0.445 |
| ExactDPM (Hong et al., ) | 24.54 | 59.88 | 36.49 | 69.18 | 23.77 | 21.23 | 15.11 | 0.445 |
| Direct Inv (Ju et al., ) | 27.22 | 54.55 | 62.86 | 84.76 | 25.02 | 22.10 | 4.3 | 50 |
| TurboEdit (Wu et al., ) | 29.52 | 44.74 | 26.08 | 91.59 | 25.05 | 22.34 | 0.981 | 0.569 |
| GNRI (Ours) | 30.22 | 40.15 | 20.10 | 98.66 | 26.15 | 23.26 | 0.521 | 0.445 |

### 5.4 Seed Interpolation and Rare Concept Generation

**Table 2: Image interpolation and centroid finding. In interpolation, two images $x^{1},x^{2}$ are inverted to generate images between their seeds $z_{T}^{1},z_{T}^{2}$. In centroid-finding, a set of images is inverted to find their centroid. Acc and FID scores improved using GNRI as the inversion method.**
|  | Interpolation | Centroid |  |  |
| --- | --- | --- | --- | --- |
|  | ACC $\uparrow$ | FID $\downarrow$ | ACC$\uparrow$ | FID $\downarrow$ |
| DDIM Inversion | 51.59 | 6.78 | 67.24 | 5.48 |
| ExactDPM (Hong et al., ) | 52.01 | 6.13 | 68.14 | 5.32 |
| ReNoise (Garibi et al., ) | 52.51 | 6.0 | 69.14 | 5.22 |
| GNRI (ours) | 54.98 | 5.91 | 70.18 | 4.59 |

**Table 3: Inversion Quality Impact on Rare Concept Generation: We assess image generation using a pre-trained classifier’s accuracy, comparing NAO (Samuel et al., ) (with DDIM inversion), ReNoise (Garibi et al., ), ExactDPM Zhang et al. () and GNRI. We report average per-class accuracy for Head (over 1M samples), Medium, and Tail (rare classes $<$ 10K samples). GNRI enhances rare and medium concept accuracy without sacrificing overall performance.**
|  | ImageNet1k in LAION2B |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- |
|  | Head | Medium | Tail | Total Acc | FID | $\hat{T}_{Init}$ | $\hat{T}_{Opt}$ |
|  | n=235 | n=509 | n=256 |  |  | (sec) | (sec) |
| Methods | #>1M | 1M>#>10K | 10K># |  |  |  |  |
| DDIM inversion | 98.5 | 96.9 | 85.1 | 94.3 | 6.4 | 25 | 29 |
| ExactDPM (Hong et al., ) | 98.1 | 96.8 | 85.3 | 94.1 | 7.1 | 240 | 32 |
| ReNoise (Garibi et al., ) | 98.5 | 97.0 | 85.3 | 94.4 | 6.9 | 24 | 28 |
| GNRI (ours) | 98.6 | 97.9 | 89.1 | 95.8 | 6.3 | 17 | 25 |

We evaluate the benefit of our inversion in a second task, of selecting latent seeds for image generation. It has been shown (Samuel et al., ; ) that text-to-image diffusion models poorly generated rare concepts, and this can be improved by selecting better noise latent $z_{T}$.
To address this issue, SeedSelect (Samuel et al., ) takes a few images of a rare concept as input and uses a diffusion inversion module to iteratively refine the obtained seeds. These refined seeds are then used to generate new plausible images of the rare concept. NAO (Samuel et al., ) extends this by introducing new paths and centroids for seed initialization. Both methods rely on DDIM Inversions, crucial for initial seed evaluation. GNRI provides an alternative for precise inversion seeds, aiming for improved image quality and semantic accuracy. It’s important to highlight that most inversion techniques (Mokady et al., ; Wallace et al., ; Huberman-Spiegelglas et al., ; Brack et al., ) are not applicable in this context as they necessitate extra parameters for image reconstruction, which impedes the straightforward implementation of interpolation.

Interpolation and centroid finding:
We evaluate GNRI by following the experimental protocol of (Samuel et al., ) and compare images generated by four methods: DDIM, ReNoise (Garibi et al., ), ExactDOM (Hong et al., ) and GNRI. Evaluation is conducted based on FID score and image accuracy, assessed using a pre-trained classifier. See details in (Samuel et al., ). Results are presented in Table 2. Notably, initializing with GNRI seeds consistently results in higher-quality images both in interpolation paths and seed centroids.

Rare concept generation:
We further show the effect of our seed inversions on the performance of NAO centroids with SeedSelect for rare concept generation. Specifically, we compared images generated by SeedSelect, initialized with NAO using DDIM inversion, ReNoise, ExactDPM, and GNRI, as shown in Table 3). We followed the evaluation protocol of (Samuel et al., ; ) on ImageNet1k classes arranged by their prevalence in the LAION2B dataset (Schuhmann et al., ). This dataset serves as a substantial “in the wild” dataset employed in training foundation diffusion models, such as Stable Diffusion (Rombach et al., ). Image quality is measured by FID and accuracy of a pre-trained classifier. For more details see (Samuel et al., ; ).
The results, summarized in Table 3, demonstrate that our inversion method significantly boosts performance, both in accuracy and FID, compared to other methods. Furthermore, our inversions yield a more precise and effective initialization point for SeedSelect (Samuel et al., ), resulting in notably quicker convergence without compromising accuracy or image quality, in all categories (head, medium, and tail) along with a high gap at tail.

## 6 Summary

Image inversion in diffusion models is vital for various applications like image editing, semantic augmentation, and generating rare concept images. Current methods often sacrifice inversion quality for computational efficiency, requiring significantly more compute resources for high-quality results. This paper presents Guided Newton-Raphson Inversion (GNRI), a novel iterative approach that balances rapid convergence with superior accuracy, execution time, and memory efficiency. Using GNRI opens the door for high-quality real-time image editing due to its efficiency and speed. GNRI requires no model
training, fine-tuning, prompt optimization, or additional parameters, and is compatible with all pre-trained diffusion models and deterministic schedulers.

## Appendix A Newton method for Multivariable Scalar Function

In this section, we present the formulation of Newton’s method for zero crossing of a multi-variable scalar function. For a vector $\mathbf{x}\in\mathbb{R}^{D}$ and a function $f(\mathbf{x}):\mathbb{R}^{D}\rightarrow\mathbb{R}$ we are looking for the roots of the equation $f(\mathbf{x})=0$. Assuming that the function $f$ is differentiable, we can use Taylor expansion:

$$ $f(\mathbf{x}+\delta)=f(\mathbf{x})+\nabla f\leavevmode\nobreak\ \delta^{T}+o(| |\delta||^{2})$ (A1) $$

For every $\delta,\mathbf{x}\in\mathbb{R}^{D}$. The idea of Newton’s method in higher dimensions is very similar to the one-dimensional scalar function;
Given an iterate $\mathbf{x}^{k}$, we define the next iterate $\mathbf{x}^{k+1}$ by linearizing the equation $f(\mathbf{x}^{k})=0$ around $\mathbf{x}^{k}$ as above, and solving the linearized equation $f(\mathbf{x}^{k}+\delta)=0$. Dropping the higher orders we then solve the remaining linear equation for $\delta$.
In fact, in the scalar case, the solution for $\delta$ is not unique, with each solution potentially determining a different direction of propagation of the iterate $\mathbf{x}^{k}$ in the high-dimensional input space. The optimal direction cannot be known in advance, and some have proposed selecting the propagation direction by imposing constraints, such as minimizing $\delta$ (Polyak & Tremba, ).
We adopt a simple, non-sparse, yet effective solution, that allows the scheme flexibility in deriving the direction of propagation:

$$ $\delta=-\frac{1}{D}\left[\frac{1}{\frac{\partial f}{\partial x_{1}}},\frac{1}{ \frac{\partial f}{\partial x_{2}}},...,\frac{1}{\frac{\partial f}{\partial x_{ D}}}\right]f(\mathbf{x}^{k})$ (A2) $$

where $x_{i}$ indicates the $i$-th component of vector $\mathbf{x}$. The above can be easily proven as a solution by substituting $\delta$ into Eq. A1, dropping the higher orders setting it equal to zero. Using the relation, $\delta=\mathbf{x}^{k+1}-\mathbf{x}^{k}$ we get the final iterative scheme:

$$ $x_{i}^{k+1}=x_{i}^{k}-\frac{1}{D}\frac{f(\mathbf{x}^{k})}{\frac{\partial f}{ \partial x_{i}}(\mathbf{x}^{k})}$ (A3) $$

Note that this equation is component-wise i.e. each component is updated separately, facilitating the solution.

When $f>0$ and does not have zero crossings, the NR procedure may converge to a (non-zero) minima. A common example is the function $f(x)=x^{2}+1$ that has no real roots, but NR converges to its minimum. We believe that an analysis of this type of convergence within the Newton-Raphson scheme, falls outside the scope of this paper.

## Appendix B GNRI Scheme Contraction Mapping

Fixed-point iterations are guaranteed to converge if the operator is contractive in the region of interest. Formally, a mapping $T$ is contractive on its domain if there exists $\rho<1$ (also called Liphshitz constant) such that
$\|T(x)-T(y)\|\leq\rho\|x-y\|$, $\forall(x,y)\in X$, where $\|\cdot\|$ denotes a norm. In the following we will demonstrate empirically that our GNRI scheme introduces contraction mapping, a property needed for convergence of the scheme. Contractive mappings ensure that each iteration moves closer to the limit, with the distance shrinking at a predictable rate. The rate of contraction (the Lipschitz constant) can be used to give bounds on how fast the error decreases, helping to assess the efficiency of the scheme.
Next, we show empirically that, the mapping is contractive in our region of interest.

According to Eq. 10 the GNRI scheme for the solution $\mathcal{F}(z)=0$ (defined in Eq. 9), at each diffusion time-step, is given by:

$$ $z^{k+1}_{i}=z^{k}_{i}-\frac{1}{D}\leavevmode\nobreak\ \frac{{\mathcal{F}}(z^{k })}{g_{i}(z^{k})}$ (B1) $$

where $g_{i}:=\frac{\partial\mathcal{F}}{\partial z_{i}}$ is the partial-derivative of $\mathcal{F}$ with respect to the variable $z_{i}$, while $i\in[0,1,...,D]$ indicates $z$’s components. We can rewrite the NR iteration as the fixed-point iteration $\tilde{f}(z)=z$ by setting $\tilde{f}(z)$ as:

$$ $\tilde{f}_{i}(z^{k}):=z^{k}_{i}-\frac{1}{D}\leavevmode\nobreak\ \frac{{ \mathcal{F}}(z^{k})}{g_{i}(z^{k})}$ (B2) $$

Note that the fixed point of the newly introduced $\tilde{f}$ is the solution of $\mathcal{F}(z)=0$.

Let us now define a new residual function:

$$ $\tilde{r}(z^{k}):=||\tilde{f}(z^{k})-z^{k}||$ (B3) $$

By substitution we get:

$$ $\displaystyle\tilde{r}(z^{k+1})$ $\displaystyle=||\tilde{f}(z^{k+1})-z^{k+1})||=||f(z^{k+1})-\tilde{f}(z^{k})||$ (B4) $\displaystyle\tilde{r}(z^{k})$ $\displaystyle=||\tilde{f}(z^{k})-z^{k}||=||z^{k+1}-z^{k}||$ $$

Therefore contraction mapping requirement for $\tilde{f}$: $||\tilde{f}(z^{k+1})-\tilde{f}(z^{k})||\leq\rho||z^{k+1}-z^{k}||$ can be written as:

$$ $\tilde{r}(z^{k+1})\leq\rho\leavevmode\nobreak\ \tilde{r}(z^{k})$ (B5) $$

and the contraction mapping is satisfied if $\tilde{r}$ follows an exponential decay ($r(z^{k})\leq\rho^{k}\leavevmode\nobreak\ r(z^{0})$), with some $\rho<1$. Note that $\tilde{r}(z^{k})$ given in Eq. B3 can be computed at each iteration step. In Fig. B1 we plot $\tilde{r}(z^{k})$, tested on 5,000 image caption pairs from the COCO dataset (the same as used in Sec. 5.1) showing that $\tilde{r}(z^{k})$ indeed decays exponentially (linearly in log-scale), till convergence. This behaviour therefore implies that GNRI iteration is a contraction mapping. Note that our contraction mapping does not converge to zero, namely the fixed point, since our objective function $\mathcal{F}$ may not have an exact solution. Computing contraction rate $\rho$ for GNRI shows $\rho\approx 0.50$ (at the first iteration) while $\rho\approx 0.90$ for FPI indicating the faster convergence of GNRI.

Note that we further show in the main paper the convergence of the equation residual $\hat{r}=||f(z_{t})-z_{t}||$ (EQ. 7) as an indicator for convergence toward the inversion solution.

Figure: Figure B1: Contraction Mapping. Contraction mapping for two fast diffusion models are shown (in log-scale), at each time-step. The scheme converges in 2-3 iterations with a notable contraction occurring in the first iteration, assisting the rapid convergence. $t=0$ is close to the image side and $t=3$ close to seed.
Refer to caption: extracted/5948112/figures/iclr25/RNRI_Scheme_residual.png

## Appendix C Failure Cases

Here we provide information about an analysis to find failure cases for convergence. We ran inversion and reconstruction, in scale, on COCO2017 (118k caption-image pairs) and found that in 95.4% of cases, GNRI successfully converged to a solution. Specifically, residuals went down from $\sim 1$ (for DDIM) to $<10^{-4}$ indicating convergence, and the PSNR was $>25.7$ indicating good solutions. The remaining $4.6\%$ were samples with incorrect captions, see Fig. SC1. These results imply that lack of convergence may indicate text and image miss-alignment which we consider for future work.

Figure: Figure C1: Failure cases: GNRI fails to converge where the prompt and image do not align.
Refer to caption: extracted/5948112/figures/neurips24/supp_failure.png

Figure: Figure C2: The influence of $\lambda$ on reconstruction performance. We observe that employing no regularization ($\lambda=0$) leads to poor reconstruction, while $\lambda=0.1$ typically yields the highest reconstruction accuracy. As $\lambda$ increases, reconstruction accuracy declines, possibly because assigning excessive weight to the prior undermines the root-finding objective.
Refer to caption: extracted/5948112/figures/neurips24/prior_abblation.png

## Appendix D Prior trade-off parameter

Figure C2 shows the impact of the weight parameter $\lambda$ on the reconstruction accuracy. We observe that using no regularization ($\lambda=0$) results in poor reconstruction, whereas setting $\lambda=0.1$ achieves the highest reconstruction accuracy, underscoring the importance of the prior in the objective function. There is a gradual decrease with larger $\lambda$ values, where at $\lambda=1$ the PSNR starts to decline faster, likely because placing too much emphasis on the prior compromises the residual root-finding objective.

Figure: Figure D1: (Left) Inversion Results: Mean reconstruction quality (y-axis, PSNR) and runtime (x-axis, seconds) on the COCO2017 validation set. Our method achieves high PSNR while reducing inversion-reconstruction time by a factor of $\times 4$ to $\times 14$.(Right) Evaluation of editing performance: GNRI achieves superior CLIP and LPIPS scores, indicating better compliance with text prompts and higher structure preservation.
Refer to caption: extracted/5948112/figures/iclr25/supp_sd.png

## Appendix E Seed Exploration & Rare concept generation

Figure: Figure E1: Generating rare concepts based on a few examples with the method of (Samuel et al., ) that heavily depends on the diffusion-inversion quality. In our comparison, we evaluate GNRI alongside ReNoise (refer to the discussion in the main paper). ReNoise frequently struggles to produce a realistic representation of certain objects (such as Oxygen-mask or Patas Monkey) or an accurate depiction of specific concepts (like Tiger-cat or Pay-phone). However, the use of GNRI rectifies these issues in the results. For a detailed quantitative comparison, please refer to the main paper. See the main paper for a quantitative comparison.
Refer to caption: extracted/5948112/figures/iclr25/centroid.png

Figure SE1 further displays results for the rare-concept generation task introduced in (Samuel et al., ; ). The objective is to enable the diffusion model to generate concepts rarely seen in its training set by using a few images of that concept. Both methods in (Samuel et al., ; ) utilize diffusion inversion for this purpose. In the main paper, we presented experiments showcasing the impact of our new inversion process on the outcomes of (Samuel et al., ; ).

Figure: Figure E2: Qualitative comparison for image editing: Illustrates how alternative approaches may struggle to preserve the original structure or overlook crucial components specified in the target prompt, while GNRI succeeds in editing the image properly.
Refer to caption: extracted/5948112/figures/iclr25/editing_fig_supp.png

Figure: Figure E3: Various editing with same input: Note the GNRI capability in both subtle and extensive changes as one would expect from the particular prompt change.
Refer to caption: extracted/5948112/figures/supp/supp_editing_same.png

We provide qualitative results based on NAO (Samuel et al., ) centroid computation when utilizing GNRI, contrasting them with those obtained through DDIM inversions. The illustrations demonstrate that GNRI is capable of identifying high-quality centroids with correct semantic content compared to centroids found by DDIM.

## Appendix F Additional Quantitative Results

In this section, we present a quantitative comparison between GNRI and baseline methods, utilizing latent diffusion SD2.1 (Rombach et al., ). Figure D1(a) illustrates the image inversion-reconstruction performance, while Figure D1(b) highlights the editing performance, evaluated using LPIPS and CLIP-score metrics. The results demonstrate that GNRI surpasses all other inversion methods in terms of reconstruction quality, which subsequently improves editing performance.

## Appendix G Additional Qualitative Results

In this section, we present additional qualitative comparisons involving GNRI and baseline methods, using SD2.1.

Figures E2 provide further comparisons for real-image editing. These figures illustrate how alternative approaches may struggle to preserve the original structure or overlook crucial components specified in the target prompt. For example, n the first row of Figure E2, GNRI correctly transforms the pizza into bread, while other methods generate a bad looking bread. Row three shows an example of GNRI success (bananas to oranges), where other alternatives totally fail.

Figure E3 shows the outcomes of various edits on the same image, providing additional confirmation that inversion with GNRI yields modification of the pertinent parts in the image while maintaining the original structure.

## Peer Reviews

confidence: 3
contribution: 3
initial_rating: 6
presentation: 3
questions: The objective function $F(Z_t)$ in Eq. 9, which combines the residual term $r(z_t)$ and the regularization $G(z_t)$ raises questions about its role in achieving exact root-finding. The two terms in Eq. 9 seem to contradict each other, because a solution to $r(z_t) = 0$ will yield a $G(z_t) > 0$, which further leads to $F(z_t) > 0$ if $r(z_t) = 0$. This suggests that the method solves an approximation of the original inversion problem rather than the exact roots.

In Fig. 3(a), it is observed that GNRI achieves a lower residual than the non-guided NR inversion (NRI), which is unexpected since NRI has only one objective $r(z_t) = 0$. Why does GNRI, despite its additional regularization term, achieve a better residual than NRI? What is the residual level if you set $z_t = mu_t (G(z_t) = 0)$?
soundness: 3
strengths: Originality: This paper introduces a novel approach to the diffusion inversion problem by leveraging the Newton-Raphson (NR) root-finding method with two modifications: (1) dimensionality reduction via a scalar formulation of the Jacobian and (2) a guiding regularization term. By reframing inversion as a fixed-point iteration (FPI) problem with an $L_1$ norm relaxation, this work combines numerical optimization techniques with diffusion models, addressing limitations in accuracy and speed seen in prior methods. The addition of the regularization term prevents out-of-distribution solutions.

Quality: The methodological foundation is rigorous, with well-defined derivations and justifications for each modification to the NR method. The paper also provides extensive empirical validation across multiple tasks, including image reconstruction, editing, interpolation, and rare concept generation. The inversion time to 0.4 seconds on an A100 GPU is impressive.

Clarity: The paper is generally clearly written and accessible. 

Significance: GNRI addresses the problem of achieving efficient and accurate image inversion in text-to-image diffusion models, which is central to tasks like real-time image editing and rare concept generation. The experimental results across models (e.g., Stable Diffusion, SDXL-Turbo, Flux) demonstrate GNRI’s versatility, with the potential for broad impact in real-time generative model applications.
summary: This paper addresses diffusion inversion for fast and accurate inversion within text-to-image diffusion models. The authors propose Guided Newton-Raphson Inversion (GNRI), which leverages the Newton-Raphson (NR) numerical root-finding technique to achieve efficient and high-quality inversion. By constructing a modified DDIM inversion function as the target for NR and adding a guiding term, GNRI ensures convergence within the distribution of the latent space, overcoming common challenges in achieving accurate reconstructions. This approach supports tasks like real-time image editing, image interpolation, and rare concept generation, achieving significant speedups. The authors show that GNRI could invert images within 0.4 seconds on an A100 GPU while maintaining high fidelity, compared with popular diffusion models.
weaknesses: The concept of utilizing FPI methods in diffusion inversion is not completely new (like AIDI, ReNoise).

Figure 3 compares the performance of the method with an FPI baseline, which is not specified as the author mentioned two baseline FPI methods (AIDI, ReNoise).

There is a notational inconsistency in referencing Eq. 5 instead of Eq. 4 when describing the fixed-point function $f(z)$, which could lead to confusion regarding the intended inversion function. This typo gives the impression that the method still relies on the DDIM-inversion approximation. Please clarify these notations.

---

confidence: 4
contribution: 3
initial_rating: 8
presentation: 4
questions: - Equation (10) requires one forward pass and one back-propagation through diffusion model (UNet). In fact, with the same number of fixed-point iterations, this requires additional cost for back-propagation compared to ReNoise. However, in Figure 4 and Table 1, the reported efficiency of the GNRI is almost same as the original DDIM and much better than ReNoise. Is there some points that the reviewer is missing?
- Efficiency analysis (Figure 4):  Could author provide PSNR vs NFE? It would be helpful to further understand the efficiency of inverse methods.
- Is this method also capable of multiple-object editing?
soundness: 3
strengths: - The motivation and proposed method is clearly stated.
- As the NR provides faster convergence than naive fixed-point iteration method, the proposed method achieves high-efficiency compared with SOTA inversion methods.
- Combined with SDXL-turbo and Flux, it enables real-time image editing.
- Provided experimental results well demonstrate the effectiveness of the proposed method.
summary: To improve the accuracy of DDIM inversion approximations, previous studies, such as ReNoise, have utilized fixed-point iteration. This paper introduces a specialized fixed-point iteration known as Newton-Raphson (NR), which enables faster convergence. Because a straightforward application of the NR method is infeasible for high-dimensional points, this paper reduces dimensionality by applying the L1​-norm to the fixed-point function, transforming the Jacobian into a vector. Additionally, to prevent convergence to an off-manifold fixed point, a proximal term is added to the fixed-point function, steering the solution $z_t$ toward the mean of the forward distribution $q_t(z_t|z_{t-1})$ This complete framework is termed Guided Newton-Raphson Inversion (GNRI).
weaknesses: - The reviewer does not find significant weaknesses
- Minor: Linebreak in Figure 2.

---

confidence: 4
contribution: 4
initial_rating: 6
presentation: 3
questions: I suggest the authors to include the experiments for the DDPM-Inv for both reconstruction and editing. This helps to understand the difference between using one trajectory vs sample multiple trajectory and record the noise(difference) and help us to determine which method is better. 
Also please include more visualization results for the editing tasks. This helps us to understand the difference of different methods visually.
soundness: 4
strengths: 1. The paper introduces an optimized approach to inversion in diffusion models, significantly accelerating the process. With inversion times as low as 0.4 seconds on an A100 GPU, this method is fast enough to enable interactive applications, a notable advancement over existing techniques.
2. By incorporating an L1 norm for dimensionality reduction and a regularization term to ensure conformity with the diffusion model's distribution, the method maintains high reconstruction quality, avoiding common issues like out-of-distribution errors in the generated latents.
summary: This paper introduces an efficient method for accelerating image inversion in diffusion models by refining the Newton-Raphson (NR) approach. While a direct application of NR is computationally intense due to the need for Jacobian computation, the authors optimize it with dimensionality reduction using the L1 norm and incorporate a regularization term to ensure the noise latent conforms to the original distribution of the diffusion model. Experiments on models like SDXL-Turbo and Flux showcase both the speed and quality of inversion, and additional tests demonstrate the method’s capabilities in image editing, interpolation, and generating rare objects.
weaknesses: 1. Missing comparison with DDPM Inversion (An Edit Friendly DDPM Noise Space: Inversion and Manipulations) for the inversion performance in both quality and speed. Also, please include the DDPM-Inv result on PIE-Bench for image editing. 
2. More visualization comparison of the proposed method against other methods for editing tasks. Includes more editing results from the PIE-Benchmark.

---

confidence: 4
contribution: 3
initial_rating: 5
presentation: 2
questions: 1. In L238, should it reference Eq. 4 instead of Eq. 5?
2. Could you provide more results for the rare concept generation?
soundness: 3
strengths: 1. **Innovative Approach**: The use of the Newton-Raphson method helps overcome common problems found in other iterative inversion techniques (AIDI, ReNoise, ExactDPM), making the process more efficient.
   
2. **Comprehensive Experiments**: The paper provides experiments on three practical tasks, showing that GNRI well compared to existing methods, especially in terms of speed.
summary: This paper introduces a new inversion method for image editing in T2I DMs. It uses the Newton-Raphson (NR) method to improve the speed and accuracy of the inversion process. The authors present two main contributions: applying a norm to the NR method (Sec 4.1) and using an anchoring technique (Sec 4.2). Experimental results show that they can edit real images lightning-fast.
weaknesses: 1. **Questionable Contribution**: Although the idea of employing NR seems innovative, the contribution in Sec 4.1 about applying the norm seems standard and not very innovative. Optimizing norms is a common practice in optimization. For example, on page 9 of ReNoise, the convergence analysis is performed using the l2-norm, and ExactDPM also applies gradient descent to the l2-norm. Actually, anyone who tries to apply Newton method would do the same thing naturally, because calculating Jacobian is impossible while calculating the gradient of the *norm* is very easy in PyTorch. Therefore, this point does not stand out.

2. **Similarity to Other Methods**: The guiding regularizer introduced in Sec 4.2 appears similar to Pan et al.'s Anderson acceleration, which also aims to prevent large changes from previous estimates. This similarity raises questions about the originality of the approach. On the contrary, the significant drop in performance when $\lambda=0$ in Figure C2 can be seen as an indication of the instability of NRI.

3. **Memory Usage**: The Newton-Raphson method needs gradients, which can lead to high memory use. (It must be using torch.grad, I guess.) Other methods, like AIDI and the forward step method in ExactDPM, do not rely on gradients and may be more efficient, in terms of memory usage. ExactDPM actually used a smaller GPU than A100.

4. **Some writings are confusing**: The description of $\mathcal{F}$ is confusing because it is defined in two different ways: as a function from $\mathbb{R}^D \to \mathbb{R}^D$ in L203 and again as a function from $\mathbb{R}^D \to \mathbb{R}^+$ in Eq (9). This inconsistency can mislead readers. Also, in line 577, the citation for Hong et al. is duplicated. In line 637, it should be noted that EDICT was presented at CVPR *2023*, not 2022.

5. **Lack of Results for Rare Concept Generation**: The paper does not provide strong quantitative results for generating rare concepts. For example, Table 3 does not show significant improvements over existing methods.

Meta-review: This paper proposed an image inverse and editing method based on the Newton-Raphson (NR) algorithm to find root of the implicit equation of inversion for diffusion model.  It designed a fixed-point iteration approach enabling faster convergence, guided towards the in-distribution roots. The proposed method was applied to image editing using Stable Diffusion, SDXL-Turbo and Flux with different deterministic schedulers. The approach enables fast image inversion (0.4 sec, using one A100 GPU), and was applied to image reconstruction, editing, interpolation and rare concept generation, demonstrating the efficiency and effectiveness. The reviewers mostly feel positive on the novelty and effectiveness, while the Reviewer TzFZ raised major concerns on the contribution in norm-based optimization and difference to Pan et al.'s Anderson acceleration. In the response, the authors clarified the novelty of this approach and discussed difference to Anderson acceleration with experimental results and the reviewer confirmed the novelty of this paper.  Consider the overall contribution and effectiveness recognized by the reviewers, this paper can be accepted, with the conditions that the novelty and relation to related works discussed in the post-rebuttal phases should be included into the final version.

Decision: Accept (Poster)