Learning an Image Editing Model without Image Editing Pairs

## Abstract

Abstract Recent image editing models have achieved impressive results while following natural language editing instructions, but they rely on supervised fine-tuning with large datasets of input-target pairs. This is a critical bottleneck, as such naturally occurring pairs are hard to curate at scale.
Current workarounds use synthetic training pairs that leverage the zero-shot capabilities of existing models. However, this can propagate and magnify the artifacts of the pretrained model into the final trained model. In this work, we present a new training paradigm that eliminates the need for paired data entirely. Our approach directly optimizes a few-step diffusion model by unrolling it during training and leveraging feedback from vision-language models (VLMs). For each input and editing instruction, the VLM evaluates if an edit follows the instruction and preserves unchanged content, providing direct gradients for end-to-end optimization. To ensure visual fidelity, we incorporate distribution matching loss (DMD), which constrains generated images to remain within the image manifold learned by pretrained models. We evaluate our method on standard benchmarks and include an extensive ablation study. Without any paired data, our method performs on par with various image editing diffusion models trained on extensive supervised paired data, under the few-step setting. Given the same VLM as the reward model, we also outperform RL-based techniques like Flow-GRPO.

## 1 introduction

Large-scale text-to-image models have achieved remarkable success, generating images of high fidelity that closely align with textual descriptions Ramesh et al. (2022); Peebles & Xie (2023); Kang et al. (2023). Despite these advances, text-only conditioning offers limited user control and falls short for many downstream applications (Meng et al., 2022; Gal et al., 2023). In practice, users often wish to start with an existing image to perform tasks like adjusting local attributes, changing the style, or placing an object in a new context. These *image editing* operations require precise, image-guided control that text-only prompts cannot provide.

While collecting large-scale text–image pairs is relatively straightforward (Schuhmann et al., 2021), constructing supervised datasets for editing tasks is far more challenging. As one requires a pair of images, the input and its edited counterpart, along with the text instruction, and such data is rarely available online. Early methods addressed this by synthetically generating editing pairs (Brooks et al., 2023) from a pretrained model, using zero-shot editing techniques (Hertz et al., 2023). However, synthetic datasets can quickly become outdated with new and improved base models, and they risk amplifying and propagating artifacts of the synthetic editing process. More recent approaches extract frames from videos and annotate their differences (Chen et al., 2025; Song et al., 2023b; Krojer et al., 2024). Although promising, the applicability of this strategy is constrained by the diversity of transformations present in natural video sequences, where obtaining pixel-aligned before–and–after edited pairs is nearly impossible. A final alternative is to manually create training pairs (Winter et al., 2024; Magar et al., 2025), but this can be quite laborious and does not scale as easily.

In this work, we explore the possibility of training an image editing model without any training pairs. Our key idea is to leverage supervision from Vision Language Models (VLMs) (Liu et al., 2023a), relying on their general image-understanding capabilities to check whether the generated images satisfy the editing instructions. Prior works have studied the use of specialized models or general-purpose VLMs in improving generative models along dimensions such as text-alignment and aesthetic quality, primarily using reinforcement learning (Black et al., 2024; Liu et al., 2025a). In contrast, our method is the first to explore using gradient feedback from VLMs for general instruction-following, and we distill this feedback into a lightweight generative model that can generalize to arbitrary images and edit instructions. Our final method combines the VLM-feedback with a distribution matching loss to ensure that generated outputs remain in the realistic image domain while following the edit instructions. In summary, our contributions are threefold:

- 1.
We propose NP-Edit (No-Pair Edit), a framework for training image editing models using gradient feedback from a Vision–Language Model (VLM), requiring *no paired supervision*.
- 2.
For efficient training and effective VLM feedback, our formulation combines it with distribution matching loss to learn a *few-step* image editing model. The final model remains competitive with existing baselines trained on supervised data.
- 3.
We conduct a comprehensive empirical study analyzing the impact of (i) different VLM backbones, (ii) dataset scale and diversity, and (iii) VLM loss formulation. Our findings show that performance improves directly with more powerful VLMs and larger datasets, demonstrating its strong potential and scalability.

## 2 Related Works

Diffusion-based image editing. Development of large-scale text-to-image models has enabled a wide range of downstream applications, including local image editing (Hertz et al., 2023; Meng et al., 2022), stylization (Sohn et al., 2023; Hertz et al., 2024; Jones et al., 2024), personalization and customization (Gal et al., 2023; Ruiz et al., 2023). These can broadly be viewed as different forms of image-editing capabilities. Early approaches often relied on zero-shot inference-time methods (Hertz et al., 2023; Parmar et al., 2023; Cao et al., 2023; Avrahami et al., 2023; Kim et al., 2023) or flexible but slow optimization-based techniques (Gal et al., 2023; Ruiz et al., 2023; Kumari et al., 2023).

To improve efficiency and robustness, subsequent works introduced training-based approaches (Brooks et al., 2023; Xiao et al., 2025; Chen et al., 2023; Fu et al., 2024; Sun et al., 2024). However, obtaining large datasets of image pairs remains challenging: synthetic curation (Brooks et al., 2023; Zhang et al., 2023; Zhao et al., 2024; Hui et al., 2024; Yang et al., 2024b; Tan et al., 2025; Cai et al., 2025; Kumari et al., 2025) risks becoming outdated as generative models improve, while human annotation (Winter et al., 2024; Magar et al., 2025; Ge et al., 2024; Sushko et al., 2025) is costly and labor-intensive. Recent efforts have explored constructing paired data from videos (Chen et al., 2025; Song et al., 2023b) or simulation environments (Yu et al., 2025), although these remain limited in either annotation diversity or visual realism. We also target similar image-editing capabilities but remove the need for paired data (Zhu et al., 2017), by using differentiable feedback from vision–language models instead of ground-truth edits.

Post-training for image generation.
Post-training methods typically align image generators with human preferences using either Direct Preference Optimization (DPO) (Wallace et al., 2024; Yang et al., 2024a) or Reinforcement Learning (RL) (Black et al., 2024). While early RL-based works use feedback from a simple scalar reward model (Kirstain et al., 2023; Xu et al., 2023), the paradigm has recently been enhanced by employing sophisticated Vision-Language Models (VLMs) as “judges” to provide more generic and accurate reward signals (Ku et al., 2024).

Although post-training has been successfully applied to text-to-image generation, its use for image editing models has been less explored. Concurrently to our work, EARL (Ahmadi et al., 2025) begins to address this by using a VLM-as-a-judge framework to post-train an image-editing model with RL. However, RL-based approaches often depend heavily on good initialization, typically requiring a Supervised Fine-Tuning (SFT) phase with paired editing data.
In contrast, our method leverages differentiable feedback from the VLM model, thereby obviating the need for an initial SFT stage and enables the learning of image editing models without the use of synthetically generated data.

Related to our work, Luo et al. (2025) recently introduced a method that incorporates gradient feedback from VLMs to satisfy various criteria, including the horizon line, style, and layout, in generated images. However, their framework operates in a per-example optimization setting, requiring costly LoRA fine-tuning (Hu et al., 2022) for each criterion and prompt pair, and also does not consider image editing tasks.

Few-step diffusion models.
Standard diffusion (or flow-matching) models require many sampling steps to generate high-quality images. Many prior works reduce the number of denoising steps for faster sampling by predicting larger denoising steps, including consistency models (Kim et al., 2024; Geng et al., 2024; Song et al., 2023a; Yang et al., 2024c; Song & Dhariwal, 2024; Lu & Song, 2025; Heek et al., 2024), shortcut models (Frans et al., 2024), meanflow (Geng et al., 2025), and inductive moment matching (Zhou et al., 2025). Another line distills a pre-trained multi-step teacher into a few-step student by matching ODE trajectories (Song et al., 2023a; Salimans & Ho, 2022; Geng et al., 2023), using adversarial loss (Sauer et al., 2024b; Kang et al., 2024; Yin et al., 2024a; Sauer et al., 2024a; Xu et al., 2024), or applying score distillation (Luo et al., 2023; Yin et al., 2024b; a; Zhou et al., 2024). In our framework, we adopt DMD (Yin et al., 2024b) as a distribution matching objective. This ensures that our few-step editing model’s output remains in the real-image manifold defined by the pre-trained text-to-image teacher, while using VLM-feedback to ensure the model follows the editing instructions.

## 3 Background

### 3.1 Diffusion Models

Diffusion or flow-based models are a class of generating models that learn the data distribution by denoising samples corrupted by different levels of Gaussian Noise (Ho et al., 2020; Song et al., 2021; Lipman et al., 2023). Given a real sample ${\mathbf{x}}$, a forward diffusion process creates noisy samples ${\mathbf{x}}^{t}=\alpha^{t}{\mathbf{x}}+\sigma^{t}\epsilon$ over time $t\in(0,1]$, where $\epsilon\sim\mathcal{N}(\bm{0},\bm{I})$, and $\alpha^{t}$, $\sigma^{t}$ define a noise schedule such that ${\mathbf{x}}^{T}\sim\mathcal{N}(\bm{0},\bm{I})$ and ${\mathbf{x}}^{0}={\mathbf{x}}$. The denoising model is parameterized to reverse the forward diffusion process by either predicting the noise $\epsilon$ (Ho et al., 2020) added to the sample or velocity ${\mathbf{v}}$ towards the clean sample (Liu et al., 2023b; Salimans & Ho, 2022). In our work, we follow the flow-based formulation, with the forward denoising process being a linear interpolation, i.e., $\alpha^{t}=1-t$ and $\sigma^{t}=t$. The training objective for a flow-based model, with parameters $\theta$, can be simplified to the following:

$$ $\displaystyle\mathbb{E}_{{\mathbf{x}}^{t},t,\mathbf{c},\epsilon\sim\mathcal{N}(\bm{0},\bm{I})}w_{t}||\mathbf{v}-\mathbf{v}_{\theta}({\mathbf{x}}^{t},t,\mathbf{c})||,$ (1) $$

where ${\mathbf{v}}=\epsilon-{\mathbf{x}}$ and $w_{t}$ is a time dependent weighting factor. The denoising network can be conditioned on other inputs ${\mathbf{c}}$, such as a text prompt, a reference image, or both, as in our case.

### 3.2 Vision Language Models (VLMs)

Vision Language Models (VLMs) trained from multimodal image-text data have shown exemplary visual understanding and reasoning capabilities and can serve as a general-purpose visual model.
A common strategy for training such large-scale VLMs is via visual instruction tuning (Liu et al., 2023a), which aligns a pre-trained Vision Encoder output with the input word embedding space of a pretrained Large Language Model (LLM). More specifically, the image ${\mathbf{x}}$ is encoded into a set of tokens using the vision encoder, $\mathbf{X}_{v}=g({\mathbf{x}})$. The input question regarding the image and its ground truth answer are tokenized in the LLM input embedding space as $\mathbf{X}_{q}$ and $\mathbf{X}_{a}$, respectively. A projector module, $f_{\phi}$, projects the vision-encoded tokens into the LLM word embedding space and is trained via standard autoregressive loss to maximize the probability of predicting the correct answer:

$$ $\displaystyle p(\mathbf{X}_{a}|\mathbf{X}_{v},\mathbf{X}_{q})=\prod_{i=1}^{L}p\Big(a_{i}|f_{\phi}(\mathbf{X}_{v}),\mathbf{X}_{q},\mathbf{X}_{a<i}\Big),$ (2) $$

where $\mathbf{X}_{a}=[a_{1}\cdots a_{L}]$ is of token length $L$, and $\mathbf{X}_{a<i}$ denotes all the tokens before the current prediction token index. The final loss simplifies to a cross-entropy over the total vocabulary length. In our experiments, we use LLaVa-OneVision-7B (Li et al., 2024) as the VLM that uses SigLIP (Zhai et al., 2023) vision encoder and Qwen-2 LLM (Qwen-Team, 2024), and is among the state-of-the-art VLMs of this scale.

## 4 Method

Given a pretrained text-to-image diffusion model $G_{\text{init}}$ and a dataset $\mathcal{X}=\{({\mathbf{y}}_{i},{\mathbf{c}}_{i},{\mathbf{c}}^{{\mathbf{y}}}_{i},{\mathbf{c}}^{{\mathbf{x}}}_{i})\}_{i=1}^{N}$ of reference image ${\mathbf{y}}$, corresponding edit instruction ${\mathbf{c}}$, and captions ${\mathbf{c}}^{{\mathbf{y}}}$ and ${\mathbf{c}}^{{\mathbf{x}}}$ that describe the reference and edited image respectively, we fine-tune $G_{\text{init}}$ into a *few-step* image editing model $G_{\theta}$ without requiring ground truth edited image ${\mathbf{x}}$ according to the edit instruction. Our approach, No-Pair (NP)-Edit, introduces a VLM-based loss to evaluate edit success and combines it with a distribution matching loss to ensure outputs remain within the natural image domain. Below, we first detail the construction of the dataset, then our training objective, and finally other implementation details.

### 4.1 Edit Instruction Dataset

Each dataset sample consists of a real image ${\mathbf{y}}$ as reference and an associated edit instruction, ${\mathbf{c}}$. Following prior works (Liu et al., 2025b; Ye et al., 2025), we focus on several categories of local editing operations, such as *Add*, *Replace*, *Remove*, *Adjust shape*, *Action*, *Stylization*, *Text editing*, *Color*, *Material*, and *Background* change, as well as more free-form editing tasks such as *Customization* or *Personalization* (Gal et al., 2023; Ruiz et al., 2023; Kumari et al., 2023). Candidate instructions for each type are generated using Qwen2.5-32B VLM (Qwen-Team, 2025). Given an image-instruction pair, we further query the VLM to assess its validity and to suggest the caption, ${\mathbf{c}}^{{\mathbf{x}}}$, for the edited image. For the customization task, we restrict reference images to those showing a prominent central object, either filtered from a real image corpus or generated via the pretrained model (Tan et al., 2025; Kumari et al., 2025), and prompt the VLM to generate a caption that places the object in a novel background or context. In total, for local and free-form editing instructions, our dataset consists of $\sim 3$M and $\sim 600$K reference images, respectively. The input prompt to the Qwen2.5-32B VLM for each setup is shown in Appendix [D](https://arxiv.org/html/2510.14978v1#A4).

### 4.2 Training Objective

Training a diffusion or flow-based model (Ho et al., 2020; Liu et al., 2023b) for image editing without pairs presents a unique challenge. Standard diffusion training takes as input noised versions of a ground-truth image. In our setting, no such ground-truth edited image exists; thus, we cannot construct these intermediate noisy inputs. On the other hand, directly mapping noise to the edited image in a single step is naturally challenging and yields poor fidelity (see Appendix [B](https://arxiv.org/html/2510.14978v1#A2)). To address this, during training, we propose to unroll the backward diffusion trajectory starting from noise using a two-step sampling procedure (Song et al., 2023a). Specifically, given the reference image–instruction pair $({\mathbf{y}},{\mathbf{c}})$, the editing model $G_{\theta}$ first predicts a provisional clean image $\hat{{\mathbf{x}}}_{\theta}^{0}$ from noise $\epsilon$. Then, a second step refines this estimate by feeding an interpolated noisy input back into the model:

$$ $\displaystyle\hat{{\mathbf{x}}}_{\theta}^{0}$ $\displaystyle=\epsilon-\hat{{\mathbf{v}}}_{\theta},$ $\displaystyle\text{where }\hat{{\mathbf{v}}}_{\theta}\equiv G_{\theta}(\epsilon,t=1,{\mathbf{c}},{\mathbf{y}}),\quad\epsilon\sim\mathcal{N}(\bm{0},\bm{I}),$ (3) $\displaystyle{\mathbf{x}}_{\theta}^{0}$ $\displaystyle=\hat{{\mathbf{x}}}_{\theta}^{t}-t{\mathbf{v}}_{\theta},$ $\displaystyle\text{where }{\mathbf{v}}_{\theta}\equiv G_{\theta}(\hat{{\mathbf{x}}}_{\theta}^{t},t,{\mathbf{c}},{\mathbf{y}}),\quad\hat{{\mathbf{x}}}_{\theta}^{t}=(1-t)\hat{{\mathbf{x}}}_{\theta}^{0}+t\epsilon;\epsilon\sim\mathcal{N}(\bm{0},\bm{I}),t\sim(0,1).$ $$

With the second step, the model is now trained on noisy intermediate states at timesteps determined by $t$, while being more efficient than a full backward unroll. In our method, we focus on *few-step* generation—specifically four steps—and restrict $t\in[0.25,0.5,0.75]$ in the second step. Few-step generator provides a better estimate of the denoised image, ${\mathbf{x}}^{0}_{\theta}$, at intermediate steps, which in turn enables effective VLM-based feedback. VLMs tend to give unreliable judgments when inputs are noisy or blurry (see Appendix [B](https://arxiv.org/html/2510.14978v1#A2)). This also enables faster inference and lowers training costs.

Figure: Figure 1: Method. We fine-tune a pretrained text-to-image model into a few-step image-editing model using differentiable VLM-feedback regarding edit success. In addition, we use distribution matching loss (DMD (Yin et al., 2024a)) to ensure output images remain in the natural image manifold.
Refer to caption: x1.png

VLM-based editing loss.
To evaluate whether an edit is successfully applied in ${\mathbf{x}}^{0}_{\theta}$, we define a set of template questions with corresponding ground truth answers, $\mathcal{D}_{\text{QA}}=\{(\mathbf{X}_{q_{j}},\mathbf{X}_{a_{j}})\}_{j}$, tailored to each edit category.
The VLM is instructed to answer with a binary yes and no answer, i.e., $\mathbf{X}_{a_{j}}\in\{\text{\emph{Yes}},\text{\emph{No}}\}$, and $\mathbf{X}_{\bar{a}_{j}}$ denotes the opposite response.
The loss is then a binary cross-entropy over the predicted logit difference between the tokens corresponding to the correct and opposite responses, respectively.

$$ $\displaystyle\mathcal{L}_{\text{VLM}}$ $\displaystyle=-\sum_{j}\log p(a_{j}),\text{ where }p(a_{j})=\sigma(\ell^{(j)}_{a_{j}}-\ell^{(j)}_{\bar{a}_{j}})$ (4) $$

where $\ell^{(j)}_{a_{j}}$ is the logit corresponding to the token $\mathbf{X}_{a_{j}}$, $\sigma$ is the sigmoid function, and $p(a_{j})$ is the probability of correct answer, while restricting normalization to only the *Yes* and *No* tokens, which we observe to be more effective during training (Zhang et al., 2024). Computing this loss is relatively fast, as it only requires a single forward call to the VLM per question, as opposed to autoregressive token prediction.

For each edit instruction, we use two complementary questions to compute the editing loss: (1) *Edit-verification* question to assess whether the intended edit is applied, and (2) *Identity-preservation* question to ensure the image is not over-edited and is consistent with the reference image. Specifically, for the local image-editing instructions, we verify edit success with the following question “The objective is to evaluate if the editing instruction has been executed in the second image. Editing instruction: $\{$edit instruction$\}$. Answer with a Yes or No.” except *removal* edit-type, where we directly evaluate if the intended object is removed by asking “Answer with a Yes or No if the image has $\{$object name$\}$”. For the identity-preservation question, we ask the following: “Answer with a Yes or No if the second image is exactly the same as the first image. IGNORE the changes in the second image because of the edit: $\{$edit instruction$\}$”. We provide the list of all questions along with their system and user prompts for all editing types, including free-form editing in Appendix [E](https://arxiv.org/html/2510.14978v1#A5).

Distribution matching with text-to-image teacher model. While VLM feedback evaluates the efficacy of instruction following, it does not enforce the generated outputs to remain in the real image domain. To ensure this and keep the output distribution of the generator aligned with the pre-trained model, we apply Distribution Matching Distillation (DMD) (Yin et al., 2024b; a) between the fine-tuned model, $G_{\theta}$, and the pre-trained text-to-image (teacher) model, $G_{\text{init}}$. DMD minimizes the Kullback–Leibler (KL) divergence between the real image distribution, as estimated by the teacher model, and the output distribution of the fine-tuned model. The gradient of this KL-divergence loss with respect to the generator parameters can be simplified to:

$$ $\displaystyle\nabla_{\theta}D_{KL}$ $\displaystyle=\mathbb{E}_{\begin{subarray}{c}\epsilon\sim\mathcal{N}(\bm{0},\bm{I}),t\in(0,1),{\mathbf{x}}^{0}_{\theta}\end{subarray}}\Big[-\big({\mathbf{v}}_{\text{real}}({\mathbf{x}}^{t}_{\theta},t,{\mathbf{c}}^{{\mathbf{x}}})-{\mathbf{v}}_{\text{gen}}({\mathbf{x}}^{t}_{\theta},t,{\mathbf{c}}^{{\mathbf{x}}})\big)\hskip 1.42262pt\frac{dG}{d\theta}\Big],$ (5) $$

where ${\mathbf{c}}^{{\mathbf{x}}}$ is the text caption describing the noisy edited image ${\mathbf{x}}^{t}_{\theta}$ and ${\mathbf{v}}_{\text{real}}$, ${\mathbf{v}}_{\text{gen}}$ represents the predicted velocity from the teacher and a trainable auxiliary model, $A_{\phi}$ respectively. The auxiliary model is trained along with $G_{\theta}$ to learn the current output distribution of $G_{\theta}$ using a flow-based denoising objective. This loss ensures that the edited images not only satisfy the instruction but also remain faithful to the text conditioned distribution of real images modeled by the pretrained teacher.

### 4.3 Training details

The pretrained model $G_{\theta}$ is originally designed to generate an image, ${\mathbf{x}}$, conditioned only on text ${\mathbf{c}}$. To adapt it to our editing task, we extend its conditioning to include the reference image ${\mathbf{y}}$. Following recent works (Xiao et al., 2025; Tan et al., 2025), we concatenate the VAE encoding of the reference image to the noisy target image encoding along the token sequence dimension, similar to text embedding, thereby enabling the model to attend to both text and visual conditions.

To stabilize training, in the initial few iterations, we train the model with the objective of simply reconstructing the concatenated reference image. This encourages the network to propagate content from the reference input, aligning it toward producing realistic images under joint text–image conditioning. After this, we introduce our main training objective as explained in the previous section.

The final loss for the generator is a weighted combination of the VLM-based editing loss and DMD loss. The auxiliary network, $A_{\phi}$, is updated $N_{\text{aux}}$ times for every generator, $G_{\theta}$, update (Yin et al., 2024a). Our pre-trained generative model is a 2B parameter internal DiT-based (Peebles & Xie, 2023) latent space diffusion model. The overall training pipeline is illustrated in Figure [1](https://arxiv.org/html/2510.14978v1#S4.F1) and is detailed more formally in Algorithm [1](https://arxiv.org/html/2510.14978v1#alg1) below. Other training hyperparameters are detailed in Appendix [E](https://arxiv.org/html/2510.14978v1#A5).

Figure: Algorithm 1 NP-Edit: our training method

Figure: Figure 2: Qualitative comparison on GEdit-Bench under the few-step sampling setting. For an upper-bound comparison, in the $1^{\text{st}}$ column we show results of the best multi-step sampling method (as measured by the quantitative metrics in Table [1](https://arxiv.org/html/2510.14978v1#S5.T1)). Our method performs on par or better than baseline methods across different edit types in the few-step setting. We show more samples in the Appendix Figure [10](https://arxiv.org/html/2510.14978v1#A7.F10)
Refer to caption: x2.png

## 5 Experiments

In this section, we show the results of our method on local image editing as well as more free-form image editing tasks like customization, and compare them with the state-of-the-art baseline methods.

### 5.1 Local image-editing

Benchmark. For evaluation, following prior works, we use the English subset of GEdit-Benchmark Liu et al. (2025b), which captures real-world user interactions across different edit types. We also show results on the ImgEdit (Ye et al., 2025) benchmark in Appendix [A](https://arxiv.org/html/2510.14978v1#A1).

Evaluation metric. For quantitative evaluation, we follow prior works and use GPT4o-based VIEScore (Ku et al., 2024) metric. It scores each edit on: (1) Semantic Consistency (SC) score, evaluating whether the edit instruction was followed, and (2) Perceptual Quality (PQ) score, assessing realism and absence of artifacts. Following VIEScore, for the *Overall score*, we take the geometric mean between SC and PQ for each image, and average across images in the evaluation benchmark.

Baselines. We compare our method with leading baselines, including FLUX.1-Kontext (Labs et al., 2025), Step1X-Edit (Liu et al., 2025b), BAGEL (Deng et al., 2025), OmniGen (Xiao et al., 2025), and Qwen-Image-Edit (Wu et al., 2025). Since no prior work explicitly targets few-step editing, we simply evaluate the above baselines with few-step sampling as well as their original multi-step setting for an upper-bound comparison. We also include Turbo-Edit (Deutch et al., 2024), a state-of-the-art zero-shot few-step method that requires no paired supervision (as zero-shot) and is thus closest to our setup. We use the open-source implementation of all baselines, with further details in the Appendix [F](https://arxiv.org/html/2510.14978v1#A6).

**Table 1: Quantitative evaluation on GEdit-Bench. Our method performs on par or better than baselines under the few-step setting. For multi-step sampling, it still outperforms OmiGen and remains competitive with many of the larger-scale models like BAGEL and FLUX.1 Kontext. All numbers reported in $\times 10$**
| Method | $\#$Param | $\#$Step | SC Score$\uparrow$ | PQ Score $\uparrow$ | Overall $\uparrow$ |
| --- | --- | --- | --- | --- | --- |
| Omni-Gen (Xiao et al., 2025) | 4B | 50 | 5.52 | 6.14 | 4.97 |
| BAGEL (Deng et al., 2025) | 7B | 50 | 7.02 | 6.26 | 6.14 |
| FLUX.1-Kontext (Labs et al., 2025) | 12B | 28 | 6.29 | 6.65 | 5.65 |
| Step1X-Edit (Liu et al., 2025b) v1.1 | 12B | 28 | 7.30 | 7.37 | 6.79 |
| Qwen-Image-Edit (Wu et al., 2025) | 20B | 50 | 7.94 | 7.50 | 7.36 |
| FLUX.1-Kontext (Labs et al., 2025) | 12B | 4 | 5.80 | 5.74 | 5.04 |
| Step1X-Edit (Liu et al., 2025b) v1.1 | 12B | 4 | 6.61 | 6.43 | 6.01 |
| Qwen-Image-Edit (Wu et al., 2025) | 20B | 4 | 6.82 | 6.21 | 6.06 |
| Turbo-Edit (Deutch et al., 2024) | 1B | 4 | 3.84 | 6.67 | 3.84 |
| NP-Edit (Ours) | 2B | 4 | 6.16 | 7.69 | 6.10 |

**Table 2: Free-form editing task, *Customization*, evaluation on Dreambooth. We perform better than OminiControl, DSD, and SynCD, which are trained for this task on synthetic datasets. When compared to FLUX.1-Kontext and Qwen-Image-Edit, we still perform comparably in the few-step setting. All numbers are reported in $\times 10$**
| Method | $\#$Param | $\#$Step | SC Score$\uparrow$ | PQ Score $\uparrow$ | Overall $\uparrow$ |
| --- | --- | --- | --- | --- | --- |
| DSD (Cai et al., 2025) | 12B | 28 | 6.71 | 7.41 | 6.78 |
| SynCD (Kumari et al., 2025) | 12B | 30 | 7.66 | 7.83 | 7.54 |
| FLUX.1-Kontext (Labs et al., 2025) | 12B | 28 | 8.19 | 7.45 | 7.61 |
| Qwen-Image-Edit (Wu et al., 2025) | 20B | 50 | 8.53 | 7.79 | 8.02 |
| OminiControl (Tan et al., 2025) | 12B | 8 | 6.33 | 7.82 | 6.22 |
| DSD (Cai et al., 2025) | 12B | 8 | 6.37 | 6.78 | 6.29 |
| SynCD (Kumari et al., 2025) | 12B | 8 | 7.71 | 6.84 | 7.07 |
| FLUX.1-Kontext (Labs et al., 2025) | 12B | 8 | 7.99 | 7.18 | 7.39 |
| Qwen-Image-Edit (Wu et al., 2025) | 20B | 8 | 8.08 | 7.44 | 7.62 |
| NP-Edit (Ours) | 2B | 8 | 7.68 | 7.56 | 7.33 |
| NP-Edit (Ours) | 2B | 4 | 7.60 | 7.28 | 7.10 |

Results
Table [1](https://arxiv.org/html/2510.14978v1#S5.T1) shows the quantitative result. In the few-step setting, our method achieves the best Overall and Perceptual Quality (PQ) score compared to baseline methods. When compared to their original multi-step sampling, our few-step model still outperforms OmniGen and remains competitive with BAGEL, FLUX.1-Kontext, despite being $\times 6$ smaller parameter-wise. While Step1X-Edit and Qwen-Image-Edit perform better, they are substantially larger models. Figure [2](https://arxiv.org/html/2510.14978v1#S4.F2) provides a qualitative comparison. As we can see, our method can successfully follow different editing instructions while being consistent with the input reference image. For instance, in the $6^{\text{th}}$ row (sheep color change), our approach produces a more natural edit compared to baselines. It also performs comparably to the multi-step variant for edits like lighting the candle in $4^{\text{rth}}$ row or making the person wave in $7^{\text{th}}$ row.

### 5.2 Free-form Editing: Customization

Benchmark. We use the widely adopted DreamBooth (Ruiz et al., 2023) dataset for evaluation. It consists of $30$ objects and $25$ prompts per object category. The goal is to generate the same object as shown in the reference image, but in a different context, as mentioned in the text prompt.

Baselines. We compare against state-of-the-art unified image-editing baselines such as FLUX.1-Kontext (Labs et al., 2025) and Qwen-Image-Edit Wu et al. (2025) as well as OminiControl (Tan et al., 2025), DSD (Cai et al., 2025), and SynCD (Kumari et al., 2025), which are feed-forward models trained specifically for this task on synthetic datasets.

Evaluation metric. Here as well, we use the VIEScore evaluation with a similar Semantic Consistency (SC) score to evaluate identity and text alignment and the Perceptual Quality (PQ) score to measure realism, and the geometric mean of the two for the Overall score.
We also report CLIPScore (Radford et al., 2021) and DINO (Oquab et al., 2023) similarity-based metrics in the Appendix.

Results. As shown in Table [2](https://arxiv.org/html/2510.14978v1#S5.T2), our method performs comparably to state-of-the-art methods. In the few-shot sampling setting, all the baseline methods fail to generate realistic samples at $4$ steps; therefore, we compare with them at $8$ sampling steps. Our method still results in higher fidelity samples as  Figure [3](https://arxiv.org/html/2510.14978v1#S5.F3) shows, while maintaining object identity with the reference image. Note that our method performs better than OminiControl, which is also a few-step (8) model for this task.

Figure: Figure 3: Qualitative comparison on Customization task. Our method can generate the object in new contexts while having better fidelity under few-step sampling. We show more samples in the Appendix Figure [12](https://arxiv.org/html/2510.14978v1#A7.F12).
Refer to caption: x3.png

### 5.3 Ablation

In this section, we perform several ablations to analyze the role of different components of our method, dataset scale, and stronger VLMs. All ablations are done on the local image-editing task.

Training objective. We ablate our training objective across four settings: (1) using only distribution matching loss, (2) using only the VLM-based editing loss, (3) removing the identity-preservation loss $\mathcal{D}_{QA}$, and (4) replacing the binary-cross entropy loss (Eqn. [4](https://arxiv.org/html/2510.14978v1#S4.E4)) with standard cross-entropy over the full vocabulary. Results are shown below in Table [3](https://arxiv.org/html/2510.14978v1#S5.T3). Training without the VLM-based loss and relying solely on distribution matching significantly degrades the model’s capabilities at following editing instructions. We observe that VLM-based loss is essential for maintaining consistency between input and edited images and for certain editing tasks like *Removal* (Figure [4](https://arxiv.org/html/2510.14978v1#S5.F4) and Appendix Figure [5](https://arxiv.org/html/2510.14978v1#A2.F5)). However, only training with VLM-based loss leads to unrealistic outputs (Appendix Figure [6](https://arxiv.org/html/2510.14978v1#A2.F6)), and the training eventually diverges, as evidenced by the low overall score in Table [3](https://arxiv.org/html/2510.14978v1#S5.T3), underscoring the need for DMD loss. In addition, using binary cross-entropy loss and having a question to check consistency between input and edited images improves the overall performance.

**Table 3: Training objective ablation. We compare on the GEdit-Bench using the VIEScore metric. Ablating different components of our method leads to a drop in performance, indicating its importance.**
| Method | SC Score$\uparrow$ | PQ Score $\uparrow$ | Overall $\uparrow$ |
| --- | --- | --- | --- |
| Ours | 6.16 | 7.69 | 6.10 |
| w/ only DMD | 4.93 | 7.51 | 4.93 |
| w/ only VLM | 2.03 | 3.48 | 1.93 |
| w/o VLM identity | 5.70 | 7.67 | 5.76 |
| w/ standard CE loss | 5.95 | 7.64 | 5.89 |

Dataset and VLM scale. To study the role of dataset scale, we vary the number of unique reference images in training. Our final dataset represents the maximum scale feasible under our computational resources. Table [5.3](https://arxiv.org/html/2510.14978v1#S5.SS3) shows the performance across different dataset sizes and VLM backbones. We observe consistent gains with larger datasets, suggesting that further scaling of data could yield additional improvements. Similarly, a larger parameter VLM-backbone leads to better performance, across different VLMs such as InternVL (Chen et al., 2024) and LLava-OneVision (Li et al., 2024), underscoring the promise that our method can improve as more powerful VLMs are developed.

Our method vs. Reinforcement Learning (RL). RL is a common post-training strategy for improving pre-trained models without paired supervision and can also leverage VLMs as the reward model, a similar setup to ours. Thus, we benchmark our method against Flow-GRPO (Liu et al., 2025a), a widely used RL method for text-to-image diffusion.
However, since RL relies on a reasonable initialization, we need to first train an image-editing model via Supervised Fine-Tuning (SFT) on a paired dataset (Lin et al., 2025).
We then fine-tune it with Flow-GRPO using the same Llava-OneVision reward model as in our approach. As shown in Table [5.3](https://arxiv.org/html/2510.14978v1#S5.SS3), SFT alone performs poorly, likely due to the limited quality of paired data. Our method surpasses both SFT and SFT+RL, despite requiring no paired supervision.
Fine-tuning the model with some paired data before applying our approach can slightly improve the pixel-level consistency between the input reference and output edited image (as shown in Figure [4](https://arxiv.org/html/2510.14978v1#S5.F4)), although the quantitative numbers are similar.

The Appendix provides additional results and more detailed discussion of the method’s limitations.

Figure: Table 4: Dataset and VLM scale and comparison with Reinforcement Learning on the GEdit-Bench. Increasing dataset scale and using stronger VLMs leads to increased performance. Our method also performs better than post-training an SFT model with RL (Liu et al., 2025a).
Refer to caption: x4.png

## 6 Discussion and Limitations

This paper introduces a new paradigm for enabling image editing capabilities given a pre-trained text-to-image diffusion model, without paired before-and-after edit supervision. Our approach combines differentiable feedback from VLMs to ensure editing success with a distribution matching objective to maintain visual realism. This method achieves competitive performance with recent state-of-the-art baselines trained on paired data while enabling efficient few-step generation.

Despite these promising results, our method has limitations. Without pixel-level supervision, edits may deviate from the input image in fine-grained details or fail to fully preserve subject identity. We show in Appendix [C](https://arxiv.org/html/2510.14978v1#A3) that adding a perceptual similarity loss (e.g., LPIPS (Zhang et al., 2018)) between input and edited images alleviates this to some extent, though often at the cost of editing quality. Another constraint for our method is the need to keep the VLM in GPU memory, introducing VRAM overhead. We expect ongoing advances in stronger and more efficient VLMs can help address this issue. Overall, our framework scales effectively with large unpaired datasets and highlights a path toward more flexible post-training of generative models for diverse downstream tasks.

Acknowledgment.
We thank Gaurav Parmar, Maxwell Jones, and Ruihan Gao for their feedback and helpful discussions. This work was partly done while Nupur Kumari was interning at Adobe Research. The project was partly supported by Adobe Inc., the Packard
Fellowship, the IITP grant funded by the Korean Government (MSIT) (No. RS-2024-00457882, National AI Research Lab Project), NSF IIS-2239076, and NSF ISS-2403303.

## References

- Adam et al. (2014)
Kingma DP Ba J Adam et al.
A method for stochastic optimization.
*arXiv preprint arXiv:1412.6980*, 1412(6), 2014.
- Ahmadi et al. (2025)
Saba Ahmadi, Rabiul Awal, Ankur Sikarwar, Amirhossein Kazemnejad, Ge Ya Luo, Juan A Rodriguez, Sai Rajeswar, Siva Reddy, Christopher Pal, Benno Krojer, et al.
The promise of rl for autoregressive image editing.
*arXiv preprint arXiv:2508.01119*, 2025.
- Avrahami et al. (2023)
Omri Avrahami, Ohad Fried, and Dani Lischinski.
Blended latent diffusion.
*ACM transactions on graphics (TOG)*, 42(4):1–11, 2023.
- Black et al. (2024)
Kevin Black, Michael Janner, Yilun Du, Ilya Kostrikov, and Sergey Levine.
Training diffusion models with reinforcement learning.
In *International Conference on Learning Representations (ICLR)*, 2024.
- Brooks et al. (2023)
Tim Brooks, Aleksander Holynski, and Alexei A Efros.
Instructpix2pix: Learning to follow image editing instructions.
In *IEEE Conference on Computer Vision and Pattern Recognition (CVPR)*, 2023.
- Cai et al. (2025)
Shengqu Cai, Eric Ryan Chan, Yunzhi Zhang, Leonidas Guibas, Jiajun Wu, and Gordon. Wetzstein.
Diffusion self-distillation for zero-shot customized image generation.
In *IEEE Conference on Computer Vision and Pattern Recognition (CVPR)*, 2025.
- Cao et al. (2023)
Mingdeng Cao, Xintao Wang, Zhongang Qi, Ying Shan, Xiaohu Qie, and Yinqiang Zheng.
Masactrl: Tuning-free mutual self-attention control for consistent image synthesis and editing.
In *IEEE International Conference on Computer Vision (ICCV)*, 2023.
- Cazenavette et al. (2024)
George Cazenavette, Avneesh Sud, Thomas Leung, and Ben Usman.
Fakeinversion: Learning to detect images from unseen text-to-image models by inverting stable diffusion.
In *IEEE Conference on Computer Vision and Pattern Recognition (CVPR)*, 2024.
- Chen et al. (2023)
Wenhu Chen, Hexiang Hu, Yandong Li, Nataniel Rui, Xuhui Jia, Ming-Wei Chang, and William W Cohen.
Subject-driven text-to-image generation via apprenticeship learning.
In *Conference on Neural Information Processing Systems (NeurIPS)*, 2023.
- Chen et al. (2025)
Xi Chen, Zhifei Zhang, He Zhang, Yuqian Zhou, Soo Ye Kim, Qing Liu, Yijun Li, Jianming Zhang, Nanxuan Zhao, Yilin Wang, et al.
Unireal: Universal image generation and editing via learning real-world dynamics.
In *IEEE Conference on Computer Vision and Pattern Recognition (CVPR)*, 2025.
- Chen et al. (2024)
Zhe Chen, Jiannan Wu, Wenhai Wang, Weijie Su, Guo Chen, Sen Xing, Muyan Zhong, Qinglong Zhang, Xizhou Zhu, Lewei Lu, et al.
Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks.
In *IEEE Conference on Computer Vision and Pattern Recognition (CVPR)*, 2024.
- Corvi et al. (2023)
Riccardo Corvi, Davide Cozzolino, Giada Zingarini, Giovanni Poggi, Koki Nagano, and Luisa Verdoliva.
On the detection of synthetic images generated by diffusion models.
In *IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)*, 2023.
- Deng et al. (2025)
Chaorui Deng, Deyao Zhu, Kunchang Li, Chenhui Gou, Feng Li, Zeyu Wang, Shu Zhong, Weihao Yu, Xiaonan Nie, Ziang Song, et al.
Emerging properties in unified multimodal pretraining.
*arXiv preprint arXiv:2505.14683*, 2025.
- Deutch et al. (2024)
Gilad Deutch, Rinon Gal, Daniel Garibi, Or Patashnik, and Daniel Cohen-Or.
Turboedit: Text-based image editing using few-step diffusion models.
In *SIGGRAPH Asia Conference Proceedings*, 2024.
- Fernandez et al. (2023)
Pierre Fernandez, Guillaume Couairon, Hervé Jégou, Matthijs Douze, and Teddy Furon.
The stable signature: Rooting watermarks in latent diffusion models.
In *IEEE International Conference on Computer Vision (ICCV)*, 2023.
- Frans et al. (2024)
Kevin Frans, Danijar Hafner, Sergey Levine, and Pieter Abbeel.
One step diffusion via shortcut models.
*arXiv preprint arXiv:2410.12557*, 2024.
- Fu et al. (2024)
Tsu-Jui Fu, Wenze Hu, Xianzhi Du, William Yang Wang, Yinfei Yang, and Zhe Gan.
Guiding instruction-based image editing via multimodal large language models.
In *International Conference on Learning Representations (ICLR)*, 2024.
- Gal et al. (2023)
Rinon Gal, Yuval Alaluf, Yuval Atzmon, Or Patashnik, Amit H Bermano, Gal Chechik, and Daniel Cohen-Or.
An image is worth one word: Personalizing text-to-image generation using textual inversion.
In *International Conference on Learning Representations (ICLR)*, 2023.
- Ge et al. (2024)
Yuying Ge, Sijie Zhao, Chen Li, Yixiao Ge, and Ying Shan.
Seed-data-edit technical report: A hybrid dataset for instructional image editing.
*arXiv preprint arXiv:2405.04007*, 2024.
- Geng et al. (2023)
Zhengyang Geng, Ashwini Pokle, and J Zico Kolter.
One-step diffusion distillation via deep equilibrium models.
In *Conference on Neural Information Processing Systems (NeurIPS)*, 2023.
- Geng et al. (2024)
Zhengyang Geng, Ashwini Pokle, William Luo, Justin Lin, and J Zico Kolter.
Consistency models made easy.
*arXiv preprint arXiv:2406.14548*, 2024.
- Geng et al. (2025)
Zhengyang Geng, Mingyang Deng, Xingjian Bai, J Zico Kolter, and Kaiming He.
Mean flows for one-step generative modeling.
*arXiv preprint arXiv:2505.13447*, 2025.
- Heek et al. (2024)
Jonathan Heek, Emiel Hoogeboom, and Tim Salimans.
Multistep consistency models.
*arXiv preprint arXiv:2403.06807*, 2024.
- Hertz et al. (2023)
Amir Hertz, Ron Mokady, Jay Tenenbaum, Kfir Aberman, Yael Pritch, and Daniel Cohen-Or.
Prompt-to-prompt image editing with cross attention control.
In *International Conference on Learning Representations (ICLR)*, 2023.
- Hertz et al. (2024)
Amir Hertz, Andrey Voynov, Shlomi Fruchter, and Daniel Cohen-Or.
Style aligned image generation via shared attention.
In *IEEE Conference on Computer Vision and Pattern Recognition (CVPR)*, 2024.
- Ho et al. (2020)
Jonathan Ho, Ajay Jain, and Pieter Abbeel.
Denoising diffusion probabilistic models.
In *Conference on Neural Information Processing Systems (NeurIPS)*, 2020.
- Hu et al. (2022)
Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen.
Lora: Low-rank adaptation of large language models.
In *International Conference on Learning Representations (ICLR)*, 2022.
- Hu et al. (2023)
Yushi Hu, Benlin Liu, Jungo Kasai, Yizhong Wang, Mari Ostendorf, Ranjay Krishna, and Noah A Smith.
Tifa: Accurate and interpretable text-to-image faithfulness evaluation with question answering.
In *IEEE Conference on Computer Vision and Pattern Recognition (CVPR)*, 2023.
- Hui et al. (2024)
Mude Hui, Siwei Yang, Bingchen Zhao, Yichun Shi, Heng Wang, Peng Wang, Yuyin Zhou, and Cihang Xie.
Hq-edit: A high-quality dataset for instruction-based image editing.
*arXiv preprint arXiv:2404.09990*, 2024.
- Jones et al. (2024)
Maxwell Jones, Sheng-Yu Wang, Nupur Kumari, David Bau, and Jun-Yan Zhu.
Customizing text-to-image models with a single image pair.
In *SIGGRAPH Asia Conference Proceedings*, 2024.
- Kang et al. (2023)
Minguk Kang, Jun-Yan Zhu, Richard Zhang, Jaesik Park, Eli Shechtman, Sylvain Paris, and Taesung Park.
Scaling up gans for text-to-image synthesis.
In *IEEE Conference on Computer Vision and Pattern Recognition (CVPR)*, 2023.
- Kang et al. (2024)
Minguk Kang, Richard Zhang, Connelly Barnes, Sylvain Paris, Suha Kwak, Jaesik Park, Eli Shechtman, Jun-Yan Zhu, and Taesung Park.
Distilling diffusion models into conditional gans.
In *European Conference on Computer Vision (ECCV)*, 2024.
- Kim et al. (2024)
Dongjun Kim, Chieh-Hsin Lai, Wei-Hsiang Liao, Naoki Murata, Yuhta Takida, Toshimitsu Uesaka, Yutong He, Yuki Mitsufuji, and Stefano Ermon.
Consistency trajectory models: Learning probability flow ode trajectory of diffusion.
In *International Conference on Learning Representations (ICLR)*, 2024.
- Kim et al. (2023)
Yunji Kim, Jiyoung Lee, Jin-Hwa Kim, Jung-Woo Ha, and Jun-Yan Zhu.
Dense text-to-image generation with attention modulation.
In *IEEE International Conference on Computer Vision (ICCV)*, 2023.
- Kirstain et al. (2023)
Yuval Kirstain, Adam Polyak, Uriel Singer, Shahbuland Matiana, Joe Penna, and Omer Levy.
Pick-a-pic: An open dataset of user preferences for text-to-image generation.
*Advances in neural information processing systems*, 36:36652–36663, 2023.
- Krojer et al. (2024)
Benno Krojer, Dheeraj Vattikonda, Luis Lara, Varun Jampani, Eva Portelance, Chris Pal, and Siva Reddy.
Learning action and reasoning-centric image editing from videos and simulation.
*Advances in Neural Information Processing Systems*, 37:38035–38078, 2024.
- Ku et al. (2024)
Max Ku, Dongfu Jiang, Cong Wei, Xiang Yue, and Wenhu Chen.
Viescore: Towards explainable metrics for conditional image synthesis evaluation.
In *Association for Computational Linguistics (ACL)*, 2024.
- Kumari et al. (2023)
Nupur Kumari, Bingliang Zhang, Richard Zhang, Eli Shechtman, and Jun-Yan Zhu.
Multi-concept customization of text-to-image diffusion.
In *IEEE Conference on Computer Vision and Pattern Recognition (CVPR)*, 2023.
- Kumari et al. (2025)
Nupur Kumari, Xi Yin, Jun-Yan Zhu, Ishan Misra, and Samaneh Azadi.
Generating multi-image synthetic data for text-to-image customization.
In *IEEE International Conference on Computer Vision (ICCV)*, 2025.
- Labs et al. (2025)
Black Forest Labs, Stephen Batifol, Andreas Blattmann, Frederic Boesel, Saksham Consul, Cyril Diagne, Tim Dockhorn, Jack English, Zion English, Patrick Esser, et al.
Flux. 1 kontext: Flow matching for in-context image generation and editing in latent space.
*arXiv preprint arXiv:2506.15742*, 2025.
- Li et al. (2024)
Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Ziwei Liu, et al.
Llava-onevision: Easy visual task transfer.
*arXiv preprint arXiv:2408.03326*, 2024.
- Lin et al. (2025)
Bin Lin, Zongjian Li, Xinhua Cheng, Yuwei Niu, Yang Ye, Xianyi He, Shenghai Yuan, Wangbo Yu, Shaodong Wang, Yunyang Ge, et al.
Uniworld: High-resolution semantic encoders for unified visual understanding and generation.
*arXiv preprint arXiv:2506.03147*, 2025.
- Lipman et al. (2023)
Yaron Lipman, Ricky TQ Chen, Heli Ben-Hamu, Maximilian Nickel, and Matt Le.
Flow matching for generative modeling.
In *International Conference on Learning Representations (ICLR)*, 2023.
- Liu et al. (2023a)
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee.
Visual instruction tuning.
In *Conference on Neural Information Processing Systems (NeurIPS)*, 2023a.
- Liu et al. (2025a)
Jie Liu, Gongye Liu, Jiajun Liang, Yangguang Li, Jiaheng Liu, Xintao Wang, Pengfei Wan, Di Zhang, and Wanli Ouyang.
Flow-grpo: Training flow matching models via online rl.
In *Conference on Neural Information Processing Systems (NeurIPS)*, 2025a.
- Liu et al. (2025b)
Shiyu Liu, Yucheng Han, Peng Xing, Fukun Yin, Rui Wang, Wei Cheng, Jiaqi Liao, Yingming Wang, Honghao Fu, Chunrui Han, et al.
Step1x-edit: A practical framework for general image editing.
*arXiv preprint arXiv:2504.17761*, 2025b.
- Liu et al. (2023b)
Xingchao Liu, Chengyue Gong, and Qiang Liu.
Flow straight and fast: Learning to generate and transfer data with rectified flow.
In *International Conference on Learning Representations (ICLR)*, 2023b.
- Lu & Song (2025)
Cheng Lu and Yang Song.
Simplifying, stabilizing and scaling continuous-time consistency models.
In *International Conference on Learning Representations (ICLR)*, 2025.
- Luo et al. (2025)
Grace Luo, Jonathan Granskog, Aleksander Holynski, and Trevor Darrell.
Dual-process image generation.
In *IEEE International Conference on Computer Vision (ICCV)*, 2025.
- Luo et al. (2023)
Weijian Luo, Tianyang Hu, Shifeng Zhang, Jiacheng Sun, Zhenguo Li, and Zhihua Zhang.
Diff-instruct: A universal approach for transferring knowledge from pre-trained diffusion models.
*Advances in Neural Information Processing Systems*, 36:76525–76546, 2023.
- Magar et al. (2025)
Nadav Magar, Amir Hertz, Eric Tabellion, Yael Pritch, Alex Rav-Acha, Ariel Shamir, and Yedid Hoshen.
Lightlab: Controlling light sources in images with diffusion models.
In *Proceedings of the Special Interest Group on Computer Graphics and Interactive Techniques Conference Conference Papers*, pp. 1–11, 2025.
- Meng et al. (2022)
Chenlin Meng, Yutong He, Yang Song, Jiaming Song, Jiajun Wu, Jun-Yan Zhu, and Stefano Ermon.
Sdedit: Guided image synthesis and editing with stochastic differential equations.
In *International Conference on Learning Representations (ICLR)*, 2022.
- Oquab et al. (2023)
Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy Vo, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al.
Dinov2: Learning robust visual features without supervision.
In *Transactions on Machine Learning Research (TMLR)*, 2023.
- Parmar et al. (2023)
Gaurav Parmar, Krishna Kumar Singh, Richard Zhang, Yijun Li, Jingwan Lu, and Jun-Yan Zhu.
Zero-shot image-to-image translation.
In *ACM SIGGRAPH 2023 Conference Proceedings*, 2023.
- Peebles & Xie (2023)
William Peebles and Saining Xie.
Scalable diffusion models with transformers.
In *IEEE International Conference on Computer Vision (ICCV)*, 2023.
- Qwen-Team (2024)
Qwen-Team.
Qwen2 technical report.
*arXiv preprint arXiv:2407.10671*, 2024.
- Qwen-Team (2025)
Qwen-Team.
Qwen2.5-vl, January 2025.
URL [https://qwenlm.github.io/blog/qwen2.5-vl/](https://qwenlm.github.io/blog/qwen2.5-vl/).
- Radford et al. (2021)
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al.
Learning transferable visual models from natural language supervision.
In *International Conference on Machine Learning (ICML)*, 2021.
- Ramesh et al. (2022)
Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen.
Hierarchical text-conditional image generation with clip latents.
*arXiv preprint arXiv:2204.06125*, 2022.
- Ruiz et al. (2023)
Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfir Aberman.
Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation.
In *IEEE Conference on Computer Vision and Pattern Recognition (CVPR)*, 2023.
- Salimans & Ho (2022)
Tim Salimans and Jonathan Ho.
Progressive distillation for fast sampling of diffusion models.
In *International Conference on Learning Representations (ICLR)*, 2022.
- Sauer et al. (2024a)
Axel Sauer, Frederic Boesel, Tim Dockhorn, Andreas Blattmann, Patrick Esser, and Robin Rombach.
Fast high-resolution image synthesis with latent adversarial diffusion distillation.
In *SIGGRAPH Asia Conference Proceedings*, 2024a.
- Sauer et al. (2024b)
Axel Sauer, Dominik Lorenz, Andreas Blattmann, and Robin Rombach.
Adversarial diffusion distillation.
In *European Conference on Computer Vision (ECCV)*, 2024b.
- Schuhmann et al. (2021)
Christoph Schuhmann, Richard Vencu, Romain Beaumont, Robert Kaczmarczyk, Clayton Mullis, Aarush Katta, Theo Coombes, Jenia Jitsev, and Aran Komatsuzaki.
Laion-400m: Open dataset of clip-filtered 400 million image-text pairs.
*arXiv preprint arXiv:2111.02114*, 2021.
- Shao et al. (2024)
Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, et al.
Deepseekmath: Pushing the limits of mathematical reasoning in open language models.
*arXiv preprint arXiv:2402.03300*, 2024.
- Sohn et al. (2023)
Kihyuk Sohn, Nataniel Ruiz, Kimin Lee, Daniel Castro Chin, Irina Blok, Huiwen Chang, Jarred Barber, Lu Jiang, Glenn Entis, Yuanzhen Li, et al.
Styledrop: Text-to-image generation in any style.
In *Conference on Neural Information Processing Systems (NeurIPS)*, 2023.
- Song et al. (2021)
Jiaming Song, Chenlin Meng, and Stefano Ermon.
Denoising diffusion implicit models.
In *International Conference on Learning Representations (ICLR)*, 2021.
- Song & Dhariwal (2024)
Yang Song and Prafulla Dhariwal.
Improved techniques for training consistency models.
In *International Conference on Learning Representations (ICLR)*, 2024.
- Song et al. (2023a)
Yang Song, Prafulla Dhariwal, Mark Chen, and Ilya Sutskever.
Consistency models.
In *Proceedings of the 40th International Conference on Machine Learning*, 2023a.
- Song et al. (2023b)
Yizhi Song, Zhifei Zhang, Zhe Lin, Scott Cohen, Brian Price, Jianming Zhang, Soo Ye Kim, and Daniel Aliaga.
Objectstitch: Object compositing with diffusion model.
In *IEEE Conference on Computer Vision and Pattern Recognition (CVPR)*, 2023b.
- Sun et al. (2024)
Quan Sun, Yufeng Cui, Xiaosong Zhang, Fan Zhang, Qiying Yu, Zhengxiong Luo, Yueze Wang, Yongming Rao, Jingjing Liu, Tiejun Huang, and Xinlong Wang.
Generative multimodal models are in-context learners.
In *IEEE Conference on Computer Vision and Pattern Recognition (CVPR)*, 2024.
- Sushko et al. (2025)
Peter Sushko, Ayana Bharadwaj, Zhi Yang Lim, Vasily Ilin, Ben Caffee, Dongping Chen, Mohammadreza Salehi, Cheng-Yu Hsieh, and Ranjay Krishna.
Realedit: Reddit edits as a large-scale empirical dataset for image transformations.
In *Proceedings of the Computer Vision and Pattern Recognition Conference*, pp. 13403–13413, 2025.
- Tan et al. (2025)
Zhenxiong Tan, Songhua Liu, Xingyi Yang, Qiaochu Xue, and Xinchao Wang.
Ominicontrol: Minimal and universal control for diffusion transformer.
In *IEEE International Conference on Computer Vision (ICCV)*, 2025.
- Wallace et al. (2024)
Bram Wallace, Meihua Dang, Rafael Rafailov, Linqi Zhou, Aaron Lou, Senthil Purushwalkam, Stefano Ermon, Caiming Xiong, Shafiq Joty, and Nikhil Naik.
Diffusion model alignment using direct preference optimization.
In *IEEE Conference on Computer Vision and Pattern Recognition (CVPR)*, 2024.
- Wang et al. (2020)
Sheng-Yu Wang, Oliver Wang, Richard Zhang, Andrew Owens, and Alexei A Efros.
Cnn-generated images are surprisingly easy to spot… for now.
In *IEEE Conference on Computer Vision and Pattern Recognition (CVPR)*, 2020.
- Winter et al. (2024)
Daniel Winter, Matan Cohen, Shlomi Fruchter, Yael Pritch, Alex Rav-Acha, and Yedid Hoshen.
Objectdrop: Bootstrapping counterfactuals for photorealistic object removal and insertion.
In *European Conference on Computer Vision (ECCV)*, 2024.
- Wu et al. (2025)
Chenfei Wu, Jiahao Li, Jingren Zhou, Junyang Lin, Kaiyuan Gao, Kun Yan, Sheng ming Yin, Shuai Bai, Xiao Xu, Yilei Chen, Yuxiang Chen, Zecheng Tang, Zekai Zhang, Zhengyi Wang, An Yang, Bowen Yu, Chen Cheng, Dayiheng Liu, Deqing Li, Hang Zhang, Hao Meng, Hu Wei, Jingyuan Ni, Kai Chen, Kuan Cao, Liang Peng, Lin Qu, Minggang Wu, Peng Wang, Shuting Yu, Tingkun Wen, Wensen Feng, Xiaoxiao Xu, Yi Wang, Yichang Zhang, Yongqiang Zhu, Yujia Wu, Yuxuan Cai, and Zenan Liu.
Qwen-image technical report, 2025.
URL [https://arxiv.org/abs/2508.02324](https://arxiv.org/abs/2508.02324).
- Xiao et al. (2025)
Shitao Xiao, Yueze Wang, Junjie Zhou, Huaying Yuan, Xingrun Xing, Ruiran Yan, Chaofan Li, Shuting Wang, Tiejun Huang, and Zheng Liu.
Omnigen: Unified image generation.
In *Proceedings of the Computer Vision and Pattern Recognition Conference*, pp. 13294–13304, 2025.
- Xu et al. (2023)
Jiazheng Xu, Xiao Liu, Yuchen Wu, Yuxuan Tong, Qinkai Li, Ming Ding, Jie Tang, and Yuxiao Dong.
Imagereward: Learning and evaluating human preferences for text-to-image generation.
*Advances in Neural Information Processing Systems*, 36:15903–15935, 2023.
- Xu et al. (2024)
Yanwu Xu, Yang Zhao, Zhisheng Xiao, and Tingbo Hou.
Ufogen: You forward once large scale text-to-image generation via diffusion gans.
In *IEEE Conference on Computer Vision and Pattern Recognition (CVPR)*, 2024.
- Yan et al. (2023)
Wilson Yan, Andrew Brown, Pieter Abbeel, Rohit Girdhar, and Samaneh Azadi.
Motion-conditioned image animation for video editing.
*arXiv preprint arXiv:2311.18827*, 2023.
- Yang et al. (2024a)
Kai Yang, Jian Tao, Jiafei Lyu, Chunjiang Ge, Jiaxin Chen, Weihan Shen, Xiaolong Zhu, and Xiu Li.
Using human feedback to fine-tune diffusion models without any reward model.
In *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*, pp. 8941–8951, 2024a.
- Yang et al. (2024b)
Ling Yang, Bohan Zeng, Jiaming Liu, Hong Li, Minghao Xu, Wentao Zhang, and Shuicheng Yan.
Editworld: Simulating world dynamics for instruction-following image editing.
*arXiv preprint arXiv:2405.14785*, 2024b.
- Yang et al. (2024c)
Ling Yang, Zixiang Zhang, Zhilong Zhang, Xingchao Liu, Minkai Xu, Wentao Zhang, Chenlin Meng, Stefano Ermon, and Bin Cui.
Consistency flow matching: Defining straight flows with velocity consistency.
*arXiv preprint arXiv:2407.02398*, 2024c.
- Ye et al. (2025)
Yang Ye, Xianyi He, Zongjian Li, Bin Lin, Shenghai Yuan, Zhiyuan Yan, Bohan Hou, and Li Yuan.
Imgedit: A unified image editing dataset and benchmark.
*arXiv preprint arXiv:2505.20275*, 2025.
- Yin et al. (2024a)
Tianwei Yin, Michaël Gharbi, Taesung Park, Richard Zhang, Eli Shechtman, Fredo Durand, and Bill Freeman.
Improved distribution matching distillation for fast image synthesis.
In *Conference on Neural Information Processing Systems (NeurIPS)*, 2024a.
- Yin et al. (2024b)
Tianwei Yin, Michaël Gharbi, Richard Zhang, Eli Shechtman, Fredo Durand, William T Freeman, and Taesung Park.
One-step diffusion with distribution matching distillation.
In *IEEE Conference on Computer Vision and Pattern Recognition (CVPR)*, 2024b.
- Yu et al. (2025)
Xin Yu, Tianyu Wang, Soo Ye Kim, Paul Guerrero, Xi Chen, Qing Liu, Zhe Lin, and Xiaojuan Qi.
Objectmover: Generative object movement with video prior.
In *IEEE Conference on Computer Vision and Pattern Recognition (CVPR)*, 2025.
- Zhai et al. (2023)
Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer.
Sigmoid loss for language image pre-training.
In *IEEE International Conference on Computer Vision (ICCV)*, 2023.
- Zhang et al. (2023)
Kai Zhang, Lingbo Mo, Wenhu Chen, Huan Sun, and Yu Su.
Magicbrush: A manually annotated dataset for instruction-guided image editing.
*Advances in Neural Information Processing Systems*, 36:31428–31449, 2023.
- Zhang et al. (2024)
Lunjun Zhang, Arian Hosseini, Hritik Bansal, Mehran Kazemi, Aviral Kumar, and Rishabh Agarwal.
Generative verifiers: Reward modeling as next-token prediction.
*arXiv preprint arXiv:2408.15240*, 2024.
- Zhang et al. (2018)
Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang.
The unreasonable effectiveness of deep features as a perceptual metric.
In *IEEE Conference on Computer Vision and Pattern Recognition (CVPR)*, 2018.
- Zhao et al. (2024)
Haozhe Zhao, Xiaojian Shawn Ma, Liang Chen, Shuzheng Si, Rujie Wu, Kaikai An, Peiyu Yu, Minjia Zhang, Qing Li, and Baobao Chang.
Ultraedit: Instruction-based fine-grained image editing at scale.
*Advances in Neural Information Processing Systems*, 37:3058–3093, 2024.
- Zhou et al. (2025)
Linqi Zhou, Stefano Ermon, and Jiaming Song.
Inductive moment matching.
*arXiv preprint arXiv:2503.07565*, 2025.
- Zhou et al. (2024)
Mingyuan Zhou, Huangjie Zheng, Zhendong Wang, Mingzhang Yin, and Hai Huang.
Score identity distillation: Exponentially fast distillation of pretrained diffusion models for one-step generation.
In *Forty-first International Conference on Machine Learning*, 2024.
- Zhu et al. (2017)
Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A Efros.
Unpaired image-to-image translation using cycle-consistent adversarial networks.
In *IEEE International Conference on Computer Vision (ICCV)*, 2017.

## Appendix A Additional Comparison with Baseline Methods

Local image-editing. Here, we compare on another commonly adopted image-editing benchmark, ImgEdit (Ye et al., 2025) (Basic), which covers nine local editing types across diverse semantic categories with a total of $734$ samples. Quantitative results under their proposed GPT4o-based evaluation protocol are reported in Table [5](https://arxiv.org/html/2510.14978v1#A1.T5), with VIEScore (Ku et al., 2024) results in Table [6](https://arxiv.org/html/2510.14978v1#A1.T6). Consistent with the trend observed on GEdit-Bench, our method has better or on-par performance in the few-step setting and remains comparable to many of the multi-step baselines as well. Qualitative comparisons are shown in Figure [11](https://arxiv.org/html/2510.14978v1#A7.F11), with additional examples on GEdit-Bench in Figure [10](https://arxiv.org/html/2510.14978v1#A7.F10).

Customization or free-form editing. Here, we report additional metrics commonly used for evaluation. Specifically, CLIPScore (Radford et al., 2021) and TIFA (Hu et al., 2023) to measure text alignment; and similarity in DINOv2 (Oquab et al., 2023) feature space after background masking to measure identity alignment, denoted as MDINOv2-I. We also report an overall Geometric score (Yan et al., 2023) by taking the geometric mean of TIFA and MDINOv2-I. The results are shown in Table [7](https://arxiv.org/html/2510.14978v1#A1.T7). Consistent with the VIEScore evaluation reported in the main paper, our method performs comparably with other baselines in the few-step sampling regime. We show more sample comparisons in Figure [12](https://arxiv.org/html/2510.14978v1#A7.F12).

**Table 5: ImgEdit-Bench comparison using their proposed GPT-4o based evaluation protocal and few-step sampling setting. Our method outperforms baseline methods on the *Avg* of all edit types.**
| Method | $\#$Param | Action$\uparrow$ | Bg $\uparrow$ | Style $\uparrow$ | Adjust $\uparrow$ | Replace $\uparrow$ | Add $\uparrow$ | Extract $\uparrow$ | Remove $\uparrow$ | Compose $\uparrow$ | Avg $\uparrow$ |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Qwen-Image-Edit | 20B | 3.14 | 2.83 | 3.70 | 3.25 | 3.00 | 3.52 | 1.96 | 2.71 | 3.06 | 3.02 |
| Flux.1-Kontext | 12B | 3.51 | 2.97 | 3.89 | 3.04 | 3.15 | 3.31 | 1.82 | 2.37 | 2.46 | 2.95 |
| Step1X Edit | 12B | 3.66 | 2.60 | 3.46 | 3.44 | 2.50 | 3.25 | 1.77 | 2.41 | 2.38 | 2.83 |
| NP-Edit (Ours) | 2B | 4.44 | 4.13 | 4.14 | 3.94 | 3.57 | 4.52 | 2.01 | 2.71 | 3.18 | 3.63 |

**Table 6: VIEScore evaluation on ImgEdit-Bench. Our method performs on par or better than baselines under the few-step setting. For multi-step sampling, it still outperforms OmniGen and remains competitive with many of the larger-scale models like BAGEL and FLUX.1 Kontext. All numbers reported in $\times 10$**
| Method | $\#$Param | $\#$Step | SC Score$\uparrow$ | PQ Score $\uparrow$ | Overall $\uparrow$ |
| --- | --- | --- | --- | --- | --- |
| BAGEL | 7B | 50 | 7.55 | 6.22 | 6.47 |
| FLUX.1-Kontext | 12B | 28 | 6.94 | 6.73 | 6.19 |
| Step-1X Edit v1.1 | 12B | 28 | 7.26 | 7.30 | 6.72 |
| QwenImage Edit | 20B | 50 | 8.30 | 7.77 | 7.85 |
| QwenImage Edit | 20B | 4 | 6.23 | 5.14 | 5.46 |
| FLUX.1-Kontext | 12B | 4 | 6.08 | 5.22 | 5.14 |
| Step-1X Edit v1.1 | 12B | 4 | 6.00 | 5.37 | 5.14 |
| NP-Edit (Ours) | 2B | 4 | 6.72 | 7.78 | 6.62 |

**Table 7: Quantitative evaluation of free-form editing task, *Customization*, on DreamBooth dataset. All numbers reported in $\times 10$**
| Method | $\#$Param | $\#$Step | MDINO Score$\uparrow$ | CLIP Score $\uparrow$ | TIFA $\uparrow$ | Geometric Score $\uparrow$ |
| --- | --- | --- | --- | --- | --- | --- |
| DSD | 12B | 28 | 6.55 | 3.08 | 8.71 | 7.32 |
| SynCD | 12B | 30 | 7.34 | 3.09 | 8.53 | 7.71 |
| FLUX.1-Kontext | 12B | 28 | 7.72 | 3.07 | 8.88 | 8.14 |
| Qwen-Image-Edit | 20B | 50 | 7.47 | 3.14 | 9.37 | 8.22 |
| OminiControl | 12B | 8 | 6.16 | 3.02 | 8.12 | 6.64 |
| DSD | 12B | 8 | 5.88 | 3.15 | 8.93 | 6.99 |
| SynCD | 12B | 8 | 7.11 | 3.16 | 9.11 | 7.79 |
| FLUX.1-Kontext | 12B | 8 | 7.50 | 3.08 | 8.83 | 7.98 |
| Qwen-Image-Edit | 20B | 8 | 7.29 | 3.08 | 8.96 | 7.91 |
| NP-Edit (Ours) | 2B | 8 | 6.82 | 2.97 | 8.73 | 7.54 |
| NP-Edit (Ours) | 2B | 4 | 7.03 | 3.04 | 8.89 | 7.72 |

## Appendix B Ablation Study

Figure: Figure 5: Performance for each sub edit-type. Training with only DMD loss fails to achieve certain tasks like removal and style changes. In addition, using binary cross-entropy loss and VLM identity-based questions helps improve the overall performance.
Refer to caption: x5.png

Figure: Figure 7: Unreliable VLM response on intermediate outputs of a multi-step diffusion model. Here we show a 28-step diffusion process, denoising predictions from early steps (e.g., $t=4$), which correspond to high noise levels, are blurry and semantically ambiguous. This can lead to unreliable responses from the VLM, as shown here. Therefore, we adopt a few-step diffusion model that always generates sharp images.
Refer to caption: x7.png

Figure: Figure 9: Limitation. Our method can struggle to maintain exact pixel consistency between the input and edited image. Having LPIPS (Zhang et al., 2018) loss between the input and output edited image can resolve it to an extent (top row) but at the cost of reduced editing success (bottom row).
Refer to caption: x9.png

Training objective. Here, we provide a more detailed analysis by examining performance across different editing sub-types. As a reminder, we ablated our training objective under four settings: (1) using only the distribution matching loss, (2) using only the VLM-based editing loss, (3) removing the identity-preservation question from $\mathcal{D}_{QA}$, and (4) replacing the binary-cross entropy loss as explained in Eqn. [4](https://arxiv.org/html/2510.14978v1#S4.E4) with standard cross-entropy over full vocabulary length. As shown in Figure [5](https://arxiv.org/html/2510.14978v1#A2.F5), training with only the DMD loss yields comparable performance on certain sub-edit types such as *Color change*, since DMD matches the text-conditioned score between the fine-tuned and pre-trained models, thus improving overall text alignment. However, it fails on tasks like *Removal*, underscoring the importance of VLM-based editing loss and its generalizability across diverse editing instructions. In addition, VLM-based loss also helps in maintaining better consistency between input and edited images (first row of Figure [4](https://arxiv.org/html/2510.14978v1#S5.F4) in the main paper). However, when training with only the VLM-based editing loss, there’s a gradual degradation in image quality, as Figure [6](https://arxiv.org/html/2510.14978v1#A2.F6) shows, highlighting the complementary role of distribution matching losses such as DMD.

Sampling steps. For our method, we chose to train a few-step image-editing model instead of a multi-step diffusion model, as multi-step diffusion has a noisy or blurry estimate of the final output in the early stages of diffusion. This can make it difficult to get a reliable response from the VLM, as shown in Figure [7](https://arxiv.org/html/2510.14978v1#A2.F7). On the other hand, predicting an edited image in one step is still challenging, as mentioned in the main paper, and shown here in Figure [8](https://arxiv.org/html/2510.14978v1#A2.F8). Thus few-step provides a nice balance between the two extremes of single and multi-step sampling for our purposes.

## Appendix C Limitation

A limitation of our framework is the lack of pixel-wise supervision to preserve regions that should remain unchanged under a given edit instruction. Consequently, edited images can deviate from the input image in details or spatial alignment, as shown in Figure [9](https://arxiv.org/html/2510.14978v1#A2.F9), $1^{\text{st}}$ column. While our VLM-based editing loss includes a question to check consistency between the input and edited images, it does not enforce pixel-level alignment. Empirically, we find that current VLMs struggle to detect subtle changes. To mitigate this, we experiment with an additional LPIPS (Zhang et al., 2018) loss between the input and output edited images. As shown in the last column of Figure [9](https://arxiv.org/html/2510.14978v1#A2.F9), this improves consistency but also negatively impacts editing quality, particularly for edit-types like *Removal*. Future work could explore specialized VLMs that are more sensitive to fine-grained, pixel-level differences.

## Appendix D Dataset Construction Details

Each tuple in our dataset $\mathcal{X}=\{({\mathbf{y}}_{i},{\mathbf{c}}_{i},{\mathbf{c}}^{{\mathbf{y}}}_{i},{\mathbf{c}}^{{\mathbf{x}}}_{i})\}_{i=1}^{N}$ consists of a real reference-image, ${\mathbf{y}}$, corresponding edit instruction, ${\mathbf{c}}$, and text prompt corresponding to the reference and edited image, ${\mathbf{c}}^{{\mathbf{y}}}$ and ${\mathbf{c}}^{{\mathbf{x}}}$, respectively. We use a text-image dataset corpus to select reference images. Given a reference image, we prompt Qwen-2.5-32B VLM to suggest different possible editing instructions. The system and user prompt for it are as follows:

In the above user prompt, for the good examples, we randomly select an edit instruction for each editing type out of a fixed set of manually defined edit instructions. Given edit instructions for each image, we again prompt the VLM to check the validity of the edit instruction and, if valid, to suggest a possible caption for the edited image. The system and user prompt for this is:

After filtering the list of generated editing instructions using the above procedure, our final dataset consists of approximately $3$M unique reference images with corresponding editing instructions spanning the $10$ edit sub-types. Within the constraints of our available computational resources, this represents the largest dataset we were able to construct.

For the customization task, we first instruct the VLM, to identify if the image has a prominent object in the center. We provide an in-context sample image as well to the model. The exact system and user prompt for this is:

If the VLM response predicts a valid image, we then query it again to suggest a new background context for the object category as follows:

## Appendix E Training Implementation Details

### E.1 Local-image editing

Training hyperparameters. We train on a batch-size of $32$ using Adam (Adam et al., 2014) optimizer with a learning rate of $2\times 10^{-6}$, $\beta_{1}$ as 0, and $\beta_{2}$ as $0.9$. We train for a total of $10$K iteratuions with auxiliary network, $A_{\phi}$, being updated $10$ times for every generator, $G_{\theta}$, update, following similar strategy adopted in DMD2 (Yin et al., 2024a). We train with the identity loss (Section [4.3](https://arxiv.org/html/2510.14978v1#S4.SS3)) for $250$ iterations. For faster convergence, the first $4$K training iterations are trained with a single step prediction ($t=1$ in Line 3 of Algorithm [1](https://arxiv.org/html/2510.14978v1#alg1)), and then we start the 2-step unrolling of the diffusion trajectory. The final loss is a weighted combination of VLM-based editing loss and distribution matching loss with $\lambda_{\text{VLM}}=0.01$ and $\lambda_{\text{DMD}}=0.5$. During training, we also add a “do nothing” editing task with $\mathcal{L}_{2}$ loss between the input and edited image as regularization with a $1\%$ probability. This helps the model learn to maintain consistency between input and edited images. During training, we sample the editing instruction corresponding to each subtype uniformly, except *removal*, which is sampled with $25\%$ probability. This is because, empirically, we observe that *removal* is more difficult than other edit-types like *Color change*.

Template questions for VLM-based editing loss.
As mentioned in the main paper, we evaluate VLM-based loss on two questions per edit type. Specifically for any edit type except removal, we use the following template:

For the removal edit-type, we change the first question to explicitly ask about the presence of the target object to be removed, with the ground truth answer in this case being *No*. We find it to be more effective than a generic template.

### E.2 Free-form editing (Customization)

Training hyperparameters. We reduce the warmup iterations for which we train with the identity loss to $100$ in this case, since customization often requires more drastic changes in the output image compared to the input reference image. Further, we increase $\lambda_{\text{DMD}}=2$ instead of $0.5$ as in the case of local image-editing. The rest of the hyperparameters remain the same. Both during training and inference, the input text prompt to the few-step generator, $G_{\theta}$, is in the following template: Generate the main object shown in the first image in a different setting and pose: { background scene description}. We train the 4-step model for $10$K iterations. For the 8-step model, we fine-tune for $5$K additional training steps starting from the 4-step model.

Template questions for VLM-based editing loss.
Here, we modify the questions to instead evaluate if the background context and pose are different in the generated image, i.e., editing success, and if the object identity is similar, i.e., image alignment and consistency between the input reference and edited image. The exact questions are as follows:

## Appendix F Other Baseline Details

Flow-GRPO (Liu et al., 2025a). We follow the open-source implementation of Flow-GRPO and train with the same computational budget as our method, i.e., across $4$ A100 GPUs and $2.5$ days of training. The final model is fine-tuned from a pre-trained image-editing model for $5$K iterations. During training, we collect $16$ images per-prompt with $12$ denoising steps ($28$ during inference) for computing the mean and standard deviation in GRPO (Shao et al., 2024). Following their official implementation, we train with LoRA (Hu et al., 2022) of rank $32$, $\alpha=64$, learning rate $1\times 10^{-4}$, and use the VLM to score edits on a scale of 0 to 9, which is normalized between 0-1 to get the final reward. The exact prompt used to query the VLM is derived from VIEScore (Ku et al., 2024) and is shown below.

Supervised Fine-Tuning. We train with the standard velocity prediction flow-objective for $30$K iterations with a batch-size of $32$ and learning rate $2\times 10^{-6}$ with a linear warmup of $2$K iterations. To enable classifier-free guidance, we drop the image and text conditions $10\%$ of the time.

Sampling parameters for local image-editing baselines. We follow the open-source implementation to sample images from all the baseline models for the benchmark evaluations. The turbo-edit (Deutch et al., 2024) baseline requires a caption corresponding to the edited image as well, and we use Qwen-2.5-32B-VLM to generate these captions for GEdit-Bench images.

Sampling parameters for customization baselines. Here as well, we follow the open-source implementation to sample images from all the baseline models for the benchmark evaluations. In the case of DSD (Cai et al., 2025), it employs Gemini-1.5 to convert the input user-prompt into a detailed prompt. However, we skipped this step for a fair evaluation with other methods, which do not use any prompt rewriting tools. In the case of SynCD (Kumari et al., 2025), though it supports multiple reference images as input, we evaluated it with a single reference image, to keep the setup similar to other baseline methods and Ours. For sampling images with OminiControl (Tan et al., 2025) and DSD (Cai et al., 2025), we follow their recommended prompt setting and replace the category name with “this item”.

## Appendix G Societal Impact

Our work introduces a training framework for fine-tuning text-to-image models into a *few-step* image-editing model without using paired supervision. By enabling few-step sampling, our method improves inference efficiency and reduces computational cost. Nonetheless, the broader risks of generative models, such as creating deepfakes and misleading content, also apply to our approach. Possible ways to mitigate this are watermarking generative content Fernandez et al. (2023) and reliable detection of generated images Wang et al. (2020); Corvi et al. (2023); Cazenavette et al. (2024)

Figure: Figure 10: Qualitative comparison on GEdit-Bench. We show results of our and baseline image-editing methods under the few-step sampling setting. For comparison, we also show the results of the best method with multi-step sampling, as measured by the quantitative metrics (Table [1](https://arxiv.org/html/2510.14978v1#S5.T1)), in the ${}^{1}{\text{st}}$ column. Our method performs on par or better than baseline methods across different edit types in the few-step setting.
Refer to caption: x10.png

Figure: Figure 11: Qualitative comparison on ImgEdit-Bench. We show results of our and baseline image-editing methods under the few-step sampling setting. For comparison, we also show the results of the best method with multi-step sampling, as measured by the quantitative metrics (Table [6](https://arxiv.org/html/2510.14978v1#A1.T6)), in the $1^{\text{st}}$ column. Our method performs on par or better than baseline methods across different edit types in the few-step setting.
Refer to caption: x11.png

Figure: Figure 12: Qualitative comparison on DreamBooth. We show results of our and baseline methods under the few-step sampling setting. For comparison, we also show the results of the best method with multi-step sampling, as measured by the quantitative metrics in the first column. Our method performs comparably with baseline methods on identity alignment while having better image fidelity across different concepts in the few-step setting.
Refer to caption: x12.png

## Peer Reviews

confidence: 4
contribution: 3
initial_rating: 6
presentation: 3
questions: Please check the weakness section. It would also strengthen the paper to include a comparison with the recently released I2I model *Nanobanana*.
soundness: 4
strengths: S1. In the Abstract and Introduction section (Sec. 1.), the motivation for training an image editing model using the set of unpaired images are clearly explained. The paper is built on solid and strong motivation and requiredness of the task.

S2. The proposed components (loss functions) are plausible and details of VLM usage is clearly released in the Appendix. The idea of implementing unpaired training setup using the prior knowledge of VLM and manifold constraint is a novel approach.

S3. Related to S2, each loss function plays a critical role for performance improvement, as explained in the Ablation study section (Table 3). The paper clearly addresses the requiredness of each proposed loss term.

S4. The proposed method shows strong qualitative results against baselines, including image customization tasks. Quantitative results also reported, which shows the outstanding performance on GEdit-Bench dataset and on-part evaluation result on customization task.
summary: The paper proposes NP-Edit, a novel framework for training image editing diffusion models without any paired image data. Instead of relying on costly or synthetic edit supervision, NP-Edit trains a few-step diffusion model using direct gradient feedback from a Vision-Language Model (VLM), which evaluates whether an edit follows a natural language instruction while preserving unchanged content. In addition, the method incorporates Distribution Matching Distillation (DMD) to align edited outputs in the manifold of the pretrained T2I model. Experimental results show that NP-Edit achieves editing quality competitive with state-of-the-art methods..
weaknesses: W1. The method relies on the Vision Language Model (VLM) to judge whether the edited results of the model are correct. If the VLM misinterprets the image or instruction, the training signal becomes noisy, which can negatively impact convergence and editing accuracy. A more detailed analysis on VLM feedback dependency is required. How does the bias of VLM affect the model training procedure and overall performance? Is the Distribution Matching Distillation (DMD) enough to alleviate the bias of VLM?

W2. I was wondering if the method is applicable to more difficult tasks, such as 1) object addition or duplication, 2) object moving, and 3) enlarging or shrinking the object size. Extensive experiments with additional tasks is required.

W3. The method does not rely on supervision with paired before-to-after images. However, due to the absence of paired supervision, the edited output may drift from the reference object's identity, or introduce unwanted structural changes. Is there any analysis of the corresponding phenomenon? If the phenomenon that I mentioned is not crucial, please logically explain which part of the method alleviates this.

W4. Quantitative analysis on computation overhead is required. How much time is required to train the model and edit a single image?

---

confidence: 3
contribution: 3
initial_rating: 4
presentation: 3
questions: I  notice that there is  only one  non-rigid editing instruction ( which is referred as 'action') in the paper, " a person waves a hand". How about other non-rigid instructions? In TEDBench[1] editing benchmark,  Imagic[1] and Forgedit[2] could conduct various non-rigid editing, for example, let the dog sit/jump, or  let a bird spread its wings, let a bird look backward, close an open book etc. 

I am willing to adjust my rating if  the authors could  address my concerns.
soundness: 2
strengths: 1. The topic of eliminating the requirements of image pairs  is very important for In-context DiTs in image editing task. Because image-text pair are easy and cheap to collect and scale up. Instead, image pairs are expensive and rare. 

2. The model proposes VLM-editing loss and DMD loss, which seems novel. 

3. The results surprise me because according to my personal experience, training in-context DiTs with  image-text pairs usually leads to copy-paste problem. Yet the results on image customization task demonstrate that NPEdit could alter the spatial orientation and location quite well, which is interesting.
summary: This paper aims at eliminating the image pairs which are requested by contemporary in-context DiT model, which are expensive to curate and scale. Given only a reference image and an instruction, the NPEdit model is supervised by the VLM's feedback signals. The authors further designed VLM-based editing loss and DMD loss. Rich experiments are conducted and NPEdit achieves state-of-the-art results under few-step setting.
weaknesses: 1. Inadequate experiments on non-rigid editing, which is refered to as 'action' in the paper. NPEdit only demonstrates one non-rigid editing, 'a person waves a hand', which is not persuasive. In a previous benchmark TEdBench[1], Imagic + Imagen[1]  and Forgedit + SD 1.4 [2] could conduct some hard non-rigid instructions on this benchmark, for example,  let the dog sit, jump, or  let a bird spread its wings, let a bird looking backward, close an open book etc.

2. Facial identity preservation does not seem good. VLMs are generally not good at measuring facial similarity, far worse than specific face recognition models like arcface. Using VLMs' judgements as supervision signals may not be accurate. The facial identities in the editing results of NPEdit further confirm this doubt. 

3. The paper did not explain very clearly how to obtain the editing result fed to VLM during training. It seems like an inversion method. This step is very important in the training process and should be elaborated further. 


Reference:

[1] Imagic: Text-Based Real Image Editing with Diffusion Models

[2] Forgedit: Text Guided Image Editing via Learning and Forgetting

---

confidence: 4
contribution: 3
initial_rating: 8
presentation: 3
questions: Overall, this submission makes a conceptual and practical contribution to the field of text-guided image editing by eliminating dependence on paired training data through the use of VLM-based differentiable supervision. The approach is both innovative and impactful, offering a scalable alternative to traditional supervised paradigms. The authors are suggested to address the above weaknesses to further strengthen the work.
soundness: 3
strengths: 1. The paper addresses a well-recognized bottleneck in image editing, dependence on expensive, hard-to-scale paired datasets. The motivation is timely and relevant, given the increasing reliance on generative models in open-world applications.

2. The idea of replacing supervised pairs with gradient-based supervision from a VLM is conceptually elegant. This formulation could generalize beyond image editing to other multimodal generation tasks.

3. Competitive empirical performance under the few-step diffusion setting has been demonstrated.

4. The author also includes solid ablation studies and well-motivated analyses.
summary: This paper introduces NP-Edit, a new framework for training image editing diffusion models without any paired supervision. Instead of relying on input–output image pairs or synthetic datasets, NP-Edit leverages direct gradient feedback from Vision–Language Models (VLMs) to guide model optimization. The VLM evaluates whether an edit instruction is followed and whether unchanged regions are preserved, providing differentiable supervision. To maintain realism, the method also employs a Distribution Matching Distillation (DMD) loss to align generated images with the manifold of a pretrained diffusion model. The proposed framework achieves performance on par with supervised image editing diffusion models, despite requiring no paired data. The paper includes extensive experiments and ablation studies examining the influence of VLM backbone choice, dataset diversity, and loss formulation.
weaknesses: 1. Since supervision is entirely derived from VLM feedback, the model’s performance is directly tied to the accuracy, bias, and robustness of the chosen VLM. Any systematic bias (e.g., cultural or aesthetic preferences) or failure to interpret nuanced edit instructions could propagate into the trained model.

2. Unrolling the few-step diffusion model during training and backpropagating through the VLM introduces potentially high computational and memory costs. The paper lacks a detailed analysis of training efficiency or scalability to high-resolution datasets.

---

confidence: 4
contribution: 3
initial_rating: 6
presentation: 3
questions: * **Judge diversity.** Beyond LLaVA-OneVision-7B, what happens if you ensemble multiple VLMs (or mix families) as teachers during training? The scaling table shows benefits from stronger VLMs; an ensemble could mitigate single-judge biases—did you attempt this?  
* **Human evaluation.** Can you provide a small-scale user study (e.g., 100 instructions × 3 systems, pairwise preferences for edit correctness and fidelity) to corroborate VIEScore-based gains and address potential metric bias? 
* **Failure modes by edit type.** The ablations suggest that Removal and certain style edits are harder. What specific misbehaviors occur (e.g., partial removal, texture spillover), and can targeted question templates or auxiliary losses mitigate them?
soundness: 3
strengths: * **Originality.** The paper is, to my knowledge, the first to train a general instruction-following image editor using **gradient** feedback from a VLM (rather than scalar RL rewards or hand-crafted objectives), coupled with DMD to keep edits on the teacher manifold; this is a non-trivial and timely contribution in the post-training space.  
* **Quality.** The technical formulation is clear: a two-step unroll to produce cleaner intermediate states for reliable VLM judgments; a binary-logit loss over Yes/No tokens; and a KL-motivated DMD term implemented with an auxiliary velocity predictor. Ablations convincingly show that removing either VLM loss or DMD substantially degrades performance.    
* **Clarity.** The paper supplies an end-to-end algorithm box, precise templates for VLM prompts/questions, and concrete training hyperparameters (optimizer, learning rate schedule, warm-up, weighting, sampling times), all of which aid reproducibility.  
* **Significance.** Competitive results with a 2B few-step editor against much larger systems under few-step sampling, and solid customization performance, suggest a practical path toward data-efficient editing without paired supervision. The explicit focus on few steps is also valuable for latency-sensitive applications.
summary: This paper proposes NP-Edit, a training paradigm for instruction-following image editing that removes the need for paired before/after supervision. The core idea is to unroll a few-step diffusion generator during training and optimize it end-to-end with differentiable feedback from a vision–language model (VLM) that answers templated Yes/No questions about (i) whether an edit was executed and (ii) whether identity/context was preserved; a distribution-matching distillation (DMD) loss to a text-to-image teacher constrains realism. The method trains a lightweight 2B-parameter few-step editor (4 steps by default), with results reported on GEdit-Bench and DreamBooth customization showing competitive performance with larger supervised baselines under few-step sampling, plus ablations showing the complementary roles of VLM feedback and DMD.
weaknesses: * **Evaluation dependence on automated judges.** Most quantitative evaluation relies on VIEScore (GPT-4o-based) and related automated metrics; no human perceptual study is presented. This raises concerns about metric bias and circularity (a VLM both trains and evaluates), especially for nuanced fidelity judgments. Adding a user study or double-blind human pairwise comparisons would strengthen claims. 
* **VLM supervision reliability and calibration.** The paper shows VLM judgments are unreliable on noisy/blurry intermediates—motivating few-step training—but broader issues remain: sensitivity to prompt wording and model-specific biases. More analysis of calibration or robustness (e.g., multiple VLM judges, temperature/decoding settings) would be helpful.  
* **Resource/engineering overhead.** Keeping a VLM in GPU memory is noted as a limitation; concrete throughput/memory numbers for typical hardware (and at training vs inference) are not reported, which makes it harder to assess deployability.


Decision: Accept (Poster)