EditInfinity: Image Editing with Binary-Quantized Generative Models
Authors: Jiahuan Wang
Yuxin Chen
Jun Yu
Guangming Lu
Wenjie Pei
Venue: NeurIPS 2025 poster ()
Keywords: Image Editing; Text-to-Image Generation; Autoregressive Models
Primary area: applications

Abstract: Adapting pretrained diffusion-based generative models for text-driven image editing with negligible tuning overhead has demonstrated remarkable potential. A classical adaptation paradigm, as followed by these methods, first infers the generative trajectory inversely for a given source image by image inversion, then performs image editing along the inferred trajectory guided by the target text prompts. However, the performance of image editing is heavily limited by the approximation errors introduced during image inversion by diffusion models, which arise from the absence of exact supervision in the intermediate generative steps. To circumvent this issue, we investigate the parameter-efficient adaptation of binary-quantized generative models for image editing, and leverage their inherent characteristic that the exact intermediate quantized representations of a source image are attainable, enabling more effective supervision for precise image inversion. Specifically, we propose EditInfinity, which adapts Infinity, a binary-quantized generative model, for image editing. We propose an efficient yet effective image inversion mechanism that integrates text prompting rectification and image style preservation, enabling precise image inversion. Furthermore, we devise a holistic smoothing strategy which allows our EditInfinity to perform image editing with high fidelity to source images and precise semantic alignment to the text prompts. Extensive experiments on the PIE-Bench benchmark across add, change, and delete editing operations, demonstrate the superior performance of our model compared to state-of-the-art diffusion-based baselines. Code available at: https://github.com/yx-chen-ust/EditInfinity.

EditInfinity: Image Editing with Binary-Quantized Generative Models
Jiahuan Wang"!
Yuxin Chen
Jun Yu'
Guangming Lu
Wenjie Peit"
Harbin Institute of Technology, Shenzhen
The Hong Kong University of Science and Technology
wangj Langjiahhanszhit@16..
ychenqa@connect. ust .k yujun@hit edu. wenjjtecoder@outlookcccc com
luguangmühit.ert edu. cn
Source Image
Editnfiiity (Ours)
AF Edit
Source Image
Editinfiniy (Durs)
KF Edit
Source Image
Editiffiiity (Ours)
Edit
Add Object
wearing earnings...
Bowerss on her black hair
with a red dog coller
Change Object
Source
tatinfinity It
Source
Eaitefiitty
Source Editnfinity (Dun)
bird X
Cat - fox
bench
sofa
Souce
Editnfiiity
Source Editefiiity
Sourci Editifieity
Delete Object
holdiing pink Rowers
- >-be-Bies over
seal or....
Figure 1: Our method, Editlnfinity, delivers strong performance in background preservation in unedited regions and text alignment in edited regions across diverse editing tasks, including add, change and delete object, showing clear advantages over the latest state-of oftte-o--hh-att tit-hh--rt art diffusion based method RF-Edit [53) as illustrated by representative examples.
Abstract
Adapting pretrained diffusion- based generative models for text-driven image edit- ing with negligible tuning overhead has demonstrated remarkable potential. A classical adaptation paradigm, as followed by these methods, first infers the gen- erative trajectory inversely for given source image by image inversion, then performs image editing along the inferred trajectory guided by the target text prompts. However, the performance of image editing heavily limited by the ap- proximation errors introduced during image inversion by diffusion models, which arise from the absence of exact supervision in the intermediate generative steps. To circumvent this issue, we investigate the parameter-efficient adaptation of generative models for image editing, and leverage their inherent characteristic that the exact intermediate quantized representations of source im- age are attainable, enabling more effective supervision for precise image inversion. Specifically, propose EditInfinity, which adapts Infinity, binary-quantized generative model, for image editing. We propose an efficient yet effective im- age inversion mechanism that integrates text prompting rectification and image style preservation, enabling precise image inversion. Furthermore, we devise a
Equal contribution.
Corresponding authors.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

holistic smoothing strategy which allows our EditInfinity to perform image edit- ing with high fidelity to source images and precise semantic alignment the text prompts. Extensive experiments on the PlE-Bench benchmark across 'add', "change', and 'delete' editing operations, demonstrate the superior performance of our model compared to state-of-the-an diffusion- based baselines. Code available at: htttp:://githiii com com/yx- ceee-chen-usee uuttnuustiiiiiii www.ddiinnniiiiynii
1 Introduction
Text-driven image editing aims to modify the content of an image in accordance with the given text prompts while maintaining the integrity of the unedited regions. In contrast to training-froon scratch methods [1] 222 63] that incur expensive training costs, the adaptation of pre-trained models, particularly diffusion-based generative models, with lightweight fine-tuning overhead has emerged as
a predominant paradigm for image editing, demonstrating remarkable potential [2 43).
A classical adaptation paradigm in diffusion models for image editing [29 20 2 consists of two essential steps: 1) image inversion, which aims to infer the generative trajectory along the sampling process in reverse for given source image, striving to reconstruct the image accurately, and 2) image editing, conducted along the inferred trajectory guided by the target text prompts. Consequently, the precision of image inversion is critical to the performance of image editing. Nevertheless, it is intractable to obtain the exact sampling trajectory of source image for a pretrained diffusion model. Thus, image inversion is either performed employing the deterministic sampling technique [5 42 10 15 20 56 to approximate the intermediate noisy representations along the reversed sampling path, or it is formulated as a optimization problem to finetune the pretrained diffusion model to fit the approximate intermediate results along the sampling path [29 9 Consequently, a potential limitation of this adaptation paradigm of diffusion models for image editing is that the performance of image editing is heavily constrained by the approximate errors introduced during image inversion.
To address aforementioned limitation, in this work, we investigate the parameter-efficien adaptation of binary-quantized generative models for image editing. Unlike diffusion models, dinary-quantizer generative models quantize images into discrete latent space and model the data distribution in this quantized space for generation. Thus, an inherent characteristic of binary-quantizer models is that the exact quantized representations for an arbitrary image can be directly inferred, potentially enabling more precise image inversion. Motivated by this observation, we propose Editlnfinity which adapts Infinity a mmnary-quantized generative model with powerful text-to- -omag generation capability- for image editing, following the classical 'image inversion-imag editing' adaptation paradigm.
Considering a pretrained Infinity mapping function between the distribution of textual prompts and image data distribution, performing inverse inference on the pretrained model to obtain the exact textual embedding for source image is intractable, whereas the user-provided source text prompts generally cannot precisely match with the source image. Therefore, we formulate the image inversion process of EditInfinity an optimization problem, aiming to learn an accurate textual embedding for a given source image, guided by provided source text prompts. A notable advantage of this design is that the intermediate multi-scale quantized representations by Infinity for the source image can be utilized as exact supervision to optimize the image inversion process, yielding precise image inversion and thereby, high-quality image editing. To conclude, make the following contribution.
We propose Editinfinity, which apply the classical 'image inversion-inen editing' adapta- tion paradigm to Infinity, prominent dinary-quantiin model, to investigate the parameter- efficient adaptation of binary-quantizae generative models for image editing.
We design an efficient yet effective image inversion mechanism comprising text prompting rectification and image style preservation, leveraging the quantized representations as exact supervision to enable precise image inversion.
We devise holistic smoothing strategy which allows Editlnfinity perform image editing with high fidelity to source image and precise semantic alignment to the text prompts.
We conduct extensive experiments on the PIE-Bench benchmark and comprehensively demonstrate the superior performance of our Editinfiiity compared to state-of-the-an diffusion-based approaches across diverse editing operations, excelling in both background preservation and semantic alignment with target text prompts.

---

2 Related Work
2.1
Image Editing with Diffusion Models
Image editing researches 27 16 21 31 have been predominantly driven by diffusion models [36
38 (3 and are broadly categorized into training- based and training- feaning-ffeeeeee paradigms [40]. Training-
based methods 58 39 for expensive training dataset limits practical applicability. In contrast, training-free [48
[8] 24 [2] achieve impressive editing capabilities. but their requirement
23 43
methods have emerged as a more flexible alternative, establishing an inversion-editing paradigm The inversion stage focuses accurate latent code inversion. Recent works (29 28
20
53 have
developed improved inversion samplers to case the inherent reconstruction inaccuracies. In the editing
32 25 26 leverage attention in diffusion models edit while preserving overall image structure. Despite these advances, a key limitation remains: existing methods fail to preserve both text aligmment fidelity and source image consistency. This trade-off between editability and faithfulness motivates us to investigate more robust editing frameworks.
stage, numerous methods []6 47
2.2 Autoregressive Image Generation Models
Autoregressive models have demonstrated remarkable scalability in image generation by leveraging next-token prediction, paradigm inherited from LLMs (Large Language Models) [52]. Early methods like PixeICNN [49] and PixeIRNN (50) model pixels directly, but their quadratic dependency growth makes high-resolution generation impractical. Thus, subsequent works avoid modeling the data distribution directly in pixel space and instead model it in compact latent space. As pioneering work, VQVAE [5]] constructs discrete latent space by vector quantization and learns the underlying latent distribution by autoregressive models. Recently, VAR (Visual RutoRegressive Modeling) 46 reformulates autoregressive image generation as next-scale prediction task, capturing global structural priors to achieve state-of-the-an generation quality while improving sampling speed.
The success of autoregressive models naturally extends to eext-to-image generation. Pioneering works like DALL-E [35] and CogVieu [8] unify text and image tokens within single transformer decoder. Subsequently, Parti [57] and LlamaGen [44] decouple text and image processing by employing dedicated text encoders to guide the autoregressive decoder. Then, HART (45) integrates VAR's hybrid tokenizers to improve generation quality. Latest, Infinity [14] advances autoregressive image generation by introducing Bitwise Visual AutoRegressive Modeling. establishes a new foundational model for autoregressive text-to-image models and achieves competitive results with diffusion-based approaches. As our method builds on Infinity, we outline its architecture Section
Preliminary: Infinity
Bitwise Multi-scale Residual Quantization. An image Hxx'3 is first encoded into the original feature F" which is then tokenized into bitwise multi-scale residual maps {Rx JK k= through iterative residual approximation. At scale k. residual features are computed between the original feature F and the cumulative feature Fx-1 from previous scales.
down (F PE-1, (ha, wx&) E hh xd
(1)
where down(-) performs bilinear downsampling to target resolution (hy, wxx. quantize residuals, Infinity adopts BSQ (Binary Spherical Quantization) [6]]
R = 2(*+) = Vd
sign
zk
E
hx xwxx
(2)
|
Va
Vd
Then, the cumulative feature Fk scale & computed recursively:
F up Ri, (hx, wx)) R
where up(.) denotes bilinear upsampling.
3

---

Bitwise Autoregressive Modeling, The transformer predicts residuals autoregressively across K scales, conditioned on the prompt Formally, the autoregressive likelihood is:
K
p(BLK(V(t)
II p (Ral R Rk-1, (()),
(4)
all previous scales
where Vi(.) denotes Flan-T5 [6].
To tackle the large codebook challenge, Infinity proposes the adinite-Vocabulary Classifier, which decomposes the prediction into df independent binary classifiers.
4
Method
Successful image editing requires precise content modifications that semantically align with target prompts while remaining faithful to unedited regions. To this end, classical adaptation paradigm repurposes pretrained text-to-image generative model to image editing through two steps: I) image inversion, which inversely infers the corresponding generative trajectory for the source image by reversing the sampling process, and 2) image editing, performed along the inferred generative trajectory guided by the target text prompts. Our proposed EditInfinity applies this paradigm to Infinity [14]. a ainary-qantizzed generative model, harnessing the inherent characteristics of quantized generative models to potentially achieve precise image inversion and high-quality image editing.
Ayou? "a vitse sitting on table near window" fius "the language style of this prompt is"
Source Image
(bour 448) [sos]
trainable
frozen
Norm
Trannffomer blook
Text Encoder
to learnable prompt)
fasu: source prompt
Self-Attention
instruction
Cross- Attee-tttnioon
learnable prompt
Projection
Cross-Attention
Encoder
FFN
LoRA
R0IF inversion tokens
[sos] +
Infinity
[C)
Infinity
R| source tokens
Lxw
(() quantization
R
R
R
R
R
R
Cre cross-entropy loss
Phese 1: textual prompting rectification
Phase 2: image style preservation
Figure 2: Image Inversion with Exact Supervision. Given a source image Isou and its prompt txou. we first quantize sou into exact tokens Ri.. Then, we concatenate taa with an instruction fans and learnable prompt f, which is optimized via Linv under the supervision of R Afterwards, the prompt is frozen, and LoRA applied to the FFN layers of Infinity to further reconstruct Isou-
4.1 Image Inversion with Exact Supervision
eext-to-image generative model performs image generation by learning a mapping from the distri- bution of text prompts to image data distribution. However, since the mapping function is unknown, it is intractable to inversely obtain the exact textual embedding for given image. Meanwhile, the user-provided source text prompt generally cannot precisely match the source image. To circumvent this problem, we formulate the image inversion process as an optimization problem with exact supervision to infer the text embedding precisely matched with the source image:
K
1
Line
Σ km
[R log Rine REc (t)))
(5)
K
where Cive is formulated as a cross-entropy loss applied to each inversion token 1i Compared to the diffusion-based models for image editing, key advantage of dinary-quantized generative models that the exact groundtruth of the intermediate outputs (R k in 'Infinity') ) for given image along the generative trajectory is attainable by token-wise quantization, enabling exact supervision for optimization of image inversion in Equation
Textual prompting rectification. To guide the optimization of Equation 5 toward a text embedding that matches the source image, we treat the source prompt l. sox as reference and apply text prompting tuning to rectify it into semantically aligned textual condition. Concretely, first augment tow

---

Edited Region
nou
Editing
g
frozen
sampling
target prompt
R
-Ey
Up
Fru instruction
Source image
ng
sampling
64
Edited Image
f lleamaale prompt
Raon
R : target tokens
[sos]
Infinity
R
Editing
Infinity *h
R
REX
tokens
Encoder
Decodar
FER edited tokens
Fear" "a plant sitting on
window" Fes "the language style of this pronge is",
piecewise linear
learnable prompt
pator
smoothing kerell
Encodingg Source Image
Multi-scale Autoregressive Editing
Edited Image
Decoding
Figure 3: Image Editing with Holistic Smoothing. First, source image is encoded into R At each step l of autoregressive generation, generated Ry conditioned on the concatenation of the target prompt tar, instruction linss and optimized learnable prompt I and then, is blended with Rg guided by piecewise linear smoothing kernel G, forming edited tokens D to prepare for guiding the next-scale generation. Finally, Et is decoded into the edited image.
with 20 learnable prompt tokens i and an instruction prompt fins (e.g.. "the language style this prompt is") to bridge the semantic gap between the source prompts and the solution. Second, pass too and Eins through the text encoder V (.) Infinity to obtain text embeddings (f tins). We then concatenate those embeddings with to form the textual conditioning input (V Lins), ) 4 for Infinity. Finally, we freeze all Infinity parameters and optimize only 4 using cross-entropy loss, where the supervision signals are exact tokens RT... derived from the source image.
Image style preservation. While the learnable prompt adapt the semantic content, they may fall short in preserving structural style characteristics. The low-rank bias [1] 411 (rank < dim (W)) favors smooth and global modifications to the output distribution, thereby encouraging reconstructions that preserve overall structure and appearance while avoiding overfitting this end, we employ LoRA [7] to refine the pretrained weights W with minimal overhead AW (only inserts trainable low-rank matrices into FFN layers (52]) after rectifying textual prompt. Then, the learned AW is retained during editing, allowing the model to faithfully preserve global style traits of the source image even when applying novel target prompts.
high-frequency artifacts. To
4.2 Image Editing with Holistic Smoothing
g
We aim manipulate only the desired regions while preserving the structural integrity of unedited areas. To this end, we introduce a precise token replacement strategy that enables localized, semantically aligned editing at the token level. Given the optimized learnable prompt t and LoRA AW, we perform conditional generation under the target prompt tear. instruction tims and optimized learnalle prompt t,, which ensures the edited image adheres to the target semantics while maintaining structural fidelity with the source image.
1
0
d"
Figure 4: G as a function of d, enabling smooth transitions from edited to unedited regions.
Piecewise Linear Smoothing Kernel. The core idea of our editing paradigm is to construct the edited tokens E by blending source tokens Rixx and target tokens Rig a spatially controlled manner. A direct blend will result in splicing phenomenon,
we first localize the edit with
user-provided mask M M-a standard setting in image editing (30 62] where text-only prompts often lack spatial specificity [16 37 [59]. Then, define a piecewise linear smoothing kernel G to guide the blending. Specifically, G is defined over the Manhattan distance to calculate location weights per location, in Equation
0g
H
g&._
d -T
<
d
T2
min (li - x + lj v)). (x.9)MM
(6)
1
> m2
5

---

Here, d'vl denotes the Manhattan distance from token (i.j) to the nearest token within M. The kernel G'd is designed to gradually transition from
within controllable band defined by thresholds πi and 72, which is visualized in Figure 4 Specifically, tokens within a distance of T from the edit region are assigned zero weight to encourage full preservation from target content, while those beyond 72 are fully replaced by the source. Tokens in the intermediate band are assigned weights via linear interpolation, facilitating smooth blending between source and target content. This formulation effectively suppresses boundary artifacts by promoting seamless transitions between source and edited regions.
Multi-scale Autoregressive Editing, Building on image inversion and the piecewise linear smoothing kernel G, we realize image editing multi-scale autoregressive token- olen-replaccement process. At each scale, generated target tokens are blended with source tokens under spatial weights provided by G, and the resulting edited tokens serve as context for the next scale. This coarse-to- fine schedule localizes semantic changes to the masked region while preserving global structure elsewhere Algorithm details the procedure.
Our algorithm begins by quantizing Loo to extract precise source tokens Re At each scale k Infinity(-) generates the target token Ri conditioned on
Algorithm 1 Multi-scale Autoregressive Editing
1: Inputs: source image soui target prompt ter; instruc-
tion timsi optimized learnable prompt fę;
2: Hyperparameters: scales K, resolutions(hs: wi )
previous tokens fig and prompts em- bedding [ (tear, tins); tt]. We then up- sample Ra and R to (hx, wx) and blend them under the guidance of to obtain the edited token E.e. This aligns edited regions with target seman- tics while preserving source fidelity else- where. & < K, we downsample Ex" to (hK+1, wk+1) ) to form R.. which serves the autoregressive state the next scale, allowing blended semantics
D E: encoder, Q: quantizer 4: (t tine), ty] projected into fibaa (i.e., [sos])
3: R = ((E((
5:
for = to K do
6:
R Infinity (Rtar, [Vtaar ); ti])
7:
Etar Upsample(
(a Rfar (hK, wx) )) @ (1 9)
Upsample (R (hK, WK Og
8:
if k < K then
9:
ptar = Downsample((E (1441,WH+11)
10: end if
11: end for
12: Iear D(E k)
> D: decoder
and structure propagate across scales. After traversing all scales, edited tokens Flaor are decoded to edited image har.
13: Return edited Image Iar
Delkts Object
Delits Object
SolleRow
Change Coeeii
Change Cordii
RF EAR
ORject
11.8%
13.8%
Crange
Change Ease
Nay 21/
Object
Change Orject
(Dart)
(Dunt
28.3%
Charge
F3F
Charge
P0R-2V1 201
bdil-feel [aurs
HiI.
Change
thange Resual
coblo Enw
Background Preservation
Alignment
LPIPS
Whole C1 IP Score
(a)
(c)
Figure 5: Comprehensive performance evaluation on PIE-Bench. (a) and (b) report background preservation and text alignment metrics across nine tasks. (c) summarizes user study preferences.
Experiments
5.1 Experimental Setup
Comparison Methods. We compare our method against a range of methods. (1) Open-source
methods: including Diffusion UNet models-P22 [16), MasaCtrl [. P2P-Zero 31 NTI 29
PnP-Inv 200, and NP (28 Diffusion Transformer Pondels-StableFiie (2) and RF-Edit 53
(2) Closed-son source method: Gemini 2.0 [3] current frontier of large-scale commercial model.

---

Benchmark. We conduct comprehensive experiments on PIE-Bench (Prompt- based Image Editing Benchmark) [20]. the prevailing standard in image editing evaluation. This benchmark contains 700 test cases covering nine editing types. Each case provides a source image with corresponding prompt, target editing prompt, and the editing mask.
Metries. Our evaluation employs seven carefully selected metrics across two critical dimensions. For background preservation, use four complementary metrics: PSNR and MSE for pixel-level accuracy, LPIPS [60] for perceptual similarity, and SSIM [54] for structural similarity. For text-image alignment, we report CLIP scores [34 of the whole image and the edited region with the target prompt. Additionally, we adopt IR (Image Reward (55) ), learned metric trained on human preference data,
specifically sensitive to editing failures, often assigning negative scores to failed outputs.
Implementation Details. We implement our method based on Infinity-2B 0 For editing, we set and 72 in Equation 66 Inversion is trained
NVIDIA 1.20 GPUs, and editing on a single NVIDIA L,20 GPU. Refer to Supplementary Material A.2 for more details.
Table 1: Quantitative results on PIE-Benchh In the 'Base Model' column, 'U", 'T", and 'A' represent Diffusion UNet, Diffusion Transformer, and Autoregressive models, respectively. Diffusion UNet models employ Stable Diffusion v1.4, with the exception of PnP-Inv, which utilizes v1.5. Diffusion Transformer models leverage FLUX.1-deW while Autoregressive models use Infinity.
Base Model
Background Preservation
Text Alignment
Method
Venue
PSNR1
LPIPS MSE 10
Wholet Edited
IRio7
P2P
ICLR'22
17.87
208.80
219.88
71.14
25.01
22.44
0.017
MasaCtrl
ICCV'23
22.17
106.62
86.97
79.67
23.96
21.16
-1.66
P2P-Zer
SIGGRAPH'25
20.44
172.22
144.12
74.67
22.80
20.54
-6.59
NTI 29
CVPR'23 23
27.03
60.67
35.86
84.11
24.75
21.86
2.77
PnP-Iny 20
ICLR '24
22.46
106.06
80.45
79.68
25.41
22.62
2P
WACV
26.21
69.01
39.73
83.40
24.61
21.87
2.42
StableFlow 2
CVPR'25
21.64
92.28
115.21
84.94
24.65
21.70
RF-Edit
ICML'25
T
23.22
131.18
75.00
81.44
25.22
22.40
5.18
Gemini
23.22
105.17
188.63
81.10
25.28
22.28
5.30
Editlnfinity
NeurlPS'2 25
27.95
33.08
24.27
92.12
26.41
23.47
5.88
Table 2: Evaluation of Base Models the GenEval Benchmark. When evaluating Infinity, adopt the same evaluation protocol as used for Stable Diffusion v1.4, v1.5, and FLUX. 1-dev, i.c., without prompt rewriting.
Base Model
Overall
Single Object
Two Object
Counting Colors
Position
Attribute Binding
Stable Diffusion v1.4
0.42
0.97
0.39
0.33
0.73
0.03
0.05
Stable Diffusion v1.5
0.43
0.97
0.38
0.35
0.76
0.04
0.06
FLUX.1-dey
0.66
0.98
0.81
0.74
0.79
0.22
0.45
Infinity-2B
0.66
0.98
0.78
0.83
0.25
0.53
5.2 Comparison to State-of-the-Arts
Quantitative Results. As demonstrated in Table 1 our method sets new state-of-the-art in text- driven image editing by significantly improving the trade-off hetween two key objectives: (1) rigorous background preservation and (2) precise text-aligned editing. While existing methods struggle with this inherent trade-off, our framework achieves a superior balance, outperforming all others by notable margins in both aspects. Notably, our method attains the best IR score (5.88), reflecting substantially higher editing success rates than competing methods. We further provide task-wise comparisons of LPIPS and full CLIP scores across all edit types. As shown in Figure 5 (a) and (b), these results consistently validate the effectiveness of our method across diverse editing scenarios.

---

Editlffinity (Ours)
Source Image
MasaCrrl
NTI
PnP-Inv
NP
Stable-Plow
RF-Fddt
(figer dog) swimming pond of green algue
Object
a cat with hat)
Add Object
too--parple flowers shoet of masic
Deleto
christmas illustration of santa's 's flaughing angry) face
Change Content
a light (sitting standing) the ground
Change Pose
(yellow red) apple sitting of a wooden table
Change Color
man in white shirt
standing in from of
Change Background
(trees mountains)
(yellow crochetj bird with red beak sitting on brunch
Change Material
(a bird real photo of hird) standing on tree branch
Change Style
Figure 6: Qualitative results on PIE-Bench across all nine tasks. The red mask denotes the edited
region M, expected to follow the target prompt, while other regions retain the background.
To ensure that the advantage of our method is not only attributed the stronger generative capacity of base model, we further analyze the base models used by all compared methods. Although our framework is lightweight and tailored Infinity, its reliance on the base model's generative capacity is consistent with other editing paradigms. As reported in Table we evaluate each method's 's base model on the GenEval benchmark [1]. Infinity performs comparably and even underperforms in certain tasks (e.g., two-object and counting). Nevertheless, our Infinity- based approach surpasses VT.UU-baase methods such as StableFlow and RF-Edit by large margin, demonstrating the effectiveness of our method despite the base model not having a clear advantage.
the popular FLUX [22]
Qualitative Results. Visual quality critical for evaluating image editing. Figure presents qualitative comparisons across all PIE-Bench tasks. For space, we omit P2P and P2P-Zero, which show weaker background preservation and text alignment, respectively (see Table D Our method achieves better trade-off between preserving unedited regions and accurately aligning edited regions with the target prompt. More visualizations are provided Supplementary Material A.5
diffusion models, all showing competitive performance in Table 1 The study uses 140 images from the 'random class' in PlE-Bench (20, covering all editing types. Fach of the 60 volunteers is randomly assigned 20 editing cases. For each case, they shown source image, target text prompt, and five edited results (randomly ordered and anonymized). Volunteers selected the best result via custom web interface, as shown in Supplementary Material Figure I0 Results Supplementary Material Figure (c) show 43.2% preferred our method-the highest among all approaches, confirming that maintains strong subjective visual quality.
User Study. Our method compares against two UNet-based and two transformer-
8

---

Learnalle Prompt y Letraable Procagt y Learnable Pronupt Y Leansable Prompt
Leurnable Pronut LoRA
< Learnable Prompt v/ Leamable Pronne
Source Image
LokA
LoKA
Lokk 10 Ites.
LoRAA Itec.
Y LoRA
LoRA
(+silver) cal (+sculpture) sitting next to a mirror
Figure 7: Illustrations of ablating the Learnable Prompt and LoRA.
Runtime Comparison. We conduct runtime comparison of our method and other methods on a single NVIDIA L20 GPU, measuring both in- version and editing time, shown in Table 3 A key advantage our method is efficient support for multiple edits on the same image, a com- mon real-world scenario. Once the inversion for given image completed, subsequent edits can be performed within 3.64 seconds, which over faster than other methods on aver- age, while the initial inversion time is only 4x longer than other methods on average. This de- sign effectively front-loads the computational cost, making ideal for iterative workflows.
Table 3: Runtime comparison. Time for edits on an image equals Inversion + n x Per-editing.
Method
Inversion (s)
Per-editing (s)
P2P[16]
14.40
10.28
MasaCtrl
5.19
17.45
P2P-Zero 31
13.31
62.29
NT
95.54
10.32
PnP-Imm 20
8.32
9.54
NP
9.00
10.37
StableFlow 2
13.85
27.20
RF-Edit(53]
55.48
54.07
EditInfinity
107.06
3.64
5.3 Ablation Study
Ablation studies are performed on the 'random class' of PIE-Bench, covering all types of editing and allowing an efficient and unbiased evaluation.
Ablation on Learnable Prompt and LoRA.
We design precise image inversion by lever- aging quantized tokens as exact supervision. It integrates the learnable prompt for textual rec- tification and a LoRA for style preservation. As shown in Figure 7 removing both compo- nents causes significant structural inconsisten- cies. The learnable prompt improves alignment with the target prompt but often shifts global style. Adding LoRA further restores stylistic consistency with the source image. However, prolonged training leads to overfitting, causing the model to ignore editing intents. To balance editability and fidelity, we stop training LoRA after 20 iterations, as shown in Figure
Lolk
25.4
<' Loaratire Horrgt
Leamabe Prorpt
ev
26.2
Leammbra Porrst
25.0
25.11
1082
25.6
LokA hec.
of
30
32
-]
Figure Quantitative results of ablating the Learn able Prompt and LoRA.
Ablation on Piecewise Linear Smoothing Kernel. We introduce ensure smooth transitions between edited and unedited regions and to suppress boundary artifacts. As shown in Figure 9 (c), removing results in sharp discontinuities along object boundaries (e.g., the cat's ears), confirming its effectiveness in producing seamless edits. To further examine the choice of smoothing function,
compare the linear kernel defined in Equation 6 with Gaussian kernel (1 df /2a" With proper hyperparameter tuning, the linear kernel achieves superior results, as reported in Table Complete results under both settings are provided in Supplementary Material (Tables 9 and TT).
Ablation on Mask. While our method defaults user-provice user-provided masks, can also leverage Infin- ity's 's cross-attention maps [16] for automatic mask generation without modifying the framework. Specificallyy, we automatically align differing words between the source and target prompts. After completing inversion, we input the source target prompt containing x: into Infinity(-) and extract the artss-attention map corresponding to r. threshold then applied: values above the thresh- old set to 0 (mask foreground), and others to (background). Table 5 shows that method

---

Table Quantitative results of ablating the Piecewise Linear Smoothing Kernel.
Background Preservation
Text Alignment
g
PSNR
LPIPS 10
MSE
SSIM10#1
WholeT
Edited
IR
X Gaussian kernel
31.12
24.47
13.03
93.53
25.44
23.12
2.85
28.15
32.91
24.40
92.17
26.10
23.81
4.61
Linear kernel
28.50
31.58
22.94
92.36
26.22
23.99
5.39
Table 5: Quantitative results of ablating the mask. EditInfinit-uu denotes user-provided masks, while Editlnfinity- denotes masks automatically generated via cross-attention. Best and second-bes results are shown in bold and italics, respectively.
Base Model
Background Preservation
Text Alignment
Method
PSNRT
LPIPS | MSE
SSIM
Wholet
Edited
IR 1o
NTI 29
28.08
57.94
36.10
85.17
24.71
22.51
3.63
RF-Edi
T
27.26
92.27
34.46
86.67
24.65
22.03
0.61
Editlnfinit--C
A
27.47
44.97
46.91
90.30
25.71
23.22
5.40
Editlnfinity-u
A
28.50
31.58
22.94
92.36
26.22
23.99
5.39
not highly sensitive to the source of the mask- strong performance is achieved in both cases. Comprehensive comparisons are reported in Supplementary Material Table II
Ablation on Multi-scale Autoregressive Edit- ing. By autoregressively blending source and target tokens (AR), source tokens in un-edited re- gions effectively guide the generation of editing regions. As illustrated in Figure (b). blending source tokens the end of autoregressive gen- eration (NAR, Non-Autoreegessivee results in incoherent and visually inconsistent edits. Thus, incorporating guidance at every scale is essential for producing harmonious and realistic results. The quantitative comparison is represented in Supplementary Material Table T2
w/ G
wlo g AR Rlend
wi
Source Image
NAR Bland
AR Blend
(6)
(d)
frubbit catl sitting in a pile of colorful aggs
Figure 9: Illustrations of ablating the Piecewise Linear Smoothing Kernel and Multi-scale Autore- gressive Editing.
Conclusion
We present EditInfinity, parameter-effiiemn adaptation of binary-quantized generative models for text-driven image editing, following the classical 'inversion-editing paradigm. During inversion, we formulate the process as an optimization problem supervised by the exact intermediate quantized representations. During editing, we propose holistic smoothing strategy to blend source and target tokens, preserving unedited regions while aligning with target prompts. Experiments on PIEE-Bench show that EditInfinity outperforms diffusion-basee baselines.
7 Limitation
While our method demonstrates strong performance across diverse editing tasks, it shows limitations in extreme cases such as style change, where background needs be preserved, and the image contains detailed structural patterns. In such cases, the blending between source and target tokens is constrained, which may lead to suboptimal preservation structural fidelity from the original image. Nonetheless, thanks to our image inversion strategy, which effectively learns the generative trajectory of source image, our method can still accomplish intended edits, despite slight structural degradation. In contrast, other methods often fail in such challenging scenarios. For example, as shown in Figure row while other methods are unable to convert the painted bird into a realistic one, our method successfully achieves the style change, with only minor deviation in the bird's head pose.
10

---

8 Acknowledgements
This work was supported in part by the National Natural Science Foundation of China (Grant NO. 62572145, 62176077, 62372133, 62125201 and U24B20174), in part by the Shenzhen Key Technical Project (Grant NO. JCY12002112221222298883 in part by the Guangdong Provincial Key Laboratory of Novel Security Intelligence Technologies (Grant NO. 2022B (02201212010005) and in part by "Guangdong Special Support Plan" (Grant No. 2023TQ07A78).
(AGGG202228310928050000
JSGGG0220881 1056
03006 and
References
[1] Yuval Alaluf, Daniel Garibi, Or Patashnik, Hadar Averbuch-Elor, and Daniel Cohen-Or. Cross- image attention for zero-shot appearance transfer. Proceedings of ACM Special Interest Group on Computer Graphics and Interactive Techniques Conference, pages 1-12, 2024.
[2] Omri Avrahami, Or Patashnik, Ohad Fried, Egor Nemchinov, Kfir Aberman, Dani Lischinski, and Daniel Cohen-Or. Stable flow: Vital layers for training-free image editing. arXiv preprint arXiv:2411
14430, 144300 2024.
[3] Tim Brooks, Aleksander Holynski, and Alexei Efros. Instructpix2pix: Learning to follow image editing instructions. In 1 Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pages 18392-118402 2023.
[4] Mingdeng Cao, Xintao Wang, Zhongang Qi, Ying Shan, Xiaohu Qie, and Yinqiang Zheng. Masactrl: Tuning-free mutual self-attention control for consistent image synthesis and editing. In Proceedings of the IEEE/CVF international conference on computer vision, pages 22560-22570, 2023.
[5] Ricky TQ Chen, Yulia Rubanova, Jesse Bettencourt, and David K Duvenaud. Neural ordinary differential equations. Advances in neural information processing systems, 31, 2018.
[6] Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Yunxuan Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, et al. Scaling instruction-finetuned language models. Journal of Machine Learning Research, 25(70):1-53, 2024.
[7] Jiankang Deng, Jia Guo, Niannan Xue, and Stefanos Zafeiriou. Areface: Additive angular margin loss for deep face recognition. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, June 2019.
[8] Ming Ding, Zhuoyi Yang, Wenyi Hong, Wendi Zheng, Chang Zhou, Da Yin, Junyang Lin, Xu Zou, Zhou Shao, Hongxia Yang, et al. Cogview: Mastering text-to-image generation via transformers. Advances in neural information processing systems, 34: 14:22-2--98855 2021.
[9] Wenkai Dong, Song Xue, Xiaoyue Duan, and Shumin Han. Prompt tuning inversion for text- driven image editing using diffusion models. In Proceedings of the IEEE/CVF International Conference Computer Vision, pages 7430-7440, 2023.
[10] Daniel Garibi, Or Patashnik, Andrey Voynov, Hadar Averbuch-Flor, and Daniel Cohen-Or. Renoise: Real image inversion through iterative noising. In European Conference on Computer Vision, pages 395-413. Springer, 2024.
[11) Zigang Geng, Binxin Yang, Tiankai Hang. Chen Li, Shuyang Gu, Ting Zhang, Jianmin Bao, Zheng Zhang, Houqiang Li, Han Hu, et al. Instructdiffusion: A generalist modeling interface for vision tasks. Proceedings of the IEEE/CVF Conference on computer vision and pattern recognition, pages 12709-12720, 2024.
[12] Dhruba Ghosh, Hannaneh Hajishirzi, and Ludwig Schmidt. Geneval: An object-focused framework for evaluating text-to-image alignment. Advances in Neural Information Processing Systems, 36:52132-52152. 2023.
[13] Google. Gemini 2.0 flash preview image generation. htttps//ccoudd wowddd.goole..o cote verte flash 2024. Accessed: 2025-10-06
x- ai/ htta//enerativeeee httpr/aaiiiieea/cc//////o////

---

[14] Jian Han. Jinlai Liu, Yi Jiang, Bin Yan, Yuqi Zhang, Zehuan Yuan, Bingyue Peng, and Xiaobing Liu. Infinity: Scaling bitwise autoregressive modeling for high-resolution image synthesis. arXiv preprini arXiv:24/2.04431, 2024.
[15] Ligong Han, Song Wen, Qi Chen, Zhixing Zhang, Kunpeng Song, Mengwei Ren, Ruijiang Gao, Anastasis Stathopoulos, Xiaoxiao He, Yuxiao Chen, et al. Improving tuning-free real image editing with proximal guidance. arXiv preprint TrXiv:2306.00414 2023.
[16] Amir Hertz, Ron Mokady, Jay Tenenbaum, Kfir Aberman, Yael Pritch, and Daniel Cohen-Or. Prompt-to-prone image editing with cross attention control. arXiv preprint arXiv:2208.0166 2022.
[17] Edward Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. Lora: Low-rank adaptation of large language models. Interna- tional Conference on Learning Representations, 1(2):3, 2022.
[18] Yuzhou Huang, Liangbin Xie, Xintao Wang, Ziyang Yuan, Xiaodong Cun, Yixiao Ge, Jiantao Zhou, Chao Dong, Rui Huang, Ruimao Zhang, et al. Smartedit: Exploring complex instruction- based image editing with multimodal large language models. In Proceedings of the IEEF/CVF Conference Computer Vision and Patlern Recognition, pages 8362-8371, 2024.
[19] Minyoung Huh, Hossein Mobahi, Richard Zhang, Brian Cheung, Pulkit Agrawal, and Phillip Isola. The low-rank simplicity bias in deep networks. arXiv preprint arXiv:2 10427, 2021.
[20] Xuan Ju, Ailing Zeng, Yuxuan Bian, Shaoteng Liu, and Qiang Xu. Direct inversion: Boosting diffusion-based editing with 3 lines of code. arXiv preprint arXiv:2310.00 10.0/506, 2023.
[21] Bahjat Kawar, Shiran Zada, Oran Lang, Omer Tov, Huiwen Chang, Tali Dekel, Inbar Mosseri, and Michal Irani. Imagic: Text-based real image editing with diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 6007-6017, 2023.
[22] Black Forest Labs. Flux. htt 2024.
[23] Shanglin Li, Bohan Zeng, Yutang Feng, Sicheng Gao, Xiuhui Liu, Jiaming Liu, Lin Li, Xu Tang. Yao Hu, Jianzhuang Liu, al. Zone: Zero-shot instruction-guidd local editing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6254-6263, 2024.
[24] Yaowei Li, Yuxuan Bian, Xuan Ju, Zhaoyang Zhang, Ying Shan, Yuexian Zou, and Qiang Xu. Brushedit: All-in-one image inpainting and editing. arXiv preprint 1oXXi:2442.103/16 2024.
[25] Bingyan Liu, Chengyu Wang. Tingfeng Cao, Kui Jia, and Jun Huang. Towards understanding cross and self-attention in stable diffusion for text-guided image editing. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 7817-7826, 2024.
[26) Shilin Lu, Yanzhu Liu, and Adams Wai-Kin Kong. Tf-icon: Diffusion- training-free cross-domain image composition. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 2294-23900 2305, 2023.
[27] Chenlin Meng, Yutong He, Yang Song, Jiaming Song, Jiajun Wu, Jun-Yan Zhu, and Stefano Ermon. Sdedit: Guided image synthesis and editing with stochastic differential equations. arXiv preprint arXiv:2108.011 01073, 2021.
[28] Daiki Miyake, Akihiro Iohara, Yu Saito, and Toshiyuki Tanaka. Negative- Negative-prompt inversion: Fast image inversion for editing with text-guided diffusion models. In 2025 IEEE/CVF Winter Conference on Applications of Computer Vision, pages 2063-2072. IEEE, 2025.
[29] Ron Mokady, Amir Hertz, Kfir Aberman, Yael Pritch, and Daniel Cohen-Or. Null-text inversion for editing real images using guided diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 6038 6047, 2023.
[30] Yotam Nitzan, Zongze Wu, Richard Zhang, Eli Shechtman, Daniel Cohen-Or, Taesung Park, and Michaël Gharbi. Lazy diffusion transformer for interactive image editing. In European Conference Computer Vision, pages 55-72. Springer, 2024.
12

---

[31] Gaurav Parmar, Krishna Kumar Singh, Richard Zhang, Yijun Li, Jingwan Lu, and Jun- Zhu. Zero-shot image-to-image translation. In Proceedings of ACM Special Interest Group on Computer Graphics and Interactive Techniques Conference, pages 1-11, 2023.
[32] Or Patashnik, Daniel Garibi, Idan Azuri, Hadar Averbuch- Elor, and Daniel Cohen-Or. Localizing object-level shape variations with text-to-image diffusion models. In Proceedings of the IEEE/CVF international conference on computer vision, pages 23051-23061. 2023.
[33] Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion models for high-resolution image synthesis. arXiv preprint arXiv:2307.0023 2023.
[34] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference machine learning, pages 8748-8763. PmLR, 2021.
[35] Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea Voss, Alec Radford, Mark Chen, and Ilya Sutskever. Zero-shot text-to-image generation. In International conference machine learming. pages 8821-8831. Pmlr, 2021.
[36] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684-10695, 2022.
[37] Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfir Aberman. Dreambooth: Fine tuning rext-to-image diffusion models for subject-driven generation. 2022.
[38] Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al. Photorealistic text-to- ext-to-imaae diffusion models with deep language understanding, Advances in neural information processing systems, 35:36479-36494, 2022.
[39] Shelly Sheynin, Adam Polyak, Uriel Singer, Yuval Kirstain, Amit Zohar, Oron Ashual, Devi Parikh, and Yaniv Taigman. Emu edit: Precise image editing via recognition and generation tasks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8871-8879, 2024.
[40] Xincheng Shuai, Henghui Ding, Xingjun Ma, Rongcheng Tu, Yu-Gang Jiang, and Dacheng Tao. A survey of multimodal-guider image editing with text-to-image diffusion models. arXiv preprint arXiv:2406. 14555, 2024.
[41] Kihyuk Sohn, Nataniel Ruiz, Kimin Lee, Daniel Castro Chin, Irina Blok, Huiwen Chang, Jarred Barber, Lu Jiang, Glenn Entis. Yuanzhen Li. et al. Styledrop: Text-to-image generation in any style. arXiv preprint arXiv:2306. 00983, 2023.
[42] Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02500 2020.
[43] Xue Song, Jiequan Cui, Hanwang Zhang, Jingjing Chen, Richang Hong, and Yu-Gang Jiang. Doubly abductive counterfactual inference for text-based image editing. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 2162-91771 2024.
[44] Peize Sun, Yi Jiang, Shoufa Chen, Shilong Zhang, Bingyue Peng, Ping Luo, and Zehuan Yuan. Autoregressive model beats diffusion: Llama for scalable image generation. arXiv preprint 1aXiv:2406.065255 2024.
[45] Haotian Tang, Yecheng Wu, Shang Yang, Enze Xie, Junsong Chen, Junyu Chen, Zhuoyang Zhang, Han Cai, Yao Lu, and Song Han. Hart: Efficient visual generation with hybrid autore- gressive transformer. arXiv preprint arXiv:2410. 108/2, 2024.
[46] Keyu Tian, Yi Jiang, Zehuan Yuan, Bingyue Peng, and Liwei Wang. Visual autoregressive modeling: Scalable image generation via next-scale prediction. Advances neural information processing systems, 1184839-84866 2024.
13

---

[47] Narek Tumanyan, Michal Geyer, Shai Bagon, and Tali Dekel. Plug-and-pin play diffusion features for text-driven saage-to-image translation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1921-1930, 2023.
[48] Dani Valevski, Matan Kalman, Eyal Molad, Eyal Segalis, Yossi Matias, and Yaniv Leviathan. Unitune: Text-driven image editing by fine tuning a diffusion model on single image. ACM Transactions on Graphics, 42(4))1-10, 2023.
[49] Aaron Van den Oord, Nal Kalchbrenner, Lasse Espcholt, Oriol Vinyals, Alex Graves, et al. Con- ditional image generation with pixelenn decoders. Advances in neural information processing systems, 29, 2016.
[50] Aäron Van Den Oord, Nal Kalchbrenner, and Koray Kavukcuoglu. Pixel recurrent neural networks. In International conference on machine learning, pages 1747-1756. PMLR, 2016.
[51] Aaron Van Den Oord, Oriol Vinyals, et al. Neural discrete representation learning. Advances in neural information processing systems, 30, 2017.
[52] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017.
[53] Jiangshan Wang, Junfu Pu, Zhongang Qi, Jiayi Guo, Yue Ma, Nisha Huang, Yuxin Chen, Xiu Li, and Ying Shan. Taming rectified flow for inversion and editing. arXiv preprint arXiv:2411. 1.04446, 2024.
[54] Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero Simoncelli. Image quality assessment: from error visibility structural similarity. IEEE transactions on image processing, (4))):600- 612, 2004.
[55] Jiazheng Xu, Xiao Liu, Yuchen Wu, Yuxuan Tong, Qinkai Li, Ming Ding. Jie Tang, and Yuxiao Dong. Imagereward: learning and evaluating human preferences for text-to-image generation. In Proceedings of the 37th International Conference on Neural Information Processing Systems, pages 15903-15935, 2023.
[56] Xiaofeng Yang, Cheng Chen, Xulei Yang, Fayao Liu, and Guosheng Lin. Text-to- -iatt--imggeee rectified flow as plug-and-play priors, arXiv preprint arXiv:2406. 03293,, 2024.
[57] Jiahui Yu, Yuanzhong Xu, Jing Yu Koh, Thang Luong, Gunjan Baid, Zirui Wang, Vijay Vasudevan, Alexander Ku, Yinfei Yang, Burcu Karagol Ayan, et al. Scaling autoregressive models for content-rich text-to-image generation. arXiv preprint urXiv:2206.10786 2(3):5, 2022.
[58] Kai Zhang, Lingbo Mo, Wenhu Chen, Huan Sun, and Yu Su. Magicbrush: A manually annotated dataset for instruction-guidee image editing. Advances in Neural Information Processing Systems, 36:31428---- 0el:31428-31444 31449, 2023.
[59] Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-imaae diffusion models. In Proceedings of the IEFF/CVF international conference computer vision, pages 3836-3847, 2023.
[60] Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unrea- sonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 586-595, 2018
[61] Yue Zhao, Yuanjun Xiong, and Philipp Krähenbühl. Image and video tokenization with binary spherical quantization. arXiv preprint arXiv:2400.07548, 2024.
[62] Junhao Zhuang, Yanhong Zeng, Wenran Liu, Chun Yuan, and Kai Chen. A task is worth one word: Learning with task prompts for high-quality versatile image inpainting. In European Conference on Computer Vision, pages 195-211. Springer, 2024.
[63] Xianwei Zhuang, Yuxin Xie, Yufan Deng, Dongchao Yang, Liming Liang, Jinghan Ru, Yuguo Yin, and Yuexian Zou. Vargpt-v1. 1. I: Improve visual autoregressive large unified model via iterative instruction tuning and reinforcement learning. arXiv preprint XXi::2504.02949 2025.
14

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The abstract and the introduction clearly state the claims made, including the contributions made the paper and important assumptions and limitations. The abstract and introduction provide concise overview of the research area, the novel approaches introduced, and the key contributions, ensuring that readers have clear understanding of what the paper aims to achieve and the significance of its findings.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: We have discussed the limitations of the work in detail and create a separate "Limitations" section in paper. See Section Discussion for details.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed not penalize honesty concerning limitations.
3. Theory assumptions and proofs
15

---

Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
Justification: All formulas and proofs in the paper are numbered and cross-referenced. The paper provides complete proofs of the formulas, and for those appearing the supplementary material, brief proof sketch is provided in the main text. The proofs in the paper are rigorously reasoned, adhere to accepted mathematical principles, and do not omit any critical steps. The theorems and lemmas relied upon in the proofs are appropriately referenced and cross-referenced. See Section 3 annd for details.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yss]
Justification: We have clearly demonstrated the complete algorithm in the paper using formulas and pseudocode, making easy for readers to reproduce the results in the paper. At the same time, we organizing and preparing to open source our code for readers to use. See Section for details.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
16

---

(c) If the contribution is new model (e.g.. a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: This paper does not currently provide open-access code, but it is planned to be made public in the future.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips..om cc/pu
CoodeSummissioooollt.. /lic//gudees/Codeuubbiisssss
for more details.
While encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
/ ptaaps.cc/pullcc/guuddeo Cooee///dddSSbmiccia ssi onPolicy) for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing much information possible in supplemental material (appended the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Question:
Answer: [Yes]
Justification: We provided detailed explanation of our experimental setup in the main paper, including the learnable prompt and lora training settings, as well the hyperparameter settings during inference. See Section for more details. We will attach more experimental settings in the supplementary materials.
Guidelines:
The answer NA means that the paper does include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can provided either with the code, appendix, or as supplemental material.
7. Experiment statistical significance
17

---

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification: The paper does report error bars suitably and correctly defined, does provide other appropriate information about the statistical significance of the experiments.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the enror bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall
with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation or the standard error of the mean.
It is OK to report I-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: We provide sufficient information on the computer resources (type of compute workers, memory, time exccution) needed reproduce the experiments in subsection 5.] and supplementary materials.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]
Justification: the research conducted our paper, we followed the NeurIPS Code of in all aspects.
sthiecttpps:/neurips..coo http://wwwppppppppit/acccccccaaa
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
18

---

If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: We discuss both potential positive societal impacts and negative societal impacts of the work performed in the supplementary material.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deploymennt of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [Yes]
Justification: We will prevent the misuse of the model by requiring users follow the usage guidelines. See supplementary materials for details.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
19

---

Question: Are the creators or original owners of assets (e.g.. code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: The creators or original owners of assets (e-g.. code, data, models), used the paper, properly credited and the license and terms of use explicitly mentioned and properly respected.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset. For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
assets are released, the license, copyright information, and terms of use the package should be provided. For popular datasets, paperswi thcode. com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: The code and documentation currently not open source. We organizing and preparing to open source the complete code and documentation together for readers to
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given participants and screenshots. if applicable, as well as details about compensation (if any)?
Answer: [Yes]
Justification: Our complete experimental instructions, task interface diagrams, and compen- sation details will be attached in the supplementary materials.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
20

---

According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/teview based on the requirements of your country institution) were obtained?
Answer: [Yes]
Justification: We discussed the potential risks, informed consent process, and ethical review approval of human studies in the supplementary materials.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with
human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, originality the research, declaration is not required.
Answer: [No]
Justification: The manuscript was proofread using LLM, which did not affect the research methodology.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy lhttps://neurij: /tteps///eerrrss..c//oeeeeeeeee http////feeeneee//255555555 for what should or should be described.
21

---

A Supplementary Material
The supplementary material is organized as follows:
Subsection A.1 presents two applications our method: facial attribute editing and complex- scene image editing.
Subsection
shows the supplementary implementation details of EditInfinity.
Subsection
A.3
presents more details of user study.
Subsection
A.4
provides more comprehensive ablation studies.
Subsection
A.5
exhibits additional qualitative results for supplementary.
Subsection
A.6
declares broader impacts of our proposed Editlnfinity.
Subsection
A.7
declares safeguards of our proposed EditInfinity.
Subsection
A.8
states ethical considerations for Editlnfinity.
A.1 Applications.
Facial Attribute Change. To verify that our method generalizes to unmaskable edits, where localized masks impractical, we conduct experiments on facial attribute modification. Specifically, randomly select 20 images from FFHQ to perform unmasked edits, including age, expression, skin tone. Since the setting unmasked, there is background preserve, and thus standard metrics that rely on background consistency are not applicable. In addition to retaining the Whole metric (CLIP score between the entire edited image and the target prompt), introduce ArcFace 77] for evaluating identity preservation, and CLIP-I for measuring similarity between the source and edited images. Table ( shows that our method outperforms strong baselines, including the leading Diffusion UNet model NTI [29] and the Diffusion Transformer model RF-Edit [53]. Thanks to our proposed image inversion algorithm, which effectively learns the generative trajectory of the source image, our method can accomplish the intended edits.
Table 6: Quantitative results on facial images from FFHQ.
Base Model
Method
ArcFacet
CLIP-I0
Wholet
NT
0.56
0.83
23.67
RF-Edit 53
0.61
0.79
23.54
EditInfinity
A
0.63
0.86
24.82
Complex Scene Images Editing. Given that PIE-Benchh already contains nearly 50% natural images, it serves as a comprehensive benchmark for evaluating method open-ended editing tasks. However, further assess performance on complex scenes involving multiple interacting objects, we conduct an additional evaluation on complex scene images editing. We select 20 MagicBrush images (due to time constraints) filtered by GPT-4o, comprising five samples each with 2, 3, 4. and 5 primary objects. Table demonttrasss the superiority of our method in handling complex scenes, compared to the two strong baselines, i.c., NTI 29 and RF-Edit 53
Table 7: Quantitative results on the complex scene images from MagicBrush.
Base Model
Background Preservation
Text Alignment
Method
PSNR
LPIPS 1
MSE
SSIM 1
Wholet
Edited
2T
U
8.81
452.03
1380.48
39.63
19.90
16.83
RF-Edit 53
T
26.00
121.84
33.98
84.73
24.13
18.29
EditInfinity
A
31.23
24.30
9.90
91.70
24.19
20.07
22

---

A.2 Supplementary Implementation Details.
During image inversion, we set the learning rate to 5.6875e-5 and use AdamW optimizer (51 0.9. 82 = 0.97) for both the learnable prompt and LoRA training. The two components are optimized sequentially, starting with the learnable prompt, followed by LoRA. To accelerate the convergence of training LoRA, KL--iiergence loss is introduced in addition to the standard cross-entropy loss. Typically, the learnable prompt is trained for 10 iterations, while LoRA trained for iterations. These settings may be adapted according to the specific editing scenario to optimal performance.
A.3 User Study Details.
Blind Evaluation of Image Editing Results
Plesse select the bed odinggig wait bixid the faloving ciient:
wox acceaddy the target oepeleenallte
Current Progreas:
Deginhingge
Ralis Rogare
Option
Opaes
Opeint
Cylann
Optice
Ploppe
Seed
Sond
Maci
pailigg ai maman
Pooeect callin af menan
Figure 10: Custom web interface of user study.
Each volunteers is asked select the best editing result via custom web interface specifically developed for this evaluation, as shown in Figure 10 The interface presents a source image along with its corresponding prompt, the edited region, a target prompt, and five edited results. The methods behind these results are anonymized and displayed in randomized order for each evaluation.
A.4 More Ablation Study.
Ablation Transformer LoRA. As shown in Table applying LoRA solely FFN layers yields a more favorable trade-off between background preservation and text alignment compared to other configurations. Therefore, we adopt this configuration in final design, enabling effective editing with minimal additional parameter overhead.
Table 8: Ablation on Transformer LoRA. Attn denotes both self attenttoenionn and cross-attention.
Background Preservation
Text Alignment
FFN
Attn
PSNR1
LPIPS
MSE
SSIM
Wholet
Edited
IRx
x
28.50
31.58
22.94
92.36
26.22
23.99
5.39
/
28.50
31.11
23.07
92.32
25.72
23.34
3.93
/
/
28.81
30.31
21.39
92.64
25.61
23.29
4.23
Ablation on Piecewise Linear Smoothing Kernel. Table 2 in the main body only presents better balance results for Gaussian and linear kernels. The full results under varying hyperparameter configurations of Gaussian and linear kernels are provided in Tables[aadd [0] respectively. In the case of Gaussian kernel, increasing enlarges the smooth transition zone, compromising background retention while improving text alignment. In the case of the linear kemne,, when fixing Tı and gradually increasing 72, transition zone width of the linear kernel (T2 - TI) increases accordingly. This leads to improved text alignment metrics but at the cost of degraded background preservation performance. Conversely, when 72 fixed and Ti increases, both text alignment and background preservation tend to deteriorate. These observations indicate that T and 72 play critical role in balancing edit fidelity
23

---

and content preservation. Overall, the linear kernel setting of and = 4 offers better trade-off, achieving strong text alignment (e.g.. 1 = 5.39) while keeping background distortion (e.g., LPIPS = 31.58) within acceptable limits.
Table 9: Quantitative results of ablating the Gaussian kernel.
Background Preservation
Text Alignment
PSNR
LPIPS
MSE
SSIM
Wholet
Editedf
1
1
29.40
28.59
18,45
93.01
26.09
23.79
4.74
2
28.63
31.16
21.71
92.45
26.08
23.71
4.66
3
28.15
32.91
24.40
92.17
26.10
23.81
4.61
4
28.08
33.05
25.00
92.25
26.23
23.73
4.91
Table 10: Quantitative results of ablating the linear kernel.
Background Preservation
Text Alignment
T1
72
PSNRT
LPIPS | 1SE |
SSIM 102
Wholet
Edited
IRiot
0
29.19
29.15
19.47
92.87
26.13
23.74
4.83
0
29.16
29.23
19.48
92.86
26.11
23.86
5.04
0
28.92
29.87
20.35
92.79
26.24
23.85
5.00
0
28.69
30.60
21.38
92.66
26.19
24.01
4.79
0
28.45
31.68
22.86
92.51
26.20
23.80
4.71
28.80
30.21
22.34
92.65
26.07
23.71
4.72
28.46
31.75
22.54
92.35
26.13
23.83
4.85
28.50
31.58
22.94
92.36
26.22
23,99
5.39
28.41
31.74
22.98
92.36
26.14
23.83
4.73
2 28.50
31.44
22.46
92.50
26.18
23.53
4.96
2 28.14
32.69
24.47
92.31
26.17
23.72
4.91
2
27.53
36.17
29.65
91.79
26.21
23.74
5.12
Table 11: Quantitative results of ablating the mask. EditInfinity-u denotes user-provided masks; EditInfinity-c denotes cross-attention masks. Best and second-best results are shown in bold and italics.
Base Model
Background Preservation
Text Alignment
Method
PSNR1
LPIPS|00_
SSIM MSE
Wholet Edited
IR
18.81
197.11
197.69
73.68
25.10
22.98
0.29
Masa
23.36
95.45
77.63
81.88
23.30
20.92
-3.82
P2P-Zero
20.92
161.28
137.64
77.02
22.89
21.09
-5.71
NTI 29
28.08
57.94
36.10
85.17
24.71
22.51
3.63
PnP-Inv 20
23.60
103.12
72.77
81.11
25.05
22.94
3.34
NP(88)
27.24
62.40
37.79
84.92
24.89
22.67
2.92
StableFlow
23.68
72.77
78.61
88.11
23.17
21.21
0.76
RF-Edit(53
T
27.26
92.27
34.46
86.67
24.65
22.03
0.61
Editlnfinity-c
27.47
44.97
46.91
90.30
25.71
23.22
5.40
EditInfinity-u
28.50
31.58
22.94
92.36
26.22
23.99
5.39
Ablation on Mask. Our method assumes the user provides masks. Indeed, this a well-established task setting in image editing [37] 59]. especially when text alone is insufficient for the precise user-desired editing region. This challenge accurately conveying user intent has
localization
long been recognized in controllable image generation. To enhance controllability. ControlNet [59] leverages visual priors such as edge maps, while DreamBooth 37 utilizes user-provided images to capture detailed features not easily conveyed by text.

---

While our method assumes user-provided masks by default, it can also leverage Infinity's cross- attention maps []6] for automatic mask generation without modifying the framework. Table [T] reports comprehensive comparisons and shows that our method is not highly sensitive to the source of the mask- -ttonn performance achieved in both cases.
A Ablation on Multi-scale Autoregressive Editing, By blending source tokens at each scale in an autoregressive (AR) manner, our method provides continuous guidance for editing region generation at subsequent scales. In contrast, the non-autoregreessive (NAR) approach blends source tokens only the end of each scale, without influencing the token generation process at the next scale. This leads to incoherent transitions and visually inconsistent edits, as illustrated in Figure I1 Table T2 further supports this observation: AR consistently outperforms NAR in both background preservation and text alignment. These results highlight the necessity of autoregressive guidance for achieving harmonious and realistic edits.
Table 12: Quantitative results of multi-scale autoregressive editing.
Background Preservation
Text Alignment
Blend
PSNRT
LPIPS MSE
SSIM
Wholet
Edited
IRoo
NAR
25.50
42.59
38.39
91.00 92.36
25.98 26.22
23.64
3.54
AR
28.50
31.58
22.94
23.99
5.39
Source Image
NAR
AR
caz (*with kar)
plate (stenk salmon) on it
momalain inndscupe with (flowrrs wathered grass
Figure 11: Illustrations ablating the Multi-scale Autoregressive Editing.
A.5 Additional Qualitative Results.
We present additional qualitative results to further demonstrate the effectiveness of our method, as shown in Figure T2 and 13 These results include diverse editing types across add object, change object, delete object, change content, change pose, change color, change background, change material, and change style. We also provide comparisons with state-of-the-art methods, highlighting our model's ability to preserve background details and align with target prompts in image editing.
A.6 Broader Impacts
Our proposed method enables high-quality image editing. Positive societal impacts include its potential applications education (e.g., visual content adaptation learning) and creative industries (e.g., graphic design and media production). However, potential negative societal impacts include misuse for deceptive content creation (e.g., deepfakes or misinformation). We acknowledge the dual-use nature of image generation technologies and emphasize responsible deployment.
25

---

A.7 Safeguards
To mitigate risks associated with misuse, we adopt the following safeguards:
We will release the model under research-use-only license.
Model checkpoints and code will include usage agreement that prohibits harmful or deceptive use cases (e.g.. unauthorized alteration of real people's images).
All datasets used for editing are publicly available and contain no private or personally identifiable information.
A.8 Ethical Considerations
There is no potential risks incurred by study participants in this paper. As such, Institutional review board (IRB) approval was not required.
26

---

Add Object
Change Object
Delete Object
PP
Source Image
Editinfinity (Ours)
P2P
MasaCtrl
P2P-Zero
NTI
SPivE
PnP-Inv
Stablc-Flow
RF-Edit
reading
with rock
book
fowwer--n--had two-beats-are
book
on the ground
panda
painting
dockeed-on-sheee
Figure 12: Qualitative results PIE-Bench across add, change, and delete object. The red mask denotes the edited region M, expected to follow the prompt, while other regions retain the background.
27

---

Change Content
Change Pose
Change Color
Change Background
Change Material
Change Style
Source Image
EditInfinity
(Ours)
P2P
MasaCtrl
P2P-Zero
NTI
PnP-Inv
NP
Stable-Flow
RF-Edit
looking left side looking af the camera
purple
office
close eyver
Wolter
CarToon WORLD
Figure 13: Qualitative results PIE-Bench across change content, change pose, change color, change background, change material, and change style. The red mask denotes the edited region M, expected to follow the prompt, while other regions retain background.
28


Reviews:
[{"review_id": "3Y85gGeWDh", "reviewer": "Reviewer_4PS6", "summary": "This paper proposes a method to adapt a type of pretrained text conditioned image generative models to the task of image editing. The model in question is Infinity, an autoregressive model on binary residual quantized tokens in the image scale space. The proposed method first conduct text inversion with learnable language style tokens and then performs editing with soft masking in the generation trajectory. \n\nThe proposed method is tested on PIE benchmark with add, delete, and change operations. It shows better image editing capability than other training-free baselines.", "questions": "I would like to encourage the authors provide more detailed explanation on the optimization of t_l and the blending of quantized tokens. \n\nAlso, I would like to see clarification on the experimental results. Are other baseline methods also using Inifinity as the base model? If not could the worse results for these methods be due to their worse base models?", "limitations": "Yes", "rating": 4, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths:\n+ The idea of using the Infinity model, which provides exact decomposition of images, as a trajectory to reference seems interesting. The proposed masking technique appears to provide good background preservation. \n+ Based on the provided visual results, the proposed method can accurately achieve addition, deletion, and change.\n+ The cost of adapting the pretrained text to image model is minimal thanks to the use of LoRA. \n\nWeakness\n- It is unclear how the text style token t_l is optimized or learned. \n- The blending operation is not described clearly. Since the Infinity model uses quantized tokens, it is unclear how linear blending happens between two token indexes. \nThe method seems tailored to the infinity model; it is unclear whether it can be generalized to other types of image generative models.", "was_revised": true, "final_justification": "The authors have addressed my concerns in the response. The method described with the added context seems reasonable and is achieving good editing quality. I am recommending accept.", "rebuttal": "Thanks for your comments. We will explain your concerns point by point.\n\n**Main Comment**\n\n> **Q1: I would like to encourage the authors provide more detailed explanation on the optimization of t_l and the blending of quantized tokens.**\n\n**A1:** We appreciate the reviewer’s helpful suggestion. We now explain in detail (i) how our method optimizes $t_l$, and (ii) how the blending of quantized tokens is performed. We have also refined the narrative in the main paper to make these components clearer and better integrated.\n\n**(a) The optimization of $t_l$:**\n\nFirst, we introduce learnable prompt embeddings $t_l$, whose size is fixed to 20 tokens. Second, we pass both the source prompt $t_{sou}$ and the instruction prompt $t_{inv}$ (i.e., \"the language style of this prompt is\") through text encoder of Infinity $\\Psi(\\cdot)$ to obtain text embeddings $\\Psi(t\\_{sou}, t\\_{ins})$. We then concatenate those embeddings with $t_l$ to form the textual conditioning input for Infinity. Finally, while keeping Infinity's weights frozen, we optimize only $t_l$ using cross-entropy loss, where the supervision signal comes from tokens $R_{1...K}^{sou}$ derived from the source image. \n\n**(b) The blending of quantized tokens:**\n\n(i) **The blending operation is performed on the quantized features $(\\\\{R_{k}\\\\}_{k=1}^K, R_k \\in  \\mathbb{R}^{h_k \\times w_k \\times d})$ rather than on indices**, which is formulated as $E_k^{tar} = \\mathrm{Upsample}(R^{tar}_k, (h_K,w_K)) \\odot (1-\\mathcal{G}) + \\mathrm{Upsample}(R^{sou}_k, (h_K,w_K)) \\odot \\mathcal{G}$. \n\nAt each step $k$ of the autoregressive generation, the target token $R_k^{tar}$ is generated conditioned on the concatenation of the target prompt embedding and instruction embedding  $\\Psi(t\\_{tar},t\\_{ins})$, and optimized $t_l$. It is then blended with source tokens $R_k^{sou}$ of the source image under the guidance of the piecewise linear smoothing kernel $\\mathcal{G}$. Although this formulation is already provided in Algorithm 1 of the main paper, we appreciate the reviewer’s suggestion and will enhance the textual explanation in the final version to improve clarity and emphasis.\n\n(ii) The feasibility of performing linear blending at the quantized feature level is grounded in the fact that the Infinity decoder takes quantized features as input to generate images. As such, performing interpolation in the feature space is both reasonable and commonly adopted in generative modeling [1-3], as it supports smooth and semantically meaningful transitions.\n\n[1] Alec Radford, et al. Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks[C]//ICLR. 2016.\n\n[2] Kingma D P, Welling M. Auto-encoding variational bayes[C]//ICLR. 2014.\n\n[3] Zhu J Y, et al. Toward multimodal image-to-image translation[C]//NeurIPS. 2017.\n\n\n> **Q2: Also, I would like to see clarification on the experimental results. Are other baseline methods also using Inifinity as the base model? If not could the worse results for these methods be due to their worse base models?**\n\n**A2:** This is an insightful question. Our method is a lightweight framework tailored to Infinity and indeed relies on the generative capacity of the base model. However, this dependency is not unique to our approach. We explicitly present base models used by all methods in Table 1 below and evaluate their generative performance on the GenEval benchmark [4], one of the most widely used benchmarks for assessing text-to-image generation quality. As shown in Table 2, **Infinity achieves comparable overall generative performance to the recently popular FLUX** [5], and even underperforms in certain tasks, such as two-object and counting. Nevertheless, **our Infinity-based image editing method significantly outperforms FLUX-based approaches** like StableFlow and RF-Edit, demonstrating the effectiveness of our method  spite the base model not having a clear advantage.\n\n**Table 1: Quantitative results on PIE-Bench.**\n|Method| Base Model | PSNR↑ | LPIPS(×10³)↓ | MSE(×10⁴)↓ | SSIM(×10²)↑ | Whole↑ | Edited↑ | IR(×10)↑ |\n|-|-|-|-|-|-|-|-|-|\n| P2P | Stable Diffusion v1.4 | 17.87| 208.80| 219.88| 71.14| 25.01| 22.44| 0.017|\n| MasaCtrl| Stable Diffusion v1.4 | 22.17| 106.62| 86.97| 79.67 | 23.96| 21.16| -1.66|\n| P2P-Zero| Stable Diffusion v1.4 | 20.44 | 172.22| 144.12| 74.67 | 22.80| 20.54| -6.59|\n| NTI| Stable Diffusion v1.4 | 27.03| 60.67| 35.86| 84.11   | 24.75 | 21.86| 2.77|\n| PnP-Inv| Stable Diffusion v1.5 | 22.46| 106.06 | 80.45  | 79.68 | 25.41| 22.62| 4.17|\n| NP| Stable Diffusion v1.4 | 26.21| 69.01| 39.73 | 83.40 | 24.61| 21.87 | 2.42|\n| StableFlow| FLUX.1-dev | 21.64| 92.28  | 115.21| 84.94| 24.65| 21.70| 1.88|\n| RF-Edit| FLUX.1-dev | 23.22| 131.18| 75.00 | 81.44  | 25.22| 22.40| 5.18 |\n| **EditInfinity (Ours)** | Infinity | **27.95** | **33.08**| **24.27** | **92.12** | **26.41** | **23.47**| **5.88**  |\n\n**Table 2: Evaluation on the GenEval benchmark.**\n\n|Base Model|Overall|Single Object |Two Object | Counting| Colors | Position | Attribute Binding |\n|-|-|-|-|-|-|-|-|\n| Stable Diffusion v1.4 |0.42 |0.97| 0.39|0.33|0.73|0.03|0.05|\n| Stable Diffusion v1.5 |0.43|0.97|0.38|0.35|0.76|0.04|0.06|\n|**FLUX.1-dev**|**0.66**|**0.98**|**0.81**|**0.74**|0.79|0.22|0.45|\n|**Infinity**|**0.66**|**0.98**|0.78|0.63|**0.83**|**0.25**|**0.53**|\n\nNote: When evaluating Infinity, we adopt the same evaluation protocol as used for Stable Diffusion v1.4, v1.5, and FLUX.1-dev, i.e., without prompt rewriting.\n\n[4] Dhruba Ghosh, et al. Geneval: An object-focused framework for evaluating text-to-image alignment[C]//NeurIPS. 2024.\n\n[5] Black Forest Labs. Flux. 2024."}, {"review_id": "HwRcZKg1DM", "reviewer": "Reviewer_LWWe", "summary": "This paper introduces EditInfinity, which is a text-based image editing method based on the autoregressive text-to-image generation model Infinity, which succeeds in high quality generation among vector quantized models available. The authors motivate the paper from the existing inversion-then-editing approach, where instead of inverting into a noisy latent (like in diffusion models), they initially invert the input text prompt by an extended embedding representation, which is followed by LoRA based adaptation for the reconstruction of precise details. Given a target text prompt, an input image and a region of interest specified by a mask, EditInfinity performs editing by manipulating the multi-scale quantized feature maps with a smoothed mask $\\mathcal{G}$, then reconstruct the image using these manipulated feature maps. Authors demonstrate the effectiveness of the approach by using PIE-Bench, where they compare their approach with diffusion/flow-matching based editing methods. The provided quantitative analyses and qualitative results shows the effectiveness of the method.", "questions": "- How are the masks used in the editing process obtained in the evaluations? Does the method assume that these masks are provided by the user or is there an automated pipeline to achieve this?\n- Is the method able to achieve disentanglement for un-maskable edits such as facial attribute change?\n- What are the failure cases of the method in detail?\n- Is there are timing constraint on the method, as a two stage optimization is required for editing?", "limitations": "- The limitations are very briefly addresses over a small discussion. I believe that further details should be included like failure cases as discussed over the *Questions* and *Strengths And Weaknesses* sections. In addition the authors are encouraged to include a timing based comparison due to the two stage optimization that the method requires (for each editing example). For further comments on suggested improvements over the limitations, please refer to *Questions* and *Strengths And Weaknesses* sections.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 4, "originality": 3, "strengths_and_weaknesses": "**Strengths**\n- The paper tackles the image editing problem using autoregressive image editing methods with a new inversion scheme which mainly depends on approximating the ideal generation prompt, and minor adjustments on the autoregressive model with LoRA-based optimization.\n- The proposed method achieves succeeds in background preservation compared to existing approaches\n- The editing pipeline proposed provides satisfactory results on edits that can be constrained with a region identified by a mask.\n- Authors provide ablations for the design decisions made for each of the components of the method.\n\n**Weaknesses**\n- The limitations of the method is discussed very briefly, where the authors should provide more qualitative examples for the failure cases.\n- The edits that can be performed with the proposed approach seems limited with an input mask, where providing the whole image as the editing region of interest seems to be disturbing the background preservation properties and the preservation of the input pose, structure (see Fig. 6, row 9). If there is such limitation, the authors should explicitly discuss this for transparency.\n- There are missing details in the quantitative evaluation setup and method details. For the details on the method, the proposed approach involves two-stage training per image. To fairly assess if this approach is feasible as an editing method or not, the authors should provide a time comparison with the competing approach in terms of the overhead on top of the base model. In addition, the evaluation requires an input mask for EditInfinity, where the method used to obtain these masks are not described in detail. The authors should privide such details on the evaluation setup.\n- Given the short description about the limitations from the authors, the proposed method succeeds in masked edits. While this is a valid use case for editing, it does not cover a majority of the edits to be performed. As an example, editing facial attributes is nor always maskable (e.g. adding smile). It is not clear if the method can perform such edits or not, if it can there should be examples on such detailed cases. In addition, the editing examples provided are rather simplistic cases where there is a single subject that is centered in the image. This makes the performance in on-the-wild images questionable. The authors should provide examples for such use cases.", "was_revised": true, "final_justification": "The rebuttal addressed my questions regarding the mask sensitivity, identity preservation, performance on \"on-the-wild\" images and evaluation details. After reviewing the author rebuttal and the other reviews, I am leaning towards positive.", "rebuttal": "Thanks for your careful and valuable comments. We will explain your concerns point by point.\n\n**Main Comment**\n\n> **Q1: How are the masks used in the editing process obtained in the evaluations? Does the method assume that these masks are provided by the user or is there an automated pipeline to achieve this?**\n\n**A1:** We appreciate this insightful question. Our method assumes the user provides masks. Indeed, this is a well‑established task setting in image editing [1] [2], especially when text alone is insufficient for the precise localization of the user-desired editing region. This challenge of accurately conveying user intent has long been recognized in controllable image generation. To enhance controllability, ControlNet [3] leverages visual priors such as edge maps, while DreamBooth [4] utilizes user-provided images to capture detailed features not easily conveyed by text.\n\nWhile our method assumes user-provided masks by default, it can also leverage Infinity’s cross-attention maps [5] for automatic mask generation without modifying the framework. Specifically, we automatically align differing words $x$ between the source and target prompts. After completing inversion, we input the source or target prompt containing $x$ into Infinity and extract the cross-attention map corresponding to $x$. A threshold is then applied: values above the threshold are set to 0 (mask foreground), and others to 1 (background). Table 1 shows that our method is not highly sensitive to the source of the mask—strong performance is achieved in both cases. We will include further discussion on this aspect in the final version.\n\n**Table 1: Quantitative results on the random class of PIE-Bench.** Ours-u denotes user-provided masks; Ours-c denotes cross-attention masks. Best and second-best results are shown in **bold** and *italics*.\n|Method|PSNR↑|LPIPS(×10³)↓|MSE(×10⁴)↓|SSIM(×10²)↑|Whole↑|Edited↑|IR(×10)↑|\n|-|-|-|-|-|-|-|-|\n|P2P|18.81|197.11|197.69|73.68 |25.10|22.98|0.29|\n|MasaCtrl|23.36|95.45|77.63|81.88|23.30|20.92|-3.82|\n|P2P-Zero|20.92|161.28|137.64|77.02|22.89|21.09|-5.71|\n|NTI|*28.08*|57.94|36.10|85.17|24.71|22.51|3.63|\n|PnP-Inv|23.60|103.12|72.77|81.11|25.05|22.94|3.34|\n|NP|27.24|62.40|37.79| 84.92|24.89|22.67|2.92|\n|StableFlow|23.68|72.77|78.61|88.11|23.17|21.21|0.76|\n|RF-Edit|27.26| 92.27| *34.46*|86.67| 24.65| 22.03|0.61|\n|*Ours-c*|27.47|*44.97*|46.91|*90.30*|*25.71*|*23.22*|**5.40**|\n|**Ours-u**|**28.50**|**31.58**|**22.94**|**92.36**|**26.22**|**23.99**|*5.39*|\n\nNote: Due to time constraints, experiments are conducted on the “random class” of PIE-Bench, which includes all editing types and supports efficient and unbiased evaluation of model components.\n\n[1] Nitzan Y, et al. Lazy diffusion transformer for interactive image editing[C]//ECCV. 2024.\n\n[2] Zhuang J, et al. A task is worth one word: Learning with task prompts for high-quality versatile image inpainting[C]//ECCV. 2024.\n\n[3] Zhang L, et al. Adding conditional control to text-to-image diffusion models[C]//ICCV. 2023.\n\n[4] Ruiz N, Li Y, et al. Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation[C]//CVPR. 2023.\n\n[5] Amir Hertz, et al. Prompt-to-Prompt Image Editing with Cross-Attention Control[C]//ICLR. 2023.\n\n> **Q2: Is the method able to achieve disentanglement for un-maskable edits such as facial attribute change?**\n\n**A2:** We thank the reviewer for this insightful question.\n\n**(a) Facial attribute change.** Yes, our method can realize facial attribute change. **PIE-Bench**, used for evaluation in main paper, is a comprehensive image editing benchmark covering natural human-centric scenarios. We use GPT-4o to filter editing instructions, resulting in 16 representative examples such as **'change the expression from laughing to angry'** and 'add the age of the woman'. Our method performs favorably on these cases. Please refer to Table 1 in the main paper for qualitative results, given image upload constraints.\n```\n|-- random\n|-- change_object\n     |-- natural*\n          |-- animal\n          |-- human*\n          |-- indoor\n          |-- outdoor\n     |-- artificial\n          |-- ...\n|-- add_object\n     |-- ...\n|-- delete_object\n     |-- ...\n|-- change_attribute_content\n     |-- ...\n|-- change_attribute_pose\n     |-- ...\n|-- change_attribute_color\n     |-- ...\n|-- change_attribute_material\n     |-- ...\n|-- change_background\n     |-- ...\n|-- change_style\n     |-- ...\n```\nFurthermore, we randomly select 20 face images from **FFHQ** (due to time constraints) to perform unmasked facial attribute changes, including **age, expression, skin tone**, and more. Since the setting is unmasked, there is no background to preserve, and thus standard metrics that rely on background consistency are not applicable. In addition to retaining the Whole metric (CLIP score between the entire edited image and the target prompt), we introduce ArcFace [6] for evaluating identity preservation, and CLIP-I for measuring similarity between the source and edited images. The results are reported in Table 2 below, which shows that our method achieves the best performance compared to others. Thanks to our proposed image inversion algorithm, which effectively learns the generative trajectory of the source image, our method can accomplish the intended edits.\n\n**Table 2: Quantitative results on facial images from FFHQ.** \n|Method|ArcFace↑|CLIP-I↑|Whole↑|\n|-|-|-|-|\n|NTI|0.56|0.83| 23.67 |\n|RF-Edit|0.61|0.79| 23.54  | \n|**Ours**|**0.63**|**0.86**|**24.82**| \n\nNote: Due to time constraints, we select representative methods with strong performance from both diffusion U-Net-based models and diffusion transformer-based models for comparison.\n\n[6] Jiankang Deng, et al. ArcFace: Additive Angular Margin Loss for Deep Face Recognition[C]//CVPR. 2019.\n\n\n**(b)  On-the-wild images.** Given that PIE-Bench contains nearly 50% natural images, it serves as a comprehensive benchmark for assessing our method. However, to specifically evaluate complex scenes, we select 20 MagicBrush images (due to time constraints) filtered by GPT-4o, comprising five samples each with **2, 3, 4, and 5 primary objects**. Table 3 demonstrates the superiority of our method in handling complex scenes.\n\n**Table 3: Quantitative results on the complex scene images from MagicBrush .** \n|Method|PSNR↑|LPIPS(×10³)↓|MSE(×10⁴)↓|SSIM(×10²)↑|Whole↑|Edited↑| \n|-|-|-|-|-|-|-|\n|NTI|8.81|452.03|1380.48|39.63|19.90|16.83| \n|RF-Edit|26.00|121.84|33.98|84.73|24.13|18.29|\n|**Ours**|**31.23**|**24.30**|**9.90**|**91.70**|**24.19**|**20.07**|\n\n> **Q3: What are the failure cases of the method in detail?**\n\n**A3:** We appreciate your valuable suggestion. In the final version, we will provide additional failure cases and elaborate on the limitation as follows.\n\n**Limitation.** While our method demonstrates strong performance across diverse editing tasks, it shows limitations in extreme cases such as style change, where no background needs to be preserved, and the image contains detailed structural patterns. In such cases, the blending between source and target tokens is constrained, which may lead to suboptimal preservation of structural fidelity from the original image. Nonetheless, thanks to our image inversion strategy, which effectively learns the generative trajectory of source image, our method can still accomplish intended edits, despite slight structural degradation. In contrast, other methods often fail in such challenging scenarios. For example, as shown in Figure 6, row 9 of the main paper, while other methods are unable to convert the painted bird into a realistic one, our method successfully achieves the style change, with only minor deviation in the bird's head pose.\n\n> **Q4: Is there are timing constraint on the method, as a two stage optimization is required for editing?**\n\n**A4:** We conduct a runtime comparison of our method and other methods on a single NVIDIA L20 GPU, measuring both inversion and editing time, as shown in Table 4 below. A key advantage lies in its efficiency during multiple edits on one image—a common real-world scenario. Once the inversion for a given image is completed, subsequent edits can be performed within **3.64 seconds**—over **7× faster** than other methods on average (initial inversion time only 4× longer than other methods on average). This design effectively front-loads the computational cost.\n\n**Table 4: Runtime comparison of inversion and editing.** \n|Method|Inversion|Once Editing|Twice Editing|\n|-|-|-|-|\n|P2P|14.40s|10.28s|10.28s|\n|MasaCtrl|**5.19s**|17.45s|17.45s|\n|P2P-Zero|13.31s|62.29s|62.29s|\n|NTI|95.54s|10.32s|10.32s|\n|PnP-Inv|8.32s|9.54s|9.54s|\n|NP|9.00s|10.37s|10.37s|\n|StableFlow|13.85s|27.20s|27.20s|\n|RF-Edit|55.48s|54.07s|54.07s|\n|**Ours**|107.06s|**3.64s**|**3.64s**|"}, {"review_id": "fSPbZMdWyU", "reviewer": "Reviewer_Ghm8", "summary": "**EditInfinity** introduces a novel text-driven image editing method that leverages binary-quantized generative models as an alternative to traditional diffusion-based approaches. By utilizing discrete token representations, the method enables precise image inversion with exact supervision and avoids the approximation errors typical of diffusion inversion. The authors enhance this process through a learnable prompt mechanism and LoRA-based style preservation. For the editing phase, they propose a holistic smoothing strategy that blends source and target tokens using a piecewise linear kernel, allowing for seamless and localized modifications. Evaluated on the PIE-Bench benchmark, EditInfinity achieves state-of-the-art results in both background preservation and semantic alignment, significantly outperforming diffusion-based baselines across a variety of editing tasks.", "questions": "1. How well does the method handle vague or high-level prompts that lack explicit object references or structure?\n2. The inversion and editing stages seem computationally demanding. What is the latency for a single edit?\n3. How is the edited region determined in practical scenarios? Can the method infer edit regions automatically based on differences between source and target prompts, or does it rely entirely on manual mask annotation?", "limitations": "Please refer to the weaknesses section.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 2, "strengths_and_weaknesses": "### Strengths\n\n1. **Clear Motivation and Novelty**: The paper addresses limitations of diffusion-based editing, especially inversion error, and innovatively adapts a VQ-based model, which is a less explored direction.\n2. **Thorough Evaluation**: The authors conduct extensive quantitative and qualitative experiments, including a user study, to demonstrate improvements across various editing tasks. EditInfinity outperforms existing methods in key metrics, especially in preserving unedited regions while aligning edits with prompts.\n\n### Weaknesses\n\n1. **Structured Editing Setup**: The method heavily depends on well-aligned source-target prompt pairs and user-provided region masks. This reliance on structured inputs may limit its generalizability to open-ended or ambiguous editing tasks where such precise supervision is unavailable.\n2. **Computation Cost**: The framework involves several non-trivial steps, including prompt optimization, LoRA fine-tuning, and multi-scale autoregressive token generation. This complexity could hinder real-time applications and make the method less practical.\n3. **Manual Design**: The framework includes several manually crafted components, such as user-defined editing masks and a fixed smoothing kernel. These elements introduce human priors into the pipeline, and it remains unclear how robust the approach is to noisy inputs or complex, less structured scenes.", "was_revised": true, "final_justification": "Thank authors for the clarification and effort, and I have decided to increase my rating. But the method can be further improved with more flexible setting and better efficiency.", "rebuttal": "We sincerely appreciate your thoughtful and constructive feedback. Below, we respond to each of your comments in detail and have incorporated the corresponding revisions into the final version of the paper.\n\n**Main Comment**\n\n> **Q1: How well does the method handle vague or high-level prompts that lack explicit object references or structure?**\n\n**A1:** We sincerely thank you for highlighting this concern. We provide a detailed response from the following two perspectives: ’Prompt Understanding‘ and ’Open-ended Editing Tasks‘.\n\n**(a) Prompt Understanding.** Our method is designed to operate with known editing regions and does not focus on prompt understanding. In scenarios involving vague or high-level prompts without clear object references, external multimodal large language models (MLLMs) may be integrated to enhance prompt understanding.\n\n**(b) Open-ended Editing Tasks.** Given that PIE-Bench contains nearly 50% natural images, it already serves as a comprehensive benchmark for assessing our method. However, to specifically evaluate complex scenes, we select 20 MagicBrush images (due to time constraints) filtered by GPT-4o, comprising five samples each with 2, 3, 4, and 5 primary objects. Table 1 demonstrates the superiority of our method in handling complex scenes.\n\n**Table 1: Quantitative results on the complex scene images from MagicBrush .** \n|Method|PSNR↑|LPIPS(×10³)↓|MSE(×10⁴)↓|SSIM(×10²)↑|Whole↑|Edited↑| \n|-|-|-|-|-|-|-|\n|NTI|8.81|452.03|1380.48|39.63|19.90|16.83| \n|RF-Edit|26.00|121.84|33.98|84.73|24.13|18.29|\n|**Ours**|**31.23**|**24.30**|**9.90**|**91.70**|**24.19**|**20.07**|\n\nNote: Due to time constraints, we select representative methods with strong performance from both diffusion U-Net-based models and diffusion transformer-based models for comparison.\n\n> **Q2: The inversion and editing stages seem computationally demanding. What is the latency for a single edit?**\n\n**A2:** We conduct a runtime comparison of our method and other methods on a single NVIDIA L20 GPU, measuring both inversion and editing time, as shown in Table 2 below. A key advantage lies in its efficiency during multiple edits on one image—a common real-world scenario. Once the inversion for a given image is completed, subsequent edits can be performed within **3.64 seconds**—over **7× faster** than other methods on average (initial inversion time only 4× longer than other methods on average). This design effectively front-loads the computational cost.\n\n\n**Table 2: Runtime comparison of inversion and editing.** \n|Method|Inversion|Once Editing|Twice Editing|\n|-|-|-|-|\n|P2P|14.40s|10.28s|10.28s|\n|MasaCtrl|**5.19s**|17.45s|17.45s|\n|P2P-Zero|13.31s|62.29s|62.29s|\n|NTI|95.54s|10.32s|10.32s|\n|PnP-Inv|8.32s|9.54s|9.54s|\n|NP|9.00s|10.37s|10.37s|\n|StableFlow|13.85s|27.20s|27.20s|\n|RF-Edit|55.48s|54.07s|54.07s|\n|**Ours**|107.06s|**3.64s**|**3.64s**|\n\n> **Q3: How is the edited region determined in practical scenarios? Can the method infer edit regions automatically based on differences between source and target prompts, or does it rely entirely on manual mask annotation?**\n\n**A3:** We appreciate this insightful question. Our method assumes the user provides masks. Indeed, this is a well‑established task setting in image editing [1] [2], especially when text alone is insufficient for the precise localization of the user-desired editing region. This challenge of accurately conveying user intent has long been recognized in controllable image generation. To enhance controllability, ControlNet [3] leverages visual priors such as edge maps, while DreamBooth [4] utilizes user-provided images to capture detailed features not easily conveyed by text.\n\nWhile our method assumes user-provided masks by default, it can also leverage Infinity’s cross-attention maps [5] for **automatic mask generation** without modifying the framework. Specifically, we automatically align differing words $x$ between the source and target prompts. After completing inversion, we input the source or target prompt containing $x$ into Infinity and extract the cross-attention map corresponding to $x$. A threshold is then applied: values above the threshold are set to 0 (mask foreground), and others to 1 (background). Table 3 shows that our method is not highly sensitive to the source of the mask—strong performance is achieved in both cases. We will include further discussion on this aspect in the final version.\n\n**Table 3: Quantitative results on the random class of PIE-Bench.** Ours-u denotes user-provided masks; Ours-c denotes cross-attention masks. Best and second-best results are shown in **bold** and *italics*.\n|Method|PSNR↑|LPIPS(×10³)↓|MSE(×10⁴)↓|SSIM(×10²)↑|Whole↑|Edited↑|IR(×10)↑|\n|-|-|-|-|-|-|-|-|\n|P2P|18.81|197.11|197.69|73.68 |25.10|22.98|0.29|\n|MasaCtrl|23.36|95.45|77.63|81.88|23.30|20.92|-3.82|\n|P2P-Zero|20.92|161.28|137.64|77.02|22.89|21.09|-5.71|\n|NTI|*28.08*|57.94|36.10|85.17|24.71|22.51|3.63|\n|PnP-Inv|23.60|103.12|72.77|81.11|25.05|22.94|3.34|\n|NP|27.24|62.40|37.79| 84.92|24.89|22.67|2.92|\n|StableFlow|23.68|72.77|78.61|88.11|23.17|21.21|0.76|\n|RF-Edit|27.26| 92.27| *34.46*|86.67| 24.65| 22.03|0.61|\n|*Ours-c*|27.47|*44.97*|46.91|*90.30*|*25.71*|*23.22*|**5.40**|\n|**Ours-u**|**28.50**|**31.58**|**22.94**|**92.36**|**26.22**|**23.99**|*5.39*|\n\nNote: Due to time constraints, experiments are conducted on the “random class” of PIE-Bench, which includes all editing types and supports efficient and unbiased evaluation of model components.\n\n[1] Nitzan Y, et al. Lazy diffusion transformer for interactive image editing[C]//ECCV. 2024.\n\n[2] Zhuang J, et al. A task is worth one word: Learning with task prompts for high-quality versatile image inpainting[C]//ECCV. 2024.\n\n[3] Zhang L, et al. Adding conditional control to text-to-image diffusion models[C]//ICCV. 2023.\n\n[4] Ruiz N, Li Y, et al. Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation[C]//CVPR. 2023.\n\n[5] Amir Hertz, et al. Prompt-to-Prompt Image Editing with Cross-Attention Control[C]//ICLR. 2023."}, {"review_id": "DcMLDuFtBN", "reviewer": "Reviewer_SEN3", "summary": "This paper introduces EditInfinity - an editing model built on the top of Infinity T2I. It offers the advantages compared to diffusion model because VQ based generative model has the ground truth of the intermediate outputs. It learns the learnable prompt and LORA for text prompt rectification and image style preservation, and propose a smoothing method through piecewise linear kernel and multi-scale autoregressive. It freezes the original Infinity to have minimal training costs.", "questions": "1/ Please add more comparisons with top models like gpt-4o or Gemini.\n2/ Better ablation study for multi-scale autoregressive editing instead of putting one visualization example\n3/ Better write-up for method section 4, including a/ better definition of symbols, b/ training strategy (stage 1, stage 2, ....)", "limitations": "yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths\n* This model is one of first few AR model which can have editing capabilities with great quality - this is the top strength to me\n* It freezes infinity model so the method is very efficient.\n* Some improvements for alignment and style preservation through learnable prompts and LORA\n* Blended style multi-style AR is interesting.\n* Results are good based on benchmark scores and user study\n\nWeakness\n* It missed some top editing models in close source like Gpt-4o and gemini for comparisons. \n* Ablation quality could be improved for multi-scale autoregressive editing. It only has one visualization example and it is difficult to know whether it is cherry picked. \n* <minor> It is better have a definition of all symbols in Figure 2 and Figure 3, as well as training strategy (stage 1, stage 2, ....).", "was_revised": true, "final_justification": "After reading all rebuttals and other review feedback, I am leaning to weak accept to accept. VQ based generation models are quite behind diffusion one, and this work achieves solid results across multiple edit tasks compared to diffusion base models. I think this work can be a good milestone towards VQ based editing work.", "rebuttal": "Thank you for your insightful comments. We respond to your concerns point by point as follows.\n\n**Main Comment**\n\n> **Q1: Please add more comparisons with top models like gpt-4o or Gemini.**\n\n**A1:** We sincerely thank the reviewer for pointing out this valuable concern. As shown in Table 1 below, we provide a comparison with the **closed-source Gemini**, which serves as a strong commercial baseline. Due to time constraints, we are not able to include results for GPT-4o. Nonetheless, we believe the comparison with Gemini provides a representative and meaningful reference point for assessing our method. As shown in Table 1, the thoughtful design of our method enables it to outperform even the closed-source Gemini.\n\n**Table 1: Quantitative results of multi-scale autoregressive editing.**\n| Method              | PSNR↑ | LPIPS(×10³)↓ | MSE(×10⁴)↓ | SSIM(×10²)↑ | Whole↑ | Edited↑ | IR(×10)↑ |\n|---------------------|-----------|------------------|----------------|-----------------|-----------|------------|-----------|\n| P2P                  | 17.87     | 208.80           | 219.88         | 71.14           | 25.01     | 22.44      | 0.017     |\n| MasaCtrl          | 22.17     | 106.62           | 86.97          | 79.67           | 23.96     | 21.16      | -1.66     |\n| P2P-Zero         | 20.44     | 172.22           | 144.12         | 74.67           | 22.80     | 20.54      | -6.59     |\n| NTI                   | 27.03     | 60.67            | 35.86          | 84.11           | 24.75     | 21.86      | 2.77      |\n| PnP-Inv            | 22.46     | 106.06           | 80.45          | 79.68           | 25.41     | 22.62      | 4.17      |\n| NP                    | 26.21     | 69.01            | 39.73          | 83.40           | 24.61     | 21.87      | 2.42      |\n| StableFlow       | 21.64     | 92.28            | 115.21         | 84.94           | 24.65     | 21.70      | 1.88      |\n| RF-Edit             | 23.22     | 131.18           | 75.00          | 81.44           | 25.22     | 22.40      | 5.18      |\n| Gemini              |   23.22   |  105.17       |   188.63         |    81.10        |   25.28     |   22.28     |   5.30     |\n| **EditInfinity (Ours)**   | **27.95** | **33.08**        | **24.27**      | **92.12**       | **26.41** | **23.47**  | **5.88**  |\n\n\n> **Q2: Better ablation study for multi-scale autoregressive editing instead of putting one visualization example.**\n\n**A2:** We thank the reviewer for this insightful question. Due to space limitations in the main paper, this has been discussed in Table 4 of the supplementary material.\n\n**Table 4: Quantitative results of multi-scale autoregressive editing.**\n\n| Blend       | PSNR↑ | LPIPS(×10³)↓ | MSE(×10⁴)↓ | SSIM(×10²)↑ | Whole↑ | Edited↑ | IR(×10)↑ |\n|-------|--------|----------------|--------------|----------------|---------|----------|---------|\n| NAR   | 25.50  | 42.59          | 38.39        | 91.00          | 25.98   | 23.64    | 3.54    |\n| **AR**| **28.50**  | **31.58**        | **22.94**      | **92.36**        | **26.22** | **23.99**  | **5.39**  |\n\nBy blending source tokens at each scale in an autoregressive (AR) manner, our method provides continuous guidance for editing region generation at subsequent scales.  In contrast, the non-autoregressive (NAR) approach blends source tokens only at the end of each scale, without influencing the token generation process at the next scale. This leads to incoherent transitions and visually inconsistent edits, as illustrated in Figure 9 of main paper and Figure 2 of supplementary material. Table 4 here further supports this observation: AR consistently outperforms NAR in both background preservation and text alignment. These results highlight the necessity of autoregressive guidance for achieving harmonious and realistic edits.\n\n> **Q3: Better write-up for method section 4, including a/ better definition of symbols, b/ training strategy (stage 1, stage 2, ....)**\n\n**A3:** We sincerely thank the reviewer for pointing out the need for clearer symbol definitions in Figures 2 and 3, as well as a more structured presentation of the training strategy in Section 4. \n\n(a) We have added dedicated legend areas to Figures 2 and 3 to explain the meaning of all relevant symbols, such as $t_{sou}, t_{ins}, t_{l}, R_{1...K}^{sou}, R_{1...K}^{inv}, R_{1...K}^{tar}, E_{1...K}^{tar}, \\mathcal{L}_{inv}, \\mathcal{G}$, etc. As the rebuttal does not support figures, we are unable to show the updated figures here.\n\n(b) We have clearly annotated Stage 1 and Stage 2 in Figure 2 and provided a more precise and structured definition of both stages in Section 4, as outlined below.\n\n- Image inversion stage 1: textual prompting rectification.\n- Image inversion stage 2: image style preservation.\n\n We greatly appreciate your suggestions, which will help improve the clarity and rigor of our method section."}]

Author rebuttal:

Meta-review: This paper introduces EditInfinity, which is a text-based image editing method based on the autoregressive text-to-image generation model Infinity, which succeeds in high quality generation among vector quantized models available. The authors motivate the paper from the existing inversion-then-editing approach, where instead of inverting into a noisy latent (like in diffusion models), they initially invert the input text prompt by an extended embedding representation, which is followed by LoRA based adaptation for the reconstruction of precise details. Given a target text prompt, an input image and a region of interest specified by a mask, EditInfinity performs editing by manipulating the multi-scale quantized feature maps with a smoothed mask, then reconstruct the image using these manipulated feature maps. Authors demonstrate the effectiveness of the approach by using PIE-Bench, where they compare their approach with diffusion/flow-matching based editing methods. The provided quantitative analyses and qualitative results shows the effectiveness of the method.

The reviewers found the work somewhat incremental (the paper extends the Infinity model to editing tasks) but considered that there are some benefits to autoregressive image generation (such as speed) that merit further investigation in this area. Three of the reviewers were initially negative. They raised many concerns, which the authors addressed successfully. After discussion there were still some concerns about the method reliance on user-provided masks, a limitation noted by multiple reviewers. This requirement reduces its generalizability. However, since VQ-based approaches remain relatively underexplored, the reviewers believed the paper is an overall positive contribution. In the end, the reviewers were mildly positive towards publication of the paper.
Decision: Accept (poster)