Title: EditInfinity: Image Editing with Binary-Quantized Generative Models
Venue: NeurIPS 2025
Accepted: yes
Keywords: Image Editing; Text-to-Image Generation; Autoregressive Models
Abstract: Adapting pretrained diffusion-based generative models for text-driven image editing with negligible tuning overhead has demonstrated remarkable potential. A classical adaptation paradigm, as followed by these methods, first infers the generative trajectory inversely for a given source image by image inversion, then performs image editing along the inferred trajectory guided by the target text prompts. However, the performance of image editing is heavily limited by the approximation errors introduced during image inversion by diffusion models, which arise from the absence of exact supervision in the intermediate generative steps. To circumvent this issue, we investigate the parameter-efficient adaptation of binary-quantized generative models for image editing, and leverage their inherent characteristic that the exact intermediate quantized representations of a source image are attainable, enabling more effective supervision for precise image inversion. Specifically, we propose EditInfinity, which adapts Infinity, a binary-quantized generative model, for image editing. We propose an efficient yet effective image inversion mechanism that integrates text prompting rectification and image style preservation, enabling precise image inversion. Furthermore, we devise a holistic smoothing strategy which allows our EditInfinity to perform image editing with high fidelity to source images and precise semantic alignment to the text prompts. Extensive experiments on the PIE-Bench benchmark across add, change, and delete editing operations, demonstrate the superior performance of our model compared to state-of-the-art diffusion-based baselines. Code available at: https://github.com/yx-chen-ust/EditInfinity.


Peer reviews:
comments: SUMMARY:
This paper introduces EditInfinity - an editing model built on the top of Infinity T2I. It offers the advantages compared to diffusion model because VQ based generative model has the ground truth of the intermediate outputs. It learns the learnable prompt and LORA for text prompt rectification and image style preservation, and propose a smoothing method through piecewise linear kernel and multi-scale autoregressive. It freezes the original Infinity to have minimal training costs.

STRENGTHS AND WEAKNESSES:
Strengths
* This model is one of first few AR model which can have editing capabilities with great quality - this is the top strength to me
* It freezes infinity model so the method is very efficient.
* Some improvements for alignment and style preservation through learnable prompts and LORA
* Blended style multi-style AR is interesting.
* Results are good based on benchmark scores and user study

Weakness
* It missed some top editing models in close source like Gpt-4o and gemini for comparisons. 
* Ablation quality could be improved for multi-scale autoregressive editing. It only has one visualization example and it is difficult to know whether it is cherry picked. 
* <minor> It is better have a definition of all symbols in Figure 2 and Figure 3, as well as training strategy (stage 1, stage 2, ....).

QUESTIONS:
1/ Please add more comparisons with top models like gpt-4o or Gemini.
2/ Better ablation study for multi-scale autoregressive editing instead of putting one visualization example
3/ Better write-up for method section 4, including a/ better definition of symbols, b/ training strategy (stage 1, stage 2, ....)

LIMITATIONS:
yes
TITLE: Official Review
CLARITY: 2
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-06-25

{"IS_META_REVIEW": false, "comments": "SUMMARY:\n**EditInfinity** introduces a novel text-driven image editing method that leverages binary-quantized generative models as an alternative to traditional diffusion-based approaches. By utilizing discrete token representations, the method enables precise image inversion with exact supervision and avoids the approximation errors typical of diffusion inversion. The authors enhance this process through a learnable prompt mechanism and LoRA-based style preservation. For the editing phase, they propose a holistic smoothing strategy that blends source and target tokens using a piecewise linear kernel, allowing for seamless and localized modifications. Evaluated on the PIE-Bench benchmark, EditInfinity achieves state-of-the-art results in both background preservation and semantic alignment, significantly outperforming diffusion-based baselines across a variety of editing tasks.\n\nSTRENGTHS AND WEAKNESSES:\n### Strengths\n\n1. **Clear Motivation and Novelty**: The paper addresses limitations of diffusion-based editing, especially inversion error, and innovatively adapts a VQ-based model, which is a less explored direction.\n2. **Thorough Evaluation**: The authors conduct extensive quantitative and qualitative experiments, including a user study, to demonstrate improvements across various editing tasks. EditInfinity outperforms existing methods in key metrics, especially in preserving unedited regions while aligning edits with prompts.\n\n### Weaknesses\n\n1. **Structured Editing Setup**: The method heavily depends on well-aligned source-target prompt pairs and user-provided region masks. This reliance on structured inputs may limit its generalizability to open-ended or ambiguous editing tasks where such precise supervision is unavailable.\n2. **Computation Cost**: The framework involves several non-trivial steps, including prompt optimization, LoRA fine-tuning, and multi-scale autoregressive token generation. This complexity could hinder real-time applications and make the method less practical.\n3. **Manual Design**: The framework includes several manually crafted components, such as user-defined editing masks and a fixed smoothing kernel. These elements introduce human priors into the pipeline, and it remains unclear how robust the approach is to noisy inputs or complex, less structured scenes.\n\nQUESTIONS:\n1. How well does the method handle vague or high-level prompts that lack explicit object references or structure?\n2. The inversion and editing stages seem computationally demanding. What is the latency for a single edit?\n3. How is the edited region determined in practical scenarios? Can the method infer edit regions automatically based on differences between source and target prompts, or does it rely entirely on manual mask annotation?\n\nLIMITATIONS:\nPlease refer to the weaknesses section.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-06-29"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces EditInfinity, which is a text-based image editing method based on the autoregressive text-to-image generation model Infinity, which succeeds in high quality generation among vector quantized models available. The authors motivate the paper from the existing inversion-then-editing approach, where instead of inverting into a noisy latent (like in diffusion models), they initially invert the input text prompt by an extended embedding representation, which is followed by LoRA based adaptation for the reconstruction of precise details. Given a target text prompt, an input image and a region of interest specified by a mask, EditInfinity performs editing by manipulating the multi-scale quantized feature maps with a smoothed mask $\\mathcal{G}$, then reconstruct the image using these manipulated feature maps. Authors demonstrate the effectiveness of the approach by using PIE-Bench, where they compare their approach with diffusion/flow-matching based editing methods. The provided quantitative analyses and qualitative results shows the effectiveness of the method.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n- The paper tackles the image editing problem using autoregressive image editing methods with a new inversion scheme which mainly depends on approximating the ideal generation prompt, and minor adjustments on the autoregressive model with LoRA-based optimization.\n- The proposed method achieves succeeds in background preservation compared to existing approaches\n- The editing pipeline proposed provides satisfactory results on edits that can be constrained with a region identified by a mask.\n- Authors provide ablations for the design decisions made for each of the components of the method.\n\n**Weaknesses**\n- The limitations of the method is discussed very briefly, where the authors should provide more qualitative examples for the failure cases.\n- The edits that can be performed with the proposed approach seems limited with an input mask, where providing the whole image as the editing region of interest seems to be disturbing the background preservation properties and the preservation of the input pose, structure (see Fig. 6, row 9). If there is such limitation, the authors should explicitly discuss this for transparency.\n- There are missing details in the quantitative evaluation setup and method details. For the details on the method, the proposed approach involves two-stage training per image. To fairly assess if this approach is feasible as an editing method or not, the authors should provide a time comparison with the competing approach in terms of the overhead on top of the base model. In addition, the evaluation requires an input mask for EditInfinity, where the method used to obtain these masks are not described in detail. The authors should privide such details on the evaluation setup.\n- Given the short description about the limitations from the authors, the proposed method succeeds in masked edits. While this is a valid use case for editing, it does not cover a majority of the edits to be performed. As an example, editing facial attributes is nor always maskable (e.g. adding smile). It is not clear if the method can perform such edits or not, if it can there should be examples on such detailed cases. In addition, the editing examples provided are rather simplistic cases where there is a single subject that is centered in the image. This makes the performance in on-the-wild images questionable. The authors should provide examples for such use cases.\n\nQUESTIONS:\n- How are the masks used in the editing process obtained in the evaluations? Does the method assume that these masks are provided by the user or is there an automated pipeline to achieve this?\n- Is the method able to achieve disentanglement for un-maskable edits such as facial attribute change?\n- What are the failure cases of the method in detail?\n- Is there are timing constraint on the method, as a two stage optimization is required for editing?\n\nLIMITATIONS:\n- The limitations are very briefly addresses over a small discussion. I believe that further details should be included like failure cases as discussed over the *Questions* and *Strengths And Weaknesses* sections. In addition the authors are encouraged to include a timing based comparison due to the two stage optimization that the method requires (for each editing example). For further comments on suggested improvements over the limitations, please refer to *Questions* and *Strengths And Weaknesses* sections.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 4, "ORIGINALITY": 3, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a method to adapt a type of pretrained text conditioned image generative models to the task of image editing. The model in question is Infinity, an autoregressive model on binary residual quantized tokens in the image scale space. The proposed method first conduct text inversion with learnable language style tokens and then performs editing with soft masking in the generation trajectory. \n\nThe proposed method is tested on PIE benchmark with add, delete, and change operations. It shows better image editing capability than other training-free baselines.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n+ The idea of using the Infinity model, which provides exact decomposition of images, as a trajectory to reference seems interesting. The proposed masking technique appears to provide good background preservation. \n+ Based on the provided visual results, the proposed method can accurately achieve addition, deletion, and change.\n+ The cost of adapting the pretrained text to image model is minimal thanks to the use of LoRA. \n\nWeakness\n- It is unclear how the text style token t_l is optimized or learned. \n- The blending operation is not described clearly. Since the Infinity model uses quantized tokens, it is unclear how linear blending happens between two token indexes. \nThe method seems tailored to the infinity model; it is unclear whether it can be generalized to other types of image generative models.\n\nQUESTIONS:\nI would like to encourage the authors provide more detailed explanation on the optimization of t_l and the blending of quantized tokens. \n\nAlso, I would like to see clarification on the experimental results. Are other baseline methods also using Inifinity as the base model? If not could the worse results for these methods be due to their worse base models?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 2, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-03"}