Enabling Instructional Image Editing with In-Context Generation in Large Scale Diffusion Transformer
Authors: Zechuan Zhang
Ji Xie
Yu Lu
Zongxin Yang
Yi Yang
Venue: NeurIPS 2025 poster ()
Keywords: image editing; diffusion model; diffusion transformer; instruction-based image editing
Primary area: applications
TL;DR: Enabling Instructional Image Editing in Large Scale Diffusion Transformer

Abstract: Instruction-based image editing enables precise modifications via natural language prompts, but existing methods face a precision-efficiency tradeoff: fine-tuning demands massive datasets (>10M) and computational resources, while training-free approaches suffer from weak instruction comprehension.  
We address this by proposing \textbf{ICEdit}, which leverages the inherent comprehension and generation abilities of large-scale Diffusion Transformers (DiTs) through three key innovations: (1) An in-context editing paradigm without architectural modifications; (2) Minimal parameter-efficient fine-tuning for quality improvement; (3) Early Filter Inference-Time Scaling, which uses VLMs to select high-quality noise samples for efficiency.  
Experiments show that ICEdit achieves state-of-the-art editing performance with only 0.1\% of the training data and 1\% trainable parameters compared to previous methods. Our approach establishes a new paradigm for balancing precision and efficiency in instructional image editing.

Enabling Instructional Image Editing with In-Context Generation in Large Scale Diffusion Transformer
Zechuan Zhang', Ji Xie', Yu Lu', Zongxin Yang", Yi Yang
ReLERR, CCAI, Zhejiang University
DBMI, HMS, Harvard University
Project Page:
Multi-turn Editing
Aloha Hawaii Add the text Aloha the bottom color.
Original Image
Change background scenery.
Replace SpongeBob book photo.
shorts board.
[CEdit
the evening, the fire burning.
Replace the buildiig Eiffel tower.
Convert the Image Into anime illustration
Multi-task Editing
Remove the dog
Wears diamind carring, and a golden rElD crown.
Make her hair dark green and her clothes checked.
Original Image
Holding cup tea, eye closed.
What patercoloon painting?
ID Consistent Editing
We introduce ICEdit, novel method that achieves state-of-the-an art instruction- based image editing with only 0.1% % training data required by previous SOTA methods, demonstrating exceptional generalization. The first row illustrates series of multi-turn edits, executed with high precision, while the second and third rows highlight diverse, visually impressive editing results from our method.
Figure 1:
Abstract
Instruction- image editing enables precise modifications via natural language prompts, but existing methods face a mrecison-efffciency tradeoff: fine-tuning demands massive datasets (>10M) and computational resources, while training- free approaches suffer from weak instruction comprehension. We address this by proposing ICEdit, which leverages the inherent comprehension and generation abilities of large-scale Diffusion Transformers (DīTs) through three key innovations: (1) An in-context editing paradigm without architectural modifications; (2) Minimal parameter-efffiient fine-tuning for quality improvement; (3) Early Filter Inference-
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Time Scaling, which uses VLMs to select high-quality noise samples for efficiency. Experiments show that ICEdit achieves of-h-offtheeart editing performance with only 0. 0.1% % of the training data and 1% trainable parameters compared to previous methods. Our approach establishes a new paradigm for balancing precision and efficiency in instructional image editing.
1 Introduction
In recent years, instruction- based image editing has gained considerable attention for its ability to transform and manipulate images using natural language prompts. The main advantage of instruction- based editing is its ability to generate precise modifications with minimal textual instructions. thereby opening new possibilities for both automated image processing and user-driven content creation.
Instruction-basee image editing methods are divided into finetuning- based [1, 2, 4, 5, 6, 7, 8, 9, 10] and training- frree approaches
precise cestructiio-followiin [11, 12, 13, 14, 15, 16, 17, 18, 19]. Finetuning- based methods achieve by fully finetuning pretrained diffusion models on large-scale datasets (450K 10M samples (I) with structural modifications like condition embedding [9, 5] channel adjustments [1, 2, 4]. but demand significant computational resources. In contrast, training-fres methods avoid retraining through techniques like image inversion, or attention manipulation, offering efficiency but struggling with complex instructions, which reduces precision and practical utility. This highlights critical trade-off between precision and efficiency in current methods.
Despite the dilemma above, recent advances in diffusion transformers (DiT) [20, 21, 22] suggest promising pathway. DiT architectures exhibit critical properties: (1) Scalable Generation Fidelity: Larger DiT variants (e.g., FLUX [23]), trained on vast amounts of image-text data, possess unprecedented text-to-image alignment capabilities. (2) Intrinsic Contextual Awareness. Diffusion Transformers (DiTs) leverage attention mechanisms to enable bidirectional interactions between reference and generated content, processing source and target images concurrently. This facilitates tasks like reference-guice guided synthesis [24, 25] and identity-preserve editing [26], while supporting conditional image generation without specialized alignment networks [27, 7, 28..
Although these works achieve promising results, they are unsuitable for instruction-guie guidddd image editing due their limited ability to comprehend explicit editing instructions and preserve the layout of non-editable regions. This raises a critical question: Can large scale DiT's generation capacity and contextual awareness directly address instruction-bant based image editing, while balancing precision and efficiency through intrinsic capabilities rather than external complexity?
Our experiments reveal two fundamental limitations in using DiTs for instructional image editing: (1) Poor instruction comprehension: while the model can interpret descriptive input/target prompts, struggles with direct editing instructions (e.g.. "make it..." or "change it..."); (2) Layout instability: the model often alters unchanged regions when regenerating scenes, leading to poor editing success.
To address these limitations, we suggest a two-part solution. First, we recommend turning direct editing commands into descriptive prompts that match how DiTs naturally understand information. Second, the editing challenges mainly arise from the model's 's unlearned image- to- to-image editing priors, which can potentially be addressed through lightweight fine-tuning with editing pairs or test-time adaptation [29, 30] strategies. This approach offers the potential to simultaneously resolve DiT'' two fundamental limitations in instructional editing while maintaining precision- -effciicneefccienccy balance.
In this paper, we propose ICEdit, an efficient and effective framework for instructional image editing that directly exploits the inherent comprehension and generative capabilities of large-scale DiT priors for instructional image editing. Our approach employs in-context prompts-a fixed-format prefix embedding editing instructions in a format DiT can effectively interpret ($3. Given source image (left panel), the model generates edited outputs (right panel) by jointly processing these fine-tuning -oxture-of-Experte (MoE) [31]
prompts and the input (see fig. 2(a) and fig. 5). Moreover, minimal parameter-efficee significantly improves editing success and quality, while adopting further enhances performance ($3.2). Finally, we introduce Early Filter Inference- Time Scaling, which uses vision-language models (VLMs) to evaluate noise quality during early denoising stages in rectified flow models ($3.3). This method rapidly identifies and filters out noises congruent with textual instructions, enhancing robustness and output fidelity.
2

---

IN-CONTEXT EDIT
Previoss work condition
Our Method
Perforatance Traising Efficiency
(190,
Mtikaa (14)
CL.2 Sont 25
T21 Diffusion Transformer
Inpainting DIT
Awll
lecawatiy
Reference
Edit
AI
GOOOM
Vi Cacaor
Broedee
Fastds
CLN Soor (h)
(a) Illutratiio In-Cautest Concept
(b) A Brief Comparison between Our Method Others
(c) Precision Effiicioney Our Methed
Figure 2: (a) Our concept: The original DiT model conceptualized edited images (right panel) by interpreting the reference image (left panel) and the instruction, similar finishing an artwork based on a provided sketch. (b) Comparison of model structures: Unlike prior DiT methods, our approach avoids extra position/condition encoders, easily and effectively preserving the original model structure. (c) Our method leverages minimal training data and achieves comparable performance with SOTA models.
painter who generates
We evaluate our method on the Emu Edit [3] and MagicBrush [2] benchmarks, demonstrating three key advancements: Significant Data Efficiency and Editing Quality: Achieving state-of-the-ar results with minimal training data (0.1% 1% of prior requirements). Validation of Proposed Paradigm: Outperforming recent DiT-based editing models (both T2I and inpainting variants), confirming the effectiveness of our in-context editing approach. Practical Applicability; Attaining competitive VIE score of 78.2 (compared to SeedEdit's s 75.7), demonstrating real-world viability comparable to commercial systems. These results establish novel perspective on balancing precision and efficiency (fig. 2(c)) by leveraging large-scale DìTs as priors. Our contributions include:
We explore the editing ability of large pretrained DiTs and propose an in-context editing paradigm, ICEdit, that enables instructive image editing by leveraging the model's inherent understanding and generation abilities, without requiring architectural modifications or extensive fine-tuning.
Our framework demonstrates significant improvements through minimal fine-tuning, significantly enhancing editing quality and robustness. We further propose an Early Filter Inference- Time Scaling approach that uses VLM select high-quality noise samples. This integrated strategy improves editing precision while maintaining computational efficiency.
Experimental results show that our method achieves state-of-the-art editing performance while requiring only 0.1 % of the training data compared to previous approaches, establishing novel perspective on balancing precision and efficiency.
2
Related Work
Training-fres frree editing techniques. Since the emergence of Diffusion Models, numerous training free image editing methods [11, 32, 33, 15, 17, 16, 13] have gained attention. Recently, RF-Solver [13] improves inversion precision in fectified-flow models by mitigating ODE-solving errors and leverages MasaCtrl [19] for image editing. StableFlow [14] identifies critical MM-DiT Blocks through ablation studies, injecting features only into these blocks to enhance editing capabilities. However, these methods face two key limitations: 1) manually designed modules restrict generation ability, hindering complex instruction understanding and reducing success rates; 2) editing requires carefully crafted prompts, limiting generalizability and scalability.
Finetuning- based editing methods. Most current editing models modify architectures and fine- tune on high-quality datasets [1, 2, 34, 4, 35, 10, 8, 36, 37, 38, 39, 40, 41J. InstructPix2Pix [1], MagicBrush [2], and UltraEdit [4] fine-tune diffusion UNet using original images input. MGIE [5] and SmartEdit [9] enhance instruction understanding by integrating Multimodal Large Language Model (MLLM) to encode and inject instructions into the diffusion model. However, gap exists between the embedding spaces of generative prompts and editing instructions, reducing the generalization ability of Diffusion Models and necessitating large datasets to bridge it. For instance, InstructPix2Pix generated 450K pairs, Emu Edit [3] collected nearly 10M pairs, and FluxEdit [42] used 1.2M pairs from [34] based on FLUX [23], yet the editing results remain suboptimal.
3

---

Inpet caption:
Output Caption:
Some palm
Somte palm and other plants sining en on
plants
sittng on highway overpass on cloudy day.
cloudy day with
Add a rainluw to the sky
rainbow sky.
Input caption: boys play yullow frisbre outdoors.
Output Captioon:
Two boys play with yellow frisbes outdoors saowy day.
diptych with too sido by side Sanie on the left but makee
Make it
Direct Edit Instruction
In-Context Edit Prompt
Global Descriptive Prompt
Figure 3: Input Prompt Variants for Image Editing. We evaluate three prompt formats: (1) Direct Instruction explicit editing commands provided directly; (2) In In-context Prompt instructions embedded structure "A diptych with... On the right, the same scene but finstruction : (3) Global Descriptive Prompt uses full input/output captions ("On the left (input) On the right (output)")
In-context Editing Framework Results (based Flux.1
Inpainting In-context Editiin Framework Flux.1 Fill)
Change
plant pot to
Change dog poodle.
Figure 4: Training-Free Methods Show Limited Performance. Both T2I and inpainting DiT frameworks (fig. 5) yield suboptimal results. The T2I DiT struggles to preserve the original layout, while the inpainting framework may inadvertently perform outpainting. Despite these shortcomings, both demonstrate potential in following instructions and modifying edited regions.
3 Method
In this section, we first explore in-context editing capabilities within original DiT generative models and propose our in-context edit framework for instruction- based image editing ($3.1). After thorough analysis, we perform minimal fine-tuning to significantly improve editing quality and robustness our editing paradigm ($3.2). Finally, we present early filter inference-time scaling ($3.3) to optimize initialization noise for better generation quality.
3.1 Exploration of DIT's In-context Edit Ability
In-Context Edit Framework. Inspired by recent advances in large-scale Diffusion Transformer (DiT) models [27, 26, 43, 24], which demonstrate robust contextual capabilities, investigate image editing via in-context generation. As illustrated fig. 2(a), our approach mimics an Al painter that generates edited images (right panel) by interpreting reference image (left panel) and an edit instruction. By leveraging DiT's strong generation fidelity and inherent contextual awareness, we aim enable direct image editing without requiring module modifications extensive finetuning.
We propose two training-frre free frameworks based on text-to-image to-image DiT (ICEdit-T2) T2I) and inpainting DiT (ICEdit-Inpaing) respectively, shown in fig. 5(a). For the ICEdit-T21 framework, introduce an implicit reference image injection method. We perform image inversion [44, 13, 14, 19, 18] on the reference image, preserving attention values across layers and steps. These values are injected into tokens representing the left side of diptych for image reconstruction, while the right side is generated based the edit instruction within predefined prompt during in-context generation.
Conversely, (CEdit-Inpain framework offers more straightforward approach. By accepting a reference image, we construct side-by-side image, where the left-hand side is reconstructed as reference image, and the right-hand side is the "edited" result. The whole process guided by fixed mask, which consistently covers half of the diptych, and edit prompt to produce the edited output.

---

In-context Editing Framework based on
LoRA Module
Inconiedi Generraioo
121 Diffusion Transformer
Jerrvion x N
Inverted Noise
Decoder
Bazn Layer
Source Image
Feature Injoction
N+
Demoising
T21 Diffusion Transformer
Inverted Noise
Random Noise
Projection
srop
Topket
MLP
On right, by side images of but (make the main grab basketbalg.
Routo
Targast Imsige
Norm Nod
In-context Editing Framework based on inpainting DIT
0000
:
Trainabe Parameter
Toxt Encoder
Image Encoder
Frozen Paramater
mask
Impainting DIT
concia
crop
Immaee
Nose
Standard LoftA Tuntng
"Adiptych ietuti looks soter coter peanknggh"
LoRA-NoE Turing
with Mask
Source Image
clpnych with Imake her in Image
gall
L Adspter
(a) Training-free Structure of ICEdit
(b) Finetuning Strategy for ICEdit
Figure 5: Model Structure. (a) We propose two training-free architectures for in-context editing using large-scale DiTs, adapted from (a) T2I-DiT and (b) inpainting-DiT. Both adopt diptych framework: left panel = source image, right panel editing output, consistent with Fig. 2. (b) While both show limited performance, we adopt the inpainting paradigm for further fine-tuning due to its simplicity (no image inversion required). Our method integrates parameter-efficien adaptation with dynamic expert routing for specialized feature processing.
Unlike prior DiT-based approaches [27, 26, 24], our method climinates the need for intricate position and condition encoding designs retraining, illustrated in fig. 2. Instead, it purely leverages the diptych image structure and the inherent processing capabilities of DiT.
In-Contes Edit Prompt. Diffusion models typically struggle to interpret editing instructions due mismatch between the embedding spaces of descriptive prompts and editing instructions. Previous approaches [1, 2, 3, 4, 42], rely on extensive training with large-scale editing datasets to enable generative diffusion models to understand editing instructions. In contrast, we propose leveraging the inherent contextual understanding of powerful Diffusion Transformers (DiT's to perform instruction-based image editing without heavy training.
We experimented with three prompt types to enable DiT model (c.g.. FLUX) to edit a given image, as shown in fig. 3. First, directly inputting the editing instruction into the model often fails to produce accurate results, frequently altering the entire image layout. Second, we designed an in-context edit prompt (IC prompt) that embeds the instruction within descriptive structure: "A diptych with two side-by-side images of the same scene. On the right, the scene is identical to the left but finstruction)." " This IC prompt significantly improves the model's ability interpret instructions. yielding approximately a 70% increase in editing success rate, as reported in table 3. Finally, we tested a training-free approach using global descriptive captions for both source and target images, similar to prior methods [44, 14, 16, 13]. While achieving better quality and instruction adherence, this method relies on impractical, detailed image descriptions, undermining scamless instruction based editing. Thus, we adopt the IC prompt in our framework and further refine it through fine-tuning.
Discussion of the Training-Free Framework: Figure 4 presents the editing outcomes of the T2I and inpainting frameworks on the Emu Testset, guided by the IC prompt. While both frameworks demonstrate some editing capability, their zero-shot performance is unsatisfactory, particularly in preserving unedited regions, which limits their practical utility (quantitative eval in table 3). We attribute this to the lack of learned image-to-image editing priors. This limitation could be mitigated through lightweight adjustments, such as finetuning or test-time scaling. Given that the ICEdit-T21 framework requires time-consuming image inversion, we favor the ICEdit-Inpaint framework for its straightforward operation, which facilitates further finetuning.
5

---

"Make the sky night instead of day."
with stars viidh actanaaaaaeetta sngggtioonn
The bocsoe list da illid
Seed
leage aleilaaeee... leceatod acod Sarge pddiad ag dass aur andt cdff regirearry
VLM Judge
Seed
Seed
Sourc
Seed
VLM Budge
Seed B
Seed
step
step
step
50 step
Source Image
Early Step
Filter
Increased Step for Quality
Early Step Results
Figure 6: Illstration of Inference-Timee Scaling Strategy ($3.3), The upper rows demonstrate that edit success can be assessed within few initial steps. These early results used to filter the optimal initial noise with VLM judges.
3.2 Efficient Fine-tuning for Enhanced Editing
We define our teaiinting-based editing method as a function & mapping a source image L, and edit instruction Te to the edited output 4:
M, McT), where D is the inpainting DiT, Ii is the in in-context image with I, on the left, M is a fixed binary mask reconstruct L,, and TiC is the in-context edit prompt derived from Te.
4e (
(1)
To boost performance, we curate a compact dataset (50K samples) from public sources ($44 and apply LoRA fine-tuning [45, 27] multi-modal DIT blocks, achieving 150% improvement in editing success (table 3) despite the small dataset. However, tasks like style transfer and object removal remain challenging.
single LoRA structure struggles to handle diverse editing tasks requiring distinct latent feature manipulations.
Mixture of LoRAs. To overcome these limitations, we propose Mixture-of- (MoE) inspired LoRA structure within the DiT block (fig. 5(b),, inspired by MoE architectures [46, 47, 31]. We integrate N parallel LoRA experts into the multi-modal attention block's output projection layer, using standard LoRA elsewhere. routing classifier selects experts based on visual tokens and text embeddings. Each expert, LoRA module with rank and scaling factor a, contributes to the output:
N
Output BaseLayer(a))
Σ G(+), B A,
where B: Rdxr, A: Brxk RR and G(x); is the routing probability. We use sparse MoE setup, selecting the top-k experts: G(*); = Dofttax(Toop((/l) (Top ()) where TopK k) retains the top-k entries, setting others -x, ensuring efficiency and versatility for diverse editing tasks.
3.3 Early Filter Inference Time Scaling
During inference, we find that initial noise significantly shapes editing outcomes, with some inputs producing results better aligned with human preferences (see fig. 9), a pattern supported by recent studies [30, 29]. This variability drives us investigate inference-ting time scaling to improve editing con- sistency and quality. In instruction-basse
editing, we observe that success in instruction alignment often become evident few inference steps (see fig. 6), characteristic compatible with rectified flow DiT models [48, 49]. These models traverse latent space efficiently, delivering high-quality outputs with few denoising steps-sometiin as few as one [50]. Thus, unlike generation tasks that demand more steps for detail and quality, we can evaluate edit success with only few steps.
Based on this insight, we propose an Early Filter Inference Time Scaling strategy. We start by sampling M initial noise candidates and generating preliminary m-step edit for each, where m < n (the full denoising steps). visual large language model (VLM) [51, 52, 53] then assesses these M early outputs for instruction compliance, using bubble sort-inspired pairwise comparison to iteratively pinpoint the top candidate, akin selecting the maximum value (see fig. 6). This optimal seed is subsequently refined with n-step denoising to produce the final image. Our approach quickly identifies good noise early, while VI.M selection ensures the output aligns with human preferences. Further details are provided in the supplementary materials (Sup. Mat.).

---

Table 1: Quantitative results on Emu Test set ($4.1). Following [4, 3], we compute CLIP-I and DINO scores between the source and edited image. while CLIP-out measures the distance between output caption and edited image. We also employ GPT-4o to evaluate the edited results. The Train.
Pa. means parameters finetuned for the editing task. indicates methods that rely on output captions.
Methods
Base Model
Train. Pa.
Data Usage
CLIP-IT
CLIP-Out
DINO
GPT T
InstructP2P [CVER23]
0.9B
0.45M
0.856
0.292
0.773
0.36
MagicBrush [NeuiIP223]
SD 1.5
0.98
0.47M
0.877
0.298
0.807
0.48
EmuEdit (CNP1C24
Close Source
2.8B
10M
0.877
0.306
0.844
0.72
UltraEdit (NerrPPS24]
SD
2.5B
3M
0.880
0.304
0.847
0.54
FluxEdit (boggingfoo)]
Flux. dev
12B
1.2M
0.852
0.282
0.760
0.22
FLUX.1 Fill Bogginggaae
Flux.1 Fill
0.794
0.659
0.24
RF-Solver Edit" (CML225)
Flux.1 dev
0.797
0.309
0.683
0.32
ACE+ (anoo25)
Flux.1 Fill
12B
54M
0.791
0,280
0.687
0.24
[CEidit (ours)
Flux.1 Fill
0.28
0.05M
0.907
0.305
0.866
0.68
Input
Ours
RF-Solver Edit
EmuEdit
UltraEdit
MagicBrush
InstructP2P
Add the word "EXIT"
of
the plant pot to
the background severe lightning storm
Convert the image into
Figure Comparison with baseline models on the Emu Edit test set ($4.1). Our method demonstrates superior performance in both edit-instruction accuracy and preservation of non-edited regions compared to the baseline models. Q Zoom for detailed view.
Experiment
Implementation Details. We use FLUX.1 Fill,
Table 2: Quantitative results on MagicBrush
test set. Following [4], all metrics are calculated between the edited image and GT edited image provided by MagicBrush [2].
leading open-source DiT-based inpainting model, as our backbone. For fine-tuning our hybrid LoRA-MoE module, we curated 50K- sample editing dataset, combining 9K sam- ples from MagicBrush [2] and 40K from Om- niEdit [34] to address MagicBrush's 's limitations in edit type balance, style-focused data, and do- main diversity. The model employs a LoRA rank of 32, four MoE experts, and a TopK value of 1. For inference- time scaling, we use Qwen- VL-72B [51] to evaluate image outputs.
Methods
L1 4
CLIP-I
DINO t
InstructP2P
0.114
0.851
0.744
MugicBrush
0.074
0.908
0.847
UltraEdit
0.066
0,904
0.852
FluxEdit
0.114
0.779
0.663
FLUXX1 Fill
0.192
0.795
0.669
Ro-Sol Edit*
0.112
0.766
0.675
ACE+
0.195
0.741
0.591
Evaluation Settings. We evaluated our model on Emu [3] and MagicBrush [2] test sets. For MagicBrush, which provides ground truth (GT)
ICEdit (ours)
0.060
0.928
0.853
edited images, we follow [4, 2] to compute CLIP [54, 55], DINO [56, 57], and Ll metrics, measuring

---

Source
MagicBruah
UltraEdit
SeadEdit
Oura
VIE-S (spt-4o evaluation)
053
T2.
$4-2
Make the girl wear red robe and black Jeans.
S0 Scort
FQ Score
Oveall Score
Make the basketball both hands.
We employ the VIE-score to evaluate human preference alignment and quantify improve- ments from our inference-tine scaling strategy (w/ Inf. Sea.) ($4.1 and $4.2).
Figure 8:
Table 3: Ablation study model structure configuration and inference time scaling settings.
(a) Ablation study on model structure ($4.2).
(b) Ablation of inference time scaling ($4.2).
Settings
Params CLIP-I [ CLIP-T GPT
Verifier Noise Num
Inf. Step
NFE + GPT
Training IC prompt
0.681
0.68
Training-fie free w/ IC prompt
0.794
0.273
0.24
VLM
10
110
0.78
Only MoE module 130M
0.929
0.300
0.51
VLM
6
4
74
0.72
LoRA (1=64) IC prompt
0.911
VLM
12
0.79
Ours wfo IC prompt 214M
0.896
0.300
0.62
VIM
50
350
0.80
Ours
214M
0.907
0,305
0.68
CLIP
6
50
350
0.65
divergence from GT. For the Emu test set, lacking GT, we adopt baseline assessments from [4, 3] and use GPT-4o [58], same as [34], to assess editing success (see Sup. Mat.). All models use a single default noise input, excluding our Early Filter Inference Time Sealing for fair comparison.
Conventional metrics like CLIP [54, 55] and DINO [56, 57] often misalign with human prefer- ences [59, 34, 10]. We thus employ VIE-Score [59], combining SC (instruction adherence and unedited region preservation) and PQ (visual quality) scores, computed as Overall = /SC x PQ. This metric evaluates the improvement brought about by our inference-time scaling strategy and benchmarks our model against SeedEdit [60], leading close-source model.
4.1 Comparisons with State-of-the-Art
Results on Emu Edit and MagicBrush Test Sets. We compare our model against UNet-based [1,
2, 3] and DiT-based [4, 42, 13, 7] methods (tables 1 and 2). Our model achieves SOTA COTA-coopaaable performance, with MagicBrush outputs (table closely matching GT and showing strong editing proficiency. On Emu (table 1), it aligns well with instructions while preserving image fidelity better than SOTA. GPT-based scores surpass open-source models and rival closed-source Emu Edit, despite using 0.5% training data. Compared to DiT-based models, our approach excels with fewer samples and parameters, demonstrating high efficiency. Qualitative results are fig. and Sup. Mat.
VIE-Score Eval. As shown in fig. 8, our model significantly outperforms open-source SOTA methods in editing accuracy and visual quality. Random seed testing shows performance nearing SeedEdit, and with our inference scaling strategy, it surpasses SeedEdit in overall VIE-Score Although SeedEdit achieves higher PQ scores due to its polished outputs, struggles with identity preservation in unedited regions. Our method, however, excels in maintaining fidelity in these areas (fig. 8).
4.2 Ablation Study
Model Structure. We validate our approach through experiments (table 3). The in-context edit prompt (IC prompt) significantly outperforms direct instructions in training-fre free models (70% GPT score increase) and enhances editing after fine-tuning. Our LoRA-MoB design outperforms standard LoRA, boosting quality and success rates (13% GPT score increase) with fewer parameters. Limiting adaptation to the output projection layer ("Only MoE"") reduces performance, highlighting the need for comprehensive fine-tuning across all modules.

---

Fixed Seed
Inference Scaling
Fixed Seed
Inference Scaling
Fixed Seed
Inference Scaling
Change the Image so it appears be snowing.
Make the sky night instead of day
Add double rainbow to the
Uigua
Add the word "Elegance" above
Get rid of helmet the skateboarder
Convert the
towels
anime illustration
Figure 9: Ablation Inference- Time Scaling ($4.2). Our strategy significantly enhances edit quality. For example, with the instruction "get rid of the helmet," default fixed seed incorrectly removes the character's head-a flawed outcome prevented by VLM filtering.
Eei Doubao
Imnge-to Image Applications
22025
Ours
Hand Refinement
Stylization
Watermork Berissal
Relighting
Hawall acenery
Aold dlpkelee
bottee..
with sbiny Min
Figure 10: Applications ($4.3). Our method leverages DiT's original generation ability, producing harmonious results. Without additional tuning, it shows robust generalization across diverse tasks.
Inference Time Scaling. As shown in figs. 8 and 9, our strategy markedly improves editing perfor- mance, achieving 19% increase in SC score and a 16% boost in overall VIE-Score. Quantitative experiments across various settings (table 3) demonstrate that our approach significantly reduces computational cost (number of function evaluation, NFE) while substantially enhancing performance.
Data Efficiency, As shown in fig. and table 1. our method yields significant improvements with only 0.05M training samples, achieving a 180% GPT-score increase over our training- framework while using just 0.1% of samples required by prior models. This highlights the efficiency and effectiveness of our fine-tuning strategy.
4.3 Application
Harmonious and Versatile Editing. Our method leverages the powerful generative prior of large- scale Diffusion Transformers (DiT's) to create seamless, context-aware edits that blend naturally with the original image, automatically incorporating shadow effects and style alignment shown in figs. and 10. As a versatile image-to-image framework, it excels in tasks such as hand refinement and relighting (fig. 10) and holds potential for broader applications through task-specific fine-tuning.
5 Conclusion
In this paper, we present ICEdit, novel DiT-based instruction editing method that delivers state-of- the-art performance with minimal fine-tuning data, achieving an unmatched balance efficiency and precision. We first explore the inherent editing potential of generative DiTs in training- free context, proposing our in-context edit paradigm. We then enhance its editing quality and robustness through minimal fine-tuning with the mixture of expert structure. Additionally, we introduce an early filter inference-time scaling strategy, using VLMs select optimal early-stage outputs from multiple seeds, enhancing edit outcomes. Extensive experiments confirm our method's effectiveness and showease superior results. We believe this efficient, precise framework establishes new paradigm for balancing precision and efficiency in instructional image editing

---

Acknowledgements. This work was supported by the National Natural Science Foundation of China (62441617) and the Fundamental Research Funds for the Central Universities 0226-002-00000), This work was also supported by the Earth System Big Data Platform of the School of Earth Sciences, Zhejiang University.
References
[1] Tim Brooks, Aleksander Holynski, and Alexei Efros. Instructpix2pix: Learning to follow image editing instructions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 18392-18402, 2023.
[2] Kai Zhang, Lingbo Mo, Wenhu Chen, Huan Sun, and Yu Su. Magicbrush: A manually annotated dataset for instruction-guided image editing. Advances in Neural Information Processing Systems, 36, 2024.
[3] Shelly Sheynin, Adam Polyak, Uriel Singer, Yuval Kirstain, Amit Zohar, Oron Ashual, Devi Parikh, and Yaniv Taigman. Emu edit: Precise image editing via recognition and generation tasks. arXiv preprint arXiv:2311.10083 2023.
[4] Haozhe Zhao, Xiaojian Shawn Ma, Liang Chen, Shuzheng Si, Rujie Wu, Kaikai An, Peiyu Yu, Minjia Zhang, Qing Li, and Baobao Chang. Ultraedit: Instruction-basee fine- fine-grained image editing at scale, Advances in Neural Information Processing Systems, 77:3058-3093 2025.
[5] Tsu-Jui Fu, Wenze Hu, Xianzhi Du, William Yang Wang, Yinfei Yang, and Zhe Gan. Guid- ing instruction- based image editing via multimodal large language models. arXiv preprint arXiv:2309. /7/02, 2023.
[6] Zhen Han, Zeyinzi Jiang, Yulin Pan, Jingfeng Zhang, Chaojie Mao, Chenwei Xie, Yu Liu, and Jingren Zhou. Ace: All-round creator and editor following instructions via diffusion transformer. arXiv preprint arXiv:2410.00086, 2024.
[7] Chaojie Mao, Jingfeng Zhang. Yulin Pan, Zeyinzi Jiang, Zhen Han, Yu Liu, and Jingren Zhou. Ace++: Instruction-basse basedd image creation and editing via context-aware content filling. arXiv preprint 25X.::2501.02487, 2025.
[8] Shiyu Liu, Yucheng Han, Peng Xing, Fukun Yin, Rui Wang, Wei Cheng, Jiaqi Liao, Yingming Wang, Honghao Fu, Chunrui Han, Guopeng Li, Yuang Peng, Quan Sun, Jingwei Wu, Yan Cai, Zheng Ge, Ranchen Ming, Lei Xia, Xianfang Zeng, Yibo Zhu, Binxing Jiao, Xiangyu Zhang, Gang Yu, and Daxin Jiang. Steplx-edit: A practical framework for general image editing. arXiv preprint
2025.
[9] Yuzhou Huang, Linggin Xie, Xintao Wang, Ziyang Yuan, Xiaodong Cun, Yixiao Ge, Jiantao Zhou, Chao Dong, Rui Huang, Ruimao Zhang, et al. Smartedit: Exploring complex instruction- based image editing with multimodal large language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2362-83773 2024.
[10] Yingjing Xu, Jie Kong, Jiazhi Wang, Xiao Pan, Bo Lin, and Qiang Liu. Insightedit: Towards better instruction following for image editing. arXiv preprint arXiv:2411.17322 2024.
[11] Amir Hertz, Ron Mokady, Jay Tenenbaum, Kfir Aberman, Yael Pritch, and Daniel Cohen-Or. Prompt-to-pronee image editing with cross attention control. CoRR, abs/2208.0166 2022.
[12] Chenlin Meng, Yutong He, Yang Song, Jiaming Song, Jiajun Wu, Jun- Yan Zhu, and Stefano Ermon. SDEdit: Guided image synthesis and editing with stochastic differential equations. In International Conference on Learning Representations, 2022.
[13] Jiangshan Wang, Junfu Pu, Zhongang Qi, Jiayi Guo, Yue Ma, Nisha Huang, Yuxin Chen, Xiu Li, and Ying Shan. Taming rectified flow for inversion and editing. arXiv preprint vrXiv:2411.04744 2024.
[14] Omri Avrahami, Or Patashnik, Ohad Fried, Egor Nemchinoy, Kfir Aberman, Dani Lischinski, and Daniel Cohen-Or. Stable flow: Vital layers for training free image editing, 2024.
10

---

[15] Vladimir Kulikov, Matan Kleiner, Inbar Huberman- SaabermaanSpieglgglaa and Tomer Michaeli. Flowedit: Inversion-free text-based editing using pre-trained flow models. arXiv preprint arXiv:2412.08629 2024.
[16] Xuan Ju, Ailing Zeng, Yuxuan Bian, Shaoteng Liu, and Qiang Xu. Direct inversion: Boosting diffusion-based editing with 3 lines of code. arXiv preprint 1rXiv:2310.01500 2023.
[17] Tianrui Zhu, Shiyi Zhang, Jiawei Shao, and Yansong Tang. Kv-edit: Training-free image editing for precise background preservation, 2025.
[18] Xuan Ju, Ailing Zeng, Yuxuan Bian, Shaoteng Liu, and Qiang Xu. Pop inversion: Boosting diffusion-based editing with 3 lines of code. International Conference on Learning Representa- tions (ICLR), 2024.
[19] Mingdeng Cao, Xintao Wang, Zhongang Qi, Ying Shan, Xiaohu Qie, and Yinqiang Zheng. Masactrl: Tuning-free free mutual self-attention control for consistent image symthesis and editing. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages October 2023.
[20] William Peebles and Saining Xie. Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 4195-4205, 2023.
[21] Junsong Chen, Jincheng Yu. Chongjian Ge, Lewei Yao, Enze Xie, Yue Wu, Zhongdao Wang, James Kwok, Ping Luo, Huchuan Lu, and Zhenguo Li. Pixart-o:: Fast training of diffusion transformer for photorealistic text-to-image synthesis. arXiv preprint arXiv:2310000426 2023.
[22] Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, Dustin Podell, Tim Dockhor, Zion English, Kyle Lacey, Alex Goodwin, Yannik Marek, and Robin Rombach. Scaling rectified flow transformers for high-resolution image synthesis, 2024.
[23] Black Forest Labs. Flux: Official inference repository for flux.1 models, 2024. Accessed: 2024-11-12.
[24] Chaehun Shin, Jooyoung Choi, Heeseung Kim, and Sungroh Yoon. Large-scale text-to- image model with inpainting is zero-shot subject-driven image generator. arXiv preprint arXiv:2411.15466 2024.
[25] Yuxuan Zhang, Yirui Yuan, Yiren Song, Haofan Wang, and Jiaming Liu. Easycontrol: Adding efficient and flexible control for diffusion transformer. arXiv preprint arXiv:2503.07022 2025.
[26] Lianghua Huang, Wei Wang, Zhi-Fan Wu, Yupeng Shi, Huanzhang Dou, Chen Liang, Yutong Feng. Yu Liu, and Jingren Zhou In-context lora for diffusion transformers. arXiv preprint arxiv:2410.23775, 2024.
[27] Tan Zhenxiong, Liu Songhua, Yang Xingyi, Xue Qiaochu, and Xinchao Wang. Ominicontrol: Minimal and universal control for diffusion transformer. arXiv preprint 1.X59.2411.15098, 2024.
[28] Shaojin Wu, Mengqi Huang, Wenxu Wu, Yufeng Cheng, Fei Ding, and Qian He. Less-to- more generalization: Unlocking more controllability by in-context generation. arXiv preprint arXiv:2504.00 02 06000 2025.
[29] Nanye Ma, Shangyuan Tong, Haolin Jia, Hexiang Hu, Yu-Chuan Su, Mingda Zhang, Xuan Yang, Yandong Li, Tommi Jaakkola, Xuhui Jia, et al. Inference-time scaling for diffusion models beyond scaling denoising steps. arXiv preprint arXiv:2501.09732. 2025.
[30] Zikai Zhou, Shitong Shao, Lichen Bai, Zhiqiang Xu, Bo Han, and 7eke Xie. Golden noise for diffusion models: A learning framework. arXiv preprint 2xXiv:2411.09502. 2024.
[31] Robert A. Jacobs, Michael I. Jordan, Steven J. Nowlan, and Geoffrey E, Hinton. Adaptive mixtures of local experts. Neural Computation, 3(1):79-87) 87, 1991.
11

---

[32] Ron Mokady, Amir Hertz, Kfir Aberman, Yael Pritch, and Daniel Cohen- Or. Null-text inversion for editing real images using guided diffusion models. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 6038 6047, 2023.
[33] Narek Tumanyan, Michal Geyer, Shai Bagon, and Tali Dekel. Plug-and-plas diffusion features for text-driven image-to-image translation, 2022.
[34] Cong Wei, Zheyang Xiong, Weiming Ren, Xinrun Du, Ge Zhang, and Wenhu Chen. Om- niedit: Building image editing generalist models through specialist supervision. arXiv preprint arXiv:2411.07199 2024.
[35] Qifan Yu, Wei Chow, Zhongqi Yue, Kaihang Pan, Yang Wu, Xiaoyang Wan, Juncheng Li, Siliang Tang, Hanwang Zhang, and Yueting Zhuang. Anyedit: Mastering unified high-quality image editing for any idea. arXiv preprint arXiv:2411.15788 2024.
[36] Yi Yang, Yueting Zhuang, and Yunhe Pan. Multiple knowledge representation for big data artificial intelligence: framework, applications, and case studies. Frontiers of Information Technology & Electronic Engineering. 22 22(12):1551-1555 2021.
[37] Wenguan Wang, Yi Yang, and Yunhe Pan. Visual knowledge in the big model era: Retrospect and prospect. Frontiers of Information Technology & Electronic Engineering, 26(1):1-19, 2025.
[38] Dewei Zhou, You Li, Fan Ma, Zongxin Yang, and Yi Yang. Mige++: Advanced multi-instance generation controller for image synthesis. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024.
[39] Dewei Zhou, Ji Xie, Zongxin Yang, and Yi Yang. 3dis: Depth decoupled instance synthesis for text-to-image generation. In ICLR, 2025.
[40] Zechuan Zhang, Zongxin Yang, and Yi Yang. Sifu: Side-view conditioned implicit function for real-world usable clothed human reconstruction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9936-9947, 2024.
[41] Ruisi Zhao, Zechuan Zhang, Zongxin Yang, and Yi Yang. 3d object manipulation in a single image using generative models. arXiv preprint arXiv:2501.129335 2025.
[42] Sayak Paul. Flux. --ve--ddi--vO,, 2025. Accessed: 2025-02-21.
[43] Lianghua Huang, Wei Wang, Zhi-Fan Wu, Huanzhang Dou, Yupeng Shi, Yutong Feng, Chen Liang, Yu Liu, and Jingren Zhou Group diffusion transformers are unsupervised multitask learners. arXiv preprint arxiv:2410. 2024.
[44] Chenlin Meng. Yutong He, Yang Song, Jiaming Song, Jiajun Wu, Jun- Yan Zhu, and Stefano Ermon. Sdedit: Guided image synthesis and editing with stochastic differential equations. arXiv preprint arXiv:2 00000333 2021.
[45] Edward Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. In International Conference on Learning Representations (ICLR), 2022.
[46] Albert Q Jiang, Alexandre Sablayrolles, Antoine Roux, Arthur Mensch, Blanche Savary, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Emma Bou Hanna, Florian Bressand, et al. Mixtral of experts. arXiv preprint arXiv:2401 240.00800 2024.
[47] Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. Deepseek-v3 technical report. arXiv preprint arX1v:2412.19437, 2024.
[48] Yaron Lipman, Ricky TQ Chen, Heli Ben-Hamu, Maximilian Nickel, and Matt Le. Flow matching for generative modeling. arXiv preprint arXiv:2210.02747 2022.
[49] Xingchao Liu, Chengyue Gong, and Qiang Liu. Flow straight and fast: Learning to generate and transfer data with rectified flow. arXiv preprint arXiv:2209.0300 2022.
12

---

[50] Kevin Frans, Danijar Hafner, Sergey Levine, and Pieter Abbeel. One step diffusion via shortcut models, arXiv preprint arXiv:2410. 12557, 2024.
[51] Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang. Shijie Wang, Jun Tang, al. Qwen2. 5-vl technical report. arXiv preprint 2202::3302..32233 2025.
[52] Lin Chen, Jinsong Li, Xiaoyi Dong, Pan Zhang, Conghui He, Jiaqi Wang, Feng Zhao, and Dahua Lin. Sharegpt4v: Improving large multi-mode models with better captions, 2023.
[53] Zongxin Yang, Guikun Chen, Xiaodi Li, Wenguan Wang, and Yi Yang. Doraemongpt: Toward understanding dynamic scenes with large language models (exemplified as a video agent). In International Conference on Machine Learning, pages 55976-55997. PMLR, 2024.
[54] Nur Muhammad Mahi Shafiullah, Chris Paxton, Lerrel Pinto, Soumith Chintala, and Arthur Szlam. Clip-fields: Weakly supervised semantic fields for robotic memory. arXiv preprint 1rXiv:2210.05663, 2022.
[55] Jack Hessel, Ari Holtzman, Maxwell Forbes, Ronan Le Bras, and Yejin Choi. Clipscore: A reference-free evaluation metric for image captioning. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, EMNLP 2021, Virtual Event / Punta Cana, Dominican Republic, 7-11 November, 2021, pages 7514-7528. Association for Computational Linguistics, 2021.
[56] Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy Vo, Mare Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07198, 2023.
[57] Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging properties in self-supervised vision transformers. In Proceedings of the International Conference on Computer Vision (ICCV), 2021.
[58] OpenAl. Gpt-4 technical report. arXiv preprint arXiv:2303.0877: 2023.
[59] Max Ku, Dongfu Jiang, Cong Wei, Xiang Yue, and Wenhu Chen. Viescore: Towards explainable metrics for conditional image synthesis evaluation. arXiv preprint arXiv:2312. 2023.
[60] Yichun Shi, Peng Wang, and Weilin Huang. Seededit: Align image re-generation to image editing. arXiv preprint 1..0666,, 2024.
[61] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björ Ommer. High-resolution image synthesis with latent diffusion models, 2021.
[62] Zexu Pan, Zhaojie Luo, Jichen Yang, and Haizhou Li. Multi-modal attention for speech emotion recognition. arXiv preprint arXiv:2009.04 2020.
[63] Jianlin Su, Murtadha Ahmed, Yu Lu, Shengfeng Pan, Wen Bo, and Yunfeng Liu. Roformer: Enhanced transformer with rotary position embedding, Nenrocomputing, 168::27763, 2024.
[64] Konstantin Mishchenko and Aaron Defazio. Prodigy: An expeditiously adaptive parameter-free learner. In Forty-first International Conference on Machine Learning, 2024.
[65] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer: High- resolution image synthesis with latent diffusion models. In IEEF/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022, New Orleans, LA, USA, June 18-24, 2022, pages 10674-10685. IEEE, 2022.
[66] Mingdeng Cao, Xuaner Zhang, Yinqiang Zheng, and Zhihao Xia. Instruction- based image manipulation by watching how things move. arXiv preprint arXiv:2412. 2024.
[67] Xichen Pan, Satya Narayan Shukla, Aashu Singh, Zhuokai Zhao, Shlok Kumar Mishra, Jialiang Wang, Zhiyang Xu, Jiuhai Chen, Kunpeng Li, Felix Juefei-Xu, at. Transfer between modalities with metaqueries. arXiv preprint 2rX 2025.
13

---

[68] Enze Xie, Junsong Chen, Yuyang Zhao, Jincheng Yu, Ligeng Zhu, Yujun Lin, Zhekai Zhang, Muyang Li, Junyu Chen, Han Cai, et al. Sana 1.5: Efficient scaling of training-tin time and inference-time compute in linear diffusion transformer. arXiv preprint arXiv:2501. 2025.
[69] William Fedus, Barret Zoph, and Noam Shazeer. Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity. Journal of Machine Learning Research, 23(120):1-39, 2022.
[70] Shaoxiang Chen, Zequn Jie, and Lin Ma. Llava-mole: Sparse mixture of lora experts for mitigating data conflicts in instruction finetuning milms, arXiv preprint arXiv:24011 16160, 2024.
14

---

1. Claims
Question:
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Yes
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: the Supplementary materials
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimesss
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
Answer:
[NA]
Justification: the paper does not include theoretical results
Guidelines:
15

---

The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided not)?
Question:
Answer: [Yes]
Justification: Included
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution
specific model and empirical evaluation, may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [No]
Justification:
The code and data will be available upon acceptance.
16

---

Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.co cc/
for more details.
While encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https: / /nips. wwtppblcc/opeecccccccccccccc wwtici/ggg////eeeeeeeeeiiieiisee
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if' applicable).
Providing much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer:
[Yes]
Justification:
As detailed in Sup.Mat.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation or the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
17

---

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative enror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: In Sup.Mat.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics https: //tp://eeeeu.ueeeeeeeeeee cc cc/public/Etaeie /euuuideethissuuieeelll csGuidel ines?
Answer: [Yes]
Justification: yes
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. if there is special consid- cration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: Map.Mat.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no socictal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g.. disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinfornation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
18

---

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: We publicly available data.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification:
We cite the original paper and dataset.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website). the copyright and terms of service of that source should provided.
assets are released, the license, copyright information, and terms use in the package should be provided. For popular datasets, paperswithcode com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: NA
19

---

Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moddl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: Does not invlove.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation,
or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: NA
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods this research? Note that if the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality the research, declaration is not required.
Answer: [NA]
Justification: the core method development in this research does not involve LLMss.
Guidelines:
20

---

The answer NA means that the core method development in this research does not involve LLMs as any important. original, non non-standard components.
Please refer to our LLM policy (uttps://neun)) http://aaii...ee/oeeeeeeeeeeeeee cc/Confereecee//20000 for what should or should be described.
A Preliminary
Diffusion Transformer (DiT) Model [20], employed in architectures such FLUX.1 [23], Stable Diffusion [61], and Aix [21], utilizes a transformer as denoising network to iteratively refine noisy image tokens. A DiT model processes two types of tokens: text condition tokens Cr E RMxxo and noisy image tokens X IR IR- where d is the embedding dimension, and M and N represent the number of text and image tokens, respectively. These tokens maintain consistent shapes as they pass through multiple transformer blocks within the network.
FLUX and FLUX-Fill are based on hybrid architecture that combines multimodal and parallel diffusion transformer blocks, scaled to 12B parameters. FLUX-Fill serves as both an inpainting and outpainting model, enabling modification of real and generated images using text description and binary mask. Both models are built on flow matching, a general and conceptually simple method for training generative models, with diffusion
special case.
In these models, each DiT block consists of layer normalization followed by Multi-Modal Attention (MMA) [62]. which incorporates Rotary Position Embedding (RoPE) [63] to encode spatial informa- tion. The multi-modal attention mechanism projects the position-encode tokens into query Q. key K, and value V representations. This enables the computation of attention across all tokens:
QK
MMA X]) = softmax
V,
Vd
where (CT:X] denotes the concatenation of text and image tokens, facilitating bidirectional attention.
B
Implementation Details
B.1
Dataset
As outlined in the main text, our fine-tuning dataset comprises 50K samples, volume substantially smaller than the training data utilized by state-of-the-an (SOTA) models. The dataset is exclusively derived from open-access resources: MagicBrush [2] (9K samples) and OmniEdit [34] (40K randomly selected samples). The distribution of task types within our dataset is presented in Table table 4. It is worth noting that the dataset was not rigorously curated, and illustrated fig. 11, still contains number of problematic samples. Due to the time-intensive nature of data cleaning, we opted not to perform additional filtering at this stage. However, we anticipate that future efforts involving meticulous curation and the incorporation of higher-quality datasets could further enhance model performance.
Importantly, our model demonstrates superior performance compared to those trained on Mag- icBrush [2] and the full OmniEdit 1.2M dataset [42], despite utilizing significantly less data. This underscores the effectiveness of our in-context edit methodology, suggesting that its advantages extend beyond the dataset itself.
Table 4: Dataset Statistics by Task Type
Task Type
Removal
Addition
Swap
Attribute Mod.
Style
Total
Count
13,272
11,938
5,823
11,484
10,530
53,047
B.2 Training-free Framework
Here we elaborate on the implementation of the in-context edit framework based on the T21 DIT. Our core idea is to guide the T2I DiT to generate side-by-side images, where the left side reconstructs
21

---

Remove the man
Add Text
Remove the man
atn gut pilsondy glga gainninn nimn
visss, Zag Zaglos ghs on a has pridin Litapy ieslody othn lsst drsppdde soreem
Figure 11: Presence of Some Low-Quality Samples the Dataset. Our examination reveals that the public dataset contains number of suboptimal samples, which could potentially impact the performance of our method.
the reference image, enabling the right side incorporate features from the left image and perform reference based generation.
Specifically, we first perform image inversion using the T2I model [13, 14], retaining the value features from the attention layers during the inversion process. Subsequently, we generate images using the in-context edit prompt. To achieve this, we concatenate the noise obtained from the inversion process on the left side with random noise of the same size on the right side as the input noise. is crucial to apply positional encoding to the concatenated random noise to distinguish it from the inversion noise.
During the denoising steps, we inject the retained value features from the inversion process into the inversion noise portion, while leaving the random noise portion unaltered. This ensures that the left side of the image reconstructs the source image, while the right side, by leveraging the injected value features through the attention mechanism, generates result with the same identity but conforming to the editing instructions. fig. 12 exhibits some results of this framework, which shows great potential for instruction- based image editing, despite minor artifacts in identity preservation and layout maintenance.
B.3
Finetuning and Inference
Our model employs Mixture of Experts (MoE) module with 4 expert LoRAs, each of rank 32, The routing network consists of single linear layer with TopK set to 1, which balances increased parameter capacity with computational efficiency. For other modules, we use standard LoRA with the same rank of 32, and all LoRA alpha values are set equal to their respective ranks.
The model is trained with batch size of and gradient accumulation effective batch size of 2. We utilize the Prodigy optimizer [64] with safeguard warmup and bias correction enabled, configured with weight decay of 0.01. Training is conducted on 4 A800 (80G) GPUs for one day, while inference is performed on an A100 (40G) GPU. Following [27], we optimize the parameters by minimizing the reconstruction loss between the model's predictions and the ground truth. Notably, we do not incorporate an additional load-balancing loss for the routing network, as our experiments reveal that the expert usage predicted by the router is well-balanced, with no single expert being overused. However, this approach may not hold for more complex MoE architectures, and we plan to explore this further in future work.
2 steps, resulting in
During training, input images are resized to 512x512 pixels, then formatted into 512x 1024 diptychs (side-by-side image pairs). Here we report the detailed VRAM consumption under various settings.
Memory Usage with Gradient Checkpointing:
512 resolution: 37 GB VRAM (batch size
1)
768 resolution: 39 GB VRAM (batch size 1)
22

---

Make the girl wear pink sunglasses
Change the sunglasses to white headless duck tongue cap
Alter the car color from red to blue
Make the backgroundsevere lightning storm
FUSH
Change the street appear rain-soaked.
Add the word "FLUSH" to the toilet seat.
Figure 12: Results of Praining-free framework. Our training-fiee free framework demonstrates signifi- cant potential for instruction-bas based image editing, despite minor artifacts in identity preservation and layout maintenance.
1024x 1024 resolution: 42 GB VRAM (batch size 1)
Memory Usage without Gradient Checkpointing:
512x512 resolution: 60GB VRAM (batch size = 1)
768x768 resolution: 77 GB VRAM (batch size 1)
1024x 1024 resolution: Out of memory (OOM)
Given that gradient checkpointing significantly slows training speed and the baseline models are optimized for 512 resolution, we train our model at 512 resolution without gradient checkpointing to balance computational efficiency and memory constraints.
For both training and inference, we use the in-context (IC) prompt: "A diptych with two side-by-side images of the same scene. On the right, the scene is exactly the same as the left but (instruction): " This allows the model to directly utilize editing instructions without additional adjustments. During inference, we set the guidance scale to 50 and the number of inference steps 50. When employing the early filter inference time scaling strategy, we randomly select 6 noise samples for fast inference with 10 steps, then use Qwen2.5 VL 72B (via API) to pairwise evaluate images and select one that best aligns with the editing instruction for the next round of filtering. The prompt used for filtering shown in in-fig- We also conduct experiments on parameter settings for inference time scaling, detailed in appendix D.
23

---

"You are multimodal large-language model tasked with evaluating images generated by text-to-image model. You are given three images:
1. The edited imagel.
2. The edited image2.
3. The original image.
The edit image_ I and edit image 2 are generated by the edit prompt "(instruction)" and the original image. Please carefully compare the two edited images with the original image and answer the question: which of the two edited images (edit image 1 and edit image_ 2) better aligns with the edit prompt "instruction)' and has higher quality and less artifacts.
Moreover, the areas that not intended for editing should remain identical to the original image and the style should be consistent with the original image unless the edit prompt specifies otherwise.
Answer with or 2 and briefly explain why."
Figure 13: Prompt used for VLMs during inference time scaling,
B.4
Evaluation Details
B.4.1
Baseline Details
In the main paper, we compare our method with both traditional and state-of-the-art models on the instruction-based image editing task. Below, provide brief overview of each baseline method:
InstructPix2Pix [1]: This method fine-tunes Stable Diffusion [65] using automatically generated instruction-bast image editing data. It enables instruction- image editing during inference without requiring any test-time tuning.
MagicBrush [2]: MagicBrush curates a well-structured editing dataset with detailed human annotations and fine-tunes its model using the InstructPix2Pix [1] framework. This approach emphasizes high-quality data for improved editing performance.
Emu Edit [3]: Emu Edit is closed-source model trained on large-scale dataset of 10M samples for image editing. It introduces the Emu test. dataset to evaluate editing quality.
Since only the results on the Emu test set are publicly available, we compare our method with Emu Edit exclusively on this dataset.
Flux Edit [42]: This is an open-source model on Hugging Face, fine-tuned on the Flux. dev model using 1.2M editing pairs from OmniEdit [34]. We include this baseline to highlight that our improvements stem primarily from our proposed framework rather than the base model (FLUX).
FLUX Fill [23]: It is a 12 billion parameter rectified flow transformer capable of filling areas in existing images based text description. We use it as our training-free baseline for comparison with our framework.
RF-Solver-Edit [13]: This is a training-free image editing framework based on FLUX. which requires image inversion followed by feature injection for editing, Since it cannot directly utilize instruction prompts, we provide the input and output captions from the dataset for generation. Due to the use of output captions, its CLIP-out metrie tends to be higher.
ACE++ [7]: ACE+ an enhanced version of ACE [6], trained on FLUX with a richer dataset of 700M samples, including 54M editing pairs. It integrates reference image generation, local editing, and controllable generation into a single framework. However, our tests reveal that it underperforms instruction-bast based tasks.
24

---

SeedEdit [60]: A wate-of-the-aa commercial model (based on DiT) trained on meticu- lously curated large-scale dataset. This model achieves an optimal balance between image reconstruction (preserving original content) and image re-generation (creating novel visual content). Due its limited accessibility through a web-based interface without API support, we conducted evaluations using randomly curated subset of 50 images from the EMU test dataset and in-the-wild samples. Each image was processed manually through SeedEdit's interactive web interface to ensure consistent and reliable results.
B.4.2 Metrics
MagicBrush is designed to evaluate the single-turn and multi-turn image editing capabilities of models. provides annotator-defined instructions, editing masks, and ground truth images. Following the setup of [2, 4], we use the LI metric to measure pixel-level differences between the generated image and the ground truth. Additionally, we employ CLIP similarity and DINO similarity to evaluate the overall resemblance to the ground truth.
The Emu Edit Test addresses bias and overfitting in mnnotator-defined datasets by eliminating ground truth images, while enhancing diversity through creative and challenging instructions paired with high-quality captions that capture essential elements in both source and target images. Consistent with [4], we evaluate the model's ability to preserve source image elements using CLIP image similarity and DINO similarity between source images and edited images. For text-image alignment, we employ CLIP-Out to measure the correspondence between local descriptions and generated image embeddings. Following [2, 4]. we use ViT-B/32 for CLIP and dino 16 for DINO embeddings.
Notably, we exclude CLIP text-image direction similarity due to its inconsistency in reflecting editing quality, as it frequently assigns low scores well-edited results (see fig. 14). Instead, we incorporate GPT-40 for complementary scoring. Furthermore, to mitigate benchmark quality issues such as placeholder captions (e.g.. 'a train station in city') or identical source-target captions- filter out incorrect cases before evaluation, following [4]. While the Emu Edit Test successfully reduces image-level bias by omitting ground truth images, the evaluation metries still implicitly assess the model's editing capability through feature-level comparisons.
Make the sky night instead of day
Add a double rainbow to the sky
Input
Output
Input
Output
CLIP dir: -0.021
CLIP dir: 0.403
Figure 14: Our analysis reveals limitations in using CLIP direction scores for editing performance evaluation. Notably, the while the second case shows a failed edit with an anomalously high score, highlighting the metric's inconsistency.
first example demonstrates successful edit despite receiving a low score,
GPT Evaluation. Following [59, 34, 10], we employ GPT-4o to compute the VIE-score for assessing editing performance, which better aligns with human perceptual judgment. The evaluation prompts are detailed in figs. 15 and 16, where the SC score quantifies instruction adherence and editing accuracy, while the PQ score measures perceptual quality and naturalness. Consistent with [34], we apply prreshodd-bssed binarization to the SC score, mapping it to the [0.1] range as weighting function for the final GPT evaluation score.
C Discussion
Limitations.
Despite achieving state-of-the-art editing performance with efficient tuning, our method suffers from the following limitations (fig. 17): (1) Object Movement: Instructions requiring spatial
25

---

You professional digital artist tasked with evaluating the effectiveness of Al-generated images based on the given rules. Provide your output as follows (keep reasoning concise and short): { "score": [<score|>, <score2>]. Treasoning":"" "..." 3.
Do output anything else. Two images will be provided: the first is the original Al-generated image, and the second is an edited version of the first. Your objective is to evaluate how successfully editing instruction has been executed images might sometimes look identical due editing failure. Use scale from 0 to 10 for two scores:
second image. Note that the two
1. Editing Success Score:
O: The edited image does not follow the editing instruction at all.
10: The edited image follows the instruction perfectly.
If the object in the instruction is absent in the original image, score is 0.
2. Degrec of Overediting Score:
0: The edited image is completely different from the original.
10: The edited image recognizably minimally edited yet effective version of the original.
Output the scores in list: "score": [<editing_ success>, <overediting>), where 'scorel' evaluates editing success and 'score2' evaluates the degree overediting.
Figure 15: Prompt used for evaluating SC score.
You are a professional digital artist tasked with evaluating the effectiveness of an Al-generated image. All images and humans depicted are Al generrera so privacy or confidentiality concern Focus solely technical quality and artifacts, ignoring whether the context appears natural. Evaluate based on:
Distortions
Unusual body parts or proportions Unnatural object shapes
Rate the image on scale from 0 to 10: O: Significant Al-artifacts present.
10: Artifact-free image.
Provide your output as: { ("score": <integer>, "reasoning": "..." keeping reasoning concise and short. Do not output anything else.
Figure 16: Prompt used for evaluating PQ score.
relocation (e.g.. "move the chair to the corner") may fail due to insufficient exposure to motion- oriented data in general editing datasets. Specialized datasets like [66] could address this through targeted fine-tuning. (2) Semantic Understanding Limitations: While demonstrates strong text encoding capabilities, its semantic understanding remains constrained, particularly in resolving polysemous terms (e.g., confusing "mouse" (computer device) with "mouse" (animal)). This limitation stems from its limited contextual disambiguation ability. Future work could incorporate MLLM-basee modules [8, 67] to improve semantic fidelity. (3) VLM Efficiency: Our inference time scaling relies on Qwen VL 72B ensure accurate quality assessment, smaller models (7B) often misjudge edit quality. Recent advances [34, 68] demonstrate that distilled 7B VLMs can achieve GPT-4o-level performance through specialized training, offering a promising path toward efficiency improvements.
Broader Impact. Our work may contribute to field in two aspects: (1) Accessible Image Editing Tooly: The parameter-efficien design could reduce computational resource requirements, potentially benefiting small-scale developers and individual creators. (2) Ethical Considerations: Like most generative models, our technology could potentially be misused for creating deceptive content
26

---

ISS CAD
Move the blue chair to the table
Add "mouse" " the mouse.
Figure 17: Some failure cases of our methods, such object movement, semantic ambiguity.
Editing Score vs. Training Data Size
0.8
0.7
0.6
GPT Evaluation Score
0.3
0.2
0.1
Fitted Curve
Data Points
0.0
0
10
20
30
40
50
60
70
Training Data Size (k)
Figure 18: The plot demonstrates that the model's performance improves with increasing training data, with the growth rate gradually plateauing.
(e.g., deepfakes or manipulated media). We advocate three essential safeguards for responsible
deployment: (1) implementing content provenance standards to ensure traceability, (2) maintaining human oversight in sensitive applications such as news media and legal documentation, and (3) fostering collaboration between Al developers and domain experts to establish ethical guidelines specific instruction-bas based editing scenarios. These measures align with emerging AI governance frameworks while respecting creative freedoms.
D
Additional Results
D.1 Ablation
MoE Settings. We conduct series of experiments to investigate the influence of different expert configurations. As shown in table we observe a significant improvement model performance (measured by GPT-based evaluation scores) when increasing the expert rank from 8 to 32 or the number of experts from to 4. However, further increasing the number experts does not lead to notable gains in editing performance, while significantly increasing the model's parameter count, thereby reducing efficiency. We hypothesize that as the number experts grows, the routing mechanism becomes more challenging, potentially requiring more sophisticated routing network designs and corresponding loss constraints to achieve stable performance. Future research could explore these aspects optimize the trade-off between performance and efficiency.
27

---

Table 5: Ablation of different settings.
Expert Number
Expert Rank
Params
CLIP-I
CLIP-Out 1
GPT f
1
32
120M
0.892
0.300
0.59
4
120M
0.920
0.303
0.58
4
32
214M
0.907
0.305
0.68
6
32
270M
0.914
0.305
0.66
8
32
335M
0.907
0.304
0.61
Inference Scaling Settings. We explore the impact of different parameter configurations our early filter inference time scaling strategy, as detailed table.3(b) in the main paper. Specifically, we investigate the choice of evaluator (VLM vs. CLIP), the number of random noise seeds, and the selection of early inference steps (early steps). The computational efficiency and accuracy are measured using NFE (Number of Function Evaluations, the cumulative compute count) and GPT-based evaluation scores. The total inference steps are fixed at 50, and NFE is calculated as 50 + (N (Num_noise x Early_step).
Our experiments reveal that using 10 early steps outperforms steps, as DiT may fail to generate sufficiently high-quality results with only 4 steps, leading to inaccurate VLM judgments. Increasing the number of initial noise seeds improves performance significantly from 1 to 6, but the gains diminish from 6 to 12. We attribute this to the model's 's robust editing performance across most noise samples, resulting in greater improvements compared using a single noise seed.
Additionally, we adopt the strategy from [29], which uses traditional metrics like CLIP to filter results by selecting the image-text pair with the highest CLIP score after full-step inference. However, this approach increases computational cost while degrading GPT-based evaluation scores, indicating that CLIP-based scoring does not align well with human visual preferences.
Data Efficiency. We investigated how editing performance scales with training data volume, as shown in fig. 18. Fine-tuning with just 10K samples markedly improves performance over training- free approaches. As data size increases, editing capability continues to rise, though the rate of improvement gradually diminishes. Notably, training with 70K samples yields minimal gains 50K, suggesting convergence under this setup. Future work could explore the effects of larger-scale parameter configurations and more high-quality editing data.
D.2
Analysis on Expert Selection Patterns
Implicit Expert Routing Across Layers and Steps. In our current design, the MoE module does not explicitly assign specific expert based task type instruction semantics. Instead, the MoE is integrated into each layer of the DiT backbone (e.g., 57 layers in Flux. Fill), and expert selection is every layer, diffusion step, and token. As described in the main paper, each token at each layer and timestep is dynamically routed to one of the experts. For instance, generating single image with 28 diffusion steps results in 28 57 1,596 expert selection events per token. These selections are distributed across all available experts and vary throughout the generation process, rather than consistently favoring particular expert for given task. Therefore, expert usage is inherently implicit and fine-grained, making direct attribution to instruction types non-trivial.
performed independently
Preliminary Analysis of Expert Usage Patterns. To further explore this aspect, we conducted targeted analysis of expert usage across different editing task types. Specifically, we selected three representative instruction categories from the EmuEdit test set-Addition, Removal, and Style Editing and ran 50 inference samples for each category. We then recorded the expert selection frequencies across all layers, diffusion steps, and tokens (38,133,76 selections per inference). The distribution of expert usage summarized in Table 6.
D.3 Top-K Expert Selection.
We choose the expert TopK value to be 1, as it offers a strong trade-off between computational efficiency and predictive performance. Activating only single expert per token significantly reduces
28

---

Table 6: Distribution of expert selection frequencies across editing task types. The MoE module demonstrates balanced expert utilization without strong bias toward specific tasks.
Task Type
Expert 0
Expert
Expert 2
Expert 3
Addition 25.4%
25.4%
24.1%
25.0%
Removal
25.9%
26.2%
23.2%
24.7%
Style Editing
24.7%
24.6%
25.0%
25.7%
FLOPs, memory usage, and communication overhead-erittcal factors for both training and real-time inference in deployment settings. This design choice is consistent with prior work: for instance, Switch Transformers [69]. LLaVA-MoLE [70], and Mixtral [46] collectively demonstrate that Top-1 routing enables scalable sparse MoE training with minimal accuracy loss, while higher TopK values may lead to diminishing returns and higher computational costs.
We also experimented with TopK 2 and TopK 4 using four total experts. The results on the EmuEdit test set are summarized in Table
Table 7: Performance comparison under different TopK values. Selecting more experts slightly degrades performance while increasing computational cost.
TopK Value
CLIP-1
CLIP-OUT1
TopK (Ours)
0,907
0.305
TopK =
0.891
0.301
TopK =
0.866
0.298
As shown above, increasing the number of selected experts leads slight but consistent drop in performance. Therefore, we adopt TopK = 1 as the optimal setting. We hypothesize that the lack of improvement with larger may stem from the absence of additional constraints guiding expert selection. Incorporating auxiliary losses or regularization to encourage more effective expert routing represents a promising direction for future work.
D.4 Qualitative Results
Here we present additional qualitative results, including comparisons with baseline models (figs. 19 and 20), editing results with different initial noise configurations (fig. 21), and a broader range of editing examples (fig. 22). These results further illustrate the effectiveness and versatility of our approach across diverse scenarios.
29

---

Instruction
Otiginal
Ours
EmuEdit
UltraEdit
MagicBrush
InstruedP2P
Insert the climate was rainy.
Convert the image into Gothic painting.
Change the image snowing lightly.
Instead of wood give the table white plastic texture
Figure 19: Comparison with baseline models on Emu Edit test set. Q Zoom in for detailed view.
30

---

Source
Ours
ScedEdit
UltraEdit
FINOR
PINGAA
PREKO
girl wear putterms.
Make the woman hold rod can in her loft hand and turn the background into white.
Add the word WIMBLEDON the
Add fork the left side of plate.
Remove all the trees.
Figure 20: Comparison with baseline models. Q Zoom in for detailed view.
31

---

Change the background to Hawaii scenery
Add the text "Aloha Hawaii" white color
Aloha Hawai
Aloha Hawaii
Aloha Hawari
Aloha Hawaii
Change the watercolor painting
painting
Figure 21: With different initial noise, methods generate diverse results.
32

---

Remove the purple flower's petal
Remove the shoes
golden girl's head
Put white plaid shirt the
Change the
want sketch painting
Make cloud golden
hydrant spill water
Lighten the image
Fix her fingers
Figure 22: More editing results of our method.
33


Reviews:
[{"review_id": "1JB4DepU0i", "reviewer": "Reviewer_oDrm", "summary": "This paper tackles the problem of instruction-based image editing, which aims to modify images via natural language prompts while balancing precision and efficiency. The authors propose ICEdit, a novel method that leverages the capabilities of Diffusion Transformers (DiTs) through (1) an in-context editing paradigm, (2) minimal parameter-efficient fine-tuning, and (3) an inference-time filtering strategy using Vision-Language Models. ICEdit achieves state-of-the-art results with only 0.1% of the training data and 1% of trainable parameters compared to existing approaches.", "questions": "While I believe the paper is of high quality overall, I encourage the authors to further validate the diptych-based editing framework and provide a deeper analysis of expert selection behavior in the LoRA-MoE module.", "limitations": "yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths\n\n1. The overall quality of the paper is high. \n- The writing and illustrations are clear and well-organized, making the proposed method easy to understand. The paper also includes extensive analysis, such as an exploration of scaling effects with respect to training data — an aspect I was particularly curious about while reading. Furthermore, important implementation details, including prompts for VLM-based inference-time scaling and evaluation setups, are clearly provided.\n\n2. The idea of diptych-based image editing and the corresponding method design are interesting and appear to be novel.\n\n3. ICEdit demonstrates impressive performance, achieving a strong balance between editing quality and training efficiency. This claim is well-supported by comprehensive experimental results.\n\nWeaknesses\n\n1. While the diptych-based image editing formulation is compelling, its effectiveness is not fully validated.\n- To better isolate and validate the contribution of the diptych-based design, it would be helpful to compare against a baseline that does not use the diptych structure but retains other components, such as the LoRA-MoE module and training dataset. In this setup, the baseline would generate an edited image solely based on the original image and instruction.\n\n2. Lack of analysis on the expert selection patterns in the LoRA-MoE module.\n- The paper claims that the LoRA-MoE design improves editing quality, and the final model appears to select one LoRA expert among four. It would be valuable to investigate whether there are consistent patterns in expert selection depending on factors such as task type or instruction category.", "was_revised": true, "final_justification": "I carefully read the authors’ rebuttals and the reviews from other reviewers. The rebuttals effectively address my original concerns on the effectiveness of diptych-based designs and on the LoRA-MoE model. Thus, I will maintain my original rating of “accept.”", "rebuttal": "Thank you sincerely for your positive and encouraging feedback, including remarks such as **“The overall quality of the paper is high,” “The writing and illustrations are clear and well-organized,” “Important implementation details are clearly provided,” “The idea and the corresponding method design are interesting and appear to be novel,” and “Demonstrates impressive performance.”**  We truly appreciate your thoughtful evaluation, and we’re glad to see that the key strengths of our work were recognized.\n\nWe address your inquiries and concerns point by point in the following responses.\n\n- **Answer to Weakness 1 and question 1:** **Ablation Study on the Diptych-Based Design**\n    1. Thank you for your insightful and constructive suggestion. Following your advice, we conducted an ablation study by removing the diptych structure. Instead, we adopted a channel-wise concatenation of the reference image and noise (as in InstructPix2Pix) as input to the model. We retained the MoE-LoRA module and fine-tuned on Flux.1 Dev using the **same training dataset** and **the same number of GPUs**. \n        \n        The performance comparison on the EmuEdit benchmark is summarized below:\n        \n        | Model Variant | CLIP-I ↑ | CLIP-OUT ↑ | DINO ↑ |\n        | --- | --- | --- | --- |\n        | **ICEdit (Ours)** | 0.907 | 0.305 | 0.866 |\n        | w/o Diptych | 0.837 | 0.275 | 0.748 |\n        | FluxEdit (baseline) | 0.852 | 0.282 | 0.760 |\n        \n        As shown above, **removing the diptych structure leads to a noticeable performance drop across all metrics, confirming the effectiveness of our proposed design.**\n        \n    2. A similar observation can be found in one of the baseline models already included in our paper. As shown in Table 1 and Table 2, the **FluxEdit** baseline—which also uses a channel-wise concatenation of the reference image and noise, and is trained from scratch on the full 1.2M OmniEdit dataset—achieves relatively poor results, further supporting our findings.\n    3. We believe the performance degradation in the non-diptych setting stems from the fact that **channel-wise concatenation alters the input encoding pattern, forcing the model to adapt to a different image structure and latent space.** This often destabilizes the output unless the model undergoes heavy retraining, which is especially challenging for large-scale DiT backbones. We find this to be a very meaningful point and will include a discussion on it in the revised version of the paper.\n    \n- **Answer to Weakness 2: Expert Selection Patterns in the LoRA-MoE Module**\n    \n    Thank you for your thoughtful question. We agree that analyzing expert selection behavior is a valuable direction, and we address it in two parts below.\n    \n    1. **Implicit Expert Routing Across Layers and Steps**\n        \n        In our current design, the MoE module does not explicitly assign one expert based on task type or instruction semantics. Instead, the MoE is integrated into **each layer** of the DiT backbone (e.g., 57 layers in Flux.1 Fill), and **expert selection is performed independently at every layer, every diffusion step, and for each token**. As described in the main paper (lines 161–165), each token at each layer and timestep is routed to one of the experts dynamically. For example, generating a single image with 28 diffusion steps results in **28 × 57 = 1,596 expert selection events per token**. These selections are distributed across all available experts and vary throughout the generation process, rather than consistently favoring a particular expert for a given task. Therefore, expert usage is inherently **implicit and fine-grained**, making direct attribution to instruction types non-trivial.\n        \n    2. **Preliminary Analysis of Expert Usage Patterns**\n        \n        To further explore your suggestion, we conducted a targeted analysis of expert usage across different editing task types. Specifically, we selected three representative instruction categories from the EmuEdit test set—**Addition**, **Removal**, and **Style Editing**—and ran **50 inference samples** for each category. We then recorded the expert selection frequencies across all layers, diffusion steps, and tokens (38,133,76 selections per inference).\n        \n        The distribution of expert usage is summarized below:\n        \n        | Task Type | Expert 0 | Expert 1 | Expert 2 | Expert 3 |\n        | --- | --- | --- | --- | --- |\n        | Addition | 25.4% | 25.4% | 24.1% | 25.0% |\n        | Removal | 25.9% | 26.2% | 23.2% | 24.7% |\n        | Style Editing | 24.7% | 24.6% | 25.0% | 25.7% |\n        \n        These results show that expert usage remains **relatively balanced across different instruction types**, indicating that **all experts are actively contributing within the model** rather than being underutilized. This also reinforces our earlier point that expert selection is inherently implicit and fine-grained, and does not directly align with high-level semantic categories such as task type.\n        \n        We consider this an insightful direction and plan to conduct more in-depth analysis in future work to further understand expert dynamics and potential specialization patterns."}, {"review_id": "72C6n5MALn", "reviewer": "Reviewer_oCfi", "summary": "This paper proposes ICEdit, an instruction-based image editing framework built upon a large-scale Diffusion Transformer (DiT). By leveraging an in-context prompt structure, a lightweight LoRA-MoE fine-tuning mechanism, and a noise filtering strategy during inference (Early Filter Inference-Time Scaling), ICEdit achieves high-quality editing performance under extremely limited training data and parameter constraints. This work advocates for minimal intervention and architectural modification to activate the native contextual understanding and generative capabilities of DiT, thereby balancing the trade-off between editing precision and inference efficiency observed in traditional methods.", "questions": "Please refer to weaknesses", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strength:\n1.\tThe paper proposes ICEdit, which leverages the original Diffusion Transformer (DiT) model to perform instruction-based image editing through an in-context approach—offering a novel and practical direction.\n2.\tIt introduces the Early Filter Inference-Time Scaling strategy, which utilizes a vision-language model (VLM) to filter early-stage outputs, mitigating instability caused by initial noise.\n3.\tIt also presents a LoRA-MoE structure to enhance the model’s capability in handling diverse tasks; this combination of LoRA and MoE is both meaningful and effective in practice.\n4.\tThe introduction, technical explanation, and experimental sections are all well-structured, with clear and coherent writing throughout.\n\nWeakness:\n1.\tExisting works such as Prompt-to-Prompt and Textual Inversion have explored similar directions, so it would be helpful to clarify how this approach differs from those prior attempts.\n2.\tThe paper also appears to lack an analysis of the model’s limitations—for example, a discussion of failure cases or incorrect edits could provide a more balanced evaluation.\n3.\tWhile the overall experiments are thorough, the choice of TopK=1 in the MoE method is not sufficiently justified and would benefit from further explanation.", "was_revised": true, "final_justification": "The author's response has addressed most of my concerns, including clarifications on distinctions from prior methods, discussions on the limitations of the adopted approach, and additional analysis regarding parameter selection. Therefore, I am inclined to raise the original score. I appreciate the author's efforts during the rebuttal process.", "rebuttal": "Thank you sincerely for recognizing our method as *“**offering a novel and practical direction**”* and *“**both meaningful and effective in practice**,”* as well as for your positive comments on the overall structure and clarity of our writing. We truly appreciate your encouraging feedback.\n\nWe’ll address your questions and concerns point by point below, and we strictly adhere to the double-blind review policy.\n\n- **Answer to Weakness 1: Difference from Previous Methods such as P2P and Textual Inversion**\n    1. **Our proposed training-free approaches are fundamentally different from prior methods such as P2P and Textual Inversion.**\n        - **Conceptual distinction**: **Our method is designed based on our proposed in-context editing paradigm (Fig. 2a)**, where the DiT model is reinterpreted as a \"painter\" that generates the edited image (right panel) by understanding both the reference image (left panel) and the instruction. This is conceptually distinct from previous training-free approaches like P2P or Textual Inversion, which do not operate under such a paradigm.\n        - **Difference from P2P**: Prompt-to-Prompt [Hertz et al., 2022] relies on descriptive prompts and cross-attention feature swapping to modify generation behavior. In contrast, our method performs instruction-guided editing, where the model is directly guided by the instruction to perform edits—**without requiring complex operations such as attention map replacement, addition, or deletion**.\n        - **Difference from Textual Inversion**: Textual Inversion [Gal et al., 2022] learns a new token to represent a visual concept from multiple reference images, and generates novel instances of that concept. This is a **different task altogether** from instruction-guided editing of a **single input image**, which Textual Inversion is not designed to handle.\n        - While we do adopt general-purpose techniques such as inversion and feature injection in our T2I-based variant — similar to P2P or Textual Inversion — **these are widely used in the field** (e.g., in MasaCtrl, PnP Inversion, RF-Solver, StableFlow) and serve only as tools to implement our paradigm rather than define it.\n        - Importantly, after exploring training-free realizations, we further refine the proposed framework via **few-shot fine-tuning with minimal data**, achieving **significant performance gains** that far surpass previous methods. This highlights that our proposed **in-context editing paradigm** is not only novel, but also demonstrably more effective than prior attempts.\n- **Answer to Weakness 2: Limitations of our model**\n    1. **We have included a discussion of the model’s limitations in the supplementary material (Section C, lines 151–162 and Fig. 7).** Due to space constraints, this section was not included in the main paper, but we will make its presence more prominent in a future revision.\n        \n        Specifically, our model may produce incorrect edits or failure cases in the following aspects:\n        \n        - **Object movement**: Instructions involving spatial relocation (e.g., \"move the chair to the corner\") may not be executed correctly. This is likely due to the lack of motion-oriented examples in general editing datasets. Targeted fine-tuning with specialized datasets could help address this issue.\n        - **Semantic understanding limitations**: Some failures arise from the model’s limited ability to resolve contextual ambiguities. Incorporating MLLM-based modules in future work could enhance semantic fidelity and instruction interpretation.\n        \n        That said, these limitations do **not compromise the overall significance** of our results. Compared to prior work, our method offers a **more efficient** and **higher-precision** framework for high-quality image editing, and demonstrates consistent advantages across multiple benchmarks.\n        \n- **Answer to Weakness 3:** **Choice of expert selection**\n    \n    Thank you for recognizing the thoroughness of our experimental evaluation. Here are explanations of the expert selection.\n    \n    1. **We choose expert TopK = 1 because it offers** **a strong trade-off between computational efficiency and predictive performance (as mentioned in lines 175-183 in Sup. Mat.)**. Activating only a single expert per token significantly reduces FLOPs, memory usage, and communication overhead—critical for both training and real-time inference in deployment settings. This design is well-supported by prior work: for instance, **Switch Transformers** [Fedus et al., 2022] and **LLaVA-MoLE** [Liu et al., 2024] demonstrate that Top-1 routing enables scalable sparse MoE training with minimal loss in accuracy. Furthermore, **Mixtral** [Beaumont et al., 2023] shows that while TopK > 1 may yield marginal gains in some cases, it comes at the cost of significantly increased computation and memory, which may not align with real-world constraints. Given these trade-offs and our deployment-oriented motivation, we adopt TopK = 1 as a practical and effective choice. We will add a clarification and references in the final version.\n    2. We have experimented with settings using TopK = 2 and TopK = 4 with a total of 4 experts. The results on the EmuEdit test set are summarized below:\n        \n        \n        | TopK Value | CLIP-I ↑ | CLIP-OUT ↑ |\n        | --- | --- | --- |\n        | **TopK = 1 (Ours)** | **0.907** | **0.305** |\n        | TopK = 2 | 0.891 | 0.301 |\n        | TopK = 4 | 0.866 | 0.298 |\n        \n        **As shown above, increasing the number of selected experts leads to a slight but consistent drop in performance.** Therefore, we chose **TopK = 1** as the optimal setting. We will clarify this design choice more explicitly in a future version of the paper.\n        We hypothesize that the lack of improvement with larger K may be due to the absence of additional constraints to guide expert selection. Incorporating auxiliary losses or regularization to encourage more effective expert routing is a promising direction for future work.\n        \n\nWe hope the explanation above resolves your concern. Please feel free to let us know if further clarification is needed.\n\nReferences:\n\n- Hertz, Amir, et al. \"Prompt-to-prompt image editing with cross attention control.\" *arXiv preprint arXiv:2208.01626* (2022).\n- Gal, Rinon, et al. \"An image is worth one word: Personalizing text-to-image generation using textual inversion.\" *arXiv preprint arXiv:2208.01618* (2022).\n- Cao, Mingdeng, et al. \"Masactrl: Tuning-free mutual self-attention control for consistent image synthesis and editing.\" *Proceedings of the IEEE/CVF international conference on computer vision*. 2023.\n- Ju, Xuan, et al. \"Direct inversion: Boosting diffusion-based editing with 3 lines of code.\" *arXiv preprint arXiv:2310.01506* (2023).\n- Wang, Jiangshan, et al. \"Taming rectified flow for inversion and editing.\" *arXiv preprint arXiv:2411.04746* (2024).\n- Avrahami, Omri, et al. \"Stable flow: Vital layers for training-free image editing.\" *Proceedings of the Computer Vision and Pattern Recognition Conference*. 2025.\n- Fedus, William, Barret Zoph, and Noam Shazeer. \"Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity.\" *Journal of Machine Learning Research* 23.120 (2022): 1-39.\n- Chen, Shaoxiang, Zequn Jie, and Lin Ma. \"Llava-mole: Sparse mixture of lora experts for mitigating data conflicts in instruction finetuning mllms.\" *arXiv preprint arXiv:2401.16160* (2024).\n- Jiang, Albert Q., et al. \"Mixtral of experts.\" *arXiv preprint arXiv:2401.04088* (2024)."}, {"review_id": "rncHrjtnmN", "reviewer": "Reviewer_NHpf", "summary": "This paper proposes an in-context learning approach for instructional image editing, introducing two frameworks: one built on a text-to-image (T2I) DiT model and another based on an inpainting DiT model. The latter incorporates a parameter-efficient fine-tuning technique to further boost performance. Experimental results show that the method achieves state-of-the-art performance with minimal training data.", "questions": "My main concerns are outlined in the weaknesses section. To summarize, I would like the authors to address the following points in the rebuttal:\n\n- What is the motivation for presenting the T2I DiT framework, given that it is not used in the experiments? This raises questions about the clarity and focus of the contributions.\n- Important details about the dataset are missing, and clarification is needed to fully understand the experimental setup.", "limitations": "Yes, in supplemental materials. I would suggest to move it to the main paper.", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths\n\n1. The idea of leveraging minimal, parameter-efficient learning is interesting.\n2. The experimental results appear promising and demonstrate strong performance.\n\nWeaknesses\n\n1. The paper’s presentation is unclear, making it difficult to follow and obscuring the main contributions. It seems that the Mixture-of-Experts (MoE) fine-tuning is a key contribution. However, the authors present two methods: one with fine-tuning and one without. This dual-approach is confusing—if fine-tuning is a central contribution, the motivation for including the text-to-image DiT method (which does not involve fine-tuning and requires computationally expensive image inversion, as mentioned in Line 146) is unclear.\n\n2. Assuming the main experiments are based on the fine-tuned inpainting DiT model—which takes a mask as input—it is not explained how these masks are generated in the training dataset.\n\n3. Since the training data appears to be carefully curated, it raises concerns about potential data overfitting. Would the model still perform well if the training data were randomly sampled from standard datasets? A comparison with randomly sampled training data would help clarify this.", "was_revised": true, "final_justification": "I appreciate the authors' response. Since they proposed renaming the method in the experiment to address my confusion, I am willing to raise my rating.", "rebuttal": "Thank you for your positive feedback and for recognizing the value of our approach and strong performance of our method. We appreciate that very much!\n\nWe greatly appreciate your feedback and will address your comments and concerns point by point in the following responses.\n\n- **Answer to Weakness 1 and Question 1: Clarification of Our Motivation and Methods**\n    \n    We believe there may have been **a misunderstanding or oversight** regarding the clarity of our motivation and method.\n    \n    Other reviewers have provided positive comments on this aspect, noting that “**the writing and illustrations are clear and well-organized, making the proposed method easy to understand,**” and commending the **“extensive analysis”** and **“clearly provided implementation details”** (**Reviewer oDrm**); that the **“method design is interesting and appears to be novel”** (**Reviewers oDrm and ZGQZ**); and that **“the introduction, technical explanation, and experimental sections are **all well-structured, with clear and coherent writing throughout**”** (**Reviewer oCfi**).\n    \n    We will further clarify our motivation and method in the response below, with references to the relevant sections and figures in the paper to aid your understanding.\n    \n    **Explanation:**\n    \n    1. **Clarification of motivation:**\n        - Our primary goal is to leverage the generative capacity of large-scale DiT models for instruction-based image editing *without architectural modifications or heavy retraining* (as noted in Lines 19–26, 35–39).\n        - To this end, we propose a novel **in-context editing paradigm** (Fig. 2a), where the DiT model is reinterpreted as a \"painter\" that produces edited images (right panel) by understanding the reference image (left panel) and the instruction.\n        - **To explore the feasibility of this paradigm, we first introduce two training-free methods.** Although they demonstrate initial instruction-following capabilities, they suffer from **poor instruction comprehension** and **layout instability** (as shown in Figs. 3 and 4, and discussed in Lines 40–43).\n        - To address these issues, we enhance the editing capability under our paradigm (the inpainting-based paradigm) by integrating **IC Prompt**, **parameter-efficient fine-tuning via MoE-LoRA with minimal data**, and a **test-time scaling strategy** (Lines 44–60).\n        - Remarkably, this approach achieves **state-of-the-art performance** using only **0.1% of the training data** compared to prior models.\n    2. **Clarification of two training-free approaches:**\n        - **The two training-free approaches share the same core purpose** — to validate our proposed **in-context editing paradigm** (Fig. 2a). Whether based on a T2I DiT or an inpainting DiT architecture, both aim to generate an edited result (right panel) conditioned on the reference image (left panel) and instruction (Lines 110–119).\n        - **These variants serve as an important proof of concept**: despite the absence of fine-tuning, both models demonstrated some ability to perform in-context editing, indicating that **DiT models inherently possess the potential to support this new editing paradigm**.\n        - However, **we also found that the performance of these training-free variants is limited**. As shown in Fig. 4, the outputs are often unstable or of low quality. Quantitatively, both variants achieved similarly low GPT scores (**0.24**), as reported under “Training-free w/ IC prompt” in **Table 3(a)**—far below the results of our fine-tuned models.\n        - Therefore, the claim that *“the T2I DiT framework is not used in the experiments”* is inaccurate—it was explicitly used as one of our training-free baselines to validate the feasibility of our paradigm.\n        - Motivated by the observed limitations of the training-free variants, we proceeded to fine-tune the inpainting-based DiT model, which is more computationally efficient than its T2I counterpart. The inpainting DiT directly supports denoising-based editing, avoiding the need for inversion and reducing runtime by roughly half (Lines 140–147).\n        \n        In summary, the training-free experiments establish two key findings （Lines 140–147）:\n        \n        (1)**DiT models are capable of in-context editing**\n        \n        (2)**Fine-tuning is necessary to fully realize the potential and stability of the proposed paradigm**.\n        \n    3. **The finetuned model:**\n        - Building on the inpainting DiT, we improve editing capability through a combination of parameter-efficient MoE-LoRA fine-tuning on a small dataset and a test-time scaling strategy (Sections 3.2, 3.3, and 4), ultimately achieving SOTA performance.\n        - **Our contributions span from proposing the motivation and validating the paradigm with training-free methods, to designing a lightweight fine-tuning strategy and inference-time enhancement for stable, high-quality editing** （Lines 69-78, Lines 243-251）.\n        - **Focusing solely on the fine-tuning or on the T2I architecture overlooks the broader scope of our work** — we are introducing a **novel and general in-context editing framework** that unifies learning-free and learning-based components under a single paradigm.\n    \n- **Answer to Weakness 2: Use of Mask in the Model**\n    1. **Our model does not require any manually created local masks.** **As stated in the main paper (lines 18–19, 149–152)** and **illustrated in Figure 5**, we simply use a fixed-size binary mask that matches the resolution of the input image. This mask is always applied to the right part of the diptych, while the left part provides the unmasked reference image.\n    2. Although we utilize an inpainting model (as our final solution), **our system functions like any other instruction-based image editing method**: the user only needs to provide an instruction and an image—**no explicit indication of the edit region is needed**. **Our model is capable of automatically identifying and editing the appropriate regions based on instruction semantics**. We believe this misunderstanding may have led to confusion about our method’s design. We appreciate your question and will clarify this point more explicitly in future revisions to avoid such misinterpretations.\n\n- **Answer to Weakness 3 and Question 2: Details About the Training Dataset**\n    1. **Our training data was not carefully curated but rather randomly sampled from standard datasets.** As stated in the supplementary material **(lines 20–28)**, *“the dataset was not rigorously curated,”* and it indeed contains many **problematic samples**, as illustrated in Supplementary Figure 1. We did **not** perform any additional filtering or cleaning of the dataset. Therefore, the assumption that our dataset was carefully selected is incorrect.\n  2. Regarding the OmniEdit data we use, we provide details on our download and usage process here. Since the OmniEdit dataset contains over 500 parquet files with a variety of edit types, we sampled a subset of files across different types to maintain diversity. **All parquet files were directly released by the dataset authors and randomly generated**—**we did not apply any post-selection or filtering**. Specifically, we selected 17 out of the 571 files: 1, 25, 50, 100, 115, 130, 160, 175, 225, 250, 300, 350, 425, 525, 566, 568, and 570. These files cover various editing types, including addition, replacement, removal, and attribute modification. The last few files (566–570) correspond to style editing, which constitutes only a small portion of the full dataset.\n  3. Importantly, **the use of randomly sampled data did not negatively impact model performance**. As shown in Supplementary Figure 8, increasing the amount of randomly sampled training data leads to consistent improvements, demonstrating a clear **scaling effect**. **This suggests that our model generalizes well without overfitting to any curated subset.**\n  4. Due to space limitations, **the discussion of dataset details was included in the supplementary material (Section B)**. We apologize if this caused you to overlook this information, and we will make these details more prominent in the main paper in future revisions. Besides, we will release the full codebase and all training configurations to facilitate reproducibility. The exact data sources, file IDs, and download instructions for the subset of OmniEdit used in our experiments will be clearly documented in the open-sourced repository.\n\nWe hope the explanation above resolves your concern. Please feel free to let us know if further clarification is needed."}, {"review_id": "Owg09EWjUD", "reviewer": "Reviewer_ZGQZ", "summary": "The paper proposes ICEdit for the task of image editing. ICEdit minimizes the alteration of existing image diffusion transformer architecture, and requires very low fine-tuning efforts, but can still achieve similar and even better quality compared to baseline methods. ICEdit also utilizes an early filter inference-time scaling approach to select high-quality noise samples for generation efficiency and quality. ICEdit shows promising results on standard image editing benchmarks and with qualitative results.", "questions": "1. Does the proposed pipeline need to fine-tune a VLM judge? In the paper, it indicates to use a QwenVL-72B, may I ask the authors if any fine-tuning about image aesthetics is required for better judging?\n2. In the visual illustration, I see ICEdit support object removal. May I ask if the task of object location manipulation (e.g., move the apple from left to right in the image) is also supported?\n3. Though requiring much less computational resources to finetune compared to training-based baselines, utilizing VLM (especially a 72B model) and inference with multiple seeds may pose a computational difficulty. Would the author mind clarifying the computational cost during inference?", "limitations": "Yes, the authors adequately addressed the limitations and potential negative societal impact of their work.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strength:\n1. The proposed design choice on image editing prompting is carefully studied.\n2. The proposed pipeline requires minimal fine-tuning effort and architectural changes, but results are improved compared to the baseline method.\n3. The proposed test-time scaling with the early-stage intervention is very interesting and quite novel to image generation.\n\nLimitations:\n1. Even though the design choice of utilizing an inpainting-based approach is carefully studied and explained, this can be a limitation if users would like to have end-to-end editing with a prompt without any instruction on locations.", "was_revised": true, "final_justification": "I am now fully convinced by the method’s pipeline, especially with the authors’ clear explanation regarding the need for user-provided location information or manual masks during inference, which I found quite surprising. Additionally, the use of VLM API calling appears to be an efficient and user-friendly approach, particularly since the huge VLM model does not require any fine-tuning. While object movement remains a challenge, I understand that this is a common limitation in image editing tasks. Overall, I am satisfied with the rebuttal, and most of my concerns have been fully addressed. Therefore, I would like to increase my score and recommend acceptance.", "rebuttal": "Thank you sincerely for recognizing our design choices and acknowledging that our method is both novel and contributes to improved results. We truly appreciate your encouraging feedback.\n\nWe address your questions and concerns point by point in the following responses.\n\n- **Answer to Limitation 1:** **Clarification Regarding Location Instruction and Inpainting Design**\n    1. Thank you for your comment. We would like to clarify a possible misunderstanding: **our method does not require any user-provided location information or manual masks**. \n        - Although we adopt an inpainting-based architecture, **the mask used during training and inference is a fixed binary mask of the same resolution as the input image**, always applied to the half of the diptych (**as stated in the main paper lines 18–19, 149–152** and **illustrated in Figure 5**). This design choice is internal to the model and does **not** require any user interaction or localization input.\n        - From the user’s perspective, **our system operates** **fully end-to-end**: **the user provides only an image and a natural language instruction**—**no specification of edit regions is needed**. **Our model is capable of automatically identifying and editing the appropriate regions based on instruction semantics**. Therefore, the model remains fully compatible with prompt-only editing workflows and does not pose limitations in terms of usability or interactivity.\n        - We appreciate the opportunity to clarify this point and will make it more explicit in future revisions to avoid similar misunderstandings.\n\n- **Answer to Question 1:  Clarification on the Use of VLM Judge and Fine-tuning**\n    1. Thank you for your insightful question. **In our current pipeline, we directly use the pre-trained Qwen-VL 72B model without any additional fine-tuning.** We chose this model because of its strong zero-shot capabilities, which we found sufficient for reliable judgment of image-editing quality and alignment. \n    2. We also experimented with smaller pre-trained variants such as Qwen-VL 7B and 32B. However, we observed that their judgment ability was less robust and less consistent across diverse editing cases, which led us to prefer the 72B version despite its larger scale.\n    3. That said, we agree that fine-tuning a smaller vision-language model (VLM) on editing-specific or aesthetics-related data is a promising alternative. Recent works such as **SANA 1.5**, **OmniEdit**, and **ImgEdit** have demonstrated that lightweight, fine-tuned VLMs can serve effectively as judges when trained on targeted datasets.\n    4. Our primary goal in this work was to explore the *inference-time scaling strategy* for editing models. Leveraging a powerful model like Qwen-VL 72B helped us validate the core effectiveness of our algorithm. In future work, we plan to explore replacing it with smaller, fine-tuned models to reduce computational costs while maintaining evaluation reliability.\n\n- **Answer to Question 2: Object Movement Editing Ability**\n    1. As discussed in the supplementary material (lines 151–155) and illustrated in Supplementary Figure 7, one known limitation of our model is its handling of **object movement**. Specifically, instructions involving spatial relocation (e.g., \"move the chair to the corner\") may lead to suboptimal results. **This is likely due to the limited exposure to motion-oriented transformations in standard editing datasets.** We believe this limitation could be mitigated through targeted fine-tuning on specialized datasets, such as the one introduced in *Cao, Mingdeng, et al., \"Instruction-based Image Manipulation by Watching How Things Move.\"*\n    2. Addressing object movement remains an important direction for future work, and we plan to explore the integration of such specialized datasets to enhance our model’s spatial reasoning and relocation capabilities.\n    3. **That said, this limitation does not undermine the overall contribution of our work.** Compared to prior methods, our approach provides a **more efficient** and **higher-precision** framework for high-quality image editing, and consistently outperforms existing baselines across multiple benchmarks.\n\n- **Answer to Question 3: Clarification of Computational Cost During Inference**\n    1. Thank you for raising this important point. We agree that directly running a 72B VLM model can be computationally intensive. **However, in our implementation, we access the Qwen-VL 72B model via the official API, which eliminates any local GPU memory overhead (as mentioned in Sup. Mat. lines 82-84).** In practice, each API call (i.e., a single instruction-image pair evaluation) takes approximately **2 second**, making it efficient and practical in most real-world scenarios.\n    2. As for our editing model, the base inference setup on an A100 GPU consumes approximately **33 GB of memory and requires around 10 seconds to generate one edited image**. That said, with **CPU offloading** and lightweight quantization strategies such as **SVDQuant** [Li, Muyang, et al., *\"Svdquant: Absorbing Outliers by Low-Rank Components for 4-bit Diffusion Models,\"* arXiv:2411.05007], we can reduce the memory requirement to **under 10 GB**, significantly lowering the barrier to deployment.\n        \n        These optimizations make our approach much more accessible while still maintaining strong performance.\n        \nWe hope this response addresses your concern, and we’re happy to further clarify if there are any remaining questions.\n\n**References:**\n\n- Xie, Enze, et al. \"Sana 1.5: Efficient scaling of training-time and inference-time compute in linear diffusion transformer.\" *arXiv preprint arXiv:2501.18427* (2025).\n- Wei, Cong, et al. \"Omniedit: Building image editing generalist models through specialist supervision.\" *The Thirteenth International Conference on Learning Representations*. 2024.\n- Ye, Yang, et al. \"Imgedit: A unified image editing dataset and benchmark.\" *arXiv preprint arXiv:2505.20275* (2025).\n- Cao, Mingdeng, et al. \"Instruction-based image manipulation by watching how things move.\" *Proceedings of the Computer Vision and Pattern Recognition Conference*. 2025.\n- Li, Muyang, et al. \"Svdquant: Absorbing outliers by low-rank components for 4-bit diffusion models.\" arXiv preprint arXiv:2411.05007 (2024)."}]

Author rebuttal:

Meta-review: This paper introduces ICEdit, an instruction-based image-editing framework that contains three components: (1) an in-context “diptych” prompting scheme, (2) a lightweight fine-tuning module, and (3) an Early-Filter Inference-Time Scaling strategy. Extensive experiments across multiple benchmarks demonstrate that ICEdit achieves superior performance.

Overall, reviewers find the technical contribution solid and the experimental results convincing. But meanwhile, some concerns are raised: 1) the effectiveness of the diptych prompting scheme should be more carefully ablated; 2) insufficient analysis on the MoE expert behaviour; 3) more details are needed regarding the usage and computational overhead of the 72B VLM; 4) it is unclear how inpainting masks are generated; and 5) the presentation clarifty can be further enahnced.

The rebuttal is considered, which successfully addresses most of these concerns. As a result, all reviewers unanimously recommend acceptance. For the final version, please ensure that all promised revisions from the rebuttal are fully integrated to ensure the clarity and quality of this NeurIPS publication.
Decision: Accept (poster)