The Promise of RL for Autoregressive Image Editing
Authors: Saba Ahmadi
Rabiul Awal
Ankur Sikarwar
Amirhossein Kazemnejad
Ge Ya Luo
Juan A. Rodriguez
Sai Rajeswar
Siva Reddy
Christopher Pal
Benno Krojer
Aishwarya Agrawal
Venue: NeurIPS 2025 poster ()
Keywords: Image editing
Autoregressive
MLLM
GRPO
RL
Reasoning
Primary area: applications

Abstract: While image generation techniques are now capable of producing high-quality images that respect prompts which span multiple sentences, the task of text-guided image editing remains a challenge. Even edit requests that consist of only a few words often fail to be executed correctly. We explore three strategies to enhance performance on a wide range of image editing tasks: supervised fine-tuning (SFT), reinforcement learning (RL), and Chain-of-Thought (CoT) reasoning. In order to study all these components in one consistent framework, we adopt an autoregressive multimodal model that processes textual and visual tokens in a unified manner.
We find RL combined with a large multi-modal LLM verifier to be the most effective of these strategies.
As a result, we release **EARL**: **E**diting with **A**utoregression and **RL**, a strong RL-based image editing model that performs competitively on a diverse range of edits compared to strong baselines, despite using much less training data. Thus, EARL pushes the frontier of autoregressive multimodal models on image editing. We release our code, training data, and trained models at [https://github.com/mair-lab/EARL](https://github.com/mair-lab/EARL).

The Promise of RL for Autoregressive Image Editing
Saba Ahmadi 1 Rabiul A2 Ankur Sikarwar 1.2+
Amirhossein Kazemnejad
Ge Ya Luo 1.2 Juan A. Rodriguez' 1,46 Sai Rajeswar 1..6
Siva Reddy 13,6,77 Christopher Pal 1,5,6,7 Benno Krojer
Aishwarya Agrawal
Mila Quebec Al Institute Université de Montréal
McGill University
École de Technologie Supérieure (ETS) 'Polytechnique Montréal SSeviiceow
"Canada CIFAR Al Chair
Abstract
While image generation techniques are now capable of producing high-quality images that respect prompts which span multiple sentences, the task text-guided image editing remains challenge. Even edit requests that consist of only few words often fail to be executed correctly. We explore three strategies to enhance performance wide range of image editing tasks: supervised fine-tuning (SFT), reinforcement learning (RL), and Chain-of-Thiee Thhoughhtt (CoT) reasoning. order to study all these components in consistent framework, we adopt autoregressive multimodal model that processes textual and visual tokens in unified manner. We find RL combined with large multi-modal LLM verifier to be the most effective of these strategies. As a result, we release EA RL: Editing with Autoregression and RL, strong RL-based image editing model that performs competitively on a diverse range of edits compared to strong baselines, despite using much less training data. Thus, EARL pushes the frontier of autoregressive multimodal models on image editing. We release our code, training data, and trained models at https://github. wwwmmairmmm....../o... lab/EARL
Introduction
have seen impressive progress on open-ended image generation in recent years. At this point, the latest text-to-image models However, synthesizing images from prompt alone is often not sufficient for end-users and broader ML applications. In reality, a person might want alter highly specific details in given image instead of creating one from scratch. Beyond direct user applications, e.g., in the domains of robotics and planning, one might want "imagine into the future" with an image editing model acting as simulator [59 [6] 71. e.g.. "how does this scene look like if the robot pushes the mug?". : In both cases, the model must faithfully preserve all details of the original image while modifying elements intended for editing.
With internet-scale image-text pairs [4T] and diffusion models 40
39 23
can often adhere to detailed prompts that span several sentences (5
13
From a capability perspective, most current editing models 61 g [6]] cover arguably simpler object and attribute edits (replace, change color, add, ...). yet only few works 26 4 tackle complex edits that require c.g., action understanding or reasoning (spatial, counting, physical dynamics). From a modeling perspective, a standard recipe to improve editing is to apply supervised fine-tuning (SFT) to diffusion-baase image generation model g 61 [6], rarely incorporating more recent post- training methods such as reinforcement learning (RL). parallel line of work introduces additional bounding- box conditioning, either explicitly provided by the user [31] or implicitly predicted [16), leaving these methods far from an end-to-ent solution. Hence, in this paper we ask: What is the
'denotes equal contribution
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

most effective approach to address both simple and complex edits with a unified end-to-end model? And specifically, what are the key learning paradigms that can move the field forward?
To this end, we conduct a series of experiments with three different learning paradigms (SFT, RL and chain-of-thought reasoning) and mixes of data. However, diffusion-based models would not directly allow a consistent setup where all training approaches could be plugged in out of the box. For this reason we choose Emu3 (50) as our starting point, a fully autoregressive generative model which was pre-trained on captioning and image generation. Since Emu3 is a unified image and language generation model, we can easily use it to study CoT reasoning and online RL methods such as GRPO 42), on top of SFT. We note that RL for visual generation with diffusion or flow-matching At the end of our exploration, we arrive at a simple recipe and propose E.A RL: Editing with Autoregression and RL, a fully autoregressive generative model that trains on simple and complex editing data during the SFT and RI. post-training stages. Specifically, we find the combination of GRPO with a strong MLLM- verifier to be the most effective. While variants of CLIP-Score are more commonly used verifiers [30 32), they often require finetuning on preferences and lack fine-grained understanding [60, J. Instead, we identify large MLLMs with fine- ane-grained
non-trivial 6 32).
understanding such as Qwen2.5-VL-777 as effective verifiers for a broad range of edits.
We empirically show that EARL performs well across many types of edits by evaluating on 6 diverse benchmarks in both IID and OOD settings. We achieve better results than prior state-of ff-te-of-thh-ar models on the OmniEdit [22, AURORA (26), and VisMin (2) benchmarks. Moreover, our method outperforms the strongest prior work Omnigen [56]. despite using five times less data, and also surpasses baselines that use comparable amount of data, such as AURORA (26). See Fig. 1 for samples from EA before and after applying our RL. recipe, showing improved performance even on challenging tasks that require spatial understanding. Notably, modeling textual and visual generation
as one autoregressive stream has emerged as a new exciting paradigm [50 45 46, [55, and we push the frontier of such models for the image editing task, outperforming prior related work EditAR (38) Finally, we highlight a surprising finding from our in-depth analysis of different training paradigms: teaching the model to explicitly reason about the intermediate steps (chain-of-thouu style reasoning) before generating the actual edit does not seem to improve performance, and sometimes even hurts. Our contributions are as follows:
1. We release EARL: A unified end-to-end editing model that performs well on the whole spectrum of edit types.
2. EARL outperforms the strongest open-source diffusion baseline (Omnigen [66 ) while also
outperforming a more comparable fully autoregressive editing model EditAR 38
3. A simple yet effective online RL pipeline for training an autoregressive editing model.
4. A systematic analysis of how training objectives interplay and at what stages to bring in simple vs. complex edits. The results interestingly reveal that various CoT reasoning settings do not bring any clear improvements. Our findings also show that complex edits not beneficial during the SFT stage but are effective in the RL post-traiming stage.
Our results show that autoregressive models, while underexplored for image editing, can be highly competitive. When paired with RL, they surpass strong diffusion-based base- lines on both simple and complex edits. This demonstrates the power of combining autoregressive generation with RL for controllable and high-fidelity image editing.
EARL Recipe:
2 Related Work
Image Editing Models. Enabled by the development of diffusion models with open-ended text- conditioning for image generation 40 39] image editing models can now receive any text prompt in similar manner and typically depend on these diffusion models [ 17. Research 40 35] has
demonstrated that models such as Stable Diffusion
40 can be zero-shot transformed into an editing
model by modifying the sampling procedure 335 or attention maps 22 To achieve better results,
components such as additional input U-Net channels are added to pre-trained image generation models followed by fine-tuning on curated editing data [6] 12 3 These training datasets consist of triplets of input image, an edit instruction describing the change, and a ground-truth edited image output. These triplets rarely occur "naturally" (i.c. only in forums (3) and therefore need
2

---

Imput
only
Input
only
Input
only
Tamn color obot ya whill
Adl some lettering wo surfbourd socurd
Uharge the wirce couch browe couch
Reciose the bectted CLETO in window nurtiin
Pour dl the orrnng contert from the bog imco big sood
Bemoos the linds from the imege.
Qualitative comparison between SFT-only and EARL across diverse editing instruc- tions. EARL extends the SFT model by leveraging reinforcement learning to better align image edits with natural language prompts. While both models handle simple edits reasonably well, EARL exhibits clear improvements in precise editing on simple as well complex edit instructions. Simple edit instructions are shown in blue, and complex edit instructions are shown in pink.
Figure I:
to be sourced from synthetic image generation pipelines (22), human-in-the-loe annotation 61].
videos 266 44 or simulation [26 36). Various works also adopt more structure into the editing task by restricting the model to edit certain regions of the image [I]] conditioning bounding boxes or keypoints [37]. In a more recent line work, image generation is learned jointly with text generation in unified multimodal models that autoregressively predict arbitrary sequences of textual and visual tokens [50 (55. Concurrent with our work, BAGEL [12] trains a unified transformer model that integrates LLMs and diffusion models, achieving strong editing performance (with and without reasoning) through large-scale pretraining and a powerful base model. However, autoregressive models less explored for image editing these models remain less powerful than their diffusion counterparts. Recent work [38] explores using autoregressive models for image editing and achieves competitive results with diffusion baselines. However, they only study the SFT training paradigm, while we study SFT, RL post-training, and CoT Reasoning, cover a wide range of edit types, and outperform their results.
Reasoning in Image Generation. L.M reasoning has been adapted to enhance image generation models through additional conditioning or planning (54). Models like GLIGEN (31 and Layout- GPT [1] use L.LMs to predict bounding boxes and scene layouts to direct object placement before generation. In these works, the layout generation step is unimodal, relying solely on LLMs. However, for image editing tasks, incorporating multimodal information from both the original image and the edit instruction is essential to determine effective layout. GoT (Generation Chain-of-Thought) [14] applies CoT to visual generation and editing tasks. It first generates reasoning in text, analyzing semantic and spatial relationships the input image, before generating the edited images using diffusion model. Additionally, PARM+ (21] introduces a reflection mechanism to self-correct generated images, further enhancing the model's reasoning capabilities. Another approach focuses on improving the prompts used for image editing by utilizing large language model (LLM) or multimodal LLM (MLLM) [17], which better equipped for both text and image understanding. While previous work has applied reasoning to image editing tasks, our approach systematically explores and evaluates of-houof- (CoT) reasoning in different settings involving both simple and complex edits.
RL. for Image Generation. Reinforcement learning has emerged as a powerful tool for finetuning image generation models, particularly diffusion models, to better align with human preferences (6].
3

---

SFT
SFT w/ Reasoning
RL
cthinking: lountify ievl coordinates. Move
M
r2
r3
up </thiningg
Next Token Prediction
Next Token Prediction
Next Token Prediction
Put the left hand on the leg while drinking from glass
Put the left hand on the leg while drinking from glass
Put the left hand on the leg while drinking from the glass
Figure 2: Autoregressive Image Editing Approaches. In supervised fine-tuning (SFT), we train an autoregressive model based on the standard image editing setup: triplets of source image, edit instruction, and target image. In SFT with reasoning, the model is supervised to generate chain- of-thought (CoTT) reasoning traces before generating the final edited image. Finally, we study reinforcement learning (RL) training of the SFT checkpoint, using edit quality verifiers as reward signals.
Preference-basee based methods such as Diffusion-DPO 48 and D3PO [57 bypass explicit reward models by directly learning from pairwise human feedback. DDPO 66 further adapts diffusion models to hard-to-specify objectives such as aesthetic quality and compressibility, using rewards based on multimodal models (prompt-image alignment). Although RL for image generation is gaining momentum through human preference and multimodal reward signals, its application to image editing remains underexplored. HIVE 63 collects human feedback on edited images to learn reward functions that capture user preferences, but such datasets are costly and difficult to scale. InstructRL4Pix (30] addresses this challenge by using attention-
reward signals for localized, instruction-driven editing. Meanwhile, GRPO has demonstrated stable and efficient training of autoregressive large language models. Concurrent work such as Flow-GRPO 32) and SimpleAR [49] apply GRPO flow matching and autoregressive models, respectively. Building on this success, we adopt GRPO for unified image editing with the autoregressive Emu3 model [50]. We leverage strong multimodal model (58) as a reward function, utilizing its robust image-text alignment and zero-shot prompting to specify and evaluate editing preferences effectively.
3 Training Paradigms for Autoregressive Image Editing
To our knowledge, EARL the first introduce an RL post-training paradigm for autoregressive image editing. While RL-based approaches have recently improved diffusion- editors, their potential in AR models has remained unexplored. Furthermore, we present the first systematic and controlled comparison of three major training strategies: SFT, RL, and CoT reasoning, within a unified AR framework for image editing. In this section, we formally describe the three training paradigms explore. All our experiments build upon the Emu3 base model [50) (see A. A. for more details), a large autoregressive multimodal model that unifies image and text generation in single autoregressive stream and is trained from scratch.
In the image editing task, the input consists of an original image and a textual edit instruction, and the output is the corresponding edited image. For training Emu3 on the image editing task, both the original and edited images are tokenized into vision tokens using an image tokenizer, while the edit instruction is tokenized into language tokens. These vision and language tokens are then modeled jointly by single causal transformer (i.e. a LLaMA-style architecture (7)). The task is framed as sequence-tor grquence-to-equenee to-sequence generation: the model takes as input a token sequence [æ...... 2MM) formed by concatenating the vision tokens of the input image with the language tokens of the instruction, and generates an output sequence vision tokens of the edited image which is used for teacher forcing. Below, we detail the learning objective for each of the training paradigms explore Fig. 2):
[......, Cr), where y = [.,,.... : .9T] represents the ground truth

---

Supervised Fine- Tine-Tuning (SFT) We employ the standard next-token prediction objective to process interleaved image-text sequences. Training minimizes the cross-entropy loss:
(() (x,)))
Σ log mo
yets 3:)
where me denotes the model and D is the labeled training dataset consisting of edit instructions paired with ground-truth edited images.
Reinforcement Learning (RL) Post-Training We use Group Relative Policy Optimization (GRPO) 42] in our RL pipeline to post-train the model following the initial SFT stage. GRPO initializes a trainable policy model Ty and a frozen reference model from the SFT checkpoint. For given input prompt z, the model generates group of G responses y/1 1/2 ... yG based on the current policy model Teaa- The optimization goal is to maximize the following objective:
[(0) = E fo Ne aodd
Σ
G
min
As, clip
To(Vi.t
Vi,eer) e, 6,11 c,1+t) 2.) -KL [Ne Trel]
iTaaaa (iitz
iTraa (Bitt (e,
Here, A, = (r---il u)/a denotes the advantage of response yi, where r; is the reward of the i-th response. and the and standard deviation of rewards across the response group (v: The hyperparameters e and B3 control the clipping range and the Kullback-Leibler (KI.) penalty, respectively. To compute the reward rı. we first detokenize the vision tokens in yı back into an image. The reward is then computed using the MLLM verifier, which evaluates the quality of the image editing. This RL objective optimizes the model generate higher-quality edits while maintaining training stability via the KL divergence term. For detailed overview of the GRPO method, see the pseudocode in App. A.2
RL Verifier: Any MLLM with strong image understanding capabilities can be used for verification. verifier to evaluate the generated edits based on the following criteria from VIEScore 27 (1) Edit Success whether the intended modification was accurately applied: (2) Overedit whether any unintended changes were introduced; (3) Natural Look - how well the edit blends with the original image; and (4) Artifacts whether the image contains visual distortions or anomalies. For criteria 1 and 2, the inputs to the model are the edit instruction, input image, and edited image. For criteria 3 and 4, only the edited image is provided. The individual scores are then aggregated into single reward signal, ranging from 0 to 10.
In particular, use Qwen2.5-VI--72B 4
Chain-of- ohhought (CoT) Reasoning Explicitly generating intermediate reasoning steps before the final output a widely adopted technique that improves performance on complex reasoning tasks 20 [33] This is known as the Chain-of-Thouse (CoT) reasoning. We extend CoT reasoning to the field of image editing. To teach CoT reasoning to Emu3, we finetune Emu3 with CoT supervision by prepending the response y with a tokenized reasoning chain (see Sec. 4. for details of synthesizing ground-truth reasoning traces).
Experimental Setup
4.1 Training Details
Implementation Our base model is Emu3-8B [50], a state-of-the-art autoregressive multi- modal model with unified image-text generation capabilities. For SFT, we initialize from Sall/Emu--Stagee weights, set learning rate of 1e 4, an effective batch size of 128 with 4 GPUs, a per-device batch size of 4, and 8 gradient accumulation steps. We use validation loss to stop training. For RL post-training, we use KL divergence coefficient of 3e 4 and a learning rate of 6. The RL model trained with 8 rollouts per edit instruction and a batch size of 128 and
We omit the dependence for brevity.
5

---

Table 1: Datasets, covered edit types, and their share of the full training corpus.
Dataset
Type
Sub-type
Size
Omni 15
S
Object (add, remove, replace), Attribute (modify), Scene, Style
750K
HumanEdit (3
C
Object (add, remove, replace, relation, action counting)
4.6K
MagicBrush
Object (add, replace, remove), Attribute (modilly), OCR (modifyy), Action (modify)
8.7K
VisMin 2
C
Object (add'semove), Attrbute (replace). Count fadd/remove), Spatial (swap)
50K
Aurora-Kubric 26
Spatial, Countngg Attribute
50K
Aurora-ActionGenomee
Human Pose (action)
7.8K
SomethnngSomcthing-222
19
Action
50K
Editing Data (Input)
Step-by-Step Reasoning (Output)
Input Image
Edited Image
Reasoning:
The source image shows
person holding a painted ceramic mug... The object to be edited is the mug within bounding box [[111, 112, 201, 210]]... Changes include turning the mug color to pink... The edited image will show a person holding a pink pointed ceramic mug.
Edit Instruction: Change the ceramic mug to be painted in a pink color
Target region: [[11) 112, 201, 210]]
Figure 3: Example of step-by-step reasoning generated by Qwen2.5-VL-72B using standard editing data (input image, edit instruction, target image, and bounding box).
training continues until the reward plateaus. To enhance training stability, we adopt fully online policy gradient approach, performing single gradient update at each RL step [25]. All images are resized 256 256 by maintaining their original aspect ratio. Further details the training setup and compute efficiency are provided in App. F.
Training Datasets Our dataset is divided into two categories based on the complexity of the edits: Simple Edits (S) and Complex Edits (C). Simple Edits (S): This category includes relatively simple local edits such as single-object and attribute changes, well global edits such as style and environment changes. These types of edits are common in large-scale synthetic datasets, such as OmniEdit [52] with 750K samples. Existing models generally perform well on these tasks. Complex Edits (C): These edits involve more advanced operations, including counting, spatial, and action modifications, where current models often struggle. Datasets like Aurora-AG (26), Aurora-Kubric (66), VisMin (2, and Something-SSmmetiing v2 [19] contain such challenging edits. Additionally, use real- world edit requests curated with human in the-loop guidance, such as Human-Edit (3) and MagicBrush [61], which include complex object/attribute changes. Data in the complex edit category is significantly scarcer, e.g., MagicBrush has 8K samples.
We provide details for each dataset in Tab. Our dataset comprises 750K OmniEdit samples, while C combines the above mentioned C datasets with 171K unique samples. For datasets in C with fewer than 50K samples, we upsample them to 50K, resulting in a dataset of size 300K for supervised fine-tuning. For RL post-training, we randomly sample from the respective data pool (S or C) at each iteration, using unique samples per step with 8 rollouts per sample. We first experiment with a smaller setup using pool of 1,600 samples for various ablations. To improve further, we then train with a total of 32K samples over the course training. This provides more diverse data, allowing the model to benefit from broader coverage across edit types.
For CoT reasoning supervision, we generate chain-of-thoughh data using multimodal large language model (MLLM), Qwen2.5-VL-722 V-L-72B [58), following a prompting strategy similar to [1]. The input to the MILM consists of standard editing data: an input image, an edited image, textual edit instruction describing the desired change, and bounding boxes specifying the edit region (if available). For action edits, we also include person keypoints. The generated CoT data follows a step-by-stey structure, including description of the input image, bounding box coordinates of objects to be edited in the input image, bounding boxes of objects be added the target image, the edit action, and description of the final edited image. For complex edit datasets, apply few-shot prompting to

---

synthesize CoTs, while for the simple edit dataset (S), we reuse CoTs from 14] An example is shown in Fig. and our prompting templates and full examples are provided the App. B.2
4.2 Evaluation Setup
We evaluate on diverse suite of image editing tasks, ranging from simple object, attribute, and style modifications to complex editing tasks such as counting, actions, and spatial relations.
metric since it outperforms traditional metrics like LPIPS [62] in terms of human correlation (0.3821 for GPT-4o versus 0. 142 for LPIPS (27]). VIEScore scores edits from 10 across four criteria (as explained in Sec. 3). For evaluation, we use GPT40-mini due to its high quality and cost efficiency and confirm that GPT40-mini variant aligns with human judgment not only on edits from the original human study [27] but also on various complex edits; details are provided in App. CJ Note that while our metric and reward are both based on VIEScore, we use separate MLLMs for evaluation (PPT4o-min) and RL verifier (Qwen2.5-VL-72B)) to reduce the risk of metric hacking, i.c. overfitting to MLLM-fpcccfic biases.
Evaluation Metric
We adopt VIEScore 27
Evaluation Benchmarks We benchmark on datasets covering both simple (OmniEdit (52), EmuEdit (43) and complex edits (MagicBrush [6]], Aurora [26). and 12EBench (34). EmuEdit and 12EBench serve as outtt--distribution (OOD) evaluations, with 12EBench including unseen edit types such as lowlight enhancement. We also repurpose VisMin [2), originally image understanding benchmark, into an editing benchmark by generating edit instructions from captions (App. B.2.2). These datasets span wide range of edit types and difficulty levels, enabling robust evaluation. To manage VIEScore API costs, we use a 1000-sample subset for 12EBench and EmuEdit.
Baselines We compare our model to diffusion- based baselines. including MagicBrush 61., In- structPix2Pix [8). Aurora (26), and Omnigen, which is the SOTA image editing model [56]. We also compare with EditAR (38], which, to the best of our knowledge, the only fully autoregressive image editing model.
5 Results
5.1 Teaching Emu3 Image Editing with Supervised Fine-TTree Fine-Tuning
Simple Editing The first row of the Supervised Fine-Tuning section in Tab. presents the results of SFT trained on simple data (SFT (S)). It achieves the highest score on OmniEdit (5.73) and average score of 3. 3.88, outperforming MagicBrush (3.32) and InstructPix2Pix (3.26), but underperforming Aurora (4.17), EditAR (4.20), and Omnigen (4.70). Notably, Omnigen benefits from a stronger base model with superior image generation performance (GenEval [T8] 0.70) and large-scale finetuning (~ 4M image editing samples). In contrast, our model finetunes from a weaker base model (Emu3, GenEval 0.64) and uses significantly less data (750K' image editing samples), resulting in lower overall performance. Also, we see that all models, including Omnigen, perform significantly worse on complex editing benchmarks compared to simple editing benchmarks, highlighting challenges in spatial edits, changes in object count, and human actions. Our trained SFT (S) follows this trend. We next explore how to improve SFT performance on both simple and complex edits.
Complex Editing To improve the ability to handle complex edits, we explore two SFT strategies: joint training on the combined simple and complex edit data (SFT (S+C)) and a two-stage curriculum that first finetunes on simple edits, then on complex edits (SFT (S+C) two-stage). As shown in Tab. 2 joint training (SFT (S+C)) reduces average performance compared to simple-only finetuning (SFT (S)) from 3.88 to 3.32 across both simple and complex editing tasks. In particular, the performance drop is significant on simple edit benchmarks, dropping from 5.73 to 4.64 on OmniEdit and from 3.66 to 2.89 on EmuEdit. We hypothesize that this degradation due to the large distributional shift between simple and complex edits; mixing them early in training may hinder the model's 's ability to generalize across either. In contrast, the two-stage curriculum partially recovers average performance (3.69) and improves results on some complex edit benchmarks (e.g., VisMin, MB). This suggests that allowing the model to first acquire basic editing capabilities from simple data makes subsequent finetuning complex tasks more effective, especially given that Emu3 has not been exposed to any
7

---

Table 2:
SFT and RL model variants for image editing fine-tuning and posiningining respectively. S stands for data used in simple editing types, and C stands for data from complex editing types. and $ denote Simple and Complex Edit benchmarks, respectively. denotes the best- rest-performing prior state-of-the-an method, a diffusion- based model trained from scratch using approximately ~5x data. Bold numbers indicate the best performances across all methods. Green numbers indicate the performance gain of EARL compared to the SFT (S) baseline.
Model/Data
Base Model
OmniFdit!
EmuuEdit
AURORA
MB
VisMin'
I2E.Bench
AVG
Magicbrush
SD v1.5
3.28
3.01
3.64
3,48
3.0G
3.32
InstrnctPix2Pix
SD vl.5
3,97
3.24
3.05
3.12
2.94
3.23
3.26
Aurora
SD vL.5
4.50
4.40
4.12
4.62
3.82
3.58
4.17
EditAR
LlamaGen
5.29
3.79
3.84
4,54
4.20
Omnigenn
5.68
5.00
4,10
4.68
4.09
4,68
4,70
Supervised Fine-tuning
Emu3
3,66
3.58
3.19
3.57
SFT (S+C)
Emu3
4.64
2.89
2.81
2.89
3.91
2.77
3.32
SIT (S+C) two-stage
Emn3
4.23
3.29
3,60
3,40
4.56
3.07
3.69
Post-training
SFT (S)
Emu3
6.07
4.13
3.47
3.53
3.34
3.80
4,06
SFT
(C)
Ema3
5.94
4.12
3.84
3.92
4.09
3,90
4.30
SET (S) RL (S+C)
Emu3
6.33
4,28
3,99
4.26
4.48
4,08
4.57
(S+C)
Emu3
3.86
SFT (S+C) RL (S+C)
Emu3
5,70
4,09
3,97
4.35
4.97
3.84
4.49
SFT (S-C) two-stage + RL (C)
Emu3
4.21
3.16
3.05
3.33
4.16
2.99
3.48
SPT (S+C) two-stage (8+C)
Emu3
5.29
3.89
3.85
4.20
4,70
3.56
4,25
RL Post-training Scaled
EARL SFT RL (S+C)
4,93
A EARL SFT RL (8+C) SFT(S)
+0.66
+0.81
+0.69
+1.33
+1.36
+0.60
+0.92
editing data during pretraining, Overall, the SFT results show that supervised finetuning is insufficient to effectively learn complex editing tasks.
5.2 Pushing Image Editing with RL Post-training
In this section, we present results showing that RL post-training substantially improves image editing performance. Starting from the SFT (S) model trained only simple edits, apply RL post-training under three settings: RL (S), which uses only simple edit data; RL (C), which uses only complex data; and RL (S+C), which uses both. Tab. 2 shows that all RL variants outperform the SFT baseline. RL (C), trained on a disjoint set of complex edits, outperforms RL (S) across all complex (C) benchmarks without significant drop simple (S) benchmarks. This contrasts with the SFT setting, where adding complex data degraded simple-edit performance. These results indicate that incorporating complex data during RL helps the model learn complex editing while preserving performance on simple edits. The largest gain comes from balancing simple and complex data, i.e., RL (S+C). The best-performing setup, SFT((S) RL(S+C), improves the average score from 3.88 to 4.57, surpassing MagicBrush, InstructPix2P55 Aurora, and EditAR, and remaining competitive with Omnigen. As shown TTb. 11 even best-of-5 sampling from the SFT model cannot match RL performance, confirming the gains stem from policy optimization. In addition to accuracy, we analyze inference efficiency in App. F.2 at 256 256, EARL runs about 4x faster than the SOTA Omnigen with similar quality, while remaining 2x slower than lightweight editors.
Applying RI. to models pre-trained both simple and complex data (SFT (S+C) and two-stage) yields modest gains over SFT, with average scores to 4.49, still below RL on the simple-only base (SFT (S) RL (S+C)) Fig. shows stable RL training SFT(S) - RL(S+C) with consistent reward improvements and rising VIEScore on OmniEEdit, indicating healthy learning dynamics. The two-stage SFT (S+C) - RL (C) variant reaches only 3.48, far below SFT (S) RL (S). Because SFT (S+C) variants underperform relative to SFT (S), their RL counterparts also remain weaker. We hypothesize that supervised finetuning on complex edits may degrade the base model's 's core capabilities, limiting RL's ability to recover or improve performance. This aligns with findings in LLM research, where base model capability critically influences RI. finetuning success [20] Thus, while complex data is essential, naively including it during SFT can constrain RL's effectiveness.
Scaling RL Training with More Steps and Data To further improve performance, we scale the best-performing setup: SFT(S) - RL(S+C), by increasing the duration of RL training to 2000 steps and using larger data pool of 300K (S+C) samples. At each step, 16 unique examples sampled,

---

(a) Training Reward
(b) OmniEdit Benchmark
7.0
7.00
VIEScore
VIEScore
6.75
6.50
5.3
4.5
5.75
4.0g
200
400
6.0
800
1000
5.00
200
300
400
500
0.00
700
800
9000
1900
Training Iteration
Training Iteration
Reward
Edit Success
Artifacts
Naturainess
Overedit
Perceptual Quality
Figure 4: (a) Training curves showing the reward progression, with different aspects of reward. and (b) VIEScore on OmniEdit increases with RL training iterations
Simple Replace
Complex edit: Positional change
Replace the handlebar mustache with beard.
7,48
Swap the bicucle and the ambulance.
7,48
6.48
Complex edit: Counting
Complex
6.48
4.58
Bring his body.
X 5.29
Figure 5: Example reward scores for various types of image edits using Qwen2.5-VI--7BB. Higher scores reflect better alignment with the intended edit prompt.
resulting in 32k total samples seen (20x more than the 1.6k samples from earlier runs). This scaled configuration achieves the best overall results and referred to EARL in our evaluations. As shown in the last section of Tab. 2 EARL surpasses all baselines (MagicBrush, InstructPix2Pix, Aurora, EditAR, and the SOTA Omnigen), achieving an average score of 4.80 compared to 4.70 for Omnigen. In particular, on OmniEdit, AURORA, and VisMin benchmarks, EARL achieves the best results over prior models. We also achieve strong performance on the out-of-distribution benchmarks like 12EBench and EmuEdit.
Qualitative Analysis of RL Post-training We observe that the SFT model sometimes produces noisy artifacts or over-edits the surrounding regions. also fails to achieve successful edits consis- tently. However, when sampled multiple times, at least one output usually satisfies the correct edit (see examples in Fig. 5). Our verifier captures key aspects of edit quality: edit success, over-editing, presence of artifacts, and naturalness, guide RL. training toward more reliable outputs. Qualitative inspection shows artifacts are nearly eliminated, edits are more precise, and success rates improve after applying RL. Fig. illlustrates instances where the verifier successfully handles both simple and complex edit tasks. Nevertheless, the reward model is limited by the multimodal image-text understanding Qwen2.5-VL-72B. For example, in complex edits, such changing higher counts (e.g., six to four), the reward becomes less reliable (see App. D.I for more details). This also explains why the performance is higher on simple edits than complex edits in EARL. Importantly, strong reward model is crucial for effective RL. Our 72B verifier demonstrates significantly better alignment with human judgment than smaller 7B counterpart (see Appp D.2 and App. D.3). To better assess FARL's robustness and remaining limitations, we report CLIP similarity and FID metrics App. E.2] provide fine-grained qualitative analyses in App. H.5) and include detailed comparison
with the only autoregressive baseline EditAR [88 in App. E.3

---

5.3 Studying Chain-of-thought Reasoning for Editing
Table 3: Performance of EARL variants with chain-of-thought supervision. i and represents Simple
and Complex Edit benchmarks, respectively.
Model/Data
OmniEdit'
EmuEdit
AURORA
MB'
VisMin
12EBench
AVG
SFT (S)
5.73
3.66
3.58
3.19
3.57
3.59
3.88
SFT think (S)
4.34
3.76
2.88
3,36
3,46
3.21
3.50
SFT think (S+C) two-stage
1.44
1.41
1.03
1.58
2.45
1.20
1.52
SFT think (S) + RL (S)
4.99
3.73
3.33
3,48
3.11
3.46
3.68
SFT think (S) + RL (C)
4.36
3.67
2.94
3,59
3.08
3.16
3.47
EARL SFT think (S) RL (S+C)
4.65
3.78
3.23
3.67
3.39
3.36
3.68
We evaluate the effect of incorporating chain-of-thought reasoning supervision image editing performance. Since Chain-of- ofTThhtht (CoT) reasoning involves alternating between generating text and images, it requires interleaved image-text generation. However, our base model Emu3 was not pretrained for this type of multimodal generation, which presents additional challenges.
We compare two SFT variants with CoT reasoning supervision: (1) SFT think (S), and (2) SFT think (S+C) using a two-stage approach. Results in Tab. show that, despite the success of chain-of-thouge reasoning in large language models [20). SFT think (S) (3.50 avg.) does not improve visual editing performance compared to SFT (S) (3.88 avg.). We also see that adding complex reasoning data (C) during SFT hurts performance, two-stage setup compared to SFT think (S). This is consistent with Section 5.1, where including complex reasoning data (SFT (S+C)) and the two-stage (SFT (S+C) two-stage) led to performance degradation compared to SFT (S). Next, we apply RL on top of SFT think (S), using the RL(S), RL (C), and RL (S+C) variants. We leave out applying RL on top of SFT think (S+C) two stage too weak. We perform RL post-training for up to 2,000 steps, or until divergence, using 16 unique samples per step. Applying RL on SFT think (S) yields a slight improvement across two settings; RL (S) and RL (S+C). On average, RL (S) improves performance from 3.50 (SFT (S)) to 3.68, while RL (S+C) also improves from 3.50 (SFT (S)) to 3.68. These observations align with findings in the LLM literature. First, COT reasoning tends to help only once model surpasses a capability threshold
shown by the drop in performance in the SFT think (S+C)
is
10
53
Second, RL provides limited benefit when the base model is still sub-optimal 33
Quality of Reasoning Chains The SFT think model generates plausible reasoning: correctly identifies target regions, plans edits, and describes intended outcomes. However, its final outputs often show lower edit accuracy, reduced naturalness, and more artifacts compared to the standard SFT model (see App. E.6 for qualitative examples). These results suggest that although the model learns to generate appropriate reasoning, it does not effectively apply it during generation of the edited image. We hypothesize that this limitation is due to the model's 's lack of pretraining on interlcaved image-text- image data. While the model can generate plausible reasoning chains, it struggles to integrate them effectively for image enhancement, likely because was not trained integrate these modalities in cohesive manner. We leave further exploration of this issue to future work.
Conclusion
This work delivers the first systematic comparison of supervised finetuning, reinforcement learning, and CoT reasoning for text-guided image editing within a unified autoregressive framework. Directly motivated by this analysis, we introduce a novel autoregressive image editing model. EARL. EARL performs on par with the strongest open-source basclines while using less data, and sets new bar for multimodal autoregressive models for editing. While SFT alone proves insufficient for handling complex edits, we find that RL significantly improves performance, enhancing the model's 's overall edit success and ability to handle tasks involving spatial reasoning and dynamic interactions. We also address the question of when to introduce complex edits during the various training stages: we found that bringing in complex edits is not helpful during the SFT stage, but is beneficial during the RL post-training stage. Lastly, the COT reasoning supervision experiment did not lead to consistent improvements, highlighting the need for further research and stronger autoregressive base models with strong reasoning capabilities. We discuss limitations and broader impact of our work in App. G
10

---

7 Acknowledgments
We acknowledge the valuable feedback provided by Qian Yang. Le Zhang, and Oscar Manas on an early draft of the paper. The technical support extended by the Mila IDT and TamlA teams in managing the computational infrastructure greatly appreciated. During this project, Aishwarya Agrawal was supported by the Canada CIFAR AI Chair award.
References
[1] Saba Ahmadi and Aishwarya Agrawal. An examination of the robustness of reference-free image captioning evaluation metries. In Yvette Graham and Matthew Purver, editors, Findings of the Association for Computational Linguistics: EACL 2024, pages 196-208, St. Julian's s, Malta, March 2024, Association for Computational Linguistics.
[2] Rabiul Awal, Saba Ahmadi, Le Zhang, and Aishwarya Agrawal. Vismin: Visual minimal- change understanding. In A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang, editors, Advances in Neural Information Processing Systems, volume 37, pages 107795-107822 Curran Associates, Inc., 2024.
[3] Jinbin Bai, Wei Chow, Ling Yang, Xiangtai Li, Juncheng Li, Hanwang Zhang, and Shuicheng Yan. Humanedit: A high-quality human-rewarded dataset for instruction-based image editing. arXiv preprint arXiv:2412.04288 2024.
[4] Jinze Bai, Shuai Bai, Yunfci Chu, Zeyu Cui, Kai Dang, Xiaodong Deng. Yang Fan, Wenbin Ge, Yu Han, Fei Huang, Binyuan Hui, Luo Ji, Mei Li, Junyang Lin, Runji Lin, Dayiheng Liu, Gao Liu, Chengqiang Lu, Keming Lu, Jianxin Ma, Rui Men, Xingzhang Ren, Xuancheng Ren, Chuanqi Tan, Sinan Tan, Jianhong Tu, Peng Wang, Shijie Wang. Wei Wang, Shengguang Wu, Benfeng Xu, Jin Xu, An Yang, Hao Yang, Jian Yang. Shusheng Yang. Yang Yao, Bowen Yu, Hongyi Yuan, Zheng Yuan, Jianwei Zhang, Xingxuan Zhang, Yichang Zhang, Zhenru Zhang, Chang Zhou, Jingren Zhou, Xiaohuan Zhou, and Tianhang Zhu. Qwen technical report. arXiv preprint arXiv:2309. 16609, 2023.
[5] James Betker, Gabriel Goh, Li Jing, Tim Brooks, Jianfeng Wang, Linjie Li, Long Ouyang, Juntang Zhuang, Joyce Lee, Yufei Guo, et al. Improving image generation with better captions. Computer Science. https://edn. openai. com/papers/dalll-- pdf. 2(3):8, 2023.
[6] Kevin Black, Michael Janner, Yilun Du, Ilya Kostrikov, and Sergey Levine. Training diffusion models with reinforcement learning. In The Twelfth International Conference on Learning Representations, 2024.
[7] Kevin Black, Mitsuhiko Nakamoto, Pranav Atreya, Homer Rich Walke, Chelsea Finn, Aviral Kumar, and Sergey Levine. Zero-shot robotic manipulation with pre-trained image-editing diffusion models. In The Twelfth International Conference Learning Representations, 2024.
[8] Tim Brooks, Aleksander Holynski, and Alexci A Efros. Instructpix2pix: Learning to follow image editing instructions. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 15392-184022 2023.
[9] Ethan Chern, Jiadi Su, Yan Ma, and Pengfei Liu. Anole: An open, autoregressive, native large multimodal models for interleaved image-text generation. arXiv preprint arXiv:2407.061333 2024.
[10] Hyung Won Chung, Le Hou, S. Longpre, Barret Zoph, Yi Tay, William Fedus, Bric Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, Albert Webson, Shixiang Shane Gu, Zhuyun Dai, Mirac Suzgun, Xinyun Chen, Aakanksha Chowdhery, Dasha Valter, Sharan Narang, Gaurav Mishra, Adams Wei Yu, Vincent Zhao, Yanping Huang, Andrew M. Dai, Hongkun Yu. Slav Petrov, Ed H. Chi, Jeff Dean, Jacob Devlin, Adam Roberts, Denny Zhou, Quoe V. Le, and Jason Wei. Scaling cdaructionfinettuee language models. ArXiv, abs/2210.1 1111110..111,,... 2022.
[11] Guillaume Couairon, Jakob Verbeek, Holger Schwenk, and Matthieu Cord. Diffedit: Diffusion- based semantic image editing with mask guidance. In The Eleventh International Conference on Learning Representations, 2023.
11

---

[12] Chaorui Deng, Deyao Zhu, Kunchang Li, Chenhui Gou, Feng Li, Zeyu Wang, Shu Zhong, Weihao Yu, Xiaonan Nie, Ziang Song, Guang Shi, and Haoqi Fan. Emerging properties in unified multimodal pretraining, 2025.
[13] Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling rectified flow trans- formers for high-resolution image synthesis. In Forty-first international conference on machine learning, 2024.
[14] Rongyao Fang, Chengqi Duan, Kun Wang, Linjiang Huang, Hao Li, Shilin Yan, Hao Tian, Xingyu Zeng, Rui Zhao, Jifeng Dai, et al. Got: Unleashing reasoning capability of multimodal large language model for visual generation and editing. arXiv preprint arXiv:2503. 10639, 2025.
[15] Weixi Feng, Wanrong Zhu, Tsu-jui Fu, Varun Jampani, Arjun Akula, Xuehai Hc. Sugato Basu, Xin Eric Wang, and William Yang Wang. Layoutgpt: Compositional visual planning and generation with large language models. Advances in Neural Information Processing Systems, 36:18225-1822 18250, 2023.
[16] Yutong Feng, Biao Gong, Di Chen, Yujun Shen, Yu Liu, and Jingren Zhou. Ranni: Taming text-to-image diffusion for accurate instruction following. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4744-4753, 2024.
[17] Tsu-Jui Fu, Wenze Hu, Xianzhi Du, William Yang Wang, Yinfei Yang, and Zhe Gan. Guiding instruction-based image editing via multimodal large language models. ArXiv, abs/2309.17102. 2023.
[18] Dhruba Ghosh, Hannaneh Hajishirzi, and Ludwig Schmidt. Geneval: An object-focused framework for evaluating text-to-image alignment. Advances in Neural Information Processing Systems, 36:52132-52152,, 2023.
[19] Raghav Goyal, Samira Ebrahimi Kahou, Vincent Michalski, Joanna Materzynska, Susanne Westphal, Heuna Kim, Valentin Haenel, Ingo Fruend, Peter Yianilos, Moritz Mueller-Freitag. al. The" something something" video database for learning and evaluating visual common sense. In Proceedings of the IEEE international conference on computer vision, pages 5842- 5850, 2017.
[20] Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-rl: Incentivizing reasoning capability in Ilms via reinforcement learning. arXiv preprini arXiv:2501. 2025.
[21] Ziyu Guo, Renrui Zhang, Chengzhuo Tong, Zhizheng Zhao, Peng Gao, Hongsheng Li, and Pheng-Ann Heng. Can we generate images with cot? let's verify and reinforce image generation step by step. arXiv preprint 21..::::2500.990 2025.
[22] Amir Hertz, Ron Mokady, Jay Tenenbaum, Kfir Aberman, Yael Pritch, and Daniel Cohen-Or. Prompt-to- toopptt image editing with cross attention control. 2022.
[23] Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840-688 2020.
[24] Xuan Ju, Ailing Zeng, Yuxuan Bian, Shaoteng Liu, and Qiang Xu. Direct inversion: Boosting diffusion-based editing with 3 lines of code. arXiv preprini 13000150 2023.
[25] Amirhossein Kazemnejad, Milad Aghajohari, Alessandro Sordoni, Aaron Courville, and Siva Reddy. Nano aha! moment: Single file "rl for Ilm" library.
com/
McGill 1EE:111NLPPAA WWW./NNLPOnnnon aaanno--hammmmmentt 2025. GitHub repository.
[26] Benno Krojer. Dheeraj Vattikonda, Luis Lara, Varun Jampani, Eva Portelance, Christopher Pal, and Siva Reddy. Learning action and reasoning-centric image editing from videos and simulation. In A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomezak, and C. Zhang, editors, Advances Neural Information Processing Systems, volume 37, pages 38035-38078. Curran Associates, Inc., 2024.
12

---

[27] Max Ku, Dongfu Jiang, Cong Wei, Xiang Yue, and Wenhu Chen. Viescore: Towards explainable metries for conditional image synthesis evaluation. arXiv preprint arXiv:2312.14867, 2023.
[28] Max Ku, Tianle Li, Kai Zhang, Yujie Lu, Xingyu Fu, Wenwen Zhuang, and Wenhu Chen. Imagenhub: Standardizing the evaluation of conditional image generation models. arXiv preprint arXiv:2310.01596 2023.
[29] Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. Efficient memory management for large lan- guage model serving with pagedattention. In Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles, 2023.
[30] Tiancheng Li, Jinxiu Liu, Huajun Chen, and Qi Liu. Instructri4pix: Training diffusion for image editing by reinforcement learning. arXiv preprint arXiv:2406.0999 2024.
[31] Yuheng Li, Haotian Liu, Qingyang Wu, Fangzhou Mu, Jianwei Yang, Jianfeng Gao, Chunyuan Li, and Yong Jae Lee. Gligen: Open-set grounded text-to-image generation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 2251 2023.
[32] Jie Liu, Gongye Liu, Jiajun Liang, Yangguang Li, Jiaheng Liu, Xintao Wang, Pengfei Wan, Di Zhang, and Wanli Ouyang. Flow-grpo: Training flow matching models via online rl. arXiv preprint S55i:220550047000 2025.
[33] Zichen Liu, Changyu Chen, Wenjun Li, Penghui Qi, Tianyu Pang, Chao Du, Wee Sun Lee, and Min Lin. Understanding rl -e-zerollkee training: A critical perspective. arXiv preprint 2rXiv:2503.27783 2025.
[34] Yiwei Ma, Jiayi Ji, Ke Ye, Weihuang Lin, zhibin wang, Yonghan Zheng, Qiang Zhou, Xiaoshuai Sun, and Rongrong Ji. 12EBench: A comprehensive benchmark for instruction- based image editing. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024.
[35] Chenlin Meng. Yutong He, Yang Song, Jiaming Song. Jiajun Wu, Jun-Yan Zhu, and Stefano Ermon. SDEdit: Guided image synthesis and editing with stochastic differential equations. In International Conference on Learning Reprexentations, 2022.
[36] Oscar Michel, Anand Bhattad, Eli VanderBilt, Ranjay Krishna, Aniruddha Kembhavi, and maanuuage-guieed guided 3d-aware image editing. Advances in Neural
Tanmay Gupta. Object 3dit: Language Information Processing Systems, 36:3497-3516 2023.
[37] Chong Mou, Xintao Wang, Liangbin Xie, Yanze Wu, Jian Zhang, Zhongang Qi, and Ying Shan. T2i-adapter: Learning adapters dig out more controllable ability for text-to-image diffusion models. In Proceedings of the AAAI conference on artificial intelligence, volume 38, pages 4296 4304, 2024.
[38] Jiteng Mu, Nuno Vasconcelos, and Xiaolong Wang. Editar: Unified conditional generation with autoregressive models. arXiv preprint arXiv:2501.04669 2025.
[39] Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion models for high-resolution image synthesis. arXiv preprint arXiv:2307.01952. 2023.
[40] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEF/CVF conference on computer vision and pattern recognition, pages 10684-10695, 2022.
[41] Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, et Laion- 5b: An open large-scale dataset for training next generation image-text models. Advances in neural information processing systems, 35:25278-25222 2022.
13

---

[42] Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint #xXv:24422033300, 2024.
[43] Shelly Sheynin, Adam Polyak, Uriel Singer, Yuval Kirstain, Amit Zohar, Oron Ashual. Devi Parikh, and Yaniv Taigman. Emu edit: Precise image editing via recognition and generation tasks. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 8871-8879, 2024.
[44] Tomáš Souček, Dima Damen, Michael Wray, Ivan Laptev, and Josef Sivic. Genhowto: Learn- ing to generate actions and state transfornations from instructional videos. arXiv preprint arXiv:2312.07322, 2023.
[45] Chameleon Team. Chameleon: Mixed-modal early-fusion foundation models. arXiv preprint arXiv:2405.09818. 2024.
[46] Shengbang Tong, David Fan, Jiachen Zhu, Yunyang Xiong, Xinlei Chen, Koustuv Sinha, Michael Rabbat, Yann LeCun, Saining Xie, and Zhuang Liu. Metamorph: Multimodal under- standing and generation via instruction tuning. arXiv preprint arXiv:2412. 14164, 2024.
[47] Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, et al. Llama 2: Open foundation and fine-tuned chat models, 2023.
[48] Bram Wallace, Meihua Dang, Rafael Rafailov, Linqi Zhou, Aaron Lou, Senthil Purushwalkam, Stefano Ermon, Caiming Xiong, Shafiq Joty, and Nikhil Naik. Diffusion model alignment using direct preference optimization. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 8228-8238, 2024.
[49] Junke Wang, Zhi Tian, Xun Wang, Xinyu Zhang, Weilin Huang, Zuxuan Wu, and Yu-Gang Jiang. Simplear: Pushing the frontier of autoregressive visual generation through pretraining, sft, and rl. arXiv preprint arXi::2504. //45, 2025.
[50] Xinlong Wang, Xiaosong Zhang, Zhengxiong Luo, Quan Sun, Yufeng Cui, Jinsheng Wang, Fan Zhang, Yueze Wang, Zhen Li, Qiying Yu, Yingli Zhao, Yulong Ao, Xuebin Min, Tao Li, Boya Wu, Bo Zhao, Bowen Zhang, Liangdong Wang, Guang Liu, Zheqi He, Xi Yang, Jingjing Liu, Yonghua Lin, Tiejun Huang, and Zhongyuan Wang. Emu3: Next-token prediction is all you need, 2024.
[51] Zhou Wang, A.C. Bovik, H.R. Sheikh, and E.P. Simoncelli. Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing, 1300)6600-11,, 2004.
[52] Cong Wei, Zheyang Xiong, Weiming Ren, Xeron Du Ge Zhang, and Wenhu Chen. Omniedit: Building image editing generalist models through specialist supervision. In The Thirteenth International Conference on Learning Representations, 2025.
[53] Jason Wei. Xuezhi Wang, Dale Schuurmans, Maarten Bosma, brian ichter, Fei Xia, Ed H. Chi, Quoe V Le, and Denny Zhou. Chain of thought prompting elicits reasoning in large language models. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho, editors, Advances in Neural Information Processing Systems, 2022.
[54] Chenfei Wu, Shengming Yin, Weizhen Qi, Xiaodong Wang. Zecheng Tang, and Nan Duan. Visual chatgpt: Talking, drawing and editing with visual foundation models. arXiv preprint WwXiv:2303004611, 2023.
[55] Chengyue Wu, Xiaokang Chen, Zhiyu Wu, Yiyang Ma, Xingchao Liu, Zizheng Pan, Wen Liu, Zhenda Xie, Xingkai Yu, Chong Ruan, et al. Janus: Decoupling visual encoding for unified multimodal understanding and generation. arXiv preprint arXiv:2410.138.. 2024.
[56] Shitao Xiao, Yueze Wang, Junjie Zhou, Huaying Yuan, Xingrun Xing, Ruiran Yan, Shut- ing Wang, Tiejun Huang, and Zheng Liu. Omnigen: Unified image generation. CoRR, abs/2409. 1340, 2024.
14

---

[57] Kai Yang, Jian Tao, Jiafei Lyu, Chunjiang Ge. Jiaxin Chen. Weihan Shen, Xiaolong Zhu, and Xiu Li. Using human feedback to fine-tune diffusion models without any reward model. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8941-8951, 2024.
[58]
Qwen An Yang, Baosong Yang. Beichen Zhang, Binyuan Hui, Bo Zheng. Bowen Yu, Chengyuan Li. Dayiheng Liu, Fei Huang, Guanting Dong, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang. Jianxin Yang, Jiaxin Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Mei Li, Mingfeng Xue, Pei Zhang, Qin Zhu, Rui Men, Runji Lin, Tianhao Li, Tingyu Xia, Xingzhang Ren, Xuancheng Ren, Yang Fan, Yang Su, Yi-Chao Zhang, Yunyang Wan, Yuqi Liu, Zeyu Cui, Zhenru Zhang, Zihan Qiu, Shanghaoran Quan, and Zekun Wang. Qwen2.5 technical report. ArXiv, abs/2412.15115. 2024.
[59] Sherry Yang. Yilun Du, Seyed Kamyar Seyed Ghasemipour, Jonathan Tompson, Leslie Pack Kaclbling, Dale Schuurmans, and Pieter Abbeel. Learning interactive real-world simulators. In The Twelfth International Conference on Learming Representations, 2024.
[60] Mert Yuksekgonul, Federico Bianchi, Pratyusha Kalluri, Dan Jurafsky, and James Zou, When and why vision-language models behave like bags-of-words, and what to do about it? In The Eleventh International Conference on Learning Representations, 2023.
[61] Kai Zhang, Lingbo Mo, Wenhu Chen, Huan Sun, and Yu Su. Magicbrush: A manually annotated dataset for instruction-guide image editing. Advances Neural Information Processing Systems, 36, 2024.
[62] Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unreason- able effectiveness of deep features as perceptual metric. In CVPR, 2018.
[63] Shu Zhang, Xinyi Yang, Yihao Feng. Can Qin, Chia-Chih Chen, Ning Yu, Zeyuan Chen, Haiquan Wang. Silvio Savarese, Stefano Ermon, Caiming Xiong, and Ran Xu. Hive: Harnessing human feedback for instructional visual editing. ArXiv, abs/2303.099 2300109618,,,, 2023.
15

---

Overview of Appendix
A Background
B Training Datasets
VIEScore Alignment with Human Judgment
D Analysis of the Verifier Model
Analysis of Model Outputs
Experimental Setup and Compute Resources
G Limitations and Broader Impact
H Behind the Scenes
I Author Contributions
A Background
A.1
Emu3
Emu3 [50] (8B parameters) is an autoregressive multimodal model. It extends the LLaMA-2 147 architecture by integrating vision tokenizer DEBER-MOVQGAN for encoding images into discrete tokens and a text tokenizer (QwenTokeniizer) for processing textual inputs. Unlike diffusion-based models [56], Emu3 generates text and visual tokens in a unified manner.
In this work, data. During pre-training, Emu3 formats image-text multimodal inputs in a structured format (shown below), incorporating special tokens to distinguish between the different modalities:
the base version of Emu3, which is pre-trained exclusively text and image
"[BOS] [caption text) [SOV] [resolution info [SOT] (vision tokens) | [EOV] [EOS]"
where [BOS] and [EOS] mark the beginning and end of the sequence, [SOV] and [EOV] define the boundaries image metadata such as resolution, image tokens, and [SOT] indicates the start of vision tokens. Additionally, [EOL] and [EOF] are included within vision tokens denote line and frame breaks, respectively. The resolution info section contains relevant details to image resolution.
A.2 GRPO: Group Relative Policy Optimization
Algorithm 1: Group Relative Policy Optimization
Input:
initial policy model To... reward models r&: task prompts D; hyperparameters c, B, P Output: final policy model Ty
Function Group Relative Policy Optimization:
Tg Te
Tief f- TB. for step = 1 to M do
Sample batch D, from D;
Update the old policy model Tew Te
Sample G outputs {or).
Te,, (-19) for each question q Db
Compute rewards {r:) for each sampled output o; by running ro
Compute Å,, for the t-th token of o; through group relative advantage estimation for GRPO iteration / to μ do
Update the policy model my by maximizing the GRPO objective
Algorithm I presents the GRPO algorithm, where we prepare the advantages using the current policy estimate the gradients. noted in [Sec. we set backward batch size equal to the rollout batch size, which effectively reduces the second Toop single iteration. This means policy gradients are estimated in fully online manner, with advantages always computed from the current policy. This design choice contributes to improved stability during training.
16

---

B Training Datasets
B.1 Dataset Composition
Our goal of building unified model requires exposure to a wide range of edit types both simple (e.g., object, attribute, or style changes) and complex (c.g., action changes, counting changes, or spatial relations). No single dataset offers comprchensive coverage of all these edit types, and existing models are typically developed a fragmented manner, each targeting only a subset of edits.
To address this, we consolidate several existing datasets to form a unified training pool that spans the full spectrum of edit types. OmniEdit [52] serves as our largest source of simple editing examples. For complex edits involving actions, relations, counting, and other real-world applications which are significantly less represented than simple edits we incorporate data from VisMin (2), Aurora (26).
Human-Edd Edit
3 and MagicBrush 61].
B.2 Creating Chain-of-Thoughee from Existing Editing Datasets
This section describes the chain-of-thought datasets for image editing and the associated prompt designs. Image editing datasets typically lack reasoning chains. To address this, we synthesize reasoning chains using Multimodal Large Language Model (MLLM) through in-context learning (prompting), based on existing resources such as the input image, edited image, and edit instruction. We use Qwen2. 5- 5-VL-72B (58) as the MILM.
In diffusion models, masking a common form of conditioning. To mimic this, we collect bounding boxes - either by using those provided in the dataset or by estimating them from available masks and pixel differences. These bounding boxes serve as conditioning inputs in the reasoning chain. The model analyzes the image, edit instruction, and target region (via the bounding box) to generate step-by-step reasoning process, referred to as the thinking field. The system prompt and annotated few-shot examples are provided in App. B.2. and App. B.2.3 respectiiiy..
For VisMin, we use image-text pairs from the VisMin [ dataset, where the two images differ minimally making them well-suited for editing tasks. Since examples of complex edits, such as those involving counting or spatial relations, are scarce, repurposing VisMin provides useful coverage for these cases. The change can be inferred from the two image captions, which we use to guide Qwen2. 5VL-72B through prompting with few-shot demonstrations. Additional bounding box metadata is used to generate both the edit instruction and the corresponding thinking field, detailed in App. B.2.2
B.2.1 LLM system prompts for creating reasoning chains
Prompt for In-context Learning of Chain-of-thought Editing
Your taak ls generate step-by-atup edit plan for the given edit taok based oa the input ixage and edit instruction. guidel
rasaoning
thorough
cally
1. Analyze the source image in depth, describing its nain elenenta, context, and any zelevant visual detaila.
2. Identify the object to be edited, providing specific datails about tha appearance, role the and
dist.inuishhing fenturea,
3. Clearly specify the area to be edited, including bounding box coordinates if provided, and explain why this region
ix chamen in ralation to the object and the acaee.
4. Detail the exact changes to be made to the object arwa, refarancing viaual attributes, position, atyle, and hav
perforred 5. Deacribe the expected result, after adit, focusing how the editad inage should appear, how the new changed
previous object integrates with the acese, and any requiseenta for preserving surcouninn alerenta.
step loginally follows and provides sufficient depth for clear, actionable edit plan.
your responze
minbared
staps.
Input: containe:
Edit: Instruction:
The instruction specifying the ezact wdit to be nade the input
Imput Image:
cource image
ecsted.
Edited Image: The editad trage for refermnce (to inderatand the edit instruction is applied).
Ragion Coordinates: The region ta be edited (specifina la the aren to be add ted).
responze
follouing
"thinking":
*clnsett dotailed procedural editing stepss"
17

---

B.2.2 LLM prompting for converting VisMin image-text pairs to editing task
Prompt for In-context Learning of Chain-of tftougght Editing
Your generate step-oy- sopp-o--teep step edst plan
on inpet image and Your ressoning should thorough and lolic atructored. Pollow these guidel tnwx:
given
1. the image
describing its main eleamte, appearance,
relerant vieual detaile. the somne,
Identify object.
edited, providing apecific dataila
distingutehing features.
3. Clearly specity the area to be edited, including bounding box coordinates if provided, and explain why this region
ix chosen relation the object and the acene.
changes be made the edit should perforzed to maintain realien and coherence.
Detail the
object area, referencing vieual attributes, positicn, style,
5. Deacribe the repectad reault. after the adit, fomusing on how the edited image should appear, how the nev or changed requirenent for preserving surrounding elerents.
object intogrates with the
Fornst your responce as a concise, numbered list of stope. Ensure each step logically follows fron the previous one and provides sufficient depth for clear, actiorable edit plan.
Isput containe:
instruction epecitying the exact odit to Imput Image: The source image to be edsted.
Edit Instraction:
the imput imago.
Edited Image: The editad image for referance (to inderatand how the edit instraction is applied).
Coordinatee:
region
edited (epociries
edited).
Your response must be in the folloving 3500 format:
"odst, instruction": "cinsert edit request hore>", "thinking": "cInsert detailed procedural editing stepa>"
B.2.3 Reasoning chain examples
In the case of the VisMin dataset 22. which contains two minimally changed image-text pairs, we do not receive explicit edit instructions. Instead, we infer these instructions by prompting multi-modal large language model (MLLM) in few-shot setting, using both the input and the edited image captions as context. The VisMin dataset covers various categories, including object, attribute, counting, and relationship. For object and attribute categories, where the task typically involves editing a specific region in the image, the target edit region (bounding box) is sufficient to generate the reasoning chain. The model infers the edit instruction by understanding the object and its properties within the defined region. In contrast, the relationship category, which describes spatial changes between objects, requires both the source image coordinates and the edited image coordinates to derive the appropriate edit instruction. For counting tasks, the reasoning chain is created by comparing the source image's 's bounding boxes with the removed bounding boxes in the edited image. This allows the model infer the necessary edit to reflect the changes in object counts. We provide several examples of this process in App. B.2.3
For other image- datasets, where edit instructions are explicitly provided, we prompt the MLLM with the input image, edited image, and edit instruction along with bounding box coordinates or keypoints for the objects or persons involved. This structured input allows the model to generate the reasoning chain by clearly understanding both the task and the necessary image modifications. For instance, the reasoning chain might involve identifying objects, spatially manipulating them, or removing them based on the given instructions and bounding boxes. The output reasoning chain provides step-by-step breakdown of how the image should be edited to achieve the specified changes, ensuring that each instruction is logically followed. We also illustrate this process with further examples in App. B.2.3]
Chain-of-thought Reasoning Examples (VisMin dataset)
*INPUT*: {
"iapat, image" "See the input image",
"descripsion": "resove glasses from face',
tanpecttont "A person vearing g glaesen in a photo.". "odited, caption"" *A perzoa without glassee photo.".
"target_edit_regtonn [250, 100, 350, 150]
"thinking": "OUIPUT": {
1. image shows
glasses. Aala
2. region [250, 100, 360, 150), choving glaccee.
18

---

3. Remove the glassee shile preserving natural ckin texcure. Vala chould thor without glasses,",
"INPUT": {
"source, www.eiinggaa...e... coppioo *A bowl that la located the laft of cat". "edited, ixageed_ssggg_cpptiioo
that located the laft boel",
["s bowl*, [200, 150, 300, 250]], [* cat", [100, 225, 200, 276])
isage_condda"
(*a bou2*, [100, 225, 200, 2761),
Ias cat", [200, 150, 300, 250]]
).
"dirror"
insit__iattuctiini "Hove the boul to where the cat i2, and Lhw to sherw the boul is.",
"thinking": image rove boul from [200, 150, 300,
2. ssap pox
to [100, 225, 200, 275), and aimiltmeeusly rulocate the cat from [100, 225, 200, 275] to [200, 150, 300, 260]..
3. This will maintain each object's and ortentation while reverning their left-right rel relettionhhpp
"INPUT': {
www..ggccpppcion................ caption": "Two bicycles and notorcyclo."" "sourco weo__iiao_ccorde
"cource, image,
l'bicycic', [10, 171, 138, 266]], ("bicycle", [213, 171, 341, 256]] ("aotorcyclo", (331, 196, 802, 264]
}. "edited saage_capcion bassovedbourdiigggooo [[213, 171, 341, 256]]
bicycla notorcycle.",
).
"DUTFUT" { "odit,
second bicycle.",
"thinking": "1 The acene shova two bicycles and notorcycle.
2. To reduce one bicycle, welll resove
bicycle located [213, 171, 341, 256), leaving first [10, sotorcycle (331, 196, 502, 264] untauched.
3. This saistaing proper spacing between the rosaining objects.'
Chain-of-thought Reasoning Examples (VisMin dataset)
'INPUT":
"edit__inatreee inatruction:: "Rensvw bicyclaa", starget_edit_regton"" [160, 389, 275, 431]
"QUIFUT":
Shinkingg-haatn... th "Stepe to renore the bicycles:
1. The goal in to ranove the bicycles from the irage.
2. Identify objecte
sheele, defining
hirycles are found vithin haunding box [140, 389, 275, 431].
3. Use the bounding box to isolate and erase the the eurrounding elemenss.
bicycles while
image chould appear if bicyclos never thero.".
C VIEScore Alignment with Human Judgment
We use the VIEScore metric as our evaluation metric for the image editing task. VIEScore is based on MLLM prompting to judge the quality of given edited image w.r.t the given editing prompt and the source image. Specifically, the MLLM prompted twice, once judge the semantic quality (alignment with the prompt and overediting), and perceptual quality (such as realism and artifacts). VIEScore was originally only shown to be effective on ImagenHub (28 which contains primarily object- and attribute-centrr centric edits. We verify in Tab. 4haat correlation with human judges also convincing on various other edit types, i.e. many the complex edits in this paper such as spatial edits (WhatsUp column), action edits (Something Something, Action-Genome, Epic Kitchen columns) or counting (Kubric column). For this we correlate human judgements initially collected for AURORA-BEAA Bench [26] and VIEScore with GPT4o-mini as a backbone (which use throughout the
19

---

paper our main metric). We note that human judges in 126 were asked to only judge the semantic alignment and not lower level visual properties such as aesthetics. Thus it is not surprising that the correlation of the semantic component of VIEScore is higher. Overall we find correlations to be on par or sometimes exceeding with those shown in the original paper [27]. For example in the original paper, the overall VIEScore (with the best model GPT4 at the time) had correlation of 0.382 with
human raters on MagicBrush. We find the lowest correlations, as expected, is on hard action-centric prompts in the Epic Kitchen subset of AURORA-Bench.
Table 4: Correlation between VIEScore (GPT44-mini) and human judges on the 8 subtasks of AURORA- [26]. Specifically, we correlate the overall VTEScore and the semantic sub-score with humans. We note that human judges in (26) were asked to only judge the semantic alignment and not lower level visual properties such as aesthetics.
Metric
MagieBrush
Something Something
Action Genome
Epic Kitehen
Whatsup
Kubric
CLEVR
Emu-Edit
AVG
Overal 0.5353
0.3676
0.3585
0.1764 0,2462
0.5285
0.2649
0.5600
0,4000
0.4328
Semantic
0.6300
0.4305
0.3830
0.6011
0.3529
0.6187
0,5612
0.5288
D Analysis of the Verifier Model
D.I Qualitative Examples of Reward Model Limitations in Complex Edits
Instruction
Input
Output
Output
Output
Output
Output
Output
Output
Ousput a
rumber of birds from ton to thrice.
Rewarde:
0.0
0.0
4.5
20
Hemoow
apples except the
Rewards:
59
Moving mouse closee semotte
Rewards:
9.0
Swop posit ona of the and bloycle
Rewards:
6.0
Move towards the right
Rowords:
3.0
4.5
4.8
6.5
6.5
6.9
6.9
6.9
Figure 6: Examples of limitations of the verifier for the complex edits. The reward values correspond to the VIEScore given by our verifier.
Our reward model shows strong performance on simple edits involving object and attribute changes but exhibits limitations when verifying complex edits, as illustrated by several qualitative examples in Fig. 6
In Row 1, the task is to reduce the number of birds from to This large-count change difficult for the verifier to handle accurately, reflected in inconsistent scores across different outputs. For instance, although output and are quite similar, the rewards they receive differ significantly (0.0 vs. 4.5). This indicates uncertainty in reliably evaluating edits that involve large numerical changes. In contrast, Row is an example of small count changes. requires removing all apples except one. The verifier assigns consistently high scores to the correct generations (outputs 3 to 8), suggesting that smaller count changes are easier to verify, likely because the verifier is more exposed to such data during training.
20

---

In Row 3. the task involves moving mouse closer to a remote. Although the model generally assigns higher scores better samples, it struggles with inconsistencies: in output the mouse's 's appearance changes noticeably, and in output 7, the hand shows some distortion. This underscores the difficulty of verifying subtle positional changes compared simpler edits, such as additions and removals.
For Row 4, the edit requires swapping the positions of a white jeep and a bicycle. The verifier correctly favors outputs that accurately reflect this positional change and exhibit fewer artifacts and distortions, demonstrating sensitivity to spatial relationships and artifacts. For example, outputs 2 and 3 still show traces of the car in its original position. indicating both leftover artifacts and failure to correctly reflect the position swap. However, obtaining fully reliable reward signals for fineggraied spatial relationships still remains challenging.
Finally, Row involves moving a person toward the right side. Although the verifier prefers outputs where the person's 's position is more towards the right, it does not take into account the changes in person's appearance.
It is important to note that while verifier scores vary, the model can still provide meaningful signals for some complex edits such as detecting lower object counts or changes position to a certain extent, helping guide improvements in edit quality.
D.2 VIEScore Evaluation with Qwen2.5-VI.-22B and Alignment with Human Judgment
We also report the correlation between Qwen2.5-VL-72B's judgments (as captured by VIEScore) and human judgments on AURORA-Bench (columns show results for individual subtasks, and "AVERAGE" denotes the aggregate performance). While the absolute correlation values are moderate (around 0.4), they are consistent with those reported in the original VIEScore paper (0.382) [27]. which established VIEScore as a reliable automatic evaluation metric. This consistency supports the reliability of Qwen2.5-VL-72B
verifier. Morcover, the results indicate that Qwen2.5-VL. 72B performs robustly not only object- and attribute-centre edits but also on more complex edits such as spatial (WhatsUp) and action Boonethigg-Somethiieg Action-Gennt Genomm)..
Table 5: Correlation results between Qwen judgments and human judgments across different datasets.
Although the absolute values modest, they comparable to the established VIEScore benchmark.
Metric
MagicBrush
Something Something
Action Genome
Epic Kitchen
Whatsup
AVERAGE
Overall 0.5377 0.6104
0.3084
0.4385
0.2393
0.4420
0.3971
Semantic
0.3192
0.4392
0.2886
0.4556
0.4309
D.3
Impact of Verifier Choice on RL Performance
Our main experiments employ Qwen2.5-VL-72B the reward model (verifier) for RL post-training. We selected Qwen2.5-VL-72B because of its reliable reward feedback and strong performance on fine-grained vision-language tasks, comparable GPT-4o. We also explored alternative verifiers to study their effect on reward signal quality and overall training performance. Notably, amall-sized multimodal LLMs often produced noisy outputs, particularly in tasks requiring cusrruction-ollowiinn and prompt adherence. We tested an instruction tuned model, Qwen-7B. As shown in [Tb.. 6 replacing Qwen2.5-VL-72B with Qwen-7B led to significant drop in performance: from average score of 3.88 with SFT alone to 2.81 with RL post-training. This highlights that smaller models cannot provide sufficiently accurate stable feedback for AR-based RL training, underscoring the importance of a strong verifier.
Table 6: Impact of verifier choice on model performance (higher is better).
Method
AURORA
MagicBrush
OmniEdit
12EBench
Vismin
EmuEdit
Average
SFT (S)
3.58
3.19 2.66
5.73 4.05
3.59
3.57
3.66
3.88
SFT (S) RI. (S+C) [Qwen-7B]
2.55
2.60
2.32
2.69
2.81
21

---

E
Analysis of Model Outputs
E.1 Breakdown of VIEScore
To better quantify model behavior, show four components of VIEScore: Edit Success and OverEdit, which reflect semantic alignment with editing instructions, and Looks Natural and No Artifact, which capture perceptual quality. Tab. 7 reports these metrics for both the SFT(S) baseline and our best-performing configuration, SFT(S) RL(S+C). As shown in the AVERAGE column, RL notably improves Edit Success, demonstrating stronger alignment with user instructions, while maintaining comparable performance on OverEdit and preserving perceptual quality in terms of Looks Natural and No Artifact.
Table 7: Breakdown of VIEScore into four components: Edit Success, OverEdit, Looks Natural, and No Artifact comparing SFT(S) and SFT(S) - RL(S+C) across six benchmarks. Values are reported / SFT(S) RL(S+C), with the higher score shown in bold.
Metric
OmniEdit
EmuEdit
AURORA
MagieBrush
VisMin
12EBench
AVERAGE
Edit Success
6.1/7.0
3.5/4.2
3.0/3.6
2.6/4.1
3,2/4.3
3.7/4.3
4.7/
OverEdit
73/775
75/7.0
7.066.6
78/7.0
5.7/6.5
7,0/6.9
Looks Natural
6.7/6.9
6.3/6.0
5.8/5.7
6.3/5.8
6.666.8
5.0/5.1
6.1/6.0
No Artifact
75/7.7
6,9/6.7
6,7/6.7
6,96644
7.3/7.6
5.8/5
6.9/68 6.8
VIEScore
5.7/6.3
3.7/433
3.6/4.0
3.2/4.3
3.6/4.5
3.6/4.1
3,9/
E.2 CLIP Similarity and FID Evaluation
To provide a more comprehensive and fair evaluation beyond VIEScore, we further report two widely used metrics: CLIP similarity and FID (Fréchet Inception Distance). These metries capture complementary aspects of model performance: CLIP similarity measures the semantic consistency between generated and ground-truth edited images (1 higher is better), while FID quantifies image realism and distributional closeness (L lower is better). We observe that EARL achieves competitive performance on both CLIP similarity and FID, remaining within the same range as strong baselines such Omnigen, Aurora, and EditAR. This demonstrates that EARL preserves semantic alignment and image quality while providing comparable fidelity to existing methods. Results are averaged across three benchmarks: MB, VisMin, and OmniEdit.
Table 8: CLIP Similarity (ViT-B/32) Edited Image vs. Ground Truth. Higher scores indicate stronger semantic alignment.
Method
MB
InstructPix2Pix
Aurora
Omnigen
EditAR
EARL SFT(S)
CLIP
0.91
0.85
0.89
0.90
0.88
0.88
Table 9: FID (Inception V3)
Edited Images Ground Truth.
Lower scores indicate higher
image realism.
Method
MB
InstructPix2Pix
Aurora
Omnigen
EditAR
EARL -RL(S))
FID
40.59
53.20
47.22
45.22
49.47
49.64
E.3
Additional Comparison with EditAR
As EditAR 38 is the only existing autoregressive baseline for image editing, we conduct a more comprehensive comparison with it. Our best-performing setup (SFT(S) RL(S+C)) achieves an average VIEScore of 4.57, surpassing EditAR's 4.20 across all benchmarks. EditAR also reports results on PIEBench [4] for image editing. As shown in Tab. 10 EARL outperforms EditAR on five metrics, including Structure Distance score, PSNR, LPIPS (62,, MSE, and SIM (5I) On CLIP similarity scores computed using ViT-Large- 14 the whole image (CLIP-W) and on the edited regions (CLIP-E), EARL. slightly underperforms compared to EditAR. However, our scores remain comparable to EditAR, demonstrating strong perceptual quality and better background preservation.
22

---

Table 10: PIEBench results comparing EARL with EditAR (most similar work to ours). SD refers to Structure Distance and CLIP-W, CLIP-E refers to CLIP similarity scores on the whole image and edited regions, respectively.
Method
Base Model
PSNR
LPIPS
MSE
SSIM f
CLIP-W
CLIP-E 1
EditAR
LlamaGen
39.43
21.32
117.15
130.27
75.13
24.87 24.44
21.87
EARL
Emu3
35.00
23.71
112.40
77.00
79.17
21.34
Table 11: Best-of-N evaluation results for SFT and RL-tuned models across six benchmarks
Model
OmniEdit
EmuEdit
AURORA
MB
VisMin
I2EBench
AVG
SFT (S)
5.73
3.66
3,58
3.19
3,57
3.59
3.88
SFT (S) [Best of 5]
6.13
4.21
3.89
3.63
4.09
4.00
4.33
SFT (S) + RL (S+C)
6.33
4.28
3,99
4.26
4.48
4.08
4.57
SFT (S) + RL (S+C) [Best of 5]
6.54
4.66
4.08
4.66
4.69
4.36
4.83
E.4 Best-of-N Evaluation
We further evaluate model performance under best-of-N sampling regime to separate the effects of sampling diversity from policy optimization. As shown Tab. IT] best-of-5 sampling substantially improves SFT results (3.88 - 4.33), indicating significant headroom through better candidate selection. However, the RI.-tuned model still outperforms SFT even when both use best-of-5 (4.33 - 4.83), demonstrating that policy optimization contributes improvements beyond what sampling alone can achieve. Moreover, the small gap between RL mangle-sample and RL best-of-5 (4.57 4.83) suggests that the RL policy is already producing high-quality outputs with minimal reliance on sampling, highlighting the effectiveness of our training approach.
E.5 Fine-Grained Evaluation of EARL
Qualitative Fig. shows EARL SFT (S) - RL. (S+C) performing four types of image edits: counting, action, spatial, and simple; each illustrated by three side-by-side input/output examples. In the counting row, EARL correctly removes one poodle and two toy cars but fails to remove one egg from the third image. In the action row, it successfully takes the white cup out and opens the orange bag in the first two examples, but cannot make the person stand up in the final example. For spatial edits, the model effectively interprets spatial relationships by correctly removing the left fire hydrant and adding a man to the left of the road sign, but fails to add a picture to the left of the woman in the third example. Finally, in the simple edits row, EARL recolors pink and successfully erases palm tree with clean backgrounds, but fails to edit the third image and removes the bowling bowl instead of the truck. Overall, these examples highlight the model's strong potential performing complex image edits, while also indicating some challenges that remain to be addressed.
alien spaceship
Quantative The quantitative results Tab. 12 14 demonstrate that EARL SFT (S)
RL (S+C) consistently outperforms the baseline SFT (S) across multiple fine tin-grained editing tasks and datasets. For instance, in [Tbb. o3 12EBench, improvements are evident in counting. direction perception, object manipulation, and style alteration metries. Similarly, Tab. 14]shows gains in style. attribute modification, environment changes, and object addition//emooal on OmniFEit. Tab.. 15 further confirms these trends on EmuEdit, with better performance object addition, background editing, and both global and local changes. Lastly, Tab. 16 highlights improved counting and spatial relation accuracy on VisMin. On AURORA, as shown Tabb. 17, there is no improvement action edits for EARL SFT (S) + RL (S+C) when compared to SFT (S), mainly because SFT (S) itself was very poor on action edits. Overall, these results underscore the effectiveness of combining SFT with RL for diverse kinds of edit types.
E.6 Qualitative Evaluation of Reasoning and Edits in the SFT-Think Model
Fig. 8 presents qualitative examples comparing two model variants: SFT(S), and SFT think(S).
23

---

Counting Edits
Input
EARL
Input
EARIL
Input
EARL
poods
two Thyy
She 409
Action Edits
Take the whate cup out of ousboudd
Ouen be orarge boy lutther with both Foadss
Make the parton standuu upright
Spatial Edits
sgn.
Beeoora
Simple Edits
Tum colr
Pemove
Remove
Figure 7: Qualitative examples of EARL across diverse edit types counting, action, spatial, and simple.
Table 12: Fine-grained evaluation of SFT and RL model variants on 12EBench (Low-level Editing).
Haze Deblurring
Lowlight Enhuncement
Noise
Rain
Shadow Removal
Snow Removal
Watermark
Model/Edit Category
SFT (S)
2.36
3.73 3.51
2.56 3.03
2.36 3.04
3.98
4.60
2.59
3.82
EARL SFT (S) RI. (S+C)
2,33
4.74
4.98
3.16
4.28
Table 13: Fine-grained evaluation of SFT and RL model variants on 12EBench (High-level Editing).
Model/Edit Category
Counting
Direction Perception
Object Removal
Object Replacement
Background Replacement
Color Alteration
Style Alteration
Region Accuracy
SFT (S)
2.74
1.82
4.52
2.49
3.47
5.74
5.34
4.36
EARL SFT (8) - (S-C)
4,08
3.45
4.76
3.40
4.72
5.98
5.80
5.06
Table 14: Fine-grained evaluation of SFT and RL model variants OmniEdit.
Model/Edit Task Category
Style
Attr. Mod.
Env
Swap
Addition
Removal
SFT
5.52
6.30
6.12
5.69
SFT (S) (S+O)
5.99
6.95
6.68
6.39
6.00
6.30
Table 15: Fine-graines evaluation of SFT and RL. model variants on EmuEdit.
Model/Edit Task Category
Add
Remove
Background
Text
Color
Style
Global
Local
SFT
2.36
4.91
2.47
2.15
4.91
4.89
4.14
3.78
EARL SFT (S+C)
4.46
4.81
3.24
3.16
5.85
5.13
4.77
4.47
This example involves changing the color of wooden rabbit sculpture to brown, while preserving its carved details and natural background.
Example 1

---

Table 16: Fine-grained evaluation of SFT and RL model variants on VisMin.
Model/Edit Task Category
Counting
Spatial Relation
SFT (S)
4.22
2.91
EARL SFT (S) RL (S+C)
4.14
Table 17: Fine-grained evaluation of SFT and RL model variants on AURORA. "Action Genome subset of the Aurora benchmark specifically contains complex action edits.
Edit Task
SFT (S)
EARL SFT (S) RL (S+C)
*Actoon
3.09
2.96
SFT (S): Performs successful edit without reasoning, changing the color to brown while preserving the background unchanged.
SFT think (S): Uses Chain-of- Thhough (CoT) reasoning, including scene description, object identification, and bounding box localization. It understands the edit instructions and grounds the plan well. However, the edited rabbit appears unnatural and loses some details.
Observation: Although SFT think(S) demonstrates strong reasoning and grounding capabilities, it introduces artifacts or unnatural features in the edited image compared to the simpler SFT(S) variant.
Example 2 This example involves replacing an antique wooden radio with typewriter, maintaining the spatial layout and removing the original radio's 's elements.
SFT (S): Successfully replaces the radio with the typewriter, keeping the proportions and orientation consistent.
SFT think (S): Uses Chain-of- Chain-of-Thought (CoT) reasoning to carefully plan the replacement, including brand details, but shows minor inconsistencies by mentioning two different brand names (IBM and Makoka). The final edited image presents slight visual distortions around the edges and keys.
Observation: While the reasoning model provides detailed planning and grounding, it some- times produces inconsistent details-suse as mentioning two different brand names (IBM and pattern of hallucination in the reasoning process that can reduce visual quality compared to the non-reasoning model.
This example involves transforming green forested area around small building into a garden with blooming flowers while keeping the building unchanged.
Example 3
SFT (S): Successfully changes the environment to a blooming garden without reasoning, pre- serving the building and overall composition.
SFT think (S): Uses Chain-of-Though (Co'T) reasoning to plan the seasonal transformation carefully, resulting in well-grounded and detailed edit.
Observation: Both models perform the edit well, delivering accurate and natural-looking results.
In summary, the reasoning model can ground and plan edits, but may sometimes reduce visual quality by introducing artifacts and unnatural details compared to the no-reasoning model.
F
Experimental Setup and Compute Resources
F.1
Training Details
For SFT training, we used the AdamW optimizer with weight decay 0.1 and parameters 31 = 0.9. 32 0.95, and x 10-6 The learning rate followed cosine schedule with a minimum value of 10-6 and gradient clipping was applied with maximum norm of 5.0. Training employed
25

---

Input
Prompt
SFT (S)
Make rabbit to be brown.
SFT think (S): CoT Reasoning
CoT BBox
SFT think (S)
soutsirrage image frature unorden robi winn humaneld moue sye. the kuiigra wite subuat writing Me lunad nwu
aerfers The mebih ciraled areed give
oenh whit conernaa i the ddrated fecore reber" '2. The oblecto ot acth the mebot so aitun operitaly todk and acpernees" "3 dieeres 250
senaifc to selecte
Change
The cou undernttoue sccune givig a ranis and ratural apusrraces iniia realnin debildd ad an lo batures The budigound senai a suand ratanal serting cantaring
moniy nenvel in
oraled acare
Input
Prompt
SFT (S)
Replace antique radio with typewriter.
SFT think (S): CoT Reasoning
CoT BBox
SFT think (S)
Thereaaee iivep look
hos wreg che mhethe ergnider tncuency anda spaaia gala the fort with pover avice vible the fye iite' 22. The cbject 2 adiad is andta rado, ipiciiiatly desrried de venci sie upoccal bnnaanng
ko" The
wxiiin
socee spice calope tha camoraane abrivera sac vodde case nett will be ver nust in ded out creake waa teretton brsedd mame (Malccer be eleary wbible the tem aps, and
mestue joaa jranessr dp depioyi tor papuerii
oscal purlic raflect the coiginall satiig conta..
Input
Prompt
SFT (S)
Transform setting to spring with blooming flowers.
SFT think (S): Reasoning
CoT BBox
think (S)
The sours image depicts srail recanguae dar builing with a grean estrror and piók coor dtantic in grean foreted araa. telking apeeass te dangls modion eis thin prsaile memooo or avou enite; bleding ooorisaty
san andiniaatio notural sovitamee aeoudd the huiding cowdllddy the pawo fander wwaa The spec volked as be the bounding too couninsko 15, 200, 2503 on cumpane ny undiv bredled avac"" "4, Hoples the givan lavel with vi snd sumig sumeggaatinn pardin Wild in biooc ny dowens, enaurig the uutting sama ma and anged Ask puthwoy boude gispts the entracc and scjuntth the bdlage io enttanca the zotingiim ma feal "S. The edited image will desid smail raca iguaar builing with grean estarior and perk coos arccaddedll by a vibertapprn ng pa dun fidc with blcomingg flwer pathway leads up la the entance. and th soire vouder newne, arn ambooes tpicall at sprinfiss owring with booning fower and nanse meseod and growth
mesimmmend 19 The olject
Figure 8: This figure shows image editing examples from SFT(S) without reasoning, SFT think(S) with Chain-of-Thoughh (CoT) reasoning. While the reasoning model can understand instructions, plan, and ground edits, it may introduce artifacts and unnatural details compared to the non-reasoning model.
DeepSpeed ZeRO stage 3 for memory efficiency, with mixed precision enabled using 6f (bf16). We train the model for maximum of epochs.
For RL training, rewards were computed using VIEScore through a vLLM API server running Dwen2.5-VI-72BBo 4 NVIDIA H100 GPUs. The training was conducted separately on different server for 2000 steps, with early stopping based on reward plateaus, also using four NVIDIA H100 GPUs.
26

---

F.2 Training Efficiency Comparison
We compare the training data size, compute resources, and training duration of EARL with existing baselines, as shown in Tab. 18 EARL achieves competitive performance with roughly 5x less umage-editing data (752k vs. 4M samples) and significantly fewer GPUs (8 vs. 104) compared to Omnigen [56], demonstrating superior data and training efficiency. Note that the higher GPU demand for Omnigen expected, as its model is trained from scratch. contrast, EARL, builds on pretrained models, reducing computational requirements. This highlights that EARL offers practical efficiency gains while maintaining strong performance.
Table 18: Comparison of training data size, compute resources, and training duration for EARL and prior methods. "_" indicates values not reported.
Model
Training Data
Compute
Duration
EARL SFT(S) RL(S+C)
~752 samples (SFT: ~550k,
8xA100L
108h
(SFT: ~60h,
RL: 32k)
RL: ru48)
InstructPix2Pix 8
313k samples
8xA100
25.5h
MagicBrush 61
~10 samples (built Instruct-
2 Ax1000
Pix2Pix)
Aurora 26
289k samples (built on Instruct-
2xRTX A6000
16h
Pix2Pix)
Omnigen 56
~0.1 samples (incl. 4.4M edit-
104xA800
ing examples)
EditAR 38
1.5M samples
8xA100
F33
Inference Speed Comparison
We integrated EARL with vLLM 29 which enables fast autoregressive decoding through Page- dAttention and optimized parallel token generation, significantly reducing latency and improving throughput compared to standard autoregressive decoding. We evaluated inference speed on 50 samples using single A A10OL GPU, running all models resolution of 256 x 256 (except Ed- itAR, which was trained and tested at 512 x 512). The total times to generate 50 samples were as follows: EARL: 52.7 s. EditAR: 66.19 s. MagieBrush: 23.6s, InstruciPix2Pix: 23.7 s. Aurora: 23.7s, s, and Omnigen: 200 Compared to Omnigen, EARL is roughly 4x faster while achieving competitive editing performance (see Table demonstrating the benefits of optimized autoregressive generation. Although EARL is about 2x slower than diffusion- based baselines such as MagicBrush InstructPix2Pix, and Aurora, this trade-off is justified by its significant improvements in editing quality.
G Limitations and Broader Impact
G.1
Limitations
First, the model's performance heavily depends on the coverage of our training data. Although we curated diverse set of simple and complex edit triplets, long-tail concepts such fine-grained cultural artifacts, specialized scientific diagrams, and underrepresented geographic scenes remain sparsely represented. This limited coverage can lead to brittle behavior when the model encounters out-of-distribution inputs.
Second, our reinforcement learning approach relies on single frozen vision-language verifier, which, despite being a state-of-the-an MLLM, has inherent limitations. The verifier can be imperfect and inherits biases from its pretraining corpus. It struggles particularly with verifying complex edit types involving spatial relationships and action changes. Although these edits are underrepresented verifier's training data, qualitative and quantitative analyses indicate that the verifier still provides meaningful learning signals to guide the model.
the
Lastly, our training data depends synthetic data generated via diffusion models, which include au- tomatic filtering to reduce noise. However, some noisy examples remain, such as image deformations or outputs that do not accurately reflect the edit instructions, due to imperfections in synthetic data generation. These factors can introduce noise in training. However, some these can be improved during RL most-training RL post-training does not need the ground-tru -ruth labels for edited images.
27

---

separate model G.2 Broader Impact
Positive Impacts Text-guided image editing systems such as our RL-enhanced approach has the potential to amplify human creativity and promote accessibility. By accepting natural language instructions, the model helps lower the barrier for designers, educators, and hobbyists who lack advanced editing expertise, enabling rapid iteration visual concepts. Beyond direct applications, our RL pipeline offers a way to overcome the requirement of ground edited images.
Potential Negative Impacts While figh-fidelity image editing offers many benefits, also poses risks. Such technology can be misused to create convincing misinformation or deepfakes. Addition- ally, if the base model or the vision-language verifier contains demographie biases, reinforcement learning may inadvertently amplify these biases. Importantly, our model is developed strictly for research purposes and is intended to advance scientific understanding rather than for deployment in real-world applications. We encourage ongoing efforts to implement safeguards and promote ethical use alongside further development in this area.
H Behind the Scenes
We started this project with a clear goal: to build a single, unified model that edits images guided only by text instruction would reason and plan the edits, generating all necessary guidance on its own. The rationale behind this is that diffusion models a dominant approach to image editing are typically built with some form of user conditioning to control for faithful editing, and they lack unification built for different edit types. We wanted have a unified model that does all sorts of edits and treats user conditioning as a learnable task via reasoning. To achieve this, we needed model capable of generating both images and text in one combined sequence, so we searched for an interleaved image-text transformer model.
pser-provided masks, bounding boxes, conditions. The model itself
The first model found is Meta's 's Chameleon model 45], as it fits the criteria well. It had good image generation capabilities, but the model was not open source. So, we pivoted to Anole ()), an open-source variant inspired by Chameleon. We trained Anole to do image editing without any reasoning input (the sanity check one can do). Unfortunately, the results were rough: images suffered from distortion and artifacts, making clear we needed something stronger.
Then came Emu3 [0). When it was released, we quickly tested it, fine-tuning with small, complex editing datasets. Results initially fell short- until added large amounts of simpler, high-quality edit data. This mix showed improvements: the model began handling simplistic edits well; also, some hope for counting, spatial, and even some action edits successfully.
Our main goal is to integrate reasoning to eliminate the need for user- providdo conditioning. To this end, we introduced reasoning data structured "chain of thought" prompts guide the model's editing process, as detailed extensively in our paper. Surprisingly, models trained with reasoning mostly underperformed compared to those without it. We tried many variants: changing the data pool, simplifying edit tasks, shortening the chain-of- of-thought length, and varying training data size, but none made a significant difference. We suspect this gap is due the base model's capacity the quality of the reasoning data. Even feeding ground reasoning directly as input during fine-tuning failed to boost results as much expected.
One positive result we had with the no-reasoning variant that we were able to get close to state-of- the-art. Since RL has been gaining traction lately, we were eager to explore its potential by applying it to our problem, but we needed to build a strong SFT model first. We tested SFT models with generating multiple samples given fixed prompt. One interesting observation was that at least one of these samples did well on simple edit tasks (c.g. changing object, attribute), and sometimes even a complex one (e.g. changing a count). That flicker of success motivated us to integrate RL on top of SFT-ed models - to push the model towards consistently generating better edits. Compared to our exploration of teaching
autoregressive model editing through reasoning, which for most part unsuccessful for a long while, RL experiments were more promising early on clearly moving our interest to go further with the RL route.
This journey has been a rollercoaster of setbacks and great excitement when RL was working consistently. Yet, each experiment deepened our understanding of the delicate balance between
28

---

data quality, reasoning guidance and RL bringing us closer to that elusive goal of truly unified, instruction-gun -uidedd image editing.
I Author Contributions
Saba Ahmadi, Rabiul Awal, and Benno Krojer initiated the project. Saba Ahmadi led the design and implementation of the EARL for auto-regressive image editing. Saba and Rabiul initiated the idea of incorporating reasoning into image editing. Rabiul and Ankur Sikarwar led the data generation for the CoT reasoning experiments. Ankur also led the evaluation. Amirhossein Kazemnejad led the design and the implementation of the reinforcement learning pipeline. Ge Ya Luo and Benno Krojer helped with evaluation metric selection. Rabiul, Saba, Ankur, Amirhossein, and Juan A. Rodriguez ran experiments for different stages of the project. Benno wrote the introduction and provided guidance on individual sections. Saba, Rabiul, Ankur, and Amirhossein led the writing of the remaining sections with feedback from the Pls. Aishwarya Agrawal, the lead Pl, guided the project from the start, with additional guidance from Siva Reddy, Christopher Pal, and Sai Rajeswar various stages.
29

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
The abstract and introduction are the summary of the paper's contributions.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how be expected to generalize other settings.
much the results
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
We discuss limitations in the supplementary material.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
30

---

Justification:
No theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification:
We provide details about training and implementation in Section 4
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm.
(b) the contribution is primarily a new model architecture, the paper should describe the architccture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reprodueibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
31

---

Answer: [Yes]
Justification:
We will release the data and code.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.co cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https guwdes/ddeeeeee Cooeeeoodeesssseeeeee //nips.com / ssiniiiiccypyy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Question:
Answer: [Yes]
Justification:
Yes, provided in Section
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification: We evaluated models across six benchmarks, each with around 1000 samples. This large sample size likely leads to low overall variability.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the enror bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
32

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We discuss in the supplementary material.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't 't make it into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics httpgp///eaarrss..c//a/l///hccat
Answer: [Yes]
Justification: Yes, we do.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
We discuss broader impact of our work in the supplementary material.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no socictal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
33

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Question:
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, scraped datasets)?
Answer: [No]
Justification: We finetune our models on top of pretrained models and our finetuning datasets are all publicly available.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g.. code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification:
We follow protocol.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the Datasets.papersstt paperswi thcode. cow/ datasets package should be provided. For popular datasets,
has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
34

---

If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: We will be releasing well-documented code and data.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/mmoel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: Not relevant.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question:
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: Not relevant.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
35

---

Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that if the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Answer: [Yes]
Justification: We use LLMs at different stages of training and evaluation as described in Section[4]
Guidelines:
The answer NA means that the method development in this research does involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy for what should or should be described.
36


Reviews:
[{"review_id": "txqyaBeGJ1", "reviewer": "Reviewer_W5tw", "summary": "The paper introduces the Editing with Autoregression and RL (EARL) model, which integrates autoregressive generation with reinforcement learning (RL) to enhance image editing tasks. Leveraging a multimodal large language model (MLLM) verifier, such as Qwen2.5-VL-72B, EARL optimizes the editing process by refining the model's outputs based on feedback from the verifier. The authors compare this approach with other strategies, including supervised fine-tuning (SFT) and chain-of-thought (CoT) reasoning. Experimental results demonstrate that EARL outperforms diffusion model baselines in both simple and complex editing tasks, offering higher data efficiency.", "questions": "1. When using Qwen2.5-VL-72B as a reward model, has its scoring preference for different types of edits been validated?\n\n2. The paper mentions that EARL \"uses less data,\" but it does not quantify the differences in training time and GPU memory usage compared to other methods. For example, under the same hardware conditions, how many times longer does the total time of EARL's SFT+RL process take compared to the diffusion model?\n\n3. The paper discusses the potential of combining autoregressive models with RL in image editing but does not examine the potential drawbacks of their token-by-token generation mechanism for complex edits (such as global semantic consistency). For example, when processing the instruction \"swap the positions of two objects in the image,\" does the autoregressive model introduce spatial reasoning errors due to the pixel-level generation order?", "limitations": "Qwen2.5-VL-72B, as a validator, may have training data biases (such as preferences for certain object categories or language expressions), but the paper does not analyze the impact of these biases on the editing results (such as whether they lead to over-editing or under-editing in specific scenarios).", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 4, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths：\n\n1. The authors compare supervised fine-tuning (SFT), reinforcement learning (RL), and chain-of-thought (CoT) reasoning within a unified autoregressive framework. They find that RL significantly enhances complex editing tasks, such as spatial transformations and action understanding, whereas CoT does not lead to consistent improvements. This underscores the clear motivation for integrating autoregressive generation with RL.\n\n2. The experimental validation encompasses a mixed dataset of simple and complex editing tasks, evaluating model generalization across six benchmarks, including OmniEdit and Aurora. Notably, the model demonstrates strong competitiveness in out-of-distribution (OOD) scenarios.\n\n3. This paper is well-written and easy to read.\n\nWeaknesses：\n\n1. This paper proposes integrating autoregressive models with RL for image editing. However, similar approaches have been explored in the NLP domain, such as RLHF applied to large language models (LLMs). Additionally, the application of RL in image generation, exemplified by methods such as Diffusion-DPO and InstructRL4Pix, is not particularly novel. The core innovation of EARL lies more in the integration of RL within an autoregressive framework, representing a model architecture combination rather than a technical breakthrough.\n\n2. The autoregressive token-by-token generation mechanism of autoregressive models may result in slower inference speeds compared to diffusion models. However, the paper does not provide a comparison of inference times between EARL and other AR or diffusion-based models such as EditAR (CVPR  25) and MagicBrush (NeurIPS 23), making it difficult to assess its potential for real-time applications.\n\n3. This paper employs Qwen2.5-VL-72B as the sole RL verifier, without comparing its performance against other multimodal large language models (MLLMs) such as LLaVA and MiniGPT-4. This limitation hinders the assessment of the verifier's impact on the overall results.\n\n4. The paper does not provide a quantitative comparison of training times and GPU memory usage between EARL and other models. This omission hinders a comprehensive evaluation of EARL's data efficiency advantages.", "was_revised": true, "final_justification": "This paper is well-written and easy to read. The proposed method, which integrates autoregressive models and reinforcement learning (RL) for image editing, represents a novel contribution to the field. While I raised some questions during the initial review stage, the authors have effectively addressed most of these concerns in their rebuttal. I thus believe this paper meets the acceptance criteria.", "rebuttal": "Thank you for the thoughtful review. We're glad you found the paper **well-written** and **easy to read**. We appreciate the **recognition of our unified comparison** of SFT, RL, and CoT, and the **clear motivation for integrating autoregressive generation with RL**. We're also encouraged by the **acknowledgment of our comprehensive evaluation** and the model’s **strong competitiveness in OOD scenarios**.\n\n> ...  similar approaches have been explored in the NLP domain, …  Diffusion-DPO and ,... The core innovation of EARL …, representing a model architecture combination rather than a technical breakthrough\n\nWhile RL has been explored in NLP and diffusion-based image generation, applying RL to autoregressive (AR) image editing is both novel and underexplored, as also acknowledged by R3: “Novelty: Introduces a simple yet effective RL post-training pipeline for autoregressive image editing, a relatively unexplored area.” The rapid improvements in image editing tasks have been led by diffusion‑based approaches, which have outpaced autoregressive models [1, 2] . Our work challenges this by showing that applying RL post-training to AR models significantly boosts performance, making them competitive with diffusion-based models while being more data-efficient.\n\nMoreover, our work is the first to conduct a systematic analysis of SFT vs RL for image editing. Through comprehensive experiments, we show that supervised fine-tuning (SFT) is not sufficient for achieving strong performance on complex edits such as spatial rearrangement, counting, and actions due to the lack of high quality and large scale paired datasets for such complex edits (see Table 1 for the sizes of available complex datasets). In contrast, RL post-training with a strong VLM verifier **does not require paired data** and **significantly helps improve the performance on complex edits**. Our strongest model SFT(S) → RL(S+C) significantly outperforms SFT on complex edits.\n\nPost‑submission, we ran **scaled RL post‑training** in our best setup (SFT(S) → RL(S+C)), which **outperformed Omnigen** with an average score of 4.8 vs. 4.7 across benchmarks. Please see our response **`EARL Scaled RL Post‑Training`** to **Reviewer 2**  for details.\n\n\n[1] MagicBrush. arXiv preprint arXiv:2306.10012.\n\n[2] Instructpix2pix. arXiv preprint arXiv:2211.09800.\n\n> The autoregressive token-by-token generation … may result in slower inference speeds compared to diffusion models. …  potential for real-time applications.\n\nWe have integrated EARL with vLLM [3], which enables fast autoregressive decoding via PagedAttention [3] and optimized parallel token generation. This significantly reduces latency and improves throughput over standard AR decoding.\n\nWe evaluated inference speed on 50 samples using a single A100L GPU. All models were run at 256×256 resolution, except EditAR which was trained and tested at 512×512. Reported times are the total time to generate 50 samples. EARL: 52.7 s, EditAR: 66.19 s, MagicBrush: 23.6 s, InstructPix2Pix: 23.7 s, Aurora: 23.7 s, Omnigen: 200 s. We make the following observations regarding inference speed:\n\n**EARL vs. Omnigen.** EARL is roughly **4× faster than Omnigen** while achieving competitive editing performance (see Table 2), highlighting the benefit of autoregressive generation with optimized decoding.\n\n\n**EARL vs. weaker diffusion baselines.** EARL is about **2× slower than MagicBrush, InstructPix2Pix, and Aurora**, but we believe this **trade-off is justified by the significant improvements in editing quality** (Table 2).\n\n\n**Clarifying EditAR results.** Since EditAR was trained on 512×512 images, we reported its runtime at that resolution (66.19s for 50 samples). EditAR generates higher-resolution outputs, so its runtime naturally is higher.  At 256×256, it produces noisy samples (46.30s for 50 samples).\n\n[3] Efficient Memory Management for Large Language Model Serving with. Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles.\n\n\n>This paper employs Qwen2.5-VL-72B as the sole RL verifier, without comparing … (MLLMs) such as LLaVA and MiniGPT-4. This limitation hinders the assessment of the verifier's impact on the overall results.\n\nWe appreciate the reviewer’s suggestion regarding alternate verifiers. We selected **Qwen2.5-VL-72B** because it demonstrates strong performance on fine-grained vision-language tasks [1], comparable to GPT-4o, making it a reliable source of reward signals. In contrast, earlier VLMs such as **LLaVA** and **MiniGPT-4** **struggle** with **instruction-following and prompt adherence**, and in our experiments their outputs were too noisy to extract usable rewards.\n\nTo further validate the reviewer’s point, we also tested a more recent **Qwen-7B** model, which is instruction-tuned and closer in scale to LLaVA and MiniGPT-4 (~7B). However, we observed that the performance of our model degrades significantly when using such a weaker verifier. For example, under our best-performing setup (SFT(S) → RL(S+C)), the AR model trained on Qwen‑7B achieved an average score of **2.81**, compared to **3.88** with SFT alone, showing clear degradation. This confirms that smaller models cannot provide sufficiently accurate or stable feedback for AR-based RL training, underscoring the need for a stronger verifier such as Qwen2.5-VL-72B. Given the limited rebuttal period, we were not able to perform RL post-training with other families of MLLMs. We plan to include these results in the camera-ready version.\n\n\n\n| Baseline                                     | AURORA | MagicBrush | OmniEdit | I2EBench | Vismin  | EmuEdit | AVERAGE |\n|----------------------------------------------|--------|------------|----------|----------|---------|---------|---------|\n| **SFT (S)**                                  | 3.58   | 3.19       | 5.73     | 3.59     | 3.57    | 3.66    | 3.88    |\n| **SFT (S) → RL (S+C)— Qwen7B** | 2.55  | 2.66      | 4.05    | 2.60    | 2.32  | 2.69  | 2.81   |\n\n\n[1] Qwen2.5 Technical Report. arXiv:2412.15115.\n\n> … quantitative comparison of training times and GPU memory usage between EARL and other models. …\n\t>... EARL \"uses less data,\" .. quantify the differences in training time.  …\n\n We compare the training time, GPU usage, and data size for EARL and available baselines below:\n\n\n| Model           | Training Data                                                                        | Compute Resources                        | Duration (Hours)                       |\n|-----------------|--------------------------------------------------------------------------------------|------------------------------------------|-----------------------------------------|\n| **EARL (ours)** | ~752k samples   (SFT: ~750k, RL: 1.6k)                                     | 8×A100L   | 84h (SFT: ~60h,RL: ~24h) |\n| InstructPix2Pix | 313k samples                                                                         | 8×A100                              | 25.5h                                   |\n| MagicBrush      | ~10k samples   (built on InstructPix2Pix)                                          | 2×A100              | – (Not reported in the paper)         |\n| Aurora          | 289k samples   (built on InstructPix2Pix)                                        | 2× RTX A6000                              | 16h                                    |\n| Omnigen         |~0.1B samples, including ~3.7M image editing examples                                                         | 104×A800                                 |  – (Not reported in the paper)                     |\n| EditAR          | 1.5M samples                                                | 8×A100                                   | – (Not reported in the paper)     |\n\nAs seen, EARL achieves competitive performance using ~5× less image editing data (752k vs 3.7M image editing samples) and far fewer GPUs (8 vs. 104) than Omnigen, highlighting its superior data and training efficiency. The higher GPU usage in OmniGen is expected, since their model is trained from scratch. In contrast, EARL builds on pretrained models, which naturally reduces computational requirements. This comparison suggests that EARL provides practical efficiency benefits while maintaining competitive performance.\n\n> When using Qwen2.5-VL-72B as a reward model, has its scoring preference for different types of edits been validated?\n\n> Qwen2.5-VL-72B, as a validator, may have training data biases …\n\nQwen2.5‑VL‑72B, as a reward model, may indeed reflect biases from its training data. While we did not conduct a full bias analysis, we provide evidence in Appendix Section D showing some inconsistencies, particularly on fine‑grained edits such as counting or spatial relations. Importantly, we validated its reliability by measuring correlation with human judgments on AURORA‑Bench (see **`VIEScore Evaluation with Qwen2.5‑VL‑72B and Alignment with Human Judgment`**, response to **Reviewer 3**). These results show strong agreement with human ratings across diverse edit types, supporting its use as a reward signal.\n\n\n> The paper discusses the potential of combining autoregressive models with RL in image editing but … potential drawbacks of their token-by-token generation …\n\nToken-by-token generation in autoregressive models can lead to error propagation and accumulation, especially in SFT-only settings, where early generation mistakes compound over time and introduce artifacts. In contrast, our RL-trained models show significantly fewer artifacts, as the verifier provides strong reward signals that encourage globally coherent and artifact-free outputs. As shown in Appendix Section E, performance breakdown across different edit types (Tables 2-6) shows consistent improvements in edit quality across diverse edit types such as counting, spatial relations, etc. This indicates that the verifier’s reward signal helps guide the model toward more accurate and coherent edits."}, {"review_id": "Rgidk6GpcJ", "reviewer": "Reviewer_EDq6", "summary": "The paper introduces EARL (Editing with Autoregression and RL), a two-stage framework for text-guided image editing. Stage 1 performs standard teacher-forced fine-tuning on 750 K “simple” and 300 K “complex” edit pairs; Stage 2 optimises the same network with GRPO, drawing rewards from Qwen-VL via VIEScore’s four sub-metrics (edit success, over-edit, naturalness, artefacts). Evaluations on OmniEdit, VisMin, I2E-Bench and three in-house splits show that RL lifts the average VIEScore, edging out diffusion-based MagicBrush and Aurora but still trailing OmniEdit.", "questions": "None", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Pros:\n\n1.The prose is clean and easy to follow, with all key stages (SFT, RL, evaluation) described in a logical order.\n\n2.Unifies simple and complex edits in a single autoregressive transformer.\n\n3.Beats diffusion baselines MagicBrush/Aurora on average VIEScore without diffusion back-bones.\n\nCons:\n\n1.Over-stated novelty:\n\n(1)“First RL for autoregressive editing” ignores prior EditAR and InsightEdit work. \n\n(2)GRPO application to vision is incremental; similar RLHF tricks already common in LLM papers.\n\n2.Evaluation gaps & reward leakage\n\n(1)Same Qwen-VL judge is used both as the RL reward and as the test metric, inviting reward-hacking.\n\n(2)No cross-metric checks (e.g., CLIP-SIM, FID) and zero human preference studies.\n\t\n3.Missing baselines\n(1)Lacks head-to-head numbers against EditAR, EmuEdit (XL), OmniGen diffusion + RL, and other recent autoregressive editors.", "was_revised": true, "final_justification": "This paper is well-written and easy to follow, and with most of my initial questions addressed in the rebuttal, I will maintain my rating.", "rebuttal": "Thank you for the constructive review. We are glad you found the **prose clean and easy to follow**, and we appreciate your **recognition of the paper’s logical structure across SFT, RL, and evaluation**. We're also encouraged by your **appreciation of our effort to unify simple and complex edits in a single autoregressive transformer**, and the **model’s ability to beat diffusion baselines like MagicBrush and Aurora on average VIEScore without relying on diffusion backbones**.\n\n>Evaluations on OmniEdit, VisMin, I2E-Bench and three in-house splits show that RL lifts the average VIEScore, edging out diffusion-based MagicBrush and Aurora but still trailing OmniEdit.\n\n\nWe believe the reviewer is referring to Omnigen (the model), since OmniEdit is one of the benchmarks we evaluate on, where we already outperform the other baselines.\n\n**`EARL Scaled RL Post‑Training`**  Post submission, we further scaled up the duration of the RL training for our best-performing setup (SFT(S) → RL(S+C)) to 2000 steps with 32k samples (vs. 1000 steps and 1.6k samples).\n\n\n| Model   | OmniEdit | EmuEdit | AURORA | MagicBrush | VisMin | I2EBench | AVG  |\n|---------|----------|---------|--------|------------|--------|----------|------|\n| EARL    | **6.39** | 4.47    | **4.27** | 4.52       | **4.93** | 4.19     | **4.80** |\n| Omnigen | 5.68     | **5.00** | 4.10   | **4.68**   | 4.09   | **4.68** | 4.70 |\n\nWe see that with scaling-up, EARL now outperforms even Omnigen on AVG, despite using ~5× less image editing data (782k image editing sample for the scaled up EARL vs 3.7M image editing samples for Omnigen), demonstrating the effectiveness and scalability of our approach.\n\n>1.Over-stated novelty: (1)“First RL for autoregressive editing” ignores prior EditAR and InsightEdit work. (2)GRPO application to vision is incremental; similar RLHF tricks already common in LLM papers.\n\n1- While EditAR is an autoregressive model, it does not use reinforcement learning, and InsightEdit relies on diffusion-based generation rather than autoregressive methods—and also does not include RL in its pipeline. To our knowledge, EARL is the first to apply reinforcement learning to autoregressive image editing, a combination not explored in prior work.\n\n2- While RL has been explored in NLP and diffusion-based image generation, applying RL to autoregressive (AR) image editing is both novel and underexplored, as also acknowledged by R3: “Novelty: Introduces a simple yet effective RL post-training pipeline for autoregressive image editing, a relatively unexplored area.” The rapid improvements in image editing tasks have been led by diffusion‑based approaches, which have outpaced autoregressive models [1, 2]. Our work challenges this by showing that applying RL post-training to AR models significantly boosts performance, making them competitive with diffusion-based models while being more data-efficient. \n\nMoreover, our work is the first to conduct a systematic analysis of SFT vs RL for image editing. Through comprehensive experiments, we show that supervised fine-tuning (SFT) is not sufficient for achieving strong performance on complex edits such as spatial rearrangement, counting, and actions due to the lack of high quality and large scale paired datasets for such complex edits (see Table 1 for the sizes of available complex datasets). In contrast, RL post-training with a strong VLM verifier **does not require paired data** and **significantly helps improve the performance on complex edits**. Our strongest model SFT(S) → RL(S+C) significantly outperforms SFT on complex edits.\n\n\n[1] Magicbrush: A manually annotated dataset for instruction-guided image editing. Advances in Neural Information Processing Systems\n\n[2] Instructpix2pix: Learning to follow image editing instructions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition\n\n\n> 2.Evaluation gaps & reward leakage (1)Same Qwen-VL judge is used both as the RL reward and as the test metric, inviting reward-hacking. (2)No cross-metric checks (e.g., CLIP-SIM, FID) and zero human preference studies.\n\n\n1- This is a misunderstanding—**we do not use the same model as the reward model and as the evaluation metric**. As described in Section 4.2, we use Qwen2.5-VL-72B as the RL reward model and GPT-4o-mini as the evaluation metric. This separation was intentional to avoid reward hacking and to ensure fair evaluation. \n\n2- We chose **VIEScore as it shows stronger Metric-to-Human (M-H) correlations compared to traditional metrics** [3]. For example, in text-guided image editing, VIEScore (0.3821) outperforms LPIPS [4] (0.1142), and in subject-driven image editing, VIEScore (0.4637) surpasses CLIP-I [5] (0.3058), demonstrating closer alignment with human judgment. Our **human studies further confirm this**, showing high alignment between VIEScore (0.4328) and human preferences (**see Appendix C**: VIEScore Evaluation with GPT-4o-Mini and Alignment with Human Judgment).\n\n\nAlso, in Section 5.2, we compare EARL with EditAR using a broader set of metrics beyond VIEScore—including PSNR, LPIPS, SSIM, and CLIP similarity—to provide a more complete evaluation. \n\nAs for FID, we note that VIEScore already captures key aspects such as image realism and artifacts—core elements FID is designed to assess. However, to address the reviewer’s concern, we report FID and CLIP similarity scores between the ground truth and generated images averaged across all benchmarks that provide the ground truth edited image (MagicBrush, VisMin and OmniEdit). \n\n\n\n\n\n\n### CLIP Similarity (ViT-B/32) – Edited Image vs. GT Edited Image (↑ higher is better)\n| Baseline              | MB   | VisMin | OmniEdit | AVG  |\n|---------------------|------|--------|----------|------|\n| Magicbrush          | 0.93 | 0.91   | 0.88     | 0.91 |\n| InstructPix2Pix     | 0.86 | 0.86   | 0.85     | 0.85 |\n| Aurora              | 0.92 | 0.90   | 0.86     | 0.89 |\n| Omnigen             | 0.92 | 0.92   | 0.88     | 0.90 |\n| EditAR              | 0.87 | 0.91   | 0.84     | 0.88 |\n| EARL SFT(S)→RL(S+C) | 0.88 | 0.88   | 0.89     | 0.88 |\n\n\n### FID (InceptionV3) – Edited Images vs. GT Edited Images (↓ lower is better)\n| Method              | MB    | VisMin | OmniEdit | AVG   |\n|---------------------|-------|--------|----------|-------|\n| Magicbrush          | 25.70 | 42.55  | 53.52    | 40.59 |\n| InstructPix2Pix     | 46.03 | 53.26  | 60.30    | 53.20 |\n| Aurora              | 32.89 | 48.05  | 60.71    | 47.22 |\n| Omnigen             | 37.84 | 40.30  | 57.53    | 45.22 |\n| EditAR              | 46.68 | 39.76  | 61.97    | 49.47 |\n| EARL SFT(S)→RL(S+C) | 47.41 | 47.16  | 54.34    | 49.64 |\n\nWe see that EARL’s scores remain in the same range as Omnigen, Aurora and EditAR for both FID and CLIP, showing it preserves semantic alignment and image quality at a comparable level to other strong baselines.\n\n\n[3]  VIEScore: Towards Explainable Metrics for Conditional Image Synthesis Evaluation. arXiv preprint arXiv:2312.14867.\n\n[4] The unreasonable effectiveness of deep features as a perceptual metric. CVPR, 2018.\n\n[5]  Learning Transferable Visual Models From Natural Language Supervision. arXiv preprint arXiv:2103.00020.\n\n\n>  3.Missing baselines (1)Lacks head-to-head numbers against EditAR, EmuEdit (XL), OmniGen diffusion + RL, and other recent autoregressive editors.\n\n\n\nWe already include comparisons with EditAR in our evaluation (Table 3). Since EditAR was not open-sourced at the time of submission, we could not  additionally report its performance on all the other benchmarks we have in our paper (Table 2) However, now it’s opensource, so we have run these additional evaluations. In our best-performing setup (SFT(S) → RL(S+C)), EARL achieves 4.57 versus EditAR’s 4.20 average VIEScore across all benchmarks, demonstrating superior performance. We will update the paper with the new EditAR results.\n\n\n\n| Baseline              | AURORA | MagicBrush | OmniEdit | I2EBench | VisMin | EmuEdit | AVERAGE |\n|------------------------|--------|------------|----------|----------|--------|---------|---------|\n| EditAR                | 3.79   | 3.84       | 5.29     | 3.84     | 4.54   | 3.88    | 4.20    |\n| EARL SFT(S)→RL(S+C)   | 3.99   | 4.26       | 6.33     | 4.08     | 4.48   | 4.28    | 4.57    |\n\nAs for EmuEdit (XL), to the best of our knowledge it is not publicly available. Lastly, as for OmniGen diffusion + RL, to the best of our knowledge, no prior work combines OmniGen (a diffusion based model) with reinforcement learning. Our goal is to demonstrate that autoregressive models are a strong alternative to diffusion-based models. Exploring reinforcement learning for OmniGen is an interesting direction, but it is outside the scope of this work."}, {"review_id": "xP6DgdiDgs", "reviewer": "Reviewer_vNkF", "summary": "This work investigates three training paradigms—supervised fine‑tuning (SFT), reinforcement learning (RL), and chain‑of‑thought (CoT) reasoning—for a unified autoregressive multimodal model (Emu3) on text‑guided image editing. The authors propose EARL, which applies GRPO‑based RL post‑training with a large MLLM verifier (Qwen2.5‑VL‑72B) and demonstrate that this approach yields significant gains over SFT alone, achieving competitive or superior performance to strong diffusion‑based baselines across six benchmarks, including IID and OOD settings. They also explore CoT reasoning but find it does not consistently improve editing quality.", "questions": "1. **Can RL discover better reasoning paths than pre‑collected CoTs?** If the off‑the‑shelf CoTs are suboptimal, can the RL stage improve not only editing quality but also generate superior chain‑of‑thought trajectories?  \n2. **Why omit a pure complex‑data SFT baseline?** The paper mixes simple and complex edits for SFT but does not show results when fine‑tuning only on the complex subset. How would SFT trained solely on complex examples perform?  \n3. **Best‑of‑N evaluation for SFT**: Figure 4 shows variability across samples, but only average scores are reported. Can the authors report best‑of‑N (e.g., best out of 5 or 10 samples) metrics for the SFT model?  \n4. **Best‑of‑N comparison between SFT and RL**: If best‑of‑N sampling is feasible, can the authors provide best‑of‑N results side‑by‑side for both SFT and the RL‑tuned model to isolate the benefit of policy optimization versus sampling strategy?", "limitations": "See weakness and questions.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "## Strengths\n- **Novelty**: Introduces a simple yet effective RL post‑training pipeline for autoregressive image editing, a relatively unexplored area.  \n- **Comprehensive evaluation**: Benchmarks on both simple (OmniEdit, EmuEdit) and complex (Aurora, MagicBrush, VisMin, I2EBench) tasks, with clear gains quantified via VIEScore.  \n- **Strong baseline comparison**: Matches or outperforms diffusion‑based methods (e.g., Omnigen) while using less data and provides a direct comparison to the only other autoregressive editing model, EditAR.  \n- **Insightful analysis**: The ablation on SFT data mixes and the investigation of CoT reasoning provide useful lessons about when and why each paradigm succeeds or fails.  \n- **Clarity and reproducibility**: Detailed descriptions of datasets, hyperparameters, and training settings support reproducibility.\n\n## Weaknesses\n- **Verifier dependency**: The approach hinges on a heavyweight MLLM reward model, which may be impractical for many users and introduces potential bias if the verifier’s understanding differs from human judgment.  \n- **Base model constraints**: Emu3’s limited image generation fidelity caps overall editing quality; stronger bases might change the relative benefits of RL vs. SFT.  \n- **Limited error analysis**: While average VIEScore gains are reported, detailed failure cases (e.g., semantic misalignments) are only qualitatively discussed, leaving some behaviors insufficiently characterized.  \n- **Scalability to high resolution**: All experiments use 256×256 images; it remains unclear how the pipeline scales to higher resolutions common in real‑world editing.", "was_revised": true, "final_justification": "During the rebuttal phase, the author addressed my concern about best-of-N evaluation and explained the details about model choice. Given I assigned a high score, I'll maintain my score.", "rebuttal": "Thank you for the detailed and positive review. We're glad you found our RL post-training pipeline **novel and effective**, our evaluation **comprehensive**, our experimental comparisons **strong**, our analysis of SFT data mixes and CoT reasoning **insightful**, and our paper **clear and reproducible**.\n\n\n>Verifier dependency: ... heavyweight MLLM reward model, ... impractical for many users \n\nWe use Qwen2.5‑VL‑72B as the backend for VIEScore to provide high-quality reward signals during RL training. The full pipeline runs on a single 8×A100 node, which is feasible for many academic labs.\n\n>  ... potential bias if the verifier’s understanding differs from human judgment.\n\n**`VIEScore Evaluation with Qwen2.5‑VL‑72B and Alignment with Human Judgment `** To address the reviewer’s concern, we report the correlation between Qwen's judgement (as captured by VIEScore) and the human judgments on AURORA‑Bench (rows report the breakdown across subtasks and All denotes the aggregate performance). We see that these correlation values are in the same ballpark as those reported in the VIEScore paper (0.3821) [1] which established VIEScore as a reliable automatic evaluation metric. Thus these results validate the reliability of Qwen as a verifier. Also, the results show that Qwen is reliable not only on object and attribute centric edits but also on more complex edits such as spatial (WhatsUp) and action (Something‑Something, Action‑Genome). \n\n\n### Correlation Results  \n\n| Dataset    | Overall  | Semantic |\n|------------|----------|----------|\n| MagicBrush | 0.5377   | 0.6104   |\n| Something  | 0.3084   | 0.3192   |\n| AG         | 0.4385   | 0.4392   |\n| EPIC       | 0.2393   | 0.2886   |\n| Whatsup    | 0.4420   | 0.4556   |\n| All        | 0.3971   | 0.4309   |\n\n[1] VIEScore: Towards Explainable Metrics for Conditional Image Synthesis Evaluation. ACL 2024.\n\n\n>Base model constraints: Emu3’s limited image generation fidelity caps overall editing quality; stronger bases might change the relative benefits of RL vs. SFT.\n\n\nWe agree that Emu3’s image fidelity places an upper bound on final editing quality. We chose Emu3 because it is currently the only open‑source autoregressive model that offers both sufficient image generation quality and semantic understanding of the prompts, and can be further fine‑tuned for the task of  image editing as well as optimized with RL (e.g., GRPO). Stronger AR base models are not available, making empirical comparisons infeasible at this time.\n\nRegarding the relative benefits of RL vs. SFT, while we are unable to run RL on stronger AR models directly, this trend in the literature supports our expectation that RL's advantages may be amplified as base model quality improves. \n\nMoreover, even with stronger base models, obtaining high-quality paired data for supervised fine-tuning—especially for complex edits like spatial rearrangement or action changes—remains a key bottleneck. In contrast, our RL approach does not rely on paired data and can generalize better to such tasks by leveraging verifier-based reward signals.\n\nTogether, these factors suggest that RL remains essential and may offer increasing benefits as base model quality improves.\n\n>Limited error analysis: While average VIEScore gains are reported, detailed failure cases (e.g., semantic misalignments) are only qualitatively discussed, leaving some behaviors insufficiently characterized.\n\nTo provide a clearer picture of model behavior, we break down VIEScore into its four aspects: edit success and over-edit (capturing semantic alignment), and naturalness and absence of artifacts (capturing perceptual quality). We report the scores for each of these aspects comparing SFT(S) and our best-performing setup, SFT(S) → RL(S+C). The results in the AVERAGE column show that RL significantly improves the performance on Edit Success, indicating stronger semantic alignment with the edit instructions, without causing significant drop in the other aspects such as OverEdit, and perceptual quality metrics such as Looks Natural and No Artifact.\n\n\n**SFT(S) vs. SFT(S) → RL(S+C)**  \n(Values are shown as `SFT(S) / SFT(S) → RL(S+C)`, with the higher score in bold)\n\n| Metric          | AURORA       | MagicBrush    | OmniEdit     | I2EBench     | Vismin       | EmuEdit      | AVERAGE     |\n|-----------------|--------------|---------------|--------------|--------------|--------------|--------------|-------------|\n| **Edit Success**   | 3.0 / **3.6** | 2.6 / **4.1** | 6.1 / **7.0** | 3.7 / **4.3** | 3.2 / **4.3** | 3.5 / **4.2** | 3.7 / **4.6** |\n| **OverEdit**       | **7.0** / 6.6 | **7.8** / 7.0 | 7.3 / **7.5** | 6.8 / **6.9** | **5.7** / 6.5 | **7.5** / 7.0 | **7.0** / 6.9 |\n| **Looks Natural**  | **5.8** / 5.7 | **6.3** / 5.8 | 6.7 / **6.9** | 5.0 / **5.1** | 6.6 / **6.8** | **6.3** / 6.0 | **6.1** / 6.0 |\n| **No Artifact**    | **6.8** / 6.7 | **6.9** / 6.4 | 7.5 / **7.7** | 5.8 / **5.9** | 7.3 / **7.6** | **6.9** / 6.7 | **6.9** / 6.8 |\n| **VIEScore**       | 3.6 / **4.0** | 3.2 / **4.3** | 5.7 / **6.3** | 3.6 / **4.1** | 3.6 / **4.5** | 3.7 / **4.3** | 3.9 / **4.6** |\n\n\n\n>Scalability to high resolution: All experiments use 256×256 images; it remains unclear how the pipeline scales to higher resolutions common in real‑world editing.\n\nDue to the limited resources as an academic lab, we conducted all experiments at 256×256 resolution. However, the method is fully scalable to higher resolutions given sufficient compute. Scaling does not require any architectural changes—only minor adjustments for tokenizing higher-resolution images during training. We also note that prior works such as InstructPix2Pix, MagicBrush, and Aurora, all developed in academic labs, were trained at the same 256x256 resolution.\n\n\n\n>Can RL discover better reasoning paths than pre‑collected CoTs? If the off‑the‑shelf CoTs are suboptimal, can the RL stage improve not only editing quality but also generate superior chain‑of‑thought trajectories?\n\nIn our experiments, we did not observe clear emergent changes in the structure of the reasoning chains after RL post‑training. A possible explanation is that the pretrained model was not explicitly optimized for reasoning or chain‑of‑thought style supervision, which limited the scope for RL to substantially modify or restructure such trajectories. As a result, RL mainly improved editing quality without noticeably altering the reasoning style.\n\n>Why omit a pure complex‑data SFT baseline? The paper mixes simple and complex edits for SFT but does not show results when fine‑tuning only on the complex subset. How would SFT trained solely on complex examples perform?\n\nWe trained an SFT model only on complex edits, but the available datasets are small for complex edits; for example, MagicBrush and ActionGenome each have only about 8K examples. That’s not enough to build a strong, balanced training set, especially since the Emu3 base model first needs to learn the task of editing (its pretraining does not cover image editing task, only image generation task). To adapt it for editing and achieve decent editing performance, we need a sufficiently large dataset.\n\nHowever, to address the reviewer’s concern, we trained a model (EARL SFT(C)) using the available complex subset and present the results here, compared against EARL  SFT(S):\n\n**Comparison of Baselines: SFT(S) vs. SFT(C)**  \n\n| Metric | AURORA | MagicBrush | OmniEdit | I2EBench | Vismin | EmuEdit | AVERAGE |\n|--------|---------|------------|----------|----------|--------|---------|---------|\n| EARL – SFT(S) | 3.58 | 3.19 | 5.73 | 3.59 | 3.57 | 3.66 | 3.88 |\n| EARL – SFT(C) | 2.78 | 2.71 | 2.93 | 2.67 | 3.90 | 2.68 | 2.94 |\n\nEARL – SFT(C) performs much worse than EARL – SFT(S), even on complex benchmarks. This confirms that with the limited complex-only data, the model fails to effectively learn the complex editing tasks.\n\n\n>Best‑of‑N evaluation for SFT: Figure 4 shows variability across samples, but only average scores are reported. Can the authors report best‑of‑N (e.g., best out of 5 or 10 samples) metrics for the SFT model?\n\n>Best‑of‑N comparison between SFT and RL: If best‑of‑N sampling is feasible, can the authors provide best‑of‑N results side‑by‑side for both SFT and the RL‑tuned model to isolate the benefit of policy optimization versus sampling strategy?\n\nThank you for the insightful suggestion. We ran best‑of‑N evaluations for SFT(S) and SFT(S) → RL(S+C) to separate sampling gains from policy optimization. Results are shown below.\n\n\n### Best‑of‑N Evaluation Results  \n\n| Model                         | AURORA | MagicBrush | OmniEdit | I2EBench | Vismin | EmuEdit | AVERAGE |\n|-------------------------------|--------|------------|----------|----------|--------|---------|---------|\n| EARL SFT (S)                  | 3.58   | 3.19       | 5.73     | 3.59     | 3.57   | 3.66    | 3.88    |\n| EARL SFT (S) [Best_of_5]      | 3.89   | 3.63       | 6.13     | 4.00     | 4.09   | 4.21    | 4.33    |\n| EARL SFT (S) → RL (S+C)       | 3.99   | 4.26       | 6.33     | 4.08     | 4.48   | 4.28    | 4.57    |\n| EARL SFT (S) → RL (S+C) [Best_of_5] | **4.08** | **4.66** | **6.54** | **4.36** | **4.69** | **4.66** | **4.83** |  \n\n\n\nWe observe that:\n**SFT(S) with vs. without best-of-5:**  Best‑of‑5 results are significantly higher than single‑sample results, indicating substantial room for improvement through policy optimization, thus further motivating our RL post-training approach.\n\n\n**SFT(S)-with best-of-5 vs. SFT (S) → RL (S+C):** RL outperforms SFT-with best‑of‑5-sampling, showing that **RL training provides improvements beyond what can be achieved via sampling alone**.\n\n\n**SFT (S) → RL (S+C) with vs. without best-of-5:** Best‑of‑5 on top of the RL post-trained model gives the highest performance overall. The gap between RL single‑sample and RL best‑of‑5 shows how close our RL model is to an “ideal” RL‑trained policy. We see that the gap is quite small, establishing the effectiveness of our trained policy."}, {"review_id": "RoUYBTNgQh", "reviewer": "Reviewer_WQB5", "summary": "Authors propose EARL, a strong autoregressive image editing method trained with GRPO. EARL besta other autoregressive editing methods and matches the performance of strong diffusion based editing methods.", "questions": "-", "limitations": "yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 4, "originality": 3, "strengths_and_weaknesses": "Strengths:\n- Shows a supervised finetuning, reinforcement learning, and CoT reasoning for image editing.\n- Achieves the best AR editing performance matching Diffusion editing performance.\n- Good qualitative analysis on types of images selected and rejected in the GRPO.\n\nWeaknesses:\n- EditAR results are only shown with PIE Bench. There is need for evaluating EditAR on the general and complex image benchmarks tested in Table 2 - OmniEdit, EmuEdit, etc.\n- There is no evidence that COT works for image editing and yet the authors spend a significant portion of the paper discussing that.\n- Why wasn't the training done with more data similar to OmniGen?\n- Only GRPO explored; PPO/LORA-PPO comparisons would strengthen generality of strength of RL.\n- No clear reasoning as to why SFT (S) → RL (S+C) is the best combination. Why are some of the other SFT(.) → RL (.) combinations not as good.\n- Some discussion on how would one extend this method to diffusion image editing would be helpful.", "was_revised": true, "final_justification": "Good paper with decent motivation and experiments.\n\nI'll keep my score.", "rebuttal": "Thank you for the positive feedback. We appreciate your recognition of our **wide study of supervised finetuning, reinforcement learning, and CoT reasoning for image editing**. We are glad you recognized that **our model yields the best AR editing performance**, **competitive with diffusion methods**, and appreciated the **qualitative analysis of GRPO-selected and rejected samples**.\n\n>EditAR results are only shown with PIE Bench. There is need for evaluating EditAR on the general and complex image benchmarks tested in Table 2 - OmniEdit, EmuEdit, etc.\n\nAs mentioned in Section 5.2, EditAR was not open-sourced at the time of submission, so we reported its results only on PIEBench (which is the only benchmark on which EditAR reports results for image editing). However, the code is now available, and we have evaluated EditAR on all the benchmarks used in our study, including OmniEdit, EmuEdit, AURORA, MagicBrush, VisMin, and I2EBench. In our best-performing setting (SFT(S) → RL(S+C)), EARL outperforms EditAR  on the AVERAGE metric, further confirming the effectiveness of our RL-based autoregressive editing approach. We will include these results in the final version.\n\n\n| Baseline              | AURORA | MagicBrush | OmniEdit | I2EBench | VisMin | EmuEdit | AVERAGE |\n|------------------------|--------|------------|----------|----------|--------|---------|---------|\n| EditAR                | 3.79   | 3.84       | 5.29     | 3.84     | 4.54   | 3.88    | 4.20    |\n| EARL SFT(S)→RL(S+C)   | 3.99   | 4.26       | 6.33     | 4.08     | 4.48   | 4.28    | 4.57    |\n\n> There is no evidence that COT works for image editing and yet the authors spend a significant portion of the paper discussing that.\n\nOur goal was to systematically evaluate the three training paradigms: SFT, RL, and SFT with CoT reasoning. This setup provides a comprehensive understanding of their impact on AR image editing. We included CoT because it has shown promise in NLP reasoning tasks. So, even though CoT did not lead to performance improvements,  we believe our findings will **inform** future work in image editing.\n\n\nWe also note that Reviewer 1 finds this comparative analysis valuable, and specifically highlights that CoT’s limited effectiveness in our experiments  “underscores the clear motivation for integrating autoregressive generation with RL.” Including CoT thus contributed to a more complete and grounded evaluation of training strategies in this setting.\n\n>Why wasn't the training done with more data similar to OmniGen?\n\nOmnigen uses ~5× more image editing data (752k image editing sample for EARL vs 3.7M image editing samples for Omnigen) but as an academic lab, we operate under limited computational resources. Our focus was on systematically comparing training paradigms (SFT, RL, CoT) rather than scaling data. That said, our method is fully scalable, and users with more resources can readily extend it with larger datasets.\n\n**`EARL Scaled RL Post‑Training`** To demonstrate the scalability of our method,  we scaled up the RL training of our the best-performing setup (SFT(S) → RL(S+C)) to 2000 steps with 32k samples (vs. 1000 steps and 1.6k samples used in the review version), The resulting model, EARL, achieves strong performance compared to Omnigen  on AVG (as shown in the following table),—demonstrating the scalability of our approach.\n\n| Model   | OmniEdit | EmuEdit | AURORA | MagicBrush | VisMin | I2EBench | AVG  |\n|---------|----------|---------|--------|------------|--------|----------|------|\n| EARL    | **6.39** | 4.47    | **4.27** | 4.52       | **4.93** | 4.19     | **4.80** |\n| Omnigen | 5.68     | **5.00** | 4.10   | **4.68**   | 4.09   | **4.68** | 4.70 |\n\n>Only GRPO explored; PPO/LORA-PPO comparisons would strengthen generality of strength of RL.\n\nWe focused on GRPO due to its strong stability and efficiency in large-scale autoregressive training compared to methods like PPO, as demonstrated in recent LLM literature [1]. While comparing with PPO or LoRA-PPO could further validate the generality of our approach, our primary goal was to establish the viability of RL for autoregressive image editing and to provide the community with a strong RL-based AR model that can be further built on top of.\n\n[1] DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning. arXiv:2501.12948.\n\n>No clear reasoning as to why SFT (S) → RL (S+C) is the best combination. Why are some of the other SFT(.) → RL (.) combinations not as good.\n\nWe found that **SFT(S)** produces the strongest and the most stable base model. This is because high quality paired data (required for SFT) is easily available for simple edits, allowing the model to reliably learn the basic editing task. In contrast, **SFT(S+C)** underperforms due to the limited size, diversity, and quality of the complex edit data (e.g., spatial reasoning, actions), which results in more noisy  learning.\n\nFor the RL stage, **training on both simple and complex edits (S+C)** proves most effective. RL on only complex edits **(RL(C))** suffers from a strong distribution shift relative to the base model and often fails to produce usable outputs, making it hard for GRPO training to receive meaningful learning signals. RL on only simple edits **(RL(S))** is stable but lacks coverage for harder tasks. **RL(S+C)** balances stability and coverage: simple edits provide reliable gradients, while complex edits gradually extend the model’s capabilities. \n\n> Some discussion on how would one extend this method to diffusion image editing would be helpful.\n\nExtending our verifier-based RL setup to diffusion models is a promising direction. Since our verifier operates on the final output image, the overall setup is, in principle, applicable to both diffusion and autoregressive models. The key challenge lies in adapting the GRPO algorithm for the diffusion modeling family. However, recent work such as Flow-GRPO [2] demonstrates that adapting policy gradient methods like GRPO is feasible for diffusion model families. While this remains an active area of research and is beyond the scope of our study, our strong results underscore that autoregressive models already offer a simple, effective, and practical solution for image editing, thus establishing autoregressive modeling as a strong contender to diffusion modeling for image editing.\n\n[2] Flow-grpo: Training flow matching models via online RL. arXiv preprint arXiv:2505.05470."}]

Author rebuttal:

Meta-review: This paper introduces the Editing with Autoregression and RL (EARL) model. Initially, the reviewers raised concerns about the lack of inference-time comparisons between EARL and other AR or diffusion-based models, limited key novelty, verifier dependency, and insufficient discussion of experimental results. However, after the rebuttal, the authors successfully addressed most of these concerns, including clarifications regarding novelty, and all reviewers provided positive feedback. The AC has carefully reviewed the paper, the reviewer comments, and the rebuttal, and agrees that the paper is well-motivated, clearly written, and supported by thorough experiments. Therefore, the AC recommends acceptance. It is encouraged that the authors incorporate all additional experiments and discussions from the rebuttal into the final version.
Decision: Accept (poster)