EditScore: Unlocking Online RL for Image Editing via High-Fidelity Reward Modeling
Authors: Xin Luo
Jiahao Wang
Chenyuan Wu
Shitao Xiao
Xiyan Jiang
Defu Lian
Jiajun Zhang
Dong Liu
Zheng Liu
Venue: ICLR 2026 Poster (2026)
Keywords: Image Editing
Reinforcement Learning
Reward Model
Primary area: applications to computer vision, audio, language, and other modalities

Abstract: Instruction-guided image editing has achieved remarkable progress, yet current models still face challenges with complex instructions and often require multiple samples to produce a desired result. Reinforcement Learning (RL) offers a promising solution, but its adoption in image editing has been severely hindered by the lack of a high-fidelity, efficient reward signal. In this work, we present a comprehensive methodology to overcome this barrier, centered on the development of a state-of-the-art, specialized reward model. We first introduce $\textbf{EditReward-Bench}$, a comprehensive benchmark to systematically evaluate reward models on editing quality. Guided by this benchmark, we develop $\textbf{EditScore}$, an efficient model to evaluate the quality of instruction-guided editing. Through meticulous data curation and filtering, EditScore effectively matches the performance of learning proprietary VLMs. Furthermore, coupled with an effective self-ensemble strategy tailored for the generative nature of EditScore, our largest variant even surpasses GPT-5 in the benchmark. We then demonstrate that a high-fidelity reward model is the key to unlocking online RL for image editing. Our experiments show that, while even the largest open-source VLMs fail to provide an effective learning signal, EditScore enables efficient and robust policy optimization. Applying our framework to a strong base model, OmniGen2, results in a final model that shows a substantial and consistent performance uplift. Overall, this work provides the first systematic path from benchmarking to reward modeling to RL training in image editing, showing that a high-fidelity, domain-specialized reward model is the key to unlocking the full potential of RL in this domain. Our code, models, and benchmark will be released publicly.

Published as a conference paper at ICLR 2026
EDITSCORE: UNLOCKING ONLINE RL FOR IMAGE EDITING VIA HIGH-FIDELITY REWARD MODELING
Xin Luol:: Jiahao Wang Chenyuan Wu! Shitao Xiao
Xiyan Jiang
Defu Lian', Jiajun Zhang', Dong Liu',
Zheng Liu
University of Science and Technology of China
2 Institute of Automation, Chinese Academy of Sciences
3 Beijing Academy of Artificial Intelligence, Zhejiang University
5 Hong Kong Polytechnic University
C GitHub: http:///ggthii..omm//////eeeeeaa
ABSTRACT
Instruction-guided image editing has achieved remarkable progress, yet current models still face challenges with complex instructions and often require multi- ple samples to produce desired result. Reinforcement Learning (RL) offers a promising solution, but its adoption in image editing has been severely hindered by the lack of high-fidelity, efficient reward signal. In this work, present prehensive methodology to overcome this barrier, centered on the development of state-of-the-a specialized reward model. We first introduce EditReward- Bench, a comprehensive benchmark to systematically evaluate reward models on editing quality. Building on this benchmark, we develop EditScore, series of re- ward models (7B-72B) for evaluating the quality of instruction -uideed image edit- ing. Through meticulous data curation and filtering, EditScore effectively matches the performance of learning proprietary VLMSs. Furthermore, coupled with an ef- fective self-ensemble strategy tailored for the generative nature of EditScore, our largest variant even surpasses GPT-5 in the benchmark. We then demonstrate that a high-fidelity reward model is the key to unlocking online RL for image edit- ing. Our experiments show that, while even the largest open-source VLMs fail provide an effective learning signal, EditScore enables efficient and robust policy optimization. Applying our framework to a strong base model, OmniGen2, re- sults in a final model that shows a substantial and consistent performance uplift. Overall, this work provides the first systematic path from benchmarking to reward modeling to RL training in image editing, showing that high-fidelity domain- specialized reward model is the key to unlocking the full potential of RL in this domain. Our code, models, data and benchmark will be released publicly.
INTRODUCTION
Recently, Reinforcement Learning LLi 2017) has demonstrated immense power in advancing large language models and has also shown remarkable success in the text-to-image (T2I) domain. Works like FlowGRPO (Liu et al., 2025a) and DanceGRPO (Xue et al. 2025) have leveraged RL on flow- matching models significantly enhance T2I generation across multiple dimensions. Despite these successes, the application of RI. to the image editing domain remains largely underexplored. We posit that RI. holds significant, untapped potential for image editing. By framing editing dy- namic trial-and-error process, a policy model can discover novel and more effective editing strategies beyond what is captured in static datasets. Through a well-designed reward mechanism, the model can be progressively steered to better align with user intent and achieve a deeper, more robust editing capability across diverse scenarios.
"Equal Contribution.
1 "oorrsponndig Author.

---

Published as a conference paper at ICLR 2026
Despite its theoretical promise, applying online RL to high-resolution image editing remains formidable and largely unsolved challenge (Wei et al. 2025| Wu et al. 2025a Ahmadi et al. 20025) The primary obstacle lies in the absence of suitable reward function: reliable, efficient, and scalable oracle that can accurately score the quality of an edit given an instruction. A natural con- sideration would be employ state-of-the-an general-purpose Visual Language Models (VLMs)) like GPT-5 [OpenAl 2025) or other proprietary APIs as reward providers. However, this approach impractical for online RL, with these large VLMs are expensive query at scale. The second alternative, using powerful open-souree VLMs, resolves cost issues. However, our investigations re- veal critical performance gap. As we will demonstrate, even 1wen2.5-VL-772.. (Bai al. (025b) fails to provide a sufficiently accurate and consistent reward signal to effectively guide the policy. Directly using such models as reward functions often causes unstable training or policy collapse, showing that scale alone cannot replace specialized accuracy. Progress thus hinges on accu- rate reward model tailored to image editing, yet the community still lacks powerful open-source option-posing a major barrier to advancing RL-based editing.
In this work, we argue that the key to unlocking online RL for image editing the development of specialized, high-fidelity. and efficient reward model. We present systematic approach to build, validate, and deploy such a model. Our contributions are organized as follows:
First, enable rigorous and reprodueible research, introduce EditReward- hensive benchmark for evaluating reward models in image editing. It is organized into four main categories, comprising 13 diverse subtasks ranging from simple attribute edits complex composi- tional changes. Each entry is evaluated by expert human raters across three key dimensions: prompt following, consistency, and overall quality. All annotations have passed rigorous inter-annotator agreement checks, establishing a reliable standard for measuring reward model quality.
compre-
Second, guided by our benchmark, we develop EditScore, series of powerful reward models guided editing. Through careful data curation and filterigg and inference-time scaling, EditScore sets a new state of the art for open-source reward models in this domain, even surpassing the accuracy of leading proprietary VLMs.
(7B-72B) for evaluating the quality of instruction eestruction-guided
To validate its practical utility, we first apply EditScore in a Best-of-N selection experiment across several ttateoof-hhe-at editing models, which confirms that our reward model can effectively im- prove the performance of diverse editors. We further used EditScore for RL training, resulting in a model that shows substantial and consistent performance uplift over its base model.
In summary, our key contributions are as follows.
We propose EditReward-Bennd a new public benchmark for the direct and reliable evalu- image editing.
ation of reward models for instruction-guide
We develop and release EditScore, a series of state-of-the offthh--rt art open-source reward models for caanection-base image editing that, through our self-ensembling strategy, surpasses the accuracy of leading proprietary VLMs on this task.
We demonstrate the broad, practical utility of EditScore through a Best-of-N selection experiment, successfully enhancing the performance of multiple diverse base models.
We provide a comprehensive analysis of an end-to-end online RL application, identifying key factors for success, including the accuracy and variance of the reward signal.
2 RELATED WORK
Reward Models for Image Generation. Reward models provide signals based on specific pref- erences and can be applied in various scenarios such as automated evaluation and reinforcement learning. Most research has focused on text-to-image tasks, with works like Wu al. (202bb); Kirstain et al. 2023); ; Zhang et al. (2024b) fine-tune CL.PP-based model with human preference data and Ma et al. 2025 ; Wang et al. (025bb); Gao et al. (2025) employ VLMs as a core component for reward generation. Research on reward models for image editing is relatively underexplored: some
works focused on controlled edits based on predefined masks (Ren et al. 2024 Gong et al. 2025)
For cntructiionguider editing, Zhang al. (2024a) fine-tur tie-tunes the BLIP model (Li al. 2022
with human annotated reward signals for the image generated only by P2P Hertz et al. (0022) and
IP2P Brooks al. (2023), WhhileChene et al. 2025 leverages CLIP scores automatically construct
2

---

Published as a conference paper at ICLR 2026
training data for training LLaVA-Next-8B Liu et al. (2024) as reward model. However, they used outdated editing models to generate images that needed to be evaluated, or simply used CLIP to annotate and score the images. Meanwhile, they use limited task categories as the prompt pool. The limitations in the narrow scope of task and generation model coverage make them struggle to sup- port online RL for current models. Furthermore, they are not open-source and cannot be used and evaluated by the public. In contrast, our model enables broader evaluation of editing tasks (13 tasks) meaaii2.--imaae-peeeee
while supporting SOTA editing models (e.g., nature enables it to perform inference-tine scaling to enhance scoring accuracy (Liu et al. 2025d).
Moreover, its generative
Image Reward Model Evaluation. Establishing a reliable benchmark is critical as it ensures the objective measurement of reward modeling performance and offers clear guidance for optimization. In previous work, image reward models are evaluated on their accuracy in predicting human prefer- ences for key generative tasks, such as text-to-image generation (Xu et al.
2023 Kirstain et al. 2023
Wu et al.) 2023b a Ma al. 2025) and image editing (Ku et al. 2023b Jiang et al. (002)) Over
the past year, generative models such as GPT-Image I (Hurst et al. 20244) FLUX-Kontex. (Batifol ct al. 2025), Qwen-Image-Edit (Wu et al. 2025a) and Nano Banana (Google 2025) have achieved
significant breakthroughs in image editing, particularly in stylization, hybrid edit and text modifica- tion. With the growing power of generative models and their expanding editing capabilities, there an urgent need for reward models that can accurately assess them. This, in turn, necessitates the development of a novel and more comprehensive benchmark for evaluating edit rewards.
EDITREWARD-BENCE
3.1 OVERVIEW
RdliteearddBeent is benchmark specifically designed for systematic evaluation of reward model- ing for image editing. It covers 13 diverse editing tasks and spans 11 heterogeneous editing models for data construction, ranging from open-source baselines to state-of-the- art proprietary models (see Table for comparison). The benchmark is distinguished by the following features.
Multi-dimensional Image Evaluation Framework. EditRewardd offers three key dimen- sions for evaluating editing outcomes. These includes Prompt Following (adherence to prompts), consistency (preservation of key visual elements) and overall quality (comparison across all aspects).
Comprehensive Model Performance Spectrum. EditReward-Bente incorporates both state-of- the-art editing models and lower-peforming baselines. It effectively challenges the reward model's distinguish ability and validates the reward model's scoring performance on current SOTA models.
Extensive Task Coverage with Real-world Applicability. EditReward- also covers diverse set of editing tasks that closely align with real-world application scenarios, ensuring authentic and comprehensive evaluation of reward models for image editing.
Table 1:
Comparison of Edit Bench against existing benchmarks, highlighting its superior scale, data sources, and comprehensive task coverage. The order of subtasks within each tuple under "Task Coverage" corresponds to the order of subtasks listed for each category in Section 3.2
ImagenHub
GenAl-Bench Jiang et al. 2024
EditReward- (ours)
Feature
Ku et al. 20236
General Properties
Size
2,864 V
919
3,072
Multi-Dimeensionl Eval.
X
Proprietary Model Data
X
X
Task Coverage (Conceptual Groups)
Subject
Appearance
x)
V.X.X.)
Scene
Advanced
xx)
3

---

Published as a conference paper at ICLR 2026
3.2 CONSTRUCTION PROTOCOL
To create a robust and comprehensive benchmark, we focused on three key pillars: the diversity of editing tasks, the variety of editing models, and the granularity of our evaluation criteria.
First, to ensure comprehensive task coverage, we structure our benchmark into four main cate- gories, comprising 13 distinct subtasks curated from established datasets like GEdit-Bench-EN (Liu eet al.] 2025b) and ImgEdit-Bench (Ye et al., 2025). These categories are designed to span wide spectrum of complexity. They are: 1) Subject, which includes fundamental tasks like subject addition subject removal and subject replace; 2) Appearance, cover- ing shape-preserving edits such color alteration, inraatiim.matriaaa.............. material modi f catiinn style t rannsss fer and tone trans transformate wormsooomaiiiooo ion; 3) Scene, which tests understanding of image layout through tasks like background change and extract; and 4) Advanced, the most challeng- ing category, which requires advanced reasoning for tasks like portrait beauti iaautiicciio, text modif cation, motion change and hybrid edit.
Second, to populate our benchmark with diverse distribution of edited images, the candidate pool of outputs were generated by a diverse array of 11 generative models for data construction: StepIX- Edit (Liu et al., 2025b). Step 1epp1X-Eddd Edit v1.1 @Liu et al., 2025b),
Image-Edii (Wu et al.) (202aa).
OmniGen2 eW al. 2025b),
(Batifol et al. 2025), HLUX-Kontext-ppe prontext-pro (Batifol et al. 2025), Bagel (Deng et al. 2025). >. MagicBrush (Zhang et al. (023), Omnigen (Xiao et al.
2025). , gpt-image-1 (Hurst et al. 2024), and Gemini-2.5 2.i---..---maeeeeiieeee (Google 2025). ), including
both open-source and of- proprietary editors.
Finally, recognizing that the quality of editing results not monolithic, we designed multi- dimensional evaluation scheme for better interpretability inspired by (Ku et [023a Liu et al.)
20256). Each editing result is assessed along three distinct axes: Prompt Following (PF): measuring how faithfully the edit executes the given instruction. Consistency (C): assessing the preservation of unedited image regions. Overall Quality (O): providing a holistic comparison of edits by ac- counting for all relevant aspects.
3.3 ANNOTATION PIPELINE
To ensure the reliability of EditReward Beiiee we designed rigorous human annotation pipeline conducted exclusively by experts in generative AI, as illustrated in Figure
To ensure high-quality ground truth, we implemented a Two-Annotator Discussion Protocol. Un- like standard crowdsourcing where raters work in isolation, our protocol assigns two experts to each sample. They engage real-time discussion to analyze visual artifacts (c.g., attribute drift) and resolve discrepancies, finalizing the ranking only upon reaching a joint consensus. We validated this protocol through a controlled study comparing it against independent labeling. The results demon- strate that the discussion-basee approach significantly reduces annotation noise: for instance, in the Consistency dimension, the inter-annotator convergence rate (at a 100% agreement threshold) im- proved by 12.12%. % Furthermore, the rankings determined by single annotator pair achieved over 97% % consistency with the majority vote derived from the full expert pool. confirming that our effi- cient two-annotator setup yields results highly aligned with collective expert judgment. A detailed disagreement analysis is provided in Appendix B
Following this protocol, for each input with five output images randomly sampled from the candidate pool, the annotators were asked to rank them. Our system uniquely allowed them to group outputs of similar quality into the same tiers; for instance, a ranking of 3 12 /5 places output 3 in the top tier, outputs and 2 tied in the second, and 4 and 5 tied in the third.
These tiered rankings were then decomposed into pairwise preference tuples. Following the 3 example, this conversion yields preference pairs where higher-tiered item is preferred over a lower- tiered one, such as (3, 1), (3, 2). (1, 4) and (2. Comparisons between items within the same tier (e.g., (1) are excluded as they represent equal quality. This process resulted in a large- scale, high-fidelity dataset of 3,072 preference pairs, comprising 944 for prompt following, 890 for consistency, and 1,238 for overall quality. For the evaluation of reward models, utilize the preference prediction accuracy metric, calculated by the proportion of pairs (A, B) where the model scores the numan-preferee output higher, i.e., S(A) > S(B).
4

---

Published as a conference paper at ICLR 2026
Input image
Output 1
Output 2
Output
Output 4
Output
Instruction
Providing cankings
Prompt Following: 1/34/5
Change in the picture from asphall circuit to desert track.
generated
2. Consisiency:
134/2/3 (1-3-42222
Preference
images based different
pairs
Quality:
(1
Figure I: Illustration of the annotation process. Annotators are presented with five candidate output images and are asked rank them according to three evaluation dimensions. The final ranking is determined through consensus among multiple annotators. For example, 1|234|5 indicates that the first image is preferred over images 2, 3 and 4, which are in turn preferred over image
METHOD
4.1 EDIT REWARD
4.1.1 THE APPROACH REWARD MODELING
Inspired by (Wang et al. 2025b Wu et al. 2025c), we formulate reward modeling as conditional textual generation task. We fine-tune models from the powerful Qwen2.5-VL. series (Bai et al., 2025b) on a standard autoregressive objective to act as specialized evaluators. Our model EditScore takes (Instruction, Input Image, Output Image) input, (Reasoning, Scalar Score) output. The chain-of-thought (Wei et al.) 2022) process of output not only enhances the model's interpretability but also demonstrably improves the accuracy of the final scores.
To ensure EditScore can perform comprehensive evaluation of editing tasks, we adopt the VI- EScore framework (Ku al., 2023a), prompting the model in parallel to assess two orthogonal aspects. The first is Semantic Consistency (SC), which evaluates the degree of instruction follow- ing and consistency, ensuring specified objects are correctly modified while unmentioned regions remain preserved. The second is Perceptual Quality (PQ), which assesses overall image quality, focusing on photorealism and the absence of artifacts. The final score geometric mean of the SC and PQ scores (Sfinal SSc SpQ). yielding balanced and granular reward signal.
4.1.2 INFERENCE-TING ENSEMBLING STRATEGY
Recent work (Snell al. 2024 Liu et al. 2025ch has shown that the performance of LLMs can be substantially improved by increasing the computational budget at test time. Building on this insight, we introduce an inference- cally boost its evaluation accuracy and enhance reward stability. Our approach is straightforward: for given input triplet z (Instruction, Input Image, Output Image), we perform independent, stochastic forward passes through EditScore, and each pass generates pair (reasoning, si). Then we aggregate only the scalar scores {81, 82, .,,, 5K) to compute the final ensembled score, S using the arithmetic mean:
ensembling strategy for our generative reward model to systemati-
K Σ K
1
S firnal (z)
8i. where (reasoning, s:) = EditScore(z).
1=1
An intuitive explanation (Liu et al. reasonings be viewed distinct judgment perspectives. Aggregating the scores derived from these diverse perspectives allows the final reward to more accurately reflect the true quality of an edit, leading significant scaling effectiveness.
2025c for this effectiveness is that each of the K generated
5

---

Published as a conference paper at ICLR 2026
4.2 DATA CONSTRUCTION PIPELINE
The training data for reward model and subsequent reinforcement learning are constructed follow- ing the similar procedure. For reinforcement learning, the training data only requires the input im- ages and instructions, whereas training the reward model additionally requires the generated outputs paired with their corresponding rewards. The construction process involves three steps as follows.
Step I: Images selection and instructions creation. We start by selecting and filtering diverse set of high-quality images as editing inputs. Next, we construct series of reference instructions for various editing tasks. Using the input images and random selected reference instructions as guidance, we prompt Qwen-2.5-VL.-726 to generate editing instructions that are task-consistent with the references. After generating a large set of image- image-instruction pairs, we applying K-center greedy algorithm (Sener & Savarese. 2018) to select 1000 semantically diverse samples per task for reward generation.
Finally, we constructed 70,000 data samples for training the reward model and 60,000 samples for reinforcement learning training. The data for the reward model requires further annotation as follows.
Step II: Candidates output generation. We generate candidates output using 5 distinet editing models random selected from our model pool.
Step III: Annotate and filtering. For each candidate output, use GPT-4.1 to annotate scores. Following VIEScore (Ku et al.) (223a),, we score each output on SC and PQ and provide reasons for the assigned scores. Next, we apply filtering to the reward samples. It is conducted from two perspectives: (i) filtering based on group-wise maximum scores to remove unachievable editing tasks, and (ii) filtering by group standard deviation remove cases with low discriminability.
5 REWARD MODEL PERFORMANCE ON EDITREWARD-BENCH
5.1 EXPERIMENTAL SETUP
Our final EditScore model is obtained by fine-tuning Qwen2.5-VL with LoRA (Hu al. 2022 on our curated dataset (see Section 4.2). For evaluation, we adopt the VIEScore (Ku et al. 2024 prompt template with two key modifications: (i) enforcing a reasoning- weanning--efor-scorriii format, (ii) expanding the score range from [0, 10) to [0, 25). Following the formulation of VIEScore, we derive
Prompt Following (Spp) and Consistency (Sc) from its Semantic Consistency metric Ssc, while Overall Quality (O) is directly taken from the final score Syintt. The detailed prompt templates are shown in Appendix M
5.2
MAIN RESULTS
Results summarized in reble a significant performance gap between proprietary models and open-source counterparts. Leading proprictary models such as GPT-4.1, GPT-5 and Gemini-2.5-PP form a distinct upper tier, achieving pairwise accuracies in the 0.7-0.75 range across all dimensions. This confirms their strong, albeit imperfect, zero-shot capabilities for this nuanced task. In particular, we find that VLMs are generally stronger at assessing Prompt Following (PF) than Consistency (C), as the latter requires fine-grained comparisons between the input and output images.
In stark contrast, even the largest and most capable open-source models exhibit notable limitations. The Qwen2.5-VL. series shows clear scaling trend, yet the 72B-paraeeter variant still falls short of 0.612 overall accuracy and performs worse than random chance in Consistency judgment. The smaller 7B and 32B models fare even worse, underscoring the inadequacy off-the-shelf open- source VLMs reliable reward signals for fine-grained editing tasks. By contrast, our EditScore achieves substantial improvements: the 7B variant surpasses the 10x larger Qwen2.5-VL-7222 while the 72B variant matching the score of GPT-4.1. Moreover, scaling inference-time compute with self-ensemble (Avg@4) further boosts performance across all model sizes, with EditScore-72B es- tablishing the state of the art EditReward-HH Bench.

---

Published as a conference paper at ICLR 2026
Table 2: Benchmark results on EditReward- reporting both overall pairwise accuracy and fine-grained breakdown across four categories of edit capabilities. Pairwise accuracy measures the proportion of pairs where the model correctly assigns a higher reward score to the human-prees peeea--eeeerred output. Notably, EditScore achieves superior performance even with its compact 7B size. Avg @4 denotes the average score over forward passes.
Model Metric
GPT-4.1
GPT-5
Gemini-2.5
Qwen2.5-VL
EditScore-7B
EditScore-32B
Ed
Pro
7B
32B
72B
Base
Avg@4
Base
Avg@4
Base
Avg@4
0.673
0.703
0.458
0.498
0.540
0.592
0.722
0.638
0.736
0.755
Overall
C
0.602
0.669
0.560
0.325
0.376
0.435
0.591
0.720
0.556
0.704
0.586
0.735
O
0.705
0.755
0.722
0.432
0.563
0.621
0.659
0.727
0.680
0.733
0.703
0.763
PF
0.615
0.707
0.712
0.414
0.527
0.523
0.590
0.691
0.625
0.703
0.612
0.708
Subject
C
0,520
0.538
0.465
0.317
0.414
0.394
0.585
0.666
0.473
0.627
0.524
0.639
0
0,679
0,708
0.765
0,460
0,570
0.594
0,740
0.771
0,703
0.754
0,721
0.807
PF
0.673
0.762
0,631
0.422
0.393
0.390
0,573
0.682
0.587
0.714
0,584
0.733
Appear.
C
0.668
0.714
0,577
0,335
0.320
0.416
0.612
0,730
0.591
0.764
0.623
0,778
0
0.709
0.756
0.700
0.470
0.514
0.559
0.663
0.714
0.669
0.710
0.697
0.736
PF
0.763
0.852
0.766
0,433
0.611
0.690
0.744
0.821
0.789
0.870
0.788
0.908
Scene
C
0,682
11741
0.675
0.236
0.482
0.429
0.735
0..35
0.627
0.787
0.695
0.797
0.773
0.841
0.693
0.357
0.673
0.713
0.789
0.774
0.764
0.816
0.794
0.837
0.673
0.736
0.541
0.627
0.736
0.625
0.717
0.734
Advanced
C
0.556
0.687
0.557
0,367
0.351
0.488
0.503
0.693
0.548
0.658
0,541
0.733
0
0.686
0.746
0,724
0,410
0.553
0.657
0.529
0.683
0.631
0.699
0,650
0.721
Performance vs. TFLOPs with Self-Ensembling
Performance vs. Latency with Self- Self-Ensembling
0.78
0.76
0.76
Accuracy
0.74
accuracy
0.74
0.72
0.70
0.70
EditScore (78)
EditScora (76)
0.68
Editscore (328)
0.68
EditScore (329)
0.66
Editscore (728)
0.66
EditScore (728)
Scaling Sizo (at
Scaling Modil Size K-1)
0.64
10'
10a
10
10"
10
Compulational Cost (Proportional TFLOPS)
Latency (5)
(a) TFLOPs vs. Accuracy
(b) Latency vs. Accuracy
Figure 2: Self-ensembling offers superior rfffciencc--erformmane trade-off compared to sim- ply scaling model parameters. (a) TFLOPs vs. Accuracy: The colored solid lines (increasing K) show a steeper performance trajectory than the gray dashed line (scaling model size), indicating higher return on computational investment. confirm that the latency cost of self-ensembling is sublinear due to shared KV-cache prefilling. No-
(b) Latency vs. Accuracy: Real-world measurements
tably, EditScore-7B (K = 4) achieves higher accuracy than the single-pass EditScore-32B (K 1)
while requiring significantly less latency and hardware resources.
5.3 EFFECTIVE INFERECCETTME SCALING OF EDITSCORE
As shown in Table 2 both scaling model size and inference-time compute improve the performance of EditScore. To further investigate which dimension contributes more effectively, we normalize the comparison by using FLOPs proxy for inference cost. Figure 2 analyzes the scaling properties of EditScore from two perspectives: theoretical computational cost (TFLOPs, Left) and real-world inference latency (Right). In both plots, colored solid lines denote inference-time compute scaling (increasing ensemble size K), while the gray dashed lines represent parameter scaling (increasing model size from 7B to 72B). We observe that scaling inference-time compute yields significantly greater marginal gains than scaling model parameters. Moreover, as shown in Figure 2)b). the wall-clock latency grows sublinearly with K. This efficiency stems from the shared prefill stage across ensemble members within the attention mechanism, minimizing redundant computation. For a comprehensive breakdown of memory consumption, system throughput, and normalized hardware budgets, please refer to Appendix

---

Published as a conference paper at ICLR 2026
5.4 ABLATION STUDY
We study two key factors in EditScore design: score range granularity and output format (Table 3 Increasing the target score range generally improves pairwise accuracy for both GPT-4.1 and GPT-5, peaking around [0, 25], while overly large ranges hurt performance due regression difficulty. In parallel, requiring the model to generate a rationale before the numeric score ("reasoning score") consistently outperforms direct scoring, yielding a +0.038 accuracy gain for EditScore-7B (0.621 + 0.659), These findings highlight that both appropriately chosen score range and reasoning- output are crucial for maximizing accuracy.
Table 3: The effect of score range granularity and reason first on the pairwise accuracy of VLMs on EditReward-Bente The ranges (O,,z) x)) indicate the target score space for model outputs. The lightly shaded column highlights our chosen configuration for optimal performance.
Method
Score
Range
Output Format
[0, 10]
[0. 20]
[0. 25]
[0, [0,30]
reasoning score
score
GPT-4.1
0.691
0.701
0.705
0.689
0.705
0.695
GPT-5
0.730
0.760
0.755
0.755
0.741
EditScore-7B (GPT-4.1)
0.605
0.657
0.659
0.629
0.659
0.621
APPLICATION OF EDITSCORE IN IMAGE EDITING
6.1 EXPERIMENTAL SETUP
Evaluation Method. To evaluate the effectiveness of EditScore in improving image editing models, we design two experiments. (1) best-of-N selection: EditScore used as a selector, where the edit- ing model generates multiple candidate outputs per input and EditScore chooses the best one. We evaluate the gain on three popular models: OmniGen2 (Wu al., 20256), 1luxee-Kontext-dev [Bat- fol et al. (2025) and Swen-Image-Fdiit Wu et al. (2025a). (2) Reinforcement learning: EditScore is employed directly as a reward model to fine-tune OmniGen2 via an additional RL stage, demonstrat- ing its utility as training signal. We adopt two widely used image editing benchmarks -Edittt Bench (Liu et al., 2025b) and ImgEdit-Bencn (Ye et al. practical editing tasks, assess the improvements brought by EditScore. For efficiency, use 7B variant of EditScore for experiments with optional self-ensemble.
cover a diverse range of
RL Training. Our main experiments conducted using OmniGen2 Wu al. (2025b), while also validate the generalization of our method on Flux- Mlux-Kontext-dev [Bttfol et al. (2025). Flow- GRPO Liu et al. (2025a) is employed with hyperparameters: sampling steps T 20, group size
G = 12, number of unique prompts = 24, noise level = 0.9, and KL weight 3 0.04. Ablation studies on hyperparameters are provided in Appendix E We also reformulate the standard Flow-
GRPO equation to align with OmniGen2's 's notation; detailed derivations are given in Appendices
and
6.2 VALIDATING EDITSCORE UTILITY VIA BEST-OF-N SECTION
Before implementing full RL training, we first validate the utility of EditScore in a controlled setting. In this experiment, the editing model generates N candidate outputs per input of GEdit-Benchh and EditScore selects the best one. This setup directly measures the reward model's s ability to identify high-quality edits. rap benchmark performance as function of N. We evaluate three base models using both single-pass EditScore (solid lines) and the stronger Avg @4 version (dashed lines). The results highlight three key findings: (I) stronger reward models consistently yield better selections, (2) performance gains vary by base model, with Owen-Image- image-Edit showing the least improvement due to prior tost-training stage (Wu et al. 2025a). and (3) OmniGen2 exhibits the largest absolute potential for improvement, motivating its choice as the base model for subsequent RI. experiments.
8

---

Published as a conference paper at ICLR 2026
Boodts Ferformance
EdiSooe Shows
Performance Staliity
Ensembiingg Condibeenty Perforance
7.5
6.25
(lrght)
Score (Overan
Fax Rentrat-t
Tho 1 Montesn- diegge))
trane Edt (ugght
6.45
64
6.2
100
530
200
410
6.01
N: #sampled images (logscala)
Fairing Steps
Trarring 54ups
(a)
(b)
(c)
Figure 3: EditScore as superior reward signal for image editing. (a) Using EditScore to select the best sample among multiple outputs effectively improves VIEScore, with OmniGen2 showing the largest gain. (b) Incorporating EditScore into RL training yields stable and significant perfor- mance improvements, even surpassing the much larger Qwen2.5-VL-72D (c) RL training benefits from selff-nnsemling, which enhances the evaluation accuracy of EditScore across diverse settings.
Table 4: Controlled Evaluation of Reward Models. Using OmniGen2 the fixed base, com- pare policies trained with different reward signals. EditScore variants consistently outperform both raw VLMSs and 1PT-4.11 even the -ittr--distriuution (OOD) EmuFdii benchmark. Note that raw Qwen2.5-VL baselines fail to provide effective guidance.
GEdit-Bend
ImgEdit Overall
EmuEdit (OOD)
Reward Signal
SC
PQ
Overall
CLIP-I
CLIP-O DINO
Baseline (No RL)
6.72
7.20
6.28
3.40
0.876
0.309
0.822
RL w/ Qwen2.5-VL--7
6.87
6.86
6,22
RL w/ Qwen2.5-VL-72B
6.89
7.21
6.42
3.60
0,871
0.311
0.814
RL w/ GPT-4.1
7.24
7.41
6.73
3.66
0.884
0.312
0.843
RL w/ EditScore-7B (Avg@4)
7.20
7.46
6.68
3.63
0.900
0.310
0.870
RL w/ EditScoore-Qwen--VV--- (Avg@4)
7.27
7.77
6.89
3.62
0,891
0.310
0.861
6.3 ONLINE REINFORCEMENT LEARNING WITH EDITSCORE
We now turn to the core application of our framework: using EditScore as direct reward signal for online RL fine-tuning. This section systematically isolates the impact of reward quality, examines training stability, and evaluates generality across models and algorithms. We additionally provide qualitative visual comparisons in Appendix
6.3.1 UNLOCKING STABLE RL WITH A HIGH-FIDELITY REWARD SIGNAL
To isolate the contribution of the reward signal, conducted controlled study using Omni- Gen2 as the policy model while varying the source of supervision. As summarized in Table 4 raw instruction-tuned VLMs prove ineffective to provide meaningful guidance, resulting in unstable optimization trajectories (see Figure 3b). This underscores that parameter scale alone cannot substitute for task-specific alignment. In contrast, our specialized EditScore-7B not only stabilizes training but also demonstrates competitiveness with the proprietary GPT-4.11 surpassing it on the 0.884 CLIP-I). Furthermore, our approach benefits directly from stronger backbones: EditScore-8B (adapted from Qwen3-VL--8t (Bai et al. 2025al) achieves the highest overall performance across metrics, confirming the method's 's scalability with advancing VLMs.
verifiers; even the large-scale Qwen2.5-VL-725 fails
EmuEdit benchmark (e.g., 0.900 vs.
Beyond the reward model, we further investigate the impact of test-time compute. Figure Bc com- pares policy learning curves guided by single-pass EditScore versus 4-pass self-ensemble strat- egy (Avg@4). The ensembled signal significantly improves evaluation accuracy, leading to faster convergence and superior final performance. As discussed in Appendix [ single pass evalua- tion occasionally produces unstable reward signal, while self-ensemble alleviate it, thus providing a consistent gradient for the policy to traverse.

---

Published as a conference paper at ICLR 2026
Table 5: Generality Analysis. Left: Consistent improvements on different backbone (Flux-
Kontext-dev). Right: Compatibility with advanced RL algorithms (TempFlow-GRPO (He et al. 2025
(a) Flux-Kontext-dee Backbone
Algorithm Comparison (OmniGen2 base)
Method
SC
PQ
O
Algorithm
SC
PQ
0
Baseline
6.59
7.61
6.15
Flow-GRPO
7.20
7.46
6.68
RI. w/ Ours
7.16
7.95
6.87
TempFlow-GRPO
7.53
7.99
7.21
6.4 GENERALITY ACROSS EDITORS AND ALGORITHMS
We further assess the robustness of EditScore-7B (Avg@4) across different generative backbones and RL algorithms (Table 5.. To verify model-agnostic generality, we apply it substantially different model. Sou-KKontext-dev (Batifol et al.) 2025), observing strong improvements across all metries (+0.57 SC, +0.72 Overall). Furthermore, substituting standard Flow-GRPO with TempFlow- GRPO (He et al. 2025 wwhich uses temporally aware loss reweighting-delivee further gains (Overall score 7.21). This confirms that our reward signal universally applicable and scales robustly with advanced RL algorithms without causing reward hacking.
6.5 ANALYSIS OF ANNOTATION SOURCES AND REWARD VARIANCE
Table 6: Comparison of reward models based on annotator source. We analyze reward models (RMs) trained on data from two different annotators: GPT-4.1 and GPT-5. The resulting policy's performance is evaluated by both annotators on GEdit-Bench (Liu et al. 2025b).
Reward Model Metrics
Policy Performance t
Annotator
Component
Acc. (0) f
Score Std.
GPT-4.1 Score
GPT-5 Score
Annotator (Itself)
0.705 0.637
3.309
GPT--111
EditScore (RM)
2.868
(Better for RI.)
OmniGen22 (Policy)
6.375
5,834
Annotator (Itself)
0.755 0.638
2.942
GPT-5
EditScore (RM)
2.533
(Worse for RL)
OmniGen2 (Policy)
6.292
5.768
To understand the role of annotators in shaping RL performance, we trained two distinct EditScore- 7B reward models on identical data subsets, annotated by GPT-4.1 and GPT-5 separately. While GPT-5 offers superior annotation accuracy, the higher score variance of GPT-4.1 provides a more effective learning signal. As shown in Table 6. EditScore trained on GPT-4.1's labels, despite marginal dip in accuracy (0.637 vs. 0.638), inherits a substantially higher score standard deviation (2.868 vs. 2.533). This high- high-variance reward signal leads demonstrably better policy after RI. fine-tuning. Crucially, this improvement is consistent between both GPT-4.1 (6.375 vs. 6.292) and GPT-5 (5.834 vs. 5.768) evaluations, excluding potential evaluation bias from GEdit-Bench (Ye al. 2025). Our results thus uncover key insight: reward signal variance, rather than absolute annotator accuracy, can be the dominant factor for successful optimization of the RL-based model, corroborating similar findings from Razin et al. (2025).
7 CONCLUSION
In this work, we addressed the critical bottleneck for RL in image editing: the lack of a reliable reward signal. We established a comprehensive benchmark for reward model evaluation. Guided by our benchmark, we developed EditScore, family of specialized generative reward models that de- liver a robust, high-fidelity signal. Crucially, EditScore is not only efficient for Best-of-N selection but is the key to unlocking stable online RL where previous open-source models fail. By releasing both editRe Bench and EditScore, we provide foundational toolkit for future research into RI--based image editing and more nuanced reward modeling.
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Saba Ahmadi, Rabiul Awal, Ankur Sikarwar, Amirhossein Kazemnejad, Ge Ya Luo, Juan A Ro- driguez, Sai Rajeswar, Siva Reddy, Christopher Pal, Benno Krojer, al. The promise of for autoregressive image editing. arXiv preprint arXiv:2508.011/9 2025.
Michael S. Albergo, Nicholas M. Boffi, and Eric Vanden-Fijnden. Stochastic interpolants: A uni- fying framework for flows and diffusions. CoRR, abs/2303.08897,, 2023, doi: 10,48550/ARXIV. 2303.08797.
org/0. 48550/arXiv.
Shuai Bai, Yuxuan Cai, Ruizhe Chen, Keqin Chen, Xionghui Chen, Zesen Cheng, Lianghao Deng, Wei Ding, Chang Gao, Chunjiang Ge, et al. Qwen3-vl technical report. arXiv preprint arXiv:2511.21631 2025a.
Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, al. Qwen2. 5-vl technical report. arXiv preprint arXiv:2502. 13923, 2025b.
Stephen Batifol, Andreas Blattmann, Frederic Boesel, Saksham Consul, Cyril Diagne, Tim Dock- horn, Jack English, Zion English, Patrick Esser, Sumith Kulal, et al. Flux. kontext: Flow match- ing for in-context image generation and editing latent space. arXiv e-prints, pp. arXiv-2506, 2025.
Tim Brooks, Aleksander Holynski, and Alexei A Efros. Instructpix2pix: Learning follow image editing instructions. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 18392-18402, 2023.
Sherry X Chen, Yi Wei, Luowei Zhou, and Suren Kumar. Adice: Automatic dataset creation and scorer for instruction-guider image editing evaluation. arXiv preprint arXiv:2507.07311. 2025.
Chaorui Deng, Deyao Zhu, Kunchang Li, Chenhui Gou, Feng Li, Zeyu Wang, Shu Zhong, Weihao Yu, Xiaonan Nie, Ziang Song, et al. Emerging properties in unified multimodal pretraining. arXiv preprint arXiv:2505. /4683, 2025.
Yu Gao, Lixue Gong, Qiushan Guo, Xiaoxia Hou, Zhichao Lai, Fanshi Li, Liang Li, Xiaochen Lian, Chao Liao, Liyang Liu, al. Seedream 3.0 technical report. arXiv preprint arXiv:2504. 1346, 2025.
Yuan Gong, Xionghui Wang. Jie Wu, Shiyin Wang, Yitong Wang, and Xinglong Wu. Onereward: Unified guided image generation via multi-task human preference learning. arXiv preprint 1rXiv:2508.2/066 2025.
Google. Introducing gemini 2.5 flash image. en/int
goog eelll.loooo....o.....o .omm --tieemmiii--------- fassffflsh-immmn image/ 2025. Accessed: 2025-009-8.
Xiaoxuan He, Siming Fu, Yuke Zhao, Wanli Li, Jian Yang, Dacheng Yin, Fengyun Rao, and Bo Zhang. Tempflow-grpo: When timing matters for grpo in flow models. arXiv preprint 043444 2025.
Amir Hertz, Ron Mokady, Jay Tenenbaum, Kfir Aberman, Yael Pritch, and Daniel Cohen-Or. Prompt-to-promp image editing with cross attention control. arXiv preprint arXiv:2208.0166 01626, 2022.
Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. Lora: Low-rank adaptation of large language models. ICLR, 1(2:3, 2022.
Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Os- trow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card. arXiv preprint arXiv:2410.21276, 2024.
Dongfu Jiang, Max Ku, Tianle Li, Yuansheng Ni, Shizhuo Sun, Rongqi Fan, and Wenhu Chen. Genai arena: An open evaluation platform for generative models. Advances in Neural Information Processing Systems, 37:79889-799088 2024.

---

Published as a conference paper at ICLR 2026
Yuval Kirstain, Adam Polyak, Uriel Singer, Shahbuland Matiana, Joe Penna, and Omer Levy. Pick- a-pic: An open dataset of user preferences for text-to- -iage generation. Advances neural information processing systems, 36:36652-366663 2023.
Max Ku, Dongfu Jiang. Cong Wei, Xiang Yue, and Wenhu Chen. Viescore: Towards explainable metrics for conditional image synthesis evaluation. arXiv preprint arXiv:2312.1 14867, 2023a.
Max Ku, Tianle Li, Kai Zhang, Yujie Lu, Xingyu Fu, Wenwen Zhuang, and Wenhu Chen. Ima- genhub: Standardizing the evaluation of conditional image generation models. arXiv preprint arXiv:2310.01596 2023b.
Max Ku, Dongfu Jiang, Cong Wei, Xiang Yue, and Wenhu Chen. Viescore: Towards explain- able metrics for conditional image synthesis evaluation. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Proceedings of the 62nd Anmual Meeting of the Association for Computa- tional Linguistics (Volume 1: Long Papers), ACL 2024, Bangkok, Thailand, August 1 11-16. 2024, pp. 12268-1 2290.. Association for Computational I Linguistics, 2024. doi: 10.18653/V 220/66////12224444 1/2024. ACL-LONG.6633. htt.hhtttps///oii... http://ooo.orgg////
ac1--oo....looooo.. 663
Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. Blip: Bootstrapping language-image pre- training for unified vision-language understanding and generation. In International conferecce machine learning. pp. 12888-12900. PMLR, 2022.
Yuxi Li. Deep reinforcement learning: An overview. arXiv preprint arXiv: 2017.
Yaron Lipman, Ricky T. Q. Chen, Heli Ben-Hamu, Maximilian Nickel, and Matthew Le. Flow matching for generative modeling. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.net, 2023. URL https://cpenxeviier net/ forum? .nt.://orun...--PPMRCCCCCCC Pd=PPVMRDDJI19
Haotian Liu, Chunyuan Li, Yuheng Li, Bo Li, Yuanhan Zhang. Sheng Shen, and Yong Jae Lee. Llavanext: Improved reasoning, ocr, and world knowledge. 2024.
Jie Liu, Gongye Liu, Jiajun Liang, Yangguang Li, Jiaheng Liu, Xintao Wang, Pengfei Wan, Di Zhang, and Wanli Ouyang. Flow-grpo: Training flow matching models via online rl. arXiv preprint S.Kiv:2505.054700 2025a.
Shiyu Liu, Yucheng Han, Peng Xing, Fukun Yin, Rui Wang, Wei Cheng, Jiaqi Liao, Yingming Wang, Honghao Fu, Chunrui Han, et al. Steplx-edit: A practical framework for general image editing. arXiv preprint MxXiv:554.1776.1 2025b.
Zijun Liu, Peiyi Wang, Runxin Xu, Shirong Ma. Chong Ruan, Peng Li, Yang Liu, and Yu Wu. org/
Inference-time scaling for generalist reward modeling, 2025c. URL
abs/2504.0243 02495
Zijun Liu, Peiyi Wang, Runxin Xu, Shirong Ma, Chong Ruan, Peng Li, Yang Liu, and Yu Wu. Inference-time scaling for generalist reward modeling, arXiv preprint arXhv:2504.02495, 2025d.
Yuhang Ma, Xiaoshi Wu, Keqiang Sun, and Hongsheng Li. Hpsv3: Towards wide-spectrum human preference score. arXiv preprint arXiv:2508.037788 2025.
OpenAI. Gpt-5 is here. fttps:://opeaa..oom
2025. Accessed: 2025-09-18.
Noam Razin, Zixuan Wang, Hubert Strauss, Stanley Wei, Jason D Lee, and Sanjeev Arora. What makes reward model a good teacher? an optimization perspective. arXiv preprint DrXiv:2503.15477, 2025.
Yuxi Ren, Jie Wu, Yanzuo Lu, Huafeng Kuang, Xin Xia, Xionghui Wang, Qianqian Wang, Yixing Zhu, Pan Xie, Shiyin Wang, et al. Byteedit: Boost, comply and accelerate generative image editing. In European Conference on Computer Vision, pp. 184-200. Springer, 2024.
John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimovv Proximal policy optimization algorithms. CoRR, 701.0007777777777 2017. URL http: / //trxi/axvvvv.v/.aa/ org/ abs/
1707. 06347
12

---

Published as a conference paper at ICLR 2026
Ozan Sener and Silvio Savarese. Active learning for convolutional neural networks: A core-set approach. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30- May 3. 2018, Conference Track Proceedings. OpenReview.net, 2018. net
URl.httss: wtt.htttss///ppeeeeeeee http://pppeevvveeeee /tt.torumt/paaeoooo
forum? a -uk
Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. Deepseckmath: Pushing the limits of mathematical reasoning in open language models. CoRR, abs/2402.03300, 2024. doi: 10.48550AARXI122200005500 URL
48550/ /a1/s///axvv..o w4w.x.ii..40..... 2432003300
Charlie Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. Scaling Ilm test-time compute optimally can be more effective than scaling model parameters, 2024. URL. htttp:://axxi..oo
abs / 2408 03314
Yang Song. Jascha Sohl-Dickstein, Diederik P. Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In 9th International Conference on Learning Representations, ICLR 2021, Viruull Event, Austria, May 3-7, 2021. OpenReview.net, 2021. URL https:://penrewieew http://wwwwweett.nt.t/ee/
PXTIG12RRHS
Robert J Tibshirani and Bradley Efron. An introduction to the bootstrap. Monographs on statistics and applied probability, 57(1):1 1993.
Yibin Wang, Zhimin Li, Yuhang Zang, Chunyu Wang, Qinglin Lu, Cheng Jin, and Jiaqi Wang- Unified multimodal chain-of-thought reward model through reinforcement fine-tuning. CoRR, abs/2505.03318,, 2025a. doi: 10.48550/RXIV.25000 033118. URL httthtts////oo.coo http//to..ogg....
48550/arXiv 03318
Yibin Wang, Yuhang Zang, Hao Li, Cheng Jin, and Jiaqi Wang. Unified reward model for multi- modal understanding and generation. arXiv preprint arXiv:2503.05236. 2025b.
Hongyang Wei, Baixin Xu, Hongbo Liu, Cyrus Wu, Jie Liu, Yi Peng, Peiyu Wang. Zexiang Liu, Jingwen He, Yidan Xietian, et al. Skywork unipie 2.0: Building kontext model with online rl for unified multimodal model. arXiv preprint arXiv:2509.04544 2025.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoe V. Le, and Denny Zhou. Chain-of-thought prompting elicits reasoning in large language models. In Sanmi Koyejo, S. Mohamed, A. Agarwal, Danielle Belgrave, K. Cho, and A. Oh (eds.), Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurIPS 2022, New Orleans, LA, USA, November 28 De- cember 9, 2022, 2022. URL htLlhttp:///aaaee .ips capppprr pttppppaaeellleeeappe///////// hash/ 9d5 961 155243
htm
Chenfei Wu, Jiahao Li, Jingren Zhou, Junyang Lin, Kaiyuan Gao, Kun Yan, Sheng-ming Yin, Shuai Bai, Xiao Xu, Yilei Chen, et al. Qwen-image technical report. arXiv preprint arXiv:2508.02322 2025a.
Chenyuan Wu, Pengfei Zheng, Ruiran Yan, Shitao Xiao, Xin Luo, Yueze Wang, Wanli Li, Xiyan Jiang, Yexin Liu, Junjie Zhou, al. Omnigen2: Exploration advanced multimodal generation. arXiv preprini 2506v:2506. 1877, 2025b.
Jie Wu, Yu Gao, Zilyu Ye, Ming Li, Liang Li, Hanzhong Guo, Jie Liu, Zeyue Xue, Xiaoxia Hou, Wei Liu, Yan Zeng, and Weilin Huang. Rewarddance: Reward scaling in visual generation, 2025c.
URL htLttt:s:///axi org/ abs/2509 08826
Xiaoshi Wu, Yiming Hao, Keqiang Sun, Yixiong Chen, Feng Zhu, Rui Zhao, and Hongsheng Li. Human preference score v2: A solid benchmark for evaluating human preferences of text-to- image synthesis. arXiv preprint arXiv:2306.094 09341, 2023a.
Xiaoshi Wu, Keqiang Sun, Feng Zhu, Rui Zhao, and Hongsheng Li. Human preference Better aligning text-to-image models with human preference. In Proceedings of the IEEE/CVF International Conference Computer Vision, pp. 2096-2105, 2023b.
13

---

Published as a conference paper at ICLR 2026
Shitao Xiao, Yueze Wang, Junjie Zhou, Huaying Yuan, Xingrun Xing, Ruiran Yan, Chaofan Li, Shuting Wang. Tiejun Huang, and Zheng Liu. Omnigen: Unified image generation. In Proceed- ings of the Computer Vision and Pattern Recognition Conference, pp. 12294-13304 2025.
Jiazheng Xu, Xiao Liu, Yuchen Wu, Yuxuan Tong, Qinkai Li, Ming Ding, Jie Tang, and Yuxiao Dong. Imagereward: Learning and evaluating human preferences for text-to-image generation. Advances in Neural Information Processing Systems, 36: 16:15593-1155 15935, 2023.
Zeyue Xue, Jie Wu, Yu Gao, Fangyuan Kong, Lingting Zhu, Mengzhao Chen, Zhiheng Liu, Wei Liu, Qiushan Guo, Weilin Huang, et al. Dancegrpo: Unleashing grpo on visual generation. arXiv 2025.
preprint arXiv:2505.0788
Yang Ye, Xianyi He, Zongjian Li, Bin Lin, Shenghai Yuan, Zhiyuan Yan, Bohan Hou, and Li Yuan. Imgedit: unified image editing dataset and benchmark. arXiv preprint arXiv:2505.2027. 20275, 2025.
Kai Zhang, Lingbo Mo, Wenhu Chen, Huan Sun, and Yu Su. Magicbrush: A manually annotated dataset for instruction senectiionnnideed image editing. Advances in Neural Information Processing Systems, 36:31428 1111422--1449, 2023.
Shu Zhang, Xinyi Yang, Yihao Feng, Can Qin, Chia-Chih Chen, Ning Yu, Zeyuan Chen, Huan Wang, Silvio Savarese, Stefano Ermon, et al. Hive: Harnessing human feedback for instructional visual editing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9026 9036, 2024a.
Sixian Zhang, Bohan Wang, Junqiang Wu, Yan Li, Tingting Gao, Di Zhang, and Zhongyuan Wang- Learning multi-dimensional human preference for text-to-image generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8018-8027, 2024b.
Yaowei Zheng, Richong Zhang, Junhao Zhang, Yanhan Ye, Zheyan Luo, Zhangchi Feng, and Yongqiang Ma. Llamafactory: Unified efficient fine-tuning of 100+ language models. In Pro- ceedings of the 62nd Annual Meeting of the Association for Computational Linguisties (Volume 3: System Demonstrations), Bangkok, Thailand, 2024. Association for Computational Linguis- tics.
org/ ats///b///2/00 230./223.13372
A STATEMENT ON THE USE OF LARGE LANGUAGE MODELS (LLMS)
In adherence to the ICLR 2026 submission guidelines, this section details the use of a Large Lan- guage Model (LLM) assistant during the preparation of this manuscript. The LLM, acting as a research and writing co-pilot, played significant role in refining the manuscript's structure, lan- guage, and presentation. The authors maintained full intellectual control throughout the process and take complete responsibility for all content.
The precise role of the LLM can categorized as follows:
The authors wrote the initial drafts for all sections of the paper, providing the key technical details, experimental results, and core arguments. The LLM was then used extensively as interactive writing assistant
Manuscript Writing and Polishing:
Enhance Clarity and Conciseness: Rephrasing long or complex sentences to im-
prove readability and flow.
Improve Academic Tone: Suggesting more formal and professional vocabulary and sentence structures appropriate for top-tier conference submission.
Correct Grammar and Syntax: Performing comprehensive proofreading identify and correct grammatical errors, typos, and awkward phrasing.
-Suggest Alternative Phrasing: Providing multiple options for expressing a single idea to avoid repetitive language).
Technical Formalization: For complex mathematical sections, such the derivation of the SDE from the ODE in Appendix F the authors provided the core mathematical steps and handwritten notes. The LLM assisted in translating these steps into clear, well- structured narrative and formatting them professionally in LaTeX. The LLM did not gen- erate novel mathematical proofs but rather helped in their presentation.
14

---

Published as a conference paper at ICLR 2026
Throughout this collaborative process, every suggestion and piece of text generated by the LLM was critically reviewed, edited, and approved by the human authors. The authors are solely responsible for the scientific validity, originality, and all claims made in this paper. The LLM is not considered an author.
B DISAGREEMENT ANALYSIS OF ANNOTATION PROTOCOL
To ensure the high fidelity of the EditReward-Benne dataset, we prioritized minimizing annotation noise and subjectivity. While standard crowdsourcing oflen relies on aggregating labels from in- dependent workers, evaluating image editing involves subtle visual inspections that prone to individual oversight.
To validate choice of the Two-Annotator Discussion Protocol, we conducted controlled study comparing it against a standard Single-Annotati baseline.
B.1 CONTROLLED STUDY SETUP
We recruited experts from our annotation pool and divided the validation process into two distinct rounds to annotate the same subset of samples across three evaluation dimensions: Prompt Following (PF), Consistency (C), and Overall Quality (O).
Baseline: Single Annotator. Four experts annotated the samples independently. We ana- lyzed the consistency among these independent raters.
Ours: Annotator Pair (Two-Annotator Group). Eight experts were randomly formed
into four pairs. Each pair followed proposed protocol: they examined images together, discussed visual artifacts in real-time to reach consensus. This process typically requires more time per sample than single-annotator approaches.
B.2 METRICS
We evaluate the reliability of annotation process using key metries:
1. Convergence Rate: This metric quantífies inter-annotator consistency. It calculates the probability that distinct annotation discussion groups (Annotator Pairs) Pairs) assign identical rankings to the same input.
individual experts (Single Annotators)
2. Agreement with Majority Vote: Serving a proxy for ground truth, the majority vote is derived by aggregating all available annotations for a sample. This metric measures the alignment between individual unit's ranking and the collective consensus. High agreement indicates that the judgment of a single unit reliably reflects the expert consensus, thereby justifying our efficient single-pan pases annotation strategy.
B.3 RESULTS AND ANALYSIS
Impact of Discussion on Visual Consistency. As shown in Table 7 the Two-Annotator protocol consistently outperforms the Single-Annotaton baseline across all dimensions. Notably, the most significant improvement is observed the Consistency dimension, where the strict agreement rate (100% threshold) increased by 12.12% (from 82.88% 95, 95,00%). We attribute this to the inherent nature of image editing evaluation: inconsistency often manifests as subtle background distortions minor structural artifacts that easily overlooked by single individual. The real-time discus- sion protocol acts as verification mechanism. forcing annotators to cross-check visual details and effectively eliminating such oversights.
Validation of Multi--Dimensionl Evaluation. The results also highlight the importance of our decomposed evaluation strategy. Across both annotation settings, the agreement rates for Prompt Following and Consistency consistently higher than for Overall Quality (e.g., 93-95% 90% in strict agreement). This disparity underscores the fact that "Overall Quality" inherently more subjective and prone to rater bias. By explicitly separating the evaluation into specifie, objective
15

---

Published as a conference paper at ICLR 2026
dimensions (PF and C), ensure that the majority of EditReward-Bench relies rigorous, repro- ducible criteria, rather than vague holistic impressions.
Efficiency and Reliability of Single- Annotation. Finally, Table g demonstrates that the rankings produced by single Annotator Pair achieve near-perfect alignment with the collective majority vote, averaging over 97 across all dimensions (with Consistency reaching 99.55%) This finding crucial for the scalability of our benchmark. It indicates that the consensus reached by one expert pair is statistically equivalent to the "ground truth" derived from a larger pool of annotators. Consequently, this justifies strategy of annotating each sample only once (by single pair), allowing us to maintain high-fidelity ground truth while significantly optimizing annotation costs compared multi-pass crowdsourcing methods.
Table 7: Convergence Rate Comparison. The Annotator Pair protocol consistently outperforms the Single Annotator baseline. The substantial improvement in Consistency (+12.12% confirms that the discussion protocol helps identify subtle visual artifacts often missed by individuals.
Prompt Following (PF)
Consistency (C)
Overall Quality (O)
Metric
Single
Pair (Ours)
Single
Pair (Ours)
Single
Pair (Ours)
75% Threshold
92.23%
95.24% (+3.01%)
87 87.39%
95.71 % (+8,32%) 32%
84.97%
92.01% % (+7.04 +7.00%)
100% Threshold
90.29%
93.33% (+3.04%)
82.88%
95.00 % (+12.12%)
79.06%
90.32% (+11.26%)
Table 8: Agreement with Majority Vote. Comparing Single Annotators vs. Annotator Pairs against the collective majority vote. The high agreement 97%) of Annotator Pairs justifies our efficient single-pun passspassssss annotation strategy.
Prompt Following (PF)
Consistency (C)
Overall Quality (O)
Unit ID
Single
Pair (Ours)
Single
Pair (Ours)
Single
Pair (Ours)
Unit
100.00%
97.80%
100.00%
100.00%
97.09%
97.18%
Unit
100.00%
100.00%
93.41%
99.11 10.11%
97.52%
97.56%
Unit
97.14%
100.00%
97.85%
100.00%
97.58%
98.76%
Unit 4
94.68%
97.03%
94.38%
99.10%
88.80%
94.86%
Average 97.96%
98.71%
96.41%
99.55%
95.25%
97.09%
C EFFICIENCY AND COMPUTATIONAL. COST ANALYSIS OF EDITSCORE
While the self-ensemble strategy effectively boosts reward accuracy, is crucial evaluate its com- putational overhead to ensure practicality in real-world deployments. In this section, we provide a comprehensive analysis of memory consumption, wall-clock latency, and cost-normalized perfor- mance specifically for our EditScore models.
C.1 MEMORY CONSUMPTION AND HARDWARE SETUP
A common concern with ensemble methods is the potential explosion in memory usage. However, our self-ensemble approach does not increase memory consumption relative to the base model. Since all K queries are processed by the same underlying model instance, the weights shared on the host GPU. The memory footprint is determined solely by the model parameters, not the ensemble size K.
In our experiments, we utilized NVIDIA H100 GPUs. To ensure efficient inference latency via Tensor Parallelism (TP), adopted the following hosting configurations:
EditScore-7B: Hosted on 1 x NVIDIA H100.
EditScore-32B: Hosted NVIDIA H100 (TP=2).
EditScore-72B: Hosted 4 NVIDIA H100 (TP=4).
16

---

Published as a conference paper at ICLR 2026
C.2 WALL-CLOCK EFFICIENCY AND LATENCY
Although setting K - 4 implies four logical forward passes, the impact on wall-clock latency is sublinear. In deployment (using efficient serving frameworks like sglang), the prefill stage is shared across the K diverse decoding paths. Consequently, increasing K reduces throughput but does not simply multiply latency by K.
Real- World RL Training Context. To illustrate the practical impact, we measured latencies dur- ing our Reinforcement Learning (RL) fine-tuning pipeline. single training step involves 576 reward queries. Using EditScore-7B-avgg on an 8 8XH100 cluster, the system completes all queries in approximately 18 seconds. This constitutes only 200 of the total training iteration time, con- firming that the reward calculation is not bottleneck.
C.3 COST-NORMAN NORMALIZEDD PERFORMANCE TRADE-OFF
To provide a fair comparison under a fixed hardware budget, we normalize the throughput calcula- tions to standard compute node equipped with 4 NVIDIA H100 GPUs.
Based on our hosting configurations above:
EditScore-7BB (1 GPU/instance) allows hosting 4 parallel instances, quadrupling the sys- tem throughput.
EditScore-32B (2 GPUs/instance) allows hosting parallel instances.
EditScore-72BB (4 GPUs/instance) allows hosting instance.
Table 9 presents the accuracy and the projected system throughput under this standardized budget.
Table 9: sorcieney-Performanne Trade-off under Fixed Budget (4 (4xH)00). We compare ac- curacy against the total system throughput derived from a 4-GPU budget. By leveraging parallel instances, smaller models with self-ensemble (K = 4) achieve both superior accuracy and higher throughput than larger single models.
Model Ensemble (K)
Accuracy (Overall)
GPUs per Instance (Used)
Parallel Instances
System Throughput (samples/s)
TFLOPs
(on 4xH100)
pper sample)
K
0.659
4
37.84
36.7
EditScore-7B
0.714
26.32
73.4
K
0.727
4
16.24
146.7
K
0.680
2
2
9.48
153.4
EditScore-32B
0.715
2
6.32
306.8
K
0.733
2
3,84
613.6
K
0.703
1
3,88
328.5
EditSccor-72B
K 0.763
1,60
1314.0
Key Observations.
1. High Accuracy at Higher Throughput: The analysis reveals that EditScore-7B (K 4) is a particularly efficient configuration. It achieves an accuracy of 0.727, which is signif- icantly higher than the single-pass EditScore-32B (K at 0.680. More importantly, under the same 4-GPU budget, the 7B ensemble offers nearly double the throughput (16.24 vs. 9.48 samples/s) because the hardware accommodate 4 parallel 7B instances versus only 32B instances.
2. Efficiency "Sweet Spot": While larger models (72B) provide the highest absolute accu- racy (0.763), they come at steep cost in throughput (1.60 samples/s). For high-volume applications like RL training, the 7B (K 4) setting provides the optimal balance, deliv- ering state-of-the-art reward signals with acceptable latency.
3. Scalability: The results confirm that increasing K for smaller models is a more cost- effective strategy than scaling up model size. The self-ensemble approach effectively trades abundant parallel compute (which is often cheaper than high-memory nodes) for quality improvements.
17

---

Published as a conference paper at ICLR 2026
In summary, when normalized for hardware budget, the self-ensemble strategy (K = 4) smaller models proves to highly efficient, outperforming larger baselines in both accuracy and system throughput.
D STATISTICAL SIGNIFICANCE ANALYSIS
To rigorously validate that the performance improvements reported in our main results (specifically for EditScore and the downstream RL fine-tuning) not attributed to random variance, con- ducted comprehensive statistical significance analysis using paired bootstrap tests (Tibshirani & Efron T993).
D.1 METHODOLOGY
For every comparison between baseline model A and our method B, performed 10,000 boot- strap resampling rounds. In each round, we resampled the evaluation pairs from the test set with replacement and recomputed the performance metric for both models obtain the difference
6, = ) ((i)
Based the distribution of these 10,000 differences, computed:
95% Confidence Intervals (CI): The range within which the true performance difference lies with 95% probability. An interval strictly above zero indicates significant improve- ment.
2. One-sided p-values: The proportion of bootstrap resamples where our method failed to outperform the baseline (i.e., SB I SA). A p-value < 0.05 indicates statistical signifi- cance.
D.2 SIGNIFICANCE EDITSCORE PERFORMANCE
We compared EditScore against the baseline reward model Qwen2.5-VI. across three model scales (7B, 32B, and 72B) EditReward-Bentn As detailed in Table TO EditScore demonstrates statis- tically significant improvements across all metrics and scales. Notably:
For the 7B and 32B models, the p-values are extremely low (ss 0), indicating robust supe- riority.
Even at the 72B scale, where baselines are stronger, EditScore maintains significance with p < 0.05 across dimensions, with the difference in Consistency (C) and Overall (O) being particularly significant (p S 0.0005).
Table 10: Statistical Significance on EditReward-Bench. We report the mean scores, the per- formance delta (A), the 95% Confidence Interval (CI) of the delta, and the one-sided p-value. All improvements are statistically significant (p < 0.05).
Size
Metrie
Baseline (Qwen2.5-VL.)
EditScore (Ours)
A (Ours Base)
95% CI of A
p-value
PF
0.458
0.592
+0.134
0.0360, 0.1271
0.0001
7B
C
0.325
0.591
+0.266
0.2202, 0.3079
< 0.0001
0
0.432
0.659
+0.227
0.0703, 0. 0.1486
< 0.0001
PF
0.498
0.638
+0.140
00.0763. 0.1515)
< 0.0001
32B
C
0.376
0.556
+0.180
00.1337, 0.2101
< 0.0001
0
0.563
0.680
+0.117
[0.0137, 0.0808
0.0033
PF
0.540
0.635
+0.095
[-0.0042, 0.0646]
0.0466
72B
C
0.435
0.586
+0.151
[0.0292. 0.1101]
0.0005
0.621
0.703
+0.082
0.0242, 0.0889
0.0002
18

---

Published as a conference paper at ICLR 2026
D.3 SIGNIFICANCE RL FINE-TUNING IMPROVEMENTS
We further validated the improvements observed in the Reinforcement Learning (RL) experiments on GEdit-Bench. Here, compared the RL policy trained with the Baseline Reward Model (Qwen2.5-VL-72BB against the policy trained with EditScore-7B-avg4.
Table I1 confirms that using EditScore as the reward signal leads to highly significant improvements (p RJ 0.0002) across all evaluation metries: Semantic Consistency (SC), Perceptual Quality (PQ), and Overall score (O). This indicates that the gains provided by EditScore are robust and translate effectively to downstream policy optimization.
Table 11:
Statistical Significance on GEdit-Bench (RL Frine-Tuning Comparison between RL policies trained with the Baseline Reward vs. EditScore Reward. The high positive A and low p- values confirm the robustness of our RL gains.
Metric
RL (Baseline Reward)
RL (EditScore Reward)
A Improvement
95% CI of A
p-value
SC
6.89
7.20
+0.338
0.1448, 0.5276
0.0003
PQ
7.21
7.46
+0.235
0.1121, 0.3569
0.0002
O
6.42
6.68
+0.282
00.1084, 0.4516
0.0003
E
ADDITIONAL ABLATION STUDIES ON HYPERPARAMETERS OF RL
In this section, we present detailed analysis of the influence of key RL hyperparameters the performance of our proposed method. Specifically, we investigate the impact of the number in- ference steps during sampling and the group size (i.e., the number of samples generated per prompt) used during Reinforcement Learning (RL) training. All results reported in this section are based on the GEdit-Benche Liu al. (0255)).
E.1
IMPACT OF INFERENCE STEPS
Table 12: Ablation studies on RL hyperparameters evaluated on GEdit- Liu et al. 2025b).
(a) Impact Inference Steps (T").
(b) Impact of Group Size (under fixed global batch size).
Steps
SC
PQ
O
Unique Prompts
Group Size
SC
PQ
12
7.35
7.75
6.98
36
8
7.36
7.89
7.07
16
7.42
7.80
7.11
24
12
7.53
7.99
7.21
20
7.53
7.99
7.21
18
16
7.26
7.80
6.90
We first examine the effect of varying the number of inference steps (T) employed by the flow model. Table 12a summarizes the model performance with T' E (12, 16, 20).
As illustrated in Table 12a increasing the number of inference steps leads consistent improve- ments across all evaluated metrics (SC, PQ, and O). This trend aligns with expectations, as a larger number of steps enables the SDE solver to generate higher-fidelity samples that more accurately re- flect the underlying model distribution. Consequently, this yields more reliable reward signals dur- ing training. Conversely, reducing the steps may introduce discretization errors or artifacts, thereby degrading the quality of reward estimation and subsequent policy updates.
E.2 IMPACT OF GROUP SIZE
Next, we analyze the trade-off between the group size (i.c., the number of samples generated per prompt) and the diversity of prompts within training batch. To ensure fair comparison, maintain a constant total computational budget (i.e., the total number of tokens processed per update remains fixed). Therefore, increase in group size necessitates proportional decrease in the number of unique prompts included in each batch.
The results presented in Table 12b We observe optimal configuration group size of 12:
19

---

Published as a conference paper at ICLR 2026
Small Group Size (8): While this setting allows the model to encounter greater diversity of unique prompts (36), the variance advantage estimation for each prompt increases due to fewer samples. This instability can lead noisy and suboptimal policy updates.
Large Group Size (16): Although advantage estimation becomes more stable with larger group size, the significant reduction in unique prompts (18) limits batch diversity. This lack of diversity appears hinder the model's ability to generalize effectively, resulting in a notable performance drop (c.g., the score decreases to 6.90).
Optimal Group Size (12): Our empirical results indicate that a group size of 12 yields the best performance. This setting strikes an effective balance, providing sufficient samples for stable advantage estimation while maintaining adequate prompt diversity to ensure robust generalization.
F DERIVATION OF THE SDE FORMULATION FOR FLOW MATCHING
This appendix derives the stochastic differential equation (SDE) underlying policy sampling in our Flow Matching model. Unlike prior work such as Flow-GRPO (Liu et al. 2025a), our base model OmniGen2 defines the probability path from noise at / = 0 (xo) to data at the convention in some other studies. This leads to a different drift term in the resulting SDE. The derivation proceeds in three steps: (i) connecting the deterministic ODE and general SDE via the Pokkke--llanck equation, (ii) relating the score term to the learned vector field, and (iii constructing the discretized transition kernel.
(x1), reversing
F.1 CONNECTING THE PROBABILITY FL.w ODE AN SDE
Our Flow Matching model (Lipman et al. 2023 is trained to approximate the vector field et (xx) of a probability flow ODE, which deterministically transports samples from a noise distribution Po to a data distribution
dx, = 0, (xt)
We wish find an equivalent SDE of the general form
dx: = fxx,, G(t)dW,
(3)
that generates the same marginal probability density path px(x) for all t E [0, 1]. Here, f is the drift term, G(t) is the diffusion coefficient (we assume it is sate-indepenneet), and W, is a standard Wiener process.
The evolution of the probability density px(x) is deseribed by the Fokker-Plande equation (Song et al. 2021). For the deterministic ODE in equation it is:
0p.(x) - (x(())xx))(((
Öt
For the SDE in equation 3 the Fokker-Planee equation is:
(xr(x) V (xxx)) V [G())
(5)
Øt
For the two processes to be equivalent (i.e., to share the same (x)). the right-hand sides of equa- tion and equation must be equal. This allows us to solve for the SDE drift term f(x, t):
V [usp.] - [fpd] + 1 V. [VGGPp))
2
[v.p.] V [ c
UcP: = fp: 2
G2
(8)
Dividing by pt and rearranging yields the expression for the drift:
G(t)2
f(x,,t) = =x +
log (x.)).
(9)
2
20

---

Published as a conference paper at ICLR 2026
Substituting this back into equation
we obtain the equivalent SDE:
G(t)² V Vlog dt G(t)dW,
dx. = U4 (xa) +
(10)
2
We define that noise at t O (xa) and ending with the data sample at t 1 (x1). equation I0 don't need a reverse SDE. This SDE's drift consists the original ODE vector field plus a term proportional the score function, V log (x+).
F.2 EXPRESSING THE SCORE
LEARNED VECTOR FIELD
The SDE in equation 10 is not yet practical for sampling, as the score function V log Pa (xx) is unknown. However, for the specific probability path used in Flow Matching, this score can be expressed entirely in terms of the known vector field en (xx), which is approximated by our trained model ng. The derivation follows similar approach to that for stochastic interpolants (Albergo al., 2023
Our derivation begins with the linear interpolation path: x, = (1 - t)xo + tx,, where Xp ~ N and is data sample. The velocity of this path is defined as the conditional expectation of the instantaneous change:
dbxc
Ue (xx)))
E
(11)
dt
Since the time derivative is
X1 Xo. the vector field becomes:
0e (x,) = E[x, [x,]
[[x]
(12)
A key identity, derived from Tweedie's formula in score-based modeling, connects the marginal score V log (x( to the posterior mean of the initial noise sample Xo:
le [[x
(13)
1 1-t
Our goal thus to express E [[xx| xx] using di (x²). To do this, we take the conditional expectation of the path definition itself:
E x x₂] = (1 t) xx] =E[x] [x]].
(14)
We now have a system of two linear equations equation T2 and equation 14 with two unknowns
(E [E[xo|xx) and E E (x,). We solve this system for E|x, (.). Rearranging equation 12
gives
E[x||x/ = (x.)) + [[xo Substituting this into equation
14
x; (1 tx x + ((x((x)) + E[x (x:))
X; = (1 - t + t
+ (x,.
X; [x tex (x:).
(15)
This gives us the desired expression:
Exxp|x = X2 tw (x:).
(16)
Finally, substituting equation [6] back into the score identity ( (quuaionn 13). arrive the final, practical expression for the score function:
X₁ to (xxe.
v log (x(:)
(17)
This result allows us compute the score any point (xe,t) using only the output of our trained vector field model vy (x,,
F.3 FINAL SDE AND DISCRETIZED TRANSITION PROBABILITY
We now substitute the practical expression for the score equation 17 into the SDE from equation 10 For simplicity, let's assume a constant diffusion G(t)
σ² x;
tu; tx,()))
dx, = 0y (x,)
dt + adW,
(18)
2
1-
21

---

Published as a conference paper at ICLR 2026
This is final SDE, where the drift is expressed entirely in terms of the learned vector field ea (approximated by vg). Unlike some prior work that requires deriving a reverse-time SDE, we can directly use this forward- SDE for model sampling.
To implement this for our RL policy, we discretize equation 18 with step size At:
using the Euler-Maruyamm scheme
o² Xx tua (x t)
X++Ar=
(x(,
At+oVAt
(19)
2
where c ~ I).
This defines the stochastic transition probability of our policy, 70(x++Ar c). is a Gaussian distribution:
no (X++ c) Nxi++ E))
(20)
with mean and covariance given by:
σ² tup t) --t
(x t)
Xx
18 (xe,
At
(21)
Σ 0² I
(22)
This completes the derivation from the deterministic ODE concrete, sampleable stochastic pol- icy for reinforcement learning.
G GRPO ON SDE FLOW MATCHING
With the stochastic policy me defined by the SDE in Appendix optimize our generative model using GRPO (Shao et al., 2024). GRPO is efficient online algorithm and lightweight than PPO (Schulman et al.) 2017), well-suited for large generative models.
The optimization process unfolds over trajectories generated by our SDE-based policy. For each input condition c, we generate a group of G trajectories. Each trajectory consists of T"" discrete steps, obtained by iteratively applying the Euler-Maruyama update from equation 19 to produce final image xi. Our EditScore model then assigns a terminal reward r; to each resulting image.
The GRPO objective maximizes clipped surrogate function over each step of the trajectories:
(O) É, min (0),,,, clip(,, (0), 1-0)),--,,, e, +, +c)4.)
(23)
where is clipping hyperparameter, and the advantage A, is computed based the terminal rewards within the group. For the final step of the trajectory, the advantage is calculated as:
A r; (ecam(fr1......7
(24)
std std(fri,...,raa}}}
For intermediate timesteps (t < T), the advantages are typically computed using Generalized Ad- vantage Estimation (GAE).
Critically, in our SDE framework, the importance ratio pt (+) is the ratio of the single-step state transition probabilities defined equation 20 under the current policy ua and the old policy Te that generated the data:
To Xx Arl A
Pe())=
(25)
Tuw Xx
H IMAGES CATEGORIES
Figure iillustrates the distribution of input image categories in our editing dataset.
QUALITATIVE RESULTS
In this section, present qualitative results of OmniGen2 after reinforcement learning. The Fig- ure showws examples across several image editing tasks. These results validate the improvement of editing outcomes achieved by RI. training editing models.
22

---

Published as a conference paper at ICLR 2026
Figure 4: Distribution of input image categories. Table 13: Benchmark results on EditRewardd-ennt
reporting both overall pairwise accuracy and
fine-grained breakdown across four categories of edit capabilities. Pairwise accuracy measures the proportion of pairs where the model correctly assigns a higher reward score to the human-preferred output. Notably, EditScore achieves superior performance even with its compact 7B size. Avg@4 denotes the average score over 4 forward passes.
Model Metric
GPT-4.1
GPT-5
Gemini-2.5
Qwen2.5-VL
EditScore-7B
EditScore332B
EditScore-72B
Pro
7B
32B
72B
Base
Avg@4
Base
Avg@4
Base Av
PF
0.654
0,750
0.697
0.493
0.490
0.624
0.615
0.735
0.628
0.714
0.676
0.735
Overall
C
0,570
0.687
0,592
0,350
0.402
0.510
0.594
0.701
0.601
0.709
0.610
0,728
0.695
0.748
0.745
0.423
0.661
0.664
0.715
0.670
0.732
0.694
0.751
0,568
0.702
0.631
0.462
0.497
0.611
0.600
0.678
0,590
0.671
0.578
0.676
Subject
C
0.449
0.607
0.441
0,350
0.359
0.419
0.524
0.645
0.499
0.640
0.463
0.617
0
0.666
0.770
0.738
0,356
0,575
0.689
0,703
0.764
0.646
0.746
0,718
0.763
PF
0.587
0.695
0.633
0,397
0.321
0,552
0.511
0.661
0.532
0.632
0.650
0.712
Appear.
0.639
0.763
0.604
0.401
0.481
0.635
0.773
0.656
0.787
0.765
0
0.666
0.709
0.742
0.489
0.507
0.578
0,663
0.696
0.657
0.706
0.652
0.718
PF
0.776
0.855
0.798
0.655
0,571
0.719
0.736
0.890
0.757
0.895
0.826
0.845
Scene
C
0.644
0.750
0.690
0.296
0.494
0.609
0.739
0.749
0.665
0.753
0.654
0.830
0
0.800
0.804
0,803
0.353
0.629
0.756
0.686
0.705
0.735
0.809
0.772
0.823
0.723
0.787
0.758
0.529
0.607
0.660
0.663
0.769
0.687
0.735
0.701
0.747
Advanced
C
0.557
0.642
0,648
0.325
0.411
0.558
0,536
0.649
0.592
0.662
0.659
0.725
0.691
0.742
0.723
0.446
0.632
0.673
0.623
0.700
0.671
0.707
0.676
0.739
J DATA ANNOTATION USER INTERFACE
To ensure the collection of high-quality, fine-grained preference data for EditRewardBench. we developed specialized web-based annotation interface. This section provides a visual overview of the interface and its key features, designed to facilitate our multi-dimensional, tiered ranking protocol. The two main components of the interface are shown in Figure 6
As illustrated in Figure 6 the interface presents expert raters with all necessary information for single annotation task. The design is centered around two core principles of our methodology:
Multi Ditie-iimmniin Evaluation: The lower panel requires annotators to provide three in- dependent rankings for Instruction Following, Consistency, and Overall Quality. This decomposed approach, clearly separated in the UI. ensures that each aspect of the edit is evaluated independently, preventing issues where, for example, visually pleasing but semantically incorrect edit might be unfairly favored.
Tiered Ranking Support: key feature of our design is the direct implementation of tiered ranking. As shown in the lower panel's input fields, annotators not foreed into strict linear order. Instead, they can group multiple images of perceived equal quality into the same tier using a pipe ('I') separator. For instance, a ranking of ||/4|// indicates that output I is in the top tier, outputs 4, 5, and 2 are tied in a middle tier, and output is in the lowest tier. This method more accurately captures nuanced human judgments and allows for the efficient generation of a dense graph of preference pairs from single annotation.
The interface also includes progress tracking and quality control features, ensuring a smooth and reliable annotation workflow for our expert raters.
23

---

Published as a conference paper at ICLR 2026
K DETAILS ON THE EXPERIMENTAL SETUP
This appendix provides detailed specifications for the training of our reward model, EditScore, and for the reinforcement learning of our policy model, OmniGen2-Edit.
K.1 REWARD MODEL. TRAINING (EDITSCORE)
Hyperparameters. Our reward models are fine-tuned from the Qwen2.5-VL series using the MILM SFT framework of LLaMA-Factory (heng et al. 2024). For all model sizes, we use a con- sistent set of hyperparameters. We employ the Adam W optimizer with learning rate of 1.0 x 10-4 The models are trained for epochs with a maximum sequence length of 8192. We use large effective batch size of 32, achieved with a per-device batch size of and 4 gradient accumulation steps across our 32-GPU setup. To ensure efficient training, we utilize LoRA (Hu al. 2022) with rank (r) of 32.
Compute Resources. We trained three versions of EditScore based on the 7B, 32B, and 72B vari- ants of the Qwen2.5-VI VL model. All training conducted on a nigh-perffrrance cluster consisting of 4 interconnected nodes, totaling 32 NVIDIA H100 (80GB) GPUs.
K.2 REINFORCEMENT LEARNING FINE-TUNING
Hyperparameters. For the online RL fine-tuning of OmniGen2-Edie we use the GRPO algo- rithm. The SDE sampling process is configured with T = 20 discrete timesteps and a diffusion coefficient a = 0.9. Key GRPO hyperparameters are fixed across experiments: in one step the global batch size is 288, the group size is G 12. The PPO clipping hyperparameter is flow = 10 Chigh = 5 10 10-4, and the learning rate is 4 * 10 The KL penalty coefficient 8. which regularizes the policy shift from the SFT initialization, is set to 0.04. The policy model is also trained using LoRA with the same configuration as the reward model (r = 32, a = 64).
Compute Resources. The online RL fine-tuning phase, which involves iterative sampling from the policy model and subsequent updates, was performed on a cluster of 32 NVIDIA H100 (80GB) GPUs.
L LIMITATIONS
While EditScore demonstrates robust performance across wide range of editing scenarios, granu- lar analysis reveals specific limitations. Table p4epenntts a detailed breakdown of pairwise accuracy across all 13 subtasks. We observe that while EditScore performs exceptionally well on objective tasks-such as Subject Removal and Text Modification vanced category. Specifically, highly subjective tasks like Portrait Beautification (Beaut) and complex compositional tasks like Hybrid Edit (Hyb) prove to be the most challenging. For instance, the baseline EditScore-78 achieves only 0.284 accuracy on Beaut and 0.519 on Hyb. However, is worth noting that our self-ensemble strategy (Avg @4) significantly mitigates these weaknesses, boosting the accuracy on Beaut to 0.569 and Hyb to 0.571, demonstrating that the model possesses the latent knowledge to evaluate these tasks but requires multiple reasoning paths to reduce variance.
coddifiattoon-perfrraaac performance drops noticeably in the Ad-
Figure further visualizes two representative failure cases that highlight these challenges.
The left case illustrates the difficulty of handling abstract and subjective instructions. Given the prompt "Make the man handsome," " the output image applies a subtle smoothing effect to the skin. However, EditScore fails to perceive this nuanced improvement, penalizing the edit with Semantic Consistency (SC) score of 0/25 by stating "no visible difference." This suggests that EditScore may be overly conservative lack the fine-grained visual acuity required to detect subtle aesthetic enhancements.
The right case demonstrates the limitations in processing compositional instructions. The prompt requires two distinct actions: removing the radiator and changing the cat's color to brown. While the model successfully identifies the removal of the radiator, it fails to rec- ognize that the second constraint not met-the cat appears with severe orange artifacts
24

---

Published as a conference paper at ICLR 2026
rather than natural brown. EditScore grants high score (22/25), indicating a tendency to "over-reward" partial success in complex chains of thought, failing to strictly penalize the unfulfilled portion of the instruction.
We attribute these failure modes to two primary factors. First, the inherent capability of the base VLM:: even state-of-the-art VI.Ms struggle with high-resolution, pixel-level discernibility (needed for beautification). Second, the data distribution: the training data for subjective tasks like beau- tification naturally contains higher human disagreement, making it difficult for the reward model to learn a sharp decision boundary. Furthermore, complex compositional samples are less frequent in current datasets compared to simple single-step edits. Addressing these limitations through the curation of more granular, steti-step instruction data and leveraging stronger base models remains a critical direction for our future work.
25

---

Published as a conference paper at ICLR 2026
Instructions
Input
OmniGen2
OmniGen2 w. RI.
Adjust the background glass wall.
Change the color of clephant to pink
Switch American animation effect.
Add boy wearing headphones operating luptop in front of the computer.
Remove the freight train
Replace the baby puppy.
want change the letters this picce clothing "DIOR".
ALENTOO
DOR
DIOR
Chinge the time to night
Figure 5: Qualitative results on image editing task.
26

---

Published as a conference paper at ICLR 2026
HaPPY COaCH
Taak
Cereet: S031
Tosk Type weo.ggoot.uooggg
twns 3VS Labeled (etp Dra daaoy/M40pi (2044 3225-39-3
Progrox 31M8 (K.09)
Descrption
Inttruction Following Banking
Conoldterey Ronkinn
Inages
coneieeecy stlinage quality
Legga
LARE
al Quality Ranking
Quality lauss
L2945
Suare Sande
Opersior Fectbakk
sedingondde tyou
Previcus
+ Next
Figure 6: Screenshots of our custom-built annotation interface for EditRewardBBechh (Top) The upper panel presents the expert rater with the visual stimuli: the original input image and set of five candidate edited outputs from various models. (Bottom) The lower panel contains the interactive components. It displays the user instruction ("Task Description") and provides three separate input fields for our core evaluation dimensions: Instruction Following, Consistency, and Overall Quality. This panel clearly shows the tiered ranking mechanism, where the pipe symbol ('l") is used to group images of similar quality.
27

---

Published as a conference paper at ICLR 2026
Table 14: Detailed of EditScore across 13 distinct editing subtasks on EditReward-Bents While EditScore performs robustly on objective tasks (e.g., Subject Removal, Text Change), it exhibits limitations in highly subjective tasks like Portrait Beautification (Beaut) and complex composi- tional tasks like Hybrid Edit (Hyb). Abbreviations: Add: Subject Addition, Rmv: Removal, Rep: Replace; Col: Color Alt., Mat: Material Mod., Sty: Style Transfer, Tone: Tone Trans.; BG: Back-
ground Change, Ext: Extract; Beaut: Portrait Beautification (ps human), Text: Text Mod., Mot:
Motion Change, Hyb: Hybrid Edit (compose).
Subject
Appearance
Scene
Advanced
Model
Avg.
Add
Rmv
Rep
Col
Mat
Sty
Tone
BG
Fxt
Beaut
Text
Mot
Hyh
EditScore- (Base)
0.681
0.658
0.718
0.284
0.578
EditScore- 7B (Avg@44)
0.708
0.833
0.753
0.725
0.674
0.703
0.755
0.758
0.791
0.569
0.858
0.733
0.571
0.727
EditScore- 720 (Buse)
0.646
0.741
0.776
0.715
0.651
0.667
0.745
0.750
0.837
0.358
0.877
0.689
0.675
0.703
EditScore- (Avg@4)
0.743
0.824
0.700
0.744
0.703
0.766
0.505
0.744
0.766
"Make the man handsome."
"Remove the radiator the right side of the image, and change the color of the black and white cat to brown."
Overall score: 0/25
Overall Score: 10.48/25
SC: There is visible difference between the original and the edited image... Score: 0/25
SC: The radiator longer visible, and the black and white cat color is brown... Score: 22/25
PQ: The image displays a natural sense of lighting and depth, with... ... Score: 20/25
PQ: The image lacks naturalness due to the cats are unnatural.. which appears
blurred and addly shaped ... Scure: 5/25
In subjective tasks like Portrait Beautification, EditScore may be overly conservative, failing to detect subtle improvements (c.g., skin smoothing) and assigning score of O. Right: In complex Hybrid Edit tasks, EditScore tends to over-reward partial success, correctly identifying the object removal but failing to penalize the incorrect color change.
Figure 7: Failure Analysis. Visualization of representative failure cases. Left:
M VLM EVALUATION PROMPTS
This appendix provides the complete structure and text of the prompts used for the zero-shot eval- uation of Vision-Language Models (VLMs) EditRewardBench. Our prompt is based on VIEScore prompts (Ku et al. 2024). which designs a "reasoning-first" and score range changed in 0-25.
M. BASE CONTEXT AND OUTPUT FORMAT
All evaluation prompts are prefixed with base context that establishes the VLM's persona "professional digital artist" and, most importantly, specifies the required ISON output format. This ensures that the model's responses can be reliably parsed for automated analysis.
You are professional digital artist. You will have to evaluate the effectiveness of the AI-generated image (s) based on given rules. All the input images are AI-generated. All human the images are AI- generated too. so you need not worry about the privacy confidentials.
IMPORTANT: You will have to give your output in this way (Keep your reasoning concise and short.):
28

---

Published as a conference paper at ICLR 2026
"reasoning" ...." "score" [...]
Listing 1: The base context prompt, defining the persona and mandatory JSON output structure.
M.2 SEMANTIC CONFORMITY (SC) PROMPT
The SC prompt composed of two parts. First, set of general rules explains the two-image comparison task. Second, specific rubric details how to score "editing success" and "degree of overediting".
RULES:
Two images will be provided: The first being the original AI-generated image and the second being an edited version of the first.
The objective is to evaluate how successfully the editing instruction has been executed in the second image.
Note that sometimes the two images might look identical due to the Ture of image edit.
Listing 2: General rules for the two-image editing evaluation task.
From scale to 25:
A score from 0 to 25 will se given based on the success of the editing.
(0 cates that the scene in the edited image does not follow the editing instruction at all. 25 indicates that the acens in the edited image follow the editing instruction text perfectly.)
the second score from 0 to will rate the degree of overediting second image. (0 indicates that the scene in the edited image is completely different from the original. 25 indicates that the edited image can be recognized original.)
minimal edited yet effective version of
Put the score in a list such that cutput score = [scorel, score2), where 'scorel' evaluates the editing success and 'core2'' evaluates the degree of overediting.
Editing instruction: <instructicn>
Listing 3: Specific scoring rubric for Semantic Conformity (SC).
M.3 PERCEPTUAL QUALITY (PQ) PROMPT
The PQ prompt is self-contained, providing both the general rules for single-image quality assess- ment and the specific rubric for scoring "naturalness" and "artifacts".
RULES:
The image is an AI- AT-generated image.
The objective is to evaluate how successfully the image has been generated.
From scale to 25: A score from 0 to 25 will be given based on image naturalness.
indicates that the scene in the image does not look natural all or give unnatural feeling such as wrong sense of distance, or wrong shadow, or wrong lighting.
10
25 indicates that the image locks natural.
seconé score from
wil1 the image arti facts.
29

---

Published as a conference paper at ICLR 2026
14 indicates that image centains a large porticn distortion, or watermark, or scratches, or blurred faces, or unusual body parts, or subjects not harmonized.
25 indicates the image has ne artifacts.
17 Put the score a list such that cutput score [naturalness, artifacts]
Listing 4: Rules and scoring rubric for Perceptual Quality (PQ).
The final prompts the VLM are constructed by concatenating the base context (Listing I with the respective rule and rubric sets for the SC (Listings and and PQ (Listing 4 evaluations. This modular design allows for clear and targeted assessment of each evaluation dimension.
30


Reviews:
[{"review_id": "Y8nt61blVF", "reviewer": "Reviewer_bWV1", "summary": "The paper tackles the missing piece for online RL in instruction-guided image editing by building a high-fidelity, domain-specialized reward model. It first introduces EditReward-Bench, a 3,072-pair human-rated benchmark that scores edits on prompt following, consistency, and overall quality. Guided by this benchmark, the authors train EditScore reward models at multiple scales (7B-72B) and use a simple self-ensemble at inference to boost accuracy, matching or surpassing proprietary VLMs on the benchmark. Finally, they show that EditScore unlocks stable and effective online RL, improving strong editors like OmniGen2 and outperforming open VLMs both for Best-of-N selection and policy optimization.", "questions": "Benchmark: How were prompts/edit types sampled and how representative are they?\nRaters: What was raw disagreement and how do results change without consensus filtering?\nReproducibility: Will you release frozen candidate sets or open substitutes for proprietary outputs?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper is original in framing instruction-guided image editing as an online RL problem that is unlocked by a domain-specialized reward model and a purpose-built benchmark, plus a simple self-ensemble that improves reward fidelity. Experimental quality is strong, with a clear scaling study, competitive baselines for Best-of-N selection and policy optimization, and consistent gains on prompt following, consistency, and visual quality. Clarity is high: the problem setting, evaluation axes, and training pipeline are well motivated and easy to follow, and the empirical claims map cleanly to the stated goals. The work is significant because it removes a key barrier to applying online RL in generative editing, yielding practical improvements on strong editors and delivering reusable infrastructure that can benefit future systems and studies.", "weaknesses": "Despite solid results, the work’s evidence base and scope feel narrower than the claims. The benchmark is small and curated for perfect agreement, which can hide genuine preference diversity and inflate metrics; include disagreement analyses, release non-consensus pairs, and recruit a broader rater pool. Reproducibility is brittle because the candidate pool depends on proprietary editors; ship frozen artifacts or open substitutes so others can rerun everything. Generality is underexplored since online RL is shown mainly on one editor with one algorithm; add ablations across multiple editors, RL methods, and hyperparameters, plus stress tests on out-of-distribution edit types. The self-ensemble that yields top reward accuracy likely raises cost and latency; report cost-normalized accuracy, wall-clock time, and memory, and compare against single-model baselines at equal budgets. Finally, claims about reward variance aiding optimization are intriguing but not causally established; isolate variance from annotator accuracy with controlled studies and replicate the effect on additional datasets and editors.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "drxUNtNJl4", "reviewer": "Reviewer_g3mS", "summary": "The paper tackles a critical bottleneck in applying online reinforcement learning (RL) to instruction-guided image editing: the lack of a high-fidelity, efficient reward signal. The authors argue that existing proprietary VLMs are too costly for online RL , while large open-source VLMs are not accurate enough, leading to training instability. To this end, the paper presents a comprehensive, three-part solution:\n1. This paper introduces EditReward-Bench, a new, meticulously annotated benchmark for evaluating image editing reward models (RMs). \n2. Guided by this benchmark, they develop EditScore, a series of specialized generative reward models (7B-72B) that achieve SOTA accuracy, even surpassing proprietary models like GPT-5. \n3. This paper provides the first successful demonstration that a specialized RM (EditScore) can unlock stable and effective online RL training for image editing, whereas general-purpose VLMs fail. The final RL-tuned policy shows substantial performance gains on standard editing benchmarks.", "questions": "None", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "* The authors constructed a benchmark for the image editing domain to evaluate Edit Reward Models. Based on this benchmark, they built a series of editing Reward Models, which were then used with Flow-GRPO to enhance the editing capabilities of the base model, OmniGen2.\n\n* The authors commit to releasing the proposed benchmark, the weights of the Reward Models, and the weights of the optimized OmniGen2 model. This will provide definite value to the community.\n\n* The tricks proposed in the methodology, such as self-ensemble, Reasoning-First, and using an RM with higher reward variance, are reasonable and lead to substantial performance improvements.", "weaknesses": "* I believe the experiments need improvement. The paper currently only reports experiments on OmniGen2. It remains unclear whether the method is effective for other editing models of different sizes and types, such as the simpler InstructP2P (SD1.5-based), the Unified Model-based Bagel, or even training a LoRA for the larger diffusion-based Step1X-Edit.\n\n* The methodological innovation is limited. The Flow-GRPO paper [1] already demonstrated that its method can be used to improve editing models. The concept that high-variance RMs are preferable to high-accuracy RMs has also been proposed in prior work [2]. Furthermore, self-ensemble and Reasoning-First are common techniques in reward model training.\n\n* The authors state they constructed \"70,000 data samples for training the reward model and 60,000 samples for reinforcement learning training,\" but they do not plan to open-source this data. This makes it difficult for the community to fully reproduce the paper's results. Although the benchmark and model weights will be released, this lack of training data makes the contribution somewhat less exciting to me.\n\n[1] Flow-grpo: Training flow matching models via online rl, NeurIPS 2025\n\n[2] What makes a reward model a good teacher? an optimization perspective, arXiv 2025", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "7Yg2Z9ZYvA", "reviewer": "Reviewer_rtMd", "summary": "This paper addresses a critical bottleneck in applying reinforcement learning to image editing: the lack of a reliable and efficient reward model. The authors introduce **EditReward-Bench**, a comprehensive benchmark for evaluating reward models in instruction-guided image editing, and **EditScore**, a family of open-source reward models that outperform existing open-source and even some proprietary VLMs. Through extensive experiments, they demonstrate the utility of EditScore in both Best-of-N selection and online RL training, achieving stable and significant improvements over strong baselines.", "questions": "1. The training data for EditScore is constructed using a pipeline that involves generating instructions and candidate outputs from a pool of 11 editing models, including state-of-the-art proprietary ones. Could you please clarify the measures taken to ensure that **EditReward-Bench's test set does not contain images or near-identical instructions from the training data**? Given the overlap in the model pool and data sources (e.g., GEdit-Bench), a detailed description of the train/test split protocol is crucial to validate the benchmark's integrity and prevent overestimation of generalization performance.\n2. The paper states that EditReward-Bench annotations \"have passed rigorous inter-annotator agreement checks.\" For transparency and reproducibility, could you provide the **specific agreement metrics used (e.g., Kendall's Tau, Spearman correlation, or percentage agreement)** and their quantitative results? Furthermore, please detail the **number of annotators per sample** and the process for resolving disagreements to achieve the final consensus ranking.\n3. The performance improvements presented in Tables 2 and 5, while substantial, are reported as point estimates. Have you conducted tests for **statistical significance** (e.g., paired bootstrap tests or t-tests over the benchmark subsets) to ensure that the gains from EditScore and the RL fine-tuning are not due to random chance? Reporting p-values or confidence intervals would strengthen the claims.\n4. The paper excellently demonstrates EditScore's strengths. However, a thorough analysis of its **failure modes and limitations** is equally important. Could you provide examples or a discussion of edit types or instructions where EditScore consistently fails or provides unreliable rewards? For instance, does it struggle with highly abstract or compositional instructions (\"make it look more joyful\") ? A dedicated \"Limitations\" section would be valuable.\n5. The self-ensembling (Avg@4) strategy is shown to be highly effective. However, it incurs a 4x computational overhead during inference. Could you provide a more detailed analysis of this **trade-off between performance and efficiency**, such as the actual increase in inference latency? This is critical for practitioners considering the deployment of EditScore in resource-constrained environments.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "**Novel Contribution:** The paper makes a clear and timely contribution by addressing the underexplored problem of reward modeling for image editing. The release of both the benchmark and the model fills a gap in the community.\n\n- **Rigorous Benchmarking:** EditReward-Bench is well-designed, covering 13 diverse editing tasks and 11 editing models, with human-annotated multi-dimensional evaluations.\n- **Strong Empirical Results:** EditScore consistently outperforms open-source VLMs and even matches or surpasses proprietary models like GPT-4.1 in some settings. The self-ensembling strategy is simple yet effective.\n- **Practical Utility:** The paper convincingly demonstrates the real-world impact of EditScore through Best-of-\\(N\\) selection and online RL experiments, showing stable training and performance gains.\n- **Insightful Analysis:** The study on reward variance vs. annotator accuracy is a valuable finding that could influence future reward modeling efforts.", "weaknesses": "- **Limited RL Scope:** The RL experiments are conducted only on OmniGen2. While justified, it would be beneficial to see how EditScore generalizes to other base models.\n- **Computational Cost:** The self-ensembling strategy, while effective, increases inference cost. A more detailed discussion on the trade-off between performance and efficiency would be helpful.\n- **Comparison with More Baselines:** While the comparison with Qwen2.5-VL is thorough, it would be useful to see how EditScore compares to other recent open-source reward models (e.g., those from HPSv3 or ImageReward).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Ue2Oszsuvx", "reviewer": "Reviewer_tH54", "summary": "This paper introduces EditReward-Bench (a benchmark for reward models) and EditScore (a family of generative reward models) for image editing. The paper first collects human preference and ranking data from annotators based on candidate images and asked to rank them based on certain qualities. This data is then used to train generative reward models, which are compared for agreeableness to the human rankers.", "questions": "- how are table 2 scores computed exactly?\n- how are the aggregation scores computed?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 4, "strengths": "- this paper introduces a solid dataset and reward model for RL training.\n- this paper is decently written, but some sections (eg, explaining how the scores of table 2 of EditReward-Bench) could be rewritten. Otherwise, it's clear.\n- the methodology for data collection seems sound.", "weaknesses": "- I am concerned about the use of candidate images from current models as part of the data labeling process. Indeed, if there is an area where all the candiate models struggle, although we would be able to provide a relative ranking, the maximal reward would only be obtained from the model generalizing, not any supervision.\n- section 6.4.1 compares two different models for optimization, and as OmineGen2 was chosen for high pass@k, a better comparison would be using the same base model. It is hard to tell if this truly is the result of a better reward model, or a more amenable base model.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)