EditScore: Unlocking Online RL for Image Editing via High-Fidelity
  Reward Modeling
Authors: name: Xin Luo; name: Jiahao Wang; name: Chenyuan Wu; name: Shitao Xiao; name: Xiyan Jiang; name: Defu Lian; name: Jiajun Zhang; name: Dong Liu; name: Zheng liu
Published: 2025-09-28 14:28:24

Abstract: Instruction-guided image editing has achieved remarkable progress, yet
current models still face challenges with complex instructions and often
require multiple samples to produce a desired result. Reinforcement Learning
(RL) offers a promising solution, but its adoption in image editing has been
severely hindered by the lack of a high-fidelity, efficient reward signal. In
this work, we present a comprehensive methodology to overcome this barrier,
centered on the development of a state-of-the-art, specialized reward model. We
first introduce EditReward-Bench, a comprehensive benchmark to systematically
evaluate reward models on editing quality. Building on this benchmark, we
develop EditScore, a series of reward models (7B-72B) for evaluating the
quality of instruction-guided image editing. Through meticulous data curation
and filtering, EditScore effectively matches the performance of learning
proprietary VLMs. Furthermore, coupled with an effective self-ensemble strategy
tailored for the generative nature of EditScore, our largest variant even
surpasses GPT-5 in the benchmark. We then demonstrate that a high-fidelity
reward model is the key to unlocking online RL for image editing. Our
experiments show that, while even the largest open-source VLMs fail to provide
an effective learning signal, EditScore enables efficient and robust policy
optimization. Applying our framework to a strong base model, OmniGen2, results
in a final model that shows a substantial and consistent performance uplift.
Overall, this work provides the first systematic path from benchmarking to
reward modeling to RL training in image editing, showing that a high-fidelity,
domain-specialized reward model is the key to unlocking the full potential of
RL in this domain.