UniREditBench: A Unified Reasoning-based Image Editing Benchmark
Authors: name: Feng Han, name: Yibin Wang, name: Chenglin Li, name: Zheming Liang, name: Dianyi Wang, name: Yang Jiao, name: Zhipeng Wei, name: Chao Gong, name: Cheng Jin, name: Jingjing Chen, name: Jiaqi Wang
Organization: Fudan University

Recent advances in multi-modal generative models have driven substantial
improvements in image editing. However, current generative models still
struggle with handling diverse and complex image editing tasks that require
implicit reasoning, underscoring the need for a comprehensive benchmark to
systematically assess their performance across various reasoning scenarios.
Existing benchmarks primarily focus on single-object attribute transformation
in realistic scenarios, which, while effective, encounter two key challenges:
(1) they largely overlook multi-object interactions as well as game-world
scenarios that involve human-defined rules, which are common in real-life
applications; (2) they only rely on textual references to evaluate the
generated images, potentially leading to systematic misjudgments, especially in
complex reasoning scenarios. To this end, this work proposes UniREditBench, a
unified benchmark for reasoning-based image editing evaluation. It comprises
2,700 meticulously curated samples, covering both real- and game-world
scenarios across 8 primary dimensions and 18 sub-dimensions. To improve
evaluation reliability, we introduce multimodal dual-reference evaluation,
providing both textual and ground-truth image references for each sample
assessment. Furthermore, we design an automated multi-scenario data synthesis
pipeline and construct UniREdit-Data-100K, a large-scale synthetic dataset with
high-quality chain-of-thought (CoT) reasoning annotations. We fine-tune Bagel
on this dataset and develop UniREdit-Bagel, demonstrating substantial
improvements in both in-domain and out-of-distribution settings. Through
thorough benchmarking of both open-source and closed-source image editing
models, we reveal their strengths and weaknesses across various aspects.