RealUnify: Do Unified Models Truly Benefit from Unification? A
  Comprehensive Benchmark
Authors: name: Yang Shi; name: Yuhao Dong; name: Yue Ding; name: Yuran Wang; name: Xuanyu Zhu; name: Sheng Zhou; name: Wenting Liu; name: Haochen Tian; name: Rundong Wang; name: Huanqian Wang; name: Zuyan Liu; name: Bohan Zeng; name: Ruizhe Chen; name: Qixun Wang; name: Zhuoran Zhang; name: Xinlong Chen; name: Chengzhuo Tong; name: Bozhou Li; name: Chaoyou Fu; name: Qiang Liu; name: Haotian Wang; name: Wenjing Yang; name: Yuanxing Zhang; name: Pengfei Wan; name: Yi-Fan Zhang; name: Ziwei Liu
Published: 2025-09-29 15:07:28

Abstract: The integration of visual understanding and generation into unified
multimodal models represents a significant stride toward general-purpose AI.
However, a fundamental question remains unanswered by existing benchmarks: does
this architectural unification actually enable synergetic interaction between
the constituent capabilities? Existing evaluation paradigms, which primarily
assess understanding and generation in isolation, are insufficient for
determining whether a unified model can leverage its understanding to enhance
its generation, or use generative simulation to facilitate deeper
comprehension. To address this critical gap, we introduce RealUnify, a
benchmark specifically designed to evaluate bidirectional capability synergy.
RealUnify comprises 1,000 meticulously human-annotated instances spanning 10
categories and 32 subtasks. It is structured around two core axes: 1)
Understanding Enhances Generation, which requires reasoning (e.g., commonsense,
logic) to guide image generation, and 2) Generation Enhances Understanding,
which necessitates mental simulation or reconstruction (e.g., of transformed or
disordered visual inputs) to solve reasoning tasks. A key contribution is our
dual-evaluation protocol, which combines direct end-to-end assessment with a
diagnostic stepwise evaluation that decomposes tasks into distinct
understanding and generation phases. This protocol allows us to precisely
discern whether performance bottlenecks stem from deficiencies in core
abilities or from a failure to integrate them. Through large-scale evaluations
of 12 leading unified models and 6 specialized baselines, we find that current
unified models still struggle to achieve effective synergy, indicating that
architectural unification alone is insufficient. These results highlight the
need for new training strategies and inductive biases to fully unlock the
potential of unified modeling.