UniGenBench++: A Unified Semantic Evaluation Benchmark for Text-to-Image
  Generation
Authors: name: Yibin Wang; name: Zhimin Li; name: Yuhang Zang; name: Jiazi Bu; name: Yujie Zhou; name: Yi Xin; name: Junjun He; name: Chunyu Wang; name: Qinglin Lu; name: Cheng Jin; name: Jiaqi Wang
Published: 2025-10-21 14:56:46

Abstract: Recent progress in text-to-image (T2I) generation underscores the importance
of reliable benchmarks in evaluating how accurately generated images reflect
the semantics of their textual prompt. However, (1) existing benchmarks lack
the diversity of prompt scenarios and multilingual support, both essential for
real-world applicability; (2) they offer only coarse evaluations across primary
dimensions, covering a narrow range of sub-dimensions, and fall short in
fine-grained sub-dimension assessment. To address these limitations, we
introduce UniGenBench++, a unified semantic assessment benchmark for T2I
generation. Specifically, it comprises 600 prompts organized hierarchically to
ensure both coverage and efficiency: (1) spans across diverse real-world
scenarios, i.e., 5 main prompt themes and 20 subthemes; (2) comprehensively
probes T2I models' semantic consistency over 10 primary and 27 sub evaluation
criteria, with each prompt assessing multiple testpoints. To rigorously assess
model robustness to variations in language and prompt length, we provide both
English and Chinese versions of each prompt in short and long forms. Leveraging
the general world knowledge and fine-grained image understanding capabilities
of a closed-source Multi-modal Large Language Model (MLLM), i.e.,
Gemini-2.5-Pro, an effective pipeline is developed for reliable benchmark
construction and streamlined model assessment. Moreover, to further facilitate
community use, we train a robust evaluation model that enables offline
assessment of T2I model outputs. Through comprehensive benchmarking of both
open- and closed-sourced T2I models, we systematically reveal their strengths
and weaknesses across various aspects.