Easier Painting Than Thinking: Can Text-to-Image Models Set the Stage,
  but Not Direct the Play?
Authors: name: Ouxiang Li, name: Yuan Wang, name: Xinting Hu, name: Huijuan Huang, name: Rui Chen, name: Jiarong Ou, name: Xin Tao, name: Pengfei Wan, name: Fuli Feng

Text-to-image (T2I) generation aims to synthesize images from textual
prompts, which jointly specify what must be shown and imply what can be
inferred, thereby corresponding to two core capabilities: composition and
reasoning. However, with the emerging advances of T2I models in reasoning
beyond composition, existing benchmarks reveal clear limitations in providing
comprehensive evaluations across and within these capabilities. Meanwhile,
these advances also enable models to handle more complex prompts, whereas
current benchmarks remain limited to low scene density and simplified
one-to-one reasoning. To address these limitations, we propose T2I-CoReBench, a
comprehensive and complex benchmark that evaluates both composition and
reasoning capabilities of T2I models. To ensure comprehensiveness, we structure
composition around scene graph elements (instance, attribute, and relation) and
reasoning around the philosophical framework of inference (deductive,
inductive, and abductive), formulating a 12-dimensional evaluation taxonomy. To
increase complexity, driven by the inherent complexities of real-world
scenarios, we curate each prompt with high compositional density for
composition and multi-step inference for reasoning. We also pair each prompt
with a checklist that specifies individual yes/no questions to assess each
intended element independently to facilitate fine-grained and reliable
evaluation. In statistics, our benchmark comprises 1,080 challenging prompts
and around 13,500 checklist questions. Experiments across 27 current T2I models
reveal that their composition capability still remains limited in complex
high-density scenarios, while the reasoning capability lags even further behind
as a critical bottleneck, with all models struggling to infer implicit elements
from prompts. Our project page: https://t2i-corebench.github.io/.