Easier Painting Than Thinking: Can Text-to-Image Models Set the Stage, but Not Direct the Play?
Authors: Ouxiang Li
Yuan Wang
Xinting Hu
Huijuan Huang
Rui Chen
Jiarong Ou
Xin Tao
Pengfei Wan
Xiaojuan Qi
Fuli Feng
Venue: ICLR 2026 Poster (2026)
Keywords: Text-to-Image Generation
Reasoning
Benchmark
Primary area: datasets and benchmarks

Abstract: Text-to-image (T2I) generation aims to synthesize images from textual prompts, which jointly specify what must be shown and imply what can be inferred, which thus correspond to two core capabilities: ***composition*** and ***reasoning***. Despite recent advances of T2I models in both composition and reasoning, existing benchmarks remain limited in evaluation. They not only fail to provide comprehensive coverage across and within both capabilities, but also largely restrict evaluation to low scene density and simple one-to-one reasoning. To address these limitations, we propose **T2I-CoReBench**, a comprehensive and complex benchmark that evaluates both composition and reasoning capabilities of T2I models. To ensure comprehensiveness, we structure composition around scene graph elements (*instance*, *attribute*, and *relation*) and reasoning around the philosophical framework of inference (*deductive*, *inductive*, and *abductive*), formulating a 12-dimensional evaluation taxonomy. To increase complexity, driven by the inherent real-world complexities, we curate each prompt with higher compositional density for composition and greater reasoning intensity for reasoning. To facilitate fine-grained and reliable evaluation, we also pair each evaluation prompt with a checklist that specifies individual *yes/no* questions to assess each intended element independently. In statistics, our benchmark comprises 1,080 challenging prompts and around 13,500 checklist questions. Experiments across 38 current T2I models reveal that their composition capability still remains limited in high compositional scenarios, while the reasoning capability lags even further behind as a critical bottleneck, with all models struggling to infer implicit elements from prompts.

Published as a conference paper at ICLR 2026
EASIER PAINTING THAN THINKING: CAN TEXT-TO- IMAGE MODELS SET THE STAGE, BUT NOT DIRECT THE PLAY?
Ouxiang Li Yuan Wang', Xinting Xinting Hu Huijuan Huang"; Rui Chen", Jiarong Ou?, Xin Tao?t Pengfei Wan Xiaojuan Qi Fuli Feng'
University of Science and Technology of China, *Kling Team, Kuaishou Technology. "The University of Hong Kong i Loxxmmail 1. usta.edu.com
2oyhul412@gmaiii com, jiangsutx gma com
Homepage
Leaderboard
Code
Benchmark
Dataset
Aeceectied the Rrownin
Froept kathin forurie
attipeoo deggs
daake and
Pooege bailiag codere linken Aler
baat el prositently poaka coLEAI
shite bu Heweeer berr
Made Sectea
wher watlee ceneiong ond hanging showe Iustances)
Abductive
(35 Instaaces)
6/6 Lso
20/33 a
Composition
Easy!
Hard!
Inductivnn
616 1,00
28/35 = eg
Ressnning
(b) Composition: DPG- Bench
(c) Composition: Boe-CoRe (Ouns
Year aodenn
Poospe itamma iappaed to
Poougt: Gevenite the neude arge temato being squnceed sightly is fir.
side will pey Ind in Eace lakey. callen feal and The take repe snspeet mnteritg whetg resting the rhemmu to directy the transway take beloy
Behavioe Outcome)
(1 Behavier Outenmee)
Deductive
(PP)
Belwering (868)
Belestard
Fair!
Hard!
5/8 0,675
(3) Evaluation Taxomomy
(d) Reasoning:
R2I-Bench
(e) Reassning: 121-CoReBench (Ours)
Figure 1: Overview of our T21 TZI-COREBENCH. (a) Our benchmark comprehensively covers two fundamental T2I capabilities (i.e., composition and reasoning), further refined into 12 dimensions. (b-e) Our benchmark poses greater challenges to advanced T21 models, with higher compositional density than DPG-Bencl (Hu et al., 2024) and greater reasoning intensity than R2I- (Chen et al., 2025b), enabling clearer performance differentiation across models under real-world com- plexities. Each image is scored based on the ratio of correctly generated elements.
ABSTRACT
Text-to-image (T2I) generation aims to synthesize images from textual prompts, which jointly specify what must be shown and imply what can be inferred, which thus correspond to two core capabilities: composition and reasoning. Despite re- cent advances of T2I models in both composition and reasoning, existing bench- marks remain limited in evaluation. They not only fail to provide comprehensive coverage across and within both capabilities, but also largely restrict evaluation low scene density and simple one-to-one reasoning. To address these limitations, we propose T22-CORREBENC evaluates both composition and reasoning capabilities of T2I models. To ensure comprehensiveness, we structure composition around scene graph elements (in- stance, attribute, and relation) and reasoning around the philosophical framework of inference (deductive, inductive, and abductive), formulating a 12-dimensional evaluation taxonomy. To increase complexity, driven by the inherent roal-world complexities, we curate each prompt with higher compositional density for com- position and greater reasoning intensity for reasoning. To facilitate fine-grained and reliable evaluation, we also pair each evaluation prompt with a checklist that
comprehensive and complex benchmark that
Work done during internship at Kling Team, Kuaishou Technology. Cooresponding authors. Project lead.

---

Published as a conference paper at ICLR 2026
specifies individual yes/no questions to assess each intended element indepen- dently. In statistics, our benchmark comprises 1, 080 challenging prompts and around 13, 500 checklist questions. Experiments across 38 current T2I models re- veal that their composition capability still remains limited in high compositional scenarios, while the reasoning capability lags even further behind critical bot- tleneck, with all models struggling to infer implicit elements from prompts.
INTRODUCTION
Recent developments in text-to-image (T2I) generative models are advancing toward high-quality image generation that adheres to user instructions. In real-world applications, textual prompts are usually concise yet underspecified (Hutchinson et al., 2022; Zhong et al., 2023), conveying not only explicit descriptions about what must be depicted, but also implicit contextual cues for generating coherent and plausible images. These correspond to two fundamental capabilities required for faith- ful T2I generation: composition and reasoning. As shown in Fig. 1, composition aims to correctly generate all explicit visual elements in the prompt, including instances (e.g., lomato), attributes (e.g.. wooden), and relations (e.g., next to); reasoning aims generate visual elements implicitly inferred from the prompt (e.g., ripe tomato is squeezed tightly in fist + the tomato juice bursts out).
Predominant T2I models, primarily based on diffusion (Ho et al., 2020; Ho & Salimans, 2022; Peebles& & Xie, 2023) and autoregressive paradigms (Sun et al., 2024; Li et al., 2024b), demonstrate strong performance on simple compositional tasks (Huang et al., 2023a: Ghosh et al., 2023) but still struggle with complex compositional tasks involving multiple visual elements (Hu et al., 2024; Wu et al., 2024) as well as reasoning tasks (Niu et al., 2025; Chen et al., 2025b). Recently, T2I models multimodal LLMs (MLLMs) (Chameleon, 2024;
enhanced with large language models (LLMs) Xie et al., 2024; Deng et al., 2025a; Wu et al., 2025a; Xu et al., 2025a;b) have emerged, which offer stronger text modeling and cross-modal alignment. This paradigm brings new expectations to handle more complex scenarios involving high compositional density and reasoning intensity.
Given these developments and challenges, is increasingly important to establish fair and holistic evaluation of T21 models that systematically assesses both composition and reasoning capabilities. Early efforts (Huang et al., 2023a; Ghosh al., 2023; Li et al., 2024a) focus on evaluating basic composition capabilities with a limited number of visual elements. Subsequent benchmarks further extend the number of visual elements in composition (see Fig. 1 (b)) (Hu et al., 2024; Wu et al., 2024; Zhou et al., 2025) and evaluate certain reasoning capabilities (e.g., behavioral reasoning in Fig. (d)) (Fu et al., 2024; Niu al., 2025; Chen et al., 2025b). These existing benchmarks exhibit two limitations. (1) Lack of comprehensiveness: Most benchmarks focus on either composition or reasoning in isolation, and their underlying taxonomies are largely heuristic, which prevents them from systematically capturing all relevant evaluation dimensions. (2) Lack of complexity: While some benchmarks increase the number of visual elements in composition, they remain limited to low scene density and fail to reflect the compositional complexity of real-world applications (e.g., generate a bustling modern kitchen in Fig. (c)). More importantly, current caasoning-orientee benchmarks mainly target single-step inference (e.g., one behavior -> one outcome), thus overlook- ing the multi-step causal chains inherent to real-world scenarios (see Fig. 1 (e)).
To address the above limitations, we introduce T2I-COREBENCH, a Composition and Reasoning Benchmark for systematic evaluation of T2I models. To ensure comprehensiveness, as illustrated in Fig. I (a), our taxonomy jointly covers composition and reasoning. For composition, we follow the scene graph structure (Johnson et al., 2015; Chang et al., 2021) and define three basic dimensions to fully depict a compositional scene: instance, attribute, and relation. We also include text rendering to capture the unique challenges of generating texts with precise content and layout. For reasoning, we adopt tripartite framework of deductive, inductive, and abductive reasoning, as well-established in philosophical literature (Peirce, 1934; Zalta et al., 2003; Godfrey-Smith, 2009), and refine it into eight dimensions tailored to T2I scenarios. To increase complexity, as summarized in Table 1, we design each dimension with higher compositional density and increased reasoning difficulties com- pared with existing benchmarks. For composition, we increase the number of visual elements (~ 20 per prompt) to simulate semantically dense scenarios. For reasoning, complexity is introduced along one-to-mant mo-y (i.e., one behavior > multiple outcomes) and many-to-one (e.g., multiple premises > one conclusion) inferences, reflecting the intricate reasoning patterns in real-world applications.
2

---

Published as a conference paper at ICLR 2026
Table 1: T21 benchmark comparison. Our T2I-COREBENCH comprchensively covers 12 evalu- ation dimensions spanning both composition ( (M Multi-Instannce MA Multi-Atrribute, MR Multi- Relation, TR Text Rendering) and reasoning ( LR Logical Reasoning, BR Behavioral Reasoning, HR Hypothetical Reasoning, PR Procedural Reasoning, GR Generalization Reasoning, AR Ana- logical Reasoning, CR Commonsense Reasoning, and RR Reconstructive Reasoning). The symbols denote different coverage levels: indicates high compositional (visual elements 5) reasoning (one-to-many or many-to-one inference) complexity, O indicates simple settings (visual elements < one-to-one inference), and O indicates coverage.
Reasoning
Composition
Benchmark
Deductive
Inductive
Abductive
MI
MA
MR
TR
L.R
BR
HR
PR
GR
AR
CR
RR
T21-CompBench (Huang al., 2023a)
C
O
O
100
O
GenEval (Ghosh et al., 2023)
0
0
GenAT-Bench (Li et al., 2024a)
0
0
DPG-Bench (Hu et al., 2024)
0
0
0
0
0
0
0
ConcepiMix (Wu al., 2024)
0
0
O
0
0
0
0
TlIF-Bench (Wei et al., 2025)
0
0
LongBench- (Zhou et al., 2025)
PRISM-Bench (Fang al., 2025)
UniGenBench (Wang et al., 2025)
Commonsense- T2T (Fu al., 2024)
PhyBench (Meng et al., 2024)
WISE (Niu et al., 2025)
T2I-Reasone BencB (Sun al., 2025)
R21-Bench (Chen et al., 2025b)
OneIG- Gennch (Chang et al., 2025)
T21-COREBENCH (Ours)
To enable fine-grained and reliable evaluation, we pair each textual prompt with a checklist of in- dependent yes/no questions, assessing whether the generated image faithfully captures both explicit and implicit visual elements. The generated images are then evaluated against these checklists by Gemini 2.5 Flash (Gemini Team, 2023), an MLLM evaluator selected for its strong alignment with human judgments and efficiency scale. In total, T2I-CORFBENCH encompasses 12 well-defined dimensions, with 1, 080 challenging prompts and approximately 13, 500 checklist questions. In ex- periments, benchmark 38 current T2I models across architectures and scales, including diffusion models, autoregressive models, and unified models. Our study shows that composition capability in T2I generation steadily improving, with open-source models gradually narrowing the gap with closed-source counterparts, whereas the overall performance remains inadequate in high composi- tional scenarios. Most notably, reasoning capability lags significantly behind: even the state-of-the- art (SOTA) models fail to reliably infer implicit visual elements from prompts, making reasoning the central bottleneck for advancing T21 generation. Our contributions can be concluded as follows:
We introduce T2I-COREBENT and complexity in T2I evaluation, covering both composition and reasoning capabilities through 1, 080 challenging prompts across 12 dimensions.
the first benchmark that jointly emphasizes comprehensiveness
We pair each prompt with a human-verified checklist of individual yes/no questions, for a total of around 13, 500 questions across the benchmark. This facilitates fine-grained and reliable assess- ment of whether the generated images faithfully capture both explicit and implicit elements.
We conduct comprehensive evaluations on 38 current T2I models and conclude valuable insights, revealing that composition, though steadily improving, still remains unsolved in complex scenar- ios, whereas reasoning lags markedly behind and stands as the central bottleneck.
2 RELATED WORKS
Text-to-Image Generative Models. In recent years, T2I generation has witnessed significant ad- vancements, with its rapid development largely driven by the emergence of diffusion models (Ho
3

---

Published as a conference paper at ICLR 2026
Prompt: A vibrant community garden. full foom daring spring moening, where rows russed bedh showease In the foregoondd bright red climb wonden tnalis whire purple levender bushes velease thei fragrance along the gravel palli, surrouaded by taracottu. planters owrrllowin
Quen-
hisople hisople 2 Prosppe
T2I Model
Prompt Design Guidelines
pearbyy weathered wooden
Rike
Rute 2
colorfil momgolbl
Taxonomy
① Task Goal 2
Checklis Construction Rules
and putuniss Soweeer this passoful garden contains no nomhomy cacti or socenennt plants Freil hetto like buill and mint dedicated sections, while fayy guts.
B Checklist Eval
Input
Instruction
LRMs:
OpenAI
Gemini * Claude
MLLM
Quastion))
Outpud
Benchmark Data
Are there caised beds the imgee?
a Ceerect: 25
Verification
Peotupt:
0-0-
there thorny coett
Score:
25
there pata image?
33
(a) Data Generation
(b) Evaluation Protocol
Figure 2: Overview of our T2I-COREBENCH pipeline.
et al., 2020; Ho & Salimans, 2022; Rombach et al., 2022; Wang et al., 2024; 2026). Predominant models, including the Stable Diffusion series (Esser et al., 2024). the Flux series (Black Forest Labs, 2024), and the DALL-E series (Ramesh et al., 202 1), have led to substantial improvements in compositional text-image alignment. To better align with the textual modality at the token level, autoregressive (Sun al., 2024; Li al., 2024b; Tian al., 2024; Han et al., 2025) and unified mod- els (Chameleon, 2024; Xie et al., 2024; Chen et al., 2025c; Deng et al., 2025a; Chen et al., 2025a; Wu al., 2025a) have emerged in LLM-like architecture, demonstrating remarkable performance in composition tasks as well as reasoning tasks due to their autoregressive paradigm. Meanwhile, some approaches (Guo al., 2025b; Li al., 2025b; Liao al., 2025; Duan et al., 2025) are exploring integrating reasoning into T2I generation to handle more complex and controllable tasks.
Text-to-Image Evaluation Benchmarks. Driven by the explicit or implicit nature of T2I genera- tion, which requires both composition and reasoning. Early T2I benchmarks (Huang et al., 2023a; Ghosh et al., 2023; Li et al., 2024a) primarily target composition tasks with explicit visual elements. Subsequent benchmarks (Hu et al., 2024; Wu et al., 2024; Wei et al., 2025; Zhou et al., 2025; Fang et al., 2025) complicate the prompt with more detailed visual elements, yet still fall short in captur- ing the real-world challenge of high compositional density. In parallel, reasoning-orienter bench- marks (Fu et al., 2024; Meng et al., 2024; Niu et al., 2025; Chen et al., 2025b; Chang et al., 2025; Wu et al., 2025c; Sun et al., 2025; Wang et al., 2025; Li et al., 2025a; 2026) are gaining prominence as T21 models progress in reasoning tasks, including reasoning dimensions such as commonsense, logical, and causality. However, they primarily focus simple one-to-one inference, overlooking more complex multi-step reasoning prevalent in real-world scenarios. Furthermore, their taxonomy of both capabilities mostly heuristic, thereby failing cover all relevant dimensions in evaluation.
3 T2I-COREBENCH
In this section, we introduce "2I-COREBENCH as shown in Fig. 2, benchmark designed evalu- ate both composition and reasoning capabilities under real-world complexities, including high com- positional density and reasoning intensity. We first formulate a comprehensive T21 evaluation tax- onomy with complexity specified for each dimension in See. 3.1. Building upon this taxonomy, we then outline the benchmark construction details Sec. 3.2 and statistical analyses in Sec. 3.3.
3.1 EVALUATION DIMENSIONS
To address the limitations of previous benchmarks, which evaluate composition and reasoning in iso- lation using heuristic taxonomies, we formulate a comprehensive evaluation taxonomy that unifies both capabilities and reflects real-world generation challenges, as shown in Table 2.
Composition. Inspired by scene graph structures (Johnson et al., 2015; Chang et al., 2021), a visual scene (e.g., image) can be fully described by three components: instances, attributes, and rela- tions. Based on this, we define three corresponding dimensions under real-world complexities, i.e., MI Multi-Instance MA Multi-Aftribute, and MR Mulli-Relation, evaluate compositional capabil-
4

---

Published as a conference paper at ICLR 2026
Table 2: Definition of the 12 evaluation dimensions in our TOICCOREBENCH. Each dimension is described with its definition, along with a complexity number that quantifies the bolded element, driven by the density of visual elements in composition and the intensity of inferences cone-to-many or many-to-on in reasoning. More detailed descriptions can be found in Appx. A.I.
Dimension
Definition
#Complexity
Composition
MI Multi-rnstance
Generate muuttppe instances single image.
25
MA Multi-Atribbtee
Bind multiple attributes single subject. Connect multiple relations within unified scene.
20
Multi Relatiinn
TR Text Rendering
Render multiple texts with content fidelity and layost accuracy.
15
ER Logical Reasoning
Solve premise-based pizzles through mulii-ste inference.
5
BR Behavioral Reasoningg
Tufer visual outcomes from initial states and subsequent behaviors.
8
Reavoning
MIR Hypothetical Reasoning
Apply counterfactual premises and propagate their effects across items.
10
PR Procedural Reasoning
Reason ordered multi-step procedures derive final scene.
GR Generdditution Reasoning
Indure rules from exumples and andly them to complete new scenes.
AR Analogical Reasoning
Transfer relational rules from a source domain to a target domain.
CR Communsense Reasoning
Complete scenes by inferring unstated commonsense elements.
~v 5
RR Reconstrictvv Reasoning
Reconstruct plausible intiial states by tracing bockward from observed clues.
ities. Moreover, we introduce TR Text Rendering as separate dimension to account for its unique complexity in content and layout accuracies of texts, as shown in Fig. (a).
Reasoning. In T2I generation, prompts inevitably involve implicit visual elements, making reason- ing a fundamental capability. To ensure a comprehensive evaluation, we adopt a tripartite framework of reasoning in philosophical literature (Peirce, 1934; Zalta et al., 2003; Godfrey-Smith, 2009), i.e., deductive, inductive, and abductive reasoning. This framework provides a rigorous foundation for reasoning types, on which we define eight reasoning dimensions tailored to T2I scenarios.
Deductive Reasoning is the process of drawing conclusions from given premises, ensuring that if the premises hold, the conclusion cannot be false. In T2I scenarios, this means generating images determined by the premises, based on which we define LR Logical Reasoning, BR Behavioral Reasoning, HR Hypothetical Reasoning, and PR Procedural Reasoning, as shown in Fig. (b).
Inductive Reasoning is the process of inferring conclusions from observed regularity patterns rather than from explicit premises. In T21 scenarios, this corresponds to inferring visual elements from underlying structural patterns in examples, based on which we define GR Generalization Reasoning and AR Analogical Reasoning, as shown in Fig. 3 (c).
Abductive Reasoning is the process of reconstructing the most plausible explanation from obser- vations. In T21 scenarios, this entails reconstructing hidden causes or unstated commonsense that best explain the visual observations, based on which we define CR Commonsense Reasoning and RR Reconstructive Reasoning, as shown in Fig. 3 (d).
3.2 BENCHMARK CONSTRUCTION
Building upon the evaluation dimensions defined in Sec. 3.1, we now construct T2I-COREBENCH through a standardized pipeline, as shown in Fig. 2. In our setup, each evaluation sample consists of a prompt, which guides T2I generation, and a checklist, which enables point-by-point verification of the generated visual elements. To systematically generate benchmark data across all dimensions, we design a unified instruction template, including: (1) Task Goal, outlining the evaluation objective of each dimension as described in Sec. 3.1: (2) Prompt Design Guidelines, specifying principles for constructing diverse and complex prompts as detailed in Sec. A.I; and (3) Checklist Construction Rules, defining how to decompose the target scene into atomic, objective, and verifiable questions. All samples undergo rigorous human verification to ensure quality and reliability in Appx. A.3.
Prompt Design for Generation. Since our benchmark features prompts with high compositional density and reasoning intensity, previous strategies prove inadequate: human-writte written prompts (Otani et al., 2023; Niu et al., 2025; Chang et al., 2025) are iabor-intensive and lack scalability, while template-bel based prompts (Huang et al., 2023a; Ghosh et al., 2023; Wu et al., 2024) are rigid and lim- ited in scene diversity. To overcome these issues, we leverage Large Reasoning Models (LRMs) to assist data construction, exploiting their broad knowledge to cover diverse scenes (Lee et al., 2023)
5

---

Published as a conference paper at ICLR 2026
Muti Instoncc (MI)
Poongt A lecty demertoryy wheel classeoe
The foo berh des
Mult Attricutc (MA)
Pronpe: snggfrenttenne dholowek dragee coner the been lesss oupir
lokle spale odler meg and coutree prich whi lerg viacckend and
Eier Votterse innmeers watitteg The dugas red seretee with
platti
Thase
pastacal alos rllasing wal park pilli
no masted larronsca Like doaes pullas poine, lopdaa the Sccia pasiy
The diaski hald Sutain Bisw colaing sux sucallle cable The drggeic deger
veemeom gol oer
and ckfiche
shent hemes logg
of cae
abloart
Chackian U. there attocck Dh keaget
Chacilis
Fita dasaas poure, call machuded disdwadd?
thtt chaldiaadd the imag??
moker dactord compeNane?
Multi-iaaation (1l1))
Povege
masles exprrmmed
Text Toanderiing ing (TT)
Pronge
aod is billking kiii
Man the froe
carrio cartng boad On bu couttin tais ure thrse applic and group
pasi beize dar, dien Advared Pai Julac Forrada ia maader kidc tod The packass thodd banz cnfermation actionn ACTIVE INCALLIENT tip Tegrelen tavgg and ba 1-d bolle On Teats
carrla the conrar some colees, Dait namber it une than
MEDIHEALTH PLUS
Lan smell longo feon laok an will iile the sall lov
inckle Fiis
ant ger book whe maser
Arradvi and Staee oiler
Checkbat:
pooner, in chat
Chrolint fertes SARIHAA TH FLEIS
[E- wikee sealer Eo moms saclin of spples
ferted SE0OORATTI protucalin the fion poa'
prowert
(a) Composition (i.e,,
MI
MA.
MR.
TR
Logital Roasonin Lut
Prongh reege lock
Nor aod Resecrn uy (041
Preege teadio The mutble in skgned aoil doets sump lnto paa bakker rilid sib
wenng fiell
you
and
the sodes
escinil The sabeta cake rtsieees (2) 3ed pubet oae "ike robe zue the burmase Blas sbbect uay. have De acinet (1 Doom root mre iabes adot Aroae lis the othir two score tall toe cutt The like cobot has an trab tare
rid rideedeed wake, vdach sate an whis about of paper The action that jas
lhe
magriay lie over be fint canaaoi the laa depat the nos afte aoniiiigg elico lanz camplindl faalbed
Cherklar:
Chocktat:
t saic o chared on il leaee ofle beeving
blur
Jusal the donisom kanly reg fhe cotthe
gimes
give
Tranpe Depict a butting day- theet dtimectes la the world evey vetedci
ITT)
Pronge tlarzas the thai racay after putferaze all of tolledtng the sipe 1. Hust seuar shett perple staued paper Full
ebude pertect iceans dnsted crekes Fresert yolo on, for ookere dike book lis 111 like wollen, wil so pai
dyy
fild
sing ofiee dk ao and coreer den wolld
pallout Dadke Ediling au sode fire kodd low stands spright and what
the
Checkti::
tad vhheda dppiced perfect aquace?
the sake.
Procctance"
12 Are bex's whoch depicedd pertect squarat
gien
at. drret hamp keop normal tores
Ans bet estmdddd loot zwrtrraabl
12. Dea the light paù soup to somal beews
tn cone thoat dartact and best sightty dowrowert
(b) Deductive Reasoning (i.e., LR
BR
HR
PR
Proways Thes a renoe Bui coaltr a Divbss Chantar band on a 'Dety's Dogpait Stady the caapate andersand
Proege Jan as bnacccona dapper the folloitig ronal proputian (I) each bas mactly ass aldos al auch becuaoo
krample The soapciee the the for clownn with hh wntdl slen, "Ticke veoied The
leatt ()) adjecmt
(ful
5 The Eouegle
palio cores the waker modle vellice conde Jaawing booke la fhr Edlewing the saee giesssttion pronille lor The laagy
dee
inalags pelctisiia rake thalee
rutil siree and wyntict
"Gerriiitttee Bides
Chriklat
fee go palled pillee?
Cheeliie
32 Exinecsttyy palkng dunse
Jos skde benagza imgtti
of the soting siyy wish doade
Bose the seciperal cledd pattir cover the crtic day aveat
(c) Inductive Reasoning (i.e..
GR
AR
Pronge: Terostler date were, mefiaing any snreisary red woold londing
Pronge the fevkene sisoeaal saave mi merrky with bokor that gie Dax lucding and
conducting chacksup oerviie lonker
lakm caddihy de heartheak. The sime standa doss by arciiin the doy's band leep a cale. meniks stod countr the buckgrzand sade cotie (lown light
Conenttie Tati Recresttcc and pobinnc high-ipood photopaaa of pordie und skggla raeneed jan after the thatt boes coeplited captating the tunter the tha io
has
Io ls Heikknng homm deoenn and even burtinn bible"
fo- Hoalladiie bellleking prlrrr miriin iy wilk"
m Dany Seay bylyy for
for imudls
(d) Abductive Reasoning (i.e., CR.
RR
illustrating (a) composition and (b-d) reasoning ca- pabilities across 12 dimensions (see Appx. C.3 for complete versions). Each dimension is designed to incorporate complexity tailored to its unique characteristics, allowing more challenging evaluation under real-world scenarios, and supports fine-grained evaluation with human-verified checklists.
Figure 3: Examples from
and strong reasoning capability to produce complex prompts (Zhong et al., 2024; Guo et al., 2025a). In practice, the Prompt Design Guidelines specify how ensure sufficient diversity, semantic den- sity, and reasoning complexity while keeping the prompt coherent, as detailed in Appx. A.2.
Checklist Design for Evaluation. Evaluating generations in complex scenarios requires more than existing metries: (1) CLIPScore (Hessel et al., 2021) fails to account for multiple explicit elements and implicit reasoning outcomes; and (2) direct MILLM-based scoring (Li et al., 2024a) requires the model itself to infer intended outcomes with accumulated errors. To facilitate fine-grained and reliable evaluation of both explicit and implicit visual elements, we follow previous visual-question- answering- based evaluation paradigms (Hu et al., 2023; Yarom et al., 2023; Cho et al., 2023a;b), by pairing each prompt with a checklist of independent yes/no questions (with the correct answer always "Yes"). Specifically, we define set of Checklist Construction Rules to decompose the target scene into atomic questions covering instances, attributes, relations, and reasoning outcomes in a verifiable manner, as detailed in Appx. A.2.
6

---

Published as a conference paper at ICLR 2026
Number of Questtions
Number- of Tokons
Oosposiion Rasoming
Compuettin
Manaorinng
Mann
Mown 10.53
200
100
AR
RR
MA
1R
BR
PR
AR
RR
(a) Prompt Token Distribution Figure 4: Statistics of our TEI-COREBENCH showing (a) prompt-token lengths and (b) checklist-
(b) Checklist Question Distribution
question counts. Our benchmark exhibits high complexity in both composition and reasoning capa- bilities, with an average prompt length of 170 tokens and an average of 12.5 questions per sample.
Evaluation Protocol. Following previous protocols (Hu et al., 2023; 2024), we introduce an MLLM evaluator, i.e., Gemini 2.5 Flash (Gemini Team, 2023), conduct automatic evaluation by framing each item as a binary visual question answering task (i.e., scored as "O" for "no" and "1" for "yes") in Fig. 2 (b). This protocol leverages the atomic checklist design, where each question targets an unambiguous visual element, ensuring inherent compatibility with MLLMM-assed evaluation.
3.3 STATISTICS AND ANALYSIS
To mitigate stylistic homogeneity and potential bias arising from relying on a single LRM (e.g., using the same model to generate prompts and produce images often yields inflated performance since they share similar training data), we employ three SOTA LRMs for data construction, including Claude Sonnet 4 (Anthropic, 2025), Gemini 2.5 Pro (Gemini Team, 2023), and OpenAl o3 (OpenAl, 2025). In statistics, for each of the 12 evaluation dimensions, we collect 30 samples with each of the three LRMs, resulting in total of 12 dimensions x 30 prompts 3 LRMs 1, 080 generation prompts and 13, 536 questions in evaluation checklists, as detailed Fig. 4.
EXPERIMENTS
4.1 EXPERIMENTAL SETUP
Evaluated Models. We evaluate 38 current T2I models across architectures and parameter scales,
covering both open- and closed-models. The open-source pool includes 28 models: (1) Dif-
fusion Models: SD-3-Mcdium, SD-3.5-Medium, SD-3,5-Larg (Esser et al., 2024), FLUX.1- schnell, FLUX.I-dew, FLUX.I-Krea-der (Black Forest Labs, 2024), FLUX.2-dev. FLUX.2- (Black Forest Labs, 2025), PixArt-a: (Chen et al., 2023), PixArt- E (Chen et al., 2024), HiDream-I1 (Cai al., 2025), Qwen-Image Qwen-Image-2512 (Wu al., 2025a), HunyuanImage-3.0 (Cao et al., 2025), Z-Image-Turbo (Z-Image Team, 2025), LongCat- Image (LongCat Team, 2025); (2) Autoregressive Models: Infinity-8B (Han et al., 2025), GoT- R1-7B (Duan et al., 2025); and (3) Unified Models: BAGEL, BAGEL w/ Think (Deng et al., 2025b), show-o2-1.50 show-o2-7B (Xie et al., 2025). Lanus-Pro-I2 Janus Prous- (Chen et al., 2025c), BLIP3o-4B, BLIP3o-8B (Chen et al., 2025a), OmniGen2-7B (Wu et al., 2025b). We fur- ther include 10 closed-source commercial models, including: Seedream 3.0 (Gao et al., 2025), Seedream 4.0 (Seedream et al., 2025), Seedream 4.5 (ByteDance, 2025), Gemini 2.0 Flash, Nano Banana, Nano Banana Pro (Gemini Team, 2023), Imagen 4, Imagen 4 Ultra (Google, 2025), GPT- Image (OpenAI, 2025a) and GPT-Image- 1.5 (OpenAl, 2025b).
klein-4B, FLUX.2-klein
Evaluation Details. To facilitate automatic evaluation, we introduce Gemini 2.5 Flash (Gemini Team, 2023) the MLLM evaluator, which exhibits strong languuee performance aligned with humans (see Appx. C.I) at relatively low cost. Considering the possible unavailability of closed-source APIs in the future, also report evaluation results with the open-source MLLMs, including: Qwwen2.5vLL-72B-mm 32h-Thinking and Qwen3- VL-30B-A3----- Thinking (Bai et al., 2025a). More comprehensive and up-to-date evaluation results with these MLLM evaluators In evaluation, we report the mean score across all samples within each dimension as its final score for that dimension. More details can be found in Appx. B.
(Bai et al., 2025b), Qwen3-VL--B--Thnnking Qwen3-VL.
available our de Leaderboard.

---

Published as a conference paper at ICLR 2026
Table 3: Main results on our 121-COREBENCH assessing both composition and reasoning capa- bilities evaluated by Gemini 2.5 Flash. #Params. indicates the number of parameters in the image generation component, e.g., diffusion transformer (Peebles & Xie, 2023). The shaded Mean column denotes the mean score for each capability. The best and second-best results are marked in bold and underline for open- and possedmodelss respectively. More comprehensive and up-to-date evaluation results with additional MILMM evaluators are available on our e
B Leaderboard.
Composition
Reasoning
Model
PParams.
Overall
MI
MA
MR
TR
Meun
LR
BR
HR
PR
GR
AR
CR
RR
Mean
Diffuzion Models
SD-3-Medium
2B
59.1
57.9
35.4
9.5
40.4
22.1
21.1
35.3
51.0
37.4
47.3
35,0
27.1
34,5
36.5
SD-3.5-Mediam
59.5
60.6
33.1
41.0
19.9
20.5
33.5
53.7
33.4
52.7
33.9
Largee
57.5
60.0
32.9
41.5
22.4
34.2
52.5
35.5
35.9
FLUX. I-schnell
12B
65.4
63.1
47.6
22.4
49.6
25.0
25.1
40.9
64.7
47.6
54.0
39.6
21.9
40.0
43.2
FLUX. I-dev
12B
58.6
60.3
44.1
31.1
48.6
24,8
23.0
36.0
61.8
42.4
57.2
36.3
30.3
39,0
42.2
FLUX.1 1- -Keea-dev
12B
70.7
71.1
53.2
28.9
56.0
30.3
26.
44.5
70.6
50.5
57.5
46.3
28.7
44.3
48.2
HLUX 2-dev
328
89.5
83.4
72.7
933.
84.7
48.4
33.5
53.4
83.1
62.3
64.3
62.1
26.9
54.2
64.4
FLUX FLUX.2-Alein-1B
4B
80.0
77.1
63.5
53.0
68.4
34.6
30.5
49.5
77.3
58.1
58.1
48.8
25.2
47.7
54.6
FLUUX.2-keein-9
85.8
80.6
68.0
77.7
78.0
32.9
52.4
90.6
60.2
52.0
PixArt-
0.68
40,2
42.2
14.2
25.0
11.6
30.4
22.6
44.4
23.7
24.1
PixAst-E
0.68
47.2
49.7
23.8
2.8
30.9
14.7
18.3
26.7
39.2
25.7
44.9
33.9
24.3
28.5
29.3
HiDream-H1
17B
62.5
62.0
42.9
33.9
50.3
34,2
24.5
40.9
53.2
34,2
50.3
46.1
31.7
39.4
43.0
Qwen-Image
20B
81.4
79.5
65.6
85.5
78.0
41.1
32.2
48.2
75.1
56.5
53.3
61.9
26.4
49.3
58.9
Qwen Twen-Image-2511
20B
88.5
52.5
21.9
91.9
83.2
42.7
34.6
53.6
82.0
62.0
57.2
60.3
21.7
51.7
62.4
Z-Image-Tubo
6B
79.5
72.2
62.7
83.9
74.6
36.9
28.8
48.7
74.0
56.2
55.8
52.0
26.0
47.3
56.4
LongCa-lImage
81.4
74.5
70.8
48.5
75.5
72.5
61.1
54.1
Auturegresive Models
Infinity-SB
SB
63.9
63.4
47.5
10.8
46.4
28.6
25.9
42.9
62.6
47.3
59.2
46.9
24.6
42.3
43.6
GoT-R1-7B
7B
48.8
55.6
32,9
6.1
35.8
22.1
19.2
31,3
49.2
34,8
46.2
32,1
14.6
31,2
32.7
Unified Models
BAGEL
14B
61.9
65.2
45.8
9.7
16.4
23.4
21.9
33.0
51.6
31.2
50.4
32.4
29.3
34.1
38.2
BAGEL w/ Think
14B
57.7
60.8
37.8
2.2
39.6
25.5
25.4
33.9
58.6
53.5
56.9
41.6
39.8
41.9
41.1
show-o2-1.58
1,58
59.5
60.3
36.1
4.6
40.1
21.6
21.8
37.1
47.7
39.9
44.7
29.0
24.0
33,2
35.5
show-o2-7B
59.4
61.8
38.1
10.4
23.2
23.1
37.5
51.6
40.9
47.2
32.2
34.6
Janus-Pro-18
51.0
54.5
33.8
2.9
35.5
24.7
13.4
7.1
15.1
6.7
13.D
21.5
Janus Pro-7B
7B
54.4
59.3
40.9
7.5
40.5
19.8
20.9
34.6
22.4
11.5
30.4
8.7
9.8
19.8
26.7
BLIP3o-4B
4B
45.6
47.5
20.3
0.5
28.5
14.2
17.7
26.3
36.3
37.6
37.8
31.3
24,8
28.2
28.3
BLIP3o-SB
SB
16.2
50.4
21.1
0.5
30.3
14.8
20.7
28.3
39.6
43.4
51.0
35.9
20.1
31.8
31.3
OmniGen2:77B 7BBB
7B
67.9
64.1
48.3
19.2
49.9
24.7
23.2
43.3
63.1
46.1
54.2
36.5
24.1
39.4
42.9
Huny manloganno 3.0
80B
84.9
81.2
63.7
85.7
9
39.6
32.8
51.4
72.4
54.1
54.1
57.0
27.7
48.6
58.7
Closed-Source Models
Seedrcam
79.9
78.0
63.7
67.3
33.6
50.3
75.1
54.9
61.7
50.3
Seedream 4.0
91.5
84.5
75.0
93.6
86.1
26.3
54.1
60.7
85.8
85,9
77.1
71.6
47.9
69.9
75.3
Seodream 4.5
92.0
87.1
22.2
96.2
88.1 3
71.1
58.3
64.5
87.6
87.9
76.9
73.5
50.9
71.3
77.0
Gemini 2.0 Flash
67.5
68.5
49.7
62.9
62.1
39.3
39.7
47.9
69.3
58.5
63.7
51.2
39.9
51.2
54.8
Nano Banana
85.7
77.9
72.6
86.3
80.6
64.5
64.9
67.1
85.2
84.1
83.1
71.3
68.7
73.6
75.9
Nano Banana Pro
91.9
85.5
83.4
98.1
89.7
90.8
73.6
77.8
90.7
90.4
84.2
77.0
76.7
82.7
85.00
Imagen 4
82.8
74.
66.3
90.2
78.4
51.8
56.8
82.8
79.5
73.3
65.9
70.0
Imagen 4 Ultra
90.0
80.0
73.2
86.2
82.4
63.6
62.4
66.1
88.5
82.8
83.0
76.3
72.9
76.1
GPT Image
84.1
75.9
72.7
86.4
79.8
59.0
54.8
65.6
87.3
76.5
82.0
70.9
56.1
69.0
72.6
GPT--mage-11
87.0
83.4
76.9
94.7
85.5
62.2
67.0
22.2
90.8
83.9
84.5
77.5
59.0
24.6
78.2
4.2
MAIN RESULTS
As shown in Table 3, we evaluate a wide range of T2I models on our T21-COREBENCH, revealing valuable insights into their strengths, weaknesses, and advancements, particularly in handling real- world scenarios that require high compositional density and reasoning intensity:
(1) Composition shows steady progress but remains unsolved, particularly in complex scenar- ios. Across all models, we observe consistent gains on composition tasks with T2I model iterations. For composition, the best closed-source model is Nano Banana Pro (89.7), while the best open- source model is FLUX.2-dev (84.7), which already approaches the advanced closed-source models. Nevertheless, composition in complex scenarios still remains challenging: even Nano Banana Pro struggles with attribe binding (MA: 85.5) and multi-relatin generation (MR: 83.4), high- lighting that fine-grained compositional generation is still an open problem.
8

---

Published as a conference paper at ICLR 2026
Lagical Aressing (LA); Genarate image showing foxr containes amauged outtiise sne bas, isc. leg, and botkl. Exch coraane collectacls ene tipe frutt and exactly one color The
winte table. The
Oneen-mrege
Nano Benana
GPT-inagg
frula ur: apple lunase unag
mupt The coloo
the juas and yoloe Use thas dus (0 The
costaiee inmedandy la kit of the aod outeiner The cottain de uppi in lax (1) The leg.
yello und but poilinn a (0 The botkl cousin the lemase and aut green The los contins leruggg (6) The comtiner gestion & D)) The ontaiter pisittion i's not the boitle.
"I Preeniss""
Aeevitoo Pronpie o clste talee lupte socrdd a gesss recangdar whose open diplars is receilig single
now fou ditind conannee feot Bal
6/12 0.50
margt cesting
bottle with namow visbly contaiiing ripe bauan. standing spiiht bermiing
After Prompt Rewrittng:
frorth
geggle
Quwin-maage
Nano Banana
-image
Checkhe: (All 12 Questiona)
Answer:
c1. la the container pusitio a jac?
the container position botde?
Work!
container pocition bloe?
07. Is the container pocition red!
12. Dove the continer at prsition contain grape?
Owen-Inage
Namio Banons
GPT-
Plysshhetiol Raaoning Depler citysttrent
every delvery
perfect polise
())a yellne mnstrecdds taay line dhte--tooper public
lamp, (9) padattaan deylgaatzaaenn
(12)
Rewriton Pronge In tright dayrime nollon, a beatiig rty-tteet interraction elled with tratic whose wheels are all perfect apureee the nscred creemoak yyyalo bor oer with feee starpoomined equane wherll udec bonde aan iodae with matching square schelll ficther bock a boworing rnd deuble-deden hea numbee forwant BB is unn apace whodd the focermandd only the Joner lapp and thees of pedestinas supping ofl the curb aze vadis the leac of till inilic Light pali dapplooi green.
After Prompt Rewriting:
Qwen--maae
Nano Banana
GPT-image
Checklit: (All Questions)
01. Are the tast's whechs depieted as perfect squares?
02, Art the but's which depicted se perfect qquano?? Art the asdan's whech depicted perfect squaces?
iP Not Work!
07, pedestran's is shoes leep their
soles?
12. Dors the trafTi-llghtt pole keep iss normal form"
Figure 5: Qualitative examples before and after prompt rewriting. In some reasoning dimensions
(e.g., LR ) the primary challenge lies in textual reasoning, and prompt rewriting is highly effective. However, tasks such as transforming wheels into squares in HR remain difficult even after prompt rewriting, indicating that textual reasoning alone is insufficient and other mechanisms are required.
Table 4: Impact of prompt rewriting on reasoning dimensions. We evaluate two leading open- and closed-source models from Table 3, respectively. The subscripts Red and I Green indicate the relative increase or decrease compared to their original evaluation results before prompt rewriting.
Reasoning (After Prompt Rewriting)
Model
LR
BR
HR
PR
GR
AR
CR
RR
Mean
--Keaadvre
64.9/s1.8
49.8
54.9
77.9
74.6
711
61.5p15.1
69.2
65.5
Qwen-Image
85.1
59.6
64.2
84.6
80.3
7177
71.9
64.5
72.7
Nano Banana
86.5+22.0
67.7
73.7
88.8
83.2,o.s
81...17
72.4..
72.
78.2 7/..
GPT-Image
85.2+28.2
71.0/18.a
78.8
87.1
82.2
85.9
75.1
73.9
19.9.10.8
(2) Reasoning remains the primary bottleneck, as even the SOTA models struggle with multi- step inferences. Despite achieving the highest overall score, Nano Banana Pro achieves only 82.7 in reasoning (7.0 below its composition score), and shows weak performance on several reasoning
dimensions (BR: 73.6, HR: 77.8, CR: 77.0, RR: 76.7). This gap is even more striking for open- source models: Qwen-Image-2512 reaches 83.7 in composition but only 51.7 in reasoning (32.0 points lower). These results indicate that current T2I models still struggle to infer implicit visual elements from prompts, underscoring reasoning as the central unsolved challenge in our benchmark.
(3) Diffusion models show a modest overall edge, and encoder-side instruction understanding is increasingly critical. Among open-source models, diffusion models exhibit a slight average ad- vantage over autoregressive and unified models, although performance variance remains high and no paradigm consistently dominates. More notably, recent T2I models increasingly adopt stronger instruction encoders, which has become a key direction for performance gains. For example, Qwen- Image-2512 leverages the Qwen2.5-VV VL encoder (Bai et al., 2025b). which provides strong multi- modal instruction understanding (Liu et al., 2023), and achieves the leading overall performance.
4.3 IMPACT OF PROMPT REWRITING
Prompt rewriting entails explicit textual reasoning before synthesis, and the rewritten prompt is then fed to the generator, which has been used in prior T2I methods and evaluations (Betker et al., 2023; Niu et al., 2025; Deng et al., 2025a). In our evaluation, BAGEL w/ Think (Deng et al., 2025a) enables its encoder (i.e., LLM) to conduct intermediate reasoning on the original prompt and rewrite

---

Published as a conference paper at ICLR 2026
it with explicit visual elements, such as attribute changes, action outcomes, and implicit cues. The rewritten instruction is then passed to the image generator. Compared with its baseline BAGEL in Table 3, BAGEL w/ thinking improves mean reasoning from 34.1 to 41.9 and achieves leading open-source scores on GR (53.5) and RR (39.8), but its composition drops from 46.4 to 39.6. These gains come from inferring implicit visual elements through intermediate reasoning, while the drop shows that such reasoning may omit explicit elements and divert attention from direct composition.
To study rewriting in model-agnostic way, we adopt OpenAl o3 (OpenAI, 2025) to rewrite orig- inal prompts (Appx. B.3) and evaluate the effect across models in B2I-COREBENCH in Table 4. We conelude the following insights: (1) Native reasoning capability constitutes key direction for future T2I models. Weaker models (e.g., FLUX.I-Krea-dee
achieve greater im- provements over 20 points, as rewriting compensates for their limited native reasoning capability. In contrast, stronger models (e.g., Nano Banana, GPT--mmage show marginal or negative effects, since their native reasoning already captures such benefit. (2) Unified models provide intrinsic ad- vantages for T21 reasoning. GPT-Image and Nano Banana, both unified models for native image generation, consistently outperform most counterparts across reasoning dimensions even without large rewriting gains. This indicates that such architectures not only better internalize textual reason- ing but also support more cohesive text-image integration, offering inherent advantages and future promise for integrated reasoning. (3) Textual reasoning only is insufficient in our benchmark. Despite overall improvements, prompt rewriting cannot fully address all T2I reasoning scenarios, e.g. the best model GPT-Image scoring below 80 on BR. HR, CR, and RR. This because T2I generation is inherently multimodal, often requiring multimodal reasoning beyond textual infer- ence, while prompt rewriting can only modify the text and cannot mitigate inherent visual biases or text-image coupling. Fig. 5 shows that even with an explicit instruction for square wheels af- ter prompt rewriting in HR. the model still fails due to the tight coupling between car wheels and their circular shape. To achieve more faithful T2I generation, future work should explore more multimodal interaction mechanisms (e.g., interleaving reasoning (Huang et al., 2025)).
5 CONCLUSION
In this paper, we present TZI,COREBENCH, a comprehensive benchmark designed to evaluate both composition and reasoning capabilities of T2I models. Through detailed taxonomy of 12 dimen- sions, we evaluate both composition and reasoning challenges under real-world complexities. Our evaluation of 38 models reveals clear progress in composition, yet also highlights persistent chal- lenges in both capabilities when faced with real-world complexities involving high compositional density and reasoning intensity, with reasoning remaining the primary bottleneck.
ETHICS STATEMENT
With the introduction of the T2I-COREBENCH benchmark, we anticipate continuous improvements in both composition and reasoning capabilities of T2I models, leading to increasingly realistic and faithful Al-generated content. While these advancements bring substantial opportunities, they also raise concerns about the proliferation of Al-generated content, which may overwhelm creative indus- tries and lead to issues around copyright and authenticity. As the boundary between human-created and Al-generated works blurs, there is growing need for well-defined frameworks to clarify owner- ship, prevent misuse, and promote transparency. Solutions such as watermarking, content detection, and regulations are crucial to address these ethical challenges and ensure that innovation is balanced with responsible Al development and use.
REFERENCES
Anthropic. Introducing claude 4. https::/www anthropic wwwpccww////cccecccc/oc///c/ee 2025. Announces Claude Opus and Claude Sonnet
May
Shuai Bai, Yuxuan Cai, Ruizhe Chen, Keqin Chen, Xionghui Chen, Zesen Cheng, Lianghao Deng, Wei Ding, Chang Gao, Chunjiang Ge, Wenbin Ge, Zhifang Guo, Qidong Huang, Jie Huang, Fei Huang, Binyuan Hui, Shutong Jiang. Zhaohai Li, Mingsheng Li, Mei Li, Kaixin Li, Zicheng Lin, Junyang Lin, Xuejing Liu, Jiawei Liu, Chenglong Liu, Yang Liu, Dayiheng Liu, Shixuan Liu,
10

---

Published as a conference paper at ICLR 2026
Dunjie Lu, Ruilin Luo, Chenxu Lv, Rui Men, Lingchen Meng. Xuancheng Ren, Xingzhang Ren, Sibo Song, Yuchong Sun, Jun Tang, Jianhong Tu, Jianqiang Wan, Peng Wang. Pengfei Wang, Qi- uyue Wang, Yuxuan Wang, Tianbao Xie, Yiheng Xu, Haiyang Xu, Jin Xu, Zhibo Yang, Mingkun Yang, Jianxin Yang, An Yang. Bowen Yu, Fei Zhang. Hang Zhang, Xi 7hang, Bo Zheng, Hu- men Zhong, Jingren Zhou, Fan Zhou, Jing Zhou, Yuanzhi Zhu, and Ke Zhu. Qwen3-vl technical report, 2025a. URL https: / //tpxaaavvv///////////eea//aaav. org/ aaaa/bbs//2511 21631.
Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. Qwen2. 5-vl technical report. arXiv preprint arXiv:2502. 2025b.
James Betker, Gabriel Goh, Li Jing, Tim Brooks, Jianfeng Wang, Linjie Li, Long Ouyang, Juntang Zhuang, Joyce Lee, Yufei Guo, et al. Improving image generation with better captions. Computer Science. https://edn. openai. com/papers/dalle-33 pdf. 2(3):8. 2023.
Black Forest Labs. Flux. https: htttp:://ithhb.... cww.bbackcccccccccc.c http://laccccoosst---eee fforest-labe ffux, 2024. 2024.
Black Forest Labs. FLUX.2: Frontier Visual Intelligence. http:: httpb//oogoouuu/aoo/eo flux-2, 2025.
Kay Henning Brodersen, Cheng Soon Ong, Klaas Enno Stephan, and Joachim M Buhmann. The balanced accuracy and its posterior distribution. In 2010 20th international conference on pattern recognition, pp. 3121-3124. IEEE, 2010.
ByteDance. Seedream 4.5. https:://// / seed. bytedance. cttpeddaanecccoeeeeeem www.//////eeecccccoo
2025.
Qi Cai, Jingwen Chen, Yang Chen, Yehao Li, Fuchen Long, Yingwei Pan, Zhaofan Qiu, Yiheng Zhang, Fenghin Gao, Peihan Xu, et al. Hidream-il: A high-efficient image generative foundation model with sparse diffusion transformer. arXiv preprint arXiv:2505.22705 2025.
Siyu Cao, Hangting Chen, Peng Chen, Yiji Cheng, Yutao Cui, Xinchi Deng, Ying Dong, Kip- per Gong, Tianpeng Gu, Xiusen Gu, et Hunyuanimage 3.0 technical report. arXiv preprint 23955,, 2025.
Team Chameleon. Chameleon: Mixed-modal early-fusion foundation models. arXiv preprint arXiv:2405.09818 2024.
Jingjing Chang. Yixiao Fang, Peng Xing, Shuhan Wu, Wei Cheng, Rui Wang, Xianfang Zeng, Gang Yu, and Hai-Bao Chen. Oneig-bench: Omni-dimensional nuanced evaluation for image generation. arXiv preprint arXiv:2506.07977, 2025.
Xiaojun Chang. Pengzhen Ren, Pengfei Xu, Zhihui Li, Xiaojiang Chen, and Alex Hauptmann. A comprehensive survey of scene graphs: Generation and application. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(1):1-26, 2021.
Jiuhai Chen, Zhiyang Xu, Xichen Pan, Yushi Hu, Can Qin, Tom Goldstein, Lifu Huang. Tianyi Zhou, Saining Xie, Silvio Savarese, et al. Blip3-o: A family of fully open unified multimodal aodeessarcchteectuee, training and dataset. arXiv preprint arXiv:2505. 09568, 2025a.
Junsong Chen, Jincheng Yu, Chongjian Ge, Lewei Yao, Enze Xie, Yue Wu, Zhongdao Wang, James Kwok, Ping Luo, Huchuan Lu, et al. Pixart-o: Fast training of diffusion transformer for photore- alistic text-to-image synthesis. arXiv preprint arXiv:2310.004266 2023.
Junsong Chen, Chongjian Ge. Enze Xie, Yue Wu, Lewei Yao, Xiaozhe Ren, Zhongdao Wang, Ping Luo, Huchuan Lu, and Zhenguo Li. Pixart-o: Weak-to -o-soong training of diffusion transformer for 4k text-to-imann to--ttggoigeeeeeee generation. In European Conference on Computer Vision, pp. 74 Springer, 2024.
Kaijie Chen, Zihao Lin, Zhiyang Xu, Ying Shen, Yuguang Yao, Joy Rimchala, Jiaxin Zhang, and Lifu Huang. R2i-bench: Benchmarking maasonnnggdrivven to-itoo-mmae generation. arXiv preprint 2rXiv:2505.23493 2025b.
11

---

Published as a conference paper at ICLR 2026
Xiaokang Chen, Zhiyu Wu, Xingchao Liu, Zizheng Pan, Wen Liu, Zhenda Xie, Xingkai Yu, and Chong Ruan. Jamus-pro: Unified multimodal understanding and generation with data and model scaling, arXiv preprint arXiv:2501.17811 2025c.
Jaemin Cho, Yushi Hu, Roopal Garg, Peter Anderson, Ranjay Krishna, Jason Baldridge, Mohit Bansal, Jordi Pont-Tuset, and Su Wang. Davidsonian scene graph: Improving reliability in fine- grained evaluation for text-to-image generation. arXiv preprini arXiv:2310. 18235, 2023a.
Jaemin Cho, Abhay Zala, and Mohit Bansal. Visual programming for step-by-step text-to-image generation and evaluation. Advances in Neural Information Processing Systems, 36:6048-6069, 2023b.
Chaorui Deng, Deyao Zhu, Kunchang Li, Chenhui Gou, Feng Li, Zeyu Wang, Shu Zhong, Wei- hao Yu, Xiaonan Nie, Ziang Song, Guang Shi, and Haoqi Fan. Emerging properties in unified multimodal pretraining. arXiv preprint arXiv:2505. 14683, 2025a.
Chaorui Deng, Deyao Zhu, Kunchang Li, Chenhui Gou, Feng Li, Zeyu Wang, Shu Zhong, Weihao Yu, Xiaonan Nie, Ziang Song, et al. Emerging properties in unified multimodal pretraining. arXiv preprint ar arXiv:2505. 14683, 2025b.
Chengqi Duan, Rongyao Fang. Yuqing Wang, Kun Wang, Linjiang Huang, Xingyu Zeng. Hong- sheng Li, and Xihui Liu. Got-rl: Unleashing reasoning capability of milm for visual generation with reinforcement learning. arXiv preprint arXiv:2505. /7022,, 2025.
Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, al. Scaling rectified flow transformers for high-resolution image synthesis. In Forty-first international conference on machine learning. 2024.
Rongyao Fang, Aldrich Yu, Chenggi Duan, Linjiang Huang, Shuai Bai, Yuxuan Cai, Kun Wang, Si Liu, Xihui Liu, and Hongsheng Li. Flux-reate Flux-reason-óm & wrism-bench. A million-see scale text-to- image reasoning dataset and comprehensive benchmark. arXiv preprint arXiv:2509 09680, 2025.
Xingyu Fu, Muyu He, Yujie Lu, William Yang Wang, and Dan Roth. Commonsense-12i chal- lenge: Can text-to-image generation models understand commonsense? arXiv preprint arXiv:2406.07546, 2024.
Yu Gao, Lixue Gong, Qiushan Guo, Xiaoxia Hou, Zhichao Lai, Fanshi Li, Liang Li, Xiaochen Lian, Chao Liao, Liyang Liu, al. Seedream 3.0 technical report. arXiv preprint arXiv:2504.1 1136,, 2025.
Yunfan Gao, Yun Xiong, Xinyu Gao, Kangxiang Jia, Jinliu Pan, Yuxi Bi, Yixin Dai, Jiawei Sun, Haofen Wang, and Haofen Wang. Retrieval-augmented generation for large language models: A survey, arXiv preprint arXiv:2312.1099 2(1), 2023.
Gemini Team. Gemini: a family of highly capable multimodal models. arXiv preprint
arXiv:2312.11805, 2023.
Dhruba Ghosh, Hannaneh Hajishirzi, and Ludwig Schmidt. Geneval: An object-focused framework for evaluating text-to-image alignment. Advances in Neural Information Processing Systems, 36: 52132-52152, 2023.
Peter Godfrey-Smin Smith. Theory and reality: An introduction the philosophy of science. In Theory and reality. University of Chicago Press, 2009.
Google. Imagen 4. https::/deeppffe
google/modeesse immge/// 2025.
Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Pciyi Wang, Xiao Bi, et al. Deepseek-rl: Incentivizing reasoning capability in Ilms via reinforcement learning. arXiv preprint arXiv:2501.122 2025a.
Ziyu Guo, Renrui Zhang. Chengzhuo Tong, Zhizheng Zhao, Rui Huang. Haoquan Zhang, Manyuan Zhang, Jiaming Liu, Shanghang Zhang, Peng Gao, et al. Can we generate images with cot? let's verify and reinforce image generation step by step. arXiv preprint arXiv:2501. 13926, 2025b.
12

---

Published as a conference paper at ICLR 2026
Jian Han, Jinlai Liu, Yi Jiang, Bin Yan, Yuqi Zhang, Zehuan Yuan, Bingyue Peng, and Xiaob- ing Liu. Infinity: Scaling bitwise autoregressive modeling for high-resolution image synthesis. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 15733-15744, 2025.
Jack Hessel, Ari Holtzman, Maxwell Forbes, Ronan Le Bras, and Yejin Choi. Clipscore: A reference-free evaluation metric for image captioning. arXiv preprini urXiv:2104.08718, 2021.
Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance.
arXiv preprint
12X.5 2022.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840-6851, 2020.
Wenyi Hong, Wenmeng Yu, Xiaotao Gu, Guo Wang, Guobing Gan, Haomiao Tang, Jiale Cheng, Ji Qi, Junhui Ji, Lihang Pan, et al. Glm-4.1 v-thinking: Towards versatile multimodal reasoning with scalable reinforcement learning. arXiv preprint arXiv:2507.01006 2025.
Xiwei Hu, Rui Wang. Yixiao Fang, Bin Fu, Pei Cheng, and Gang Yu. Ella: Equip diffusion models with Ilm for enhanced semantic alignment. arXiv preprint 24X3v:2403.05/33 2024.
Yushi Hu, Benlin Liu, Jungo Kasai, Yizhong Wang, Mari Ostendorf, Ranjay Krishna, and Noah A Smith. Tifa: Accurate and interpretable text-to-image faithfulness evaluation with question swering. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 20406-20417, 2023.
Kaiyi Huang, Kaiyue Sun, Enze Xie, Zhenguo Li, and Xihui Liu. T2i-compbench: A compre- hensive benchmark for open open-worid compositional text-to-image generation. Advances in Neural Information Processing Systems, 36:78723-78747 2023a.
Lei Huang, Weijiang Yu, Weitao Ma, Weihong Zhong, Zhangyin Feng. Haotian Wang, Qianglong Chen. Weihua Peng, Xiaocheng Feng, Bing Qin, et al. A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions. ACM Transactions on Information Systems, 2023b.
Wenxuan Huang, Shuang Chen, Zheyong Xie, Shaosheng Cao, Shixiang Tang, Yufan Shen, Qingyu Yin, Wenbo Hu, Xiaoman Wang. Yuntian Tang, al. Interleaving reasoning for better text-to- image generation. arXiv preprint arXiv:2509.0669 2025.
Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Os- trow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card. arXiv preprint arXiv:2410.21276 2024.
Ben Hutchinson, Jason Baldridge, and Vinodkumar Prabhakaran. Underspecification in scene snseription-to-deppcttte tasks. In Proceedings of the 2nd Conference of the Asia Pacific Chapter of the Association for Computational Linguistics and the 12th International Joint Conference Natural Language Processing (Volume 1: Long Papers), pp. 1172-1184, 2022.
Justin Johnson, Ranjay Krishna, Michael Stark, Li-Jia Li, David Shamma, Michael Bernstein, and Li Fei-Fei. Image retrieval using scene graphs In Proceedings of the IEEE conference com- puter vision and pattern recognition, pp. 3668-3678, 2015.
Alycia Lee, Brando Miranda, and Sanmi Koyejo. Beyond scale: the diversity coefficient as a data quality metric demonstrates Iims are pre-trained on formally diverse data. In ICML Workshop Challenges in Deployable Generative AI, International Conference on Machine Learning (ICML), 2023.
Baiqi Li, Zhiqiu Lin, Deepak Pathak, Jiayao Li, Yixin Fei, Kewen Wu, Tiffany Ling, Xide Xia, Pengchuan Zhang, Graham Neubig, et al. Genai-bench Evaluating and improving compositional text-to-visual generation. arXiv preprint arXiv:2406. /3743 /3743, 2024a.
Hongxiang Li, Yaowei Li, Bin Lin, Yuwei Niu, Yuhang Yang, Xiaoshuang Huang, Jiayin Cai, Xiao- long Jiang. Yao Hu, and Long Chen. Gir-bench: Versatile benchmark for generating images with reasoning. arXiv preprint 2x.iv22510.11 1026, 2025a.
13

---

Published as a conference paper at ICLR 2026
Ruihang Li. Leigang Qu, Jingxu Zhang, Dongnan Gui, Mengde Xu, Xiaosong Zhang, Han Hu, Wenjie Wang, and Jiaqi Wang. Genarena: How can we achieve human- aligned evaluation for visual generation tasks? arXiv preprint wxXv::2602.06013. 2026.
Shufan Li, Konstantinos Kallidromitis,, Akash Gokul, Arsh Koneru, Yusuke Kato, Kazuki Kozuka, and Aditya Grover. Reflect-dit: Inference-time scaling for text-to-image diffusion transformers via in-context reflection. arXiv preprint arXiv:2503. 12271, 2025b.
Tianhong Li, Yonglong Tian, He Li, Mingyang Deng, and Kaiming He. Autoregressive image generation without vector quantization. Advances in Neural Information Processing Systems, 37: 56424-56445, 2024b.
Jiaqi Liao, Zhengyuan Yang. Linjie Li, Dianqi Li, Kevin Lin, Yu Cheng. and Lijuan Wang. Imagegen-cot: Enhancing text-to-image in-context learning with chain-of-thought reasoning. arXiv preprini arXiv:2503./9312 2025.
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. Advances in neural information processing systems, 36:34892-34911 2023.
LongCat Team. Langeat-image technical report. arXiv preprint arXiv:2512.07554 2025.
Fanqing Meng, Wenqi Shao, Lixin Luo, Yahong Wang, Yiran Chen, Quanfeng Lu, Yue Yang. Tian- shuo Yang, Kaipeng Zhang, Yu Qiao, al. Phybench: A physical commonsense benchmark for evaluating text-to-image models. arXiv preprint arXiv:2406. 1802, 2024.
Yuwei NIu, Munan Ning, Mengren Zheng, Weiyang Jin, Bin Lin, Peng Jin, Jiaqi Liao, Chaoran Feng, Kunpeng Ning, Bin Zhu, et al. Wise: A world knowledge-inforee informed semantic evaluation for text-to-image generation. arXiv preprint arXiv:2503.07263 2025.
OpenAI.
Gpt-4o-image, 2025a.
http:://ppenal..om//nnio
introducing-45 image generation/.
OpenAl.
Gpt-image-1.5. 2025b.
https://openai..oom/iddd
new www--chatpttinnce is As-herr/.
OpenAl.
Introducing o3 and 04-mini.
https://openal.com//nnco
April 2025.
Mayu Otani, Riku Togashi, Yu Sawai, Ryosuke Ishigami, Yuta Nakashima, Esa Rahtu, Janne Heikkilä, and Shin'ich ichi Satoh. Toward verifiable and reproducible human evaluation for text-to- image generation. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 14277-14286, 2023.
William Peebles and Saining Xie. Scalable diffuusin models with transformers. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 4195-4205, 2023.
Charles Sanders Peirce Collected papers of charles sanders peirce, volume 5. Harvard University Press, 1934.
Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea Voss, Alec Radford, Mark Chen, and Ilya Sutskever. Zero-shot text-to-image generation. In International conference on machine learning, pp. 8821-8831. Pmlr, 2021.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEF/CVF confer- ence on computer vision and pattern recognition, pp. 10684-10695, 2022.
Team Seedream, Yunpeng Chen, Yu Gao, Lixue Gong, Meng Guo, Qiushan Guo, Zhiyao Guo, Xiaoxia Hou, Weilin Huang, Yixuan Huang, et al. Seedream 4.0: Toward next-generation multi- modal image generation. arXiv preprint arXiv:2509.2042 2025.
Kaiyue Sun, Rongyao Fang, Chengqi Duan, Xian Liu, and Xihui Liu. T2i-reasonbench: Bench- marking massonigg-ifformer text-to-image generation. arXiv preprint arXiv:2508. 17472, 2025.
14

---

Published as a conference paper at ICLR 2026
Peize Sun, Yi Jiang, Shoufa Chen, Shilong Zhang, Bingyue Peng, Ping Luo, and Zehuan Yuan. Autoregressive model beats diffusion: Llama for scalable image generation. arXiv preprint 0.Xiv:2406.06625, 2024.
Keyu Tian, Yi Jiang, Zehuan Yuan, Bingyue Peng. and Liwei Wang. Visual autoregressive modeling: Scalable image generation via next-scale prediction. Advances in neural information processing systems, 37:84839-8448 2024.
Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdh- ery, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models. arXiv preprint ar arxiv: 11171, 2022.
Yibin Wang, Zhimin Li, Yuhang Zang, Yujie Zhou, Jiazi Bu, Chunyu Wang, Qinglin Lu, Cheng Jin, and Jiaqi Wang. Pref-grpo: Pairwise preference reward-based grpo for stable text-to-image reinforcement learning. arXiv preprint arXiv:2508.2075/1 2025.
Yuan Wang, Borui Liao, Huijuan Huang, Jinda Lu, Ouxiang Li, Kuien Liu, Meng Wang, and Xiang Wang. Thinking with frames: Generative video distortion evaluation via frame reward model.
arXiv preprint ar arXiv:2601.0 004033, 2026.
Zhicai Wang, Ouxiang Li, Tan Wang, Longhui Wei, Yanbin Hao, Xiang Wang, and Qi Tian. Prior preserved text-to-image personalization without image regularization. IEEE Transactions on Cir- cuits and Systems for Video Technology, 2024.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824-24837, 2022.
Xinyu Wei, Jinrui Zhang, Zeqing Wang, Hongyang Wei, Zhen Guo, and Lei Zhang. Tiif-benhh How does your 12i model follow your instructions? arXiv preprint wrXiv:2506.02166 2025.
Chenfei Wu, Jiahao Li, Jingren Zhou, Junyang Lin, Kaiyuan Gao, Kun Yan, Sheng-ming Yin, Shuai Bai, Xiao Xu, Yilei Chen, et al. Qwen-image technical report. arXiv preprint arXiv:2508.02334, 2025a.
Chenyuan Wu, Pengfei Zheng, Ruiran Yan, Shitao Xiao, Xin Luo, Yueze Wang, Wanli Li, Xiyan Jiang, Yexin Liu, Junjie Zhou, et al. Omnigen2: Exploration to advanced multimodal generation. arXiv preprini ar0iv:2506. (CCT:::550..8871,, 2025b.
Xindi Wu, Dingli Yu, Yangsibo Huang, Olga Russakovsky, and Sanjeev Arora. Conceptmix: A compositional image generation benchmark with controllable difficulty. Advances Neural formation Processing Systems, 77.86004-86041 2024.
Yongliang Wu, Zonghui Li, Xinting Hu, Xinyu Ye, Xianfang Zeng, Gang Yu, Wenbo Zhu, Bernt Schiele, Ming-Hsuan Yang, and Xu Yang. Kris-hench: Benchmarking next-level intelligent image editing models. arXiv preprint 150. 2025c
Yongliang WU, Yizhou Zhou, Zhou Ziheng, Yingzhe Peng, Xinyu Ye, Xinting Hu, Wenbo Zhu, Lu Qi, Ming-Hsuan Yang, and Xu Yang. On the generalization of sft: A reinforcement learning perspective with reward rectification. arXiv preprint arXiv:2508. 05629, 2025d.
Jinheng Xie, Weijia Mao, Zechen Bai, David Junhao Zhang, Weihao Wang. Kevin Qinghong Lin, Yuchao Gu, Zhijie Chen, Zhenheng Yang, and Mike Zheng Shou, Show-o: One single transformer to unify multimodal understanding and generation. arXiv preprint arXiv:2408. /2528, 2024.
Jinheng Xie, Zhenheng Yang, and Mike Zheng Shou. Show-o2: Improved native unified multimodal models. arXiv preprint arXiv:2506, 15564, 2025.
Yiyan Xu, Wenjie Wang, Yang Zhang, Biao Tang, Peng Yan, Fuli Feng, and Xiangnan He. Per- sonalized image generation with large multimodal models. In Proceedings of the ACM on Web Conference 2025, pp. 264-274, 2025a.
15

---

Published as a conference paper at ICLR 2026
Yiyan Xu, Jinghao Zhang, Alireza Salemi, Xinting Hu, Wenjie Wang, Fuli Feng, Hamed Zamani, Xiangnan He, and Tat-Seng Chua. Personalized generation in large model era: A survey. Pro- ceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 24607-24649, 2025b.
Zijun Yao, Yantao Liu, Yanxu Chen, Jianhui Chen, Junfeng Fang, Lei Hou, Juanzi Li, and Tat-Seng Chua. Are reasoning models more prone hallucination? arXiv preprint WrXiv:2505.33646. 2025.
Michal Yarom, Yonatan Bitton, Soravit Changpinyo, Roce Aharoni, Jonathan Herzig, Oran Lang, Eran Ofek, and Idan Szpektor. What you see is what you read? improving text-image alignment evaluation. Advances in Neural Information Processing Systems, 36:1601-1619, 2023.
Z-Image Team. Z-image: An efficient image generation foundation model with single-strean diffu- sion transformer. arXiv preprint arXiv:2511.22699 2025.
Edward N Zalta, Uri Nodelman, Colin Allen, and John Perry. Stanford encyclopedia of philosophy, 2003.
Shanshan Zhong, Zhongzhan Huang, Weushao Wen, Jinghui Qin, and Liang Lin. Sur-adapter: Enhancing text-to-image pre-trained diffusion models with large language models. In Proceedings of the 31st ACM International Conference on Multimedia, pp. 567-578, 2023.
Tianyang Zhong, Zhengliang Liu, Yi Pan, Yutong Zhang, Yifan Zhou, Shizhe Liang, Zihao Wu, Yanjun Lyu, Peng Shu, Xiaowei Yu, et al. Evaluation of openai ol: Opportunities and challenges of agi. arXiv preprint arXiv:2409,18@ 18486, 2024.
Yucheng Zhou, Jiahao Yuan, and Qianning Wang. Draw all your imagine: A holistic bench- mark and agent framework for complex instruction-based image generation. arXiv preprint arXiv:2505.24787, 2025.
Jinguo Zhu, Weiyun Wang, Zhe Chen, Zhaoyang Liu, Shenglong Ye, Lixin Gu, Hao Tian, Yuchen Duan, Weijie Su, Jie Shao, et al. Internv13: Exploring advanced training and test-time recipes for open-source multimodal models. arXiv preprint arXiv:2504. 10479, 2025.
16

---

Published as a conference paper at ICLR 2026
A BENCHMARK CONSTRUCTION DETAILS
A.1 EVALUATION DIMENSION DETAILS
Composition is the process of integrating multiple visual elements (i.e., instances, attributes, and relations) into coherent image that faithfully reflects the textual prompt, based which we define MI Multi-Instance, MA Mulii-Atribulee MR Multi-Relation, and TR Text Rendering.
Multi-Instanee (MI) refers to generating multiple instances within single image. In setup, in- stances are organized into a coherent thematic scene, with scene details expressed through narrative descriptions rather than disjointed lists to preserve contextual coherence. We also include existential negation (Li et al., 2024a) by specifying absent instances (e.g., there is no apple) alongside those that must appear. To increase complexity, each prompt specifies ~ 25 instances on average, creating high-density scenarios that challenge faithful instance composition.
Multi-Attribute (MA) refers binding multiple attributes to a single core subject. The attribute set spans a wide range of categories: physical properties (e.g.. color, material, texture, shape, lighting), numerical attributes (e.g.. numerals and quantities), states and conditions (e.g., appearance and life- cycle), and abstract and stylistic traits (e.g., emotion and style). Similarly, all attributes are integrated in a unified thematic scene with narrative descriptions and existential negation. To increase com- plexity, each prompt assigns 20 verifiable attributes to single subject, achieving high attribute density while testing precise and consistent attribute binding.
Multi-Relatie (MR) refers to scenes where multiple relations connect instances. We define rela- tions spanning spatial (e.g., on the left), interaction (e.g., holding), comparative (e.g., larger than), compositional (e.g., handle on door), and numerical (e.g., twice many as) relations. Similarly, all relations are incorporated in a unified thematic scene with narrative descriptions. To emphasize more relations rather than more instances (i.e., MI), each prompt specifies more than 10 instances and ~ 15 relations, fostering complex and precise relational structures.
Text Rendering (TR) refers to rendering structured multiple texts within specified scene, focusing on both content fidelity and layout precision. To simulate real-world scenarios, we adopt a hierar- chical text structure in prompts, comprising main titles, section headers, and itemized entries. To further increase textual complexity, we incorporate special formats and symbols, including varied letter cases (e.g., ALL CAPS), curreney signs (e.g., S), punctuation marks (e.g., &). trademarks (e.g., TM), etc. Each prompt specifies ~ 15 texts and corresponding layouts, simulating complex real-world applications, including 2D posters and 3D shop signs.
Deductive Reasoning is the process of drawing conclusions from given premises, ensuring that if the premises hold, the conclusion cannot be false. In T2I scenarios, this means generating im- ages determined by the premises, based on which we define LR Logical Reasoning (multiple premises -> one conclusion), BR Behavioral Reasoning (behaviors inevitable outcomes), HR Hypothetical Reasoning (counterfactual premises - affected items), and PR Procedural Reasoning (ordered procedures + cumulative results).
Logical Reasoning (LR) refers to solving premise-based puzzles through multi-step deductive in- ference rather than direct scene description. In our setup, prompts are formulated as set of in- terdependent premises, which leads to a deterministic scene regarding object attributes and spatial relations. To guarantee diversity of logical structures, we define various reasoning forms (e.g., de- ductive elimination, conditional chaining, causal reasoning) and reasoning scenarios (e.g., spatial arrangement, attribute matching, state transition). Each prompt contains independent premises and requires multiple reasoning hops to ensure reasoning complexity.
Behavioral Reasoning (BR) refers to inferring the visual outcomes that inevitably follow from an initial state and subsequent behaviors falling dominoes). In our setup, prompts specify only the initial state and behavior(s), leading to logically inevitable and visually salient outcomes involving both affected and unaffected items, which the model must then distinguish through reasoning. To increase complexity, each prompt involves compound or sequential actions that deterministically lead to 8 observable outcomes, leading to both logically inevitable and visually salient outcomes.
17

---

Published as a conference paper at ICLR 2026
Hypothetical Reasoning (HR) refers predefining a counterfactual premise that contradicts real- world physics and propagating its effects across both affected and unaffected items within a scene. The model must internalize this rule itself (e.g., every vehicle's 's wheels are perfect squares instead of circles) and enforce it uniformly in different forms of interaction. To increase complexity, prompts designed with 10 objects engaging in varied interactions, where both positive (rule applied) and negative cases (rule not applied) must be correctly distinguished in the same image.
Reasoning (PR) refers reasoning over an ordered sequence of procedures, where visual elements incrementally transform and only the final scene is expected (e.g., folding paper into a crane). In our setup, prompts are structured as multi-step procedures, each building on the previous to produce cumulative and interdependent changes rather than direct outcome description. To increase complexity, prompts are designed as ~ 5 explicit procedures, each building on the previous to create cumulative and interacting transformations, while omitting direct outcomes so the model must infer the intermediate steps necessary to reach the complete result.
Procedural
Inductive Reasoning is the process of inferring conclusions from observed regularity patterns rather than from explicit premises. In T2I scenarios, this corresponds to inferring visual el- ements from underlying structural patterns in examples, based on which we define GR Gen- eralization Reasoning (generalization rules from examples -> new case) and AR Analogical Reasoning (analogical rules from source domain + target domain).
Generalization Reasoning (GR) refers to inducing generalization rules from several examples and applying them to new scenarios with missing visual elements. our setup, each prompt introduces two to three examples that collectively correspond to unified rule pattern, comprising both variant (changing across examples) and invariant (constant across examples) components, which the model must extrapolate to complete new scene with omitted details. To ensure complexity, each prompt is designed to ~ 8 such rules and to ensure generalization complexity.
Analogical Reasoning (AR) refers to transferring specific analogical rules from the source domain (e.g., A relates to B) to structurally parallel target domain (e.g., C relates to D). In our setup, each prompt specifies source domain rules through detailed anchored example (e.g., hexagonal structure of a honeycomb). while the target domain provides only core elements (e.g., clouds arranged like a honeycomb) without describing the analogical outcome. Each prompt is designed as 5 distinct analogical rules, each of which must be consistently transferred from the source to the target domain.
Abductive Reasoning is the process of reconstructing the most plausible explanation from ob- servations. In T2I scenarios, this entails reconstructing hidden causes or unstated common- sense that best explain the visual observations, based on which we define CR Commonsense Reasoning (indispensable elements unstated commonsense) and RR Reconstructive Rea- soning (plausible hidden causes t observed clues).
Commonsense Reasoning (CR) refers to completing scene by invoking commonsense knowledge that is logically required yet unstated. In our setup, each prompt describes a scene with CCR implicit indispensable elements. To ensure complexity, each prompt typically requires ~ independent commonsense inferences, covering six diverse domains from: physical (e.g., a light bulb without electricity + does not shine), chemical (e.g.. mixing vinegar and baking soda bubbles form), biological (e.g.. bat in daytime - sleeps upside down), social (e.g., a doctor treuting patients + wears a white coat), functional (e.g., cutting vegetables - requires a knife), and cultural (e.g., Thanksgiving table in the U.S. - turkey exists) commonsense.
Reconstructive Reasoning (RR) refers to tracing backward from observations to their most plausi- ble initial states in the absence of explicit descriptions. In our setup, each prompt presents a static "observation" containing ~ 5 indirect yet diagnostic clues, akin to evidence at a scene. The model must integrate these clues to infer and render the most plausible "cause" through abductive reason- ing. To ensure diversity, prompts cover varied inferential scenarios, such as event reconstruction, intent inference, state rewind, and environmental storytelling.
18

---

Published as a conference paper at ICLR 2026
A.2 DATA GENERATION DETAILS
To curate the benchmark data in our T21-COREBENCI, we follow a standardized data construction
pipeline using LRMs, with tailored generation instruction for each dimension shown in Fig. 2. This instruction mainly includes three parts: (1) Task Goal, (2) Prompl Design Guidelines, and (3) Checklist Construction Rules. Each sample comprises high-complexity prompt and fine-graines checklist, jointly designed to ensure both semantic richness and verifiability, As shown in Fig. 6, we take MI Multi-Instance dimension as concrete example for detailed illustration.
Generation Instruction LRMs (Multi-Instance)
1. Task Goal
Main Category: Composition
Subcategory:
Specific Goal: To systematically evaluate the model's ability to generate multiple instances within a single image.
I.. Prompt Design Principles
General Principle: Diversity and Scalability
To construct comprehensive and robust benchmark, the test must only be sufficiently large but also diverse across multiple dimensions, ensuring evaluation of general capabilities rather than overfitting to specific templates. Diversity should reflected the following aspects:
Visual & Thematic Diversity: Prompts should cover wide range of sceney (e.g.. indoor, outdoor, outer space), instances (e.g., animals, artifacts, geometric shapes, humans), attributes (e.g., color, material, state, emotion), and themes (e.g., daily life, history, science fiction, fantasy).
2. Structural & Relational Diversity:
The challenge mechanisms of prompts should vary, including changes in logical structures, spatial relations (absolute, relative, topological), aurribute binding complexity (single, multiple, shared attributes). and constraint types (affirmative "is", negative "is not", exclusive "either...or..."
Guideline I: Unified Theme
Explanation: broad and inclusive core scene should be
that all elements remain logically coherent under unified theme, providing a stable background and atmosphere.
Note: All test instances must be common, macroscopic, and visually discernible. Avoid abstract (e.g., labor disputes), atmospheric (e.g., soft sunlight), overly fine-grained (e.g., e. the hands of pocket watch) instances.
Guideline 2: Existential Negation
Explanation: To further test the ability follow exclusion constraints, prompts contain expres- sions specifying certain instances are absent from the scene. To maintain naturalness, negations should be phrased in descriptive or indirect forms (beyond explicit "there no finstance"").
Note: All negation expressions should be organically dispersed throughout the prompt, rather than clustered the end listed separately.
Guideline 3: Precise Quantification
Explanation: Each prompt should specify around 25 independent instances (counting both present and negated ones), with one-fifth of them expressed through existential negation.
Note: Avoid mere cnumcrations; use connected expressions to improve fluency.
Guideline 4: Narrative Description
Explanation: Prompts should avoid simply listing elements separated by commas. Instead, con- nective or locative expressions (e.g., "beside there is ...", "on top of ... lies...", ...", "in the corner stands ...") should be used to describe spatial relations, making the prompt resemble coherent scene description rather than rigid checklist.
III. Checklist Construction Rules
19

---

Published as a conference paper at ICLR 2026
1. Core Objective:
Decompose complex instructions into series of independent, verifiable atomic capability points enable fine-grained evaluation of generated images.
2. Question Format Requirements:
Form: Each quest must be closed yes/no interrogative.
Orientation: Questions must be designed such that the correct answer is "Yes". That is, when the generated image satisfies the corresponding requirement, the answer should "Yes".
3. Principle of Comprehensiveness and Atomicity
Explanation: To enable precise error attribution, the checklist must be both comprehensive and fine-grained, which should be decomposed into the smallest, non-divisible "atomic" points.
Implementation: Avoid assessing multiple attributes with single question. For example, instead of asking "Is the object the center a green cylinder."", decompose into: the object in the center a cylinder?"
"Is the object in the center green?"
Tags Usage Instructions
Explanation: Tags categorize the capability dimension assessed by each question, enabling more ane-ggrined multi-dimensional data analysis.
Tag Scope and Description:
instt ancaenee.eo poo: Evaluates instance presence, i.e., whether specified instance appears in the image. Question template: Is/Are there (a) [instance] the image?
-instance_nee Evaluates instance absence, ie., whether specified instance required to be absent does not appear. Question template: Is/Are there [instance] in the image?
Remark Field Specification
Explanation: No content is required, and leave it as an empty ""
IV. Output Structure
Each benchmark entry is organized in unified structured ISON format, defined as follows:
"IItem ID)":
"Main Class": "The core capsbility category tes:ed by this Stem", *Sub class"; "A specifle sub-dimension",
"Brompt"; "The complete textual instruction imput to the 727 model", "Checklist";
"question": "quastion 12", "cags": ("rag A*] )
"question": "Question 2?", "tags": ("Tag R*)
"Remark": "An optionel metadata field"
Figure 6: Generation instruction for LRMs MI Multi-Instance) in our T2I-COREBENCI.
Prompt Generation Prompt Design Principles. We first include a general principle termed Di- versity and Scalability, which requires variability in both visual themes and structural relations. Subsequently, we introduce sign constraints tailored to each evaluation dimension, including: (1) Unified Theme, (2) Existential Negation, (3) Precise Quantfficatio, and (4) Narrutive Description.
of dimension- specific guidelines, which articulate concrete de-
Checklist Generation in Checklist Construction Rules. Each complex prompt is decomposed into fine-grained, atomic yes'no questions, ensuring that the correct answer is always "Yes". To support precise capability attribution, questions annotated with fine-grained tags, which evaluate the presence anstancc.poo or absence (instance_ned) of specific instances. All samples follow unified ISON schema with optional Remark field for metadata.
Data Filtering and Refinement. To reduce model-specific bias and enrich stylistic and structural diversity, employ three different LRMs', each contributing 100 samples (i.e., prompt + check- list), resulting in x 100 = 300 candidates for this dimension. Afterwards, we apply a multi-stage
Claude Sonnet 4 (Anthropic, 2025), Gemini 2.5 Pro (Gemini Team, 2023), and OpenAl a3 (OpenAI, 2025)
20

---

Published as a conference paper at ICLR 2026
You are an Al quality auditor for text-to-image generation.
Your task is analyze the given image and answer yes/to question based solely its visual content. question may relate to the presence a specific object, its attributes, or relationships between multiple elements in the image.
You will also be given the original prompt used to generate the image. The prompt may provide additional context to help interpret the question, but must never be used supply assume visual details.
Your judgment must rely entirely on the image itself. The image must contain clear, unmistakable visual evidence to justify a "yes" answer the prompt cannot compensate for missing or ambiguous content.
Respond with:
"yes" only if the answer is clearly and unambiguously yes based solely on the visual content. The visual evidence must be strong, definitive, and require no assumptions or guesses.
"no" in all other cases including if the relevant visual detail is missing, unclear, ambiguous, partially shown, obscured, only suggested.
Even if the image closely matches what is described in the prompt, you must rely on visible evidence alone. If the relevant detail cannot be confirmed visually with certainty, answer "no". Ambiguity equals no.
conditional questions, answer "yes" only both the condition and the main clause are clearly and unambiguously true in the image. If either part is false or uncertain, respond "no".
Do not provide any explanation, justifiction,, extra text. Only return a single word: either "yes" or "no".
Example input:
Prompt: "A golden retriever running in a grassy field under the sun." Question: "Is there sun in the image?" ?
Example output: "yes"
Example input:
Prompt: "A white cat sitting red couch in modern living room."
Question: "Is the couch present, is red in color?" Example output: "no"
Figure 7: Evaluation instruction for MLLM evaluator in our T21-COREBENCI.
filtering pipeline: (1) Feasibility check: prompts that fail to produce coherent or renderable images, discarded. (2) Redundancy removal: overly similar or template-like like cases are filtered out to preserve thematic and structural diversity across the dataset. (3) Saaman-in-the-laeee refinement: the remaining candidates are iteratively veri- fied by annotators, who correct borderline cases, refine unclear descriptions, and ensure strict align- ment with the dimension-specifi, guidelines (detailed Appx. A.3). Through this process, the 300 candidates are distilled into compact set of 3 x 30 90 high-quality, guideline-aliged samples.
whose visual elements are ambiguous or unverifiable,
A.3 HUMAN VERIFICATION
Since LRMs are prone to hallucination (Huang et al., 2023b; Yao et al., 2025; Wu et al., 2025d) (e.g., not always reliably following the input instruction), all generated prompts and checklists are subject striet human verification for correctness. Given the inherent complexity in verification, engage
21

---

Published as a conference paper at ICLR 2026
You are prompt rewriting assistant. The given Prompt may involve reasoning steps or logical deduc- tions. Your task is to rewrite the Prompt into clear, dircet, maage-focused description suitable for a text-to-image model. During rewriting, perform all necessary reasoning yourself so that the output contains only the final objects, attributes, and spatial or relational details to be shown in the image. The rewritten Prompt must be fully self-contained, visually descriptive, and contain no reasoning steps or instructions. Write the output as a single continuous paragraph no bullet points, lists, or line breaks.
Examples:
Prompt: Generate an image of three robots in a laboratory. Each robot has a different color (red, blue, green) and holds different tool thanmer, wonner wrenck). The robots make statements: (1) Red mbot say:: 'Rlue robot the hammer' (2) Blue robot says: 'I have the scanner" (3) Green robot says: 'Red robot is lying" (4) Red whot also says: 'I have the wrench. (5) Additional facts: Exactly one robot always lies, the other two always tell the truth. The lying robot has an antenna on its head, while trath telling robots have anterna.
Output: Generate on image of three robots standing a laboratry: the red robot holding a hammer and has an anternaa its head, the blue robot holding scauner without an antenna, and the green robot holding wrench without antenna.
Prompt: Generate photo of Rube Goldberg- stylbeer--tyleeeeeeeee choin reaction in classmom, captured at the final momeras. The initial setup contains a taut elastic cord placed just before line of standing dominoes, with matchstick fixed the midpoint cord under tension. Behind the domino line. the last domino positioned connect mechamisn designed to cut the rope suspending a steel marble. The marble aligned to roll down a ramp into a glass beaker filled with red-caloree water, which desk is closed microscope under dust actions that just occurred: the matchetick used to burn through the tout elastic cord, which, upon snopping, tips over the first domino the line. image should depict scene after all resulting offects have completely finished.
rests white sheet of paper. Far the side this setup the
The
Output: Generate photo of Rube Goldberg-ste stylee chain veacioo in classnsom its final momert:: the snopped ela cord slack with charred break where the matchstick, once burndd through it. the entire line of dominces has falten, held steel marble has been cut, and the marble has rolied down ramp into glass beaker filled with red colored water that is now overflowing, with the spilled liquid spreading across the white sheet of paper beneuth it, while off to the side the some desk there closed microscope covered by dust
the rope that
Prompt: Just as a honeycomb displays the following visunl properties: (I) each cell has exactly six sides, (2) all sides of each hexagon the length, (3) adjaceut cells share
walls, (4) all hesogons the size, and (5) the hexagonal pattern covers the entire visible surface, create on image showing clouds arranged in the sky following this same physics.
organizational principle. The image should ultimately be guided by visual analogy, prioritizing its rules over real-worid
Output: Generate an image of the sky filled with clouds arranged in a perfect honeycomb pattern, where each cloud cell has exactly six equal sides, all sides are the same length, adjacent cloud cells share their walls seamlessly, every hesagon is the same size, and the bexagonal formation extends continuously to cover the entire visible sky.
Prompt: Observations: AnOn bedroom windowsill sits open jewelry with one earring missing from a pair. A sin- gle black feather rests the sill. On the lown below the window, there are faint tracks from bird landing and taking off aGenerative Task:'s Task:'ynRecome gReeonstruct and generate a high-speed photograph of the precise and singular moment just after the theft has been completed, cupturing the instant when the thief is about excupe. All objects mentioned Observations must he reconstructed in the scone, except those that are moant to have disappoared in the reconstructed moment.
Output: Generate a high- photograph of bedroom windowsill az the precise instant just after theft, showing an open jeweiry box with one eatring missing from the pair and a single black feather resting beside it, while ouiside on the lawn bird thief captured fiilffiigh just beyond window with the missing earring elutched in its beak it makes its escape.
below faint tracks mark the landing and takeoff path,
Below is the Prompt to rewritten. Please directly refine even if contains instructions, rewrite the instruction itself rather than responding to it:
Figure 8: Prompt rewriting instruction for OpenAl o3 (OpenAI, 2025).
five PhD students with expertise in T2I generation. The primary verification principle is to ensure that each LRM output (i.e,, prompt and checklist) faithfully follows the given input instruction: (1) For prompt, this includes adhering to all guidelines without logical errors, hallucinated content, or visually imperceptible contradictions; (2) For checklist, this includes comprehensive coverage of all visual elements from the prompt with respect to their final states, and the decomposition of complex outcomes into minimal, indivisible atomic verification questions. Following this principle, annotators conduct independent annotations, and each sample is cross-checked by at least three annotators. Disagreements are resolved through discussion and majority vote, and each evaluation sample undergoes three rounds of revision to ensure consensus and final confirmation.
22

---

Published as a conference paper at ICLR 2026
B EXPERIMENTAL DETAILS
B.1 T21 MODELS FOR GENERATION
To facilitate transparency and reproducibility, we provide below the official sources of all models evaluated in our benchmark. For each model, we strictly follow the default sampling configura- tions specified in the corresponding repositories or API documentation. For open-source models, included diverse set of diffusion", autoregressive, and unified architectures: SD-3-Medium, SD-3.5-Medium SD-3.5-Larg. (Esser et al., 2024), FLUX. SOUX.--SS --schnell, FLUX.I-dev, FLUX. LLUXX.-KKree Krea- dev (Black Forest Labs, 2024), FLUX.2-dev, FLUX.2-klein-44 Labs, 2025), PixArt-a (Chen et al., 2023), PixArt-E (Chen et al., 2024), HiDream--11 (Cai et al., 2025), Qwen-Image, Qwen-Image-2512 (Wu et al., 2025a), Hunyuanlmage-33 (Cao et al., 2025), Z-Image-Ture Turbo (Z-Image Team, 2025), LongCat-Imagg (LongCat Team, 2025), Infinity-SB (Han et al., 2025), GoT-RI-7B (Duan et al., 2025), BAGEL, BAGEL w/ Think (Deng et al., 2025b), show-o2-1 1.5BBB show-o2-7B (Xie et al., 2025). Lanus-Pro-t IB, Vanus-Pro-T BLIP3o-4B, BLIP3o-88 (Chen et al., 2025a), and OmniGen2-7B (Wu et al., 2025b). For closed- source commercial models, we rely their official API endpoints, which guarantee that our eval- uation reflects the current production-level configurations of these services: Seedream 3.0 (Gao et al., 2025), Seedream 4.0 (Seedream al., 2025), Seedream 4.5 (ByteDDance, 2025), Gemini 2.0 Flash, Nano Banana, Nano Banana Pro (Gemini Team, 2023), Imagen 4, Imagen 4 Ultra (Google, 2025), GPT-Image (OpenAI, 2025a), and GPT-Image-I. 1.5 (OpenAI, 2025b). All evaluated models are implemented using their default configurations from the official repositories, with a fixed ran- dom seed applied whenever supported to ensure reproducibility. All experiments are conducted on eight GPUs, with four images generated per prompt to ensure robust and reliable evaluation.
FLUX.2-Klein-9B (Black Forest
(Chen et al., 2025c),
B.2 MLLM INSTRUCTION FOR EVALUATION
In our benchmark, evaluation is conducted au- tomatically using an MLLM as the checklist an- swerer. Specifically, we provide each generated image together with its associated prompt and evaluate it against the checklist in question- by-question manner, where the MLLM receives only a single yes/no question at a time. This design avoids interference between different questions, ensures that each judgment relies solely on visible evidence, and thereby im- proves both the accuracy and consistency of the evaluation. Herein, we list all MLLMs em- ployed in our evaluation together with their of- ficial sources, so that the evaluation setup can be faithfully reproduced. Closed-source mod- els are accessed via their official API endpoints, which guarantee that our evaluation reflects the current production-level configurations of these services: GPT-4o (Hurst et al., 2024), OpenAI o3, OpenAI o4 mini (OpenAI, 2025), Gemini 2.5 Pro, Gemini 2.5 Flash, Gemini 2.5 Flash Lite, and Gemini 2.0 Flash (Gemini Team, 2023). Open-source models are implemented with their default inference settings from their of- ficial repositories: Diven2.5-VL772B-11e
Table 5: Human alignment study across differ- ent MILMs on four compositional dimensions, evaluated with balanced accuracy (%). The best and second-best results are marked in bold and underline for open- and -ooeess
MLLM
MI
MA
MR
TR
Mcan
Qwen2 Mieen.---V--272-ststee
81.3
63.1
64.2
73.7
70.6
InteraVL3-788
70.8
56.8
56.5
67.7
62.9
78.0
61.3
60.3
71.8
67.8
Qwen3-VL-BB -B--I----B-I
72.0
56.2
56.6
65.4
62,5
Qwend-VL-8B- -BB-hhikng
79.6
68.9
70.7
76.2
73.8
Qwen3-VL Innssuue
63.4
60.6
73.3
69.5
WWea3--L-322BTTiiae
72.9
75.4
79.8
77.5
Quen3-V1--30--00 A38---3n-ss
83.1
61.9
59.1
744.2
69.6
Qwen3-VL-30B -LB-
82.5
73.9
75.4
27.2
22.4
GPT-4g
78.3
67.5
63.6
72.0
70.3
OpenAl o3
83.5
77.8
80.4
86.8
82.1
OpenAl 04 mini
81.9
74.7
77.0
83.0
79.1
Gemini 2.5 Pro
83.4
76.5
82.2
88.4
82.6
Gemini 2.5 Flash
83.8
76.9
78.0
85.7
81.1
Gemini 2.5 Flash Lite
69.1
60.1
58.0
74.5
65.4
Gemini 2.0 Flash
73,5
61.0
67.7
77.1
69.8
(Bai et al., 2025b), VL3-78V (Zhu et al., 2025), ELM4.5V-106B (Hong et al., 2025). Dwen3-VL--BB-nnsruue VL-32B-Instrucc Diven3-VL-22BB -
Qwen3- ----------B----- Thinkinnn Qwen3- ---------30B----- Instruct, and Dowen3-VL-300 A3B-Thinking (Bai et al., 2025a). To ensure the reproducibility of results, set the temperature coefficient to zero during all model evaluations whenever supported. The evaluation instruction for the MILM evaluator is presented in Fig. 7, which strictly emphasizes reliance on the image content
Herein, flow-based generative models are framed as variants of the diffusion paradigm within unified contimuous-time (ODE/SDE) framework.
23

---

Published as a conference paper at ICLR 2026
without assuming any detail from the prompt and prior knowledge from the evaluator itself, thereby alleviating hallucinations and ensuring reliable evaluation.
B.3 PROMPT REWRITING DETAILS
The detailed instruction for prompt rewriting in Sec. 4.3 is illustrated in Fig. 8.
C ADDITIONAL EXPERIMENTS
C.I HUMAN ALIGNMENT STUDY
To further validate the effectiveness of employing MILMs as substitutes for human evaluation, we compare MLLM-based judgments with those of human annotators. Specifically, we focus four dimensions (i.e., MI, MA, MR, and TR) ) which capture the fundamental visual elements of evalua- tion: instance, attribute, relation, and text. As the questions in the remaining eight reasoning dimen- sions can also be decomposed into these same elements, evaluating these four dimensions could be sufficient. In experiments, the human annotation results, we hire professional annotators who are highly experienced in image and video annotation. The annotation pipeline begins with the distribution of detailed guidelines, followed by training and trial annotations to ensure consistency. The annotators then carry out the primary annotation (first round), after which the results undergo secondary and tertiary rounds of verification through full inspection, ensuring high-quality and reliable results. Considering the im- balance in the human human-annotate annotated ground -ruth results (e.g., the number of correctly generated visual elements in GPT--mmaee generations is substantially greater than that of incorrect ones), we introduce balanced uccuracy (Brodersen et al., 2010) to provide fair and robust evaluation.
images from GPT-Image along these four dimensions. For
As shown in Table 5, closed-source MLLMs significantly outperform open-source ones in recog- nizing these fundamental visual elements, with OpenAl o3 and Gemini 2.5 Pro achieving the best performance. Considering the trade-off between performance and API cost, we select Gemini 2.5 Flash as our evaluator for scage evaluation (i.e., its API cost is about 1/4 of that of Gemini 2.5 Pro, while performance drops by around 1%). Meanwhile, considering the possible unavailability of closed-souse cousrd APIs in the future, we also report evaluation results using Q VLL-7BB-Instrrett VL--3----------T-ieee
-wen3-VL--BB-TE -hinkinng, convenient evaluation, which are available our
and 832 Leaderboard.
for more
C.2 FINE-GRAINED ANALYSES
Notably, we further annotate each question from the checklist with fine-graines labels to capture their complexity and types for subset of dimensions, including: composition (MI, MA,, TR and reasoning (LR, BR. HR. GR), ) which facilitates fine-grained analyses, including:
MI Multi-Instances The positive (POS) label is used to evaluate instance existence, verifying whether a specific instance mentioned in the prompt is exactly present in the image (e.g.. "there is an apple"). In contrast, the negative (NEG) label is used evaluate instance non-existence, verifying whether an instance explicitly required to be absent in the prompt does not appear in the image (e.g.. "there is no banana").
MA Multi-Attribute: The positive (POS) label is used to evaluate attribute accuracy, verifying whether the attributes of an existing instance, such as color, material, or state, are correctly ren- dered (e.g., "a red ball"). In contrast, the negative (NEG) label is used to evaluate attribtte exclusion, verifying whether the instance adheres to the constraint of not possessing a specific attribute (e.g., "a ball with no red color").
TR Text- Rext-Rendering: The content (CON) label is used to evaluate the accuracy of the generated textual content, focusing on what rendered, such as whether the spelling of words is correct whether special symbols are properly displayed. The layout (LAY) label is used to evaluate the accuracy of the text's position, layout, and spatial relationships, focusing on where the text appears, such as whether title is placed at the top.
LR Logical Reasoning: The 0-hop label corresponds to cases where the prompt requires only direct observation without additional inference (e.g., "a red cube on the table"), the 1-hop label
24

---

Published as a conference paper at ICLR 2026
Table 6: Fine-grained analyses on our T2I-COREBENCH for both composition (MI MA. 7R) and reasoning (LR. BR, HR, GR) dimensions, evaluated by Gemini 2.5 Flash. Values highlighted in red indicate special exceptions, which are further discussed in the analysis. The best and second- best results are marked in bold and underline for open- and closed- closed-models. respectively.
Composition
Reasoning
Model
MI
TR
LR 1-hop
GR
NEG
O-hop
m-hop
POS
NEG
POS
Diffusion Models
SD.3-Medium
$2.6
89.5
54.8
78.0
7.2
10.3
30.0
20.5
20.3
11.7
57.9
7,8
64.6
58.4
22.9
SD.3.5 M.ediu
$1.6
95.8
57.3
82.0
8.1
11.5
24.9
21.8
18.2
12.2
53.4
8.3
60.3
50.0
21.9
SD.3.5. Largee
49.8
93.3
56.5
83.4
13.4
16.2
28.8
24.1
21.0
14.1
56.2
11.5
58.3
53.4
23.1
FLUUX.I-schre I-schnell
$9.8
91.5
60.3
81.7
20.2
21.9
30.4
20.9
24.4
12.2
74.8
10.8
73.9
67.2
33.8
F-UX.I-ddv
$1.1
93.9
57.0
81.8
29.2
30.1
30.0
26.8
24.3
11.5
68.0
8.1
65.8
59.5
30.0
FLUXXI-Kreaadd
67.5
85.5
69.7
82.0
26.9
26.7
38.4
26.8
29.4
13.6
75.2
15.6
77.1
70.8
36.2
FLUX.2-dev
87.8
98.3
83.0
85.5
93.7
92.1
70.6
41.8
41.5
19.2
88.3
24.9
85.6
90.9
43.3
FLUXX22klein
76.7
96.4
76.3
83.8
42.8
62.3
46.4
32.7
30.4
17.3
82.6
19.5
82.5
86.0
39.1
FLU.X22kleiin99
83.4
97.4
79.9
84.8
75.2
80.8
62.8
40.9
37.1
18.6
88L3
25.8
82.5
87.7
41.6
PixArt-
28.2
96.9
36.7
75.8
2.7
3.9
12.7
13.2
11.4
7.9
26.3
6.3
36.0
31.9
16.2
PixAn-E
36.3
98.2
45.1
78.4
1.7
3.6
19.2
114
13.7
12.2
43.0
9.4
44.2
34,3
19.4
HiDream-II
54.8
98.5
58.9
82.7
32.0
30.5
40.9
25.5
33.5
13.1
68.8
12.6
71.5
48.5
23.6
Qwen-Image
85.6
61.9
79.8
79.2
84.3
86.1
58.6
40.5
35.8
18.2
86.6
16.3
83.5
82,1
39.3
Owen-Image-2512
88.0
91.7
82.7
82.6
91.1
92.5
60.8
38.2
37.7
20.9
88.3
21.6
88..7
88.0
45.0
Z- Z-Image-Turbo
78.6
84.6
71.2
80.5
84.1
85.4
51.5
34.6
31.9
15.3
80.9
16.5
84.1
77.0
42.0
CaggCaaaaageee
79.2
92.8
73,1
82.9
63.3
60.5
55.1
38.2
34.2
23.4
82.9
18.5
81.3
80.3
67.9
Autoregressive Models
Indinity-88 -B
57.2
94.8
60.6
82,9
7.3
12.9
40.6
31.4
24.3
14.6
70.0
16.0
72.2
66.6
34.1
GoT-R1-7B
38.3
97.3
50.9
83.6
3.8
7.2
27.6
19.1
20.3
11.2
50.8
8.3
55.2
50.5
24.2
Unijied Model
BAGEL BAGEL Think
96.4
62.5
83.6
4.3
30.1
22.0
11.0
65.3
8.6
58.0
40.3
24.3
82.2
23.0
17.2
58.1
11.7
56.3
sbow-o2-1.58
81.7
10.1
sbow-o2-7B
59.7
10.3
Junus Pro-BB
10.7
Pro-7B
11.2
59.4
11.6
59.0
BI.IP3o44D
41.7
34.6
BI.IP3--SB
44.9
20.2
35.9
10.0
OmmiGen2-7B
32.9
73.5
12.4
77
HunyuanImago-MM
81.1
82.7
84.4
54.5
32.7
36.4
19.8
82.2
26.1
79.8
78.4
Slssrd-Souree Models
Seedream 3.0
82.2
68.4
76.9
84.3
39.0
$2.1
49.1
37.7
31.8
21.0
83.2
19.6
84.6
81.6
36.4
Seedream
91.0
94.0
84.5
84.9
95.2
92.6
82.2
82.2
74.4
45.9
86.6
42.6
80.5
90.9
82.2
Seedream 4.5
90.9
97.6
87.8
82.6
97.8
94.6
86.8
81.4
64.4
50.2
90.4
47.8
82.3
91.9
84.8
Gemini 2.0 Flach
61
96.7
66.1
84.6
60.9
66.2
50.9
40.9
34.1
28.8
81.9
28.9
68.4
77.9
45.5
Nano Banana
83.0
98.7
76.5
87.6
84.7
87.4
83.9
77.3
$7.3
60.2
83.1
50.6
84.7
88.6
80.6
Nano Banana Pro
90.8
97.8
85.2
86.1
98.7
97.9
92.3
91.8
91.1
69.8
88.9
66.9
88.9
91.6
89.3
Imagen
81.0
92.7
72.7
85.6
90.4
88.9
60.5
4644
38.3
42.9
85.6
34.5
81.8
84.6
75.0
Imagon Ultra
88.4
97.7
79.1
86.9
85.6
85.8
81.3
70.9
55.2
56.6
84.1
51.7
82.1
86.6
79.6
GPT-Image
80.9
99.2
74.3
86.0
85.7
87.5
77.1
65.5
52.3
46.8
85.6
54.7
76.6
80.8
72.7
GPT-Image-115
84.5
98.6
82.8
87.5
95.6
94.3
84.3
68.2
54.4
63.3
81.4
61.9
79.2
89.5
79.6
corresponds to cases that require a single step of logical inference (e.g., "the larger of two objects is on the left"), whereas the multi-hop (m-hop) label corresponds to that require multiple chained inferences (e.g., "if the dog is behind the fence, and the fence is behind the house, then the dog behind the house").
BR Behavioral Reasoning: The positive (POS) label is used to evaluate the model's core be- havioral reasoning capability by verifying whether the image presents the inevitable visual conse- quences triggered by the behavior described in the prompt but not explicitly stated (e.g., "a glass is knocked over the water spills onto the floor"). ) In contrast, the negative (NEG) label is used to identify elements that remain unaffected by the behavior, preserving their original state (e.g., "knocking over glass of orange juice does not affect the egg placed beside if").
25

---

Published as a conference paper at ICLR 2026
hR Hypothetical Reasoning: The positive (POS) label is used to verify the visual results that directly follow from the hypothetical rule, where the corresponding objects satisfy the assumed premise and therefore should exhibit the specified change or characteristic (e.g.. "ifthe the wheels are assumed to be square, the car should display square wheels"). Conversely, the negative (NEG) label is used to verify that objects not meeting the hypothetical premise remain unaffected, ensur- ing that the model does not mistakenly apply the hypothetical rule to inapplicable objects (e.g., "other parts of the car not mentioned in the hypothesis should remain unchanged").
GR Generalization Reasoning: The invariant (INV) label used to evaluate features the target scene that remain unchanged, representing the "common constant attributes" summarized across multiple examples (e.g., "all birds have wings"). In contrast, the variant (VAR) label is used to assess whether the model can follow cross-example variation logic to generate systematic changes in certain attributes within the target scene (e.g., "the color of each bird changes across different scenes while their shape remains the same").
We report the fine-grained analyses in Table 6, and conclude the following interesting insights: (1) Most models find NEG cases easier than POS, though few notable exceptions emerge. Across MI, MA. BR. and HR, models consistently score higher on NEG cases than POS ones, suggesting that is generally easier to avoid conditions than to satisfy them. This trend is especially pronounced in reasoning tasks [RR, HR), where models are better at confirming the absence of change than at predicting correct outcomes. Interestingly, few advanced models deviate from this pattern: both Qwen-Image and Seedream 3.0 slightly favors POS over NEG in MI indicating their limitations in handling negative constraints. (2) Performance on the two sub- dibbeiimsssons of Text Rendering is strongly correlated, suggesting that both content and layout must be jointly optimized. In the TR dimension, models that achieve high accuracy in textual content (CON) also tend to perform well in layout fidelity (LAY), and vice versa. This strong correlation implies that effective text rendering requires coordinated progress in both semantic correctness and spatial arrangement, as deficiencies in either aspect can significantly impair overall performance. (3) A clear stepwise effect observed in Logical Reasoning, with multi-hop problems being consistently more difficult than O-hop/1- hop ones. Across models, performance in LR declines noticeably as the number of reasoning hops increases, with multi-hop questions scoring lower than 1-hop, which in turn score lower than O-hop. This pattern reflects the increasing complexity introduced by multi-step dependencies, indicating that current models struggle to maintain reasoning consistency over longer inferential chains. (4) In Generalization Reasoning, models handle invariant patterns more reliably than variant ones. Within the GR dimension, scores on the invariant subset (INV) are consistently higher than those on the variant subset (VAR). This indicates that models are more adept at identifying and preserving shared, stable patterns, but struggle when required to generalize over systematic variations. The performance gap reveals core challenge in enabling models to reason beyond fixed regularities toward flexible pattern adaptation.
C.3 QUANTITATIVE EXAMPLES AND COMPARISONS
Due to page limits, we include the complete set of illustrative examples and cross-model qualitative comparisons in Fig. 9 and Figs. 10, 11, 12. These figures showcase composition and three key dimensions of reasoning (i.e., deductive, inductive, and abductive), providing a fuller picture beyond the main quantitative results in the text.
D LLM USAGE STATEMENT
In this work, L.MS are used solely as general-purpose assistive tools. Specifically, we use them to (1) provide suggestions for improving grammar and clarity of writing, (2) help organize section structures, and (3) assist in generating candidate prompts and checklists during the benchmark con- struction stage, which are subsequently verified and refined by human annotators. Importantly, all research ideas, experiment designs, and final scientific claims are developed and validated by the au- thors themselves. The LLMs do not contribute to the originality of research concepts conclusions, and are therefore not considered contributors or co-authors. The authors take full responsibility for all content presented in this paper, including any text initially drafted with L.M assistance.
26

---

Published as a conference paper at ICLR 2026
E LIMITATIONS AND DISCUSSION
Limitations. While our T2I-COREBENCH provides a comprehensive and challenging benchmark for assessing both compositional and reasoning capabilities, we also observe several limitations in evaluation: (i) Our study focuses solely on T21 generation, leaving out other emerging modalíties such as video generation and interactive multimodal generation, which pose additional temporal and contextual reasoning challenges. (ii) Although our checklist-based evaluation ensures consistency and objectivity across dimensions, certain aspects could benefit from finer- aner-grained metrics. For ex- ample, text rendering is currently assessed at the sentence level, whereas character-level accuracy could offer a more detailed perspective. (iii) Our benchmark primarily evaluates generative faithful- ness with respect to prompt semantics, without considering non-semantic aspects such as aesthetics, realism, and diversity. The dataset largely focuses on objects and animals, with limited coverage of human-centric or face-related cases, which may reduce relevance to certain real-world applications. Expanding the benchmark to include human-related scenarios, together with broader non-semantic dimensions, is an important direction for future work. (iv) Our benchmark is currently limited to En- glish prompts, while multilingual capabilities remain largely unexplored; extending the benchmark multiple languages represents important direction for future work.
To address the identified challenges of T2I generation in complex composition and rea- soning scenarios, we identify four promising research directions for future work: (i) The develop- ment of more diverse and challenging training data, particularly with multi-element and reasoning- oriented supervision, is essential for enabling stronger generalization across complex tasks. (ii) The integration of LLMs and MLLMs into T2I pipelines should be advanced, leveraging their strong language modeling and cross-modal reasoning capabilities to improve semantic understand- ing and alignment in complex generation scenarios. (iii) The incorporation of LLM-style reasoning paradigms, e.g., Chain-of-Thougee Augmental-ugmeeneed Generation (Gao et al., 2023), into T2I pipelines facilitate intermediate inference before image generation, thereby improving the extraction of implicit visual elements from complex prompts. (iv) The exploration of reasoning mechanisms during generation is also needed, by explicitly integrating visual reasoning steps into the generation process to support more detailed and controllable outputs. We hope this benchmark and analysis can facilitate future research toward building T2I models into both "set the stage" and "direct the play".
Discussion.
(Wei et al., 2022). Self-Coonistency (Wang et al., 2022), and
27

---

Published as a conference paper at ICLR 2026
Mulli-Insiance (MI): A liveyy elementarry school classroon during art period, where colorful student artwork decorates the walls above rows of small desks and bright yellow chairs. The teacher's desk holds red apple, coffee mug, and seattered peneils, while large whitehoard displays today's lesson plan written in blue marker. Near the windows, patted plants thrive an the windowsll next to boxes tistuss and hand sanitiner. Art supplies overflow from plastic bins: crayons, scissors, glue sticks, and constraction paper every imaginable color. However, won't find any electronic tablets or computers in this traditional classroom, as the school maintains hands on learning approach. There are also no musical instruments like drums gultars present, keeping the focus purely visual Students' backpacks hang hooks along back wall, while a globe sits prommently on a corner table beside stacks of picture books. The bulletin board showcases gold star stickers and student certificates, and alphabet letters march along the wall border above the chalkboard.
"35 Instances"
8D 3.5 Medum
FLUKI KOCD dov
Oaees, maga
Socdrcom 40
Nano Banana BEION
GPT Image
21/35 0.60
22,35 0.63
28/35 0,30
33/35 0.94
30/35 0.96
29/35 0.33
Multi- Multt-Atrribte Attribute (MA): A single mechanical clarkwork dragon constructed from brass and copper gears in Victorian inventor's workshop. The dragon medium-sised with articulated joints and visible clockwork mechanisms throughout body. scales individual overlap like medieval armoz, and its eyes glowing amber gemstones. The dragon has four legs with mechanical claws, and its wings are made copper sheets hrass ribbing spine, releasing small puffi of white vapor. The dragon's head features three rotating gear assemblies visible through transparent crystal panels. Its tail segnented with spring loaded joints and ends in a sharp brass apear paint. The dragon's chrst houses a large central clockwork heart glows with warm golden light and produces visible ticking mation. Intricate engravings Victorian flourisbes decorate the brass surfaces, and tiny copper wires connect various mechanical components. The dragon is not corroded, maintaining its polished metallic appearancs. Small brass screws and bolts are visible at every joint, and delicate flligree work adorns the wing membranes. The dragon shows no signs of rust and contains modern electronic companents.
plates
not organic, being entirely mechanical constraction.
steam vents are pasitioned along
'25 Attributes
BD-3 5-Meciamm
FLUK---KNeedee
Soan--hagg Iragee
Suncharn 4.
Nara Barner
OPT-
(5/25-
18/25 0.72
23/225
23/25
19/25
15/25
Multi- Relli-- (MR): kitchen scene with an experienced chef wearing a white hut standing behind an old, worn wooden counter. The chef is holding large kniffe cutting carrots on cutting board, shiny, new pot sits on top gas stove to the counter; the pot noticeabll newer than the counter. A black cat is sitting under the counter facing the chef. On the counter, there are three apples and group of currots. The number carrots is twice the number apples. The three apples are arranged in frunt of the cutting board. Also on the counter are and their number that lies open between the apples and the catting bourd. The recipe book is thicker than the cutting board. A kitchen towel hangs from a book on the behind stove, shaker sits next to the recipe on the counter.
less than the number apples. A wonden spoon inside the pot. The chef pointing recipo book
3D 35 Medium
FLUX1 Kroa dor
Owion imagc
Scccrcam 4.
Naro Bononz
GPT Imago
8/21 0.38
(4/221 0.67
16/2/ 0,76
17/21 0,31
h/21 0.81
17/21 0.31
Text- Rendering (TRI: Creste pharmaceutical product packaging box with detailed multi-level text bierarchy. The main product MEDHEALTH PLUS should be displayed in large blue letters on the front panel. Below that, show Advanced Pain Relief Formula' smaller black text. The parkage should have four information sections: 'ACTIVE INGREDIENTS' (top left), DOSAGE INSTRUCTIONS (top right). "WARNINGS & PRECAUTIONS (bottocr-efft) and MANNFACCTRER INTO' (bottum-zight). Under ACTIVE INGREDIENTS, list 'Buppooee 400mg'. Aceeaminophen 325mg', and Caffeine 65mg.. Under IOSSAGE INSTRUCTIONS, show 'Adulte 1-2 tablets', Every 6-8 hours', and "Max dablets/day'. Under WARNINGS PRECAUTIONS, display Do not exceed dosage', Consult doctor if pregmant' and Keep away from children! Under MANUFACTURIR INFO, list 'MediCorp International, 'Lot i M11-2224-456', and Txp. 12/2025 Add small plus symbol (+) next to 'huprofen 400mg and Wdults: tablets' only. not add symbols and Store below C"
any other elements the side panel, include FDA APPROVED
'20 Texts 20 Layouts"
Mocium
FLUX Koca dDV
Owen -mage
Seodream 4,0
Nano Banana
GPT-
MEDOISALTH
MEDIHEALTH
7/40 = 0.17
12/40 0,30
32/40 0.80
38/40 0.95
37/40 0.93
38/40 0.95
Figure 9: Quantitative examples of composition dimensions (i.e., MI, MA,
TR
28

---

Published as a conference paper at ICLR 2026
Logicut Reasoning (I.R): Generaate an image of three robols in lab, Each robot has a different color (red, blue, green) and holds a different tool
'Bhee robot has hammerr: Blue rohot saya: have the (3) Green robot says: Red robot is lying (4) Red robot also says. have the wreach: (5) Additional facts: Exactly one robot always lies, the other always tell the truth. The lying rohot
(harmmer, wrench). rohots make statements, (1) Red robot
its head, while truth telling robots have antenna.
"5 Premisess
Checkllst::
red rohot
its head?
04, the rohot holding the hammer?
02. Does the blue robot have no antenna on its head?
05. the blue robot holding the scammer?
Does the green robot have
its head?
06. the green robot holding the wrench?
3D 3.G Medium
FLUXT Kres dav
Owen
mago
Socdrcam 4.0
Nano Banana
SPT Imago
0.50
0.67
5/6
1//6
4/66 0.67
Rehavioral Reasoning (BR): Generate platto of a Rabe Goldberg style chain resction in claseroom, captured at the final moment. The initall setup contains taut clastic cord placed just before line standing deminoes, with matchstick fived at the midpoint of the cord under tension. Behind domino line, the last domino is positioned to connect to a mechanism designed to cut the rupe suspending a steel marble The murble glass beaker filled with cel-cedered water, which rests white sheet of paper.
aligned to roll down ramp on the same desk closed microstopee under a dust cuver. The actions that just occurred: the matchstick is used burn through the taut elastic cord, which, upen snapping. tips over the first domino the line. The image should depict the after all resulting effects have completely finished.
the side of this setup
"8 Outcoanes*
Checklist:
01. the matchstick charred or blackened after burning?
05, the steel marble inside the glass beaker?
02, the elastic cord visibly broken after being burned through?
06. Is the white paper under the beaker stained with red sploshes?
03. Are of the dominues in the line lying flat desk?
07. the micrescope still on the desk, far from experiment?
04. Does the rope holding the steel marble uppear be cut?
68. the dust cover still on the microscope and completely dry?
9h-38--Wedim
Il UK dew
Qoown- magg
Soodrcom 4.0
Neno Faraea
:PT-image
WAUZZARR
o/3 0,00
1/8 0.13
2/8 0,2
2/8 0,25
5/3 0.63
4/1 0.50
Hyputheticul Reusoning (HH): Depict bustling city-street intersection. In this world, every vehicle's wheels perfect squares instead of Present: 1) a yellow taxi car, red double decker delivery bicycle, 4) police motorcycle, gray sedam, 6) street-swen sweeeer trakk. 7) pedestrian's shoes, 8) strect lamp, 9) fire hydrant, a public trash bin, 13) blue mailbox, and 12) a traffle light pole. Render daylight realism.
Items"
Checklist: 01. the wheels depicted as perfect squares?
07. pedestrian's shoes keep their normal soles?
02. Are the bus's wheels depicted perfect squares?
05. Does the street lamp keep its normal form?
U3. bicycle wheels depicted as perfect squares?
hydrant leeep normal form?
04, Are the motoreyele wheels depicted perfect squares?
10. Does the trash hin keep its normal form?
US. sedan's wheels depicted perfect squares?
keep
06. Are the street sweeper truck wheels depicted perfect squares?
12. Does the traffle light pole keep its normal form?
FLUK1 Krea dcv
Owen mago
Socdrcam
Imago
4/12 0.35
6/12 0.50
6/12 0.50
6/12 0.50
4/12 0,42
6/12 0.50
Procedural Reusoning (PR): Illustrate final scene after performing all of following sleps in ordervatt, Place a square sheet purple arigami paper flat on wooden table.1n2. Fold the paper diagonally corner to corner and crease sharply, then unfold in3, Fold along the other dingorad and crease, then unfold to reveal an X-shuped crease pallern ind, Collupse the paper inward along the creuses to form a square base, las. Continue folding create the traditional bird base, then pull out the neck, head, and two wings form crane yn6. Spread the wings gently the crane stands upright and centre it un the table 'aRender the tabletup exactly as appears once six steps are complete.
Procedures"
Checklist:
a finished origami crune made from purple paper present un the table?
Are both wings extended outward horizontally?
04. the crane's is head distinct and bent alightly downward?
Is crane standing upright without external support? 05. unfolded scrap paper
30-355-Muiiu
FLUX wow
Sundhwan 4.0
Nami Bar
GPT-inngge
615 0.00
0/5 0,00
4/5 = 0.0
5/5 Lo00
5/5 1,00
5/5 1,00
Figure 10: Quantitative examples of deductive reasoning dimensions (i..,, LR, BR HR, PR
29

---

Published as a conference paper at ICLR 2026
Generalization Reasoning (GR): This system that creates a Diiviee Chariot' based on a Deity's Domain: Study the examples to understand the rules'n'n cules'n'nE The source is the Sky' domain, with charioteers, a primary metal of gold. and Sun' symbol. The result golden dhariot that floats without wheels. lt pulled by 2 griflins (Rule: Sky -> Grillins/Foutingg Sea Hippucampi Wheels; creature count charioleer count). A large, golden rays of light (Rule: Sun > light rays. Trident water trails). The chariot made of glowing energy and metal and set against cloudy sky. innnEsamplee The chariot with wheels made swirling water. It is pulled by hippocampus (Rule: Sky -> Grillis/Foonling Sea -> Hipporampi/V Chaenp////Wele Wheels; creature count charloteer count). A large, silver trident emblem is the front of the chariot (Rale: metal determines chariot and emblem material), Thee chariot is followed by tmals swirling water (Rule: Sun light mys. Trident -> water trails). The chariot is made glowing energy and metal and is against stormy Now apply this exact system. Generate with charioleers, primary metal of silver, and a 'Sun' symbol.
emblem is the front of the chariot (Rule: metal determines chariot and emblem material), The chariot emits soft
the SSea' domain, with charioteer, ptimary metal of silver, and 'rident' symbol. The result silver
the chariot from the following
The 'Sky' domain, B Genendization Rules"
Checklist: Is the chariot being pulled by griffins!
05. there a large sun-shaped emblem on the front of chariot??
02. Are there exactly 3 griffins polling the charlot?
06. la the material of the embllem also silver?
Does chariot flool and have no wheels?
07. Does the chariet emit sall rays light?
04, la the body of the chariot made of silver?
08. the setting sky with clouds?
3D 36 Medium
FLUXI1 Kina dev
Owen Image
Scedrcam 4.0
Nano Banana
SPT Image
5/3 6,38
0.25
6/3 0.63
7/8
Anulogicul Reasoning (ARU: Just buneycomb displays the following visual properties: (1) each cell has escctlly six sides, (2) all sides of each hexagon the same length, adjacent cells share walls, (4) hesagons entire visible surface, create an image showing clouds araanged in the sky following this same organizational principle. The image should ultimately guided
sine, and the hevagonal pattern the
visual analogy, prioritizing rules real- world physics.
Analogical Rules"
Chocklist: each cloud formation exactly six sides?
cloud bexagons the size?!
02. Are all sides of each cloud hexagon the same length?
05. Does the hesagonal cloud pattern cover the entire visible sky area?
adjacent cloud Formations share
walls?
52-- B-Vediumm
T-HONN--ORY
Chown- image
Seecteam 6.0
Namo Baraan
GPTT-mage
0/5
3/5
4/5 0.30
Figure 11: Quantitative examples of inductive reasoning dimensions (i.e., GR, AR] )
Deseribe realistic seene, including any necessary real world details make believable. In veterinary clinic's examination room, a velerinarian conducting a check up on nervous terrier. The vet leaning over the animal listen carefully to its heartbeat. The stands close stroking the dog's head to has been prepared next to small vial. On the wall, an X-ray film is clipped onto an illuminated light box.
Commonsense Reasoning
calm. On the stainless steel counter in the background, single syringe
Commonsense"
Chocklist: 01. Is the veterinatian using a stethescope to listen to the dog's heartheat? elevated metal examination table?
Is dog positioned
Is the velerinarian wearing professional allire suitable for medical enviromment, such as scrubs or lab cual?
Is the needle the prepared syringe the counter still covered with its protective
us. Does the illuminated X-ray on the wall display the skeletal structure of an animal?
3 -ediumm
RLUKI-Kma--O
Quvnn
Seecteam 4.0
Nano
3PT--mago
2,55 0.40
2/15 0.40
2/55s 0.40
3/5 s 0.60
3/5 = 0.60
3/5 0.60
Reconstructive Reasoning (RR): OhservationelaCts hedroom windowsill single black feather reats on the sill. On the lawn below the window, there are faint trackes from bird landing and taking offieiineneeettee TecesaReenttttttt and generate high-speed photograph of the precise and singular moment just after the theft has been completed, capturing the instant when the thief is about to escape. All objects mentioned in Observations must be reconstructed in the scene, except those that are Clues"
open jewelry box with earring missing from pair.
meant have disappeared the reconstructed moment.
Checklist:
Is bird, such
magpie, visible the scene?
Is there open jewelry box the windowsll??
Is the bird holding a shiny earring in its beak?
05. Is single matching eurring still visible inside the jewelty box?
la the bird depicted in mid- flght, taking off from windowsll??
50-3.5 um
RLUXX dev
Seetheem
4.0
Neno Farana
GPT-Amage
0.2.
2/5 0.40
3/5 0.60
5/5
1,00
Figure 12: Quantitative examples abductive reasoning dimensions (i.e,, CR, RR )
30


Reviews:
[{"review_id": "PwsL8HnnRy", "reviewer": "Reviewer_pbo7", "summary": "This paper introduces T2I-COREBENCH, a high-density, checklist-based benchmark to jointly test composition (multi-instance, multi-attribute, multi-relation, text rendering) and reasoning (eight subtypes derived from deductive/inductive/abductive patterns) for text-to-image models. Evaluating 28 recent models, the authors find a clear gap: models can “paint” the specified elements but struggle to “think” through multi-step or implicit requirements, especially in dense scenes. Reasoning, not composition, is the current bottleneck.", "questions": "- How did you de-correlate prompt/checklist style from the evaluator to avoid agreement-by-style?\n- Did you experiment with soft scores or multiple MLLM votes, and if not, why stick to strict yes/no, which can make dense prompts disproportionately harsh?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- Well-motivated split between composition vs. reasoning, with 12 concrete dimensions.\n- Dense prompts + yes/no checklists enable fine-grained, scalable evaluation.\n- Broad study on 28 models gives useful community signals about where T2I still fails.", "weaknesses": "- Evaluator dependence and possible bias. The pipeline crucially depends on an MLLM to judge success. The paper would be stronger with a multi-evaluator or human-heavy subset to rule out evaluator-specific artifacts .", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "u4wGoRHEPS", "reviewer": "Reviewer_UbDT", "summary": "The paper presents T2I-CoReBench, a benchmark for evaluating the compositional and reasoning abilities of text-to-image (T2I) models. The benchmark dissects model performance across multiple reasoning dimensions—logical, behavioral, hypothetical, and generalization reasoning—along with compositional skills such as multi-instance, multi-attribute, and text rendering. The authors argue that current T2I models excel in simple composition but struggle with structured reasoning that connects textual causality and visual outcomes. Comprehensive experiments compare state-of-the-art open- and closed-source models, uncovering systematic reasoning failures.", "questions": "1. Is there any bias in the prompt generated synthetically? \n2. Is there any distinction between the generated images of open and closed source models?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The authors categorize reasoning into fine-grained dimensions (e.g., logical, behavioral, hypothetical, abductive), offering a structured framework that is more interpretable than single-score benchmarks.\n\n2. The inclusion of over a dozen models, spanning proprietary and open-source variants, makes the findings broadly representative and informative.", "weaknesses": "1. Some reasoning categories overlap (e.g., behavioral vs. hypothetical), raising questions about independence and orthogonality among evaluation axes.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "anmuh8XhZS", "reviewer": "Reviewer_jqux", "summary": "This paper introduces T2I-CoReBench, a Composition and Reasoning Benchmark for systematic evaluation of T2I models. T2I-CoReBench addresses research gaps in comprehensiveness and complexity, evaluating both composition and reasoning within a broader taxonomy. The dataset is constructed by an automatic pipeline, using LRMs to generate data points given instructions, and further examined by humans. Each data point is accompanied by a checklist, utilizing Google Gemini Flash 2.5 as the evaluation protocol to assess checklist accomplishment and provide a quality evaluation score. Comprehensive experiments evaluate 28 T2I models with different architectures and find that most models fall short on overall performance.", "questions": "See weaknesses above.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "1. Solid contribution in constructing a comprehensive and challenging benchmark on T2I. T2I-CoReBench comprehensively covers composition and reasoning tasks in a detailed taxonomy. \n2. Broad evaluation of different models offers great insights: The paper benchmarks 28 T2I models, covering open-source and closed-source models, as well as different architectures (Diffusion, Autoregressive, Unified). This makes the results comprehensive and convincing. Insights on \"Prompt Rewriting\" in Section 4.3 are intriguing.\n3. Good Presentation Quality.", "weaknesses": "1. Evaluation relies on a single indicator, which cannot comprehensively measure the quality of T2I: There is only one indicator, the achievement rate of checklist questions, which cannot measure such as the overall rationality and stylization of the pictures. Given the examples, it also mainly examines item appearance in images, regardless of model hallucinations, etc.\n2. Both the composition and reasoning tasks have their own previous benchmarks. Merely merging the two tasks into a more comprehensive benchmark limits their necessity.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "M3aCJz2zVE", "reviewer": "Reviewer_DGHz", "summary": "The authors introduce T2I-CoReBench, a benchmark for evaluating both composition and reasoning capabilities of text-to-image (T2I) generation models with 12 dimensions. The benchmark comprises 1,080 prompts and ~13,500 checklist visual questions, enabling fine-grained evaluation with an MLLM-based yes/no answerer. Experiments on 28 T2I models (diffusion, autoregressive, and unified architectures) reveal that while composition is steadily improving, reasoning remains the main performance bottleneck. Further, prompt rewriting shows some effectiveness in bridging this gap but remains limited in scenarios requiring deeper visual reasoning.", "questions": "1. This is minor point, but I feel the title is hard to understand. I didn't understand what title meant before reading the paper, and didn't expect to read text-to-image evaluation benchmark paper at all. Now I guess that authors meant composition and reasoning by 'set the stage' and 'direct the play', but still, such connection wouldn't be clear for many new readers.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The authors provides the first benchmark that evaluate both composition and reasoning capabilities of T2I models.\n\n2. The main findings (reasoning lags behind composition & prompt rewriting helps but still there’s gap) is well supported with large experiments (e.g., table 3) and valuable for future research direction.", "weaknesses": "1. **Evaluation method.** In L143-145, the authors mentioned `We propose an automatic checklist-based evaluation protocol ... individual yes/no questions ... allows fine-grained and reliable assessment' as a second contribution. However, such automatic checklist-based evaluation protocol (i.e., question generation followed by question answering) was already extensively studied in previous works [e.g., A, B, C, D], but none of them have been cited or discussed. I don't think authors can claim evaluation methodological novelty, which is separate from benchmark construction. The authors should cite these works and clarify their contributions.\n\n- [A] Hu et al., TIFA: Accurate and Interpretable Text‑to‑Image Faithfulness Evaluation with Question Answering. ICCV 2023\n- [B] Yarom et al., What You See is What You Read? Improving Text‑Image Alignment Evaluation. NeurIPS 2023.\n- [C] Cho et al., Visual Programming for Text‑to‑Image Generation and Evaluation. NeurIPS 2023\n- [D] Cho et al. Davidsonian Scene Graph: Improving Reliability in Fine‑grained Evaluation for Text‑to‑Image Generation. ICLR 2024.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)