Everything in Its Place: Benchmarking Spatial Intelligence of Text-to-Image Models
Authors: Zengbin Wang
Xuecai Hu
Yong Wang
Feng Xiong
Man Zhang
Xiangxiang Chu
Venue: ICLR 2026 Poster (2026)
Keywords: Text-to-Image Generative Evaluation
Spatial Intelligence
Primary area: datasets and benchmarks
TL;DR: A new text-to-image benchmark (SpatialGenEval) to evaluate, and a fine-tuned dataset (SpatialT2I) to improve how text-to-image models handle complex spatial intelligence.

Abstract: Text-to-image (T2I) models have achieved remarkable success in generating high-fidelity images, but they often fail in handling complex spatial relationships, e.g., spatial perception, reasoning, or interaction. These critical aspects are largely overlooked by current benchmarks due to their short or information-sparse prompt design. In this paper, we introduce SpatialGenEval, a new benchmark designed to systematically evaluate the spatial intelligence of T2I models, covering two key aspects: (1) SpatialGenEval involves 1,230 long, information-dense prompts across 25 real-world scenes. Each prompt integrates 10 spatial sub-domains and corresponding 10 multi-choice question-answer pairs, ranging from object position and layout to occlusion and causality. Our extensive evaluation of 23 state-of-the-art models reveals that higher-order spatial reasoning remains a primary bottleneck. (2) To demonstrate that the utility of our information-dense design goes beyond evaluation, we also construct another SpatialT2I dataset. It contains 15,400 text-image pairs with rewritten prompts to ensure image consistency while preserving information density. Fine-tuned results on current foundation models (i.e., Stable Diffusion-XL, Uniworld-V1, OmniGen2) yield consistent performance gains (+4.2%, +5.7%, +4.4%) and more realistic effects in spatial relations, highlighting a data-centric paradigm to achieve spatial intelligence in T2I models.

Published as a conference paper at ICLR 2026
EVERYTHING IN ITS PLACE: BENCHMARKING SPA- TIAL INTELLIGENCE OF TEXT- TEXT-TO-IMAGE MODELS
Zengbin Wangl.2, Xuecai Hu Yong Wang' 1 Feng Xiong', Man Zhang?, Xiangxiang Chu
AMAP, Alibaba Group. "Bijjing University of Posts and Telecommunications
+ Project Page: https: //gi thub. Com/AMAP--O. WWW.AP-MM/Spaa.. i a GenEval
ABSTRACT
Text-to-image (T2I) models have achieved remarkable success in generating high- fidelity images, but they often fail in handling complex spatial relationships, e.g.. spatial perception, reasoning. or interaction. These critical aspects are largely overlooked by current benchmarks due to their short or information-sparss prompt design. In this paper, we introduce SpatialGenEval, new benchmark designed to systematically evaluate the spatial intelligence of T2I models, covering two key aspects: (I) SpatialGenEval involves 1,230 Iong, information-dense prompts across 25 real-world scenes. Each prompt integrates 10 spatial sub-domains and pairs, ranging from object posi- tion and layout to occlusion and causality. Our extensive evaluation of 23 state- of-the-art models reveals that higher-order spatial reasoning remains a primary bottleneck. (2) To demonstrate that the utility of our information-dense design goes beyond evaluation, we also construct another SpatialT2I dataset. It contains 15,400 text- exxt-image pairs with rewritten prompts to ensure image consistency while preserving information density. Fine-tuned results on current foundation mod- els (i.e., Stable Diffusion-XLL -I, Uniworld-VI V1, OmniGen2) yield consistent perfor- mance gains (+4.2%, +5.7%, +4.4% and more realistic effects in spatial relations, highlighting data-centric paradigm to achieve spatial intelligence in T2I models.
corresponding 10 multi-choice question-answa
INTRODUCTION
Recent developments in text-to-image (T2I) generation have demonstrated remarkable progress to- wards photorealistic and high-fidelity image generation (Zhang al., 2023; Bie et al., 2024). This advancement has been largely driven by architectural innovations, evolving from early generative adversarial networks (GAN) (Goodfellow al., 2020) to the dominant diffusion paradigms (Rom- bach et al., 2022). These paradigms are often augmented by powerful LLM text encoders (Black Forest Labs, 2024; Wu et al., 2025a) or integrated into unified multimodal architectures that merge generation and understanding capabilities (Deng et al., 2025; Chen et al., 2025b; OpenAl, 2024b; Lin et al., 2025; Xie et al., 2024; Chu et al., 2025b). Among these SOTA T2I models, a core success lies in their ability to render the fundamental 'whal' of a scene. They exhibit strong compositional capabilitics in generating specified objects and binding them to their corresponding attributes (e.g., color, material, shape), thus achieving high fidelity for basic semantic prompt following.
However, the limitations of these models become apparent when the task shifts from merely gener- ating "what" is in scene to precisely depicting 'where' objects are located, 'how' they are arranged, and "why" they interact within complex real-world scenes. As illustrated by the error cases in Fig- ure 1, even SOTA T21 models (Wu al., 2025a; Deng al., 2025; OpenAI, 2024b) often fail on such fine-grained prompts. They may misplace objects, incorrectly orient them, disregard relative numerical comparisons, or fail to render causal interactions. These are not minor aesthetic flaws, but signal a fundamental shortcoming: a lack of spatial intelligence (Yang et al., 2025a), the core abilities of spatial perception, reasoning, and interaction with real-world scenes.
Notably, these complex spatial failures are largely overlooked by current benchmarks. As in Figure/ and Table 1, significant number of current benchmarks (Wei et al., 2025; Ghosh al., 2023; Huang
Work done during the internship at AMAP, Alibaba Group. Project leads and corresponding authors.

---

Published as a conference paper at ICLR 2026
Spatial Perception & Layout Tou smalll black and large yellow dog tridge right,
Spatial Ressoning: five red groomts Iopel...
The brick's bomauet los
Spatial Interaction The wind blowing,
traveler's cleskk billow bchinal
Prompt Type: Long & Sporse Info. the before simplicity
foad began
Prompt Type: Long & Dense Info.
doable in
with
dvep slge
dapeni muteppially postiiond
behind
leb,
boantfflly with
fyy
ganmetry dies, hamanious gron.
"V"
coprrcic
eonlight duup
Question Type:
Question Type:
Ber kgy
Yasos-No solections (numbess or lows)
Oueri- mutti choice questions (nunbers: of
Figure I: (Top): Error cases around spatial perception, reasoning, and interaction from GPT--mage- I (OpenAI, 2024b), Qwen-Image (Wu et al., 2025a), and Bagel (Deng et al., 2025). (Bottom): A comparison of prompt and evaluation formats across current benchmarks (Wei et al., 2025).
et al., 2023) are structured around short or information-spars. prompts. This design inherently confines their scope to verifying the presence of objects. their attributes, or simple binary relations. Additionally, these are typically assessed using noarse-grained metrics, such as classification (Ghosh et al., 2023) or limited yes-or-no questions (Wei et al., 2025). Although valuable for assessing basic composition, these evaluations fall far short of probing a model's spatial capabilities, thus failing to capture critical deficiencies in higher-order reasoning interaction. This highlights the need to explore long, information-dena spatial-aware prompts and fine-grate ine-grained evaluations for a comprehensive assessment of spatial intelligence.
To address this gap, we introduce SpatialGenEval, a new benehmark designed to systematically evaluate the spatial intelligence of T2I models. SpatialGenEval has two core features: (1) Long & Information-dent & Spatial-aware Prompts: SpatialGienEval involves a hierarchical decomposition of spatial intelligence into 4 domains (spatial foundation, perception, reasoning, and interaction), and 10 corresponding sub-domains, covering a comprehensive range of spatial abilities from object position and layout occlusion and causality. Based on these, we construct 1,230 prompts across 25 real-world scenes. Each prompt is designed to integrate all 10 sub-domains, making it inherently long, information-dent and suitable for comprehensive spatial evaluation. (2) Anmi-dimensional & Multi-choice Evaluations: For fine-grained evaluations, each prompt is paired with 10 meticulously crafted mini-dimensional multiple-choice question-ansver pairs, enabling precise identification of model's 's successes and failures in all defined spatial capabilities.
To demonstrate our data's utility beyond evaluation, we follow the same principles of our Spa- tialGenEval to create another 1,230 prompts and corresponding 12,300 QAs. After filtering out low-quality design scenes from the initial 1,230 prompts, the remaining 1, 1,100 prompts are sent to 14 top-performing spen-source T2I models (accuracy > 50% on SpatialGenEvall for image generation. The resulting images are sent to the MLLM (i.e., Dww22.5-VLL722B) for evaluation, yielding a total of 15,400 text-image pairs. Subsequently, powerful MILM (i.e., Gemini 2.5 Pro (Comanici al., 2025)) refines their corresponding prompts to ensure better text-image alignment while preserving the original information density. Crucially, fine-tuning SOTA models like Stable Diffusion (Rom- bach et al., 2022), UniWorld-V1 (Lin et al., 2025), and OmniGen2 (Wu et al., 2025b) with this dataset significantly boosts their spatial intelligence, validating the data-centric approach as a viable and effective pathway for model improvement. In summary. our contributions are threefold:
We introduce SpatialGenEvaa,, new benchmark to systematically evaluate complex spatial intelli- gence in T2I models. It leverages 1,230 information-dent prompts, each covering 10 spatial sub- domains and paired with 12,300 corresponding multiple-choice questions to evaluate model's understanding beyond what to generate, to where, how, and why.
Our extensive evaluation of 23 state-of-the-art models reveals a universal performance bottleneck in spatial reasoning- While models excel at basic object composition, their accuracy falls when faced with tasks requiring higher-order spatial understanding, such relative positioning, occlu- sion, and causality, revealing this as a primary barrier to current T2I capabilities.
Beyond evaluation, we explore a spatial-aware dataset (SpatialT2D), designed as a practical data-centric solution to improve the spatial intelligence of existing models. Fine-tuning results
2

---

Published as a conference paper at ICLR 2026
SENLL
DUN
HUL
Srapadooo
(a) 4 domains, sub-domains (b) 25 real-world
(c) Overall Compurisons
Detailed Compurisons
Figure 2: Overview of the SpatialGenEval benchmark and key results. The benchmark structured around (a) 10 spatial sub-domains and (b) 25 real-world scenes. (c) The evaluation of 23 SOTA T21 models shows the overall performance ranking and (d) detailed capability breakdown.
Benchmarks
Prompt
Omni-
QA
Object Attribuse Position Orientation Layout Comparison Proximity Ocelusion
Motion Causal
Length
Eval
Type
T21 B2I-CompBench
S
Yes/No
0
GenEval
S
Detect
DPG--ench
10000
Wise
S
O
Score
IllF-Bench
1.
Yes/Noa
Yes/No
SpatiaGGenEval
L
Multi-Choice
Table 1: Comparisons between our SpatialGenEval and current T2I Benchmarks. "L" and "S"
denote long and short prompt.
and O denote full, partial, and coverage. respectively.
yield significant and consistent performance gains (+4.2% on Stable Diffusion-XLe +5.7% on UniWorld-VI +4.4% on OmniGen2).
2 SPATIALLGENEVAL BENCHMARK
In this section, we introduce SpatialGenEVva,, a new benchmark designed to evaluate the spatial in- telligence of text-to-image models with dense information and omni-dimensional evaluations. An overview of SpatialGenEval is shown in Figure 2. Following, we first outline the key design prin- ciples of SpatialGenEval in Sec. 2.1. Subsequently, we present the focused spatial aspects and their definitions in Sec. 2.2, ranging from object position and layout to occlusion and causality. Fi- nally, we detail the full benchmark construction pipeline in See 2.3, covering both the generation of information-dens. prompts and their corresponding 10 omni-dimentional question-answer pairs.
2.1 KEY PRINCIPLES
Long & information-dense prompts: A primary limitation of current T2I benchmarks is their reliance short information-sparse prompts (Wei et al., 2025; Ghosh et al., 2023; Huang et al., 2023; Niu et al., 2025), often confined to simple object-attribute pairs or simple relations. To better capture the complexity of real-world scenes and probe model's ability synthesize intricate information, SpatialGenEval is designed to utilize longer and information-dense prompts that are densely packed with multiple and interdependent spatial constraints.
Omni-dimensiooall multi-choice questions: Instead of coarse-grainee metrics about objects, at- tributions, and simple spatial relations a yes-or-no selection (e.g., "Is there any dice in the image?"), evaluate models across all distinct sub-domains of spatial intelligence in multi- choice format. For each prompt, 10 multi-choice questions across all dimensions are generated, allowing for fine-grained diagnosis of where a model succeeds or fails.
Image-dependent answer (no answer leakage): Recent MMStar (Chen et al., 2024c) has vealed a significant flaw in MLLMs: some MLLMs can generate answers without accessing im- ages. For this case, do not send the "text-to-image prompt" to the evaluator prevent "answer leakage". Additionally, the questions are also checked with humans to avoid "answer leakage"..
Refuse to answer (not guess): For each multiple-choice question, the MLLM evaluation task is instructed to select the best-matching option. To avoid cases where the model is forced to select an incorrect option when all options (A/B/C/D) are not faithful to the generated image (Yu et al., 2023), we include another "E: None" option to account for such generation failures.
3

---

Published as a conference paper at ICLR 2026
S1: Object Category
S2: Object Attribution
Prompt
Prompt
tea lieaar two childire badi-ro back on cakertt oull fecepeaed coi,, lokdr lags packas look, conis froe the bay d-nt, lo pain pertally bloded by phal todkly boar.
Canwor Or the gromn enveciing crose fries lis tipe in crese The timern, erncked-shell seall on right fa: bekial. both faons In facid las. dind.
Toeikte tencchiitinn, what ofher regretes reuntioned in the sommet by can, teedy Bucapack crayn,, paper
What decutieed snvihutoc of stal rightt
Window curtans, plani keep,
und ansefbohelled tiry sediltrry
Beann cradhnl-sttlled
53: Spatial Position
S4: Spatial Orientation
S5: Spatial Layout
Prompt:
Prompt:
Prompt:
In RRO To dirrly of for uis aiapi rille. spurd
unilt living ccottal tocillar craebs fore ands thene osteriu bladds
Under glve ask old saryyaller astoal is sence, facing live oitiing in sentictrele
from phadd vie, where faraly the poorto sake leoaed?
Suned todilae's movereer, what toddier Baing tesortle?
Q: How childen amarged awuad mveytailler? staggtttr
epann
frven rubdec Jack The sridine None
the
Is career
Noez
The celarfat blocks
S6: Spatial Comparison
Spatial Proximity
S8: Spatial Occlusion
Prompt
Prampt:
Prompt
i is. The ned chair is the reeerrend apgears swice to large the tikee. chair teckgroand.
coro, The bkodde's loot whecl portally coclaced by mechhy neterpaer stand
sinis Ha fax portiallyy abourre by vileaut price ferm leavos amangal is mooe her
G: Hore dezr the appares die the red shais conapare to The bike chair?
G: Accoding ts dhe scune, stutebios neaily lhz beysk?
G; Whsi is patially stocuinn dhe womarts face? suuntill's t: The hamnninghid C.Hervanam The ribrust fern laves Note
A: Rol and blue wane 8t Red terice leneer then Mare O. Rodl felf May DJ Red in dlace tieos lagar fann shee E: None
A Theopationss
9. B.Aliiliing
C: A anitic light
Dr. Annnerpaaer stand
Nooo
99: Spatial Motion Interaction
S10: Spatial Causal Interaction
Prompt:
cono, facalo a sellow Hilei the centor jumping to spike
Catice: Usde a spuffght lirge opera singer in tasedd saeds Mage The soend
lett
glmns cansing to
O. What is fac prizary ausios dowsried betwons the plase und the rallephall
G. What is dieee ou glan dutiorngg
A: The player savina the valley The plaer becppir velkyball The is the volleyboal C: The glayer junging to spilt the sollechlll Nozs
A singer seadiig
B. The has beapstiigt
bitteg
colloping
Figure 3: Examples of SpatialGenEval. Each image is generated from information-dense prompt covering all 10 spatial sub-domains and evaluated with 10 corresponding multiple-choice questions.
2.2 FOCUSED ASPECTS
Motivated by advances in spatial cognition (Ruan et al., 2025; Malanchini al., 2020) and recent studies about spatial intelligence (Yang et al., 2025b;a; Cai et al., 2025; Stogiannidis et al., 2025), the definition of spatial intelligence has evolved to include perception, reasoning, and interaction with the environment. Following this comprehensive view, we define spatial intelligence in Spatial- GenEval through a hierarchical framework that begins with Spatial Foundation (representing objects and their attributions), moves Spatial Perception (perceiving their arrangement in space), advances to Spatial Reasoning (inferring relationships between them), and culminates in Spatial Interaction (understanding dynamic events and their causes). Representative examples are shown in Figure 3.
Spatial Foundation (S1/S2). This domain evaluates the model's ability to generate semantically correct objects, focusing compositional completeness and attribute binding, including: (S1) OB- ject Category evaluates compositional completeness by testing the model's ability to generate men- tioned objects without omission or hallucination. (S2) Object Attribution evaluates attribute bind- ing by examining whether the model correctly assigns attributes (e.g., color, shape, material) to their designated objects, preventing attribute leakage.
Spatial Perception (S3/S4/55). Building upon the foundational generation of objects, this domain evaluates the model's s ability to interpret and render its geometric and relational arrangements on the 2D canvas. It focuses on the accurate translation of spatial language into visual form, through three sub-dimensions: (S3) Spatial Position evaluates the localization of an object using absolute (e.g., top-left, bottom) or relative (e.g., to the right of the book, to his left side) terms. (S4) Spatial Orientation focuses on rotational alignment, such as generating objects with specified facing direc- tions (c.g., facing left, upside down). A common challenge is that models often Ican towards default poses (e.g., front view). (S5) Spatial Layout assesses the model's understanding of multi-object arrangements. This extends beyond individual positions to collective configurations, such as linear sequences (e.g., in a line from left to right), circular formations, or other specified group structures. This sub- dibenssno is crucial for testing the comprehension of group-level spatial patterns.
Spatial Reasoning (S6/S7/S8). This domain moves beyond direct perception to assess a model's higher-order cognitive ability to understand and render abstract, implicit, and 3D-aware spatial rela-
4

---

Published as a conference paper at ICLR 2026
(a) Domain & Scene
(b) Prompt & 10 Omni-QAs Generation
(c) T2I Generation Omni-Eval
Sccne: Clessroom
Domains: Descriptions of 10 sub-domains
Where the position of vase?
Generated Prempt:
B: Berween nbet and bock C: To right of book On tog book None
A: To left oobett
10 sub- domains
Select All
On a woaden tablesog, a resty robot, clear glaes viied, and blue baok are lined up from left right
absect
position
layout
attribution
Whe is pariilly obaazing
book??
The robut, facing right, taller than the book but shorter than the vase.
A-Thhrabufs bady The spillad wotrr
C: tigoing vase The uollatps olge Nooe
oriontation
comperison
With its
cleir glass saree, the pushing
Select
scenes
One
procimitt
motion
The vase is tipping, spilling water and pactially obscuring the book.
consal interaction
occlusion
7/10
4/10
5/0
4/20
Figure 4: SpatialGenEval Construction Pipeline. (a) The process begins by selecting one of 25 real- world scenes and combining with the definitions of all 10 spatial sub-domains. (b) The MLLM sequentially synthesizes an information-dense prompt that integrates all 10 constraints, along with 10 corresponding omni-dimensional QA pairs. (c) T2I models generate an image from the prompt, which is then evaluated against the QA pairs to yield fine-graine fine-grained spatial intelligence score.
tionships, involving: (S6) Spatial Comparison evaluates the model's grasp of relative quantitative attributes. This involves generating objects that adhere to comparative statements about their prop- erties, such size (e-g., three times taller than), quantity, or length. This tests whether the model can perform reasoning rather than merely generating objects in isolation. (S7) Spatial Proximity focuses on the fine-grained physical distance between objects. It challenges the model to render precise interactions like "touching", "closest to", or "far from". This subdimension is critical for assessing the model's ability to control object boundaries and depict intimate spatial relationships, which are often overlooked in favor of simple co-occurrence. (S8) Spatial Occlusion assesses the model's 's implicit understanding of 3D scene structure and depth. This requires generating scene where one object partially fully obscures another (e.g., the vase is partially obscuring the book). Success in this area indicates a more sophisticated world model that can reason about viewpoint and object layering, moving beyond flat, 2D composition.
Spatial Interaction (S9/S10). This is the most advanced domain, evaluating the model's ability to depict dynamic events and physical causality. It moves beyond static scene composition to test whether a model possesses a rudimentary understanding of physics and temporal progression. This capability is divided into two distinct but related forms of interaction: (S9) Spatial Motion Interac- tion focuses on generating objects in dynamic states or mid-action sequences. It requires capturing specific temporal moment, such "a dog jumping over log" "a mid-fliht ball".. This tests the model's ability to convey movement through pose, trajectory, and contextual cues rather than relying on static or canonical placements. (S10) Spatial Causal Interaction evaluates the capacity of the model to illustrate explicit cause-effect relationships between objects or environments. Ex- amples include "a rock hitting water and causing ripples" or "a hammer striking a nail into wood". Success in this dimension implies that the model can reason about functional physical relationships and translate latent dynamics into visually consistent and logically plausible images.
2.3 BENCHMARK CONSTRUCTION
Following the definition of 10 spatial-aware aspects, the construction for the SpatialGenEval bench- mark involves two main stages: information-dense spatial-aware prompt generation (Sec. 2.3.1) and the generation of their corresponding omni-dimensional question-answer pairs (Sec. 2.3.2).
2.3.1 INFORMATION-DENSEE & SPATIAL--WARE PROMPT GENERATION
Automated prompt generation. As illustrated in Figure 4, we instruct Gemini 2.5 Pro (Comanici et al., 2025) using two inputs: specific scene from a curated set of 25 real-world scenes (i.e.. nature, indoor, outdoor, human, and design, as detailed in Appendix 1) and the definitions of 10 spatial sub-domains. The model's task is to seamlessly integrate all 10 spatial constraints into a single, fluent, and logically sound prompt based on one given scene. The target length is set to approximately 60 words, balancing compatibility (77 tokens) with CLIP encoders (Radford et al., 2021) and the need for high information density. The meta instruction is shown in the Appendix A.4.
refinement. While powerful, MLLMs generate prompts that are stylisti- cally awkward, logically unsound, or unfairly complex. To guarantee prompt quality, each MLLM- generated output meticulously reviewed by human experts with the same guidebook. For example,
5

---

Published as a conference paper at ICLR 2026
(1) disjointed phrases like "There is a robot. It is rusty." will be combined into natural phrase like "A rusty robot". (2) Logical impossibilities like a cyclical layout ("A is left of B, B is left of C, is left of A") will be identified and corrected. (3) Furthermore, to ensure that focus is spatial reasoning rather than lexical knowledge, ambiguous or unusual words (e.g.. vermilion, nos- talgic, futuristic, bustling, tranquil) will be removed or replaced with more common synonyms (e-g., vermilion - bright red). Totally, this validation process yields our final set of 1,230 high-quality, information-dense prompts across 25 real-world scenes. More details are shown Appendix A.2.
2.3.2 OMNI-DIMENSIONAL QAS GENERATION
Automated mini-dimensional QAs generation. For each of the 1,230 prompts, we instruct a powerful Multimodal Large Language Model (i.e., Gemini 2.5 Pro) to automatically generate 10 multiple-choice questions, with each question targeting exactly one of the 10 spatial sub-domains. To guide the model effectively, its input for each prompt includes three parts: the prompt, the defi- nitions of all 10 spatial sub-domains, and a set of example questions. The model is then instructed to generate all 10 QA pairs at once. Each generated pair consists of the question, ground-truth answer drawn from the prompt, and three other plausible but incorrect options designed to test a model's s detailed understanding. The meta instruction is shown in the Appendix A.4.
Human-ing -oetan-i-tth-logge refinement. Following automated generation, every QA pair undergoes a rig- orous human validation process with the same guidebook confirm that: (1) To prevent answer leakage, the explicit task of human annotators is to identify and eliminate the question containing an explicit answer, where the answer lies in the question. For example, the question "What is the layout of the leaves that arranged in a circle?" should be revised into "What is the layout of the leaves in the image?". (2) Refuse to answer (not guess): After the above validation, we programmatically ap- pend "E: None" option in each question. This allows the evaluator to refuse forced choice when none of the options are faithful to the generated image. More details are shown in Appendix A.2.
3 EXPERIMENTAL RESULTS
3.1 SETUP
Text-to-Image models. We test wide range of well-known text-to-image models with diverse
model architectures and model scales, covering 23 open-source and closed-source models: (1) Dif- fusion Models:
(Rombach et al., 2022), PixArt-series (Chen et al., 2023;
2024b), Flux-series (Black Forest Labs, 2024), pla2ground-v2.55 (Li et al., 2024), SANA-1.5 (Xie
et al., 2025), and more recent Qwen-Image (Wu et al., 2025a). (2) AutoRegressive Models: Omni- Gen2 (Wu et al., 2025b), NextStep-1 (NextStep Team et al., 2025), and Infinity (Han et al., 2025).
(3) Unified Models: Janus-Pro (Chen et al., 2025b). Show-o (Xie et al., 2024), UniWorld-Vi (Lin et al., 2025), UniPic-v2 (Skywork Multimodality Team, 2025), and Bagel (Deng et al., 2025). (4) Closed-source Models: DALL-E-3 (Ramesh et al., 2021), GPT-Image-I (OpenAI, 2024b), Nano Banana (Gemini-2.5-Flash---aage (Google, 2025), and Seed Dream 4.0 (ByteDance, 2025).
MILM as a judge. If not specific, we formulate the evaluation as a zero-shot, multiple-choice VQA task, using an open-source MLLM (i.e., Owen2.5-VL-72B (Bai et al., 2025)) as the primary evalu- ator. This choice leverages the model's 's SOTA capabilities while ensuring long-term reproducibility and avoiding reliance on closed-source APIs. For comparison with closed-source MLLM, we also evaluate with 255-06 (OpenAI, 2024a), and the results are presented in Appendix A.3.
Evaluation details. (1) For the evaluation, the MLLM evaluator is presented with a generated image and its 10 corresponding questions from 10 sub-domains. The evaluator's 's task is to select the most accurate description from five options: four plausible choices (A-D) derived from the prompt and a crucial fifth option, "E: None". To enhance the stability of this automated evaluation and reduce randomness, we implement a 5-round voting mechanism (Wang et al., 2022). A response is consid- ered correct only if the MLLM selects the ground-truth answer in at least 4 of the 5 rounds. The final score is then reported as the accuracy on each of the 10 spatial sub-domains, calculated as the percentage of correctly answered questions. (2) For time cost, conduct the evaluation 8 H22 GPUs and deploy vllm framework (Kwon et al., 2023) in local environment. The evaluation time cost is around 1.8 seconds per image, completing all 1230 generated images in about 40 minutes.

---

Published as a conference paper at ICLR 2026
Model
Size Overall
Spuliall Foundation
Sputial Perception
Spatial Remsoning
SSpatial Interaction Causal
Object
Attribate
Position Orientation Layout Comparison Proximity Occlusion
Motion
Random
19.8
20.1
19.3
19.8
19.8
19.7
20.3
19.5
19.6
20.1
19.8
Diffusion Generative Model
SD-1.5 1.5
0.86B
28.5
8.5
33.7
19.5
29.2
38.2
12.8
37.7
15.6
42.0
47.6
PisArt-alphu
SD-XL
3.5B
41.2
25.7
52.8
32.0
40,9
49.3
19.1
50.7
22.4
56.7
62.0
Playaround-v2.5
2:58
41.4
27.0
55.8
41.5
49.0
18.5
22.4
55.2
63.3
PixAnt-sigaa
0.6B
51.0
42.8
67.6
43.7
49.1
58.0
25.9
57.6
27.0
67.0
71.0
SD-3-M
2B
54.6
52.6
72.1
46.9
$0.7
62.4
26.1
62.7
27.4
71.4
74.1
SD-3.5-1L
8B
54.0
52.4
72.0
44.7
$2.0
62.7
25.4
61.3
27.4
69.4
72.6
SANA 1.5
4.88
53.8
48.5
70.0
47.3
51.1
62.2
25,9
59.9
28.5
70.0
75.0
FLUX. 1-dev
12B
56.5
51.7
73.8
50.0
55.5
66.7
28.2
62.9
28.9
73.1
73.8
FLUX. I-krea
128
58.5
58.0
75.4
50.7
55.7
67.1
28.3
66.7
28.0
76.0
78.8
Qwen-Image
61.0
77.2
56.7
2.
AutoRegressvv Gemerative Model
NextStep-1
148
55.0
45.4
69.0
46.7
52.3
64.0
26.7
62.5
32.0
73.7
77.4
OmniGen2
4B
56.4
51.5
73.6
55.9
55,5
65.4
26.0
64.2
27.3
72.0
72.6
Infinity
73.0
72.6
Unified Generative Model
Janus Pro
7B
50.6
30.9
62.0
43,2
47.4
60.2
26.3
60.2
31.5
70.2
74.3
Show-o
13B
52.8
41.7
68.3
46.7
48.2
60.8
26.6
61.1
28.9
69.5
75.8
UniWorld-V1
54.2
46.8
71.3
53.1
64.0
26.1
62.0
26.8
69.6
72.4
UniPic-v2
9B
54.3
41.4
69.1
44.9
51.0
63.3
27.8
63.1
30.0
75.1
77.1
Ragel
73.7
54.0
Closed-source Generative Model
DALL-E-3
54.8
51.1
67.9
41.5
$2.9
63.3
28.4
62.4
28.0
75.2
77.4
GPT-Image-1
60.5
56.3
74.1
53.3
58.9
70.4
31.4
66.8
30.2
80.9
82.2
Nano Banana
61.7
58.5
75.3
55.5
58.9
70.9
31,8
68.7
33.5
81
82.2
Seed Dream 4.0
62.7
59.9
80.2
57.2
58.9
70.1
32.1
68.3
33.8
83.0
83.8
Table 2: SpatialGenEEal leaderboard based on Qwen 2.5 VL (72B). "Random": random selection.
3.2 MAIN RESULTS OF SPATIALGENEVAL BENCHIMAR
Overall findings of SpatialGenEval. Building on the overall leaderboard results (Table 2) of Spa- tialGenEval across 23 open- and closed-sourc, generative models, several key findings revealed regarding model performance, core weaknesses, and promising development strategies.
Open-source models are eatching up to closed-source ones, yet spatial intelligence remains a significant challenge. Across all models, the overall performance on SpatialGenEval generally reflects the continuous improvement of SOTA T2I models over time. Notably, the gap between open-source and closed-source models narrowing, with the best open-source model, Qwen- Image (60.6%), now catching up to the leading closed-source model, Seed Dream 4.0 (62.7%). Despite this progress, the highest score remains around the 60-point passing threshold. highlight- ing that even SOTA models possess only a rudimentary grasp of complex spatial intelligence.
Imbalanced performance between spatial foundation and higher-order spatial intelligence. A key finding across all models is the performance gap between basic and advanced spatial skills. For spatial foundation tasks, top models like Qwen-Image and Bagel score above 70.0% on object and attribute generation. However, the performance drops tasks that require complex thinking in complex spatial perception, reasoning, and interaction. This suggests that while models can draw objects correctly, they struggle to organize them according specific rules.
Spatial reasoning emerges as the primary bottleneck. Notably, spatial reasoning is the main weakness across all spatial domains. Scores subtasks like comparison and occlusion are often below 30%, near the random selection (20%). This reveals core failure of current T2I models: they can render objects correctly but cannot bind the semantic properties of objects to the structural logic of scene, such relative size physical layering. Moreover, the most gap between open- source and closed-source models also lies in the spatial reasoning and interaction sub-domains.
Text encoder capability emerges as key determinant of spatial intelligence. Our results reveal clear trend where models with stronger text encoders, particularly those leveraging pow- erful LLMs, consistently outperform those with standard CLIP encoders. For example, the top- performing open-source model, Qwen-Image (60.6%), utilizes powerful LLM encoder. Simi- larly, models that enhance the standard CLIP architecture with more advanced encoders, such as FLUX.1 (56.5-58.5%) and SD-3 (54.0-54.6%) using T5, significantly outperform older models reliant solely on CLIP, like SD-1.5 (28.5%). This strongly suggests that a deeper understanding of complex, information-dense prompts is critical to achieve high-fidelity spatial generation.

---

Published as a conference paper at ICLR 2026
Bieic Composition Firor
Visual Perception
Reasoning
Error
Desig
25.3%
24,6%
Ousdeor
24.2%
25.5%
34.2%
16.1%
Bagal
21.7%
26.9%
36.4%
15.0%
Indour
27 454
23 155
36.0%
11.8%
Flus. dev
22.4%
25,075
15
16.0%
24.11
35.5%
Qwandhrag
30.8%
5%
38,256
21.63
28,
GPT-Image-1
25,
20
3%%
li
Eroo Peccentage (15)
Enoor Pescestage (%)
Figure 5: Distribution of error types across scenes (left, based on all T21 models) and some examples of T21 models (right) in our SpntallGenEval.
Model
GenEval
DPG-Bench
Wise
TIfF-Bench
Meta-Rank
SpatialGenFval (Ours)
Rank (Ours)
Janus-Pro
0.80
84.19
0.35
65.02
5
50,60
SD-3.5-L
0.71
84.08
0.46
66,96
4
54.00
Iluv. 0.82
83.84
0.50
71.78
2
56.50
Bagel 0.82
67.20
0.52
71.70
3
57.00
Qwen-Image 0.91
88.32
0.62
86.83
60.60
Table 3: Meta-ranking consistency of five popular models
commonly used text-to-image benchmarks (GenEval (Ghosh et al., 2023), DPG-Bench (Hu et al., 2024)). two newly proposed benchmarks (Wise (Niu et al., 2025), TIIF-Bench (Wei et al., 2025)), and our SpatialGenEval.
Model scale and architecture are two possible pathways for advanced spatial intelligence. Our results also reveal two concurrent trends driving improvements in spatial intelligence. (1) For model scale, specialized diffusion models generally follow a trend where performance corre- lates with model size. For example, the 20B Qwen-Image (60.6%) significantly outperforms the 8B SD-3.5-L (54.0%). (2) For model architecture, unified models demonstrate greater parame- ter efficiency by integrating understanding and generative abilities. For example, the 7B Bagel model (57.0%) achieves a score comparable to the much larger 12B FLUX. I 1-krea (58.5%). This highlights their potential advance spatial intelligence without relying solely on parameter count.
Failure cases analysis. Our analysis in Figure 5 provides a detailed breakdown of failure cases across scenes and models. Models first face Basic Composition and generally succeed with low er- ror rates. The next challenge of Visual Perception is more difficult. Its errors are highest in complex Nature scenes at 28.5%. A significant increase in difficulty then occurs with Relational Reasoning. This skill is the primary failure point for all models, and its error rates often exceed 35%. Inter- estingly, Motion Interaction is a less frequent source of error, with rates typically remaining below 18%. This distribution suggests that the principal barrier to achieving advanced spatial intelligence not a linear progression of skills, but a critical weakness in processing relational logic.
Correlation with other benchmarks. In Table 3, the model rankings on SpatialGenEval closely align with the meta-rankings from four other major benchmarks. This strong correlation validates our benchmark as a reliable indicator a model's overall generative capability.
Other evaluation models as the judge. To test for gudgeddeeendency, we evaluate mod- els using both GPT-4o (OpenAI, 2024a) and Qwen2.5-VLL-72B (Bai et al., 2025). As shown in Table 4, both judges produce similar model rankings and numbers. The consistency of the similar ranking and number validates the ro- bustness of our benchmark and the selected evaluator, demonstrating that its relative results are not biased by the choice of evaluator.
GPT-40-250306
Qwen2.5-VL. (72B)
Model
Overall
Rank
Overall
Rank
Janus-Pro
48.0
50.6
5
SD-3.5-I.
52.9
54.0
F-ux.
54.3
56.5
Bagel
56.6
57.0
Qwen-Image 60.8
60.6
Table 4: Overall ranking consistency of closed- and open-source evaluations on SpatialGenEval.
Human Alignment Study. To evaluate the ef- fectiveness of MILMS as evaluators, we con- duct a human alignment study with three top- performing models: the open-source Qwen2.5- VL-72B and the closed-source GPT-4o and 2e5-ni-2.5-Pro. We randomly sample 200 im- ages from Qwen-Image (8 from each of 25
MLLMs
Foundation Perception Reasoning Interaction
Qwen2.5.VL-221
$7.0
76.5
73.3
84,8
80.4
GPT-40
82.8
76.7
72.5
83.3
78.8
Gemini-2.5Prr
78.2
86.0
Table 5: Human alignment study across open- source and closed-source MLLMs based on the balanced accuracy (%).
scenes). Following the principles in Section 2, five human annotators work independently to select
8

---

Published as a conference paper at ICLR 2026
loverall Spatial Foundation|
Spatial Perception
Sputial Reasoning
Spatial Interaction
Model
Object
Attribute
Position Orientation Layout Comparison Proximity Occlusion Motion
Causal
SD-XL
41.2
25.7
52.8
32.0
40.9
49.3
19.1
50.7
22.4
56.7
62.0
SptialT21
45.4
29.0
55.7
38.6
48.3
53.6
23.0
54.8
27.2
61.6
63.2
UniWorld-V1
54.2
46.8
71.3
50.1
53.1
64.0
26.1
62.0
26.8
69.6
72.4
SptialT21
59.9
50.8
74.8
58.5
63.5
70.4
34.6
70.7
31.6
66.0
78.5
OmniGen2
56.4
51.5
73.6
55.9
55.5
26.0
64.2
27.3
72.0
72.6
SptillT2
60.8
61.8
71.8
35.3
66.3
34.2
74.8
Table 6: Quantitative fine-tuned results of recent T2I models on SpatialGenEval.
Prompt E: In a forest, a grey squirrel on left branch of a mossive oek tree drops an acorn towards a red squirrel at the bottom-center. center...er two squirrels face other. itss tree hole, faces the observer. The oak tree much taller than nearby log. The red squirrel close small pile of three brown acorns, illuminated by sunbeam.
them, wise-looking owl yellow eyes, partially obscured by
GPT-Image-1
Awen-Image
Flux.1
Bagel
UniWorld-Vi
OmniGen2
SSpafallT21
Prompt forest clearing, Family of thres-two adults a child between them-sits on and white checkered blanket. They face away from the viewer, looking up fireworks. golden retriever lies next to the child, and picnic basket is on the blanket's right side, A tall tree on the left partially blocks the fireworks. One adult pulls the child close gentle wind rusties the leaves.
Figure 6: Qualitative comparisons of recent T2I models on SpatialGenEval.
the best option based solely on the given image and 10 questions, with access the original text- to-image prompt to prevent "leakage". We measure alignment using balanced accuracy (Brodersen al., 2010). following (Li al., 2025). Table 5 shows that all MLLMs align well with human judgment, with Gemini-2.5-Pra performing best. Moreover, alignment correlates with sub-domain difficulty of each sub-domain. The alignment is higher on simpler dimensions like Spatial Foun- qution/Perception//nteraee while lower on the Spatial Reasoning sub-domain. Despite this, the alignment score still nears 80%, validating their effectiveness as evaluators in SpatialGenEval.
SUPERVISED FINE-TUNING (SFT)
To further validate the other utilities of our information-dense and omni-dimensional data, this sec- tion investigates its application in constructing new supervised fine-tuning (SFT) dataset to en- hance the spatial intelligence of existing T2I models.
Additional SFT data construction. SpatialT21 is constructed separately and has no overlap with our evaluation benchmark. The construction of SpatialT21 involves two stages:
Stage I: Prompt and Onnii-dimensional QAs generation. This stage follows the same principles of our SpatialGenFEval in Section 2.3. Totally, we obtain another 1,230 prompts and 12,300 QAs.
Stage 2: Rewrite prompt to obtain text-image pair. We curated outputs from 14 top-performing T21 models with average scores above 50% (Table 2,10), along with their generation prompts. These are processed by strong MLLM (e.g., Gemini 2.5 Pro) produce mildly rewritten prompts that better match the corresponding images, improving text-image consistency while preserving information density and all dimensions of spatial intelligence. The meta instruction of SpatialT21 construction is shown in the Appendix A.5. It is worth noting that data from "Design" scenes (130 prompts) are excluded due to low image quality. In total, we construct (1230- 130) x 14 = 15, 400 image-text pairs from 22 scenes, forming the SpatialT21 dataset for the following SFT stage.
Training details and SFT results. We fine-tune the recent UniWorld- (Lin et al., 2025), Om- niGen2 (Wu al., 2025b), and Stable Diffusion-XLL (Rombach et al., 2022) based on their official settings and our SpatialT2I dataset. As shown in Table 6, the fine-tuned models consistently achieve better spatial abilities in SpatialGenEval. Finally, select fine-tuned OmniGen2 final model,

---

Published as a conference paper at ICLR 2026
and the qualitative comparisons with recent SOTA T2I models presented in Figure 6. The gen- erated images exhibit competitive results and more realistic effects.
Ablation study of SpatialT2I and trend data scaling. To assess the impact of data quality and quantity in SpatialT21, we con- duct two experiments as follows. (1) We fol- low SPRIGHT (Chatterjee al., 2024) and se- lect three subsets arranged by increasing perfor- mance in Table 2, i.e., Unipic-v2 (54.3), Bagel (57.0), and Qwen-Image (60.6). Each subset in- cludes 1100 text-image pairs. As shown in Fig- ure 7, fine-tuning on both the diffusion model (SD-XL.) and non-diffusion model (OmniGen2) reveals that all subsets yield performance gains, and higher-scoring ones contribute more signif- icantly. (2) We observe data scaling trend in Figure 7, as performance consistently im- proves when increasing training data from 0% to 100%, by progressively adding higher-scoring subsets. These findings reveal the value of explor- ing information-dense spatial-awar data and suggest that further scaling a promising direction.
41.2
OmniGinn2
$6.4
7
41.5
56.5
42.2
56.9
42.6
57.4
$0.21
Thaang Gia
Trackg
Figure 7: (Top) Ablations of the contribution of three different SpatialT21 subsets fine-tuning performance. The selected subsets are arranged from top to bottom in order of their performance. (Bottom) Data scaling trend observed when incre- mentally adding better subsets to the training data.
5 RELATED WORK
Text-to-Image models. Text-to-image generation relies on several key architectural approaches. Diffusion models (Rombach et al., 2022) are the dominant paradigm, prized for a parallel generation process that yields high efficiency and global coherence. Innovations within this framework include scaling the generative backbone with transformer architectures (Black Forest Labs, 2024; Peebles & Xie, 2023) and improving semantic comprehension with powerful LLM text encoders (Hu al., 2024). Beyond diffusion, autoregressive models (Wu et al., 2025b; Han et al., 2025; NextStep Team et al., 2025) regard image generation as a token-by-tokee process, a sequential nature that inherently models strong dependencies between visual tokens while offering finer compositional control. A recent trend is unified multimodal architectures (Deng et al., 2025; Chen et al., 2025b; Xie et al., 2024; Lin et al., 2025; Skywork Multimodality Team, 2025), which integrate visual understanding and generation into a single model for more holistic reasoning. Totally, all these diverse models have enabled models excel at generating high-fidelity images with basic compositional elements.
Text-to-Image benchmarks. The evaluation of T2I models has co-evolved with their capabilities, leading to distinct categories of benchmarks. The first category targets foundational semantic align- ment, verifying object presence and attribute binding with metrics like object detection (Ghosh et al., 2023; Huang et al., 2023). A more recent category addresses complex instruction following and re- lational understanding, using longer prompts and question-answering formats to assess multi-object relationships (Wei al., 2025; Hu et al., 2024: Chang al., 2025; Chatterjee al., 2024). Fur- thermore, a growing number of specialized benchmarks have emerged to probe even higher-order capabilities, such as world knowledge (Niu et al., 2025), physical plausibility (Meng et al., 2024), broader reasoning skills (Chen et al., 2025a), and comprehensive quantitative understanding of T21 models' capabilities and risks (Lee et al., 2023). This evolution highlights a clear shift research focus of T2I evaluation, from object-level fidelity towards scene-level compositional logic.
CONCLUSION
In this paper, we introduce SpatialGenEval,, benchmark that systematically evaluates the spatial intelligence of text-to-image (T2I) models. is built upon a hierarchical framework and utilizes information-dense prompts to test model capabilities in scenarios of real-world complexity. Our extensive evaluation of current models reveals a stark performance disparity between basic object generation and advanced spatial tasks, pinpointing spatial reasoning as the primary bottleneck. Fur- thermore, by demonstrating the effectiveness of our supervised fine-tuning dataset, SpatialT21, we validate data-centric approach practical path toward resolving these shortcomings.
10

---

Published as a conference paper at ICLR 2026
ETHICS STATEMENT
This paper introduces SpatialGenEval, a benchmark for evaluating the spatial intelligence of text-to- image models. Its creation uses large multimodal models (MLLMs) with rigorous human oversight. All generated prompt and question-answer pairs are reviewed by human experts to ensure it is logi- cal, neutral, and free of harmful or personally identifiable information. The benchmark is based on common, real-world scenes ensure broad applicability. Our research aims to transparently identify current AI limitations, thereby fostering the development more capable and reliable text-to-iin to-image generative models.
REPRODUCIBILITY STATEMENT
To ensure reproducibility, this paper provides comprehensive methodological details, and all re- sources are made publicly available. The construction of the SpatialGenEval benchmark is detailed in Section and Appendix A.2, A.4. Our experimental setup, including the models, judges, and evaluation protocol, is described in Section 3. The creation of the SpatialT21 dataset is outlined in Section 4 and Appendix A.5. The complete benchmark, dataset, and evaluation code are available the research community for verification and future work.
REFERENCES
Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, al. Qwen2.5-VL technical report. 2025.
Fengxiang Bie, Yibo Yang, Zhongzhu Zhou, Adam Ghanem, Minjia Zhang, Zhewei Yao, Xiaoxia Wu, Connor Holmes, Pareesa Golnari, David A Clifton, al. Renaissance; A survey into ai text-to-image generation in the era of large model. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024.
Black Forest Labs. Flux: A series of fast diffusion models for high-reoolutio text-to-image synthe- sis. 2024. URL https::///// http///gggggggcceceeo/ wotbbcccc-oooeeeeeeeoceeecc http://ggn.ccc....laaattttooaoaa
Kay Henning Brodersen, Cheng Soon Ong, Klaas Enno Stephan, and Joachim M Buhmann. The balanced accuracy and its posterior distribution. In International Conference on Patlern Recogni- fion, pp. 3121-3124. IEFE, 2010.
ByteDance. Seed Dream 4.0. 2025. URL https::/reeeacch..ot doubao com/ zh./ seedream4_0/
Wenxiao Cai, laroslav Ponomarenko, Jianhao Yuan, Xiaoqi Li, Wankou Yang. Hao Dong, and Bo Zhao. Spatialbot: Precise spatial understanding with vision language models. In 2025 IEEE International Conference Robotics and Automation (ICRA), pp. 9490-9498. IEEE, 2025.
Edward Chang. Maci: Multi-agent collaborative intelligence for adaptive reasoning and temporal planning. arXiv preprint arXiv:2501.16633 2025.
Jingjing Chang, Yixiao Fang, Peng Xing, Shuhan Wu, Wei Cheng, Rui Wang, Xianfang Zeng, Gang Yu, and Hai-Bao Chen. Oneig-bench: Omni-dimensional nuanced evaluation for image generation. arXiv preprint XXXi:2556600977 2025.
Agneet Chatterjee, Gabriela Ben Melech Stan, Estelle Aflalo, Sayak Paul, Dhruba Ghosh, Tejas Gokhale, Ludwig Schmidt, Hannaneh Hajishirzi, Vasudev Lal, Chitta Baral, et al. Getting it right: Improving spatial consistency in text-to-image models. In European Conference on Computer Vision, pp. 204-222. Springer, 2024.
Boyuan Chen, Zhuo Xu, Sean Kirmani, Brain Ichter, Dorsa Sadigh, Leonidas Guibas, and Fei Xia. Spatialvim: Endowing vision-language models with spatial reasoning capabilities. In Proceedings of the JEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14455-14465 2024a.
11

---

Published as a conference paper at ICLR 2026
Junsong Chen, Jincheng Yu, Chongjian Ge. Lewei Yao, Enze Xie, Yue Wu, Zhongdao Wang. James Kwok, Ping Luo, Huchuan Lu, et al. Pixart-a: Fast training of diffusion transformer for photore- alistic coxt-to-image synthesis. arXiv preprint arX1v:3110000426, 2023.
Junsong Chen, Chongjian Ge, Enze Xie, Yue Wu, Lewei Yao, Xiaozhe Ren, Zhongdao Wang, Ping Luo, Huchuan Lu, and Zhenguo Li. Pixart-o:: Weak-to-strong training of diffusion transformer for 4k toxt-to-image generation. In European Conference on Computer Vision, pp. 74-91. Springer, 2024b.
Kaijie Chen, Zihao Lin, Zhiyang Xu, Ying Shen, Yuguang Yao, Joy Rimehala, Jiaxin Zhang, and Lifu Huang. R2i-bench: Benchmarking ceasooning-drvver text-to-image generation. arXiv preprint arXiv:2505.23493, 2025a.
Lin Chen, Jinsong Li, Xiaoyi Dong, Pan Zhang, Yuhang 7ang, Zehui Chen, Haodong Duan, Jiaqi Wang. Yu Qiao, Dahua Lin, et al. Are we on the right way for evaluating large vision-language models? Advances in Neural Information Processing Systems,
2024c.
Xiaokang Chen, Zhiyu Wu, Xingchao Liu, Zizheng Pan, Wen Liu, Zhenda Xie, Xingkai Yu, and Chong Ruan. Janus-pro: Unified multimodal understanding and generation with data and model scaling. arXiv preprint vrXiv:2501.17811 2025b.
Xinhai Chen, Zhichao Wang. Liang Deng, Junjun Yan, Chunye Gong, Bo Yang, Qinglin Wang, Qingyang Zhang, Lihua Yang. Yufei Pang, et Towards a new paradigm in intelligence-drivn computational fluid dynamics simulations. Engineering Applications of Computational Fluid Mechanics, (81))24477055 2024d.
Xiangxiang Chu, Hailang Huang, Xiao Zhang, Fei Wei, and Yong Wang. Gpg: A simple and strong reinforcement learning baseline for model reasoning. arXiv preprint arXiv:250.002546, 2025a.
Xiangxiang Chu, Renda Li, and Yong Wang. Usp: Unified self-supervised pretraining for image generation and understanding. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 2025b.
Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, et al. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capa- bilities. arXiv preprint arXiv:2507.0626/ 2025.
Florinel-Alin Croitoru, Vlad Hondru, Radu Tudor lonescu, Nicu Sebe, and Mubarak Shah. Curricu- lum direct preference optimization for diffusion and consistency models. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp.
2025.
Yanqi Dai, Yuxiang Ji, Xiao Zhang, Yong Wang, Xiangxiang Chu, and Zhiwu Lu. Harder is better: Boosting mathematical reasoning via difficulty-aware grpo and multi-aspect question reformula- tion. arXiv preprint arXiv:2601.206/ 2026.
DeepSeek-AI Team., Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. Deepseek-v3 technical report. 2025. URL httpt:tt///c /ttttp:://xxiiii org/ abs/2412 19437.
Chaorui Deng, Deyao Zhu, Kunchang Li, Chenhui Gou, Feng Li, Zeyu Wang. Shu Zhong, Weihao Yu, Xiaonan Nie, Ziang Song, et al. Emerging properties in unified multimodal pretraining. arXiv preprint arXiv:2505. 0r0e225555.488.. 14683, 2025.
Jiafei Duan, Samson Yu, Hui Li Tan, Hongyuan Zhu, and Cheston Tan. A survey of embodied ai: From simulators to research tasks. IEEE Transactions on Emerging Topics in Computational Intelligence, 6(2):230-244, 2022.
Gemini Team., Petko Georgiev, Ving Ian Lei, Ryan Burnell, Libin Bai, Anmol Gulati, Garrett Tanzer, Damien Vincent, Zhufeng Pan, Shibo Wang, et al. Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context. 2024, URL https: /arxivv.... org/ abs/ 2403. 05530.
12

---

Published as a conference paper at ICLR 2026
Dhruba Ghosh, Hannaneh Hajishirzi, and Ludwig Schmidt. Geneval: An object- foccsed framework for evaluating ext-tt--mmae alignment. Advances in Neural Information Processing Systems, 36: 2023.
Ziyang Gong,
Wenhao Li, Oliver Ma, Songyuan Li, Jiayi Ji, Xue Yang, Gen Luo, Junchi Yan, and Rongrong Ji. Space-10: comprehensive benchmark for multimodal large language models in compositional spatial intelligence. arXiv preprint urXiv:2506.07900 2025.
lan Goodfellow, Jean Pouget-Abadie, Mchdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial networks. Communications of the ACM, 63(11)1139-14 2020.
Google. model.
Introducing 2025.
gemini 2.5 flash image, our state-of-the-art image
URL bettps:/deevelpeeet googleblog con/en/
Agrim Gupta, Silvio Savarese, Surya Ganguli, and Li Fei-Fei. Embodied intelligence via learning and evolution. Nature communications, 12(1))5721, 2021.
Jian Han, Jinlai Liu, Yi Jiang, Bin Yan, Yuqi Zhang, Zehuan Yuan, Bingyue Peng, and Xiaobing Liu. Infinity: Scaling bitwise autoregressive modeling for high-resolution image synthesis. In Computer Vision and Pattern Recognition Conference, pp. 15733-15744, 2025.
Xiwei Hu, Rui Wang, Yixiao Fang, Bin Fu, Pei Cheng, and Gang Yu. Ella: Equip diffusion models with Ilm for enhanced semantic alignment. arXiv preprint urXiv:2403.05/35, 2024.
Kaiyi Huang, Kaiyue Sun, Enze Xie, Zhenguo Li, and Xihui Liu. T2i-compbench: A compre- hensive benchmark for open-world compositional text-to-image generation. Advances in Neural Information Processing Systems, 36:78723-78741 2023.
Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng. Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. Efficient memory management for large language model serving with pagedattention. In Proceedings of the ACM SIGOPS 29th Symposium Operating Systems Principles, 2023.
Tony Lee, Michihiro Yasunaga, Chenlin Meng, Yifan Mai, Joon Sung Park, Agrim Gupta, Yunzhi Zhang, Deepak Narayanan, Hannah Teufel, Marco Bellagente, et al. Holistic evaluation of text- to-image models. Advances in Neural Information Processing Systems, 36:69981-7000 2023.
Daiqing Li, Aleks Kamko, Ehsan Akhgari, Ali Sabet, Linmiao Xu, and Suhail Doshi. Playground v2. 5: Three insights towards enhancing aesthetic quality in text-to-image generation. arXiv preprint ar arXiv:2402.172455 2024.
Ouxiang Li, Yuan Wang, Xinting Hu, Huijuan Huang, Rui Chen, Jiarong Ou, Xin Tao, Pengfei Wan, and Fuli Feng. Easier painting than thinking: Can text-to-image models set the stage, but not direct the play? arXiv preprint arXiv:2509.033 0351119099666...... 2025.
Renda Li, Hailang Huang, Fei Wei, Feng Xiong, Yong Wang. and Xiangxiang Chu. Adacurl: Adap- tive curriculum reinforcement learning with invalid sample mitigation and historical revisiting. In Proceedings of the AAAI Conference on Artificial Intelligence, 2026.
Bin Lin, Zongjian Li, Xinhua Cheng. Yuwei Niu, Yang Ye, Xianyi He, Shenghai Yuan, Wangbo Yu, Shaodong Wang, Yunyang Ge, et al. Uniworld: High-resolution semantic encoders for unified visual understanding and generation. arXiv preprint arXiv:2506.03147, 2025.
Fangyu Liu, Guy Emerson, and Nigel Collier. Visual spatial reasoning. Transactions of the Associ- ation for Computational Linguistics, 11:635-651, 2023.
Jie Liu, Gongye Liu, Jiajun Liang, Yangguang Li, Jiaheng Liu, Xintao Wang, Pengfei Wan, Di Zhang, and Wanli Ouyang. Flow-grpo: Training flow matching models via online rl. arXiv preprint arXiv:2505.0544 2025.
13

---

Published as a conference paper at ICLR 2026
Margherita Malanchini, Kaili Rimfeld, Nicholas G Shakeshaft, Andrew McMillan, Kerry L Schofield, Maja Rodic, Valerio Rossi, Yulia Kovas, Philip Dale, Elliot M Tucker-Drob, et al. Evidence for a unitary structure of spatial cognition beyond general intelligence. npj Science of Learning, 5(1):9, 2020.
Fanqing Meng, Wenqi Shao, Lixin Luo, Yahong Wang, Yiran Chen, Quanfeng Lu, Yue Yang, Tian- shuo Yang, Kaipeng Zhang, Yu Qiao, et al. Phybench: A physical commonsense benchmark for evaluating text-to-image models. arXiv preprint arXiv:2406. /802, 2024.
NextStep Team, Chunrui Han, Guopeng Li, Jingwei Wu, Quan Sun, Yan Cai, Yuang Peng, Zheng Ge, Deyu Zhou, Haomiao Tang. et al. Nextstep-1: Toward autoregressive image generation with continuous tokens at scale. arXiv preprint arXiv:2508, 10711, 2025.
Yuwei Niu, Munan Ning. Mengren Zheng. Weiyang Jin, Bin Lin, Peng Jin, Jiaqi Liao, Chaoran Feng, Kunpeng Ning, Bin Zhu, et al. Wise: A world knowledge-informed semantic evaluation for text-to-image generation. arXiv preprint arXiv:2503 2025.
OpenAL. GPT-4o system card. arXiv preprint arXiv:2410.21123 2024a. URL org/abs/ 221.22411222226
OpenAL. GPT-4o-Image.
2024b.
URL
introducing-4:: were--maaggggeee-eeeee
Contributors OpenCompass. Opencompass: A universal evaluation platform for foundation models, com/ open compass opencompass.
2023. URL https:
William Peebles and Saining Xie. Scalable diffuuion models with transformers. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 4195-4205, 2023.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, al. Learning transferable visual models from natural language supervision. In International conference on machine learning. pp- 8748-8763. PmLR, 2021.
Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea Voss, Alec Radford, Mark Chen, and Ilya Sutskever. Zero-shot text-to-image generation. International Conference on Machine Learning, pp. 8821-8831 PMLR, 2021.
Nils Reimers and Iryna Gurevych. Sentence-ber Sentence embeddings using siamese bert- networks. arXiv preprint urXiv:1908, 10084, 2019.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Patlern Recognition, pp. 10684-10695, 2022.
Shouwei Ruan, Liyuan Wang, Caixin Kang, Qihui Zhu, Songming Liu, Xingxing Wei, and Hang Su. From reactive to cognitive: brain-inspired spatial intelligence for embodied agents. arXiv preprint 2rXiv:2508.17198, 2025.
Skywork Multimodality Team. Skywork unipic 2.0: Building kontext model with online for unified multimodal model. 2025. URL https: : //w///unnii/ii//. github. io/.
Ilias Stogiannidis, Steven McDonagh, and Sotirios A Tsaftaris. Mind the gap: Benchmarking spatial reasoning in dision-languan models. arXiv preprint arXiv:2503. 2025.
Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdh- ery, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models. arXiv preprint arXiv:2203.11111 2022.
Yibin Wang, Zhimin Li, Yuhang Zang, Yujie Zhou, Jiazi Bu. Chunyu Wang, Qinglin Lu, Cheng Jin, and Jiaqi Wang. Pref Pref-grpo: Pairwise preference awardd-bseed grpo for stable text-to-image reinforcement learning. arXiv preprint arXiv:2508.2075: 2025.
14

---

Published as a conference paper at ICLR 2026
Xinyu Wei, Jinrui Zhang, Zeqing Wang, Hongyang Wei, Zhen Guo, and Lei Zhang. Tiif-bench: How does your t2i model follow your instructions? arXiv preprint vrXiv:2506.02166 2025.
Chenfei Wu, Jiahao Li, Jingren Zhou, Junyang Lin, Kaiyuan Gao, Kun Yan, Sheng-ming Yin, Shuai Bai, Xiao Xu, Yilei Chen, et al. Qwen-Image technical report. arXiv preprint arxiv:2508.02324, 2025a.
Chenyuan Wu, Pengfei Zheng, Ruiran Yan, Shitao Xiao, Xin Luo, Yueze Wang, Wanli Li, Xiyan Jiang, Yexin Liu, Junjie Zhou, al. Omnigen2: Exploration advanced multimodal generation. arXiv preprint arXiv:2506. 200.:::255668877777. /8871, 2025b.
Enze Xie, Junsong Chen, Yuyang Zhao, Jincheng Yu, Ligeng Zhu, Chengyue Wu, Yujun Lin, Zhekai Zhang, Muyang Li, Junyu Chen, et al. Sana 1.5: Efficient scaling of training-time and inference- time compute in linear diffusion transformer. arXiv preprint arXiv:250..1822, 2025.
Jinheng Xie, Weijia Mao, Zechen Bai, David Junhao Zhang, Weihao Wang, Kevin Qinghong Lin, Yuchao Gu, Zhijie Chen, Zhenheng Yang, and Mike Zheng Shou. Show-o: One single transformer to unify multimodal understanding and generation. arXiv preprint arXiv:2408. /2528, 2024.
Feng Xiong, Hongling Xu, Yifei Wang, Runxi Cheng. Yong Wang, and Xiangxiang Chu. Hs-star: Hierarchical sampling for self-taught reasoners via difficulty estimation and budget reallocation. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing,
pp. 553955535555 5555,, 2025.
Jihan Yang, Shusheng Yang, Anjali W Gupta, Rilyn Han, Li Fei-Fei, and Saining Xie. Thinking in space: How multimodal large language models see, remember, and recall spaces. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 10632-10643, 2025a.
Sihan Yang, Runsen Xu, Yiman Xie, Sizhe Yang. Mo Li, Jingli Lin, Chenming Zhu, Xiaochen Chen, Haodong Duan, Xiangyu Yue, et al. Mmsi-bench: A benchmark for multi-image spatial intelligence. arXiv preprint arXiv:2505.2376. 2025b.
Weihao Yu, Zhengyuan Yang, Linjie Li. Jianfeng Wang, Kevin Lin, Zicheng Liu, Xinchao Wang, and Lijuan Wang. Mm-vet: Evaluating large multimodal models for integrated capabilities. arXiv preprint arXiv:2308.02444 2023.
Chenshuang Zhang, Chaoning Zhang, Mengchun Zhang, and In So Kweon. Text-to- diffusion models in generative ai: survey, arXiv preprint arXiv:2303.07900 2023.
Zheyuan Zhang, Fengyuan Hu, Jayjun Lee, Freda Shi, Parisa Kordjamshidi, Joyce Chai, and Ziqiao Ma. Do vision-language models represent space and how? evaluating spatial frame of reference under ambiguities. arXiv preprint arXiv:2410.17385, 2024.
15

---

Published as a conference paper at ICLR 2026
A APPENDIX
A.1
BENCHMARK STATISTICS AND ANALYSIS
This section provides detailed statistical overview (Table 7) of our SpatialGenEval benchmark, followed by a discussion of the selection principles that ensure its diversity and comprehensiveness.
Statistic
Number
Number of prompts
1,230
max length (words)
97
min length (words)
46
uverage length (words)
Numher of scenes
25 (1,230)
Nature
7 (350, 28.5%)
Indoor
4 (200, 16.3%)
Outdoor
8 (400, 32,5%)
Human
3 (150, 12.2%)
Design
3 (130, 10.6%)
Number of questions
12,300 12,300 (100%)
Multi-choice QAs
Number of spatial domains
4
Number of spatial sub-domains
10
Table 7: Statistics results of our SpatialGenEval benchmark.
Scene selection and diversity. As illustrated Figure 8, our selection of primary scenes (nature, indoor, outdoor, human, and design) and their corresponding 25 sub-scenes is designed to form the representative samples of real-world applications where spatial intelligence is crucial. This set is a practical balance between broad coverage and manageable benchmark size. Moreover, our focus on diverse, real-world scenes is also timely, developing in parallel with a similar emphasis from leading models like Qwen-Image (Wu al., 2025a) and benchmarks like OnelG-Bench (Chang et al., 2025). Specifically, the detailed scenes are as follows.
Outdoor (32.5%). As the largest category, this focuses on complex, public human environments. includes transportation hubs (Airport, Railway), recreational areas (Park, Zoo), and commercial/- cultural spaces (Shopping Mall, Art Gallery, Cafe, Library). These scenes challenge models with high-density object layouts, crowd dynamics, and understanding large-scale functional designs.
Nature (28.5%). This category spans large-scale environments from natural landscapes (Forest, Mountain, Desert, Beach, Underwater) to human settlements (Cityspace, Village). These scenes test reasoning about organic layouts, vast scales, and perspective (e.g., the relative size of moun- tains the dense arrangement of trees).
Indoor (16.3%). This category includes common, confined spaces where function dictates object placement. Scenes like Kitchen, Classroom, Living Room, and Office test model's understand- ing of strict physical constraints, containment, and the functional relationships between objects (e.g. a chair's position relative to a desk).
Human (12.2%). This category centers on people and their interactions. Scenes like Sports, Human Activities, and Portraits require reasoning about body poses, relative positions between multiple people, and human-object interactions, which are critical for depicting action and social context.
Design (10.6%). This category tests spatial intelligence in non-photorcalistic and conceptual con- texts. Scenes like Cartoon, Advertisement, and Story challenge model to generalize beyond real-world physics, understanding instead principles of artistic composition, narrative flow, and symbolic spatial arrangements.
Domain selection and diversity. Our selection of domains and 10 sub-domains is not a random list, but a structured framework based on the definition spatial intelligence. Drawing from studies in cognitive science (Malanchini et al.. 2020; Gupta et al., 2021; Ruan et al., 2025) and computer vision (Yang et al., 2025a; Stogiannidis et al., 2025; Gong et al., 2025: Yang et al., 2025b; Cai series of steps. The process starts with the basic
et al., 2025), spatial intelligence be
16

---

API Published as a conference paper at ICLR 2026
Models
Your
Resolution
Source
URL
1. Diffusion Generative Model
SD-1.5
2021.12
1024x
1024
checkpoint
hups//huggingllee..oo/ hww. vitt---iiiiio---------- -Subblleiiflllss------
alpht
2023.10
1024x
checkpoint
htpps//huggingface
wttp://nwwwa....aaaaaaeea crrPis alphasPPisAniiiaaa M-L-2-1024-M5
Playground-v2.5
2024.02
1024x 1024
checkpoint
https//hugginggacc.oo www.ayrroouuuuulluyyooo---------
SD-XL
2023.07
1024x 1024
checkpoint
htps://uugingglee wtwp:innnnneeeennaiiiieee wwwstabiiityi -ittaa/a-aaadl---sso-----aaaa--a
PixArt-sigma
2024.04
1024 x 1024
checkpoint
https//ugginggfce
SD.3.M M.M
2024.03
1024 x 1024
checkpoint
htpp://huggingfacce face www.eec...caooooe.ooeeoo. www.s//aaiittaaisssaaab....eaaee diffusioon medium
SD-3.5-1
2024.11
1024x 1024
checkpoint
httpe//ww ww monel scope cwhwoneees/A Moaiee//ae--oooeeeeeeeeeeeeeeeee -iw.aablledifiion......5. 3.55iuioon-3.--lrgee
SANA 1.5
2025.01
1024x 1024
checkpoint
haps://huggigg www..eew.......oo...oo. www.e//wweeeeteeeeeeeeeeoeeeeeee Lareeeieeeeeee-eeeeeeaae Law.eiieeee..aeeeeeeelleAAAA
5_4.8B_ 10B_11024pp_difeeeeee
FLUX.I d-dev
2024.12
1024
checkpoint
htlps//huggingfeee face wwbblcck law/FLU
F-UX.1-Krea
2025.07
1024x1024
checkpoint
/ttps// wow.aaoooooesssaaaaLUU..
Qwen-Image
2025.08
1328x1328
checkpoint
httpc/huggingfae caiQuueni/Quenn-aage
2. RegoReere Generative Model
OminiGen2
2025.06
1024x1024
checkpoint checkpoint checkpoint
htps:///iibub com' VecioeSppaaeLLaaaonnnnnnn
NextStep-1
2025,08
512x512 1024x1024
https://gttuub.com com//sepfuur-ai/Nettttpp
Infinity
2024.12
https///gttuub.ccm// com//oundaiionnnnoooooooooooooao Vision/nniiityy
3.
U Unified Generutive Model
Show-o
2024.08
512x512
checkpoint checkpoint checkpoint
https//gittubb.c com/show
Janus-Pro Pro
2025.01
384x384
https///itthbb. wonn/deopsee--ii/naane
Un
2025.06
1024x1022
https///github.. com/PKU- ShanGrrupUnniWooreeo
UniPic-22
2025.08
512x
checkpoint
httpe://ptthhh.c com/ wtt.S/yyoot/////i///eeeeeeeee AWlha////ahaacccciiiiiiiiiiiiiie
Bagel
2025.05
1024x 1024
checkpoint
https://ithubbo com Danceeeeeeeeeee/aagg
Closed-source Generative Model
DALL-E-3
2023.10
1024x1024
API https: http:///nnni.c////--aaaaa
Gma-Image-1
2025.04
1024x1024
API
httpc//platfomm.oo openai com
Nano Banana
2025.08
1024x 1024
APT https://developers http://awgg...ooo//e/oooonnnnnng www.aeeeem---------ii------ www.a//m......oo..oooo///eeeeeen
Seed Dream 4.0
2025.08
1024x1024
Table 8: The release time, resolution, model source, and URL of T21 models on our SpatialGenFval.
step of identifying what objects are present. It then moves on to perceiving their static properties and arrangements. The next step is to infer the relationships between them. The final stage is understanding the dynamics and causes of events. Our framework is designed to follow this logical progression, making a powerful diagnostic tool.
T2I model selection and diversity. To provide comprehensive overview of the current T2I land- scape, we select 23 representative models, as detailed in Table 8. Our selection spans multiple architectural paradigms, including dominant diffusion models (Rombach et al., 2022; Chen et al., 2023; 2024b; Li et al., 2024; Xie et al., 2025; Black Forest Labs, 2024; et al., 2025a), autoregres- sive models (Wu et al., 2025b; NextStep Team al., 2025; Han et al., 2025), and emerging unified architectures (Xie et al., 2024; Chen et al., 2025b; Skywork Multimodality Team, 2025; Lin et al., 2025: Deng et al., 2025). Crucially, our evaluation includes both open- source checkpoint models and leading closed-source API systems, such as DALL-E-3 (Ramesh et al., 2021), GPT-Image-1 (Ope- nAI, 2024b), and Nano Banana (Google, 2025). This dual approach allows for direct comparison between community-driven and commercial efforts, ensuring our benchmark's findings are robust and widely applicable across the entire T21 ecosystem.
Question examples. As the question examples shown in Table 9, our question design deliberately moves beyond simple checks to probe deeper and more complex spatial understanding as follows.
For Spatial Foundation, instead of asking for the presence of single object, we test comprehen- sive scene awareness by asking what else exists besides a given set of objects. For Attribute, we query for multi-atrribue combinations rather than just single, isolated properties.
For Spatial Perception, questions require models to determine an object's 's position or orientation from another object's viewpoint, not just global third-person perspective. This tests a more sophisticated understanding of relative and absolute spatial arrangements.
For Spatial Reasoning, our questions shift from qualitative judgments to quantitative analysis. For instance, Comparison tasks demand reasoning about how many times quantity by how much larger, going beyond simple "more/less" distinctions. We also probe 3D spatial understanding through Occlusion and physical distance-based Proximity.
For Spatial Interaction, we focus on the dynamics between multiple objects and their effects. Questions target the causal outcomes of these interactions, demanding a level of reasoning far beyond the description of simple, isolated motions.
17

---

Published as a conference paper at ICLR 2026
Task
Question Examples
Besides [object A] and [object BJ. what other objects are mentioned in the image?
Object
Which option can better describe [all objects] exist in the image?
Which option can better describe the attributes the [object A] [object
image?
Attribute
What is the (attribute A]. [attribute BJ, and [attribute C] of the [object] in the image?
From the third-person perspective, where is the [object] in the image?
Position
From the [object A]'s perspective, where is the [object] in the image?
From the third-persou perspoctive, what is the facing orientation of the [object] in the image?
Orientation
From [object AJ's perspective, what facing orientation of the [object BJ image?
From the third person perspective, what the [object A] and (object BJ facing orientations in the image?
How are the objects arranged the image?
Layout
How are the [object A] and (ohject BS] arranged in the image?
How are [a group of object A] and [object BJ arranged in the image?
Howw many times the quantity of [object A] that [object BJ?
Comparison
-Whut is the secemeriho/nmmmricau difference between the [object A]. Jobject BJ, and [object CJ in the image?
What is the closest object the (object A| in the image?
Proximity
Which objects ure Ineurby object A] in the image?
Iy the [object AJ fully visihle in the image?
Occlusion
What part the [object A] is partially occluding the [object B] in the image?
What is the motion interaction between the [object A] and [object B] image?
Motion
Whut is the interaction between [object A]. [object BJ, and lobject CJ in the image?
What caused the flag flutter the wind in the image?
Causal
What caused the floor to become illuminated in the image?
Table 9: Question examples of all spatial sub-domains in SpatialGenFval.
A.2 HUMAN ANNOTATION INTERFACE
To ensure the quality and accuracy of SpatialGenEval, employ rigorous two-stage human finement process. This process involves five expert annotators and requires over 168 person-hours complete. To maintain high inter-annotator agreement, all annotators first complete calibra- tion phase using a detailed guidebook. The details of the two stages are as follows. The human annotation interface is shown in Figure
Prompt refinement. In this stage, annotators revise the initial text prompts to improve clarity, fluency, and logical consistency. Specifically, they follow strict guidelines: (a) simplifying vocabulary by replacing or removing uncommon words, and (b) verifying that each prompt com- prehensively covers all 10 spatial sub-domains without omission.
QA refinement. In the second stage, annotators review the generated question-answer pairs against the refined prompts. This refinement process adheres three key principles: (a) ensuring each question precisely targets a specific spatial sub-domain, (b) removing any information from the question that could directly leak the answer, and (c) correcting phrasing inaccuracies, such as changing "in the prompt" to "in the scene" or "in the image". After refinement, each QA pair is independently validated by Qwen2.5-VL. (Bai et al., 2025) for consistency evaluation.
A.3 ADDITIONAL CLOSED-SOURCE EVALUATION
In line with current leading benchmarks (Wei et al., 2025; Li et al., 2025), we conduct a secondary alternative evaluator. Ta-
evaluation using powerful closed-source model, 2550-40- ble 10 presents the full results. The model rankings from GPT-4o are highly consistent with those from our primary evaluation using Qwen2.5-VL-72B. This consistency confirms the robustness of our main analysis, demonstrating that our conclusions are not dependent on a single evaluator.
A.4 META INSTRUCTION OF SPATIALGENEVAL CONSTRUCTION
The construction of our benchmark and dataset follows seti--auommtedd pipeline, leveraging large multimodal models (MLLMs) with human oversight. In this section, we detail the core meta in- structions used at each stage of the data generation process.
18

---

Published as a conference paper at ICLR 2026
Size Overall Spatial Foundation)
Spatial Perception
Spatial Reasoning
[Sputial Motion
Model
Ohjeect
Attribute
|Position Orientation Compparononoooon Proximity Occhusion
Causal
Diffusion Generalive Model
SD-1.5
0.86B
24.6
35.3
43.4
15.5
20.2
29.4
24.2
13.7
28.0
28.
PixArt-alpha
0.6B
35.1
44.2
28.2
39.9
38.4
11.4
35.0
17.9
35.5
Play Mayground-v255
3.5B
38.6
54.6
64.2
29.8
41.0
42.5
12.3
39.7
19.4
37.6
44.3
SD-XL
38.6
Pix PixArt-sigma
0,6B
48.0
66.7
74.4
39.3
47.8
51.1
16.8
50.4
26.4
49.4
58.0
SD-3-M
51.1
74.6
76.2
42.8
49.8
20.3
52.0
SD-3.5-1.
8.18
52.9
74.2
79.0
39.8
51.4
58.9
19.9
56.9
29.5
58.1
61.5
SANA 1.5
4.8B
53.0
71.5
78.0
43.3
31.5
$7.2
18.6
56.7
27.9
58.3
66.7
FLUX.I-dev
12B
54.3
74.6
78.3
56.4
27.6
FLUX. I- --rea
12B
58.0
79.5
79.8
48.9
55.9
63.8
20.7
63.7
29.8
63.6
71.9
Qwen Qwen--mage
AutoRegressive Generative Model
OmniGen2 NextStep-1
4B
52.3
73.9
75.0
50.2
51.4
60.6
18.9
57.8
23.9
53.2
57.9
14B
52.4
67.2
69.8
44.7
53.5
59.9
20.5
28.6
56.6
67.4
Infinity
8B
54.6
74.9
75.7
50.0
56.3
62.4
20.0
58.9
28.2
56.0
64.1
Unified Generative Model
Janus Pro
7B
48.0
61.1
64.1
42.7
47.4
55.4
17.2
53.3
27.6
50.2
61.2
Show-o
1.3B
48.9
63.6
69.1
44.7
48.0
56.3
18.8
51.7
24.7
51.4
60.3
UniWorld-V1
12B
50.9
72.6
71.6
45.9
52.4
59.8
18.7
54.6
22.8
54.6
56.0
UniPic-v2
64.1
Bagel
7B
56.6
76.7
79.8
45.9
55.7
62.4
20.6
62.1
31.3
64.2
67.6
4.
Moored-Source Generative Model
DALL-E-3
51.8
66.7
69.3
40.2
53.7
21.1
24.6
61.5
GPT- GPT-Image-1
59.2
70.4
73.5
52.2
61.3
65.4
24.1
67.2
30.8
73.3
74.0
Nano Banana
61.6
68.0
74.5
56.3
62.0
69.0
26.4
70.7
36.1
74.6
78.3
Seed Dreum 4.0
62.1
66.5
62.3
68.6
70.3
38.0
81.5
Table 10: SpatialGenEval leaderboard based on GPT-4o.
Stage 1: Meta instruction for prompt generation. Following the key design principles of long, information-denss and spatial-aware settings, we instruct the MLLM to synthesize a coherent scene description that incorporates all 10 spatial sub-domains based on the given scene follows.
Stage 1: Meta Instruction for Prompt Generation
[Task Description]
I am creating text-to-image evaluation benchmark to challenge the spatial intelligence of text-to-image models. You are an assistant tasked with generating 50 distinct and dynamic text-to-imape prompts based on the provided scene. Each prompt MUST involve 10 types of spatial sub-domains as follows.
[Scene] ###Scene####
[Definitions of Spatial Primary-domaing
primary spatial domains involve spatial foundation, spatial perception, spatial reasoning, and spatial interaction.
[Definitions of 10 Spatial Sub-domains]
10 spatial sub-domains involve object, attribute, position, orientation, layout, comparison, proximity, occlusion, motion interaction, and causal motion interaction.
[Instructions]
1. The prompt should clearly describe all 10 spatial sub-domains around 60 words. The generated prompts should be distinct and involve various cases.
[Output Format]
Please output your response in valid JSON format as follows.
(scene: ###SCeneettt,, prompt: ### generated prompt 1###).
(scene: ###ScenetttH, prompt: ###The generated prompt 2###).
(.....)
(scene: ###ScenetitH, prompt: ###The generated prompt 50###)
19

---

Published as a conference paper at ICLR 2026
Stage 2: Meta instruction for QAs generation. For each generated prompt, instruct the MLLM (i.e., Gemini 2.5 Pro (Comanici et al., 2025)) to create 10 corresponding multiple-choics questions, each targeting one of the 10 spatial sub-domains to enable omni mini-dimensional evaluations as follows.
Stage 2: Meta Instruction for Question-Answer Pairs Generation
[Task Description]
I am creating text-to-image evaluation benchmark to challenge the spatial understanding ability of text-to-iman models. You
assistant tasked with generating ** #0 multiple- choice question-answert answers** based on the given **tcxt-to-image prompt* Each question MUST involve of the following 10 spatial sub-domains. Please avoid including any question that introduces irrelevant or mythological information not present the prompt.
[Definitions of spatial primary-domains] 4 spatial primary-domains involve spatial foundation, spatial perception, spatial reasoning, and spatial interaction.
DDefinitions of 10 spatial sub- sub-domains]
10 spatial sub-domains involve object, attribute, position, orientation, layout, comparison, proximity, occlusion, motion interaction, and causal motion interaction.
[Output Format]
Based on the following ##iprompthiit to generate 10 multiple-choice question-answer answerss and fill in the following "questions" and "answers" fields. The questions MUST be in the same order as the "question type" field. Please avoid including any question that introduces irrelevant or mythological information not present in the prompt.
(id: ###id### scene: ###scene##HH, prompt: ##Hpromptt## question type: ###questio type####, questions: [###question 1###, ###question- 2###, ###question- an- swers: [###aaswwr- 1###,, ##Hhanswr-22HHt ##llanswer- 10###]]
Stage 3: Meta instruction of MLLM evaluation. To evaluate a generated image, we provide an image and its 10 corresponding multiple-choice questions to an MLLM evaluator. The instruction explicitly forbids the use of external knowledge and enforees Aisually-grounde answering process. The meta instruction is as follows.
Stage 3: MILM Evaluation Instruction
[Task Description]
You are tasked with carefully examining the provided image and answering the following 10 multiple-choice questions. You MUST ONLY rely on the provided image to answer the questions. DO NOT use any external resources like world knowledge or external information beyond the provided image.
MMultiple-Choice Questions]
##MMultipl--hoice Questions####
[Instructions]
1. Answer these 10 questions separate 10 lines, beginning with the correct choice option (A/B/C/D/E) and followed by a detailed reason (in the same line as the answer).
2. Maintain the exact order of the questions in your answers.
3. Provide only one answer per question.
Each answer must be
own line.
5. Ensure the index of answers matches the index of questions.
Select the option "E: None" when the image can not answer the question.
20

---

Published as a conference paper at ICLR 2026
A.5 META INSTRUCTION OF SPATIALT2I CONSTRUCTION
Meta instruction for rewritten prompt generation of SpatialT22. To create the SpatialT21 dataset, we use an MILM (ie., GPT-4o (OpenAI, 2024a)) to rewrite the original prompts to accurately de- scribe the content of the generated images, thereby correcting spatial failures in text-image align-
ment. The core instruction is as follows.
Meta Instruction for SpatialT21 Construction (Rewrite Prompt based on MLLM)
[Task Description]
You are an expert Al assistant specializing in image prompt analysis and refinement. Your task is to analyze generated image and its corresponding metadata to rewrite the original text-to-image prompt. The goal to create a new prompt that accurately describes the spatial foundation, perception, reasoning, and interaction of the generated image.
[Ippt Data]
You will receive two primary inputs for each task:
1. generated image: This the visual ground truth and your primary source of informa- tion. Your final rewritten prompt must be a faithful description of this image.
A ISON input containing the following 7 keys:
{id: ###id###, scene: ###scene#HH, prompt: #**poompt*#* question ###questions##ff ground-truth answers: ##Hanswers#HH, image path: ##ffimage path###H, answers from gen- erated image: ###mmodel preds#tHt))
[Instructions]
Your process is to rewrite the prompt fully based on the real generated image as follows. Step Pinpoint discrepancies: Your primary task systematically compare the "ground- truth answers" with the "answers from generated image".
If the answers match: This means the aspect of the image described by the question is generated correctly, and the corresponding part of the original prompt is accurate.
If the answers DO NOT match: This is critical signal. It tells you exactly where the generated image failed to follow the original prompt's instructions. The "answers from generated image" becomes your source of truth for this specific detail.
Step 2: Synthesize the rewritten prompt: Based your analysis from Step and direct observation the image, you will construct a new, cohesive prompt.
Embrace the reality: Your new prompt must be built from facts presented in "answers based on generated image" and confirmed by your own visual inspection.
Integrate corrections holistically: Do not simply swap words. Weave the corrected details (object order, actions, attributes, etc.) into a new, flowing, and descriptive sentence.
Be specific: Use precise and descriptive language that captures the nuances of the gener- ated image. If the image shows "rusty blue robot" instead of just a "rusty robot", your new prompt must include "blue."
Verify everything: Before finalizing, re-read your rewrite prompt and ensure every single clause is verifiably true by looking at the provided image.
[Output Format]
Please ONLY output your response in a valid JSON format as follows.
(id: ###id### scene: ###hscenethhh, image path: ##tlimage path##iH,, original prompt: ###tprompt##*i, rewrite prompt: ##ttrewritt prompt##h)
A.6 META INSTRUCTION OF PROMPT REWRITING
Meta instruction for prompt rewriting. We instruct Gemini 2.5 Pro (Comanici et al., 2025) to rewrite the original prompts with the specific goal of making our defined spatial dimensions more explicit and unambiguous. The meta instruction is follows.
21

---

Published as a conference paper at ICLR 2026
Model
Overall
Spatial Foundation|
Spatial Perception
Sputial Reusoning
Spotial Interaction
Object
Atrrbbute
Pusition Orientation
Iayou||oomaaisnnn Comparison
Proximity Occlusion
Motion
Causal
SD-3.5-L
54.0
72.0
44.7
52.0
62.7
25.4
61.3
27.4
69.4
72.6
Rewriting
56.3
53.6
73.5
49.4
52.4
65.0
27.7
65.5
27.6
72.8
75.7
OmniGen2
56.4
51.5
73.6
55.9
55.5
65.4
26.0
64.2
27.3
72.0
72.6
Rewriting
58.5
53.8
75.6
60.1
55.5
68.4
30.5
63.8
27.7
75.0
75.0
UniWorld-V V1
46.8
71.3
50.1
53.1
64.0
26.1
62.0
26.8
69.6
72.4
Rowriting
55.9
72.1
53.8
53.4
66.0
29.1
64.8
26.9
71.5
73.8
Qwen-Image
60.6
61.0
77.2
55.6
56.7
69.7
28.6
67.7
30.8
78.1
80.2
Rewriting
61.7
62.8
80.8
57.6
56.4
70.2
29.7
68.6
30.4
79.0
81.1
Table 11:
Quantitative results recent T2I models based on prompt rewriting.
Meta Instruction of Prompt Rewriting to Improve Spatial Intelligence
[Task Description]
You are an expert Text-to-Image prompt rewriter. You will receive a long, information-dense text-to-image prompt designed to evaluate the spatial intelligence of generative models. Your task is to rewrite this prompt to accurately and clearly describe its contents, attributes, spatial perception, spatial reasoning, and spatial interaction. The ultimate goal is make the rewritten prompt suitable for downstream text-to-image generation.
[Ippt Data]
ISON input containing the following keys:
(id: ###id### scene: ### prompt: ###propt##** question type: ###question type####, question: ###questions##tt, ground-truth answers: ###answersHHHt)
[Instructions]
Step 1: Analyze Input: Carefully examine all keys in the imput JSON, especially the prompt, questions, and ground-truth answers.
Step 2: Deconstruct the Scene: Systematically go through each of the 10 question-answer pairs. Each answer provides non-negotiable fact about the scene's final state (e.g., what objects exist, their attributes, exact positions, layout, relative sizes, occlusion state, and the result of any interactions).
Step 3: Synthesize the Rewritten Prompt:
Construct a clear prompt by integrating 10 facts you confirmed in the previous step. Start with the foundational elements and layer in the perceptual, reasoning, and interactional details.
Step 4: Review your rewritten prompt to ensure it is single, clear paragraph.
[Output Format]
Please ONLY output your response in valid ISON format follows.
(id: ###id### scene: ###'scenettitt, question type: ###question type###,, original prompt: #i#prromtt#** rewrite prompt: ##ttrerrite prompt##t)
A.7 IMPACT OF PROMPT REWRITING TO IMPROVE SPATIAL INTELLIGENCE
To explore other solutions to improve the spatial intelligence of T21 models, we explore another potential method: prompt rewriting. We instruct Gemini 2.5 Pro (Comanici al., 2025) to rewrite the original prompts with the specific goal making our defined 10 spatial dimensions more ex- plicit and unambiguous. These enhanced prompts are then sent to the text-to-image models (across diffusion-based autoregressive-basel and unified-based) for evaluation. The results in Table 11 show that prompt rewriting is viable strategy for improving model performance. The detailed breakdown reveals three key insights:
Enhanced prompt decomposition is valuable path for T21 models. The results show that all mod- els benefit from rewriting, confirming that a model's core ability to deconstruct complex prompts is critical bottleneck. Notably, the improvement is more pronounced for models that initially struggle with text reasoning (e.g., SD-3.5-L shows a +2.3% gain), suggesting that improving this native capability is promising research direction.
Rewriting achieves greater gains on explicit spatial relationships. Rewriting proves highly effec- tive at resolving textual ambiguity in categories like Position, Comparison, and Layout. This leads
22

---

Published as a conference paper at ICLR 2026
substantial score increases (e.g., +4.5% in Comparison for OmniGen2, +4.7% in Position for SD-3.5-L), demonstrating its ability to clarify relational instructions.
Minimal impact on implicit visual reasoning. Conversely, rewriting offers little benefit for com- plex visual reasoning tasks like Occlusion and Orientation. This indicates the failure stems not from text comprehension, but from core lack of 3D and physical reasoning in the generator. Such challenges require solutions beyond prompt engineering, such as specialized fine-tuning or unified-based design for joint optimization.
A.8 THE USE OF LLMS
This work utilizes several multimodal large language models (Comanici al., 2025; OpenAI, 2024a;; Bai et al., 2025) for benchmark construction, SFT data construction, and improving paper writing. We transparently document their roles, and our rationale for choosing them is as follows.
Gemini 2.5 Pro (Comanici et al., 2025). We leverage Gemini 2.5 Pro for the initial data generation phase of SpatialGenEval and prompt rewriting for improvement, including (1) information-denss prompt generation, (2) omni-dimensional QA pairs generation, and (3) prompt rewriting to search for enhanced text understanding for generation. In preliminary comparisons against other MLLMs such as GPT-4o (OpenAI, 2024a), Qwen2.5-VL- VL--BBB (Bai et al., 2025), DeepSeek- V3 (DeepSeek- Al Team. et al., 2025), and Gemini- 1-r (Gemini Team. et al., 2024), our human evaluations find that Gemini 2.5 Pro demonstrates a superior creative capability for generating novel, com- plex, and diverse content from scratch. This observation aligns well with the recent performance comparisons of MILMs (OpenCompass, 2023).
Dwen2.5-VLL-722 (Bai et al., 2025). We employ Qwen2.5-VL-720. as the primary evaluator (judge), which renowned for its strong open-source multimodal understanding capabilities. This choice ensures the long-tern reproducibility of our evaluation results and validates our findings independently of closed-source APIs
GPT-4o (OpenAI, 2024a). GPT-4o serves two critical roles in our pipeline: (1) acting as the
secondary closed-source evaluator (judge) for benchmark, and (2) performing the prompt rewriting for the SpatialT21 dataset. We select it due to its widespread adoption, consistently stable multimodal understanding, and superior inference speed, which are crucial for our large- scale evaluation and data refinement tasks.
A.9 DISCUSSION
A.9.1 DISCUSSION ABOUT THE CHOICE GEMINI 2.5 PRO CREATE T2I PROMPTS
The choice of Gemini 2.5 Pro stems from its strong creative ability (less repetition than others across 50 prompts within the same scene). Specifically, the construction of SpatialGenEval benchmark concentrates two abilities follows.
Strong instruction following ability: As stated in Section 2.3, the prompt generation process must strictly follow the instructions to generate each text-to-image prompt based on given scene (e.g., classroom) while seamlessly integrating all 10 pre-defined spatial sub-domains.
Strong creative ability (less repetition): As stated in Appendix A.4, we instruct the model to generate 50 distinct rext-to-image prompts all at once. This requires strong creative ability to avoid generating repetitive outputs.
During the implementation, we conducted controlled experiment where we instructed three top- performing MLLMs (Gemini 2.5 Pro, GPT-4o, and Qwen2.5-VLL72BB) for prompt generation. We found that all three models demonstrate strong instruction following ability, but Gemini 2.5 Pro outperformed the others in terms of creative ability. This observation stems from: (1) consistent decision from all five human annotators, and (2) Gemini 2.5 Pro shows lower average sentence similarity (Reimers & Gurevych, 2019) across all text-to-image prompts within the same scene, i.c., Gemini 2.5 Pro (0.4938) GPT-4o (0.5125) QWw-2.5-VVL-72B (0.5548).
23

---

Published as a conference paper at ICLR 2026
Model
Spatial Foundation
Spatial Perception
Spatial Reasoning
Spatial Interaction
Overall
No Image Input Random Choice
7.3
19.2
13.5
28.1
16.9
19.7
19.7
19.8
20.0
19.8
Qwen-Image
69.1
60.7
42.4
79.2
60.6
Table 12: Analysis of the potential visual bias from MLLM predictions.
A.9.2 DISCUSSION ABOUT THE POTENTIAL BIAS FROM MLLM PREDICTIONS
To mitigate the potential biases from model predictions (e.g., which favors reliance on implicit world knowledge answer, other aesthetic and stylistic preferences), our evaluation protocol robust with three safeguards.
No external knowledge instruction. The MLLM is explicitly instructed: DO NOT use any external resources like world knowledge. This discourages guessing based on prior knowledge.
Refuse to answer (add another "E: None" option). The inclusion of the "E: None option" is crucial. If the image does not provide the necessary visual evidence to answer the question, the evaluator is instructed select "E: None", preventing forced-choice guessing.
Majority voting. We employ 5-round voting mechanism, where response is only considered correct if the ground-truth answer is selected in at least 4 of the 5 rounds. This enhances evaluation stability and reduces the impact of random inference.
Questions designed to be aesthetic-agnostee The task of MILM evaluator is perform direct Visual Question Answering (VQA) based on factual spatial correctness (e.g.. "How are the children arranged around the storyteller?"). The questions have no aesthetic stylistic preference.
Finally, to empirically validate that our questions are visually grounded, we conduct new ablation study: evaluating the questions without any image input. The results Table 12 show that overall accuracy drops to 16.9%, lower than the random guess accuracy of 19.8%. These results indicate that although the model may exhibit some slight biases (e.g., 28.1% vs. 20.0% in Spatial Interaction), their impact remains minimal. This result also strongly indicates that without visual context, the model cannot deduce the correct answers and often correctly refuses to answer (by selecting "E: None", which marked as an incorrect answer since the ground truth is A-D).
A.9.3 DISCUSSTON ABOUT THE RELIANCE OF MILM FOR EVALUATION.
Although current MILMs remain working handling highly complex spatial reasoning (Liu al., 2023: Chen et al., 2024a; Zhang et al., 2024), we would like to clarify that current leading MLLMs are suitable and capable in evaluating the questions in our SpatialGenEval benchmark, based on three key aspects:
Design philosophy: The primary burden of spatial intelligence of our SpatialGenEval lies in the generative T21 model, not the VLM evaluator: The T2I model must integrate the long, information- dense prompt to create an image with all defined spatial relationships. In contrast, the MLLM's task is simplified simple visual checks based on the direct visual evidence. This design choice decouples the complex reasoning required for generation from the simpler verification required for evaluation.
Problem difficulty: Unlike benchmarks designed to push the boundaries of MLLM spatial under- standing. SpatialGenEval focuses on evaluating generative tasks and consists of simpler QAs. Unlike recent spatial intelligence benchmarks such VSI-Bench (Yang al., 2025a). MMSI- Bench (Yang et al., 2025b), and Space-10 (Gong et al., 2025), which target understanding task and focus on more abstract or complex long-range reasoning (e.g., estimating real-world distances path planning). SpatialGenEval is targeted to generative task and concentrates evaluating more simpler visual relationships like relative position, layout, occlusion, and interaction. These tasks can be verified directly from the image's visual content, making them more suitable for current MLLMs.
Human alignment: Current leading MLLMs align well with human evaluation in our Spatial- GenEval. Our human alignment study in the Section 3 provides direct numerical validation that current MLLMs are capable of handling our designed questions. The results reveal an exception- ally high correlation between the MLLM and human evaluations.
24

---

Published as a conference paper at ICLR 2026
A.10 LIMITATIONS
While our work establishes a new benchmark for evaluating spatial intelligence, we acknowledge two limitations. (1) Scale concern: The construction of SpatialGenEval relies semi-automated. human-in-the-loop pipeline. While this ensures high data quality, the process is labor-intensive and presents challenges for scaling the benchmark an even larger size or new domains. (2) Scope expansion concern: Our hierarchical framework of 10 spatial sub-domains, while comprehensive, an abstraction of the near-infinite complexity of real-world spatial phenomena. More nuanced or dynamic spatial effects (Chen et al., 2024d; Chang, 2025), such as fluid dynamics, complex deformations, or multi-agent predictive interactions, are worth future exploration.
A.11 FUTURE WORK
Our work encourages shift from evaluating simple fidelity complex compositional logic and opens several avenues for future research. (1) Dataset-level extension: Our key principles of information-dense prompts and omni-dimensional evaluation can extended to other generative capabilities, such as stylistic control, textual rendering, and world knowledge. Extending this frame- work seat-to-video is also crucial step for assessing spatio-tempon level extension: Leveraging information-denss settings by increasing object number is promis- ing approach improve fine-grained perception and generation (Li et al., 2025; Chatterjee al., 2024). (3) Post-training strategies: Beyond simple fine-tuning, SpatialT21 enables exploring more advanced data-centric strategies. This includes curriculum learning (from simple to complex spatial tasks (Croitoru et al., 2025; Xiong et al., 2025)) and reinforcement learning (Wang et al., 2025; Liu et al., 2025; Chu et al., 2025a; Li et al., 2026; Dai et al., 2026) from MLLM feedback to further boost model performance.
reasoning. (2) Object-
A.12 BROADER IMPACT
Improving spatial intelligence can lead to more useful tools for artists, architects, and designers by enabling precise control over object placement and scene layout. (2) Embodied Al: Models with a stronger grasp of spatial relationships are a crucial step towards advancing Al agents for robotics and embodied intelligence (Duan al., 2022), which must understand and interact with the physical world.
(I) Creative tools:
25

---

Published as a conference paper at ICLR 2026
Porest
Mountain
Bench
Desert
Underwater
Cityspace
Village
Portrait
Human Activity
Sport
Office
Living
10
Zoo
Airport
Railway
Art Gallery
Shopping Mall
Story
Figure 8: Samples of generated results from all selected 25 scenes Spatial
26

---

Published as a conference paper at ICLR 2026
Prompt:
deep forest, girl right, taller boy left, harding its goow, large squarrels
illuminsting They facing each other.
nearby, background center, partially
arranged
Question-Answer Pairs:
Which option can better describe the object categories in the scene? Options: A: Boy, garl, apple, lanter, chair, tree, squirrels, B: Boy, girl, apple, lantern, bench, tree, squirrels, fem, (: Bay, girl, orange, larp, bench, tree, chippunks, fem, D: Non, wonan, apple, lantern, bench, tree, squirrels,
Object
Which option can better describe the attributes of the lantem? Options: A: The Lantern is glowing blue, Bi The lantern is mode of brass, The lantern i5 din and yel low, D: The lantern is red, E: None
Abribute
third-person perspective the
located?
Position
girl,
path,
of
From thard-person perspective, what is the tacing ortentation of the boy and the girl? Options: A: They are facing away froe each other, They are both lacing each other,
facinng observer, C: They
boy facing the
None
How are two squirrels arranged on other, Side-by-side, C: In circle around trunk, different
tree? Options:
above the
Layout
branches,
What is the height difference betveen the boy and the girl? Options: A: The bey is taller than the girl, B: They are the sase height, C: The boy is such taller than the girl, D: The girl is taller than the boy, E: None
Compariso:
Proximby
Where the blue Lantern placed? Options: A: On the ground, bench, C: In the boy's hand, D: Hang from the
E: None
the wouden
Is the boy fully visible in the scene? Options: At Yes, he is completely visible, tt: No, he is hidden by the girl, C: No. he is partially hidden large fer, D: No. he is obscured by the tantern's Light, E: Mone
Occksion
What the notion interaction
the and the to the
Options: The
Motion
the boy, 8: The girl is handing dancing together, D: The is taking the lantern fron the girt, E: None
girl
They
B
girl illuminated? Options: selling, None
Caunal
bright,
squirrels ho
ORFFF
RUSI
Figure 9: Illustration depicting the human annotation interface, where the experts are presented with the prompt and corresponding question-anse -nswer pairs for final refinement.
27

---

Published as a conference paper at ICLR 2026
Prompt: A double exposure portrait features a woman with dark hair in profile, facing left. Her silhouette is filled with a landscape: on the right, the sun peeks over a large mountain. On the left, two bare trees stand in river, the right one taller. Five birds fly toward the mountain in V formation, and bright sunlight casts sharp shadows of the trees.
Spatial Foundation:
S2: Spatial Foundation: Actribution
S3: Spatial Position
Qt: Desides a woman, tree, and birds, what other nutural lestures mentiened as
Q2: What are the described sttriuues the woman's hair and the trees?
QS: Where are the two hore trees located within the lindscape inside the silhomet??
part the lundscope?
Blonde hair, lasfy tress
B. Dack buis bere tree
On the right side, on the mountain
A: moumain and river
C: Red hair, tall mees Nowe
D: Dork hair, green trees
the center, new the hinds
R: A fonal and a lake
On the left side, a On right
C:A desert
valley clouds
E: Nase
S4: Spatial Orientation
S5: Spatial Layout
facing oriettrion of in portrit?
binds orringdd sky?
straight line
In scattered group
A: She facing forward looking the viewor
lommation
circle
Nose
Rt She in profiic, lacing sight
C) Her back is to the viewer Di She profile. facing Ieft
F: None
S7: Spatial Preximily
Q?: What the spenial relationship between the two tros and the siver?
86: Spatial Comparion
The tros are streding next the river D: The wres are far away from the river The standing the river
Q6: two hare troes compare in hright?
left tallee
height
ziver
The one taller
None
Their heights are not mentioned
None
SB: Spatial Orclusion
SP: Spatial Mation Interaction
S10: Spatial Mation Causal Interaction
Q6: Which object is partillly obscuring the sun?
()) What the direction of the binds flight in relution to the landscupe?
Q10 Acconding the sorse, what the direct couse of the treed' sharp shadows??
The
8: The woman's silhoutte
They are flying away from the mountain
The dackness the womnar's hair
C: The binds
B: They are flying woardd the mosntsin
R: The reflaction from the river
D: The lange mounoin
C: They are flying soward the crees
C: The briht surlight
None
They flying circle above the river None
The mountain blocking the light
Figure 10: A data sample from the SpatialGenEval benchmark, where the generated image is chal- lenged by 10 question-answer pairs across 4 key dimensions: Spatial Foundation (S1, S2), Spatial Perception (S3-S5), Spatial Reasoning (S6-S8), and Spatial Interaction (S9, $10). The generated images are from 12 top-performing T21 models from top-left to bottom-right, i.e., Qwen-Image (Wu ct al., 2025a), GPT-4o (OpenAI, 2024b), Flux. -krea (Black Forest Labs, 2024), Infinity Han et al. (2025), Bagel (Deng et al., 2025), Flux.1-dev (Black Forest Labs, 2024), OmniGen2 (Wu et al., 2025b), NextStep-1 (NextStep Team et al., 2025), DALL-E-3 (Ramesh et al., 2021), SD-3-M (Rom- bach al., 2022), UniWorld-V V1 (Lin et al., 2025), SD-3.5-L (Rombach et al., 2022).
28


Reviews:
[{"review_id": "2YbMJApKbh", "reviewer": "Reviewer_vwE4", "summary": "The paper introduces a new text-to-image generation benchmark focused on testing spatial capabilities. The benchmark consists of 1.2k prompts, generated by an LLM prompted with different combinations of domains and themes, and covering 10 axes of spatial capability. After refinement, for evaluation, an MLLM is given the prompts and asked to produce QA pairs aligning with the 10 spatial aspects. Newer T2I models are tested on this benchmark and findings show that spatial comparison and occlusion are two main bottlenecks in improving spatial capabilities.", "questions": "1. What informs the choice of Gemini 2.5 Pro as the model to create T2I prompts?\n2. How strongly do individual per-question scores in the QA portion align with human judgment? While the strong correlation with other benchmark numbers is convincing, if the goal is a breakdown of model capabilities by sub-domain, it is important that these scores also correlate strongly with human or other benchmark judgment.\n3. In human refinement of the QA pairs, you mention revising questions that can be answered from the prompt alone. Is this determined by whether a human can answer correctly without an image, or a model? Often, models can still end up being biased towards the correct answer, even if it is not apparent from human inspection.\n4. Is there any analysis of bias towards/against image quality or image style, independent of spatial correctness? E.g., do models that output photorealistic vs digital art-style images exhibit higher or lower scores on the benchmark despite having similar true spatial capabilites?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 2, "presentation": 4, "contribution": 3, "strengths": "The work introduces previously untested axes of T2I capability, and provides a large benchmark designed to test models along these new dimensions. The benchmark consists of LLM-generated prompts with automated model-based evaluation, where the generation prompts and evaluation QA pairs are refined manually by humans. While the integrity of the model-based evals could be contested, the diversity of prompts and structured scoring (evenly across the clearly defined 10 sub-domains) is clear.\n\nThe benchmark also remains inclusive through information-dense prompts that are short enough to fairly evaluate older T2I models while being difficult enough to distinguish between frontier models.", "weaknesses": "In the analysis of error rates and failure cases, the authors mention \"This clear hierarchy demonstrates that models learn skills in a specific order\". However, the hierarchy is not clear from Figure 5; it seems for example that motion, which appears after relational reasoning, is already easier. Some clarification would be helpful here.\n\nIt is unclear how impactful the contribution of the training data is. While the results show that SFT on the new data improves performance on their benchmark, this is to be expected since the method of training data generation/collection is closely tied to the benchmark. It would be helpful to see numbers from other benchmarks showing whether this improvement also carries into other metrics and maintains the same rank correlation.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "KKaNoSfhBO", "reviewer": "Reviewer_WonS", "summary": "The paper proposes a new T2I benchmark, SpatialGenEval, designed to evaluate the spatial understanding of T2I models. This benchmark features long, information-dense prompts that incorporate complex spatial descriptions, offering a more challenging evaluation compared to previous benchmarks. It categorizes the prompts into four main domains and ten subdomains to enable a more detailed analysis of model performance.\nThe paper also introduces a new evaluation metric that employs a Vision-Language Model (VLM) to analyze the generated images and answer multiple-choice questions based on them. Using this benchmark, the authors conduct a comprehensive evaluation of four different architecture of T2I models, covering both pioneering and state-of-the-art approaches. The results highlight the challenges these models face when generating images with more advanced spatial relations—particularly those involving 3D spatial reasoning.\nFinally, the paper demonstrates the utility of the proposed benchmark by fine-tuning a T2I model using the benchmark prompts. The fine-tuned model shows some improvements in spatial understanding in image generation.", "questions": "There are no additional questions beyond those asked in weaknesses.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "- The paper introduces a new benchmark with complex and information-dense prompts, which significantly improve upon previous benchmarks that rely on shorter prompts and limited number of spatial relations.\n- The paper also introduces a new evaluation protocol that incorporates a vision-language model (VLM) to assess the completeness of generated images through visual question answering, specifically designed to evaluate spatial understanding.\n- The benchmark includes human experts in refining prompts and evaluating questions to ensure quality, soundness, and appropriate complexity.\n- The benchmark reveals performance gaps across different categories of spatial relations, enabled by the detailed annotations proposed in the benchmark.\n- The paper illustrates the usefulness of the evaluation framework for improving model performance in text-to-image generation.\n- Well-document and easy to follow paper. Especially figures make the paper easy to follow and understand.", "weaknesses": "• Although the paper proposes a benchmark with complex spatial relations, the evaluation protocol raises some concerns. As demonstrated by several prior works VLMs still struggle with spatial understanding—particularly in 3D spatial relations[1, 2, 3]. Therefore, the interpretation of the results is limited, as it remains unclear whether the observed issues stem from the T2I model itself or from the evaluation protocol.\n\n• Although the authors demonstrate that fine-tuning improves performance, the paper does not explore other potential methods or strategies to address the lack of spatial understanding in T2I models.\n\n• While the authors incorporate a comprehensive human-in-the-loop process for dataset creation, the study does not include a comparison against a human baseline. This is significant, especially when introducing a new evaluation protocol, as it is unclear whether the proposed metric aligns with human judgment. The significant of VQA settings for evaluation is drop without showing this can compared with previous methods.\n\n• The error analysis is based solely on four domain types and relies on numerical accuracy. A qualitative, human-centered analysis of failure cases is needed to provide deeper insights into model weaknesses.\n\n[1] Liu et al. Visual Spatial Reasoning, TACL 2023.\n\n[2] Chen et al. SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilities, CVPR 2024. \n\n[3] Zhang et al. Do Vision-Language Models Represent Space and How? Evaluating Spatial Frame of Reference under Ambiguities, ICLR 2025.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "mXtzcGkR8C", "reviewer": "Reviewer_WeDg", "summary": "The paper introduces a new benchmark for critically evaluating the spatial understanding of current text-to-image models.", "questions": "* L85 - L86: It's not clear what the sentence means. Does it mean high-scoring samples from T2I models?  Also, which prompts are being referred to here?\n* Figure 1: Are the bottom formats from different eval frameworks? If so, which ones?\n* Could the authors provide comparisons to Nano Banana [1] in Figure 2 and in general?\n* Could the authors provide some broader trends from benchmark results? For example, do models with LLMs as text encoders (QwenImage [2], for example) show better performance than the ones without?\n* \"Refuse to answer (not guess)\" -- how is this taken into consideration during the evals?\n* Could the authors consider using the SPRIGHT [3] fine-tuning strategy and see if that helps? The strategy is to only train on images where the number of objects exceeds a certain threshold.\n* The SFT experiments include no diffusion-based models. Could the authors include an experiment with a diffusion-based model, say QwenImage?\n\nMisc:\n\n1. Some missing references: SPRIGHT [1], HEIM [4]\n\nReferences:\n\n[1] Nano Banana; https://gemini.google/us/overview/image-generation/?hl=en\n\n[2] Qwen-Image Technical Report; Wu et al.; 2025\n\n[3] Getting it Right: Improving Spatial Consistency in Text-to-Image Models; Chatterjee et al.; 2024\n\n[4] Holistic Evaluation of Text-To-Image Models; Lee et al.; 2023", "limitations": "", "rating": 2, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "* Well-motivated problem statement\n* Diverse evaluation dataset\n* Well-guiding principles behind dataset construction\n* Diverse results from many different models", "weaknesses": "* Unclarity in writing\n* Lack of QA in MLLM-synthesized prompts\n* Emphasis on using 77 tokens (L259) despite the fact that modern models go well beyond this context length", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "mX4e7Baml6", "reviewer": "Reviewer_awqq", "summary": "The paper introduces SpatialGenEval, a new benchmark designed to systematically evaluate the spatial intelligence of T2I models. SpatialGenEval consists of 1,230 long, information-dense prompts across 25 real-world scenes  and 10 spatial sub-domains. SpatialGenEval moves beyond existing benchmarks by creating information dense prompts and evaluate 21 state-ofthe-art models. Evaluation reveals that higher-order spatial reasoning remains a primary bottleneck in existing models. Further, SpatialT2I dataset is generated, which consists of 15,400 text-image pairs, fine-tuning on which leads to performance improvement in models such as Uniworld-V1 and OmniGen2.", "questions": "Please refer to weaknesses.", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 4, "contribution": 4, "strengths": "Having worked on spatial relationships in T2I models, I thank the authors for working on a very timely problem and doing a good job of it!\n\n1. The motivation is outlined very well and the paper is easy to follow with supporting examples for easier readability.\n2. Existing benchmarks, as rightly pointed by the paper, only present simpler prompts. SpatialGenEval does a comprehensive job of covering 10 spatial constraints, enabling deeper probing of models. \n3. Holistic evaluation of multiple models, both open and closed-source along with difference architectures such as diffusion / AR models.\n4. While the prompts and corresponding questions are generated from VLM's, a subsequent round of human validation is performed, which ensures a higher quality and avoids potential bias emanating from a proprietary LLM.", "weaknesses": "1. Reliance on VLM for evaluations - This is the major weakness of the current work. Generated images are evaluated with VLM's which themselves struggle with spatial reasoning. For example, a question such as :\n \"Based on the toddler's movement, what is the toddler facing towards?\" -- A: The ficus tree B: Away from the rubber duck C: The colorful blocks D: The window E: None ; will be difficult for most VLMs to answer as well. Despite Table 3 and 4, where meta comparisons to other benchmarks are shown, performing an alignment with human judgement will solidify the findings. \n\n2. What is the design choice behind having all prompts having all the 10 spatial constraints? Incremental complexity would paint a better picture of where models fail; additionally since all these constraints are essentially \"prompts\"; some of these shortcomings might just be text encoder shortcomings instead of the core image generator blocks. This also increases complexity on the VLM's to \"generate all 10 QA pairs at once\".\n\n3. While the intention is correct, some prompts/questions are ambiguous; for example in Spatial Causal Interaction - \"Under a spotlight, a large opera singer in a tuxedo stands center stage. <···>. The sound wave hits the glass, causing it to shatter\" - Since its an image, the glass can be shattered by a multiple number of possibilities; such as a rock thrown by someone in the crowd -- How are these cases handled?\n\n4. The SFT contribution seems like an after thought, some more details will be helpful; for example: was the text encoder fine-tuned as part of the experiments? Please refer to SPRIGHT (https://arxiv.org/abs/2404.01197) where authors discuss useful training details for improving spatial relationships in T2I models and covers multiple related data-centric nuances mentioned about in the current work such as relative sizes, spatial proximity etc.\n\nI will increase my score if the above points (especially 1) are addressed.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)