SpaCE-10: A Comprehensive Benchmark for Multimodal Large Language Models in Compositional Spatial Intelligence
Authors: Ziyang Gong
Wenhao Li
Xianzheng Ma
Songyuan Li
Zhaokai Wang
Songze Li
Jiayi Ji
Xue Yang
Gen Luo
Junchi Yan
Rongrong Ji
Venue: ICLR 2026 Poster (2026)
Keywords: Multimodal Large Language Model
Evaluation Benchmark
Compositional Spatial Intelligence
Primary area: datasets and benchmarks

Abstract: Multimodal Large Language Models (MLLMs) have achieved remarkable progress in various multimodal tasks. To pursue higher intelligence in space, MLLMs require integrating multiple atomic spatial capabilities to handle complex and dynamic tasks. However, existing benchmarks struggle to comprehensively evaluate the spatial intelligence of common MLLMs from the atomic level to the compositional level. To fill this gap, we present SpaCE-10, a comprehensive benchmark for compositional spatial evaluations. In SpaCE-10, we define 10 atomic spatial capabilities, which are combined to form 8 compositional capabilities. Based on these definitions, we propose a novel hierarchical annotation pipeline to generate high-quality and diverse question-answer (QA) pairs. With over 150+ hours of human expert effort, we obtain over 5k QA pairs for 811 real indoor scenes in SpaCE-10, which covers various evaluation settings like point cloud input and multi-choice QA. We conduct an extensive evaluation of common MLLMs on SpaCE-10 and find that even the most advanced MLLM still lags behind humans by large margins. Through our careful study, we also draw several significant findings that benefit the MLLM community. For example, we reveal that the shortcoming of counting capability greatly limits the compositional spatial capabilities of existing MLLMs. We will release the code and benchmark soon.

Published as a conference paper at ICLR 2026
SPACE-10: A COMPREHENSIVE BENCHMARK FOR MULTIMODAL LARGE LANGUAGE MODELS IN COM- POSITIONAL SPATIAL INTELLIGENCE
Ziyang Gong", Wenhao Li Oliver Ma Songyuan Li', Zhaokai Wang Songze L
Jiayi Ji Xue Yang' 1, Gen Luo Junchi Yan', Rongrong Ji²
SCS & SAIS & SAL, Shanghai Jiao Tong University
2 Xiamen University *Shanghai Artifical Intelligence Laboratory
"Sun Yat-sen University Fudan University National University of Singapore
Code https:///ittunt wtt.chhw..coomcccooommooeeeeeeee wwtp://///wwnnaaa/aaaaeeaea
ABSTRACT
Multimodal Large Language Models (MLLMs) have achieved remarkable progress in various multimodal tasks. To pursue higher intelligence in space, MLLMs require integrating multiple spatial capabilities, even for handling sim- ple and normal tasks. However, existing benchmarks struggle to comprehensively evaluate the spatial intelligence of common MILMs from the atomic level the compositional level. To fill this gap, we present SpaCE-10,, comprehen- sive benchmark for compositional spatial evaluations. In SpaCE-10. we define 10 atomic spatial capabilities, which are combined to form 8 compositional ca- pabilities. Based on these definitions, we propose novel hierarchical annotation pipeline to generate high-quality and diverse beestion-answer (QA) pairs. With over 150+ hours human expert effort, we obtain over 5k QA pairs for 811 real indoor scenes in SpaCE-10, 10, which covers various evaluation settings like point cloud input and multi-choice QA. We conduct an extensive evaluation of common MLLMs SpaCE-10 and find that even the most advanced MLLM still lags be- hind humans by large margins. Through our careful study, we also draw several significant findings that benefit the MILMM community. For example, that the shortcoming of counting capability greatly limits the compositional spatial capabilities of existing MLLMs. The Evaluation code has been released.
reveal
INTRODUCTION
Recent years have witnessed the rapid development of multimodal large language models (MLLMs) (OpenAl 2025) Bai al. (2025) Wang et al. (2025), which continually narrows the gap between machines and humans in multimodal tasks (Liu et al. 2024). The significant progress has motivated researchers to pursue higher machine intelligence in the real world (Huang et al., 2023 Hong al. 2023 Luo al. 2025). Focus the scene in Fig. [ imagine you about to head out the door and tell your home robot, forgot my watch, please bring to me. remember it's near the nightstand." To succeed, the robot must know what a watch is, plan to the nightstand, reason about spatial relations near, localize the watch among distractors, retrieve it, and return under di- verse changing viewpoints. Solving this 'simple, everyday task' requires the on-the-fly composition of a diverse set of spatial capabilities. This raises a central question: Can current MLLMs master these spatial capabilities and compose them seamlessly in real-world scenarios?
While existing benchmarks have made valuable explorations of the spatial intelligence of multi- modal large language models (MLLMs) (Yang et al., 2024 Ma al., 2024 Linghu et 2024 Yang et al. [025b] Jia et al. (2025). they seldom make these capabilities explicit or design tasks
that systematically combine them. Early benchmarks (Azuma et al. 2022; Ma al.) 2022 Yan
et al. 2023 Ye et al. 2022 Chen et al. 2020) mainly focus on the assessment of less-combined
"Equal Contribution
1 "Corrsponndence authors

---

Published as a conference paper at ICLR 2026
Spatial Capability Evaluation (SpaCE-10)
QA Demos
Alomic Capability Pool
Qt: Which opeaon does not ocmmon any objects in the scene?
Ci Object Recogattion
gry carpet, black TV
R. grann plani, Nack TV
ON
C. green plant, brown bod
D Ieomn doee, gray TV
CTt Spatial Lacalliationn
Capobility: CI, US, CT, CMC
Entlyy Presence
C3 Spotial Relationship
Ci Size Compurion
When facing
objecs, which descripcions correct?
A. Pant is lefl of TV.
Platl un the of TV: right dhor
CS Counting
C. Lamp right
Capability: CI,, C2, CT,
00- 00-Spatial Relationship
Knowledgr
CT Matti-riw Fimioa
Qb: Which following deariptions cumust?
Aming the oéjocts sdjscett the bed, there obyost can som iiome, but nonz for cleaning the floor.
Cle Furward Thisking
u. Among the choerss agjacett to the Cabneet, there etject can be seed provdee infe-tigtcttion, but none venilation C.
Obsrrcation
Capability CLCC
Reasoning
Perfermance Comparisso on SpaCEE
From DifTree 3D Point Cloads
Highly compeesive QA Types
6.2%
Gornad S4Tiab Thaah-
Datseets
Collect Cloud Snapehots
Integration
12.0%
LLme Viice F2R
384 Iluman Expert (lours
Human Expert Hlurs
17.95
Quedta VI
800+ Indocr Scenarice
Compositional QA
Morma Perkermaer
Accuracy
Figure 1: Overview of SpaCE-10 benchmark. SpaCE-10 10 takes over 150 human expert hours to collect 5k+ QA pairs in 811 indoor scenes, which can evaluate MLLMs from 10 atomic capabilities to 8 compositional capabilities. Through evaluations, SpaCE-10 indicates that even the most ad- vanced MLLM still lags far behind humans by large margins. Green cirle means the correct answer.
capabilities like object recognition and spatial localization, while compositional ones still remain be defined and evaluated. Recent benchmarks (Yang et al. 2024 2025b Jia et al. 2025 aim to evaluate the spatial intelligence of MLLMs through more compositional questions, but still fail to reflect the role of different spatial capabilities in compositional reasoning. More importantly, ex- isting spatial benchmarks struggle to satisfy the evaluation needs of existing MLLMs in terms of scenes, modalities, and question types, etc. As shown Tab. [ the number of scenarios in existing benchmarks is usually less than 400, which makes it difficult to cover various practical situations.
To fill these gaps, this paper proposes SpaCE- 10 (Spatial Capability Evaluation). a capability-focus question-answer (QA) benchmark with an atomic capability pool (Fig. D.. This pool highlights the 10 core spatial capabilities (C1-C10) for MLLMs in real-world deployment. In SpaCE-10, 10, there are 8 meticulously designed and systematically combined QA types, each of which covers more than atomic capabilities. Hence, SpaCE-10 can not only assess the Compositional Spatial Intelligence (CSI) of MILMs but also reflect the impact of different atomic capabilities in spatial comprehension.
Based on this design principle, we propose an innovative hierarchical annotation pipeline in SpaCE- 10. Specifically, we collect over 800 real indoor scanned scenes from four public datasets. For each scene, we present an automated pipeline to generate structured data that can describe different types of information in the scene, e.g., appearance and relationship. Based on this information, a multi- stage cemi-automated pipeline is adopted generate basic QA pairs, conduct quality verification, and perform the capability integration. Our SpaCIE100 consists of more than 5,000 high-quality QA pairs, covering various settings of existing MILLMs, e.g., point cloud input and multi-choice ques- tion types. As shown in Tab. SpaCE-10 demonstrates greater diversity than previous benchmarks in data distribution, annotation process, and evaluation settings, showing promising all-around eval- uation ability for compositional spatial intelligence.
We conduct extensive and systematic evaluations mainstream MLLMS on SpaCE- 10, including close-source source MLLMs and nearly 50 open-source MLLMs ranging from IB to 241B. Experimental results show that even the most advanced MLLMs are still far behind humans in compositional spa- tial intelligence, iee, 53.1 1 % of GPT-5 vs. 91. 21 of human. Meanwhile, 2D MILMMS demonstrate
2

---

Published as a conference paper at ICLR 2026
Comparison of SpaCE-10 with existing spatial benchmarks. Our SpaCE-10 con- tains the most diverse scenarios and QA types, covering various evaluation settings of existing MLLMs. CSI means the investigation of Compositional Spatial Intelligence. SCN, OBJ, HM3D, 3RS, ARK represent ScanNet (Dai et al. 2017), Objaverse, Habitat-Matterort--33 (Ramakrishnan al. 2021). 3RScan (Wald et al. 2019). ) and ArkitScene (Baruch et al.) (02T). Sim. denotes
Table 1:
similarity -bseld metrics, 2D&3D means the benchmarks support both 2D imagery and 3D point cloud-based MLLMs.
Dataset
Scenario Source
Scene
Q&A
Metric
2D
Multi-Answer
CSI
BDQA Zhao al.
2022
SCN
902
Sim.
ScanQA Azuma
2022
SCN
167
10k
Sim.
FEA3DGQ Zhao
20122
SCN
3.9k
Sim.
SQAID Ma
2022
CLEVER3D Van
X
X
3D-LLM Houg el
OBLSCN.HMMD
30k
Sim.
X
X
M3DBench (LT al.
2023
SCN
1.5k
Sim.+LLM,
X
X
MSQA (Linghu el 2024
SCN,3RS.ARK
381
3.5k
LLM
X
X
VSI Yang ala 2024
SCN, SCNNN+
288
5.0k
Acc.
X
X
SpaCE-10 (Ours)
SCN, 3RS, ARK. SCN++
811
5.0k
Acc.
much stronger capabilities than 3D MLLMs on SpaCE-10, showing great potential for masge-based spatial reasoning. addition, existing MLLMS greatly fall short in multiple-ansee ing their inferior complex reasoning abilities. Our further study also reveals that the shortcoming of counting capability greatly limits the compositional spatial capabilities of existing MILMs. These findings provide valuable directions for the community to develop more capable MLLMs in terms of spatial intelligence. Overall, our main contributions are summarized as follows:
QAs. suggest-
We present SpaCE-10, a comprehensive benchmark for compositional spatial intelligence. SpaCE-10 is the most diverse benchmark that assess the capabilities of MLLMs from the atomic level to the compositional level. also covers various evaluation settings, including 3D inputs and multi-choice questions.
We propose innovative hierarchical annotation pipeline in SpaCE-10, which first produces structured descriptions scenes via an automated pipeline and then generates compositional QA pairs through multi-stage semi-automated pipeline. The hierarchical pipeline ensures the quality, diversity, and controllability of the generated QA pairs.
We conduct extensive evaluations for nearly open- and close-source MILMs on SpaCE-10 Through our in-depth analysis, draw several significant findings that will benefit the spatial intelligence of future MLLMs in the community.
2 RELATED WORK
Early works start spatial intelligence mainly with two directions: (i) 2D abstract reasoning with
logic-puzzle and geometric-panel tasks (e.g.. Raven-style matrices) (Gntthier 2022 Xiao et al.]
2024 Xu 2025 Ramakrishnan al. 2024), and (ii) simplified images with only few objects, where queries test basic relations such as above/below/ettrrigttiiz (Tong et al. 2024b Kamath 2023 Bagherinezhad et al. 2016). As attention shifts to realistic environments,
scene benchmarks (Azuma et al. 2022:10 Ma et al. 2022, Linghu et 2024, Ye et al., 2022, Li et al. 2023) emerge and expand to richer tasks, such as route planning and situated perception from specified viewpoints. However, they typically adopt point-cloud inputs and still treat spatial ability as single block. With the rise of MLLMs, newer studies (Ma et al. 2024 Yang et al., 2024 2025b Jia al.) 2025 He et al. 2025) examine 3D spatial comprehension directly from 2D images or videos. Yet most works do not make capabilities explicit or delve into them. To this end, SpaCE-10 10 defines an atomic capability pool (C1-C10) across perception and reasoning assess model's CSL. It also supports both 2D images and 3D point clouds, offering new perspective for advancing spatial intelligence in MILMMs.
3

---

Published as a conference paper at ICLR 2026
3 SPACE-10
3.1 OVERVIEW
The images in SpaCE-10 are from four 3D indoor scene scan datasets, including ScanNet++, ScanNet (SCN), 3RScan (3RS), and ARkitScene (ARK) with over 800 real indoor scenes, including a wide variety of environments such as living rooms, classrooms, bathrooms, kitchens, and more. Finally, SpaCE-10 consists of 8 QA types that are EQ (Entity Quantifica- tion). SQ (Scene Quantification), SA (Size Assessment), OO (Object-Object Spatial Relationship), OS (Object-Scene Spatial Relationship), EP (Entity Presence), FR (Functional Reasoning), and SP (Spatial Planning), Also it includes 10 atomic spatial capabilities of C1 (Object Recognition), C2 (Spatial Localization), C3 (Spatial Relationship), C4 (Size Comparison), C5 (Counting), C6 (Funci- ton Knowledge), C7 (Multi-vier nielt Fusion), C8 (Forward Thinking). C9 (Reverse Reasoning), and C10 (Situated Observation). Definition of QA and capabilities are in Sec. B Band respectively.
Construction
Analysis In Fig. 2 (a). we show the numbers of each type of QA in SpaCE-10. Among them, blue ones belong to Perception, and purple ones belong to Reasoning. In (b) and (C), we demonstrate the average vocabulary size (unique word number) and character length of the Question and Option of QA types. EQ and SQ are excluded because their options are numbers.
For Capability, in (d) we show the number of capability contained in each QA type. (e) represents the capability coverage in each QA type, revealing a three-fier hierarchy. (1) Foundation: C1 (Object Recognition), C7 (Multi-view Fusion), C8 (Forward Thinking) appear in 100% of QA types since they are prerequisites for almost every task. (2) Bridge: C2 (Spatial Localization) covers 87% and links both "quantification' and 'relation/plan' QA types. (3) Specialist: C3 (Spatial Relationship) and C10 (Situated Observation) are mid- frderreeency (45%), activated when viewpoint-dependent re- lations matter; C9 (Reverse Reasoning) attaches to causal/plan problems; C4 (Size Comparison), C5 (Counting), and C6 (Function Knowledge) appear in specific QA types. (f) shows the co-occurrence of each capability. Beyond of (d), we observe the tightest pair is C3 (Spatial Relationship) with C10 (Situated Observation) = 4/8 (OO, OS, FR. SP), which means they appear together QA types, showing that when relative relations are asked, specified viewpoint usually required. Moreover, C9 (Reverse Reasoning) appears mainly with planning-style tasks: C2 (Spatial Localization) with C9 (Reverse Reasoning)+ 3/8 (EP, FR, SP), while C3 (Spatial Relationship) with C9 (Reverse Rea- soning) = 2/8 (FR, SP) and C10 (Situated Observation) with C9 (Reverse Reasoning) = 2/8 (FR, SP). The design motivation of the atomic capability pool can be found at Sec. B
3.2 HIERARCHICAL ANNOTATION PIPELINE
Overview. As shown in Fig (a), our annotation pipeline consists of stages from data preparation to high-quality QA generation. In stage 1, we employ 3 human experts to manually collect snapshots of 3D point cloud scans from 4 to 6 different directions, with over 38 hours of human expertise to maintain the high quality. In stage 2, combine the collected snapshots and video frames to generate the structural data that describes different aspects of information in the scene. In stage 3, we leverage GPT-4o generate over 10k Basic QA covering atomic capabilities with former structural data. In stage 4, human experts again manually filter the low-quality QA pairs, costing over 112 hours of experts and resulting in over 8k+ QA pairs. Finally, in stage 5, we design template-based strategy to integrate the spatial capabilities in QA types, yielding the final QA pairs. The ablation study on the effectiveness of the annotation pipeline can be found at the Sec.
Structural Data Generation. As shown in Fig. 3 (b), this pipeline follows progressive design to generate structural data with steps: (1) Initially. 10 keyframes are selected from the video of each scenario by combining the CLIP vision encoder (Radford et al., (02T) and the k-means algorithm. (2) Based on 2D keyframes, we leverage GPT-4o to generate a 2D caption for each scene, which covers information of appearance, size, and spatial relationships. (3) We reuse GPT- 40 as an inspector to refine the 2D captions by removing incorrect and redundant information. (4) The manually collected 3D snapshots will be combined with keyframes for 3D caption generation. These high-quality snapshots contain rich global information about the whole scenes, which can provide considerable scene-level spatial information. (5) The inspector again checks and refines the 3D caption. (6) Finally, the rule-based extractor will be applied to obtain structural data for the following QA generation.
4

---

Published as a conference paper at ICLR 2026
(a) OA Type Distribution
(b) Vecabulary Size
[cl Character Length
0.00
Nalapind abc
150
20-
600
Weabhin Oze
120
Chsracter
200
0
0-
(d) Capebility Composition Number
(e) Capability Coverage
ifi Capability Co-occurrence
C10
Compoctio Number
C2-
52
C10
NO
100
Coverago 196
Figure 2: Dataset analysis of SpaCE-10. (a) Number distribution of each QA type. SpaCE-10 consists of 8 QA types that are EQ (Entity Quantification), SQ (Scene Quantification), SA (Size Assessment), 00 (Object-Object Spatial Relationship), OS (Object-Scenc Spatial Relationship), EP (Entity Presence), FR (Functional Reasoning). and SP (Spatial Planning). (b) Average vocabulary size per QA type for question, option, and average. (c) Average character length per QA type. (d) Coverage of the atomic capabilities (C1-C10). (e) The correlation between human expert accuracy and average character length across six QA types. (f) Capability co-occurrence matrix.
Gn thet to milaceeaive The food willte.
Olyea-Level Infogmation
2D Oity Level Captions
2D Refinel Captions
Exper
Snapehot
2D Mulli-iiww Images
Tbe white refrgenner
The while vrfppeatee bad rwo rioe beadles
Stage
Coptioger
kybbic acee and
Bdow de vnlinct
Inspector
ix whit sowe and white calaaai bood.
celiuets
Goneri Struttraal Data
Keyfhanes
cabeat
Suece Level Speiial Infoenndion)
Captiner
Stapebute of 3D Sozme from five dinsctions
Stage 4: Qualit Velifeetics Expert Hours,
On right
On the left side romm
"Ohject Appan The white refryymnce two vertied lundies ),-1]
"Surtial datioosaip" 1'Spatall Relationchp 1he
Extractor
the right side of
Inspector
Oo the side whil
Cipobtly loagoriiee
right
Si+ QA
Structura Date
Refined BD Captiors
3D Secre-Lovel Cappiius
(4) Outline of Pipeline
Structural Pipeline Details
Figure 3: Illustration of hierarchical annotation pipeline. We generate structural data to construct over 10k QA pairs, and performs capability integration to obtain over 5k QA pairs with 10 compositional capabilities. This process takes over 150 expert hours for data collection and filtering.
QA Generation. For QA generation, adopt 3 approaches: template-based, MLLM-based, and human-based generation. For EQ, we use template-based method, and SP is manually designed by 2 human expert. For rest QA types, we leverage GPT-4o generate. Notably, we mentioned earlier, the questions in SpaCE-10 10 are composed of multiple atomic capabilities. However, such highly integrated questions are difficult for current MLLMs to generate directly. Therefore, for SA, 00, OS, EP, FR five QA types, we propose to first generate basic version of QA, namely basic QA, and then enhance its embedded capabilities. The details of QA generation in Sec. D
Cross- Crass-Capabylity Integration Strategy. We apply three strategies to integrate cross-capabilittes:: (1) For SA, 00, and OS, we integrate an additional C7 (Multi-view Fusion). In the original setting, the four options usually refer to the same object (or object pair). We regroup multiple same-type, same-scene QAs into a single QA so that each option points todifferent objects. This forces MLLMs to search across the entire scene find all mentioned entities, enabling more holistic spatial perception. (2) For EP, we add C7 and C9 (Reverse Reasoning), expanding the question to involve multiple objects. We then reverse the question type from 'which object exists' to 'which object
5

---

Published as a conference paper at ICLR 2026
does not exist', integrating reverse reasoning capabilities. (3) For FR, add C7, C9, and C10 (Situated Observation). The basic FR question asks 'which option correctly describes the function of object near central object'. The four options are structured as 'Object' which can be used to "Function". In the integrated version, the question is simply revised to: 'Which of the following is correct?'. But Each option involves different central object, with the structure: 'Among the objects adjacent to central object, there is one can be used to 'Function', but lacks two objects can be used to 'Function'. This change prevents potential leakage of prior knowledge in options, such as object names, and places greater emphasis on the model's 's understanding of functional roles, The examples are in the Sec. D.2
3.3 QUALITY VERIFICATION
For the quality verification process, we rely on manual filtering by 3 human experts. We set up an user interface for validation and employed two human experts to perform the evaluation. In this process, the evaluation criteria include checking for incorrect options, invalid answers, missing data, or questions involving objects not present the snapshots. The low-quality data will be directly deleted. This quality control process takes over 112 hours to check and filter low-quality QA pairs. By employing human validation, we ensure that only high-quality and contextually accurate questions are retained for the final benchmark. Related visualizations are attached in the Sec. H
4 EXPERIMENTS
4.1 SETUP
In our experiments, we test nearly 50 close-source and open-source MILMS on SpaCE-10, including 3D MILMs LEO and GPT4Scene, GPT-5, IL3.5, and so on. During the evaluation, except for Leo, which is evaluated by their own framework, other MLLMs are evaluated by the LMMs- Eval (Zhang et al. 2024) with 8 frames as input, and open-source MLLMs are tested on Nvidia A100 GPUs For the response and answer aligmment, we follow the prompt of MMBench (Liu et al., 1--0 for the judgment. Notably, we removed the random assignment in LMMs-Eval, so the model's 's performance is likely to be lower than the random baseline (25%).
2024) and use
4.2 OVERALL RESULTS
Human vs. MLLMs. We first compare human and MILMs' performance on SpaCE-10. The 'Human' score is taken from the average score of 6 human experts. The results in Tab. indicate that although human performance does not meet expectations when facing with more complex numerical and reasoning tasks, the total score of 91.2% is still significantly higher than all existing MLLMs. In comparison, the best open-source MILM only achieves an average score of 55.0%, and the best close- source models only achieve score of 53.1%. These results demonstrate that the compositional spatial intelligence of MLLMs is far below the human level.
3D MLLMs vs. 2D MLLMs. In Tab. 2 we evaluate LEO and GPT4Scene as representatives of 3D-related MILMs. Notably, input to the LEO model requires point clouds of the objects relevant to the questions. To ensure fair comparison, we adjust the input for LEO by randomly sampling 1024 points from the entire scene's point clouds. The results show that LEO scored 11. % overall in SpaCE-10, which is significantly lower than GPT4Scene (34.5%). Compared MLLMs with scale < 7B, the performance of LEO-7B is also substantially lower. We argue that one of the limita- tions of current 3D MILMs is that they are designed to focus on specific objects and have difficulty processing the entire scene's 's point clouds as input. Additionally. they likely sacrifice multimodal conversational abilities for understanding scans. These results also indicate that 2D MILMS have greater potential in visual spatial intelligence comprehension than 3D MILMs. Open-Source vs. Close-Source. In close-souce MLLMS experiments, GPT-5 achieves the best overall performance, ranking 8th with score of 53.2% It excels in perception tasks, such as in SA (Size Assessment) with 69.7%, OO Sobject-Object Spatial Relationship) with 60.7%, and FR (Functional Reasoning) with 66.8%, indicating strong accuracy in recognizing size and position. Additionally, GPT-4o achieves the highest score in EQ (Entity Quantification) among all tested models, with a score of 58.3%. However, GPT-4o struggles with SP (Spatial Planning), where it scores the lowest among all

---

Published as conference paper at ICLR 2026
Table 2: Single-answer performance ranking of nearly 50 MILMs on SpaCE-10 benchmark.
Perception
Reasoning
Models
EQ
SQ
SA
00
0S
EP
FR
SP
Human
91.3
88.5
90.2
93.4
95.6
91.1
90.3
86.3
91.2
MILMS
LEO-7B Hoang et al. 2023
47
15.8
0.0
16.7
16.5
25.2
5.5
5.7
13.3
11.1
GPT-4Scche-71 2025
36
30,9
37.7
38.0
38,9
41.6
29.5
28.0
32.5
34.5
Close Source 20 MLLMS
GPT-5 O 2025
42.0
71.0
60.7
36.5
36.0
53.4
GP1--4--0-4-1 -X0 Kehiam of 2024
9
58.3
92.8
56.2
58.3
56.2
41.6
52.2
23.7
49.0
Gemini-2.00 2.0-aash- Thinsing Tean ar
20
34.3
25.6
53.1
42.6
53.8
42.2
46.7
31.2
42.2
Clande 3.7-onee
ch
14
46.0
44.3
49.1
49.1
44.3
49.3
25.0
46.2
Open Source 2D MLLMS
Scale 78
IntemVL2.5-18
Chen al 2024
33
33.0
54.1
18.8
43.6
29.9
26.7
41.0
23.7
35.3
InteVVL3-18
/Iu af 20153
24
301.7
35.7
27.9
44.6
31.6
47.8
41.9
30.0
41.4
ImemVV1...5-1f
21725
38
34.8
33.8
33.5
27.0
InterVL3328B
Zhu eral 2025
17
41.5
45.9
45.4
45.7
31.9
45.7
48.7
41.3
44.2
IntenV1.3.-22
Wiing
2025
35
35.6
28.4
42.2
45.7
32.3
20.1
45.8
20.0
34,6
Qwen2.5.VLL-8B-11
2025
34
31.7
23.3
47.1
51.7
31.6
25.5
37.0
21,2
34,8
SpaceOm
40
21.8
24.5
47.3
49.7
32.7
21.9
36.7
25.0
33.2
SpaceQees
SpaceThinker
37
32.7
22.4
46.7
50.5
33.4
22.4
36.9
24,2
34.1
VILAII5-3B (ingst
2124
15
25.0
9.1
31.7
34,6
31.6
35.3
12.9
33.7
26.1
IntenVL2.550m
Min(CPM-s4-48
27
38.1
41.1
49.0
36.5
29.3
50.0
39.0
IntemVLL.5-1B
Wain 01
21745
30
38.9
12.9
48.7
50.7
27.9
33.9
37.0
35.0
35.5
Scole 141
Qwen225-V1--78 1e-m2-5-Vi-77---niirur
ct..l
32.7
32.3
27.5
33.3
11.
30.5
57.4
OnsVV-Oeeeiiiir755 (hiood 2024
15
37.4
33.8
46.4
$7.3
34.5
43.3
61.6
21.2
45.2
MiMo-VL-RL.. 8B
Xiabin 2025
31
23.7
35.0
46.4
41.3
34.7
32.2
32.5
36.1
35.5
Cambrian-SB
Tunkereum
41
22.6
18.6
34.8
32.6
32.3
25.1
41.
23.7
29.5
20.9
IntemVLL2.555
[ 2020
21
33.2
36.0
$0.0
55,0
33.5
27.1
59.1
32.5
41.8
IntemVL3-SB
2 F1763
25
36.6
29.5
42.9
$1.7
34.5
26.6
60.6
37.5
40.0
IntemVLL3.566f
Weing
7075
26
37.1
28.5
61.7
49.8
35.4
17.6
54.8
36.3
39.5
Gemma3-128
[isun
22
39.7
IneemVL3...14B
ef
10
41.0
47.6
65.3
$2.1
34.5
45.4
54.3
30.0
48.8
V 14B==Scafee2 720
IntemVLL3.5-208-- 3.BB--00---BB [Wang
2025
8
37.4
43.1
64.1
41.4
54.1
$7.6
28.8
$1.6
MmcmVL2.5526BB.. (Ch 5314
Gemma3-27B Te 603633
23
39.4
21.7
63.5
48.5
37.8
33.2
$1.5
30.0
41.5
Swen2,5-VL-32--1:100
2025
41
19.9
26.5
48.9
36.8
32.3
31.1
30.1
32.5
32.6
Inlem/VL2.55388 (Chmn
16
38.1
36.1
64.4
$4.3
36.8
27.4
63.0
37.5
45.1
ImernV1.5-388
36.3
IntmmLL3.5W88 Wang
Y Scnle > 726
ML.V-45V Team al. (375h
7
38.9
41.1
65.1
61.1
36.7
61.2
49.3
31.3
$1.6
LLaVA-Oneiioes
67.9
46.7
67.3
$2.8
Qwen2.5-VL-72B. (Ba di
55.7
IntcnVL2.57788 Chea 2023
13
27.8
45.0
62.4
64.4
40.3
23.7
67.3
40.0
47.1
IntemVL3-788 of 0025)
6
36.8
18.2
65.3
61.6
43.8
44.4
61.3
46.3
$2.5
11
37..3
29
66.4
62.9
37.8
38.6
64.0
26.8
47.9
IntemVL3.52211B
Wing
Models proposed by RemyxAl (LMs scries,
tasks, suggesting a limitation in scene-level planning ability. In open-source MILMs, among mod- delivers outstanding performance, ranking first overall with a score of 55.0%. It outperforms other models in nearly all tasks. These results suggest that the gap between open-source and close-source models has been significantly narrowed, and some open-source MLLMs even outperform close-source models, especially in compositional spatial intelligence.
els with scale of over 72B, LL3.5-241BB
Single-Answer vs. Multiple-Answer. In this experiment, we choose two close MLLM series of similar performance to make a comparison on single-answer and multiple-answer questions. The results in Tab. show that MILMs perform significantly worse multiple-answer tasks compared to single-answer tasks. Smaller models like Intern IB and 2B score over 30.0% on single- answer tasks, while in multiple-answer tasks, their scores often fall below 5%, which is even worse than random selection. As the model size increases to 78B, MLLMs show greater robustness for different QA types. These results lead to interesting preliminary conclusion: smaller models may overfit to the single-answar task format, while larger models seem to have learned more fundamental compositional spatial intelligence. However, the Qwen series exhibits almost opposite trend, with

---

Published as a conference paper at ICLR 2026
Table 3: Performance of MLLMs on single-choice and multiple-choice QA pairs. Results show that MLLMs, especially smaller ones, tend to overfit to single-choice questions.
Single-choice (3091)
Multiple-choice (970)
Overall
Score 1
Models
SA
00
EP
FR
00
OS
FR
Single
Mulippe
multiple/ /sinle
InfornVVL..
12.5
Chen
19.8
3.9
4.7
32.3
27.0
36.6
28.8
21.7
48.0
4.7
2.0
1.3
1.5
3.5
32.4
4.1
0.13
InteraV12..4 59
Cheiee ul 2012
50.0
55.0
33.6
27.1
59.1
8.4
14.8
8.7
12.9
1.5
45.0
10.6
0.24
1nteraV1.25 380
Chen 2044
64.4
54.3
36.8
27.4
65.0
47.8
37.9
7.3
46.8
49.2
InternV11.25
62.4
12.1
51.6
Quesz
Serka
Qwen225 VL 3B Instruct
Bal al 2025
47.1
51.7
31.6
25.5
37.0
21.7
24.1
3,0
15.3
20.7
38.6
17.9
0,66
Qwen2.5-V1_ Instruct
36.9
35.3
32.3
27.6
34.2
20.2
118
14.7
13.4
10.4
35.3
0.42
Qwee2.5 Instud
48.9
36.8
31.1
30.1
6.9
8.0
35.8
Qwen2.5 VL. 72B Instrctt
15 01 0725
55.7
80.9
32.1
36.5
38.0
17.2
13.3
13.0
15.7
15.9
40.5
15,0
0.37
Others
Cambeian 8B
2024g
34.8
32.6
128
5.9
20.6
33.2
VILALS
27.5
32.7
12.4
26.7
23.3
LLaVA-One Viseo-21 221 m.al
2006
67.9
64.5
60.3
46.7
67.3
32.0
133
34.7
35.8
57.3
30.9
0.54
Table 4: Accuracy comparison on basic and compositional QA pairs.
The results reveal the
relationship between the performance and integration of capacities.
Task
Integrated
Capability
InternVL2..-1B
VL2.5-8B
Owen2.5-VL-3B
Owen2.5-VL-7B
Average-C(%))
C1,C2CA.CS
37,8
66.4
60.0
64,8
57.3
-07
18 ( 19.0)
50.0 () 16.4)
47.1 ( 12.9)
36.9 (J 27.9)
38.2 (4 19:1)
CLCC2.1.C8.C..10
52.0
56.4 51.7 (1.4.)
00
C7
43.6 (1 8.4)
55.0 11.4)
(4.5.)
46.4 (4 7.6)
os
CI.C2.3.9..10
-07
99 21.6)
44,2
(4 21.2)
$1.5 t) 19.2)
32.3 22.0)
54.3
500 () 21/0)
KP
x
C1,CL.8
-C7,C9
26.8 (1.39.8
66.6
27.J (4 48.7)
75,8
25.5 (). 37.99
63.4
27.6 (4 37.7)
65.3
67.8 26.8 [] 41.0)
FR
x
C1,C2,C3,C6.3
-C7,09C10
41.0 29.9)
70.9
59.1 ()) 30.6)
89.7
37.0 48.6)
85.6
34.2 (1.55))
85.7
42.8 []] 42.9)
83.0
high scores in the smaller models, reaching 0.46. As the parameters increase, the scores decrease, but overall, the performance remains normal.
4.3 CAPABILITY ANALYSIS
Atomic Capability vs. Compositional Capabilities. Firstly, the Tab. 4 shows that on the 5 types of questions with more compositional abilities, the performance of the four models all drops signifi- cantly. For the questions with integrating C7 (Multi-view Fusion) capability, the models' accuracy in SA (Size Assessment), OO (Object-Object Spatial Relationship), and OS (Object- Relationship) tasks decreases by 19.1%, %, 7.6%, and 21.0%, respectively. Despite the drop, the ac- curacy of SA and OO still remains at decent level. Secondly, in the EP task with both C7 and C9 (Reverse Reasoning) abilities integrated, the models' overall accuracy plummets from 67.8% without integration to 26.8%, huge drop of 41.0%. Similarly, for the FR task, which with the most compositional capacities with an additional C10 (Situated Observation), the four models' average score crashes from 83.0% to 42.8%, an even larger drop of 42.9%.
Spatial
These results partially reveal the relationship between accuracy and capability. As more abilities are incorporated, model performance declines to varying degrees, with some decreases even exceeding 50.0%. This indicates that current models have a limited grasp of integrated spatial intelligence, thereby highlighting the necessity of Sp
Spatial Capability Breakdown. To better understand the strengths and weaknesses of the current MLLMs across various atomic capabilities. we constructed a model accuracy and atomic capability score matrix (Fig. 4 to associate QA accuracy with capabilities. Notably, this matrix is an un- weighted average, so diagnoses the model's capability independent of task weights in calculating accuracy. About the calculation details, the QA and capability mapping is shown in Tab, 5 and the calculation method is shown in Sec. B Moreover, because CI (Object Recognition), C7 (Multi-view Fusion), and C8 (Forward Thinking) appear in all eight QA types (100% coverage), their scores the model's ' eight QA-type average accuracies. Our core findings are as follows.
(1) Strengths and Weaknesses: C4 (Size Comparison) is consistently the strongest atomic capabil- ity across almost all models, whereas C5 (Counting) is uniformly the weakest. This pattern indi- cates that continuous magnitude judgments are comparatively well handled, while discrete numeros-
8

---

Published as a conference paper at ICLR 2026
Hones
90.8
91.2
91.4
90.2
89.9
89.4
90.8
90.9
89.2
91.4
90.5
Multimode Largee Languigg Model
47.4
49.5
47.6
56.2
45.5
42.5
47.4
47.4
39.2
47.6
47.0
41.2
43.4
43.6
33.1
29.9
36.2
41.2
41.2
40.0
43.6
41.3
10
33.8
34.5
36.0
42.2
32.0
37.1
33.8
33.0
28.6
36.0
34.8
0wen2.541-30
33.8
35.1
35.4
47.1
27.3
30.1
33.8
33.8
27.9
35.4
33.9
Omerd 745-288
38.0
38.4
36.2
39.1
33.8
36.5
38.0
38.0
36.1
36.2
38.6
50.7
52.4
52.1
67.9
41.2
52.0
30.7
50.7
50.1
52.1
52.1
37.5
<30
Awerag
Spatial Capability
Figure 4: Results of representative MLLMs on 10 atomic capabilities of SpaCE-10. Each value reflects the model's average accuracy (%) across all question types involving the respective spatial capability (CI-C10), as defined in the benchmark's task-to-capability mapping.
ity under occlusion/clutter and across views remains a persistent perceptual challenge. Notably, while
achieves the highest overall capability (52.3%) and accuracy (55.0%) av- erage among all models, its performance on C5 (37.5%) is substantially lower than that of GPT-4o (45.5%), whose overall accuracy is 49.0%. This contrast potentially highlights that closed-source models may possess better generalization in tougher capabilities.
Interestingly, comparing GLM-4.5V, LLaVA- OneVision-7221 and VL..55L33.552411 we observe that their task-weighted overall accuracies 51.6%, 52.6%, and 55.0%, respectively. Yet GLM's capability average is only 48. 1%, below LLaVA's 52. 1%. Conversely, Intern VL..522L..5551' overall accuracy exceeds LLaVA by 2.4% while its capability average differs by only 0.3%. This comparison highlights a crucial fact: high accuracy does not necessarily indicate high capability. Accuracy in Tab. elmphassses 'task completion rate", whereas the Capability matrix in Fig. Dis is another dimension that reveals how MLLMs master di- verse spatial capabilities. Relying solely on overall accuracy can therefore mislead about model's generalizability. The takeaway that future efforts boost MILMM spatial intelligence must look beyond accuracy and shore up capabilities, such as C3 (Spatial Relationship), C10 (Situated Obser- vation), and C5 (Counting), to achieve truly robust spatial reasoning. This divergence also validates the necessity of our capability-basee analysis.
(2) Similar Accuracy # Similar Capability:
(3) Scaling improves, but does not break bottlenecks: Scaling model parameters under our evaluation framework indeed yields significant overall performance gains. For example, Intern VL3.5 improves its capability average from 34.8% at 2B parameters to 52.3% at 241B parameters, showing that larger models generally master broader set of spatial abilities. However, this scaling trend is not uniform across all capabilities. In particular, for C5 (Counting). even the largest Intern VL3.5-2411 achieves only 37.5%, far below its strength in other dimensions and still substantially weaker than human performance (89.9%). This indicates that parameter scaling alone struggles to deliver qualitative improvements in discrete numerical reasoning within spatial contexts. By contrast, models like GPT-4o, GLM-4.5V, and ALLVA--One One Vision-72B. achieve superior scores in C5. This suggests that architectural design, training strategy, and data diversity may play more decisive role than raw scaling for certain atomic spatial capabilities.
Spatial Capability Improvement Taken together, our findings point to two potential paths for im- proving spatial intelligence in MILMss (1) Counting-focused supervision: curate training sets that stress occlusion, crowding, and multi-view consistency to build reliable discrete counting ability in realistic scenes; (2) Capability- training beyond scale: while increasing model size im- proves overall accuracy and the capability average, gains on the view-conditionel relation chain of C3 (Spatial Relationship) and C10 (Situated Observation) remain limited. Thus, future work could curate training data from a capability perspective and adopt training strategies (e.g. post-training) based different capability compositions.

---

Published as a conference paper at ICLR 2026
5 CONCLUSION
In this paper, we propose SpaCE-10, a comprehensive benchmark for evaluating compositional spa- tial intelligence Multimodal Large Language Models (MLLMs). SpaCE-10 covers evaluations of MLLMs from 10 atomic spatial capabilities to 8 compositional capabilities. In SpaCE-10, we col- lect images and point clouds from 800 scenes and design a hierarchical annotation pipeline to pro- duce over 5k high-quality Through extensive evaluation on nearly 50 MILMM, we reveal critical limitations of current MILM LMs and draw several findings that are beneficial to future work in the community. We believe these stud- ies will provide an invaluable hint for future research toward human-level machine intelligence.
crestion-answer pairs, covering various evaluation settings of MLLMs.
ACKNOWLEDGMENT
This work was supported by the National Natural Science Foundation of China (62506229), Natural Science Foundation of Shanghai under 1.52R440220 Shanghai Qi Yuan Innovation Foundation, and Scientific Research Innovation Capability Support Project for Young Faculty (U40) of the Ministry of Education of China (SRICSPYR-2Y2025019)
ETHICS STATEMENT
All visual data in SpaCE- 10 are sourced from publicly available datasets, ensuring compliance with original data licenses and no privacy violations. Human experts involved in annotation and evalu- ation were informed the task purpose, and their contributions were voluntary. We checked for potential biases across scenes and QA types achieve balanced evaluation of specific MILMM capa- bilities. The benchmark and evaluation code will he released with clear usage guidelines to prevent misuse for misleading spatial reasoning applications.
REPRODUCIBILITY STATEMENT
We provide details of the data curation and evaluation in the main text and appendix to ensure reproducibility. Our source code and data are publicly released.
REFERENCES
The claude 3 model family: Opus, sonnet, haiku. URL http:ttps/app...mmttccccoooo
orq/Corpus 268232499
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Ale- man, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, al. Gpt-4 technical report. arXiv preprint arXi::2303.08774, 2023.
Daichi Azuma, Taiki Miyanishi, Shuhei Kurita, and Motoaki Kawanabe. Scanqa: swering for spatial scene understanding. In proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 19129-191399 2022.
3d question an-
Hessam Bagherinezhad. Hannanch Hajishirzi, Yejin Choi, and Ali Farhadi. Are elephants bigger than butterflies? reasoning about sizes of objects. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 30, 2016
Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. Qwen2. 5-vl technical report. arXiv preprint 250. 2025.
Gilad Baruch, Zhuoyuan Chen, Afshin Dehghan, Tal Dimry, Yuri Feigin, Peter Fu, Thomas Gebauer, Brandon Joffe, Daniel Kurz, Arik Schwartz, et al. Arkitscenes: A diverse real-world dataset for 3d indoor scene understanding using mobile rgb-d data. arXiv preprint arXiv:21 1 1.08997, 2021.
10

---

Published as a conference paper at ICLR 2026
Holger Caesar, Varun Bankiti, Alex H Lang, Sourabh Vora, Venice Erin Liong, Qiang Xu, Anush Krishnan, Yu Pan, Giancarlo Baldan, and Oscar Beijbom. nuscenes: multimodal dataset for autonomous driving. In Proceedings the IEEE/CVF conference computer vision and pattern recognition, pp. 11621-16631 2020.
Dave Zhenyu Chen, Angel X Chang, and Matthias Nießner. Scanrefer: 3d object localization in rgb-d scans using natural language. In European conference on computer vision, pp. 202-221. Springer, 2020.
Zhe Chen, Weiyun Wang. Yue Cao, Yangzhou Liu, Zhangwei Gao, Erfei Cui, Jinguo Zhu, Shen- glong Ye, Hao Tian, Zhaoyang Liu, al. Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling. arXiv preprint arXiv:2412.05271, 2024.
An-Chieh Cheng, Hongxu Yin, Yang Fu, Qiushan Guo. Ruihan Yang, Jan Kautz, Xiaolong Wang, and Sifei Liu. Spatialrgpt: Grounded spatial reasoning in visionllanguage models. Advances in Neural Information Processing Systems, 37:135062 135093, 2024.
Angela Dai, Angel X Chang, Manolis Savva, Maciej Halber, Thomas Funkhouser, and Matthias Nießner. Scannet: Richly-annotated reconstructions of indoor scenes. In Proceedings of the IEEE conference computer vision and pattern recognition, pp. 5828-58889 2017.
Corentin Gonthier. Cross-cultural differences in visuo-spatial processing and the falture-fairness of visuo-spatial intelligence tests: An integrative review and model for matrices tasks. Cognitive Research: Principles and Implications, 7(1):11, 2022.
Yuping He, Yifei Huang, Guo Chen, Baoqi Pei, Jilan Xu, Tong Lu, and Jiangmiao Pang- Egoexobench: A benchmark for first-and third-person view video understanding in mllms. arXiv preprint wrXi::2507.18342, 2025.
Yining Hong, Haoyu Zhen, Peihao Chen, Shuhong Zheng, Yilun Du, Zhenfang Chen, and Chuang Gan. 3d-lIm: Injecting the 3d world into large language models. Advances in Neural Information Processing Systems, 36:20482-20494, 2023.
Jiangyong Huang, Silong Yong. Xiaojian Ma. Xiongkun Linghu. Puhao Li, Yan Wang, Qing Li, Song- Chun Zhu, Baoxiong Jia, and Siyuan Huang. An embodied generalist agent 3d world. arXiv preprint arXiv:2311. 12871, 2023.
Song-Chun
Jiangyong Huang, Baoxiong Jia, Yan Wang. Ziyu Zhu, Xiongkun Linghu, Qing Li, Song-Chur Zhu, and Siyuan Huang. Unveiling the mist 3d dision-languag understanding: Object- centric evaluation with chain-of-and an-alssss.. Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 24570-24581, 2025a.
Kuan-Chil Huang, Xiangtai Li, Lu Qi, Shuicheng Yan, and Ming-Hsuan Yang. Reason3d: Search- ing and reasoning 3d segmentation via large language model. In International Conference on 3D Vision 2025, 2025b.
Mengdi Jia, Zekun Qi, Shaochen Zhang, Wenyao Zhang, Xinqiang Yu, Jiawei He, He Wang, and Li Yi. Omnispatial: Towards comprehensive spatial reasoning benchmark for vision language models. arXiv preprint arXiv:2506.03133, 2025.
vestigating their struggle with spatial reasoning. arXiv preprint arXiv: 19785, 2023. Amita Kamath, Jack Hessel, and Kai-Wei Chang. What's" up" with vision-language models? in-
Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Ziwei Liu, et al. Llava-onevision: Easy visual task transfer. arXiv preprint arXiv:2408. 03326, 2024.
Mingsheng Li, Xin Chen, Chi Zhang, Sijin Chen, Hongyuan Zhu, Fukun Yin, Gang Yu, and Tao Chen. M3dbench: Let's instruct large models with multi-modal 3d prompts. arXiv preprint arXiv:2312. 10763, 2023.
11

---

Published as a conference paper at ICLR 2026
Ji Lin, Hongxu Yin, Wei Ping, Pavlo Molchanov, Mohammad Shoeybi, and Song Han. Vila: On pre-training for visual language models. In Proceedings the IEEE/CVF conference on computer vision and pattern recognition, pp. (8689926699, 2024.
Xiongkun Linghu, Jiangyong Huang. Xuesong Niu, Xiaojian Shawn Ma, Baoxiong Jia, and Siyuan Huang. Multi-modal situated reasoning in 3d scenes. Advances in Neural Information Processing Systems, 00//4009040990 2024.
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. Advances in neural information processing systems, 36:34892-349 2023.
Yuan Liu, Haodong Duan, Yuanhan Zhang, Bo Li, Songyang Zhang, Wangbo Zhao, Yike Yuan, Jiaqi Wang, Conghui He, Ziwei Liu, al. Mmhench: Is your multi-modal model all-around player? In European conference on computer vision, pp. 216-233. Springer, 2024.
Gen Luo, Ganlin Yang, Ziyang Gong, Guanzhou Chen, Haonan Duan, Erfei Cui, Ronglei Tong, Zhi Hou, Tianyi Zhang, Zhe Chen, et al. Visual embodied brain: Let multimodal large language models see, think, and control in spaces. arXiv preprint 250.:22500.00223, 2025.
Ruiyuan Lyu, Jingli Lin, Tai Wang. Shuai Yang, Xiaohan Mao, Yilun Chen, Runsen Xu, Haifeng Huang, Chenming Zhu, Dahua Lin, al. Mmscan: multi-modal 3d scene dataset with hierar- chical grounded language annotations. Advances in Neural Information Processing Systems, 37: 50898-50999 2024.
Wufei Ma, Haoyu Chen, Guofeng Zhang, Celso M de Melo, Jieneng Chen, and Alan Yuille. 3dsr- bench: A comprehensive 3d spatial reasoning benchmark. arXiv preprint arXiv:2412.07825, 2024.
Wufei Ma. Luoxin Ye, Celso M de Melo, Alan Yuille, and Jieneng Chen. Spatialllm: A compound 3d-informed design towards spatially-intellgg large multimodal models. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp.
2025.
Xiaojian Ma. Silong Yong, Zilong Zheng, Qing Li, Yitao Liang, Song- Chun Zhu, and Siyuan Huang. Sqa3d: Situated question answering in 3d scenes. arXiv preprint anXin:2110.07474, 2022.
OpenAL.
Gpt-5 system
card.
https: https://cch.openaa.. /d
112443ce-ab78-45000 2025.
Zhangyang Qi, Zhixiong Zhang, Ye Fang, Jiaqi Wang, and Hengshuang Zhao. Gpt4scene: Un- derstand 3d scenes from videos with Lasgun-angguage models. arXiv preprint arXiv:2501.01428, 2025.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, al. Learning transferable visual models from natural language supervision. In International conference machine learning, pp. 8748-8763. PmLR, 2021.
Santhosh K Ramakrishnan, Aaron Gokaslan, Erik Wijmans, Oleksandr Maksymets, Alex Clegg, John Turner, Eric Undersander, Wojciech Galuba, Andrew Westbury, Angel X Chang, et al. Habittat 3d dataset (hm3d): 1000 large-scale 3d environments for embodied ai. arXiv
preprint arXiv:2 00900838. 2021.
Santhosh Kumar Ramakrishnan, Erik Wijmans, Philipp Krachenbuchl, and Vladlen Koltun. Does spatial cognition emerge in frontier models? arXiv preprint arXiv:2410.06-468 2024.
Adarsh Jagan Sathyamoorthy, Kasun Weerakoon, Mohamed Elnoor, Anuj Zore, Brian Ichter, Fei Xia, Jie Tan, Wenhao Yu, and Dinesh Manocha. Convoi: Context-aware navigation using vision language models in outdoor and indoor cuvironments. In 2024 IEEE/RSJ International Confer- ence on Intelligent Robots and Systems (IROS), pp- 1 13837-13844 IEEF, 2024.
Dhruv Shah, Ajay Sridhar. Nitish Dashora, Kyle Stachowiez, Kevin Black, Noriaki Hirose, and Sergey Levine. Vint: A foundation model for visual navigation. arXiv preprint arXiv:2306. 14846, 2023.
12

---

Published as a conference paper at ICLR 2026
Yihong Tang, Ao Qu Zhaokai Wang, Dingyi Zhuang. Zhaofeng Wu, Wei Ma, Shenhao Wang, Yunhan Zheng, Zhan Zhao, and Jinhua Zhao. Sparkle: Mastering basic spatial capabilities in vision language models elicits generalization to composite spatial reasoning. 2024.
Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, Katie Millican, et al. Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312. 1805, 2023.
Gemma Team, Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, Ramona Merhej, Sarah Perrin, Tatiana Matejovicova, Alexandre Ramé, Morgane Rivière, al. Gemma 3 technical report. arXiv preprint arXiv:2503 2025a.
V Team, Wenyi Hong, Wenmeng Yu, Xiaotao Gu, Guo Wang, Guobing Gan, Haomiao Tang, Jiale Cheng, Ji Qi. Junhui Ji, Lihang Pan, Shuaiqi Duan, Weihan Wang, Yan Wang. Yean Cheng, Zehai He, Zhe Su, Zhen Yang, Ziyang Pan, Aohan Zeng. Baoxu Wang, Bin Chen, Boyan Shi, Changyu Pang, Chenhui Zhang. Da Yin, Fan Yang, Guoqing Chen, Jiazheng Xu, Jiale Zhu, Jiali Chen, Jing Chen, Jinhao Chen, Jinghao Lin, Jinjiang Wang, Junjie Chen, Leqi Lei, Letian Gong, Leyi Pan, Mingdao Liu. Mingde Xu, Mingzhi Zhang, Qinkai Zheng, Sheng Yang, Shi Zhong, Shiyu Huang. Shuyuan Zhao, Siyan Xue, Shangqin Tu, Shengbiao Meng, Tianshu Zhang, Tianwei Luo, Tianxiang Hao, Tianyu Tong. Wenkai Li, Wei Jia, Xiao Liu, Xiaohan Zhang, Xin Lyu, Xinyue Fan, Xuancheng Huang, Yanling Wang. Yadong Xue, Yanfeng Wang. Yanzi Wang, Yifan An, Yifan Du Yiming Shi, Yiheng Huang, Yilin Niu, Yuan Wang. Yuanchang Yue, Yuchen Li, Yutao Zhang, Yuting Wang, Yu Wang, Yuxuan Zhang, Zhao Xue, Zhenyu Hou, Zhengxiao Du, Zihan Wang, Peng Zhang, Debing Liu, Bin Xu, Juanzi Li, Minlie Huang, Yuxiao Dong, and Jie Tang. Glm-4.5v and glm-4. 11-thiik-t Towards versatile multimodal reasoning with scalable reinforcement learning, 2025b. URL hRthttps::
01006
Peter Tong, Ellis Brown, Penghao Wu, Sanghyun Woo, Adithya Jairam Vedagiri IYER, Sai Charitha Akula, Shusheng Yang, Jihan Yang, Manoj Middepogu, Ziteng Wang, et al. Cambrian-1: A fully open, vision-centric exploration of multimodal llms. Advances Neural Information Processing Systems,
2024a.
Shengbang Tong. Zhuang Liu, Yuexiang Zhai, Yi Ma, Yann LeCun, and Saining Xie. Eyes wide shut? exploring the visual shortcomings of multimodal llms. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9568-9578, 2024b.
Johanna Wald, Armen Avetisyan, Nassir Navab, Federico Tombari, and Matthias NieBner. Rio: 3d object instance re-localization in changing indoor environments. In Proceedings of the IEEF/CVF International Conference on Computer Vision, pp. 7658-7667, 2019.
Weiyun Wang. Zhangwei Gao, Lixin Gu, Hengjun Pu, Long Cui, Xingguang Wei, Zhaoyang Liu, Linglin Jing, Shenglong Ye, Jie Shao, et al. Internv13. 5: Advancing open-source multimodal models in versatility, reasoning, and efficiency. arXiv preprint arXiv:2508.18265. 2025.
Benjamin Wilson, William Qi, Tammay Agarwal, John Lambert, Jagjeet Singh, Siddhesh Khan- delwal, Bowen Pan, Ratnesh Kumar, Andrew Hartnett, Jhony Kacsemodel Pontes, et al. Ar- goverse 2: Next generation datasets for self-driving perception and forecasting. arXiv preprint arXiv:2301.00499; 2023.
Yijia Xiao, Edward Sun, Tianyu Liu, and Wei Wang. Logievista: Multimodal Ilm logical reasoning benchmark in visual contexts. arXiv preprint AXi::2407.009733 2024.
L.LM-Core-Team Xiaomi. Mimo-vl technical report, 2025. URL. httpp:://axxv.org////t 0556600566
Weiye Xu, Jiahao Wang, Weiyun Wang, Zhe Chen, Wengang Zhou, Aijun Yang, Lewei Lu, Houqiang Li, Xiaohua Wang, Xizhou Zhu, et al. Visulogie: benchmark for evaluating visual reasoning in multi-modal large language models. arXiv preprint arXiv:2504.15279 2025.
Xu Yan, Zhihao Yuan, Yuhao Du, Yinghong Liao, Yao Guo, Shuguang Cui, and Zhen Li. Com- prehensive visual question answering on point clouds through compositional scene manipulation. IEEE Transactions Visualization and Computer Graphics, 30(12):44337488, 2023.
13

---

Published as a conference paper at ICLR 2026
Jianing Yang, Xuweiyi Chen, Nikhil Madaan, Madhavan Iyengar, Shengyi Qian, David F Fouhey, and Joyce Chai. 3d-grand: A million-scale dataset for 3d-llms with better grounding and less hallucination. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp- 29501-29512, 2025a.
Jihan Yang, Shusheng Yang. Anjali Gupta, Rilyn Han, Li Fei-Fei, and Saining Xie. Thinking in space: How multimodal large language models see, remember, and recall spaces. arXiv preprint arXiv:2412.1417/ 2024.
Sihan Yang. Runsen Xu, Yiman Xie, Sizhe Yang, Mo Li, Jingli Lin, Chenming Zhu, Xiaochen Chen, Haodong Duan, Xiangyu Yue, et al. Mmsi-bench: A benchmark for multi-image spatial intelligence. arXiv preprint arXi::2505.2376, 2025b.
Yuan Yao. Tianyu Yu, Ao Zhang, Chongyi Wang. Junbo Cui, Hongji Zhu, Tianchi Cai, Haoyu Li, Weilin Zhao, Zhihui He, et al. Minicpm-v: gpt-4v level milm on your phone. Nat Commun 16, 5509 (2025), 2025.
Shuquan Ye, Dongdong Chen, Songfang Han, and Jing Liao. 3d question answering. IEEE Trans- actions on Visualization and Computer Graphics, 30(3):172--1786, 2022.
Kaichen Zhang, Bo Li, Peiyuan Zhang, Fanyi Pu, Joshua Adrian Cahyono, Kairui Hu, Shuai Liu, Yuanhan Zhang, Jingkang Yang, Chunyuan Li, et al. Lmms-eval: Reality check on the evaluation of large multimodal models. arXiv preprint 2rXiv:2407.12772. 2024.
Lichen Zhao, Daigang Cai, Jing Zhang. Lu Sheng, Dong Xu, Rui Zheng, Yinjie Zhao, Lipeng Wang, and Xibo Fan. Toward explainable 3d grounded visual question answering: new benchmark and strong baseline. IEEE Transactions on Circuits and Systems for Video Technology, 33(6): 2935 2949, 2022.
Jinguo Zhu, Weiyun Wang, Zhe Chen, Zhaoyang Liu, Shenglong Ye, Lixin Gu, Yuchen Duan, Hao Tian, Weijie Su, Jie Shao, et al. Internv1:: Exploring advanced training and test-time recipes for open-source multimodal models. arXiv preprint arXiv:2504. 10479, 2025.
USE OF LARGE LANGUAGE MODELS (LLMs)
During the writing of this paper, we utilized LLM solely for language editing to improve clarity and readability. We critically reviewed and revised all Al-generated suggestions to ensure the final text accurately reflects our original intent. All intellectual contributions, including the research design, methodology, analysis, and conclusions, our exclusive work, and we take full responsibility for the academic integrity of this publication.
A OVERVIEW OF THE APPENDIX
This appendix provides an introduction of QA definition and examples, atomic capability definition, calculation of correlation between character length and human accuracy, calculation of capability score matrix, experiments on data generation pipeline, case study, annotate interface, ethic state- ment, and reproducibility statement. It is organized as follows:
In Sec. we detail the definition and example of each QA type. The examples of Basic QA and Compositional QA are also illustrated in this section. In Sec. C we introduce the definition of each atomic spatial capability and our design motivation. In Sec. [ we describe the detailed process of QA generation in Sec. annd demonstrate the examples of Basic QA and Compositional QA in Sec. D.2 The multiple-answer examples are also attached Sec. D.3 Sec, E we demonstrate the calculation method of the capability score with a simple example. In Sec. F we make ablation study on the effectiveness of the data generation pipeline. In Sec. we show the unweighted overall performance of Tab in the main paper. In Sec. HH show the case study of Basic QA quality in Sec.(H.] our curated annotation interface in Sec. H.2 and more QA cases in Sec. H.3 In Sec. we discuss more details in this paper. In Sec. I.6
announce the usage of L.LMs.
14

---

Published as a conference paper at ICLR 2026
Table 5: Mapping of QA types and Spatial Capabilities. Each type of QA is the integration of multiple capabilities.
Entily Quantilication (EQQ
Scone Quantification (5Q)
4
v
Size Assessment (SA)
4
4
v
00 Sparial Relationship (0O)
/
4
/
OS OS--Spatial Relationship (OS)
/
4
/
Eatily Presence (EP)
Functional Reasoning (FR)
Spatial Planning (SP)
B QA DEFINITION
In this section, we introduce the definition of each QA type and show the QA examples in Fig. Notably, each QA type is the integration multiple capabilities, and we also show the mapping between QA and capabilities in this figure and Tab. 5 The following definitions:
Entity Quantification (EQ): Counting the number of objects in scene.
Scene Quantification (SQ): Counting the number of regions within scene.
Size Assessment (SA): Comparing the size relationships between different objects.
Object-Object Spatial Relationship (0O): Understanding the relative spatial relationship between two objects.
Object-Scene Spatial Relationship (OS): Understanding the relative spatial relationship be- tween an object and the overall scene.
Functional Reasoning (FR): Reasoning objects that match or do not match certain functions based on relative spatial relationships.
Spatial Planning (SP): Global spatial navigation and path planning.
C CAPABILITY DEFINITION
In this section, we illustrate the definition of each capability:
CI-Object Recognition: Identify what the object is.
C2- C2-Spatial Location: Localizing an object in space.
C3 -paataia Relationship: Understanding relative spatial position relationship.
Size Comparison: Comparing the size relationship objects.
C5-Counting: Count the number of objects and scenes.
06-Function Knowledge: Understanding the function of objects.
C7-Multi-veew Fusion: Understanding spatial information from multiple views
28-Forward Thinking: Understand the forward instructions and complete tasks in the given space.
Ce-Rever Reasoning: Understand the reverse instructions and complete tasks the given space. C10-Situated Observation: Imagine standing in designated position in space and observing and understanding the scene.
In designing our capabilities, we draw on established research in spatial intelligence. Initially, beginning reference for our design is Sparkle (Tang et al.] 2024), which frames 2D spatial intelli- gence around three -bireties-Directioon. Distance, and Location. distance are split into C3 (Spatial Relationship). Since we believe that the concept of 'what and where' is one of the most fundamental abilities for real spatial intelligence, location is separated and mapped to the C2 (Spatial Location). Next, early studies (Kamath et al. 2023 Bagherinezhad al. 2016) mainly emphasize C4 (Size Comparison) and simple positional cues in clean images. Then, evaluation expands from single-scene 3D point-cloud QA (Azuma al. 2022 Ma et al. 2022
SpaCE-10, the direction and
15

---

Published as a conference paper at ICLR 2026
Entity Presence:
Sputial Plunning:
Which
options contain any
Q: Which the following descriptions are incorrec for keeping the room clean ridy? the
objects the given scene?
From entranee move to room conter, tuns bod, then tidy up the quilt.
A, lush green plass, browa I'V, blue bod, gray cabunet lamp, carpel, gray gray
From
right bed, then up the quili.
From
cabinet, clean up cahinet.
C. brown door, TV, gny window, blue cabinet
From
cobines, left smaller then close window.
D. hudh green planz, black TV, blue bed, black window
Capability: CI, C22 C3, CT, C10
Capabiliity C1,
Entity Quantification:
00 Spatial Relationship:
Q: many
given
When facing the
objects.
which descriptions conrect?
Capubility; C3,
plant
TByTh plans is on the right of the TV.
Scene Quantification:
The lamp
right the hed
objects with specific fanctions is cilleed a finetional rone (fe example, desk the workspace, clairs and ahies dining area). in given somns, sporiallyy different functional zones there? B.3 D.4 Capability: Cl. CT,
area
The mural on the right of door.
Capability: CI, C3, CT2
OS Spatial Relationship:
Q: When viewing from the entrance, which descciptions are cocnect?
Size Assessment:
morm.
Q: Which description correct?
the lett side roca..
A. The door is lower than the bed.
cabinet the object that
B. The door is wades than the TV.
C. The lamp taller than the bed.
right the Capability: CI2, C22, Ciy, C7 C10
D. The plant lower than
Capability CI, CI2 C4, C72
Function Reasoning:
Q: Which the following description comeet?
Among the ohjects adjacest to these objoctt be to store Remns lacks two objects that can cleas ventilation.
Among the objects adjacent the rhere object venillatin
chrck appesrance,
objects
provide
Among the objects adjacent the TV, there object used check appearance, but lacks objects that used provide vencilation and decocate the room.
Among the objects adjacent to the IN, there an object can used decorate the room, but lacks two objects that can used clean the floor and pooide vencilation.
Capshility: CI, C22 C3. CS, C7, CR2 CS C10
Figure 5: Examples all types of QA. The blue examples represent the perception QA, and the purple ones denote the reasoning QA. The green circles are the correct answer.
Ye et al. 2022; Zhao et al. 2022 Yan et al. 2023 Hong al., 2023 Yang et al. 2025a Lyu al.
2024 Huang et al. 2025a b) mu -uew 2D imagery (Ma et al. 2024 2025 Linghu et al. 2024
Yang et al.) 2024 2025b Jia al. 2025 Cheng et al. 2024). the focus shifts from single-vies view C3 (spatial localization) and C5 (Counting) within one scene to cross- view consistency captured by C7 (Multi-view Fusion). In parallel, SQA (Ma et al. 2022) introduces viewpoint conditioning, aligning with C10 (Situated Observation). Nowadays, with the rise of Embodied Intelligence and Reasoning MLLMs, based on these previous excellent works, further propose incorporate C6 (Function Knowledge), C8 (Forward Thinking), and C9 (Reverse Reasoning) into this atomic capability pool for manipulating basic knowledge and reasoning ability examination.
D DATA GENERATION DETAILS
D.1 GENERATION FOR EQ AND SP
For EQ, first extract the number of each object from the scan datasets' semantic labels, and then leverage a predefined template to generate QA. For the SP, we employed human experts to manually design 80 QA pairs. Each question presents complete task flow, including the navigation path, goal, goal characteristics, and actions to be performed, with potential errors in any step. Based on the prompt, the model must select either fully correct task flow containing incorrect steps.
16

---

Published as a conference paper at ICLR 2026
D.2 BASIC QA AND COMPOSITIONAL QA
In the main paper, mention that there 5 QA types (00, OS, SA, FR, SP) that will be applied
with cross-capability integration strategy. Thus, in this section, we demonstrate examples these two QAs.
(1) For 0O (Ohject-Object Spatial Relationship), OS (Object-Scene Spatial Relationship), SA (Size Assessment):
Basic-SA-Question:
When facing the front side of the objects, which description is correct?
Basic-SA-Options:
A. This red table is taller than the brown cabinet, but narrower than the brown cabinet.
B. This red table is shorter and narrower than the brown cabinet.
C. This red table is shorter, but wider than the brown cabinet.
D. This red table is taller than the brown cabinet and wider than brown cabinet.
Basic-SA-Anss Snnweerr:
In the Basic QA format, each question involves only two objects, and the scenario is restricted to a single viewpoint, ie., the front of the objects. For each scene, we generate multiple Basic QA questions, and then aggregate these options to form Compositional QA questions.
Cumposttional-SA-uuesttoor When facing the front side of the objects, which description is cor- rect?
Cimpositionll-SA-Optiiio
A. This red table is taller than the brown cabinet, but narrower than brown cabinet.
B. The blue sofa wider than the white door.
C. The green plant is taller and wider than the wooden bookshelf.
D. This red table is taller than the brown cabinet and wider than brown cabinet.
Composiiional-SA-Anssee B
In Compositional QA, we combine multiple Basic QA questions, so each question may now refer to different objects located at various positions across the scene. This forces the model to integrate information from multiple perspectives, requiring C7: Multi-view Fusion.
(2) For EP (Entity Presence), add additional C7 and C9:
Basic-nP-Question Does a red chair exist in the given scene?
Dasic-EP-Options 1:
A. Yes B. No
Basic-EP -Pnieer 1: A
Piiee-E--Quetionn Does a gray sofa exist in the given scene?
Sasic-EP-Options2 2:
A. Yes B. No
2: B
We generate multiple Basic-EP questions and then aggregate them into Compositional-EF quettttnnn
Cumpositional-EE--uuusttttto Which of the following options does not contain any objects in the given scene?
Compositional-EP-Otiinn
A. Red chair, blue sofa, green plant, red table
17

---

Published as a conference paper at ICLR 2026
B. Gray sofa, white lamp, orange carpet, pink cushion
C. Black coffee table, grey armchair, purple curtain, brown bookshelf
D. Beige ottoman, teal vase, silver lamp, golden picture frame
CnapositionallEP-Annsee
In Compositional-ER. each option contains multiple objects. Similar to Basic QA, where objects the scene may appear from different angles or positions, C7: Multi-view Fusion required. Additionally, the task shifts from direct "does this object exist?" question to more complex "which object is missing from the scene?" requiring C9: Reversed Reasoning.
(3) For FR (Functional Reasoning), we add additional C7, C9, and C1O:
Basic-FR- Fasic-FR-Question: Which option correctly describes the function of an object near the bed?
Basic-FR--Oe Dasic-FR-Options:
A. Cabinet, store items
B. Mop, clean the floor
C. Window, provide ventilation
D. Mop. provide light
Basie-FR-Answer: A
In Basic-FR, the correct answer must satisfy both functional and spatial positioning requirements. We generate multiple Basic-FR questions, which are then combined into Compositional ques- tions.
Crapositional-FR-uuesste Which of the following descriptions correct?
Cuapesitional-FR-Opttoon
A. Among the objects adjacent to the bed, there is an object that can be used store items, but lacks two objects that can be used to clean the floor and provide ventilation.
B. Among the objects adjacent the cabinet, there an object that can be used to check our appearance, but lacks two objects that can be used to provide ventilation and rest.
C. Among the objects adjacent the TV, there an object that be used to check our appearance, but lacks two objects that can be used to provide ventilation and decorate the room.
D. Among the objects adjacent to the TV, there is an object that can be used to decorate the room, but lacks two objects that can be used to clean the floor and provide ventilation.
Compositional- Fir.sittionn--FAAnweeerrrr A
In Compositional-FR, multiple central objects are combined, broadening the question to encompass the entire scene. The model must analyze the scene from various perspectives, necessitating C7: Multi-view Fusion to integrate spatial relationships and C10: Situated Observation to understand the contextual functionality of objects. Furthermore, task involves evaluating the relevance of functional descriptions, which requires C9: Reverse Reasoning to assess the appropriateness of the functions relation to the scene.
D.3 MULTI-ANSWER EXAMPLE
Each multiple-choie question is set five-option, two-correct answer question. where the evalu- ated model must select two correct answers to be considered correct. The accuracy for each type of QA is calculated in the same way single-choice questions. Compared single-choice questions, the question and option format for multiple-choice questions remains exactly the same. Below an example of the EP:
Single-Choice EP:
Question: Which of the following options does not contain any objects in the given scene?
18

---

Published as a conference paper at ICLR 2026
Options:
A. Red chair, blue sofa, green plant, red table
B. Gray sofa, white lamp, orange carpet, pink cushion
C. Black coffee table, grey armchair, purple curtain, brown bookshelf'
D. Beige ottoman, teal vase, silver lamp, golden picture frame
Answer:
Double-Choice EP:
Question: Which of the following options does not contain any objects the given scene?
Options:
A. Red chair, blue sofa, green plant, red table
B. Gray sofa, white lamp, orange carpet, pink cushion
C. Black coffee table, grey armchair, purple curtain, brown bookshelf
D. Beige ottoman, teal vase, silver lamp, golden picture frame
E. Gray sofa, grey armchair, purple curtain, brown bookshelf
Answer: B, E
E CAPABILITY SCORE CALCULAION
Capability-soon computation (for Fig. The values Fig. are computed from the per- question scores in Tab. 55 via the task task->capy mapping (CI-C10).
Let 2, denote the set of questions linked to capability C, and n, = 12 For a question q E Q,, let Score(g) be the score assigned by the task's 's evaluation rule (e.g.. 0/100 for single-answer exact match, or the task-specific percentage for multiple-anse answer). The capability score is the mean over its linked questions:
Scoro(C,)
Σ Score(q).
n4 GEQ:
no question maps to a capability (n, = 0), the entry is marked as N/A and excluded from any further averaging. When question maps to multiple capabilities, its score contributes to each linked capability (no reweighting).
Toy example. Consider capabilities C1 and C2, and questions Q1, Q2 with mappings Q1 + Cj and Q2 + {C1, C2). Suppose Score(Q:) = 80 and Score(Q2) = 90. Then Q (Q1,Q2) (n) 2) and Q2 [Q2) (n2 1), yielding
Score (Ci) 80+-00 85,
Score(C2) = 90.
This procedure produces the capability values shown in Fig.
F PIPELINE COMPONENT ABLATION
To systematically evaluate the contribution of each key component in the structured data generation pipeline to the accuracy of basic QA generation, we designed a stepwise ablation study. Specifically, selected five QA task types (SA, 00, OS, EP, FP), and for each type, randomly sampled 30 scenes, generating one question per scene. The accuracy of each generated question was manually verified by human experts.
The experimental results in Tab. Ffcearyy demonstrate the cumulative gains of each module. Using the 2D Captioner alone the baseline, the generated QA already showed relatively stable accuracy across most categories (an average of 62.0%). with particularly high accuracy in the EP task at 96.7%. This reflects the relatively low difficulty of generating questions for this category and that
19

---

Published as a conference paper at ICLR 2026
Table 6: Ablation study of structural data generation pipeline. We randomly sample 30 scenes for each type of QA and generate one QA for each scenc. The results demonstrate the effectiveness of each component.
Componeent
SA
00
EP
FP
Average
2D Captioner
50.0
50.0
46.7
96.7
66.7
62.0
Captioner
76.7 (+14.7)
Inspector
80.0
66.7
70.0
96.7
83.3
79.1 (+17.3)
Structural Dara
86.7
76.7
76.7
96.7
83.3
84.0
2D visual information sufficiently supports it. With the addition of the 3D Captioner, the overall aceuracy improved significantly (an average increase of 14.7%), indicating that 3D information effectively supplements the limitations of 2D vision and enhances the model's understanding of spatial and object attributes. Further incorporating the Inspector component led to another increase in accuracy, reaching 79.3% (a 17.3% improvement over the baseline), showing that this module plays an important role in validating and refining the details question generation. Finally, after adding structured data, the overall average accuracy reached 84.0%, 22% improvement compared to the initial baseline, fully demonstrating the critical value structured information in improving the quality of basic QA generation.
G UNWEIGHTED MAIN RESULTS
We demonstrate the unweighted single-answer performance ranking on SpaCE- in Tab.
H
CASE STUDY
H.1 BASIC QA QUALITY
In this paper, we have developed sophisticated pipeline for the automated generation of high- quality Basic QA (Basic Question Answering). In the previous section, we systematically verified the effectiveness of each component within the pipeline. This section presents two typical high- quality QA cases and delves into them with specific context-based analysis and discussion.
As shown in Fig. [6] the first case centers on the size estimation task (Basic SA). As depicted in the figure, through meticulously crafted prompts and high-quality snapshots, we steered GPT to accu- rately generate questions. When comparing the volume relationship between white wooden baby erib and wardrobe, GPT delivered impressive performance. It not only correctly identified the height and width dimensions of each object but also precisely determined their relative differences across multiple perspectives. For instance, its response accurately pointed out that the wardrobe is taller than the crib but has similar width, demonstrating a good grasp of the 3D geometric proper- ties of objects.
The second case focuses on the spatial relationship understanding task (Basic 00), as shown in the Fig. in the context of judging the spatial relationship between yellow chair and an entrance. In this example, GPT not only accurately distinguished the relative direction of "left" but also cor- rectly identified the spatial distance difference between "near" and "far". Although the question only involved one yellow chair, the distractors in the options were somewhat deceptive, which in turn enhanced the question's 's ability to assess the model's spatial understanding. This case also directly confirms the feasibility and rationality of using GPT for generating spatial intelligence QA questions. Through these two cases, we observed that powerful MLLMs, when only provided with high-quality 2D visual inputs, are already capable of understanding certain 3D spatial information, such as object size, volume relationships, and spatial orientation. This ability suggests promis- ing path for future exploration: unlike current 3D models that sacrifice conversational abilities to fit point cloud data, 2D MLLMs can still demonstrate strong spatial understanding potential even without explicitly incorporating 3D structural modeling.
20

---

Published as conference paper at ICLR 2026
Table Unweighted overall performance ranking on SpaCE-10.
Perception
Reasoning
Models
Rank
Overall
EQ
SQ
SA
00
OS
EP
FR
SP
Human
91.3
88.5
90.2
93.4
95.6
91.1
90.3
86.3
90.8
3D MILMS
LEO-7B Huang et 2025
46
15.8
0.0
16.7
16.5
25.2
5.5
5.7
13.3
12.3
GP 78 2025
31
30.9
37.7
38.0
38.9
41.6
29,5
28.0
32.5
34.6
Close Source 2D MLLMS
GPT:5 (OrenAl 2025
5
42.0
43.0
69.7
60.7
36.5
50.3
66.8
30.0
49.9
GPT-4o
2023
9
58.3
32.8
56.2
58.3
56.2
41.6
52.2
23.7
47.4
Gemini-2:0
34.3
53.1
31.2
41.2
Claude 3.7 Sonnet
14
46.0
44.3
49.1
46.0
49.1
44.3
49.3
25.0
44.1
Open Souree 28 MI.1Ms
Scude
IntemVL.2.5111 18
Chen et 2014
33
33.0
54.1
18.8
43.6
29.9
26.7
41.0
23.7
33.9
VL.
Inter VL.35-111111
Wane ara 2055
34
34.8
41.7
29.4
42.7
25.9
21.9
40.2
33.8
33.8
Intem VL
Laca 2021
41
32.2
26.8
27.0
36.6
28.8
21.7
48.2
36.2
32.2
VL..2B
IntemVL11.542
Wang et --L----V-B---siruu
2025
35
35.6
28.4
42.2
45.7
32.3
20.1
45.8
20.0
33.8
Quen2.5-V1-3BB
37
31.7
23.3
47.1
$1.7
25.5
21.2
33.6
SpaceOm
39
21.8
24.5
47.3
49.7
32.7
21.9
36.7
25.0
32.5
SpaceQwen
32
SpaceThinker
36
32.7
22.4
46.7
50.5
33.4
22.4
36.9
24.2
33.6
Lin
IntemVLL2 1B
Lase et
2024
28
34,3
23.4
50.2
50.8
16.2
21.7
$6.0
33.7
35.8
MiniCPM.y44-4B
26
38.1
32.7
41.1
36.5
29.3
30.0
38.3
IntemVLL....4B
Wang et
2125
29
38.9
12.9
48.7
50.7
27.9
33.9
37.0
35.0
35.6
SSalle 144
Qwen2.5-VL-7B--mm
ct
2025
I.1aVA-v1.5-78
[Iin 2a14
42
31.2
31.3
30.5
35.7
22.9
10.7
57.4
32.5
31.5
LLaVA-OucVisini:
(are 2024
18
37.4
33.8
16.4
57.3
34.5
43.3
61.6
21.2
41,9
MiMo-VE-RI.
30
23.7
46.4
34.7
32.2
36.1
35.2
Cambrian-88
Tonk cla A12418
43
22.6
18.6
34.8
32.6
32.3
25.1
41.4
23.7
23.9
VILAI.5-8B
45
25.7
27.5
32.7
17.2
12.4
23.7
21.8
IntemVL12.5
Casa 2021
20
33.2
36.0
50.0
$5 0
33.6
27.1
59.1
32.5
40.8
Inter VLB18
24
36.5
29.5
42.9
51.7
34.5
26.6
60.6
37.5
40.9
Wang
17.6
Gemma3-128
[eam
21
41.8
41.2
$5.1
46.5
35.6
25.0
53.2
27.5
40.7
VntenVLL3-1455
ATA
12
39.7
28.7
54.4
58.1
38.1
51.3
56.6
35.0
45.2
(Wing
41.0
46.3
Y 148 148-cS
728
InternV1.3.5 20.
2025
37.4
64.1
28.8
48,2
IntemVL2 5-266-666888
(Ch 2033
17
34,3
29.3
62.6
55 4
33.0
29.2
61.8
33.7
42.4
Gemma3-27B Telm R1M600
22
39.4
21.7
63.5
48.5
37.8
33.2
51.5
30.0
40.7
Qwen2 32n--ru-m
InterVL1..5 5.388.
(Chen 2024
13
38.1
36.1
64.4
54.3
36.8
27.4
63.0
37.5
44.7
Intern VLB3-3BB Ziu
A145
36.3
41.6
69.5
36.3
58.6
60.8
35.0
19.8
MMtmmVL.3..3355 Waag et
W Seale 726
GL.M-45VV (Team
2025h
8
38.9
65.5
61.1
36.7
61.2
49.3
31.3
48.1
L.aVA-OeeVimmin 725 L1r
2024
44.1
67.9
40.3
46.7
67.3
36.2
50.7
(Bti) dilied 2025
27
32.4
34.9
$5.7
40.9
32.1
36.5
38.0
33.7
38.0
IntenVLL.2
IntemVL3-77 38B
Zhu et
A14
3
36.8
48.2
65.3
61.6
43.8
44.4
64.3
46.3
51.3
Intem 5-2 18-
2025
2
35.8
39.1
68.2
63.5
46.2
64.2
58.6
40.0
52.0
Models proposed by RemyxAl (Space
H.2 ANNOTATION INTERFACE
To facilitate efficient annotation, we developed custom annotation tool, with the interface shown in Fig. During the annotation process, human experts are restricted viewing only 3D snapshots to judge the correctness of each QA pair; they are not allowed to view the 2D images. This design offers two main advantages: (1) It ensures high annotation speed. By examining only a small number of 3D snapshots, annotators can quickly grasp the overall layout of the scene while significantly reducing their visual workload. (2) Since 2D images contain overly fine-grained details, many of which may not be present in the 3D scene, using only 3D information to filter out incorrect questions helps ensure that the resulting QA pairs are suitable for evaluation across both 2D and 3D models. Additionally, during evaluation, to-end process not only prioritizes annotation efficiency but also reflects our rigorous commitment to data quality control.
tag erroneous questions to ensure none overlooked. This end-
21

---

Published as a conference paper at ICLR 2026
Question: Which of the following statements accurately compares the relative sizes of white wooden baby crib and wardrobe?
wwardrabe
A. The wardrobe is taller than the white wooden baby crib, but they are similar in width.
H-baby crbb
B. The white wooden baby crib is both taller and wider than the wardrobe.
W-boby
C. The wardrobe and the white wooden baby crib the same height, but differ slightly in width.
W-wtardrbbe
D. The white wooden baby crib significantly larger than the wardrobe in both height and width.
Figure 6: Showease of Basic SA QA. 'H' and "W" mean the height and width of objects, respec- tively. In this case, GPT-4o shows the precise understanding of both size and geometry.
Which of the following options best describes the spatial relationship between the yellow chair and the entrance in the provided 3D scene image?
Question:
A. The yellow chair closest to the entrance, positioned on the left side.
Entrance
B. The yellow chair is farthest from the entrance, positioned on the right side.
Entrance
C. The yellow chair is arranged at the back of the room in front of the whiteboard.
D. The yellow chair is placed diagonally opposite the entrance, near the black chair.
Figure 7: Showcase of Basic 00. This case reflects the capability in spatial relationships and QA design. Notably, all spatial relationship is based on the observation the entrance.
H.3 VISUALIZATION OF QA
We demonstrate more QA cases in this section. Specifically, figures of EP are shown in Fig.
and TO figures of 00 shown in Fig. TT and 12 figures of OS shown in Fig. [3] and 14
figures of SA are shown in Fig. [5 and 16, figures of FR are shown in Fig. I7 and 18 figures of SP
shown Fig. 19 and 20 figures of EQ and SQ are shown in Fig. 21 and 22
DISCUSSION
I.1 ERROR PATTERN ANALYSIS BASED ON VISUALIZATIONS
(1) Analysis in EP (Entity Presence): In the Entity Presence (EP) task, the dominant failure mode clearly aligns with C7 (Multi-view Fusion). This task requires the model to accurately determine which objects are absent from the given of images. As shown in Fig. 9 GPT-5 selects option B, but option explicitly mentions "a set of black-and-we structure," which is indeed present in the scene.
bookshelves with a clean and organized
22

---

Published as a conference paper at ICLR 2026
30 Stapence
Which objects dose not cxist in the given scene?
Cholces:
Say curved desk, large coffee R3y
Answar B
Correctness
Yes
Mid
Scane: 154c3110d9 (7/100).
Annotation State: No
Previous Scone
Next QA
Save
Figure 8: Interface of annotation tools.
QA Type: Entity Presence
Question: Which the following options does contain any objects in given scene?
Options:
Blue ceramic vase with intricate designs, green rug with forrl patterns, red rug with geometric shapes under dining table and a white coflee table with minimalist list design.
of blackandwhite bookshelves with clean and organized matte finish table with four chairs, black toilet paper holder with
glossy painted
C: Blue ceramic coffee mug, gray bed, queensized bed with black metal frame and blue soap dish with intricate detailing.
D: Black office chair, modern pendant chandelicr with dangling light fixtures that provide illumination, round table with glass top and smooth plastic finish.
GPT-S Answer:
GT: C
Figure 9: Visualization of Entity Presence
23

---

Published as a conference paper at ICLR 2026
QA Type: Enity Presence
Question: Which of the Kollowing options does not contain any abjects in the given scene?
Options:
small square window, rectangular oringe lamp, red dining a large white bed with a soft platform base, neally urranged with white bodding and pillows.
B: Black door with metallic handle, black bookshelf with multiple compartments, black sofa with white cushions and metal dining table.
Modemn black tall roctangular mirror and metallie standing lamp.
with sliding
white
X
D: Smooth white walls contributing to the minimalist design style, black metal coffee table, black desk and a set of tall white cabinets with drawers.
GPT-5
GT: B
Figure 10: Visualization of Entity Presence
QA Type: Object-Ohject Sparial Relationship (0O)
When facing front side the objects, which deseriptions correctf
Options:
A: The Towel is folded neatly on the floor next the Bathtub.
x
B: The Mirror is placed to the right of the Sink the wall.
The Radiator positioned left the Sink along wall.
D: The Towel is placed inside the Bathtub.
Answer:
For A and D, towel on the wall; For D, mirroe is above sink.
Figure 11: Visualization of Object-Object Spatial Relationship
24

---

Published as a conference paper at ICLR 2026
QA Type: Objjett-Ohjec Spatial Relationship (OO)
When Eacing the front side of the objects, which descriptions correct!
Question:
Options:
A: The Recessed Shelf's is mounted near the front of the Toilet.
B: The Recossed Shelf is located behind the Toilet.
The placed behind Bathtub the side room.
D: The Toilet is positioned to the right of the Sink.
GPT-5 Answer:
GT:
For A and B3 recessed shelf is located right behind the Toilet; For C, sink placod front the bathtub right side the room. For toilet is positioned the left the Sink.
Figure 12: Visualization of Object-Objerk Spatial Relationship
QA Type: Object-Scene Spatial Relationship (OS)
Question: When viewing from the cutranee, which descriptions is conrect?
Options:
and green waste bins large whiteboard spanning wall,
below it on the
Black and green waste hins situated to the left located the center.
C: Floortoceilngg windows with vertical gray blinds, with white radiatur extending below is located in the center.
A large whiteboard spanning the wall, with black and green waste bins below located the
GPT-5 Answer: C
GT: A
description contruditoor;y C, window the center. For D. these objects are located at the center.
Figure 13: Visualization of Object-Scene Spatial Relationship
25

---

Published as a conference paper at ICLR 2026
QA Type: Object- Scene Sputial Relationship
Question: viewing
which descriptions concett
Options:
A chuir draped with black security vest labeled 'SECURITY" is located in the conter.
B: A living area with an I shaped gray sofa facing flarsereen TV atop orangetonsed entertainment consolé is the left.
A round dining table surrounded by chairs with patterned upholstery is right.
A decorated green Christmas tree near glass sliding doors farthest object.
GPT-5 Answer: C
GT:
For B, the description self conirudctorr For C, window is not located at these objects are located
Figure 14: Visualization of Object-Scene Spatial Relationship
QA Type: Assessment (SA)
Question: When viewing from cutrance, which descriptions conrect?
Options:
The chair under the desk is significantly larger in size compared the green plant the cabinet..
The black vase significuntly taller than the lamp the hed's headboard.
is slightly smaller the green plant
cabinct.
D: The plant on the cabinet and the chair under the desk are approximately the same size in height.
GPT-5 Answer: D
GT: A
For B, the height of vase and lamp is similar.
Figure 15: Visualization of Size Assessment
26

---

Published as a conference paper at ICLR 2026
QA Type: Assessment (SA)
Question:
When viewing from the entrance, which descriptions correct?
Options:
The Red Wooden Cabinet is taller
than the Beige Sofa.
The Beige Sofa the Red Wooden Cabinct have approximatcly the width, the Wooden Cabinet slightly
The Coffee Table and the Beige Rug are approximately the same size, covering similar surface arcas.
The Beige Sofa is taller and wider than the Red Wooden Cabinet.
GPT-5 Answer: A
GT: A
image observe cabinct taller whose height and width are similar to the beige sofa.
Figure 16: Visualization of Size Assessment
Love
Type: Functional Reasoning (FR)
Question:
viewing from the entrance, descriptions correct?
Options:
A: Among the following objects adjacent to the window, there an object can be used to clean hands,, but lacks two objects that can be used to on und to sleep.
R: Among the following objects adjacent to the ceiling fann, there object used be used display information and sloop.
for bothing, but lacks objects that
C: Among the following objects adjacent the bookshelf, there object objects
provide light,
Among following objocts adjaceet the chair, there an object can be used to clean clothes, but lacks two objects that can be used hold blank sheets for printing and to sleep.
GPT-S Answer:
GT: C
For curtain shows that windowidon door shelf, und provide light. The bod not adjacent the shelf, thus there is no object can used to sit.
Figure 17: Visualization of Functional Reasoning
27

---

Published as a conference paper at ICLR 2026
QA Type: Functional Reasoning (FR)
Question:
When viewing from the which descriptions is correc??
Options:
At Among the following objects adjacent to the shower rod, there is an object can be used to prevent water from splashing, but lacks two objects that can be used to rest and dry your body.
x B: Among the following objects adjacent to the wooden shelf, there abject can
light, lacks objects used to provide ventilation and to sit.
provide
x
C: Among the following objects adjucent to the bath mai, there is an object can be used hang clothes, but lacks two objects that can be used to clean teeth and to sit.
Among the following objects adjucent to can be used store clothes. but lacks two objects that can be used to store books and to cook food.
there object
GPT-5 Answer:
GT A
X
Figure 18: Visualization of Functional Reasoning
QA Type:
Spatial Planning
Question: Which the following descriptions are incomeet keeping clean and tidy?
Options:
A: Starting from the entrance, first move front to red shelf, then move right to table, then arrange the chair next the table properly.
x
B: Starting from the entrance, move front to window, then wipe the rudiator under the window clean.
Starting from the shelf.
front to red shelf, then tidy the clutter
Starting from the entrance, move right front black large bed, then clean the clutter on the bed.
GPT-5 Answer: A
GT: D
For D. the bed is abviously not black. Other options
Figure 19: Visualization Spatial Planning
28

---

Published as a conference paper at ICLR 2026
QA Type: Spatial Planning (SP)
Question: Which of the following deseriptions are incorrect for keeping the room clean and lidy?
Options:
Startng from the entrunce, move left front to area, then mop this area with mog.
Sturting from the entrance, first move left front to open then right front
wipe the
clean.
entrance, move right narrow hallway, move front blackboard, then clean white chalk marks off the blackboard.
Starting
Starting from the entrunce, move front to table, then arrange the chair next to the black glass moder table neatly.
GPT-5 Answer: D
GT: D
Imagel infer AB, Combine other images and image3 infer option
Figure 20: Visualization Spatial Planning
QA Type: Eatity Quantifictiion (EQ)
Question:
How many sofas are in the given scene?
Options:
B: 2
D:
GPT-S Answer: B
GT: B
QA Type: Scene Quamtificatin (SQ)
Question: area indicated by objects with specific functions is callod functional zone (for example, a desk in the workspace, chairs and tables the dining area). In a given scene, bow many spatially different functional are there?
Options:
B:
C:
D:
GPT-S Answer: B
GT:
There only one area (living room) for rest.
Figure 21: Visualization of Entity Quantification and Scene Quantification
29

---

Published as a conference paper at ICLR 2026
Entily Quantification (BQ)
Question: How many whiteboards given scene!?
Options:
A: 2
B:
D:
GPT-5 Auswer:
GT: A
X
QA Type: Scone Quantification (SQ)
Question: area indicated by objects with specific flunctions is callod functional zone (for example, a desk the workspace, choirs and tables
the dining area). In given scene, how many spatially different functional
Options:
B:
C:
D:
GPT-S Answer: A
GT: A
There only one area for working.
Figure 22: Visualization of Entity Quantification and Scene Quantification
30

---

Published as a conference paper at ICLR 2026
The critical evidence comes from View 3, where books placed on the shelves unambiguously indi- cate that these shelves should be recognized as bookshelves. GPT-5 most likely failed to integrate this particular viewpoint, resulting in an incorrect judgment. This illustrates that insufficient cross- view fusion, rather than object recognition, is the primary cause of error in this EP example.
(2) Analysis in 00 (Object-Object Spatial Relationship): In the Fig. 11 the model shows that it can understand spatial relations to some extent, but the main errors come from C7 (Multi-view Fusion) and C10 (Situated Observation).
To determine that option D is incorrect, the model needs to combine information from View 3 and View 4. These two viewpoints together indicate that the toilet and the sink are positioned directly next to each other. Once we mentally imagine the scene from the "facing the front side of the objects" perspective, it becomes clear that the toilet located on the left side of the sink rather than the right.
This error reflects a failure not object recognition but in the integration of cross- rress-view information and the reasoning about left- left-right relationships under viewpoint changes, which correspond directly to C7 and C10.
(3) Analysis in OS (Object-Scene Spatial Relationship): For the OS task, the first example in Fig. clearly shows that the model fails on C2 (Spatial Localization) together with C7 and C10. The model selects option C. instead of A, which reflects several issues. First, GPT-5 exhibits a significant global spatial mislocalization because it incorrectly places the floor-to-ceiling windows at the center of the room rather than at the far end, which directly corresponds to error in C2. Second, to correctly choose option A, the model only needs to use the viewpoint in the last image to identify the position of the entrance. Once the entrance is recognized, the overall room layout becomes straightforward to infer, which falls under C1O. These mistakes are further compounded by the model's inability to fuse complementary views across the scene, which is typical C7 failure.
In the Fig. 14 the model again makes mistakes tied to C2, C7, and C10. Options A and C describe spatial placements that are mutually inconsistent. If the model had reasoned based on the global room layout, it would have realized that although the round dining table and chairs appear on the right in View 5, they are actually located near the center when considering the full set of views. The incorrect answer indicates that the model over-relies on isolated viewpoints, fails to integrate multi-view evidence, and does not perform viewpoint-conee conditioned reasoning about the true center of the room.
the error comes from a straightforward mistake in C4 (Size Comparison). The model selects option D instead of A, even though the chair under the desk is clearly much larger than the small green plant on the cabinet. This indicates a failure to perform basic relative size estimation. However, such mistakes are relatively rare. As shown in the main experimental results, C4 is one of the most stable and reliable capabilities across models, and GPT-5 generally performs well on size comparison tasks.
(4) Analysis in SA (Size Assessment): In Fig. 15
(5) Analysis in FR (Functional Reasoning): In Fig. 18 GPT-5 shows a clear deficiency in C6 (Function Knowledge). Option B is purposely designed to contain hidden contradiction. Both "providing natural light" and "providing ventilation" are typical functions of doors or windows. If such an object can provide natural light, it almost always can provide ventilation well, which means the two functions cannot be separated in the way option B describes. GPT-5 does not detect this implicit inconsistency and incorrectly selects B.
Option A is also intentionally challenging. The idea of "resting and drying your body" is misleading because, in View 6, although one can see a towel-like object, is not actually adjacent to the bathtub or shower area. Since the question requires adjacency, the towel does not satisfy the functional description. Correctly identifying this requires fine-grained spatial reasoning involving the relative positions of objects, which depends on C3 (Spatial Relationship).
(6) Analysis in SP (Spatial Planning): In Fig. 19 we select particularly illustrative case, since the question asks which option is incorrect. Options A and D describe almost identical navigation paths, and both include actions that are fully plausible the given environment. However, option D contains very obvious factual error: it refers to a "black large bed," while the bed shown in the scene is clearly not black. Despite being able to follow the spatial path correctly, GPT-5 still fails
31

---

Published as a conference paper at ICLR 2026
the question because it does not detect this basic visual inconsistency. This reflects clear failure in
C1 (Object Recognition).
Such error demonstrates that GPT-5 struggles to maintain stable perception and reasoning in cluttered or multi-step environments. Even when the spatial planning sequence is correct, a failure at the most fundamental visual capability prevents the model from arriving at the right answer. This example highlights that C1 remains essential, since higher-level planning depends directly on accurate object recognition.
(7) Analysis in EQ (Entity Quantification) and SQ (Scene Quantification): In Fig. 21 and 22 observe two distinct sources of error. In the EQ questions, the model's mistakes primarily stem from the limitations of C5 (Counting). Even with multiple viewpoints available, GPT-5 fails to consistently distinguish between repeated appearances of the same object and genuinely different instances. This reflect the broader challenge we observe throughout the benchmark, where counting remains one of the weakest atomic capabilities across models.
In contrast, the errors in SQ come from an insufficient understanding of C6 (Function Knowledge). Correctly determining the number functional zones requires understanding not only what objects are present but also how they define and structure the activities within the scene. GPT-5 struggles to identify which objects are truly function-defining and which ones serve secondary roles. As a result, it misclassifies the number of functional areas even when the visual evidence is sufficient.
Across these visualizations, we examine how all ten atomic capabilities from Cl to C10 contribute to failures across the eight QA types. Each example reveals different weakness, ranging from basic perception issues such as object recognition and counting to more complex challenges such as multi- view fusion, functional understanding, and situated spatial reasoning. These errors collectively show that current MLLMs still lack robust and reliable foundations for compositional spatial intelligence. The variety and consistency of these failure modes strongly underscore the necessity of SpaCE-10, which provides structured and seabbbltty-aligne benchmark to expose these limitations and guide future progress in spatial reasoning for MLLMs.
1.2 MORE DETAILED STATISTICS OF SPACE-10
SpaCE-10 contains 5,122 QA pairs in total (4,152 single-answer and 970 multiple-answer), covering 8i1 distinct indoor scenes with 6,488 multi-view images. The eight QA types contain the following number of samples: EQ: 388, SQ: 645, SA: 785, O0: 809, OS: 654, EP: 1,013, FR: 800, SP: 80.
Using the capability-task mapping defined in Table 5, we compute the precise distribution of sam- ples across the ten atomic spatial capabilities: C1: 5,122, C2: 4,477, C3: 2,343, C4: 785, C5: 1,033, C6: 2,230, C7: 5,122, C8: 5,122, C9: 1,893, C10: 2,343.
1.3
WHY 3D LLMS UNDERPERFORM 2D MLLMS?
Although 3D representations contain richer geometry, current 3D LLMs are trained on far fewer roughly QwenVL (Bai et al.)
lanality- pair data compared with 2D MLLMs. LEO (Huang et al. 2023)
28.3M 3D-text tokens, while modern 2D MLLMs (Intern (Zhu et al. 2025)
2025)) are trained over 10B 2T image-text tokens, which are about 10 to 10a times larger than 3D LLMs.
This large data gap means that 3D LLMs lack the world knowledge, reasoning priors, instruction- following patterns, and compositional skills that 2D models acquire from massive corpora. SpaCE- 10 makes this discrepancy particularly visible.
1.4 WHY WE USE 1024 POINTS FOR LEO?
LEO's tokenizer and transformer encoder trained strictly on 1024-point inputs. Increasing the resolution requires re- ve-training the entire model and is not supported by the released architecture. We therefore follow the model's official configuration ensure fair and correct evaluation.
32

---

Published as a conference paper at ICLR 2026
1.5 FUTURE EXTENTION TO OURDOOR SCENARIOS
Outdoor spatial intelligence is an essential piece of real-world applications. We also plan to ex- tend this capability-driver framework outdoor settings in our following iterations. For example. we conduct a survey and initially find some potential datasets that can support the next iteration of SpaCE-10 10. For outdoor spatial reasoning, autonomouu-driivng datasets such as nuScenes (Cae- sar et al. 2020) and Argoverse 2.0 (Wilson et al., 2023) offer large-scale multi-sensor observations (e.g., LiDAR, multi-view RGB, HD maps) that provide rich data sources for extending capabilities such as multi-view fusion, spatial localization, and forward reasoning to both 2D and 3D evaluation. For embodied outdoor navigation, models and datasets such as ViNT (Shah et al., 2023) and CoN- VOI (Sathyamoorthy et al. 2024) demonstrate the feasibility of evaluating piewpoint-dependent reasoning and long-horizen planning beyond indoor spaces. These resources can provide feasible paths for expanding SpaCE- 10 beyond indoor scenes while preserving its capability- design, and they will guide our future iterations toward a more comprehensive evaluation of spatial intelli- gence across both indoor and outdoor domains.
1.6 THE USE OF LARGE LANGUAGE MODELS
During the writing of this manuscript, LLMs are used to generate data, judge the evaluation, and slightly polish the wording and sentences, not to design the idea or write key sections. The authors take full responsibility for all content in the manuscript, including verifying the accuracy of LLM-assisted outputs and ensuring no plagiarism or scientific misconduct.
33


Reviews:
[{"review_id": "ukaDgfWf9C", "reviewer": "Reviewer_ffEP", "summary": "This paper introduces SpaCE-10, a large-scale benchmark designed to evaluate the compositional spatial intelligence (CSI) of multimodal large language models (MLLMs). The authors argue that existing benchmarks fail to disentangle and systematically assess spatial capabilities in MLLMs. SpaCE-10 defines 10 atomic spatial capabilities (e.g., object recognition, counting, spatial relationship, multi-view fusion) and combines them into 8 compositional QA tasks covering both perception and reasoning.\nThe benchmark is built upon 811 real indoor 3D scenes collected from four public datasets, producing over 5,000 high-quality question-answer pairs through a hierarchical semi-automated annotation pipeline involving both GPT-4o and human experts.\nThe authors evaluate ~50 open- and closed-source MLLMs, including GPT-5, LLaVA-OneVision, Qwen2.5-VL, and InternVL3.5. Results show that even the best models perform far below human level (≈53% vs. 91%), with major deficiencies in counting, reverse reasoning, and multi-capability composition. The authors release the dataset and code to promote further research in spatial reasoning for MLLMs.", "questions": "- How does SpaCE-10 ensure that the compositional QA pairs do not contain linguistic or visual shortcuts that allow models to guess without full reasoning?\n- Could the benchmark be extended to temporal or embodied spatial reasoning, e.g., predicting next-step navigation or multi-step manipulation tasks?\n- Given that GPT-4o is used in both generation and evaluation, how is data leakage or model bias avoided?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper convincingly identifies the lack of a unified benchmark for compositional spatial intelligence, distinguishing SpaCE-10 from prior 2D/3D QA datasets.\n- The hierarchical annotation pipeline (combining automated generation and human validation) is well thought-out, ensuring both scalability and quality.\n- By defining 10 atomic spatial capabilities and mapping them to 8 compositional QA types, the benchmark enables fine-grained capability diagnosis rather than raw accuracy comparison.\n- The inclusion of nearly 50 MLLMs, spanning 2D and 3D models from 1B to 241B parameters, provides an impressive empirical scope and meaningful comparative insights.", "weaknesses": "- Although human validation is used, the quality and potential bias of GPT-generated QAs could influence benchmark reliability; the paper could analyze this more rigorously.\n- SpaCE-10 focuses on indoor 3D environments; its applicability to outdoor or dynamic (temporal) spatial reasoning remains unexplored.\n- While overall accuracy drops are discussed, the paper could provide deeper qualitative examples showing failure modes and reasoning errors.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "fMVSqEkPpx", "reviewer": "Reviewer_sqZm", "summary": "This paper propose SpaCE-10 to address the limitation of existing benchmark in comprehensively assessing the ability of MLLMs to integrate multiple basic (the atomic level) spatial reasoning skills into complex, compositional tasks (the compositional level). The proposed benchmark defines a structured hierarchy, including 10 atomic spatial capabilities which are combined into 8 compositional capabilities. The authors design a hierarchical annotation pipeline to generate QA pairs, together with assistance from human experts.\nExperiments across models from 1B to 200+B provide in-depth analysis, with special focus on single abilities compared with their combination. These results reveal significant shortage of MLLMs in integrated spatial intelligence.", "questions": "1. Please provide a detailed breakdown of the SpaCE-10 benchmark. Specifically, what is the total number of QA pairs, the number of distinct images used, and the precise distribution of samples across the 10 atomic and 8 compositional capabilities?\n2. Please to the points in Weaknesses section.\n3. This is not a question, just to clarify that I want to assign a rate of 9 (but only allowed to choose between 8 and 10). The reason for not assigning the highest score is stated in Weaknesses#1. I suppose this work could be further strengthened if covering more practical scenes. Overall this is a good work.", "limitations": "", "rating": 8, "confidence": 5, "soundness": 3, "presentation": 4, "contribution": 4, "strengths": "- Significance: \n  1. The addressed compositional spatial reasoning ability is indeed important, challenging and useful in real world tasks.\n  2. The identified performance gap between MLLMs and humans on SpaCE-10 highlights the immediate and practical value of this benchmark in directing future model development.  \n\n- Originality: The paper offers an original and highly structured framework that 1. clearly defines 10 atomic level and 8 compositional level spatial capabilities, and 2. provides novel insights for researchers to precisely diagnose model deficiencies in 2D and 3D spatial reasoning.\n\n- Quality: \n  1. The conducted extensive evaluation of more than 50 MLLMs with diverse scales and structures validates the benchmark's utility. These results reveal common trends other than determining shortages for specific models. \n  2. This work includes both data generation, quality verification, in-depth analysis and case studies, almost covering aspect required by a benchmark.\n\n- Clarity: \n  1. This paper is well motivated and demonstrated. \n  2. The clear organization of spatial reasoning into atomic and compositional levels provides an intuitive and logical structure for evaluating intelligence in space.", "weaknesses": "1. This paper only covers indoor scenes, especially the housing scenes. While there are lots of different scenes worth investigating, such as indoor inductrial scene (in a factory), and also outdoor scenes. These could further expand the coverage and generality of the scope of this benchmark.\n2. Could also provide detialed insights for future research (further discuss the cause and potential improvements for your findings), or indicate example practical tasks for potential applications.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "kzCIcKMo2Y", "reviewer": "Reviewer_D8Pf", "summary": "The paper presents SpaCE-10, a comprehensive benchmark for evaluating compositional spatial intelligence in multimodal large language models. It defines 10 atomic spatial capabilities and systematically combines them into 8 compositional QA types, built from 811 real indoor scenes with 5k+ question–answer pairs across both 2D image and 3D point-cloud settings.", "questions": "See weakness", "limitations": "", "rating": 4, "confidence": 2, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. The paper clearly explains ten basic spatial skills and combines them into eight question types. This setup makes it easier to see which abilities the models are good or bad at, instead of only looking at overall accuracy.\n2. The data collection process is well organized, mixing automated generation with human checking to keep questions accurate and varied. The experiments on about 50 models give useful insights.", "weaknesses": "1. Overall, the benchmark is limited to indoor scenes, which narrows its scope. Real-world spatial intelligence also involves outdoor and embodied settings, for example, navigation and perception in autonomous driving or robotics. \n2. Fixing inputs to 8 images may limit multi-view reasoning. What is the performance when the number of views grows?\n3. MCQ-only setup: This misses tasks that need precise outputs (e.g., 3D grounding with (x,y,z) coordinates, path planning). What is the current status of MLLM performance on these tasks?\n4. Intuitively, reasoning directly in 3D should work better, but in this benchmark the 3D LLMs perform worse than the 2D LLMs. Would denser point clouds help? 1,024 points seem too sparse and may not capture the whole scene. Also, would it be possible to use the same 8 images with a feed-forward reconstructor (e.g., VGGT) to build a point cloud, then feed that into the 3D MLLM?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "AY1r2A8pyx", "reviewer": "Reviewer_WP4t", "summary": "The paper propose a benchmard dataset(Space-10) to evaluate MLLM's spatial understanding. It contain 5K QA for 811 real indoor scenes. The authors define 10 spatial capabilities focusing on different aspect of sptial understanding like counting, localization etc. For dataset constuction, the authors utilize multi-stage of manual and automated approaches to refine the quality. The benchmark is then used to evaluate various open and close source MLLM.", "questions": "Please see the weakness above.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. The author conducted extensive experiments on different MLLM on the proposed dataset.\n2. The paper is well written and easy to follow. Figures look nice.\n3. Evaluating on models compositional spatial capabilities is interesting and can help to understand the MLLM's capabilities.", "weaknesses": "1. The use of abbreviation and classes number is a bit complicated and hard to read.\n2. The proposed division into ten classes is not entirely convincing, and describing them as atomic seems somewhat overstated. For instance, Classes C1, C7, and C8 all exhibit 100% coverage and are included across all eight compositional capabilities, effectively making them equivalent to the overall average. This raises the question of whether defining them as separate classes provides any meaningful distinction. Similarly, Classes C3 and C10 consistently co-occur, they are either both present or both absent within the same tasks, suggesting that none of the classes are truly independent. Given this strong interdependence, referring to them as atomic does not seem appropriate.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)