Spatial Mental Modeling from Limited Views
Authors: Qineng Wang
Baiqiao Yin
Pingyue Zhang
Jianshu Zhang
Kangrui Wang
Zihan Wang
Jieyu Zhang
Keshigeyan Chandrasegaran
Han Liu
Ranjay Krishna
Saining Xie
Jiajun Wu
Li Fei-Fei
Manling Li
Venue: ICLR 2026 Poster (2026)
Keywords: Vision Language Models
VLMs
Multi Modal Language Models
Spatial Intelligence
Spatial Reasoning
Primary area: foundation or frontier models, including LLMs
TL;DR: We propose MindCube and find existing VLMs perform poorly on it. Supervising models to first generate cognitive maps and then reason upon them proves to be a quite effective approximation for spatial mental modeling from limited views.

Abstract: Can Vision Language Models (VLMs) imagine the full scene from just a few views, like humans do? Humans form spatial mental models, internal representations of unseen space, to reason about layout, perspective, and motion. Our new MindCube benchmark with 21,154 questions across 3,268 images exposes this critical gap, where existing VLMs exhibit near-random performance. Using MindCube, we systematically evaluate how well VLMs build robust spatial mental models through representing positions (cognitive mapping), orientations (perspective-taking), and dynamics (mental simulation for "what-if" movements). We then explore three approaches to help VLMs approximate spatial mental models, including unseen intermediate views, natural language reasoning chains, and cognitive maps. The significant improvement comes from a synergistic approach, "map-then-reason", that jointly trains the model to first generate a cognitive map and then reason upon it. By training models to reason over these internal maps, we boosted accuracy from 37.8% to 60.8% (+23.0%). Adding reinforcement learning pushed performance even further to 70.7% (+32.9%). Our key insight is that such scaffolding of spatial mental models, actively constructing and utilizing internal structured spatial representations with flexible reasoning processes, significantly improves understanding of unobservable space.

Published as a conference paper at ICLR 2026
SPATIAL MENTAL MODELING FROM LIMITED VIEWS
Qineng Wang'
Baiqiao Yin 1 Pingyue Zhang'
Jianshu Zhang'
Kangrui Wang's
Zihan Wang'
Jieyu Zhang'
Keshigeyan Chandrasegaran"
Han Liu'
Ranjay Krishna'
Saining Xie
Jiajun Wu
Li Fei-Feft
Manling Li"
Northwestern University
Stanford University
New York University
Uniiersity Washington
Website
Code
Dataset
ABSTRACT
Can Vision Vision-Language Models (VLMs) imagine the full scene from just few views, like humans do? Humans form spatial mental models naturally, internal representations of unseen space, to reason about layout, perspective, and mo- tion. Our MINDCUBE benchmark with 21, 154 questions across 3, 268 images exposes this critical gap, where existing VLMs exhibit near-random performance. Using MINDCUBE, we systematically evaluate how well VLMs build robust spa- tial mental models through representing positions (cognitive mapping), orienta- tions (perspective-taking) and dynamics (mental simulation for "what-if"" move- ments). We then explore three approaches to help approximate spatial mental models in VLMs, focusing on incorporating unseen intermediate views, natu- ral language reasoning chains, and cognitive maps. The significant improve- ment comes from synergistic approach, "map-then-reason" that jointly trains the model to first generate a cognitive map and then reason upon it. By training models to reason over these internal maps, boosted accuracy from 87% to 60.8% (+23.0%) (... Adding reinforcement learning pushed performance even fur- ther to 70.7% (+32.9%). Our key insight is that such scaffolding of spatial mental models, actively constructing and utilizing internal structured spatial representa- tions with flexible reasoning processes, significantly improves understanding of unobservable space.
INTRODUCTION
For Vision-Language Models (VLMs) (OpenAI, 2024) to move beyond passive perception (Li et al., 2023) to interact with partially observable environments (Yang et al., 2024), fundamental to reason about unseen spatial relationships from limited views. Consider how effortlessly human can infer the layout of a room or the hidden objects behind furniture, all by integrating information from several egocentric observations. For example, given the second viewpoint in Figure 1, human can easily infer the unseen objects behind the "plant" are the "tissue box" and the "hand sanitizer", including their position, pose, and their relationship with objects that are not simultaneously visible. We humans build and update a mental model of our surroundings, even when objects are out of sight. This is enabled by core cognitive function referred to spatial mental model Johnson-Laiie 1980; 1983): an internal representation of the environment that allows for consistent understanding and inference about space, independent of the current viewpoint. VLMs, despite their impressive progress, struggle to synthesize spatial information from limited views, maintain spatial consistency across views, and reason about objects not directly visible (Ma et al., 2025a).
This gap calls for specialized evaluation settings, which must include: (a) reasoning with partial observations where objects are occluded or out of view (such as "hand sanitizer" in the second viewpoint in Figure 1), (b) maintaining cross-view consistency across shifting viewpoints (such as through anchor objects "plant"), and (c) mental simulation to infer hidden spatial relationships (such as "what if turning left and moving forward""). To fill this gap, introduce MINDCUBE, featuring 21, 154 questions and 3, 268 images, organized into 976 multi-view groups through various types of viewpoint transformations (i.e., ROTATION, AMONG, AROUND in Figure 2). We annotate questions
Equal contribution. listed in alphabetical order. I Equal advising.

---

Our extensive evaluations of 17 oftte-of-the-at VLMs MINDCUBE reveal that both open- weight and closed-source models perform only marginally better than random guessing. This poor per- facilitate spatial mental models to reason Published as a conference paper at ICLR 2026
The Challenge: Spatial Mental Modeling from Limited Views
Views
VLMs Are Bad at Spatial Mental Modeling
Posilion & Orientation
Mental Simulation
VLM
Question
If you are at view and move to view 2. what the furthest from you?
Potted plant
B. Hand sanitizer
@
Imagine the scene
Move view
Black shelf
D. Fireplace
How to Approximate Mental Model in VLMs?
View Interpolation
View Interp. Ane
Invesfigated Seftings
GA
Instruction
Inserted
Col Ars
Reasoning Instruct
Base Perf.
on images and the question. please first think step by step then answer
+0.09%
E
SFT
+2.67%
37.81%
+0.91%
22.95%
3.52%
+12.76%
QA
Cognitive Map
Free- Form Reasoning
RL
+5.90%
Three Cognitive Scaffold Data Structures
Elicitation Methods
Configurations Performance Overview
Figure 1: Top: VLMs cannot maintain a coherent mental model when evaluating on the MINDCUBE benchmark. Bottom: We study how
help build spatial mental models through external (scal- ing of views, cognitive map input) and internal strategies fine-tuning, cognitive map elicitation). We find joint cognitive map and reasoning setting yields the highest gain (+32.86%%
with a focus objects that are not visible in the current query view. As shown in Figure 2, we systematically design question types requiring "what-if" mental simulations from the given view (such "what if turning left"), perspective taking (such as "what if taking the sofa' 's perspective"), complex relation reasoning queries (referencing either the agent or other objects).
formance motivates central question: How effectively from partial observations?
Inspired by spatial cognition (Ramakrishnan al., 2025; Lee al., 2025; Zha et al., 2025) oper- ating through visual imagery, linguistic reasoning, or explicit cognitive maps, to build consistent spatial awareness across different views, we investigate three approaches determine whether in- termediate representations can assist approximating spatial mental models in VLMs. View Inter- polation enhances the input by providing additional views and thereby offering more information using recorded video, which unexpectedly is not helpful, highlighting the importance of reasoning directly from limited views. Free-form Natural Language Reasoning verbalizes the mental simu- lation process, achieving performance gains (+2.7%). Structured Cognitive Map simulates global spatial memory from an allocentric (bird's s-eye) perspective with orientation and view augmenta- tion. Interestingly, providing ground truth cognitive maps directly to answer questions will not yield strong improvements (-5.81%). only actively engaging reasoning with a map achieves strong im- provements (+3.62%). Despite the effectiveness of reasoning
maps, building accurate spatial mental models exhibit a significant bottleneck attributed to VLMs' intrinsic ability, evidenced by low Isomorphic Rates (<: 10%) with ground truth maps during generation.
2

---

Published as a conference paper at ICLR 2026
Rotation
Question types
Question: If left. what is to your left?
third viawpoint and degreas
"What It" Dynamics
Options:
bin
rotation
lable
Pallway
Bookcose
agent-objoct
solf perspective
non-linwar
Around
Relation Query
Question: you are posilioned third viewpoint. move forward, will you get closer to the red trash bin?
Options:
agent-ooayeet
B.
No
[ggent-agem
agent-abject
perspectiion
object- object
if you are postiooed at the first viewpoint, what to the left of Among
Perspective Taking
Visual Patterns
Question:
you
Options:
perspective
Safa
Windows
perapection)
TV oolcinel
Dining Toble
meansoill
object-abject
belf perspective
Bon
Figure 2: MINDCUBE taxonomy and examples. Left: Three camera movement patterns (ROTATION, AROUND, AMONG) with corresponding spatial QA examples. Right: Four-dimensional taxonomy categorizing MINDCUBE questions types.
Recognizing this limitation, we train VLMs by constructing 10, 000 reasoning chains and ground truth cognitive maps, investigating how effectively guide spatial mental models toward achiev- ing accuracy. While SFT free-form reasoning chains proved more effective with gain of + +1.2%, guiding models to first build cognitive maps and then perform free-form reasoning over them achieved significantly better performance, resulting in total gain of +8.5%, proving scaf- folding spatial mental models via actively constructing and utilizing internal structured spatial rep- resentations with flexible reasoning processes highly effective. We also use Reinforcement Learn- ing (RL) to further boost post-SFT performance, guiding models to think in terms of building and reasoning over cognitive maps by injecting structured thinking before RL training, using our SFT model. This approach leads to significant improvement, raising task accuracy from baseline of 37.8% to 70.7%. Our empirical evidence substantiates critical finding: autonomously generating and leveraging internal mental representations help VLMS exhibit superior performance in spatial reasoning tasks, as compared conventional approaches such view interpolation or externalyysuupplee maps.
2 MINDCUBE BENCHMARK AND EVALUATION
2.1 MINDCUBE BENCHMARK
Overview. We introduce MINDCUBE, benchmark for evaluating VLMs' spatial reasoning under partial observations and dynamic viewpoints. MINDCUBE features multi-view orthogonal images paired with spatial reasoning questions, enabling fine-grained analysis of spatial mental modeling performance. It targets key challenges such maintaining object consistency across views and reasoning about occluded invisible elements.
Settings. MINDCUBE incorporates three distinct settings-Rotations Around and Among (vi- sualized left of Figure 2). In the Rotation setting, the challenge lies in interpreting multiple orthogonal views from static and rotational observation point, requiring models to form holistic understanding of the environment despite only incremental visibility shifts. The Around setting leverages occlusion to foree VLMs maintain object permanence to convert lateral (left-right) relations in frontal views into depth (front-back) cues side views. The Among setting maintain spatial consistency and overcome visibility constraints views
with partial visibility and
3

---

Published as a conference paper at ICLR 2026
Table I: Left: MINDCUBE data statistics. The number next the setting (ROTATION, AMONG, ArOund) means the total QA pairs. Numbers next to each dataset (e.g., Arkitscenes) mean QA pairs/image groups. For example, "865/53" for Arkitscenes in ROTATION means 865 QA pairs and 53 image groups from Right: Performance VLMs on MINDCUBE. Dark blue indicates the best result among all models and light blue indicates the second best result among all models.
Rotation (1081)
Method
Overall
Rotation
Among
Around
Baseline
Arkilscenes
865/53
(chance)
Random (frequency)
33.02
38,30
32,66
35.79
Self collected
216/9
Open-Weight Weight Multi Image Models
Img groups
62
I. MWVA-OOeviiion-7755 7H al. (20245)
47.43
36.45
48.42
44.19
LLaVA-Video-OUe
Zhang al. (2024d)
41.96
35.71
43.55
30.12
LongVA-7B Zhang (2024c)
29.46
35.89
29.55
24.88
Among (18204)
WLLUGGOWW3-7--21111 Ye al. (2024)
44.85
37.84
47.11
26.91
IntemVLL--8B Zhu et (2025)
37.50
26.00
42.03
36.00
WildRGB-D
17500/710
Bren2.5-Vl-7H-Inssmee Bai (2025)
29.26
38.76
29.50
21.35
Swen2.5-VL-33B-Iae
Bai et al (2025)
33.21
37.37
33.26
30.34
DL3DV-1OK
704/24
DeepSeek V1.2-Small al. (2024)
26.91
Gemma-3-122 3-auna3--12Biit Team al. (2025)
46.67
38.39
48.38
34.63
Img groups
733
Mantis 8B (pi Jiang et al. (2024)
41.05
37.65
40.23
50.99
Proprietory Models
GPT-5-2025-08-07 OpenAl (2025)
47.59
93.33
34.17
41.63
Around (1869)
Gemini-2.5-ppo-202544 Team (2025)
47.05
85.50
25.95
38.40
Soate-4-Sonnet-222255
Anthropic (2025)
44.75
48.42
44.21
47.62
DL3DV-10K
789/109
Sputial Models
RoboBrain Ji (2025)
37.38
35,80
38.28
29.53
Self collected 1080/71
SpaceMantis Chen et al. (2024a)
22.81
37.65
21.26
29.32
Spatial-MLLM al. (2025a)
32.06
38.39
20.92
32.82
Img groups
180
Space-Qwen Chen (2024a)
33.28
38.02
33.71
26.32
captured around central object with adjacent ones, each view showing the central object posi- tioned before one surrounding element. VLMs need to share information across views, deducing the overall spatial arrangement and relationships even when not all elements are visible simultancously. Table (left) summarizes the benchmark's 's overall data distribution. Details on benchmark design about settings and taxonomies and curation are provided in the Appendix B, C and B.2.2.
Dataset Curation. The MINDCUBE dataset was created through a pipeline: We first selected multi-view image groups matching taxonomy's movement patterns (Figure 2) and spatial crite- ria. These were then annotated with key spatial information. Finally, we algorithmically generated taxonomy-aligue questions with targeted distractors. Details included in the Appendix B.
2.2 EVALUATION ON MINDCUBE
We evaluate VLMs' spatial mental modeling ablities on MINDCUBE using diverse set of models (Table I, right; setup details the Appendix C). Results reveal a striking performance gap: the best model, -e--S-eee------nnee human-level C.3. While some models show strength in specific areas arcas-notable GPT-5 in ROTATION (93.33%) and Mantis-8B (SigLip) in AROUND (50.99%)-o single model excels across all cate- gories. We also observe that proprietary models generally outperform the open-source ones. Spatial fine-tuning also yielded varied outcomes without consistently reaching top performance. Overall, neither multi-image input nor spatial fine-tuning reliably improves spatial reasoning, raising a key question: How can help VLMs develop or approximate these crucial spatial reasoning ca- pabilities?
achieves only 47. 17.62% accuracy, well above chance but far from
WHICH SCAFFOLDS BEST GUIDE SPATIAL MENTAL MODELING?
To address the identified gap, we first evaluate whether structured data forms can scaffold spatial reasoning in frozen VLMSs by approximating spatial mental models under limited views.
4

---

Published as a conference paper at ICLR 2026
Views
View Interpolation
Cognilive Map
Reasoning Chain
Augmented
Plain [Obj Only)
Free-Form
Nageetatts II
Nhe
"peaction's Ox 5)
1.
"position"s
castcizes teim JAEE to sigst,
"poaition"s By 51
petrod plants so bith viiee
*eleephi
ascástaaz's
Question
"peactton's ON S4
"poaltionns F7 5]
saguomeed in Wie by poiess
Sabied the
you ore view and move to view 2. what is furthest from you?
31,
Vine
"pectsictss S1 "elano"
"actogghs Tooms"
20 vler the potted plase La
cantstver la the Porshent from
Views and Question
Examples
Three Data Structures We Studied to Approximate Spattal Mental Models
Figure 3: Grounded examples of our three data structures that approximate spatial mental models.
3.1 DATA STRUCTURES AS COGNITIVE SCAFFOLDS FOR SPATIAL MENTAL MODELS
We investigate whether certain data structures can act as cognitive scaffolds that help form spa- tial mental models in VLMs from limited visual observations. In cognitive science, spatial mental models are internal representations encoding the relative configuration of objects and viewpoints. Rather than metric-precise maps, they are schematic, manipulable constructs that support reason- ing across fragmented observations and unseen perspectives (Johnson-Laiee 1983; Tversky, 1993; Tversky et al., 1994; Tversky, 2003). For instance, humans can mentally simulate turning or infer what lies behind them, suggesting that such representations are flexible, incomplete, yet functionally effective. Drawing on this literature, we define three data structures below (detailed introduction can be found in Appendix D.I), each targeting distinct cognitive properties (integration, transformation, inference) of spatial mental models, with grounded examples in Figure 3:
View Interpolation. Interpolating between sparse views introduces perceptual continuity, echo- ing the process of mental animation (Hegarty, 1992) and supporting internal transformation such as imagined rotation. This structure scaffolds the dynamic updating capability of spatial mental models. Figure 3 shows a framframe inserting example that replaces the original question images. 2. Augmented Cognitive Map. A cognitive map is 2D schematic representation of object layouts in space. Such maps resemble Tversky's cognitive collages (Tversky, 1993), and they capture locally coherent but fragmented structures. Recent studies (Yang et al., 2024; Yeh et al., 2025) on VLM-based spatial intelligence typically adopt plain form that only encodes object positions in top-down view. We propose an augmented variant that incorporates discrete views, with both objects and views annotated by position and orientation, thereby approaching the relational consistency of spatial mental models.
3. Free Form Reasoning. Open-ended, step-by-step natural language reasoning offers a procedural approximation of how spatial models are constructed and queried. While less rigid than make structures, such reasoning reflects the inferential function of spatial mental models, especially under ambiguous or incomplete observations (Tversky et al., 1994).
3.2 EXPERIMENT SETUP
We conduct controlled experiments with fixed input formats test whether structured scaffolds can help without retraining. Each condition introduces a different structure to support internal modeling.
Configurations and Evaluation Metrics. Each experiment is defined by two orthogonal axes: In- pul Structure (what spatial evidence VLMs receive) and Output Format (the required response type). As the experimental foundation of this paper, we begin with the ten possible configurations listed in Table 2, from which we investigate a representative subset. Specifically, our grounded cognitive maps are generated using the object arrangements annotation described in Section 2. and examples for all configurations are provided in the Appendix D.3. In the frozen VLMs evaluation setup, we exclude the CGMap-Our and Plain Plain-CGMap Out settings, as VLMs tend to conflate map generation with reasoning, even when instructed otherwise. Beyond evaluating task performance using QA accuracy, we also introduce two well-defined graph metrics for generated cognitive maps: (1) Overall Similarity, weighted score combining directional and facing consistency; and (2) Iso- morphic Rate, measuring whether all pairwise object relations match the ground truth under optimal alignment. Full definitions provided in the Appendix D.2.
5

---

Published as a conference paper at ICLR 2026
Table 2: Input-output configurations used in all experiments. The suffix "-In" means the cognitive map is given to the model as input, whereas "-Out" means the cognitive map is predicted as an intermediate output before answering. "Aug" indicates maps with object and camera annotations; "Plain" indicates maps without these augmentations. VI= View Interpolation, CGMap Cognitive Map, FFR Free-form reasoning. Figure shows visual examples input structures.
Name
What the model receives (input)
What the model produces (output)
Raw-QA
Raw views question text
Direct answer
VI-1
Raw views 1 interpolated view
Direct answer
question text
VI-2
Raw views 2 interpolated views + Direct answer
question text
FFR
Raw views question text Augmented cognitive map (objects
Free-form reasoning answer
Aug-CGMap-In
Direct answer
camera) + question text
Aug-CGMap-Out
Raw views + question text
Augmented cognitive map + answer Plain cognitive map + answer Augmented cognitive map free-form reasoning - answer
Flain-CoMap-Out
Raw views question text Raw views question
Aug-coMap-FER-Out
>lain-CGMap-FFR-OU
Raw views question text
Plain cognitive map free-form rea- soning answer
IGMap-In-EER-Out
Augmented cognitive map (objects + camera) question text
Free-form reasoning - answer
Model and Evaluation Data We conduct all experiments using W.nen..5-VL-3B-Instmee (Bai al., 2025) with all evaluations performed on MINDCURE-TINN, diagnostic subset sampled from MINDCUBE, containing 1,050 questions in total. Detailed statistics are: 600 from AMONG, 250 from AROUND, and 200 from ROTATION.
3.3 DO SCAFFOLDS IMPROVE SPATIAL MENTAL MODELING WITHOUT TRAINING?
We evaluate how well the seven input configurations defined in Table 2 support spatial mental mod- eling in VLMs under limited views, without any model updates. Results are shown in Table 3 (left).
How far can structure alone go? We begin with the baseline: raw input views and direct answering (Raw-QA), which achieves 37.81% 1881% % accuracy. Adding interpolated views, which we hope to simu- late smoother perceptual transitions, leads no meaningful gain (t 0.09%). We include a further analysis VI in Appendix E.3. Similarly, providing pre-computed augmented cognitive map as direct input (Aug Saagg--GMa-IIIi severely degrades performance to 32.00% In contrast, enabling free-form reasoning (FFR) alone or combined with other settings provides a substantial boost to 41.33%. These results suggest: structure alone, whether visual or spatial, is not enough. Without engaging reasoning, VLMs struggle to leverage even well-formed spatial cues to improve spatial mental models.
Table Left: QA accuracy (%) of Swen2..5-VL-3B--nsttee on the MINDCUBE- -INY benchmark under different configs for frozen VLMs. Right: Graph metrics for two cog map output settings.
100
Config.
Overall
Rotation
Among
Around
Raw-OA
37.81
34.00
36.00
45.20
41
VI-1
37.907
35.50
37.33
41.20
Similarit
Overall
VI-2
36.50
42.80
Auq-CGMap-In
32.00.
35.00
30.50
33.20
FFR
40.48
32.00
36.00
58.00
Isomorph
11.45
CG Map
Output (Aug)
40.571
21.00
43.00
50.40
-ic Rate
16.84
CG
BLLLn--CMMp-FFR-OOO
41.331
25.00
39.67
58.40
Outout (Plain)
CGMap-In- FFRROOU
41.431
41.67
44.40
Can we prompt the model to think spatially? The answer appears to be yes. Prompting the before
model to generate cognitive map (Aug-CGMap-FFR-OUr

---

Published as a conference paper at ICLR 2026
answering leads to further improvements over free-form reasoning alone (FFR) from 40.48% to 41.43%. This suggests that generating a map may encourage the model to first form a global un- derstanding of the scene, which in turn supports more structured reasoning. Both map forms have great format-following ability, yet fail to generate accurate maps. Overall, augmented maps perform worse. In Table 3 (Right), despite generating syntactically valid maps for both formats, similarity to grounded maps is low (< 50%), reflecting limited mapping ability. Notably, both augmented and plain maps have low isomorphism rates (0%10% 7.43%). The reason that the isomorphic rate for augmented map setting is nearly zero is likely because the added view-level details increase generation errors. Detailed case examples can be found in the Appendix
Key Takeaways: Scaffolding Spatial Mental Models Frozen VLMs
Explicit reasoning is crucial for improving performance.
Reasoning acts as a necessary mechanism to ground spatial structure in frozen settings.
Passive structures (like maps input) alone and visual continuity offer little benefit.
4 CAN WE TRAIN FOR THE EMERGENCE OF SPATIAL MENTAL MODELS VIA VLMs' USE OF SCAFFOLDS?
So far, prompting frozen VLMs with external scaffolds, such as interpolated views or cognitive maps, has yielded limited gains. These techniques fail to tackle the core limitation: VLMs do not form internal spatial representations
reason through space effectively. To go further, we want to know: Can supervised fine-tuning (SFT) and Reinforcement learning (RI.) teach VL.Ms build and leverage spatial mental models from within?
4.1 DESIGNING A ROBUST EXPERIMENTAL FRAMEWORK
To ensure consistency and comparability, we inherit experimental configurations detailed in Sections 3.1 and 3.2. Specifically, we retain: (1) the two effective data scaffolds Cognitive Maps (Object- only / Object Camera) and Free-Form Reasoning, (2) the base model Qwen2.5-VL. -1--5---VL-B--strrue (3) the evaluation benchmark MINDCUBE-TiNY and (4) all established evaluation metrics. View interpolation is excluded due to its limited performance gains in carlier validations.
SFT Task Configurations. Drawing on insights from Section 3.3, we use selected configura- tions from Table 2 to evaluate the incremental impact of cognitive map generation and free-form reasoning in SFT. These include baseline QA without explicit reasoning (Raw-QA), reasoning auggeeted
guided by generated maps only (P (Plain-CGMap-O0 prompts (FFR), and a fully integrated setup that asks VLMs to generate both maps and reasoning FFF---GMMP-FF-----
Aug-CGMap-Ouu
and Pl MainnnGMap--F------
RL Task Configurations and Reward Design. We employ the VAGEN framework (Wang* et al., 2025) for VLM policy optimization, using Group Relative Policy Optimization (GRPO) (Shao al., 2024) as our core algorithm. We evaluate three RL variants: (1) RL-FFR (from scratch), which trains base model produce free-form reasoning chains; (2) RL-Aug CGMap- FFR-FFFROO Out (from scratch), which trains the model to jointly generate cognitive maps and reasoning; and (3) BLL-Aug-CGMap-FEES
(from SFT), which initializes from the strongest SFT check- point. Detailed settings can be found in the Appendix G.I.
Grounded Cognitive Maps and Free-Form Reasoning Chain. Grounded cognitive maps are not only used the input in the Aug-CGMap-In and CGMap-In-FFR-WW setting for the frozen VLMs in the Section 3.2, but also as the training and comparison data. We curate such grounded cognitive maps through a template-based method, where we always select the front image in our annotation as the "up" direction. We also manually constructed grounded reasoning chains using detailed image annotations and structured question templates, ensuring logical coherence and clear grounding in observable spatial relations (see an example in Figure 3). The detailed grounded cog- nitive maps and reasoning data generation pipelines are shown in the Appendix F.1.1 and F.1.2

---

Published as a conference paper at ICLR 2026
80-
.50
Overll Stutturk (50
80
Comerrii Buttr Sonnrraa
70
40
Ong-OOMappOuu Aug-C0Map-778-0u
Oap
Aup-CCMap-OOt Aug-
05
200
OGMpp-OU
Plais CCMap-Out
Plain MaMpa
20
Plain OGMap Out
CCMpp FFR
CCXpp
(a) QA accuracy (%)
(b) Isomorphic Rate (%)
(c) Overall Similarity (%)
Figure 4: SFT per 5 step training performance on task accuracy and graph metrics.
4.2 DO THE EMERGENCE OF SPATIAL MENTAL MODELS TRULY BENEFIT FROM EXPLICIT TRAINING?
We explore several SFT configurations (results shown Table 4), guided by series of core ques- tions. Fine-tuning directly on raw QA pairs, without spatial supervision, raises accuracy from 37.81% % to 52.28%. This suggests VLMs can absorb some spatial cues from QA data alone. We use this setup the baseline for evaluating methods that explicitly incorporate spatial structures. Primary modifications in SFT phase include adjusted training hyperparameters (detailed the Ap- pendix F.2) and the input-output configurations.
Table 4: QA accuracy (%) and cognitive map generation quality of Qwen2.5 5-V..----L-B------ -L-3B-IInstruct under both SFT and RL on MINDCUBE- TINY. FFR refers to free-form reasoning. Bolded means the best within that training category (SFT RL).
MINDCUBE- -INY QA Accuracy
Generated Cognitive Map
Config.
Rotation
Among
Sim.
Raw-GA
$2.38
34.30
52.50
66,00
AUg-CGMap-Out
54.197
35.50
53.17
71.60
74.30
43.24
Plain-cakag-Out
54.38*
35.50
$3.50
71.60
91.73
89.05
Aog-CGMap-PFR-OOuE
55 24
46,00
Blain-CGMaa-FFFR
60.76'
47.50
62.33
67.60
88.79
73.81
RI-FER (from se*atch)
50.57
8L- AL-tug-CGMae CoMap-FFR-Outt (from scratchi
$2.19
32.00
$2.00
68.80
$7.03
0.00
2
RL-PPain-CGtapp
scratch)
10.29
CL-tug-CCSaa--FEE (from SPTI
70.67
53,00
76.83
70.00
85.53
58.86
Ml-Plain-CGtap
Ifron SPT)
70.67
48.00
79.17
68.40
85.79
71.52
Can structured approximations mental models alone meaningfully improve performance?
As shown in Table4, supervised fine-tuning on explicit cognitive maps, either Augmented Plain, leads to substantial improvements in graph structure quality, with more than 30% gains in both overall similarity and Isomorphic rate. However, the effect end-task accuracy remains limited. Both augmented maps (54.19%) and Plain maps (54.38%) offer only modest gains over the fine- tuned Raw-QA (52.28%). Similarly, directly FFR also yields marginal gain (53.52%). This means that a scaffold alone is not sufficient to automatically translate into performance gains.
Generating both cognitive maps and free-form reasoning is the most effective approxima- tion. Among all configurations, the combination of generating a plain map and then reasoning Pllain-CGMpp-FRR-
yields performance gain (*8.48% compared to Raw QA-SFT), sur- passing models that rely on only map generation or reasoning alone. This suggests synergy be- tween structured spatial modeling and natural language inference. The training dynamics reveal a crucial trade-off that explains this synergy. As shown in Figure 4 (b, c), models trained solely on map generation (Plaing LLnCCMMa--Ou) learn the target structure very rapidly, quickly reaching near rear-perfect similarity and isomorphism However, their QA accuracy soon plateaus (Figure 4a), suggesting the model learns the structure without fully grasping its functional utility. In contrast, the nop-performing
Setttaa-FFR--ee FFR-Out model learns the map structure more slowly and never reaches the same level of structural perfection. Yet, its QA accuracy continues to increase and surpass all other configurations. This suggests that the joint pressure of the reasoning task forces the model not just to replicate a structure, but to build a functionally effective spatial representation, which can lead to improvement for overall spatial understanding despite being imperfect.
8

---

Published as a conference paper at ICLR 2026
Key Takeaways: Explicit Training for the Emergence Spatial Mental Models
Joint cogmap and reasoning setling yields optimal performance through synergistic effects.
Neither map generation nor reasoning alone largely outperforms the SFT QA baseline.
4.3 CAN REINFORCEMENT LEARNING FURTHER REFINE SPATIAL MENTAL MODELS?
While SFT establishes a strong baseline for spatial mental modeling, emerging evidence from mod- els like DeepSeck RI (Guo et al., 2025) suggests reinforcement learning (RL) can offer additional gains by optimizing behavior through outcome-driven feedback. We ask: Can reward-guided refine- ment help VLMs build sharper spatial models and reason more effectively?
RL lets model feel the consequences of its spatial thoughts through reward, but does that feedback alone forge genuine "mental map," or must we first teach the model what a map looks like? Table 4 summarizes three key settings and answers this question in two parts.
RL in a vacuum is not enough. Training from scratch with sparse rewards provides insufficient guidance for building robust spatial representations. When asked to produce free-form reasoning (RL-FFR (from scratch) ), the model achieves only 50.57% overall accuracy. This result, while an improvement over initial baselines, confirms that task-level rewards alone are too unstruc- tured to effectively teach spatial abstraction.
Structured outputs provide modest benefits when learned from scratch. Introducing a cognitive map structure for the policy to generate offers a scaffold for its reasoning. When starting from scratch, the simpler RL-Plain- DePlaincGMMap--0 CGMap-GFR----------- its augmented counterpart meaningful geometry, with low similarity scores and near-zero isomorphism rates. This suggests that without prior concept of "good" map, RL struggles to exploit the provided structural format, even it can learn to fill it out validly.
configuration (53.71%) slightly outperforms in QA accuracy. However, in both cases, the model fails to learn
RL performs better when it trains from SFT checkpoint. The most dramatic improvements occur when warm- warm-starting RL from an optimal SFT checkpoint. Both RL-P waw.aaannnaaaa---aeeee (from SFT) and its augmented version reach an identical, impressive 70.67% overall QA accu- racy, This represents a significant 19.91% absolute gain over the best SFT model and a 11..66% gain over the best (L-ffromscrateh approach. Crucially, while both models achieve the same peak accuracy, their underlying spatial representations differ. The P CCMip variant produces geo- metrically superior maps, with a much higher isomorphism rate (71. (71.52% 58.86%). This suggests that while RL fine-tuning can guide different initial models to the same reasoning proficiency, start- ing with a cleaner, simpler SFT scaffold (Plain) allows RL better preserve and polish geometri- cally sound internal map. These results strongly indicate that RL's primary role here is (1) polishing and refining the strong priors learned during SFT, and (2) raising the performance ceiling of SFT, enabling the model to break through previous plateaus to achieve pear-oracle-level performance.
Out
Key Takeaways: Reinforcement Learning for the Emergence of Spatial Mental Models
Combining cognitive maps with reasoning consistently improves all learning outcomes.
Starting from scratch, RL provides only marginal gains for spatial reasoning; its true power is unlocked when building upon strong SFT foundation.
RELATED WORKS
Spatial Cognition. Spatial cognition encompasses skills like mental rotation. spatial visualiza- tion. and object assembly, essential for perceiving and manipulating spatial relationships in both 2D and 3D environments (Xu et al., 2025b; Zha et al., 2025; Wang et al., 2025). At the core of these abilities are Spatial Mental Models (SMMs) (Johnson-Laird, 1980; 1983), which are internal representations that allow for consistent understanding of space. Recently much effort has been dedicated to evaluating spatial cognition in VLMs (Zhan et al., 2025; Ma et al., 2025a; Lee et al., 2025; Zhang et al., 2025). Moreover, some methods are proposed to enhance spatial understanding, such coordinate- aware prompting (Cai et al., 2024), CoT reasoning (Ma et al., 2025b; Liu et al., 2025b), explicit spatial representation alignment (Cheng et al., 2024; Chen et al., 2024a), and an RL--based approach (Pan & Liu, 2025). However, existing benchmarks (Lee al., 2025; Zhan al.,

---

Published as a conference paper at ICLR 2026
2025: Chen al., 2025; Qi et al., 2025; Zhang al., 2025; Ma et al., 2025a; Ramakrishnan et al., 2025; Tang et al., 2025b; Fu et al., 2024; Yang et al., 2024: Zhang et al., 2024a) and approaches often neglect the mental-level spatial reasoning that underpins human cognition, leaving gap between machine and human capabilities. To bridge this gap, a new approach is needed that trains VLMs reason about space only through visual data but also through mental-level spatial reasoning. aligning more closely with human spatial cognition.
Multi Views understanding. Multiview spatial understanding leverages multiple viewpoints re- construct 3D structures and overcome single-view limitations. Efficient techniques optimize view processing, while reconstruction methods (Wang et al., 2025; Liu al., 2025a; Fu al., 2025; Qu et al., 2025), view synthesis methods (Sun et al., 2018; Zhang et al., 2024e; Sargent et al., 2023) and multiview equivariant learning (You et al., 2024) enhance geometric consistency. Topological repre- sentations like Zhang et al. (2024b) encode object relations for holistic reasoning, while frameworks such as Hong et al. (2023) advance nren-vocabulary concept learning from multiview data via neu- ral fields and vision-language fusion. LMMs augmented with multiview inputs (Daxberger et al., 2025; Wu et al., 2025a; Fan et al., 2025; Zheng et al., 2025; Lee et al., 2025; Zhao et al., 2025; Xu et al., 2025a) demonstrate marked improvements in spatial tasks like geometric understanding and perspective taking. Yet. they struggle with multiview consistency understanding due to fragmented reasoning and 2D-to-3D projection ambiguities, leaving a gap for robust spatial AI.
6 CONCLUSION
We introduced MINDCUBE study how VLMs can approximate spatial mental models from lim- ited views, a core cognitive ability for reasoning in partially observable environments. Moving be- yond benchmarking, we explored how internal representations can be scaffolded through structured data and reasoning. Our key finding is that constructing and reasoning over self ceff-genrrated cogni- tive maps, rather than relying on view interpolation or externally provided maps, yields the most effective approximation of spatial mental models across all elicitation methods (input-output con- figurations, supervised fine-tuning, and reinforcement learning). Initializing RL from a well-trained SFT checkpoint further optimizes the process, pushing spatial reasoning performance to new limits.
ETHICS STATEMENT
The MINDCUBE benchmark was developed using a combination of publicly available, anonymized datasets (ArkitScenes, WildRGB-D, DL3DV-10K) and self-collected imagery. For our self-collected data, care was taken to capture indoor and outdoor scenes without including personally identifiable information (PII) or sensitive content. All human annotators involved in the data curation and eval- uation phases were compensated at rates significantly exceeding their local minimum wage.
We acknowledge several limitations and ethical considerations. The datasets used, while diverse, may not fully represent the vast range of global environments, potentially introducing geographic or cultural biases into the model's spatial understanding. Furthermore, the training, fine-tuning, and evaluation of the large-scale Vision-Language Models discussed in this paper carry a significant computational and environmental cost. While our research is intended to advance the scientific un- derstanding of Al cognition, we recognize that technologies enhancing spatial reasoning in machines could have dual-use applications.
REPRODUCIBILITY STATEMENT
To ensure the reproducibility of our findings, we have included our complete codebase for data processing, model training, and evaluation in the supplementary materials as a .zip archive. Further- more, the full MINDCUBE benchmark, encompassing all of our training data, test data, annotations, and evaluation protocols, will be released in a public repository to facilitate further research and verification by the community.
ACKNOWLEDGMENNTS
This work is in part supported by the Stanford Institute for N00014-23-1-2355 ONR MURI N00014-22- 1-2740, ONR MURI N00014-21-1-2801, and DSO National Laboratories Agreement DSOCO25017.
AI (HAD, ONR
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Anthropic. Claude 4 sonnet system card, May 2025. URL
h /wwp://h//wwwwc...././ wwt.rooic................. com/ 6 9 9a52cb Bebbbe6ebeeeee wew.lleeeeDbbbbeeeeeoooo 3dd påf. Version 20250514, ac- cessed 2025-06-23.
Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang. Peng Wang, Shijie Wang, Jun Tang. Humen Zhong, Yuanzhi Zhu, Mingkun Yang, Zhaohai Li, Jianqiang Wan, Pengfei Wang, Wei Ding. Zheren Fu, Yiheng Xu, Jiabo Ye, Xi Zhang, Tianbao Xie, Zesen Cheng, Hang Zhang, Zhibo Yang. Haiyang Xu, and Junyang Lin. Qwen2.5-vl technical report. arXiv preprint ar arXiv:2502. 13223, 2025.
Gilad Baruch, Zhuoyuan Chen, Afshin Dehghan, Yuri Feigin, Peter Fu, Thomas Gebauer, Daniel Kurz, Tal Dimry, Brandon Joffe, Arik Schwartz, et al. Arkitscenes: A diverse real-world dataset for 3d indoor scene understanding using mobile rgb-d data. In Thirty-fifth Conference on Neural Information Processing Systems Dutasets and Benchmarks Track (Round 1), 2021.
Wenxiao Cai, Yaroslav Ponomarenko, Jianhao Yuan, Xiaoqi Li, Wankou Yang, Hao Dong, and Bo Zhao. Spatialbot: Precise spatial understanding with vision language models. arXiv preprint arXiv:2406. 13642, 2024.
Boyuan Chen, Zhuo Xu, Sean Kirmani, Brian Ichter, Danny Driess, Pete Florence, Dorsa Sadigh, Leonidas Guibas, and Fei Xia. Spatialvim: Endowing lsionnlanguage models with spatial rea- soning capabilities. arXiv preprint arXiv:2401.12168. 2024a. URL https://aaxaat wwr.viorrg///oo.. abs 2200/2 1211..1266 12169.
Shiqi Chen, Tongyao Zhu, Ruochen Zhou, Jinghan Zhang, Siyang Gao, Juan Carlos Niebles, Mor Geva, Junxian He, Jiajun Wu, and Manling Li. Why is spatial reasoning hard for vlms? an attention mechanism perspective on focus areas, 2025. URL aittp:://arxive org/ /bs/ 2503.01 01773.
Zhe Chen, Jiannan Wu, Wenhai Wang. Weijie Su, Guo Chen, Sen Xing, Muyan Zhong, Qinglong Zhang, Xizhou Zhu, Lewei Lu, et al. Internvl: Scaling vision foundation models and aligning for generic visual-linguistic tasks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 144855221999 2024b.
An-Chich Cheng. Hongxu Yin, Yang Fu, Qiushan Guo. Ruihan Yang, Jan Kautz, Xiaolong Wang, and Sifei Liu. Spatialrgpt: Grounded spatial reasoning in vision language models, 2024. URL https: //arxi////vvvv org/ abs 620 6.0060000.00... 01584.
Erik Daxberger, Nina Wenzel, David Griffiths, Haiming Gang, Justin Lazarow, Gefen Kohavi, Kai Kang, Marcin Eichner, Yinfei Yang, Afshin Dehghan, and Peter Grasch. Mm-spatial: Exploring 3d spatial understanding in multimodal Ilms, 2025. URL htLpttts: / org/abs/2533.3. 13111.
Danny Driess, Fei Xia, Mehdi SM Sajjadi, Corey Lynch, Aakanksha Chowdhery, Brian Ichter, Ayzaan Wahid, Jonathan Tompson, Quan Vuong, Tianhe Yu, et al. Palm-e: an embodied multi- modal language model. In Proceedings of the 40th International Conference on Machine Learn- ing, pp. 8469-8488, 2023.
Zhiwen Fan, Jian Zhang, Renjie Li, Junge Zhang, Runjin Chen, Hezhen Hu, Kevin Wang, Huaizhi Qu, Dilin Wang. Zhicheng Yan, Hongyu Xu, Justin Theiss, Tianlong Chen, Jiachen Li. Zhengzhong Tu, Zhangyang Wang, and Rakesh Ranjan. Vlm-3r: Vision-language models aug- mented with iddtuctiionallgnnnn 3d reconstruction, 2025. URL httts:///axxxv.. 2505. 20279.
abs/
Chuanyu Fu, Guanying Chen, al. Maskgaussian: Differentiable mask pruning for efficient 3d gaussian rendering. In CVPR, 2025. URL https:ttp////..... http:s///axee www.wwww..v..e...ooooo.. 6.506.60 02751.
Xingyu Fu, Yushi Hu, Bangzheng Li. Yu Feng, Haoyu Wang, Xudong Lin, Dan Roth, Noah A Smith, Wei-Chiu Ma, and Ranjay Krishna. Blink: Multimodal large language models can see but not perceive. arXiv preprint arXiv:2404 /2390, 2024.

---

Published as a conference paper at ICLR 2026
Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-rl: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12944 2025.
Mary Hegarty. Mental animation: Inferring motion from static displays of mechanical systems. Journal of experimental psychology: learning, memory, and cognition, 18(5):1084, 1992.
Yining Hong, Chunru Lin, Yilun Du, Zhenfang Chen, Joshua B. Tenenbaum, and Chuang Gan. 3d concept learning and reasoning from multi-view images, 2023. URL https: /rrgiv.orggg///// abs/2303 11327.
Wenlong Huang, Chen Wang, Ruohan Zhang, Yunzhu Li, Jiajun Wu, and Li Fei-Fei. Voxposer: Composable 3d value maps for robotic manipulation with language models. arXiv preprint arXiv:2307.05977, 2023.
Wenlong Huang. Chen Wang, Yunzhu Li, Ruohan Zhang, and Li Fei-Fei. Rekep: Spatio- temporal reasoning of relational keypoint constraints for robotic manipulation. arXiv preprint arXiv:2409.01533 2024.
Yuheng Ji, Huajie Tan, Jiayu Shi, Xiaoshuai Hao, Yuan Zhang, Hengyuan Zhang, Pengwei Wang, Mengdi Zhao, Yao Mu, Pengju An, et al. Robobrain: A unified brain model for robotic manipu- lation from abstract to concrete. arXiv preprint 252222250.. 2025.
Dongfu Jiang, Xuan He, Huaye Zeng, Cong Wei, Max W.F. Ku, Qian Liu, and Wenhu Chen. Mantis: Interleaved mulli--imaee instruction tuning. Transactions on Machine Learning Research, 2024, 2024. URL https:/ openrevieww net/ (tt.ormmndsskttdduvaaaaaa
Philip N Johnson-Laire Mental models in cognitive science. Cognitive science, 4(1):71-115, 1980.
Philip Nicholas Johnson-Laird. Mental models: Towards a cognitive science of language, inference, and consciousness. Number 6. Harvard University Press, 1983.
Phillip Lee, Jihyeon Je, Chanho Park, Mikaela Angelina Uy, Leonidas Guibas, and Minhyuk Sung. Perspective-aware reasoning in vision-language models via mental imagery simulation, 2025. URL httpp:://arrtttt wttgabs
Bo Li, Yuanhan Zhang. Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Ziwei Liu, et al. Llava-onevision: Easy visual task transfer. arXiv preprint 033::22008. 2024a.
Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language-image pre- training with frozen image encoders and large language models, 2023. URL https://arxa org/abs/ 2200 12597.
Manling Li, Shiyu Zhao, Qineng Wang, Kangrui Wang. Yu Zhou, Sanjana Srivastava, Cem Gokmen, Tony Lee, Erran Li Li, Ruohan Zhang, et al. Embodied agent interface: Benchmarking 1lms for embodied decision making. Advances in Neural Information Processing Systems, 37:100428- 100534, 2024b.
Jacky Liang, Wenlong Huang, Fei Xia, Peng Xu, Karol Hausman, Brian Ichter, Pete Florence, and Andy Zeng. Code as policies: Language model programs for embodied control. In 2023 IEEE International Conference Robotics and Automation (ICRA), pp.
IEEE, 2023.
Lu Ling, Yichen Sheng, Zhi Tu, Wentian Zhao, Cheng Xin, Kun Wan, Lantao Yu, Qianyu Guo, Zixun Yu, Yawen Lu, Xuanmao Li, Xingpeng Sun, Rohan Ashok, Aniruddha Mukherjee, Hao Kang, Xiangrui Kong, Gang Hua, Tianyi Zhang, Bedrich Benes, and Aniket Bera. DB3d-10kk A large-scale scene dataset for deep learning- based 3d vision, 2023. URL https: /arxiv.. org/ /ag//bb///a 2.12.. 6256.
Deku Liu, Yihan Zhang, Zhe Chen, al. Citygaussianv2: Efficient and geometrically accurate reconstruction for large-scale scenes. In ICLR, 2025a. URL https: //arxiivvv.. .org httpii/ioog/////// 2411.0075
12

---

Published as a conference paper at ICLR 2026
Yuecheng Liu, Dafeng Chi, Shiguang Wu, Zhanguang Zhang, Yaochen Hu, Lingfeng Zhang, Yingxue Zhang. Shuang Wu, Tongtong Cao. Guowei Huang, Helong Huang, Guangjian Tian, Weichao Qiu, Xingyue Quan, Jianye Hao, and Yuzheng Zhuang. Spatialeot: Advancing spatial reasoning through coordinate alignment and chain-of-thought for embodied task planning, 2025b.
URL https: axxiii i v. org/ abs 2501 10074.
Haoyu Lu, Wen Liu, Bo Zhang, Bingxuan Wang, Kai Dong, Bo Liu, Jingxiang Sun, Tongzheng Ren, Zhuoshu Li, Hao Yang. Yaofeng Sun, Chengqi Deng, Hanwei Xu, Zhenda Xie, and Chong Ruan. Deepseek-vl: Towards real-world vision-language understanding, 2024. URL https:
//w.xii..rgg/aaaaaaeeeeea 2200..2433.00000 05525.
Wufei Ma, Haoyu Chen, Guofeng Zhang, Yu-Cheng Chou, Celso M de Melo, and Alan Yuille. 3dsrbench: A comprehensive 3d spatial reasoning benchmark, 2025a. URL https://arxiv.
azg/abs/2412 07825.
Wufei Ma, Yu-Cheng Chou, Qihao Liu, Xingrui Wang, Celso de Melo, Jieneng Chen, Jianwen Xie, and Alan Yuille, Spatialreasoner: Towards explicit and generalizable 3d spatial reasoning, 2025b. URL https:ttp///cmc /als://aaxxiii attp/axxvv..gg//./aa///////o.lat absa 20024.
OpenAL. Hello gpt-4o. Blog, 05 2024. URL http:://opemal.com
Bello-gpt-40e Accessed: November 22, 2024.
OpenAL. GPT-5 System Card. Technical report, OpenAl, aug 2025. URL https://cdd.
openai com/ gpt 5 system carsste--ddd...eee pdf. Accessed: 2025-08-10.
Zhenyu Pan and Han Liu. Metaspatial: Reinforcing 3d spatial reasoning in vims for the metaverse.
arXiv preprint arXiv:2533. /8470, 2025.
Jianing Qi, Jiawei Liu, Hao Tang, and Zhigang Zhu. Beyond semantics: Rediscovering spa- tial awareness in vision-language models, 2025. URL https/////a ars..r///ab..3.....
17349.
Yansong Qu, Jie Wang, et al. Drag your gaussian: Effective drag-based editing with score distillation for 3d gaussian splatting. In SIGGRAPH Asia, 2025. URL https: http:::///axxi..oo/ep p 2501. 18672.
Santhosh Kumar Ramakrishnan, Erik Wijmans, Philipp Kraehenbuehl, and Vladlen Koltun. Does spatial cognition emerge in frontier models?, 2025. URLhttps://arxit org hbgg/abs/2115
06468.
Kyle Sargent, Zizhang Li, Tanmay Shah, Charles Hermaan, Hong-Xing Yu, Yunzhi Zhang, Eric Ryan Chan, Dmitry Lagun, Li Fei-Fei, Deqing Sun, and Jiajun Wu. Zeronvs: Zero-shot novel view synthesis from single real image. arXiv:2310.17994 2023.
Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song. Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300 2024.
Shao-Hua Sun, Minyoung Huh, Yuan-Hong Liao, Ning Zhang, and Joseph J Lim. Multi-view to novel view: Synthesizing novel views with self-learned confidence. In ECCV, 2018.
Yihe Tang, Wenlong Huang, Yingke Wang, Chengshu Li, Roy Yuan, Ruohan Zhang, Jiajun Wu, and Li Fei-Fei. Uad: Unsupervised affordance distillation for generalization in robotic manipulation. arXiv preprini arXiv:2506 09284, 2025a.
Yihong Tang, Ao Qu, Zhaokai Wang, Dingyi Zhuang, Zhaofeng Wu, Wei Ma, Shenhao Wang, Yunhan Zheng, Zhan Zhao, and Jinhua Zhao. Sparkle: Mastering basic spatial capabilities in vision language models elicits generalization to spatial reasoning, 2025b. URL https: / //arxiv. wrw.vorg/aaee abs/ /4110 1...0.. 62.
Gemini Team. Gemini: A family of highly capable multimodal models, 2025. URL https: 11805.
13

---

Published as a conference paper at ICLR 2026
Gemma Team, Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, Ramona Merhej, Sarah Perrin, Tatiana Matejovicova, Alexandre Ramé, Morgane Rivière, et al. Gemma 3 technical report. arXiv preprint 2503:22503.17766, 2025.
Barbara Tversky. Cognitive maps, cognitive collages, and spatial mental models. In European conference on spatial information theory, pp. 14-24. Springer, 1993.
Barbara Tversky. Structures of mental spaces: How people think about space. Environment and behavior, 35(1))66-80, 2003.
Barbara Tversky, Nancy Franklin, Holly A Taylor, and David Bryant. Spatial mental models from descriptions. Journal of the American society for information science, 45()))656-668, 1994.
Jianyuan Wang et al. Vggt: Visual geometry grounded transformer for universal 3d reconstruction. In CVPR, 2025.
Kangrui Wang", Pingyue Zhang* Zihan Wang*. Yaning Gao*, Linjie Li Qineng Wang. Hanyang Chen, Chi Wan, Yiping Lu, Zhengyuan Yang. Lijuan Wang, Ranjay Krishna, Jiajun Wu, Li Fei- Fei, Yejin Choi, and Manling Li. Reinforcing visual state reasoning for multi-turn vlm agents, 2025. URL https::t///m...... /ttpt/gggtttt//u/// com Camm/RACEN-AAA CIIANAAI/VACEEE /VAGEN.
Qineng Wang. Zihao Wang. Ying Su, Hanghang Tong, and Yangqiu Song. Rethinking the bounds of IIm reasoning: Are multi-agent discussions the key? arXiv preprint arXiv:2402. 282.:2222./8 2024.
Wenqi Wang, Reuben Tan, Pengyue Zhu, Jianwei Yang. Zhengyuan Yang, Lijuan Wang, Andrey Kolobov, Jianfeng Gao, and Boqing Gong. Site: towards spatial intelligence thorough evaluation, 2025. URL https: /awxiv.vvvv. org/ ab// /250///b/555.... 05456.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc Le, Denny Zhou, et al. Chain-of thoutht prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824-24837, 2022.
Diankun Wu, Fangfu Liu, Yi-Hsin Hung, and Yueqi Duan. Opattal-mllmm Boosting milm capabilities in visual-based spatial intelligence. arXiv preprint arXiv:2505.22 23747, 2025a.
Junfei Wu, Jian Guan, Kaituo Feng, Qiang Liu, Shu Wu, Liang Wang, Wei Wu, and Tieniu Tan. Re- inforcing spatial reasoning in vision-language models with interwoven thinking and visual draw- ing, 2025b. URL https::/p//. / arxiv org/ abs///b//a/// 2506 09965.
Hongchi Xia, Yang Fu, Sifei Liu, and Xiaolong Wang. Rgbd objects in the wild: scaling real-world 3d object learning from rgb-d videos. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pp.
2024.
Runsen Xu, Weiyao Wang, Hao Tang, Xingyu Chen, Xiaodong Wang, Fu-Jen Chu, Dahua Lin, Matt Feiszli, and Kevin J. Liang. Multi-spatialmllm: Multi-frame spatial understanding with multi- modal large language models, 2025a. URL https: wwwaaaaivvv.... www.wiivvorg...oooooooo.ooo..oo. abs 200./5555500 2at.../255..0000 1......000 17055.
Wenrui Xu, Dalin Lyu, Weihang Wang, Jie Feng, Chen Gao, and Yong Li. Defining and evaluating visual language models' basic spatial abilities: A perspective from psychometrics, 2025b. URL
https: http://xxxvvv////// /abog//bbs//// alos/ / 22022. 11.11118 859.
Jihan Yang, Shusheng Yang. Anjali W. Gupta, Rilyn Han, Li Fei-Fei, and Saining Xie. Thinking in space: How multimodal large language models see, remember, and recall spaces, 2024. URL
https: /tttps:/aaxiv.... wwwpiiiooor//ooooo alss/ 241 14171.
Rui Yang, Hanyang Chen, Junyu Zhang, Mark Zhao, Cheng Qian, Kangrui Wang, Qineng Wang. Teja Venkat Koripella, Marziyeh Movahedi, Manling Li, et al. Embodiedbench: Comprehen- sive benchmarking multi-modal large language models for vision-driven embodied agents. arXiv preprint 1XXX::2502.0956 2025.
Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du Izhak Shafran, Karthik Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models. In International Conference on Learning Representations (ICLR), 2023.
14

---

Published as a conference paper at ICLR 2026
Jiabo Ye, Haiyang Xu, Haowei Liu, Anwen Hu, Ming Yan, Qi Qian, Ji Zhang. Fei Huang, and Jingren Zhou. mplug-ow13: Towards long image-sequence understanding in multi-mon malti-modal large language models. arXiv preprint arXivv2408.048840 2024.
Chun-Hsiao Yeh, Chenyu Wang, Shengbang Tong, Ta- Ying Cheng, Rouyu Wang, Tianzhe Chu, Yuexiang Zhai, Yubei Chen, Shenghua Gao, and Yi Ma. Sceing from another perspective: Eval- uating multi-view understanding in milms. arXiv preprint arXiv:2504. 152800 2025.
Yang You, Yixin Li, Congyue Deng, Yue Wang, and Leonidas Guibas. Multiview equivariance improves 3d correspondence understanding with minimal feature finetuning, 2024. URL https:
/ www.aii...g/aaa//eeeee.
Jirong Zha, Yuxuan Fan, Xiao Yang, Chen Gao, and Xinlei Chen. How enable Ilm with 3d capacity? a survey of spatial reasoning in Ilm, 2025. URL settps://axxiv.ott 2504. 05786.
Weichen Zhan, Zile Zhou, Zhiheng Zheng, Chen Gao, Jinqiang Cui, Yong Li, Xinlei Chen, and Xiao-Ping Zhang. Open3dvqa: benchmark for comprehensive spatial reasoning with multi- modal large language model in open space, 2025. URL https: //tpxivvvvv///////////ee/eaaaav. org/ htrgg/bb///0333 11094.
Jieyu Zhang, Weikai Huang, Zixian Ma, Oscar Michel, Dong He, Tanmay Gupta, Wei-Chiu Ma, Ali Farhadi, Aniruddha Kembhavi, and Ranjay Krishna. Task me anything. In Shirty-Eightt Annual Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2024a.
Juexiao Zhang, Gao Zhu, Sihang Li, Xinhao Liu, Haorui Song, Xinran Tang, and Chen Feng. Mul- tiview scene graph, 2024b. URL https: / arxiv. org/ abs/ 221./240..110 11187.
Peiyuan Zhang, Kaichen Zhang, Bo Li, Guangtao Zeng, Jingkang Yang, Yuanhan Zhang, Ziyue Wang, Haoran Tan, Chunyuan Li, and Ziwei Liu. Long context transfer from language to vision. arXiv preprint wxXhv:2406.16852 2024c.
Wenyu Zhang, Wei En Ng. Lixin Ma, Yuwen Wang, Jungqi Zhao, Allison Koenecke, Boyang Li, and Lu Wang. Sphere: Unveiling spatial blind spots in vision-language models through hierarchical evaluation, 2025. URL https:
: / //axiivvv.. ortoorggaaaao abs 12693.
Yuanhan Zhang, Jinming Wu, Wei Li, Bo Li, Zejun Ma, Ziwei Liu, and Chunyuan Li. Video instruction tuning with synthetic data. arXiv preprint arXiv:2410.02711 2024d.
Yuxuan Zhang, Yifan Yang, Jing Zhang, Yifang Wang, Yijun Zhang, and Ming-Hsuan Yang. Viewcrafter: Taming video diffusion models for high-fidelity novel view synthesis. In ECCV, 2024c.
Baining Zhao, Ziyou Wang. Jianjie Fang, Chen Gao, Fanhang Man, Jinqiang Cui, Xin Wang. Xinlei Chen, Yong Li, and Wenwu Zhu. Embodied-r: Collaborative framework for activat- ing embodied spatial reasoning in foundation models via reinforcement learning, 2025. URL https: /tt:://axxivi org/ awss///b//aaa abss/250 12680.
Duo Zheng, Shijia Huang, Yanyang Li, and Liwei Wang. Learning from videos for 3d world: Enhancing milms with 3d vision geometry priors, 2025. URL. htt.httt///eexxi.orrggg
2505.2468 24625.
Jensen Jinghao Zhou, Hang Gao, Vikram Voleti, Aaryaman Vasishta, Chun-Han Yao, Mark Boss, Philip Torr, Christian Rupprecht, and Varun Jampani. Stable virtual camera: Generative view synthesis with diffusion models. arXiv preprint arXiv:2503. 144899 2025.
Jinguo Zhu, Weiyun Wang, Zhe Chen, Zhaoyang Liu, Shenglong Ye, Lixin Gu, Hao Tian, Yuchen Duan, Weijie Su, Jie Shao, Zhangwei Gao, Erfei Cui, Xuehui Wang. Yue Cao, Yangzhou Liu, Xingguang Wei, Hongjie Zhang. Haomin Wang, Weiye Xu, Hao Li, Jiahao Wang, Nianchen Deng, Songze Li, Yinan He, Tan Jiang, Jiapeng Luo, Yi Wang, Conghui He, Botian Shi, Xingcheng Zhang, Wenqi Shao, Junjun He, Yingtong Xiong, Wenwen Qu, Peng Sun, Penglong Jiao, Han Lv, Lijun Wu, Kaipeng Zhang, Huipeng Deng, Jiaye Ge, Kai Chen, Limin Wang, Min Dou, Lewei Lu, Xizhou Zhu, Tong Lu, Dahua Lin, Yu Qiao, Jifeng Dai, and Wenhai Wang. In- ternv13: Exploring advanced training and test-time recipes for open-source multimodal models, www.wvvoo..o/oooooooooo.oo worrorraabs...... 2000. 0479.
2025. URL https: / / rxiv.
15

---

Published as a conference paper at ICLR 2026
Appendix
Table of Contents
A The Use of Large Language Models
17
B MINDCUBE Benchmark
17
B.1 Details for Data Collection and Annotation
17
B.2 Details of our MINDCUBE Benchmark
19
B.3 Examples
21
Evaluation on MINDCUBE
22
Prompt Templates for Evaluation
................................
22
C.2 Details in text only evaluation
................................
22
C,3 Human Evaluation
24
C.4 Evaluation Setup
25
C.5 Analysis in settings
25
C.6 Failure case analysis
29
Data Structures as Cognitive Scaffolds, Evaluation Metrics, and Input- Input-Output Con-
figurations
30
Data Structures as Cognitive Scaffolds
30
D.2 Evaluation Metries ................................
32
D.3
Prompts for All Input-Output Configurations
35
E
Which Scaffolds Best Guide Spatial Thinking in Unchanged VLMs?
43
E.1
VLM Response Examples for Configurations Section D.3
44
E.2
Additional Graph Metrics for Generated Graphs
45
E.3
Further Analysis on View Interpolation
46
Explicit Reasoning with Visual-of- ofhouhht
F
Can We Teach VLMs to Build and Leverage Spatial Representations? Supervised Fine- Tuning Data Curation
48
F.1
48
F2
Detailed Experimental Sctup
50
E3
VLM Response Examples After SFT for Configurations in Section D.3
54
E4
Detailed Graph Metric Results for SFT Craph-Relate Experiments
57
Which Part VLM Bottleneck for Spatial Understanding?
58
F6
Branching from Raw-QA SFT Checkpoint
59
E7
Experiments Other MILMS
59
E8
Prompt Sensitivity Analyses. Exploring latent spatial representations.
60
F9
60
E10
Hyperparameter Tuning Results
61
Can Reinforcement Learning Further Refine Spatial Thought Processes?
61
G.1 Detailed Experimental Sctup
61
G.2 RL Reward Design Ablation
62
G.3 VLM Response Examples After RL for Configurations in Section D.3
62
16

---

Published as a conference paper at ICLR 2026
A THE USE OF LARGE LANGUAGE MODELS
We used large language models (LLMs), including Google's Gemini 2.5 Pro and OpenAT's GPT- auxiliary tools assist with writing, editing, and conducting the literature review for this manuscript. All content was critically reviewed, fact-checked, and revised by the human authors ensure its scientific validity and originality. The authors are fully responsible for all statements and conclusions presented in this paper. Specifically, we use LLMs for polishing our wording and writing, and we use LLMs to retrieve several related works.
B MINDCUBE BENCHMARK
B.1 DETAILS FOR DATA COLLECTION AND ANNOTATION
Image Collection and Selection. Our MINDCURE benchmark comprises 3, 268 images (2,302 indoor/outdoor images from publicly released dataset and 400 self-collected images), where we im- plement comprehensive image selection methodology encompassing four distinct view dynamics, incorporating various data sources and processing procedures, as shown in Fig.2.
For rotation view dynamics, we implement a three-stage filtering strategy extract meaningful camera trajectories and key frames from ArkitScenes Baruch et al. (2021) dataset.
In the first stage, we analyze the top-down view of camera poses within each scene identify two types of trajectories; linear paths and small rotational arcs. A linear trajectory is characterized by consistently oriented cameras exhibiting significant displacement perpendicular to their viewing di- rection. A rotational are trajectory is identified when three to four camera positions demonstrate approximately 90-degree relative orientation changes while being distributed along an approximate circular arc. The second stage focuses on selecting two critical frames from the previously identified
20
(a) camera pose
(b) camera pose 2
Figure 1: Examples of camera poses in ArkitScenes
translation segments. The selection criteria mandate that: (1) the camera movement direction must be parallel to the object arrangement direction, (2) this movement should be aligned with the hori- zontal axis, (3) the first frame should only capture objects A and B, while the second frame should only capture objects and C, and (4) both frames must be free from motion blur and exhibit clear object visibility.
The third stage processes the rotation segments extract three or four key frames. These frames must satisfy several conditions: (1) the camera positions should appear to originate from station- ary rotating camera, even slight circular movement exists, (2) the camera orientations should align with standard cardinal directions (approximately 90 degrees apart), and (3) each frame should tain no more than three semantically distinct primary objects that occupy over 50% of the frame area relative to the background.
For among view dynamics, image groups are manually selected from DL3DV-10KLiN et al. (2023) and WildRGB-DD DXia et al. (2024) datasets. We employ single-stage selection process to identify
17

---

Published as a conference paper at ICLR 2026
four key frames representing cardinal viewpoints (front, left, right, and back) from 360-degree scene captures. The selection criteria are: (1) camera orientations must align with standard directions, ensuring that the central object, its background objects, and the camera's ' line of sight and parallel or perpendicular to standard scene elements such as tables or walls, (2) we reject sets where three more frames share identical semantic background information, and (3) we discard sets where three or more frames have severely occluded background objects that cannot be reconstructed from information in the other frames.
collinear
For around view dynamics, image groups are manually curated from the DL3DV-1OKLing et al. (2023) dataset and assigned sequential identifiers. The front view (designated as view 1) must provide clear visibility of all relevant information. This view is established as the reference point for subsequent views the sequence.
This structured approach to image selection and processing yields rich dataset that supports sub- sequent model training and testing procedures. The methodology ensures comprehensive coverage of spatial relationships, occlusion states, and view-dependent object characteristics across multiple viewing scenarios.
Top Pose
ArkitSScenes
Filter for Clips
Filter Views
Orthcoraphi: Vieon
Room Walls
Filtering high-quality groups
WildRGB-D
Filter for Among
Consistet Main Dijject
Diverse Semantics Beceground Dlijects
Orthographic Vieao
DL3DV- 10K
Limi Disinct
Diverse Semantics
Clear Spenia Belations
Figure MINDCUBE Bench construction pipeline.
Data Annotation. After collecting and filtering the images, we follow two-phase paradigm for annotation:
We establish systematic image annotation protocol to ensure data consistency and accuracy. The annotation framework encompasses four key dimensions: spatial relationship identi- fication, object grouping rules, semantic orientation determination, and occlusion level assessment. We provide pdf of the annotation interface in the supplementary material.
Regarding spatial relationship identification, annotators are required to identify primary object en- tities within images and determine their spatial relationships. These relationships are primarily categorized into two types: front-back relationships typically involving two primary objects, with priority given to objects directly behind as key entities; and left-right relationships encompassing two to four primary objects, where adjacent objects with front-back relationships can be considered as unified entity.
To enhance annotation efficiency and semantic completeness, this study introduces object grouping rules. Multiple objects be annotated as unified entity when they collectively form clear spatial relationships with other primary objects. Each object may include attribute descriptors (c.g., color, material) enhance semantic expression. Combined object entities must maintain distinct spatial relationships with other primary objects.
For objects with definitive semantic fronts, the following information must be recorded: the object's inherent semantic front, the object's 's orientation relative to the current viewpoint (aligned, reversed, leftward, rightward, etc.), and the object's actual projected direction within the scene.
18

---

Published as a conference paper at ICLR 2026
Occlusion levels evaluated using four-tier classification system: complete occlusion where the object entirely invisible from the current viewpoint; major occlusion where primary object features are difficult to identify; minor occlusion where primary object features remain identifiable; and no occlusion where the object is fully visible. For cases of complete occlusion, the annotation system provides multi-view scene images, ensuring object visibility in least viewpoint to support subsequent cross-view question-answerigg system training.
This annotation protocol provides structured semantic foundation for subsequent automated question-answer pair generation while ensuring data quality and consistency. Through this stan- dardized annotation process, we effectively capture key information including spatial relationships, compositional features, semantic orientations. and occlusion states of objects within scenes. Examples for automatic QA generation pipeline. Our automatic QA generation pipeline gener-
Current Frame
Visual Patterns
no linear
"What-if" Dynamics
Relation Query
Perspective Taking Level
Questions:
How did you likely from the first view to second view?
meanwhile
(Agent, Agent)
perspective
2. What is behind of you?
From chair's view, could you quilt-covered sofa?
2
Main Objects
From the TV's view, what is on your left-front side?
(Agent, Object)
other perspective11
chair with
What the left the chair from your ego-centric view?
basket
2.
(Object, Object)
perspective-2
6. What is on the left the chair from TV's view?
Figure 3: Example of different question-related label combinations to generate QA pairs.
ates different types of questions using combinations of labels. Each question's label combination is encoded in its ID (e.g.. grroonnggrop001110 _q'_1_1"), while the original object and label information preserved in the meta_info field to track the context of question generation.
B.2 DETAILS OF OUR MINDCUBE BENCHMARK
B.2.1 THREE KINDS OF INVISIBILITY SETTINGS
Rotation, In this setting, our camera remains stationary while rotating in place, capturing 2 to orthogonal views. In each view, central object remains visible in the foreground, while all views maintain equal importance in the spatial representation.
We evaluate models' understanding of spatial invisibility by asking questions such 'When positioned at particular viewpoint, what should he to your left or right (given that each view only reveals what's directly ahcad)?' "After rotating quarter half turn, what objects would be in front of you, to your left, behind you, or to your right?' We expect models to construct a comprehensive spatial understanding by leveraging the sequential nature of the views and consistent spatial cues across images (such as lighting direction), thereby demonstrating their ability reason about the complete environment despite only having access to partial visual information from each viewpoint.
Around. In this setting, we leverage occlusion phenomena to force MILLMs beyond simple 2D spatial recognition. When viewing objects from different angles, some objects become partially or fully hidden, requiring models to:
Maintain object permanence despite partial visibility
19

---

Published as a conference paper at ICLR 2026
Transform lateral relationships (left-right) from frontal views into depth relationships (front-bach) for side views
Integrate spatial information across multiple viewpoints form a coherent 3D understand- ing
This approach prevents models from relying solely on direct visual cues and instead necessitates true 3D spatial reasoning by combining information from multiple perspectives.
Among. In this setting, the camera rotates around central object, positioned between this central object and several surrounding objects. Four orthogonal views are captured, with each view showing the central object positioned in front of one of the surrounding objects.
This setup creates interesting visibility constraints across different perspectives. For instance, a surrounding object visible in one view may be invisible in another view because of the constraints imposed by the camera's 's field of view. Through establishing consistency relationships between these views, we can infer the relative positions of objects not directly visible from certain perspectives. When an object is not visible from a particular viewpoint, consistency and spatial reasoning can determine its position relative to the central object.
All views hold equal status in this framework, allowing for bidirectional establishment of invisi- bility relationships This creates a coherent spatial reasoning system where information from each perspective contributes to a complete understanding of the three-dimensional arrangement, even when direct visual confirmation is unavailable from certain angles.
B.2.2 LABEL TAXONOMY
We use image related labels for better analysis and question related labels for automatic QA gener- ation with different label combinations.
Visual Patterns. In our taxonomy of spatial configurations, classify visual patternss into distinct categories based on their geometric relationships. Linear arrangements refer to configurations where objects are positioned along a single axis, forming collinear pattern. Non-linear arrangements, conversely, are characterized by objects positioned such that the connecting lines between adjacent pairs form 90-degree angles, creating rectilinear patterns. This binary classification serves as a fundamental attribute in our spatial relationship labeling scheme, enabling precise description and analysis of scene compositions across various domains.
"What if" Dynamics. "What if" Dynamics refers to the model's capability to comprehend and reason about dynamic perspective changes occurring within images or posed questions. We concep- tualize viewpoint transitions as combinations of translation and rotation operations, resulting in four distinct categories:
Pure Translation: Cases where the viewpoint undergoes only translational movement with- out rotational change.
Scenarios involving rotational transformation of the viewpoint while main- taining its positional coordinates.
Pure Rotation:
Simultaneous -oanslaatoo-Roattionn/ (Meanwhile): Instances where both translational and ro- tational operations occur concurrently.
Sequential RereiionnRRotaiinnSSeeeee Cases where translation and rotation occur in sequence rather than simultancously. Notably, in our dataset, this category is uniquely represented through textual descriptions the questions rather than through explicit visual transformations.
The first three categories "What if" dynamics are visually demonstrated through changes in view representation, while the sequential category requires models interpret text-based descriptions of perspective changes. This taxonomy provides a systematic framework for evaluating spatial reason- ing capabilities across diverse viewpoint transformation scenarios.
Relation Query. We define three distinct categories of relation queries that capture the fundamental nature of spatial reasoning tasks:
20

---

Published as a conference paper at ICLR 2026
Agent-Agent: This pattern involves self-referential spatial positioning. where the observer must evaluate and potentially adjust their own position in space. It requires egocentric spatial reasoning and self-awareness of one's mess location relative environmental constraints.
agent-Object: This pattern focuses determining the orientation of observed object relative to the observer's position. Unlike the P-P pattern, the emphasis here is on object perception rather than self-positioning. requiring the observer to make judgments about external entities while maintaining awareness of their own reference frame.
This pattern involves reasoning about the spatial relationship between two discrete objects in the environment, independent of the observer's position. This allocen- tric spatial reasoning requires understanding relative positioning, distance, and orientation between entities without necessarily using oneself reference point.
Object-Object:
These categorizations provide a structured approach to analyzing the cogmitive demands of different spatial reasoning tasks and inform both the design of spatial question answering systems and the evaluation of human spatial cognition abilities.
Perspective Taking, We propose label called "Perspective Taking" that categorizes the complexity of viewpoint projection. This label distinguishes between three increasingly sophisticated levels of perspective reasoning:
Reasoning based on the current camera view or the observer's 's own view- point. This represents the baseline where perspective shift is required.
Self Perspective:
Other's Perspective Taking-1: The ability to determine visibility relationships from another agent's viewpoint. This involves understanding what objects are visible or occluded from different viewpoint (e.g., determining whether specific object is within the field of view of another camera). The another agent's viewpoint is usually determined by an object with clear orientation in the image.
Other's Perspective Taking-2: The ability understand how spatial relationships transform when viewed from another agent's perspective. This more advanced capability requires mental rotation and spatial transformation reason about relative positions (c.g.. deter- mining whether, from another viewpoint, object X appears to be positioned behind object Y).
This classification aligns with developmental psychology research on perspective-taking abilities, where Level-1 perspective taking typically develops earlier than the more cognitively demanding Level-2 perspective taking.
We provide performance across different categories and labels in Table1 and 2. Upon detailed analysis of model performance across various capabilities, certain trends emerge. The O-O (Object- Object) task within Relation Pattern also demonstrates generally lower scores across the board, suggesting it is less tractable problem for current models. Notably. InternV2--8B struggles with the sequence task, exhibiting the lowest score among all evaluated models in that category.
Regarding model stability, Mantis(SigLip) demonstrates robust performance in both Object Ar- rangement and Relation Pattern sections, indicating a consistent capability in these spatial reasoning tasks. Similarly, Siven2.5-VI-7B--nnssee Dynamics. In contrast, Intern shows a broader instability, with consistently lower overall scores and considerable performance fluctuations across different sub-categories, highlighting areas for further improvement its generalizability and robustness.
maintains relatively stable performance within Viewpoint
B.3 EXAMPLES
We show examples in Figure5, 6 and 4.
21

---

Published as a conference paper at ICLR 2026
Table 1: Performance of VLMs on MINDCUBE across categories.
Model
Overall
Object Arrangement
Perspective Taking
Linear
Perp.
Self
Level1
Level2
LLaVA-Video-7B-Qwennl
41.96
30.12
43.11
42.19
60.76
33.80
Mantis(SigLip)
41.04
50.99
40.08
41.20
54.43
35.41
GPT-4o
38.81
29.16
39.75
39.07
46.20
31.86
Qwen2.5-VL-3B-
33.21
30.34
33.49
32.96
46.84
36.28
Lo
29.46
24.88
29.91
28 28.81
51.90
39.83
Owen2.55VVL-7B-mmttee
29.26
21.35
30.02
28.77
46.84
36.81
deepscek-v12-small
47.62
26.91
49.63
48.32
56.33
31.11
Rohobrain
37.38
29.53
38.14
37.56
55.06
30.57
Claude-sonnet-4
44.75
47.62
44.48
45.32
49.38
31.74
Space-Mantis
22.82
29.32
22.19
22.15
45.57
33.48
VL2-8B
18.68
13.11
19.22
17.89
64.56
27.99
twen
33.28
26.32
33.95
33.06
46.84
35.63
LLaVA-Omevisioon-78
47.43
44.09
47.75
48.04
51.27
33.48
Spatial-MLLM
32.06
20.92
33.13
31.79
46.84
35.20
mPLUG-OWI3-7B
44.85
26.91
46.59
45.15
60.13
35.74
Table 2: Performance of VLMs on MINDCUBE across categories.(Part 2)
Model
Relation Pattern
Viewpoint Dynamies
A-A
A-O
0-0
Rotation
Meanwhile
Sequence
ILaVA--iido-7B--Qwen2
36.22
57.61
26.67
35.71
30.12
73.45
Mantix(Sigl.ip)
23.78
64.16
25.24
37.65
24.99
82.74
GPT-4o
49.30
48.38
16.70
32.65
31.09
59.73
Qwen2.5-VL-3B-Innte
37.85
37.51
20.65
37.37
27.88
46.05
LongVA-7B
19.72
35.49
25.58
35.89
24.67
40.50
Bren225-vI--7B-Insttee
31.41
34.67
15.63
38.76
22.87
43.76
deepscek-v12-small
43.98
68.27
25.33
37.00
32.97
87.13
Robobrain
30.94
49.18
27.37
35.80
28.79
59.66
Claude-sonnet-4
41.78
67.25
15.85
48.42
34.76
69.53
Space-Mantis
28.18
17.03
20.89
37.65
24.98
14.46
InternVL22-B
15.67
12.47
24.58
36.45
21.78
7.36
Space-Qwen
31.59
38.14
26.13
38.02
28.51
44.58
POAVA--Oevision-75 7B
42.28
65.87
29.79
36.45
33.80
84.38
Spatial-Mi MLLM
27.72
37.75
25.80
38.39
26.84
44.19
mPLUG-OW/33778
47.80
62.29
18.83
37.84
31.02
81.55
C EVALUATION ON MINDCUBE
C.1 PROMPT TEMPLATES FOR EVALUATION
Evaluation Prompt Prefix
Based on these images, answer the question based this rule: You only need provide *ONE* correct answer selecting from the options listed below. For example, if you think the correct answer 'A. above' from A. above B. under front D. behind.', your response should only be 'A. above'. The Question is:
C.2 DETAILS IN TEXT ONLY EVALUATION
In the text-only evaluation, replace the original image input with corresponding textual descrip- tions and assess the performance models based on these descriptions. The purpose of this eval- uation is to highlight how much information may be lost distorted when the visual input is sub-
22

---

Published as a conference paper at ICLR 2026
Example of Among setting
meanwhile
agent agent
salf perspective
non linear
Based on view1 and view2 showing same scene, which direction did you move from the first view secand view?
Question:
Options: Forward-left Forward-right
View1
View4
System Prompt:
Based these four images (image 2. 3, and showing the red ball from different viewpoints (front, left, back, and right), with each camera aligned with room walls and partially capturing the surroundings
mearwhiile
agent-object
self perspective
non-linear
mearwwfile
object-abject
self perspective
non-linear
If you are standing at the viewpoint presented in image 1, then you turn left and move forward, will you get closer to the light-colored sofa? Options:
Question:
From the viewpoint presented in image 1, what is to the left of the red ball?
Question:
Options: A. white-red cabinet B. light-colored sofa C. dark brown sofa school bag and cabinet
Yes No
Question:
standing at the viewpoint presented in image what behind you?
From the viewpoint presented in image 1, what is the right of the red ball?
Question:
Options: A. white-red cabinet B. light-colored sofa C. dark Optilont:::: A. white-red cabinet light-colored sofa C. dark
brown sofa D. school bag and TV cabinet
brown sofa D. school bag and TV cabinet
meanwhile
object abject
other parspective
non linear
Question: If you are positioned where the light-colored
If you are positioned where the dark brown sofa and facing the same direction, what would be to the right of the red ball from this view?
Question:
sofa is and facing the same direction, what would be to the left of the red ball from this view?
Options: dark brown sofa school bag and cabinet white-red cabinet
Options: A. school bag and TV cabinet white-red cabinet light-colored sofa
Figure 4: Example of among setting.
stituted with text-based representations, and to demonstrate the crucial role of visual data in the models' performance.
We used two types of captions: brief and dense. The brief captions provide a concise overview of the image, while the dense captions offer a more detailed description with focus on the spatial relationships between objects. Additionally, the models are evaluated using textual descriptions (text-only evaluation) based on these captions, with no access to the actual images.
Prompt for Brief Captioning Describe this image briefly.
Prompt for Dense Captioning Describe this image in detail, specifially focusing on the spatial relationship between objects.
Text-only evaluation Prompt Prefix
You need gather information about each image based the descriptions I provide below, and answer the given questions using those textual descriptions, without directly viewing the images.
Image 1: <Caption 1>
Image N: <Caption N>
23

---

Published as a conference paper at ICLR 2026
Example of Around selling
meanswhile
agent agent
salf perspactive
linear
Question:
Based on view1 and view2 showing the same scene, please determine which direction you move? Left-front B. Right front.
View (Front)
View2(Left)
(Righ()
Options: A. Forward-left Forward-right
System Prompt:
Given 3 orthogonal perspectives of a scene, they are the front view, left view and right view.
meamrhhlee
object-object
self perspective
linear
meanwhile obyect-abject
self perspective
linear
In the second image, what is the nearest object the nearest object behind of the black waste bin? Options: A. green waste bin B. blue waste C. shrubbery
Question:
If you are at the view of the secand image now, then you turn right and go straight, is the green waste bin be closer to you?
Question:
Options: A, Yes No
Question: In the third image, what is the nearest object behind of the blue waste bin.
Question: If you at the view the third image now, then you turn left and go straight, the green waste bin be closer to you?
Options: A. green waste bin blue waste bin C. shrubbery
Options: A. Yes B. No
Figure 5: Example-1 of around setting.
As shown in the Table all three models exhibit noticeable performance decline when replacing the original image input with its corresponding text-based description. Specifically, the brief cap- tions cause the most significant performance drop. For instance, RoboBrain-B experiences 7.83% decrease with the brief captions, and LLaVA-One Vision-7B drops by 12.91% % in the same condition. Even when using dense captions, which offer more detail, there still a performance reduction, although the decrease is slightly less pronounced compared to brief captions. In conclusion, while textual descriptions can convey some information, they fail to capture the richness and intricacies of visual data, leading marked reduction in performance across all models.
Table Text-only (T) evaluation vs. original evaluation with image inputs (I). The results highlight significant performance drop when the original image input is replaced with the corresponding text-based caption, particularly with the brief captions. In all cases, model performance decreases notably, underscoring that our benchmark is vision-centric.
Model
Brief (T)
Dense (T)
Original (l)
RoboBrain-SB
33.92% U7.83%
35.58% J6.17%
41.75%
LLaVA-OneVision->
34.17% 112.91%
35.92%
111.16%
47.08%
Qwen2.5-VL-7B Instrruct
27.00% U5.33%
28.75%
13.58%
32.33%
C.3 HUMAN EVALUATION
We use our Tiny Benchmark encompassing all task categories for evaluation by human annota- tors, each of whom independently answers every question. Here is the results4.
Table 4: Comparison Human and GPT-4 Performance (%)
Model/Anmotator GPT4-0
Human-max
Human-min
Human-avg
Accuracy
36.54
94.77
94.20
94.55
This observation demonstrates the disparity in spatial reasoning capabilities between humans and state-of-the- multimodal large language models, where humans exhibit superior performance in solving spatial problems that remain challenging for advanced AI systems.
24

---

Published as a conference paper at ICLR 2026
Example Around setting
View1
View2
View3
meanwhhile
agent-agent
perspective
lineer
Question:
Based on view1 and view2 showing the same scene, please determine which direction did you move? Left-front B. Right-front,
Options: A. Forward-left Forward-right
System Prompt1:
Given orthogonal perspectives of scene, they are the front view, left view and right view.
meamwhile
object-on object
self perspective
linear
meanwhilee
object object
self perspective
linear
Question:
In the second image, what is the nearest
If you are at the view of the second image now, then you turn right and go straight, is the sanitation cart be closer to you? Options: A. Yes B. No (View 123 or View 145 Used)
Question:
object the nearest object behind of the double trash can?
Options: A. sanitation cart B. bench C. battery powered vehicle D. car (View 123 or View 145 Used)
Question: lin the third image, what the nearest object
Question: If you
the view of the third image now, then you turn left and go straight, the double trash
behind of the sanitation cart?
Options: A. double trash can B. bench C. battery powered closer to you?
vehicle D. car (View 123 View 145 Used)
Options: A. Yes B. No (View 123 or View 145 Used)
System Prompt2: Given 3 orthogonal perspectives of a scene, they are the behind view, left view and right view.
meanwhile
object-object
sell perspective
linear
meanwhile object-object
sell perspective
lineer
the second image, what is the nearest object the nearest object behind the double trash can? Options:
Question:
Question:
If you are at the view the second image now, then you turn right and go straight, is the sanitation cart be closer to you?
sanitation cart bench C. battery powered View 645 Used)
vehicle D car (View
Options: A. Yes B. No (View 623 or View 645 Used)
Question: If you are at the view the third image now, then you turn left and go straight, the double trash Options: double trash can bench battery powered closer to you? Options: A. B. (View 623 or View 645 Used)
Question:
I= the third image, what is the nearest object
behind of the sanitation cart?
vehicle D. (View 623 View 645 Used)
Figure 6: Example-2 of around setting.
C.4 EVALUATION SETUP
To comprehensively evaluate model performance, we conducted experiments on a diverse suite of models. This suite includes models with native multi-image reasoning capabilities (e-g., LAaVA- Onevision (Li et al., 2024a), LLaVA- (Zhang et al., 2024d), mPLUG-OW13 (Ye et al., 2024), InternVL2.5 (Chen et al., 2024b), Qwen VL2.5 (Bai et al., 2025), LongVA (Zhang et al., 2024c), DeepSeck-VL2 (Lu et al., 2024)), Gemma3 Team et al. (2025), models fine-tuned interleaved image-text data (e.g., Mantis (Jiang et al., 2024)), leading proprietary APIs (e.g.. GPT-5, Claude- 4-Sonnet), and models specifically fine-tuned for spatial reasoning tasks (e.g., RoboBrain (Ji et al., 2025), Space-Mantis (Chen et al., 2024a), Space-Qwel Qwen (Chen et al., 2024a), and Spatial-MILLM Wu al. (2025a)).
C.5 ANALYSIS IN SETTINGS
C.5.1 AROUND
First, we examine the relationship between occlusion degree and response accuracy across four visibility levels (fully visible, mostly visible, mostly occluded, fully occluded) to determine whether performance degrades proportionally with increasing occlusion. Second, we investigate the impact of camera height variation within the same lateral viewpoint, different vertical perspectives yield distinct occlusion patterns that may challenge the model's ability to maintain spatial coher- ence. These paradigms evaluate whether models perform consistently when transferring spatial
25

---

Published as a conference paper at ICLR 2026
relationships across viewpoints, particularly in scenarios with significant object size discrepancies where smaller objects may be completely occluded from one angle but visible from another. This multifaceted analysis approach enables more nuanced understanding of MLLMs' genuine 3D spatial reasoning capabilities beyond simple pattern recognition of 2D visual cues. We mainly evaluated GPT-4o and Qwen2.5-VI.
Occlusion Degree Analysis.
Our analysis reveals notable correlation between occlusion
Abstract spatial representation
Around
Can
Column
Can
Front View
View
Complex
Arrange
-ment
Condiment cart
Right
Back view
From the viewpoint presented in image 2, is there an blue trash can behind the brick- patterned column? A. Yes B. No. Question2: From the viewpoint presented image 3, is there an orange trash can behind the brick- patterned column? A. Yes No.
Question1:
Answer1: In image 2, the viewpoint is from the left side of the brick-patterned column, Image 4 shows the back view of the column, where blue trash can is visible. Therefore, from the viewpoint in image 2, the blue trash can is indeed behind the column. Therefore, my Answer2:In image
Yes.
viewpoint shows blue can next to brick-patterned column. Image which shows the back view, reveals an orange trash can next to the column. Therefore, from the viewpoint in image 3, the orange trash can is not visible because it is on the opposite side of the column. Therefore, answer is B. No.
GPT-4o Response
Figure 7: Example of complex spatial arrangement in around setting.
degree and model performance. Accuracy rates declined progressively with increasing occlusion, with an average decrease 50.7% between fully visible and fully occluded conditions (p <0.01). Interestingly, the performance degradation was non-linear, with precipitous drop occurring between the mostly visible and mostly occluded categories (28.7% decrease), suggesting a potential threshold effect in the models' spatial reasoning capabilities. Error analysis in Figure8 further revealed that models frequently defaulted to proximity-base based guessing when confronted with heavily occluded objects, rather than leveraging cross-view information to reason about hidden spatial relationships.
Camera Height Impact Analysis. Varying camera heights significantly affected model per- formance through different occlusion patterns. High-angle perspectives yielded 24.8% higher accuracy than eye-level views by revealing tops of partially occluded objects and providing better scene context. This advantage was most pronounced in dense arrangements where top-down angles exposed spatial gaps between objects otherwise invisible from eye-level. Models clearly benefited from the holistic understanding afforded by elevated viewpoints, where global spatial relationships became more apparent. In contrast, eye-level perspectives with more severe occlusions led to poorer spatial reasoning, suggesting limited ability to mentally reconstruct hidden scene elements from partial visual information.
The integration of findings across all three dimensions indicates that current MLLMs pos- sess partial but inconsistent 3D spatial reasoning capabilities. The models' performance appears heavily influenced by the visibility of key reference points across multiple viewpoints, suggesting
26

---

Published as a conference paper at ICLR 2026
Around
View
Left View
Different
Height
Right
Right
Right
View
View 2
View
Question: Based these three images (image 1, 2, and 3) showing the same scene from different viewpoints (front, left, and right): From the viewpoint presented in image is there a table behind the trash can? Yes No."
Answer1: In image 3, we see the trash can positioned against the wall with visible table behind The angle the shot does not show any part of table behind the trash can.
Answer1: - In image 1, we see a table and chairs adjacent the trash
wall.
In image 2, the view from the the table extends from the wall, and the trash closely beside
Therefore, my answer is B. No.
In image 3, the view from the right toward the trash can shows the area where the table located.
Answer2: In image 3, we see trash can placed against wall with partial view of the floor and part of another object that appears to be table. The angle suggests that there were table behind can, it would be visible or at least partially seen. However, the image only shows the trash can and part of the floor, without any indication of table being present behind it.
From this viewpoint, while there's direct view behind the trash can, based on previous images that illustrate the spatial arrangement, table should visible next to the trash can, nat directly behind it. Therefore, my answer is
trash
Image shows trash can the right of table with two chairs. Image reinforces this by showing the table and chairs aligned along the wall, with the trash can just beyond the table. Image shows the view from the side of the trash where the table not visible directly behind it, suggesting is adjacent rather than directly behind from this viewpoint. Therefore, my answer No
Answer2:
Therefore, my answer is B. No.
Answer3: image we the trash can positioned in front a wall with a table visible behind it. The perspective shows that the table is indeed located behind the trash can when viewed from this angle. Therefore, my answer is A, Yes.
Answer3: In image 1, a table is visible next trash can. Image 2 confirms the same table and trash can placement from a different angle. Image 3 shows the trash can in the foreground, and table is visible directly behind it. Therefore, my answer is
Qwen-7B-Instruct Response
GPT-4o Response
Figure 8: Example of different camera heights' impact in around setting.
27

---

Published as a conference paper at ICLR 2026
LLVA-Onnviiilon
LavA-Onevislon
Spaceowen
intern..s 5-VL
SpaceOwen
Intern 5-VL
GPT-40
GPT-40
120
CC (Consistent Changeai
(Whrogg Vayy)
Changes!
100
Proportion (%)
80
60
40
20
Attribute Invariance
Quantity Sensitivity
Figure 9: Paired question-answen. inconsistency in two tests. We report the proportions of IC, CC and WW. Notably, SpaceQwen has a highest inconsistency(around 40%). GPT-4o and LLaVA- Onevision exhibit more balanced performance.
reliance on visual correspondence matching rather than true 3D mental modeling. The observed asymmetries in spatial transformation paradigms further support this hypothesis, as robust reasoning system would demonstrate consistent performance regardless of the specific transformation required.
C.5.2 AMONG
While the correctness of one single question indicates how often model answers an isolated ques- tion correctly, it does not capture whether the model remains consistent when presented with objects with different visual attributes and different number of central objects while retaining the original peripheral scenes.
To investigate this, also propose different tests:
Attribute Invariance Test. We modify only the visual attributes (e.g., color, category) of the central object while keeping the spatial configuration of all objects unchanged, as shown in Figure10. robust spatial reasoning system should maintain consistent answers, spatial relationships remain invariant despite superficial attribute changes.
Quantity Sensitivity Test. We increase the number of central objects (c.g., from one to three) while retaining the original peripheral objects, as shown in Figure I. This modification is hypothesized enhance reasoning performance, as additional central objects provide more reference points for establishing cross-view correspondences and consistency.
We also propose to look into the proportions of paired questions in tests where the answers are consistent with one another. First, we classify each paired instance into three scenarios: 1) CC(Both Correct) when the model answers both the primary and paired question correctly, 2) WW (Both Wrong) when it fails both versions, and 3) IC (Inconsistent) when the model answers version correctly but fails the other.
As shown in Figure 9, report the proportions of IC (in consistent) outcomes across 4 MLLMs in two tests - two open-source (Intern2.5-VL, and closed-source GPT-4o. We have several observations: 1) SpaceQwen exhibits notably high inconsistency score (around 40%) on both tasks, 2) LLaVA-Onevisia remain fairly balanced inconsistency and high performance across tests, while Intern VI. vary significantly across tests.
a spatial model (SpaceQwen)
28

---

Published as a conference paper at ICLR 2026
Figure 10: Examples Attribute Invariance Test.
CB
Figure 11: Examples Quantity Sensitivity Test.
Our systematic evaluation demonstrates MLLMs can achieve attribute-invariant spatial reasoning but struggle to utilize additional reference objects effectively. This highlights the need for: (1) enhanced geometric reasoning architectures, and (2) comprehensive benchmarks evaluating both attribute invariance and quantity sensitivity in 3D spatial understanding.
C.6 FAILURE CASE ANALYSIS
The observed pattern of errors indicates that models primarily rely on local relationship match- ing rather than inferring global spatial configurations, which represents a critical gap compared to human-like spatial reasoning abilities. Future architectural improvements should therefore focus enhancing transitive spatial inference mechanisms and view-invariant scene representation to sup- port more robust reasoning across multiple perspectives.
29

---

Published as a conference paper at ICLR 2026
Question: Based on these four different viewpoints (front, left, back, and right,, you positioned the first viewpoint, then you turn right and move wor you get closer to the window?
Based on these four different viewpoints (front, left, back, andright), if you are positioned third viewpoint, what is behind you?
Question:
Options: A. No B.Yes
<Reasoning>:
Options: A. Washing machine White wood rack C.White gray curtains White
1. The first image shows a view with a vent.likely the front view. second image has window iisibleiiddccting view opposite the first.
window
The third image features large cardboard, potentially a covered window one
<Reasoning>:
The third viewpoint shows the right side of the shoe, with the washing machine visible background.
4. The fourth image shows plain walgsggggt the view
opposite the image.
Based these, the four views are likely structured roughly rectangular layout. Starting at the first view (front with vent). tumnnng right would bring you wall (fourth image), away from the window (second image). Therefore, my answer is A, No.
Therefore, you are positioned the third viewpoint,the washing machine behind you. Therefore, my answer Washing machine.
Figure 12: Failure case analysis. We show GPT4-o's reasoning process. In case 1, the model is unable to establish the spatial location corresponding to each view; In case 2, the model confuses the subject of the "behind" relationship.
D DATA STRUCTURES AS COGNITIVE SCAFFOLDS, EVALUATION METRICS, AND INPUT-OUTPUT CONFIGURATIONS
In this section, we provide detailed descriptions of the three data structures employed as cognitive scaffolds to approximate spatial mental models in VLMs, followed by formal definitions of the evaluation metrics employed across all experiments. Furthermore, we show the prompts for all the input-output configurations that were used across the following experiments.
D.1
DATA STRUCTURES AS COGNITIVE SCAFFOLDS
The human ability to navigate and reason about space, especially with incomplete information, is largely attributed to the formation of internal spatial mental models. These models, extensively studied cognitive science, are not necessarily veridical, metric-perfect replicas of the environment. Instead, they often schematic and even distorted, yet functionally effective representations. These models can be especially useful for understanding the environment spatial layouts for agentic set- tings Yao et al. (2023); Wang et al. (2024), such embodied scenarios Liang al. (2023); Driess et al. (2023); Huang et al. (2023; 2024); Li et al. (2024b); Yang et al. (2025); Tang et al. (2025a). Pioneering work by Barbara Tversky suggests that these internal constructs are more akin to "cog- nitive collages" - flexible assemblies of spatial information gleaned from various perspectives and experiences, rather than rigid, map-like blueprints Tversky (1993). These "cognitive collages" al- low for the integration of fragmented observations and support reasoning across unseen perspectives. Johnson-Lan Laird Johnson- Laird (1983) posits that mental models, including those for space, serve as "strucfural analogs of the world," enabling individuals to simulate and infer spatial relationships, such as determining the relative positions of objects from sequential descriptions (e.g., "A is to the left of B; B is in front of C"). Research by Tversky Tversky et al. (1994) has also demonstrated that individuals can construct rich, multi-dimensional mental representations even from linear, descrip- tive texts, and subsequently query these models from various psychological viewpoints.
Inspired by these cognitive theories, we explore three distinct data structures designed to act as cognitive scaffolds for VLMs. When VLMs presented with limited visual input, these structures aim to approximate different facets of human spatial mental modeling: dynamic updating, integrated spatial layout representation, and inferential reasoning.
30

---

Published as a conference paper at ICLR 2026
D.1.1 VIEW INTERPOLATION FOR DYNAMIC UPDATING
Human spatial mental models are not static; they continuously updated with new sensory infor- mation and through mental simulation, such as imagining change in viewpoint. To approximate this dynamic updating and mental animation capability Hegarty (1992), we employ view interpo- lation. This technique aims to bridge perceptual gaps between discrete, sparsely sampled views by generating intermediate visual frames.
In our experiments, view interpolation is implemented by inserting synthetic frames between consecutive views provided to the model. For instance, "I interpolated frame") is speci- fied, one new frame is generated and inserted between Vn+1 (e.g., between View and View 2). Similarly, "2 interpolated frames" would mean two syn- thetic frames are inserted in sequence between V, and Va+1- For the interpolated frames, either define a heuristic function to choose from the original datasets Baruch et al. (2021); Xia et al. (2024) where we sampled our data, mediate frames for those image groups without. This approach intended to provide a smoother perceptual experience, potentially aiding the VLM in tracking object relations and maintaining spa- tial consistency across viewpoint shifts. (Refer to Figure in the main paper for conceptual illustration)
Our Setting:
initial view Vn and the subsequent view
Stable Virtual Camera Zhou al. (2025) generate inter-
D.1.2 COGNITIVE MAPS FOR INTEGRATED SPATIAL LAYOUTS
core aspect of spatial cognition is the ability to form an allocentric (world-centered) or survey- like understanding of an environment, capturing the relative locations of objects. Tversky Tversky (1993; 2003) highlights that such representations often involve different frames of reference and hierarchical structures. Cognitive maps in our context are 2D schematic representations that attempt to embody this integrated spatial layout.
Our Setting: We investigate two variants of cognitive maps, both represented structured data (e.g.. JSON-like objects), to capture the spatial layout:
We provide 2D grid map of the scene that is related to the question be answered.
The map uses a 10x 10 grid, where [O, 0] is the top-left corner and [9, 9] is the bottom- right comer (i.e., bird's-eye view).
Directions defined as follows:
up towards the top of the grid (decreasing y-value)
right = towards the right of the grid (increasing x-value)
down = towards the bottom of the grid (increasing y-value)
left = towards the left of the grid (decreasing x-value)
inner into the 2D map (perpendicular the grid, pointing away from you) outer = out of the 2D map (perpendicular to the grid, pointing toward you) The map contains:
objects a list of all important items in the scene with their pos i in facing - indicating the direction an object is oriented (when applicable)
views representing different camera viewpoints in the scene
Augmented Cognitive Map: This version explicitly integrates the observer's perspective by encoding the positions and orientation
(facing directions) of the camera viewpoints within the map, alongside the objects and their locations. For instance, as depicted in our data examples (refer to Figure 3, Cognitive Map Augmented panel), augmented map might define a list of objets with their name and position (e.g., "Tissue box": "position":
[5, 5] }), and separate list of views detailing each camera's name (e.g., "View 1 1), posstioon (e.g., [3, 5]), and facing direction (e.g., "up").
Plain Cognitive Map (Object Only): This is more simplified, object-centric representation. It primarily focuses on the spatial locat ions of objects and, for some objects, their intrinsic ori arientatiin itly embedding camera view information within its structure. For example (refer to Figure 3, Cognitive Map Plain panel), plain map might list objects like "Potted plant" with
(facing direction) from top-down survey perspective, without explic-
31

---

Published as a conference paper at ICLR 2026
its position (e.g.. [5, 61) and facing direction (e.g.. "down") "), and another object like "Sofa" " with only its position (e.g., [4, 5]). This type of map still allows for reasoning about object-to-object relationships and, where specified, object orientations, but abstracts away the explicit camera viewpoints that generated the scene understanding.
In both map types, coordinates represent positions on 2D grid, and facing directions can be categorical (e.g.. "up". "down", "left". "right", "outer", " "inner"). These structures aim to provide the VLM with an explicit, albeit potentially imperfect, schematic the environment that it can then learn to generate and utilize for spatial reasoning tasks.
As for the format, our ISON format has been widely adopted as a computational model providing a flexible structure for VLMs, designed to offer bird's-eyc view representation encoding the relative positions and orientations of objects Yang et al. (2024). This representation aligns, at a high level, with the functional principles of cognitive maps in cognitive science. Our goal is equip VLMs with a scaffold that approximates the functional role of a cognitive map to enable explicit reasoning, rather than replicating its exact neurological basis.
The use of JSON is a principled choice for interfacing with text-native VLMs, following standard practices for eliciting structured outputs. VLMs fundamentally operate on sequences of language tokens, making JSON a naturally fitting text-based format. JSON provides structured and com- putationally effective means evaluate complex spatial outputs, constituting one of the standard methods for eliciting structured knowledge from LLMs and VLMs. Although differentiable vector- ized representations represent a promising research direction, current integration attempts have been widely recognized as ineffective, particularly owing to limitations in VLM comprehension.
D.1.3 FREE FORM REASONING
Spatial mental models are not just static representations; they are actively used for inference and problem-solving Tversky et al. (1994). To approximate this procedural and inferential aspect, we utilize free-form reasoning, implemented as a natural language Shain-of-Though (CoT) Wei et al. (2022) process. This encourages the VLM to externalize its step-by-step reasoning process when deducing an answer to a spatial query.
Our Setting: The VLM is prompted to generate textual reasoning chain before outputting the final answer. This process is guided by a three-step principle, exemplified by the reasoning chain shown in Figure 3, the reasoning chain panel. For the steps shown in that example, they are: (1) Initial Observation and Grounding: The model first processes each available view, identifying key objects and their immediate spatial relationships within that specific viewpoint. For instance, the example chain begins with: "In View 1, I see a potted plant, tissue box, and hand sanitizer from left to right, with a sofa behind. This step grounds the reasoning in direct visual evidence from individual perspectives. (2) Cross- View Integration and Environment Consolidation the model attempts identify consistent objects or environmental cues across the different views to recognize that they depict the same underlying 3D scene. The example reasoning continues: "In View 2, I see the same petted plant, sc both views are from the same envi ronment. This step is crucial building unified understanding of the space from discrete observations. (3) Question- Spatial Inference: Finally, based on the specific question posed and the integrated understanding from the previous steps, the model performs step-by-step logical and spatial inferences to arrive at the answer. In the example, this involves relating the object positions across views relative to the observer's 's position in View 2: "Since the hand sanitizer is rightmost in View 1, it's spatially furthest behind the potted plant when looking in View 2.
In View the potted plant is closest to me, so the hand sanitizer is the furthest from me."
D.2 EVALUATION METRICS
To quantitatively assess how these data structures affect the performance of VLMs the spatial mental modeling presented in MINDCURE, and to evaluate the quality of the generated cognitive
32

---

Published as a conference paper at ICLR 2026
maps, employed the following metries: (1) QA Accuracy, and (2) Graph Metrics for Generated Cognitive Maps.
D.2.1 QA ACCURACY
QA Accuracy serves as the core metric for evaluating task performance. It quantifies the proportion of questions that the vision-language model (VLM) answers correctly out of the total number of questions. higher QA Accuracy indicates better alignment between the model's 's responses and the ground truth.
The metric is formally defined as:
Nearrct
QA Accuracy
=
x 100%
Nooal
where Neoreet denotes the number of correctly answered questions, and Noval is the total number of questions evaluated.
D.2.2 GRAPH METRICS FOR COGNITIVE MAPS
To quantitatively evaluate the quality of generated cognitive map, we use of structured graph- based metrics. The overall process consists of several key steps:
1. Validity Check. First, we ensure that the generated map is syntactically and semantically valid-i.e. has a correct JSON format, contains interpretable object positions, and includes least one valid object.
Rotation Normalization. Since we do not enforce a fixed orientation for generated maps (to allow for flexible generation from vision-language models), evaluate the similarity between the generated map and the ground truth across a set of 3D rotations. We always choose the best-aligned rotation to compute our similarity scores.
Structural Matching. We define relation graph between object pairs in each map. capturing di- rectional and proximity-based relationships. A part of the evaluation is determining whether these relationships in the ground truth are preserved in the generated map.
4. Similarity Metrics. We compute coverage (how many ground-truth objects present), direc- tional similarity (relative spatial relations), and facing similarity (object orientation). These are aggregated into overall similarity score.
5. Ootation-Invarian Isomorphism. We also evaluate whether a generated map is graph- isomorphic the ground truth under any allowed 3D rotation, providing strict measure of structural correctness.
Below, we provide precise mathematical definitions for each of these components.
Notation. cognitive mup finite set of objects O , on where each object o; associated with (i) a 2-D position vector p; = (I,, 1)) E IR2 and (ii) an optional facing label fa {up, right, down, left, inner, outer) {Ø). For two maps, we distinguish (1) the ground-trutt map (O+ pp. f*) and (2) a generated map (O9, f
The set of objects that appear in both maps
Extended directional relation. We define directional proximity- based relationship between any ordered object pair (o,, o,) based on their spatial arrangement and optional facing annotations. TTis relation is captured via the function:
33

---

Published as a conference paper at ICLR 2026
right
[y 2y |> W/ M/ and 2j 2ii
left
[yy x; |> 1/;
ykl and zj. < 2i
down
[y 261 2 [ xi and Dj>>//s V/i
dir(os,
up
1y 341 2 JT 2% and Vj Vi,
inner
[pj p:112 < ó and (J: = inner
ʃj = outer),
outer
[p: P:l12 < 6 and (fr = outer V fj = inner),
Ø
otherwise,
with threshold 0.5 as in the implementation. These relations form relation matrix:
R(o,, dir(o;;
Coverage. Coverage measures how many ground-truth objects are successfully retrieved in the generated map:
109 O
Cov
E [0,1]]
Directional similarity. We now evaluate how well the generated map preserves the directional relationships among object pairs from the ground truth. Define:
P {(o,, E O x Oc li R (' & #0..
Then the directional similarity score is given by:
Sar ((o,,0,) E P' (o(o, R2 (oi, (oi,
E [0.1], 1],
p
which corresponds the proportion of directional relations in the ground truth that are correctly matched in the generated map.
Facing similarity. For objects with defined facing directions, we compare their orientation across the two maps:
F = for E O 1 #
Then:
Stuce Ho EF 1 fiHl E 10 1].
[F
Overall similarity. nation:
To aggregate the directional and facing similarities, we use a weighted combi-
S Sair + (1 ax) Stnce E [0. 1],
where a = 0.7 places greater emphasis on spatial layout than orientation.
Rotation-invariaa isomorphism. To ensure fair comparison regardless of orientation, we define set of 3D rotations: R = (RI.... Rm, }. including all 90" turns about the z-axis, and one 90" turn about each the z- and y-axes.
We say the maps matrices match completely:
isomorphic if there exists rotation such that their relation
3 k E (l....,m. V oi, Og E E R2 (o, o,) 12 (o,, o;),
where R2 is the relation matrix computed after applying Rg to the generated map.
Graph validity. Finally, a generated map deemed valid if: (1) It is well-formed JSON, (2) All fields conform to expected formats and constraints, and (3) At least one object has valid position.
Together, the tuple (Cov, Sair, Stice Soverlll- Iso provides a comprehensive, rotation- aware eval- uation of how closely generated cognitive map matches ground truth structure and orientation.
34

---

Published as a conference paper at ICLR 2026
D.3 PROMPTS FOR ALL INPUT-OUTPUT CONHIGURATIONS
Below, we provide all prompts for the input-output configurations we investigate in our work.
D.3.1 EXAMPLE FOR RAN-OA
Example Prompt for Raw-QA
[Task]
Your task is to analyze the spatial arrangement of objects in the scene by examining the provided images, which show the scene from different viewpoints. [Answer Instruction]
You only need to provide *ONE* correct answer selecting from the options listed below. For example, if you think the correct answer is A. Above' from 'A. Above B. Under C. Front D. Behind', your response should **only** be <ansser>> Above</ansswe>>". [Question]
Based on these four images (image 1, 2, 3, and 4) showing the white jar from different view- points (front, left, back, and right), with each camera aligned with room walls and partially capturing the surroundings: From the viewpoint presented in image 4, what is to the left of the white jar?
A. Table with cups on it B. Clothes rack C. Bed sheet with a floral pattern D. White head- board
D.3.2 EXAMPLE FOR FFR
Example Prompt for FFR: Free-Form Reasoning
[Task]
Your task is to analyze the spatial arrangement of objects in the scene by examining the provided images, which show the scene from different viewpoints. [Answer Instruction]
Please do step by step reasoning first, then give your final answer. For example, you think the correct answer AA. Above' from A. Above B. Under C. Front D. Behind', your response should be this format: (repacc> with your reasoning here)></hink>><<nswrr>>>> [Question]
Above<<annswr>>"
Based on these four images (image 1, 2, and 4) showing the white jar from different view- points (front, left, back, and right), with each camera aligned with room walls and partially capturing the surroundings: From the viewpoint presented in image 4, what is the left of the white jar?
A. Table with cups on it Clothes rack Bed sheet with a floral pattern D. White head- board
35

---

Published as a conference paper at ICLR 2026
D.3.3
EXAMPLE FOR VI-1 AND VI-2
Prompt for VI-1: View Interpolation with Frame
[Task]
Your task is to analyze the spatial arrangement objects the scene by examining the provided images, which show the scene from different viewpoints. [Answer Instruction]
You only need to provide *ONE* correct answer selecting from the options listed below. For example, if you think the correct answer is 'A. Above' from 'A. Above B. Under C. Front D. Behind', your response should **only** be <<answer>A A. Above</answer>>. [Question]
Based on these 8 images showing the white jar from different viewpoints (from front (image 1) to left (image 3), from left (image 3) to back (image 5), from back (image 5) to right (im- age 7), from right (image 7) back to front (image 1)), with each camera aligned with room walls and partially capturing the surroundings: From the viewpoint presented in image 7, what to the left of the white jar?
A. Table with cups it B. Clothes rack C. Bed sheet with floral pattern D. White head- board
Prompt for VI 22: View Interpolation with 2 Frames
[Task]
Your task is to analyze the spatial arrangement of objects in the scene by examining the provided images, which show the scene from different viewpoints.
[Answer Instruction]
You only need to provide *ONE* correct answer selecting from the options listed below. For example, if you think the correct answer is A. Above' from 'A. Above B. Under C. Front D. Behind', your response should **only*** be <<nnswe>>A. Above<<aanswer>. [Question]
Based on these 12 images showing the white jar from different viewpoints (from front (im- age 1) left (image 4), from left (image 4) back (image 7), from back (image 7) to right
36

---

Published as a conference paper at ICLR 2026
(image 10), from right (image 10) back to front (image 1)), with each camera aligned with room walls and partially capturing the surroundings: From the viewpoint presented in image 10, what the left of the white jar?
A. Table with cups on it B. Clothes rack C. Bed sheet with a floral pattern D. White head- board
D.3.4 EXAMPLE FOR AUG-CGMAP-ID
Prompt for Aug-Gu CGMap In: Grounded Augmented Cognitive Map as Input
[Task]
Your task is to analyze the spatial arrangement of objects in the scene by examining the provided images, which show the scene from different viewpoints. Also, we provide you a cognitive map that shows the general layout for the scene. Please the cognitive map reason and answer the question. [Answer Instruction]
You only need to provide *ONE* correct answer selecting from the options listed below. For example, if you think the correct answer is A. Above' from 'A. Above B. Under C. Front D. Behind', your response should **only** be <answer>>. Above</answer>> [Cognitive Map Format]
We provide you 2D grid map of the scene that is related to the question you should answer. Below the description of the map:
The map
10x10 grid where [0,0] is the top-left corner and [9,9] is at the bottom-
right corner
Thee map is shown in the bird's view
Directions are defined as:
up towards the top of the grid (decreasing y-value)
right towards the right of the grid (increasing x-value)
down towards the bottom of the grid (increasing y-value) left towards the left of the grid (decreasing x-value)
inner straight into the 2D map (perpendicular to the grid, pointing away from you)
outer straight out of the 2D) map (perpendicular to the grid, pointing towards you) "objects"" lists all important items the scene with their positions
"facing" indicates which direction object is oriented towards (when applicable) "views" represents the different camera viewpoints in the scene
Below is the cognitive map of the scene related to the question. Please use it reason and answer the question.
**jjoon
"ob j
("name": "white jar", "position": [5, 5]},
("name": "bed sheet with floral pattern",
"position": [5, 811,
("name": "white headboard", "position": [2, 5]),
("name": "clothes rack", "position": [5, 2],,
("name": "table with cups on it", "position": [8, 51}
},
"views": [
["name": "Image 1", "position": [5, 61, "facing": "up"),
("name": "Image 2", "position"; [4, 5], "facing": "right"),
37

---

Published as a conference paper at ICLR 2026
["mame"" : "I "Image 3", "position": [5, 4], "facing": "down")
("name": "Image 4 "position": [6, 5], "facing": "left")
]
1..
[Question]
Based on these four images (image 1, 3, and 4) showing the white jar from different view- points (front, left, back, and right), with each camera aligned with room walls and partially capturing the surroundings: From the viewpoint presented in image 4, what is to the left of the white jar?
A. Table with cups on it B. Clothes rack C. Bed sheet with a floral pattern D. White head- board
D.3.5 EXAMPLE FOR AUG CCMM--O0
Prompt for Bagg-GMMp--aa Out: Ask VLM to Output Augmented Cognitive Map and Direct Answer
[Task]
Your task to analyze the spatial arrangement objects the scene by examining the provided images, which show the scene from different viewpoints. You will then create detailed cognitive map representing the scene using a 10x10 grid coordinate system.
[Rules]
1. Focus ONLY on these categories of objects the scene: {white jar, bed sheet with a floral pattern, white headboard, clothes rack, table with cups it]
2. Create a cognitive map with the following structure in the bird's view:
-A 10x10 grid where [0,0] is at the top-left corner and [9,9] the bottom-right corner
up towards the top of the grid (decreasing y)
right towards the right of the grid (increasing x)
down = towards the bottom of the grid (increasing y)
left= towards the left of the grid (decreasing x)
inner straight into the 2D map (perpendicular to the grid, pointing away from you) outer = straight out of the 2D map (perpendicular to the grid, pointing towards you)
Include positions of all objects from the specified categories
Estimate the center location (coordinates [x, y)) of each instance within provided categories
If a category contains multiple instances, include all of them
Each object's estimated location should accurately reflect its real position in the scene, preserving the relative spatial relationships among all objects
Combine and merge information from the images since they are pointing to the same scene, calibrating the object locations accordingly
Include camera positions and directions for each view
Carefully integrate information from all views to create single coherent spatial repre- sentation.
[Answer Instruction]
1. Given the provided views and main objects mentioned in the above rules, you **MUST** present your cognitive map the following JSON format **before your answer**; "json
"ob jobjects":
38

---

Published as a conference paper at ICLR 2026
("name") "object_nande sameet_mme,,,, "position": [x, yl,
"facing": "di rection"),
("name": "object_withoute wttp:e/wwiiooottooeettttiii "position": [x, yl
},
"views": [
("name": "View/Image 1", "position": [x, y],
"facing": "direction"),
("name": "View/Image 2", "position": [x, y], "facing": "direction")
]
2. Next, provide *ONE* correct answer selecting from the options. Your answer field must be the format like "A. Above".
3. In general, your response's s format should be like "Based my observation, the answer is: <cogmap><(Rppacce with your cogmap <ere)<//ogga>>><ansse <anger>>>aa with your answer h/asw<//n,,.r>>" Your option must be from the available options. [Question]
Based these four images (image 1, 2, and 4) showing the white jar from different view- points (front, left, back, and right), with each camera aligned with room walls and partially capturing the surroundings: From the viewpoint presented in image 4, what is the left of the white jar?
A. Table with cups on it Clothes rack Bed sheet with a floral pattern D. White head- board
D.3.6 EXAMPLE FOR PLA IN TN-CGMA P-OUT
Prompt for Plain Maaln- Out: Ask VLM Output Plain Cognitive Map and Direct Answer
[Task]
Your task is to analyze the spatial arrangement objects the scene by examining the provided images, which show the scene from different viewpoints. You will then create detailed cognitive map representing the scene using a 10x10 grid coordinate system.
[Rules]
1. Focus ONLY on these categories of objects the scene: {white jar, bed sheet with a floral pattern, white headboard, clothes rack, table with cups it}
2. Create a cognitive map with the following structure in the bird's view:
-A 10x10 grid where [0, is at the top-left corner and [9, 9] is at the bottom-right corner
up towards the top of the grid (decreasing y)
right = towards the right of the grid (increasing x)
down = towards the bottom of the grid (increasing y)
left towards the left of the grid (decreasing x)
Include positions of all objects from the specified categories
Estimate the center location (coordinates [x. y]) of each instance within provided categories
If category contains multiple instances, include all of them
Object positions must maintain accurate relative spatial relationships
Combine and merge information from the images since they pointing the same scene,
calibrating the object locations with grid coordinates accordingly
Carefully integrate information from all views create single coherent spatial repre-
39

---

Published as a conference paper at ICLR 2026
sentation.
[Answer Instruction]
1. Given the provided views and main objects mentioned in the above rules, you * *MUST** present your cognitive map in the following JSON format "*before your reasoning* jjso
"ob "object_cctteorr__1" ("position": [x, ylh, "object_categoo
("position": [x, yl,
"facing": "direction"),
if the object is asked for orientation
2. Next, provide *ONE* conrect answer selecting from the options. Your answer field must be the format like "A. Above"
3. In general, your response's s format should be like "Based on my observation, the answer is: <copmap><Replace with your cogmap answer hare)</answer>>,, Your option must be from the available options. [Question]
with your
Based these four images (image 1, 2. 3, and 4) showing the white jar from different view- points (front, left, back, and right), with each camera aligned with room walls and partially capturing the surroundings: From the viewpoint presented in image 4, what is the left of the white jar?
A. Table with cups it B. Clothes rack Bed sheet with a floral pattern D. White head- board
D.3.7 EXAMPLE FOR PLAIN-CGMAP
Prompt for Mlain-CGMay SFMMap--FR---- Out: Ask VLM to Output Plain Cognitive Map and Free-Form Reasoning
[Task]
Your task is to analyze the spatial arrangement of objects the scene by examining the provided images, which show the scene from different viewpoints. You will then create detailed cognitive map representing the scene using 10x grid coordinate system.
[Rules]
1. Focus ONLY on these categories of objects in the scene: {white jar, bed sheet with floral pattern, white headboard, clothes rack, table with cups it}
2. Create cognitive map with the following structure in the bird's view:
10x grid where [0, is at the top-left corner and [9, 9] is the bottom-right corner
-up towards the top of the grid (decreasing y)
right = towards the right of the grid (increasing x)
down towards the bottom of the grid (increasing y)
left towards the left of the grid (decreasing x)
Include positions of all objects from the specified categories
Estimate the center location (coordinates [x, y)) of each instance within provided categories
If a category contains multiple instances, include all of them
Object positions must maintain accurate relative spatial relationships
Combine and merge information from the images since they pointing the scene,
40

---

Published as a conference paper at ICLR 2026
calibrating the object locations with grid coordinates accordingly
3. Carefully integrate information from all views to create a single coherent spatial repre- sentation.
[Answer Instruction]
Given the provided views and main objects mentioned in the above rules, you MUST**
present your cognitive map in the following JSON format **before your reasoning**;
json
"ob "object Sareect_cttegoy_222 2": ("position": [x, yl,
1": ("position": [x, ylh
"facing": "directions
# if the object is asked orientation
2. Next, please also provide your reasons step by step in details, then provide *ONE* correct answer selecting from the options. Your answer field must be in the format like "A A. Above" 3. In general, your response's 's format should be like "Based on my observation, the answer is: <Roplap> with your cogmap reasoning here))</thnkk> <answer> with your answer herre)//answe>>" Your op- tion must be from the available options.
(Repacc with your
[Question]
Based on these four images (image 1, 2, 3, and 4) showing the white jar from different view- points (front, left, back, and right), with each camera aligned with room walls and partially capturing the surroundings: From the viewpoint presented in image 4, what is to the left of the white jar?
A. Table with cups on it B. Clothes rack C. Bed sheet with a floral pattern D. White head- board
D.3.8 EXAMPLE FOR AuG AUG-CGMAP-EFR-OrE
Prompt for Aut CCMap-MMRRR..P-PP Ask VLM to Output Augmented Cognitive Map and Free-Form Reasoning
[Task]
Your task is to analyze the spatial arrangement of objects in the scene by examining the provided images, which show the scene from different viewpoints. You will then create detailed cognitive map representing the scene using 10x10 10 grid coordinate system.
[Rules]
Focus ONLY on these categories of objects the scene: fwhite jar, bed sheet with floral pattern, white headboard, clothes rack, table with cups it}
Create a cognitive map with the following structure in the bird's view:
-A 10x10 grid where [0,0] is at the top-left corner and [9,9] is at the bottom-right comer
up towards the top of the grid (decreasing y)
-
right towards the right of the grid (increasing x)
down = towards the bottom of the grid (increasing y)
left towards the left of the grid (decreasing x)
inner straight into the 2D map (perpendicular to the grid, pointing away from you)
outer straight out of the 2D map (perpendicular to the grid, pointing towards you)
41

---

Published as a conference paper at ICLR 2026
Include positions of all objects from the specified categories
Estimate the center location (coordinates [x. y]) of cach instance within provided categories
a category contains multiple instances, include all of them
Each object's estimated location should accurately reflect its real position in the scene, preserving the relative spatial relationships among all objects
Combine and merge information from the images since they are pointing the same scene, calibrating the object locations accordingly
Include camera positions and directions for each view
Carefully integrate information from all views to create a single coherent spatial repre- sentation.
[Answer Instruction]
1. Given the provided views and main objects mentioned in the above rules, you * **MUST** present your cognitive map in the following JSON format **before your reasoning**; "json
"ob "objects":
("name": "object "object_name", "position"; [x, y],
"facing": "direction"),
("name": ; coojeet_.itlloott.oiiettttin "position": [x, yl
}.
"views": [
("name": "View/Image Image 1", "position": [x, y],
"facing": "direction"),
("name": "View/Image 2", "position": [x, y),
"facing": "direction")
]
2. Next, please also provide your reasons step by step in details, then provide *ONE* correct answer selecting from the options. Your answer field must be format like "A Above" 3. In general, your response's 's format should be like "Based my observation, the answer is: <cogmap><(Relace with your cogmap
with your reasoning here)</think>>> <ansser>> with your answer heree<//nsser>>", Your op- tion must be from the available options.
[Question]
Based on these four images (image 1, 2, 3, and 4) showing the white jar from different view- points (front, left, back, and right), with each camera aligned with room walls and partially capturing the surroundings: From the viewpoint presented in image 4, what is to the left of the white jar?
A. Table with cups on it B. Clothes rack C. Bed sheet with a floral pattern D. White head- board
D.3.9 EXAMPLE FOR CGMAP- DD--FFR-OO T
Prompt for CGMap-In Lin-FFR-OuR Input VLM with Grounded Cognitive Map and Output with Free-Form Reasoning
[Task]
Your task is analyze the spatial arrangement of objects in the scene by examining the
42

---

Published as a conference paper at ICLR 2026
provided images, which show the from different viewpoints. Also, provide you cognitive map that shows the general layout for the scene. Please use the cognitive map to reason and answer the question. [Answer Instruction]
Please do step by step reasoning first, then give your final answer. For example, you think the correct answer A. Above' from A. Above B. Under C. Front D. Behind', your response should be this format: (repacc>rreplace with your reasoning here)</think>> <answer> A. Above-</answer> [Cognitive Map Format]
We provide you a 2D grid map of the scene that is related the question you should answer.
Below is the description of the map:
Thee map uses a 10x10 grid where [0,0] is at the top-left corner and [9,9] is at the bottom- right corner
The map is shown in the bird's view Directions defined
towards the top of the grid (decreasing y-value)
right towards the right of the grid (increasing x-value)
down towards the bottom of the grid (increasing y-value)
left = towards the left of the grid (decreasing x-value)
inner straight into the 21) map (perpendicular to the grid, pointing away from you) outer = straight out of the 2D map (perpendicular to the grid, pointing towards you) "objects" lists all important items in the scene with their positions
"facing" indicates which direction an object is oriented towards (when applicable)
"views" represents the different camera viewpoints in the scene
Below is the cognitive map of the scene related to the question. Please use it to reason and answer the question.
*'jjonn
"ob "objects":
("name": "white jar", "position": [5, 5]},
("name": : "bed sheet with floral pattern",
"position": [5, 8]},
("name": ":
"white headboard", "position": [2, 5],,
("name": "clothes rack", "position": [5, 2]),
("name": "table with cups on it", "position": [8, 5])
},
"views":
("name" " "Image 1 "position": [5, 61, "facing": "up"},
("name": "Image 2", "position": [4, 5], "facing": "right"),
("name": "Image 3", "position"; [5, 4], "facing"; "down"),
("name": "Image 4", "position": [6, 5], "facing": "left")
]
[Question]
Based on these four images (image 1, 2, 3, and 4) showing the white jar from different view- points (front, left, back, and right), with each camera aligned with room walls and partially capturing the surroundings: From the viewpoint presented in image 4, what is to the left of the white jar?
A. Table with cups on it B. Clothes rack C. Bed sheet with a floral pattern D. White head- board
E WHICH SCAFFOLDS BEST GUIDE SPATIAL THINKING IN UNCHANGED VLMs?
The primary experimental setup for evaluating how different structural scaffolds can guide spatial reasoning in unchanged VLMs, has been comprehensively described in Section 3 of the main paper.
43

---

Published as a conference paper at ICLR 2026
This current section is dedicated to presenting qualitative examples of model outputs, serving as case studies for the different scaffolding configurations tested.
For precise details the input prompt structures and specific instructions provided to the Qwen2.5- VL-3B- for each experimental condition (c.g., Raw- QA, FFR, P Plain -FP---C--MM----------- please refer to Appendix D.3.
E.1
VLM RESPONSE EXAMPLES FOR CONFIGURATIONS IN SECTION D.3
Output for Raw QA: Input Prompt Refer to D.3.1
C. Bed sheet with a floral pattern
Output for VI: Input Prompt Refer to D.3.3
B. Clothes rack
Output for
In: Input Prompt Refer to D.3.4
C. Bed sheet with a floral pattern
Output for FFR: Input Prompt Refer to D.3.2
<think>> To determine what is to the left of the white jar from the viewpoint presented in image 4, let's analyze the scene: In image 4, the white jar is placed on a table covered with a pink tablecloth featuring heart patterns.
The table has a glass top and is positioned against a wall.
To the left of the jar, there piece of furniture that appears be a bed bed frame. Given this information, the object to the left of the white jar is most likely bed or a bed frame. </think>
<answer>C. Bed sheet with floral pattern</answer>
Output for Aug-cOMap-FFR-Ort: Input Prompt Refer to D.3.8
jjoo
"objects":
("name": "white jar", "position"; [150, 446),
"facing": "left"),
""name": "bed sheet with a floral pattern", "position": [0, 317],
"facing": "left"),
("name": "clothes rack", "position": [0, 108),
"facing": "left"),
("name": "table with cups it", "position"; [188, 200],
"facing": "left"),
("name": "white headboard", "position": [188, 317],
"facing"; "left")
}.
"views":
("name": "Image 1", "position": [0, 0], "facing": "left"),
44

---

Published as a conference paper at ICLR 2026
("name"" ; "Image 2", "position": [0, 01, "facing": "left"),
("name": "Image 3", "position": [0, 0l, "facing": "left")
("name": "Image "position": [0, 0), "facing": "left")
]
<CogMap>>
Your cognitive map
< RReaonnnng>>
From the viewpoint presented in image 4, we can see that the white jar is positioned in the foreground, facing left. The bed sheet with floral pattern is located directly behind the white jar, covering the lower part of the image. The clothes rack is visible on the left side of the image, slightly above the bed sheet. The table with cups on it is situated further back, to the right of the white jar. The white headboard is also visible, but it is not as prominent as the other objects due its position behind the bed sheet. Therefore, my answer is A. Table with cups on it.
</Answer>
Therefore, my answer is A. Table with cups on it.
Output for Plain-CGMap-FFs- Input Prompt Refer 1)3.7
'' "jjsonn
"white_jar"; ("position"; [150, 440], "facing": "left"), stable_witt_ccus_onne ("position": [187, 260], "facing": "right")
...
<CogMap>>
YYor cognitive map
Reaaoning
that the white jar is positioned on the bed sheet with a floral pattern. The table with cups on located to the right of the white jar, indicated by its position in the image. The clothes rack is not visible this particular view, and the white headboard is not directly related to the white jar or the table with cups on it. Therefore, the correct answer is A. Table with cups it. <Answer>>
From the viewpoint presented in image 4,
Therefore, my answer A. Table with cups it.
Output for CGMap In FFR Out: Input Prompt Refer to D.3.9
C. Bed sheet with a floral pattern
E.2 ADDITIONAL GRAPH METRICS FOR GENERATED GRAPHS
To quantitatively assess the quality of the cognitive maps produced by our models, we employ a suite of graph-based metrics detailed in Section C.2.2. Table presents a comparison of the cognitive maps generated by two of our methods: Aug-CGMap-FFR-OO and plain-cMMap-FFR-------
The results highlight several key findings:
High Validity: Both methods demonstrate a strong ability to generate syntactically and seman- tically valid maps, achieving high Valid Rates (94.95% for P Plain-CGMap-PERR and 92.57% % for CMMP--P-MMPPPROO This indicates that the models successfully learned the required ISON structure for the cognitive maps.
45

---

Published as a conference paper at ICLR 2026
Structural Complexity: Achieving perfect structural replication of the ground truth remains challenging, as shown by the modest Isomorphism Rates. The Plain-CGMap-FRR- method performs significantly better, with 7.43% of its maps being structurally identical (iso- morphic) to the ground truth, compared to a mere 0.10 % for the augmented map method.
Superior Similarity Performance: A clear performance difference in semantic similarity is ident. The Aug Magg-GMap--ER FFPtiaap-FF--O----- Out method, which explicitly includes camera views, achieves a substantially higher Overall Similarity (51.12%) and is superior in representing both the relative directional relationships (Avg. Dir. Sim. of 43.57 %) and the correct orientation of individual ob- jects (Avg. Facing Sim. of 68.75%). ). In contrast, while Plain- Plain-CGMap SGGMap-FFFS FFR-Out maintains higher validity and isomorphism, it lags behind in all three similarity metrics.
Table 5: Comparison of graph metrics for cognitive maps generated by different methods. The met- rics evaluate the quality of the generated maps against the ground truth. Valid Rate: percentage of syntactically and semantically valid maps. Isomorphism Rate: percentage of maps that are struc- turally identical (isomorphic) to the ground truth, accounting for rotation. Overall Sim. (Similar- ity): weighted score combining directional and facing similarity (Soverall a Sair + (1-a) -Swee Avg. Dir. Sim. (Average Directional Similarity): correctness of relative spatial relations between objects. Avg. Facing Sim. (Average Facing Similarity): correctness of object orientations. All values are percentages (%).
Method
Valid Rate
Isomorphism Rate
Overall Sim.
Avg. Dir. Sim.
Avg. Facing Sim.
Bug-CGMMp-FFRROut
92.57 94.95
0.10
51.12 37.44
43.57 28.29
68.75
Flain-CGMap-ETR-O00
7.43
58.78
E.3 FURTHER ANALYSIS ON VIEW INTERPOLATION
To rigorously assess the impact of view interpolation and ensure fair comparison, we conducted extensive additional experiments covering view selection strategies, comparisons with optimal in- terpolation settings, and scaling laws across model sizes.
(1) Smart View Selection vs. Dense Interpolation. It's possible that the dense interpolation might introduce redundancy by implementing an "Oracle View Selection" baseline. Using GPT-5 as a planner to select the top-2 most informative views from 4 images, compared this "Smart Selec- tion" against our dense interpolation setting on Qwen2.5- VLL-7.BB As shown in Table 6, the Smart Selection (36.80%) performs similarly the standard 2-view setting but is significantly inferior to dense interpolation (47.40%). This suggests that intermediate views, often perceived as redundant, provide essential spatio-temporal continuity that high-performing models utilize stitch scenes; aggressive filtering disrupts this chain.
Method (Qwen2.5-VL-7BB
Top-2 Selection (Oracle) Dense Interpolation (VI-4)
Accuracy (%)
36.80
47.40
Table 6: Comparison: Oracle View Selection vs. Dense Interpolation.
(2) Comparison with Optimal View Interpolation. To ensure faimesss we compared our Plain- Mainn-CGaap against the best possible performance of the View Interpolation (VI) baseline on Qwen2.5-VL-3BB. As detailed in Table 7, our method MalainnCMMaa--F-OOO forms the baseline even its peak performance (VI-1, 46.47%). This confirms that the Cognitive Map provides structural advantage over simply increasing visual frame density. Furthermore, adding standard Free-Form Reasoning (FFR) to interpolated views harms performance as density increases (dropping from 53.53% 40.77%) indicating that the bottleneck lies in the perception ability to organize visual floods rather than reasoning capacity alone.
47. 77.41 ()) outper-
(3) Scaling Analysis: Does View Interpolation Scale? We extended our evaluation to larger mod- els, including Qwen2.5-VL-77 Qwen3-VL-8B, VL-8B, Qwen3-VL-223 VL-235B, and GPT-5, to test higher ca- pacity naturally resolves interpolation issues. Results for 7B/8B models (Table 8) show consis-
46

---

Published as a conference paper at ICLR 2026
Method (Qwen2.5VL-3B)
VI-O (Raw)
VI-I
VI-2
VI-3
VI-4
VI-5
VI-6
VI-7
Plain-CGMap-FFR-
47.41
44.94
44.59
43.18
44.35
43.41
42.82
45.28
RawQA
43.76
46.47
45.53
44.94
44.35
44.24
45.18
44.59
FFR
45.53
43.71
43.83
41.65
41.00
40.36
40.89
40.77
Table 7: Comparison with Optimal View Interpolation (Qwen2.5-VL-3B).
tent scaling law. While Qwen2.5-VI.-7B benefits from density (peaking VI-4), Qwen3- Qwen3-VI.-BB VL-8B exhibits unstable performance despite being architecturally advanced.
Config
VI-0
VI-1
VI-2
VI-3
VI-4
VI-5
VI-6
VI-7
Qwen2.5-VL
37.80
34.90
35.60
45.30
47.40
46.50
46.80
46.80
Qwen3-VL-88
33.80
36.60
37.60
35.20
35.30
33.90
35.80
35.80
Table 8: Scaling Analysis on Qwen2.5-VL-7B and Qwen3-VL-8B.
Moreover, for massive-scale models (Table 9), performance negatively correlates with view density. GPT-5 peaks at the sparse I-frame setting (46.59%) and declines 42.35% as density increases to 7 frames. Similarly, structured mapping, interpolation artifacts act noise rather than useful signals, for SOTA models.
drops to ~366 with interpolation. This suggests that without
Model
V1-1
V1-2
V1-3
V1-4
V1-5
VI-6
V1-7
GPT-5
46.59
45.18
44.24
44.59
42.59
43.53
42.35
Qwen3-VL-235B
38.94
38.59
37.29
37.18
35.29
36.12
36.00
Table 9: Performance of Large-Scale Models (GPT-5, Qwen3-VL-235B) across view densities.
E.4 EXPLICIT REASONING WITH VISUAL-OF-THOUGHH
Explicit Visual Sketching via External Tools. While text-structured maps (e.g.. JSON) introduce symbolic order, they fundamentally remain implicit token sequences. To bridge this gap, we inves- tigated whether externalizing spatial map into explicit visual representations could further enhance reasoning. Inspired by ViLaSRWi Wu et al. (2025b), we implemented pipeline where the VLM uses an external plotting engine (Matplotlib) rather than generating text maps directly.
We designed two visual-centric configurations:
Instead of ingesting raw JSON tokens, the model receives rendered 10 x 10 grid image where objects and viewpoints are plotted explicitly. This tests the model's 's ability to comprehend provided visual layouts.
Visual-Map-as-nnuut
2. Visual-Map-for- Reeasnnnn (Ima-CGMap): We adopt multi-turn tool-use framework. The model first acts as coder to generate drawing coordinates (c.g.. (obj, [x, (t,))) ) which are executed render visual sketch. This sketch is then fed back to the model as new visual prompt to guide the final spatial reasoning. The entire multi-turn interaction pipeline and prompt design were strictly aligned with the ViSaL. framework.
Results Analysis. As shown in Table 10, visual map configurations consistently outperform their textual counterparts. Specifically, Img CGMap achieves an overall accuracy of 43.13%, surpassing the text-based P Plain Plain-CGMap CGMap (41.33%). Notably, the "Among" spatial relation benefits signifi- cantly from the visual grid (41.13% vs. 39.67%), likely because relative positioning is more intuitive in pixel space than in coordinate space. This experiment validates that leveraging external tools to create explicit visual sketches grounds the model's 's reasoning more effectively than symbolic text, offering promising direction for future development.
47

---

Published as a conference paper at ICLR 2026
Cogretive Visueization www.i//eeeeeeeeeeeeoooooooo www.://wwww.....eaaeaaaaaaaaaaee
Cogaltine Vissaloation
(a) Explicit Visual Map in Among Setting.
(b) Explicit Visual Map in Around Setting.
Figure 13: Comparison of Map Representations. The visual map, generated via external tools, renders objects and viewpoints onto a 10 x 10 grid, providing explicit geometric grounding.
Configuration
Map Modality
Overall
Rotation
Among
Around
Raw-QA
37.81
34.00
36.00
45.20
CGMap-In (Original)
Text (JSON)
41.43
37.00
41.67
44.40
Img- Ing-CGMap-In (New)
Visual (Grid)
42.10
31.50
44.17
45.60
Plain- Plain-CGMap (Original) Img-CGMap (New)
Text (JSON)
41.33
25.00
39.67
58.40
Visual (Grid)
43.13
32.75
41.13
55.06
Table 10: Performance comparison between implicit text-based maps (JSON) and explicit visual- based maps (Image/Grid), Visual configurations consistently outperform their textual counterparts.
F CAN WE TEACH VLMs TO BUILD AND LEVERAGE SPATIAL REPRESENTATIONS?
In the main paper, we demonstrated that prompting frozen VLMs with external scaffolds offers limited improvements. This highlighted a core limitation: the models themselves aren't effectively forming internal spatial representations or reasoning through space. To address this, we investigated whether supervised fine-tuning (SFT) could teach VLMs to build and leverage these spatial models internally. This section of the appendix provides further details on our SFT methodology, starting with the crucial step of data curation.
F.1 SUPERVISED FINE-TUNING DATA CURATION
Effective SFT heavily relies on the quality and nature of the training data. To teach our VLMs the desired spatial reasoning capabilities, meticulously curated two primary types of data: cognitive maps and free-form reasoning chains. These were designed to provide strong supervisory signals for the model to learn how represent and reason about space.
F.1.1 COGNITIVE MAP GENERATION
As discussed in Section D.I, cognitive maps serve as 2D schematic representations of object lay- outs. For the SFT phase, we needed to generate ground truth cognitive maps that the VLM could leam to produce. Our approach to generating these maps annotations described Section B.I. We aimed for representations that were not only accurate but also in format that the VLM could feasibly learn to generate.
grounded in the object arrangement
48

---

Published as a conference paper at ICLR 2026
The generation process automated via script that processes input JSONL files, where each line item contains scene details including images and, crucially, meta_info describing the objects, their potential orientations, and the camera viewpoint setup. For every item, the script first identifies its specific spatial arrangement "setting" (e.g., "around," "among." "translation," or "rotation") by parsing the item's unique ID. Based this setting, dedicated functions apply of predefined rules and heuristics to determine the 2D coordinates (on 10x10 grid) and facing directions for both the objects and the camera views.
For instance, in the "around" setting, objects (typically 2-4) are placed in predetermined linear arrangement near the grid's 's center (e.g.. coordinates like [4.5], [5,5]), and camera views are posi- tioned at cardinal directions relative to these objects, based on the specific camera angles pertinent to the question. In the "rotation" setting, the camera fixed the center (15,5), and its facing direction changes across views, while object positions are defined relative to the camera's 's current orientation. Similar rule-based placements are implemented for "among" (objects in a cross or T-shape with views from specific angles) and "translation" (objects arranged linearly to depict relationships like "on" or "down to") settings. Object orientations, applicable, are also assigned based on the input meta.info.
Finally, the generated layout of objects and views is formatted into a structured JSON string, representing the cognitive map. This JSON cogmap, along with templated instructional prompts (cogmap input for VI.M input format guidance and cogmap output for VLM output task description), added to the original data item. The overall generation logic is summarized in Algo- rithm
Algorithm Cognitive Map Generation
Require: Dataset D containing items with spatial arrangement annotations Ensure: Updated dataset with cognitive maps in JSON format
for all item D do
2: setting + Extract setting type from item.id
3: Initialize empty cognitive map cogmap
Position objects and views based on setting type
4:
if setting = "around" then
5:
Position 2-4 objects in a line with coordinates like [4.5], [5,5], etc.
6:
Place views at cardinal positions based on camera angles
7:
else if setting "among" then
8:
Place center object at [5,5] and surrounding objects at [5,8], [2,5], [5,2], [8,5] Position views based on specified camera angles
9:
10:
else if selting "translation" then
Position objects according their spatial relationships (e.g., "on", "down")
12:
Place views to highlight these spatial relationships
else if setting "rotation" then
14:
Arrange objects based on rotation type (clockwise, counterclockwise, etc.)
15:
Fix camera at [5,5] with varying facing directions
16:
end if
Add orientation information where applicable
17:
for all object objects do object has orientation then Add facing direction ("up", "down", "left", "right")
18:
19:
20:
end
end for
22:
Format cogmaa as structured JSON
23: Add formatted cognitive map item
24:
end for
return Updated dataset
F.1.2 FREE-FORM REASONING GENERATION
While cognitive maps provide a structured, global understanding of the scene, effective spatial rea- soning also involves procedural, step-by-step thought process. To instill this capability in
49

---

Published as a conference paper at ICLR 2026
VLMs, generated a dataset of grounded free-form reasoning chains. These chains were designed to verbalize the mental simulation process required to answer the spatial questions in MINDCUBE.
The generation of these reasoning chains was closely tied to the question-answer (QA) templates developed in Scction 2. For each specific setting (c.g., rotation, among. around), we manually constructed reasoning chains following a consistent set of principles ensure logical coherence and clear grounding in the provided visual information and the question asked.
The core principles guiding the generation of these reasoning chains were:
Initial Scene Understanding. The reasoning begins by processing each input image individu- ally. This involves identifying key objects visible in that view and noting their explicit spatial relationships with other objects within that same view. This step emulates the initial perceptual intake human might perform.
2. Cross- View Consistency and Environment Integration. After individual view analysis, the reasoning emphasizes that although different images are provided, they all depict the same un- derlying spatial environment. This is often achieved by identifying and highlighting an anchor object or a consistent set of objects that appear across multiple views, thereby helping to establish a unified mental model of the scene.
3. Question-Driver Inference. With a foundational understanding of the scene established from the views, the subsequent steps in the reasoning chain are directly guided by the specifics of the question. This involves: (1) Mental Simulation: If the question involves a hypothetical change in viewpoint "what-if" scenario (e.g.. "what if you turn left?"", the reasoning chain explicitly verbalizes this mental transformation. (2) Perspective Taking: If the question requires adopting a different perspective (e.g.. "from the sofa's perspective"), the reasoning chain articulates this shift. (3) Spatial Relationship Deduction: The chain logically deduces the queried spatial relationship by integrating information from the relevant views, applying spatial concepts (like lfft-of, behind, further from), and referencing the established mental model of the scene.
This structured approach generating reasoning chains aimed to provide clear, step-by-step exam- ples of spatial thought processes for the VLM to learn from. Figure 14, 15 and 16 show a template example combined with the filled case for ROTATION, AMONG, AROUND, respectively.
F.2 DETAILED EXPERIMENTAL SETUP
In this section, provide more granular view of the experimental parameters employed dur- ing the Supervised Fine- Finningning (SFT) phase of our research. As stated in the main text, these experiments were designed to teach Vision-Language Models (VLMs) to build and leverage internal spatial representations. The base model for these SFT experiments was Owenn255-VL-3B-Instrrue
Table 11: Training hyperparameters for SFT ex- periments with
Parameter
Value
Dataset size Epochs Learning rate Scheduler Fine-tuning type
10,000 QA pairs
3
le-5
Cosine
Full-parameter
We utilized a consistent training script for all SFT experiments, ensuring comparability across different configurations. The primary variation across these runs was the specific dataset used (datasets variable in the script), corresponding to the different SFT task con- figurations discussed in Section 4.1, such as Other hyperparameters were kept constant to isolate the effects of the different training signals.
Batch Size
256
GPUs used
2 x NVIDIA H100
Max image resolution
90,000 pixels
Min image resolution
784 pixels
Model Max Length
8192 tokens
Weight Decay
Warmup Ratio
0.03
Aug-CGMap-OUr
Max Grad Norm
Precision
BF16
Optimizer
AdamW
The core training hyperparameters are summa- rized in Table and further detailed by the provided training script.
The training was conducted using distributed setup managed by torchrun and leveraged DeepSpeed with a ZeRO Stage 3 optimization strategy for efficient full-parameter fine-tuning. Specifically, we set NPROC PER NODE 2, utilizing two NVIDIA H100 GPUs, though the
50

---

Published as a conference paper at ICLR 2026
Rotation Template Type 2
Template Reasoning Chain
This scene observed using four images. In image 1, can see (object_ in front_viewe as the main object front me. image 2, can see (object_in cooject_in_fron (froootviwe2]) as the main object front of me. In image 3, can (object_ in f__front_ [in__] the main object in front of image 4, can see (object_in iinject_in_fooe froot_ylewwl as the main object in front of me. Image is the initial view. Image is captured after a 90-degree clockwise rotation from image 1. Image 3 is after another 90-degree clockwise rotation (180 degrees from image 1). Image is after further degree clockwise rotation (270 degrees from image 1). From the perspective of image 4: "(object_in _i__in_ffott_vvvveeee is in front, "(ojjectinn _it__innfff___eeee
is to the right, ([ojjectjin crctt__in_foot_eeeeee view2)
is behind, ((object_in iinninnn_tttnoeeeooo (n__rrot__vewa] to the left. Affer turnnng 90 degrees the right:
[[object_n jn_ frooto_ rrontyviewly '[object_i in vven_yveewy is now behind, "(object_i in_front__ view4y is now to my left. The object
front, '[object_i sottt_ii_oootttt [n_tront_veew]]
to my right,
located to my behind is ((ojjectinn _i__jnnnf______ee _n_rron_viewwj....... Therefore, from the viewpoint of image 4, after a mental turn of degrees to the right, the object to my behind ((ojjectinn _ir___i______ee_ in__ront_view@] The answer is (option)
Question: If you are standing at the viewpoint presented in image and turn 90 degrees to the right, what is to your behind ?
Options:
A. Staircases
B. Vanity unit
C. Urinal
D. Yellow signboard
rotation
agent-agent
self perspective
(non-linear
Curated Reasoning Chain
This scene observed using four images. image 1, can see yellow signboard as the main object in front of me. In image can see vanity unit the main object in front of me. image 3. can see staircases as the main object in front of me. In image 4, can see urinal as the main object in front of me. Image is the initial view. Image is captured after 90-degree clockwise rotation from image Image is after another 90 90-degree clockwise rotation (180 degrees from image 1J. Image 4 is after further 90-degree clockwise rotation (270 degrees from image 1/. From the perspective of image 4: 'urinall is in front. 'yellow signboard' is to the right, 'vanity unif is behind, 'stcircases' to the left. After turning 90 degrees the right: yellow signboard' now in behind, 'urinal
front, 'vanity unit' is now my right, 'staircases' located to my behind is 'staircases', Therefore, from the viewpoint of image 4, after mental turn of 90 degrees to the right, the object to my behind 'staircases', The answer is A, Staircases
to my left. The object
Figure 14: Example reasoning chain template for ROTATION
51

---

Published as a conference paper at ICLR 2026
Among - Template Type 3
Template Reasoning Chain
this scene. lbserve four images showing different perspectives. All images feature the chain_object object. image can see (main_ (main_objec[] front of (context_obs (mnin_objectt in front of the (context_obj, _bj__t_____________ v2). In image 3, can see (main_object) in front of the (context obj_V3 image can
image 2. can see
mnjjn_ojjecff front of the (context_ob) obj__4] By observing the main objeet and its surroundings across views, and noting the rotational changes, establish their relationships. Image I is the initial view. Image 2 captured ofter a 90-degree clockwise rotation from image 1. Image 3 ofter another d0-degree clockwise rotation (180 degrees from image 1). Image 4 is ofter a further 90-degree clockwise ratafion (270 degrees from image 1). Through analyzing these perspective changes, construct a complete spatial understandiin:: When I view (context_ [ojj_oovjj V2) behind (main_object olbject) the second view, implies that the first view, (context_obx _ti__ooo___]]____ on the right sicle of (main onjjn_objectf... Similarly, when see (context_ _ojj_ V43 behind (main_objact) in the fourth view, it indicates that the first view, (context_oble V4) on the left side [main_obe mnnin_object).. To determine view, which the third view. As scontext_obj.. obj_V)j is observed behind onjjcf) in the third view, means that the first view. (context 06]_33 positioned behind This way, cann fully comprehend the spatial relafionships objects in the entire scene from the perspective image 1, So, from perspective of image 1: (context_ob) _oooxtt_o____ee_ V2) to the right of imain_objectl) (context_od) _opootto______o__ V3) to behind, and (context_ boooooova.a v4) the left of objectt the answer (option).
behind me first view, examine opposite
Question: From the viewpoint presented in image what to the right of the black stool
Options:
A. Desk
B. Office Area
C. Grey sofa
D. Two chairs on the corridor
meanwhile
object-object
self perspective
(non-linear
Curated Reasoning Chain
In this scene. I observe four images showing different perspectives. All images feature the black stool as the main object. image 1, can black stool front of the cabinet desk along comidor. image see black stool in front of the office area, image 3, can see black stool in front the two chairs on the corridor. In image 4, can black stool in front of the grey sofa. identify the position change across views, focus on the main object's ongle variation. Then. analyze the angles and relafive positions other objects on the platform to back up this observation. lnnderstand that: Imoge is the initial view. Imoge is captured after 90-degree clockwise rotation from image 1, Image 3 after onother 90-degree clockwise rotation (180 degrees from image 1). Image after a further 90-degree clockwise rotation [270 degrees from image 1). Through analyzing these perspective changes. can construct complete spatial undiessaanding: when view affice
behind black stool the second view, it implies that office area is on the right side of black stool. Similarry, when see grey sofa behind black stool in the fourth view, it indicates that in the first view, grey sofa the left side black stool. However. am still uncertain obout what lies behind me in the first view, Then, recognize that can exomine the opposite view to find out. The opposite view of the fist view is the third view. As two chairs on the corridor is observed behind black stool the third view, it means that in first view, chairs the corrddor positioned behind This way. I can fully comprehend the spatial relationships of all objects in the entire scone. So, from the perspective of image office area to the right of black stool, two chairs the corridor is my behind. grey sofa to of black stool. The answer office area
first view,
Figure 15: Example reasoning chain template for AMONG
52

---

Published as a conference paper at ICLR 2026
Around - Template Type 1
Template Reasoning Chain
Ined to determine how moved from the viewpoint in image I to the viewpoint in image 2, In [osjectlvview]] (object2_ viewl) from leff right. image
image can see (anchor_object view2). notice that sunchor_objece both views) is visible in both images, but from different ongles. analyze how the viewpoint changed from image to image 2: The (anchor_ seschor_objjct..... sejjectannllsige which visible in image becomes more prominent image 2, This suggests moved (inferred_ movement_eemcmmmmmmm movemmnt__esccipooonn The changes in object visibility and positioning between images suggest moved jwe_jnneeedeeeveeee Therefore, answer (option)
clearly
Based these two views showing the same scene, which direction did you move from the first view to the second view?
Question:
Options:
A. Forward-left
B. Forward-right
meanwhile
agent-agent
self perspective
linear
Curated Reasoning Chain
Ineed determine how moved from the viewpoint in image the viewpoint in image 2, image 1, can see: grey square planter, white square planter from left to right. In image 2. con clearly see grey square planter. notice that grey square planter visible in both images, different angles. analyze how the viewpoint changed from image to image 2: The grey square planter, which is visible image becomes
prominent in image This suggests moved forward and toward the left side of the scene. The changes in object visibility and positioning between images suggest moved forward and to the left. Therefore, the answer is A. Fowward-efft
Figure 16: Example reasoning chain template for AROUND
53

---

Published as a conference paper at ICLR 2026
2, and NPROC_PER.NODE default- ing to 4, our table and resource claims point to 2 GPUs being used for these runs. The per device train batch size set to 4, and with gradient accumalation steps at 32, this resulted in an effective batch size of 256.
script template showed CIDAVVISIBLE
1 10-5 with cosine learning rate scheduler and warmup tio of 0.03 over training epochs. We enabled full fine-tuning of the vision encoder, MM-MLP, and the LLM components True), The maximum sequence length for the model was set to 8192 tokens ---modelmmxx.enngee 8192). and the maximum image input pixels were capped 90,000 (--max pixels 90000), with minimum of 784, Gradient checkpointing was enabled to man- age memory usage (--gradi ent checkpo i nt in True). Checkpoints were saved every steps (--save_steps 5), keeping total of 12 checkpoints (- (--save_total 1 imit 12).
The learning rate
sicunemmm.viiion True --tune mmmmmp True
- --tuuunemmm
This consistent and robust experimental framework, detailed in Sections 3.1, 3.2, and herein, al- lowed for fair comparison of the different SFT strategies aimed at improving the spatial reasoning capabilities of Dowen2.5-VL-3B-Innte
F.3
VLM RESPONSE EXAMPLES AFTER SET FOR CONFIGURATIONS SECTION D.3
Output for Raw-QA: Input Prompt Refer to D.3.1
<answer>> D. White headboaard</answerr
Output for MGg-CGMMa-OOtt Input Prompt Refer to D.3.5
Based on my observation, the answer is:
<cogmap>>
'rjsonn
"ob "objects":
("name": "white jar", "position"; [5, 5]},
("name": "bed sheet with floral pattern", "position"; [5, 81),
("name": "white headboard", "position": [2, 5]},
("name": "clothes rack", "position": [5, 2]},
("name": "table with cups on it", "position": [8, 5]}
},
"views": [
("name": "Image 1", "position": [5, 6], "facing": "up"i,
("name": "Image 2", "position": [4, 5), "facing"; "right"),
("name": "Image 3", "position": [5, 4], "facing": "down"),
("name": "Image 4", "position"; [6, 5], "facing"; "left")
]
<lcogmap><<aswer>> Bed sheet with a floral pattern</answer>
Output for Plain CGMaC- -ut:: Input Prompt Refer to D.3.6
Based on my observation, the answer is:
<cogmap>
''jsonn
"white jar": ("position": [5, 5]
54

---

Published as a conference paper at ICLR 2026
"bed sheet with a floral pattern": ("position": [5, 81),
"white headboard"; ("position": [2, 5]),
"clothes rack": ("position"; [5, 2]),
"table with cups on it"; ("position": [8, 5]}
</cogmap>><amm <atp:>>>aanwwr>rae Clothes rack </answer>
Output for FFR: Input Prompt Refer D.3.2
<think><In this scene, observe four images showing different perspectives. All images feature the white jar as the main object. In image 1, can see white jar in front of the table with cups on it. image 2, can see white jar in front of the bed sheet with floral pattern. In image 3, I can see white jar in front of the clothes rack. In image 4, I can see white jar in front of the white headboard. To identify the position change across views, focus on the main object's angle variation. Then, analyze the angles and relative positions of other objects on platform back up this observation. understand that: Image is the initial view. Image 2 is captured after 90-degree clockwise rotation from image 1. Image 3 is after another 90-degree clockwise rotation (180 degrees from image 1). Image 4 is after further 90-degree elockwise rotation (270 degrees from image 1). Through analyzing these perspective changes, can construct a complete spatial understanding: when view bed sheet with floral pattern behind white jar the second view, it implies that in the first view, bed sheet with a floral pattern is on the right side of white jar. Similarly, when I see white headboard behind white jar in the fourth view, it indicates that in the first view, white headboard is on the left side of white jar. However, I am still uncertain about what lies behind me in the first view. Then, recognize that can examine the opposite view to find out. The opposite view of the fist view is the third view. As clothes rack is observed behind white jar in the third view, it means that in the first view, clothes rack is positioned behind me. This way, can fully comprehend the spatial relationships of all objects in the entire scene. So, from the perspective of image 4: table with cups on it is to the right of white jar, bed sheet with floral pattern is to my behind, clothes rack is to the left of white jar. So the answer B. Clothes rack</think>>> <answer>B. Clothes rack </answer>>
Output for Aug Cuug-CGMap-FRR--MM -Ott: Input Prompt Refer to D.3.8
Based on my observation, the answer is: <cogmap>>
''jsson
"ob "objects":
("name": "white jar", "position": [5, 5]},
("name": "bed sheet with a floral pattern",
"position": [5, 81},
("name": "white headboard", "position": [2, 5]),
("name": "clothes rack", "position"; [5, 2]},
("name": "table with cups on it", "position": [8, 5]]
},
"views":
{ ("name": "Image 1", "position": [5, 61, "facing": "up"
["name": "Image 2", "position": [4, 51, "facing": "right";,
['name": "Image 3", "position": [5, 4], "facing": "down"),
["name": "Image 4", "position"; [6, 5], "facing": "left")
55

---

Published as a conference paper at ICLR 2026
//cogmap>><thhnk> this scene, observe four images showing different perspectives. All images feature the white jar as the main object. In image 1, can see white jar in front of the table with cups on it. image 2, can see white jar in front of the clothes rack. In image 3, I can white jar in front of the bed sheet with floral pattern. In image 4, can white jar in front the white headboard. To identify the position change across views, I focus on the main object's ' angle variation. Then, analyze the angles and relative positions of other objects on the platform to back up this observation. understand that: Image is the initial view. Image 2 is captured after 90-degree clockwise rotation from image 1. Image 3 is after another 90-degree clockwise rotation (180 degrees from image 1). Image 4 is after a further 90-degree elockwise rotation (270 degrees from image 1). Through analyzing these perspective changes, can construct complete spatial understanding: when I view clothes rack behind white jar in the second view, implies that in the first view, clothes rack is on the right side of white jar. Similarly, when I see white headboard behind white jar in the fourth view, indicates that in the first view, white headboard on the left side of white jar. However, I am still uncertain about what lies behind me in the first view. Then, recognize that can examine the opposite view to find out. The opposite view of the fist view is third view. As bed sheet with a floral pattern is observed behind white jar in the third view, means that in the first view, bed sheet with a floral pattern positioned behind me. This way, I can fully comprehend the spatial relationships of all objects in the entire scene. So, from the perspective of image 4: clothes rack to the right of white jar, bed sheet with a floral pattern is my behind, table with cups it is the left of white jar. So the answer is A. Table with cups on it. it.<//tink>>>> <answer>> A. Table with cups on it it<//anwer>>
Output for Plain- -lain--CMap--F&--- Input Prompt Refer D.3.7
Based my observation, the answer is: <cogmap>>
''jsonn
" "white jar": ("position": [5, 5]},
"bed sheet with a floral pattern": ("position": [5, }}),
"white headboard"; ("position": [2, 5]).
"clothes rack": {"position": [5, 2]},
"table with cups on it": ("position": [8, 51}
</cogmap>> <think>>hhhhhh In this scene, I observe four images showing different perspectives. All images feature the white jar as the main object. In image 1, can see white jar in front of the table with cups on it. image
can see white jar in front of the clothes rack. In image 3, I can see white jar in front of the bed sheet with floral pattern. In image 4, can see white jar in front of the white headboard. To identify the position change across views, I focus on the main object's 's angle variation. Then, analyze the angles and relative positions of other objects on the platform back up this observation. understand that: Image is the initial view. Image 2 is captured after 90-degree clockwise rotation from image 1. Image 3 is after another 90-degree clockwise rotation (180 degrees from image 1). Image 4 is after further 90 90-degree clockwise rotation (270 degrees from image 1). Through analyzing these perspective changes, can construct a complete spatial understanding: when I view clothes rack behind white jar in the second view, it implies that in the first view, clothes rack is the right side of white jar. Similarly, when see white headboard behind white jar in the fourth view, it indicates that in the first view, white headboard is the left side of white jar. However, am still uncertain about what lies behind me in the first view. Then, recognize that I can examine the opposite view to find The opposite view of the fist view is the
56

---

Published as a conference paper at ICLR 2026
third view. As bed sheet with floral pattern is observed behind white jar in the third view, it means that in the first view, bed sheet with floral pattern is positioned behind me. This way, I can fully comprehend the spatial relationships of all objects in the entire scene. So, from the perspective of image 4: clothes rack is to the right of white jar, bed sheet with floral pattern is to my behind, table with cups on it is to the left of white jar. So the answer is A. Table with cups on <anink>>annwwr>>>>> A. Table with cups on it</answer>>
F.4 DETAILED GRAPH METRIC RESULTS FOR SFT GRAPH-RELATED RELATED EXPERIMENTS
This section provides a detailed look at the Supervised Fine- Tine-Tuning (SFT) training dynamics to support the main paper's conclusions. The figures below plot key metries over training steps for four map-generation settings. A comparative analysis highlights that jointly training map generation and reasoning is the most effective strategy.
key
When training on map generation alone, as in the Plain- Plain-coMap-Out and Aug-CCMap-Out settings, the graph quality metrics show rapid convergence. However, the final QA accuracy is limited, reaching 54.38% for Plain- Plain-CGMap Out and 54.19% 14. for AGMpCGMap---- Out.
In contrast, the joint training approaches (Plain-CGMap-FFR-- Aug-CGMap-FFR-Out) despite a slower initial convergence on graph quality metrics, ulti- mately achieve far superior performance in task accuracy. The SeiinnnCMa--FFee CFMMM-FFR------------
and
setting
proves to be the most effective, reaching a QA Accuracy of 60.00%. The setting also yields strong results, with QA accuracy climbing to about 65%. This demonstrates the superiority of joint training for achieving high performance in both the final task accuracy and the quality of the generated spatial representations.
Licesuppae Anlo
Arcngge Deresin Nenkarty
@t Anttray
Figure 17: Training dynamies for the Lug-CCMMa--aee setting.
Orenal Stidutt
Aectage Stadart
Ascrge Ascryg Faciag Stallery
Figure 18: Training dynamics for the Plain Maiin-CGMap-Ous Out setting.
Lecruuplacko Jeic
Arstag Dererin Stedarly
QA Uk.Jem
Figure 19: Training dynamics for the Aug-CGMapa CGMa-- FFt setting.
57

---

Published as a conference paper at ICLR 2026
Iocoupple kulo
Averig Nerein Mederte
On Aveency
2004
Figure 20: Training dynamics for the Plain Saaann-CMap-2220 F-R--FFRR -Out setting, showing superior final performance.
F.5 WHICH PART OF VLM IS THE BOTTLENECK FOR SPATIAL UNDERSTANDING?
To develop more efficient fine-tuning strategies, it is crucial to understand which component of Vision-Laanguae Model (VL vision encoder responsible for perception or the Large Lan- guage Model (LLM) responsible for reasoning-preseene standing. To investigate this, we conduct a bottleneck analysis by selectively fine-tuning different parts of the VLM and observing the impact on performance.
the primary bottleneck for spatial under-
We evaluate four distinct training configurations on the Raw QA task, with results captured at an early stage of training (step 57) assess the initial learning dynamies. The configurations are: (1) the baseline performance of the pre-trained model without any fine-tuning; (2) fine-tuning only the vision encoder while keeping the LLM frozen; (3) fine-tuning only the LLM while keeping the vision encoder frozen; and (4) the standard approach of fine-tuning all parts of the model.
Table 12: VLM Training Bottleneck Analysis (Step=57, in %). Performance measured on the MINDCUBE-TINY benchmark under the Raw-QA setting.
Training Method
Overall
Rotation
Among
Around
Raw-QA (no fine-tuning)
37.81
34.00
36.00
45,20
Freeze LLM (Vision Encoder Only)
37.81
30.50
37.00
45.60
Freeze Vision Encoder (LLM Only)
51.43
34.00
50.00
68.80
Tune All Parts
52.28
34.50
52.50
66.00
The results, presented in Table 12, offer several key insights. First, there is a dramatic perfor- mance leap from the no-fine-tuning baseline (37.81% 1% overall), but only when the language model is trained. Methods involving LLM fine-tuning achieve over 51 accuracy, underscoring the necessity of adapting the model's reasoning capabilities.
Most strikingly, the performance bottleneck is almost exclusively concentrated in the LLM. Tun- ing only the LLM (Freeze Vision Encoder) yields an overall accuracy of 51.43%, captur- ing nearly the full performance gain of end-to-end fine-tuning (52.28%). In stark contrast, tuning only the vision encoder (Freeze LLM) provides no improvement whatsoever over the baseline (37.81%), This indicates that the bottleneck is not shared between modules. For this spatial task, adapting the model's language-baas based reasoning is critical, while adapting its visual perception is surprisingly ineffective.
Intriguingly, the fact that fine-tuning only the vision encoder fails to improve performance in itself significant finding. A possible explanation is that the pre-trained visual features are already sufficient to extract the necessary objects and their properties. The core challenge of the task seems to lie not in what is seen, but in how to reason about the spatial relationships across series of views-a a task primarily handled by the LLM. In conclusion, our analysis suggests that the most significant gains come from adapting the reasoning module. For efficient tuning, freezing the vision encoder and focusing solely on the LLM proves to be a highly effective strategy, achieving nearly top-tier performance at fraction of the computational cost.
58

---

Published as a conference paper at ICLR 2026
F.6 BRANCHING FROM RAR-QA SFT CHECKPOINT
In our main experiments, we fine-tuned the model for each specific task format starting from the base pre-trained VLM. A natural question arises: curriculum-based SFT approach further im- prove performance? Specifically, we investigate whether first fine-tuning the model on the simplest task format- -aa--Aa which only requires outputting the final answer-can establish a better foundation for learning to leverage more complex reasoning formats.
To test this hypothesis, we conducted the model fully fine-tuned on the 'Raw-QA MAw-AAA' task. Then, we used this specialized checkpoint as the initial weights for further fine-tuning on other scaffolding tasks, namely Aug-CGMap- FF Rsn, and SaA---MMM----ROOR tuning, the model's output for all tasks was still constrained to be only the final answer option. This setup allows us isolate the effect of the cognitive scaffolds on the model's internal reasoning process, rather than its ability to generate complex text.
branching experiments. We took the checkpoint from
It is important to note that during this second stage of fine-
The results, presented in Table 13, show consistent and notable improvement across all branched tasks compared to their counterparts trained from scratch. For example, both Aug-CCMap-In and Aug-CGMap-FFR--O
reach impressive overall accuracy of 49.00%. Even the FF Ran method benefits from this two-stage approach, with its overall accuracy rising to 46.82%. These findings suggest that two-stage SFT strategy is highly effective. By first grounding the model in the fundamental objective of the task (i.e., finding the correct answer) and then teaching it to process and leverage more complex cognitive scaffolds, we can achieve superior spatial reasoning performance. This indicates that the model, once primed for the core task, becomes more adept at utilizing the provided spatial context, even if it does explicitly generate the reasoning chain or cognitive map.
Table 13: Performance of various methods after being fine-tuned from a Raw QA SFT checkpoint.
This two-stage training approach led to performance gains across all methods. All accuracies reported as percentages (%).
Method
Overall
Rotation
Among
Around
Raw-G -A
46.36
33.50
51.20
46.75
Aug-CGMap-In
49.00
35.50
53.20
50.50
FF Ran
46.82
37.00
50.60
47.00
Aug-CGMap-FFR-
49,00
37.00
53.20
49.75
F.7 EXPERIMENTS ON OTHER MLLMS
Our core 'map-then-reason" insight generalizes across different Visual Language Model (VLM) families. We have conducted new experiments on InternVI.-2B23 et al. (2025), and the results below (Table 14) show that the outperform others by large margin.
and Plain MaaannCGMa--PPPP DFMMap-FFR--- FFR-FFFROU -utt settings
Table 14: Performance of Intern VL3-2B after supervised fine-tuning. All accuracies are reported as percentages (%).
Config.
Overall
Rotation
Among
Around
Overall Sim.
Isom. Rate
Raw-QA FFR
54,23
34.00
55.25
68.00
56.83
40.00
58.98
65.20
Mag-CGMap--i
52.98
35,00
53.90
65.20
87.39
66.15
Mlain-CGMa--ae
51.63
31.50
50.34
70.80
93.64
90.58
71.44
58.50
75.42
72.40
87.00
66.63
Plain-CCKap-PFR-OUt
73.56
65.00
77.63
70.80
92.04
88.27
59

---

Published as a conference paper at ICLR 2026
F.8 PROMPT SENSITIVITY ANALYSES.
To examine prompt sensitivity, we compared our structured representation (S) against natural- language (NL) formulation for the three strongest variants. As shown in Table 15, results vary by the structured prompt yields 41 11.3% versus 39.71% for NL slightly outperforms S (43.43% vs. 41.43%). These findings demonstrate that the optimal prompt style is setting-dependers and support the principle that VLMs should flexibly adapt to diverse user inputs rather than rely on single "perfect" format. Consequently, our paper reports results using one consistent and representative prompt variation.
format: for
NL, whereas for CGMap-In-FFR-OUL
Table 15: Prompt Sensitivity Analysis: structured (S) vs. natural language (NL).
Configuration
Acc (S)
Acc (NL)
Bug-cGMap-FFR-
40.57%
37. 37.43%
Plain CGMap-In- In--FR-OOOR
Mlain-CGMap- CGMap- FFR-Out
41.33%
39.71%
41.43%
43.43%
F.9 EXPLORING LATENT SPATIAL REPRESENTATIONS
Despite our primary focus on explicit representations like different scaffolds, we also investigated the model's internal features understand how it encodes spatial information and forms spatial men- tal models. To determine whether the model maintains latent spatial representations and encodes a niewpoint-nnvaiian "neural line," we conducted an experiment using 100 object triplets (front/left- /right views) from our MINDCUBE. We registered Py PyTorch forward hooks at the 10th LLM layer (11m. wridddensttteen of our SFT model to capture these features. We performed three comple- mentary analyses.
Pairwise similarity. We calculated the cosine similarity and Pearson correlation coefficient (r) for activations of the same object across different views (positive pairs) and between different objects (negative pairs). The results Table 16 show that both cosine similarity and Pearson correlation were higher for positive pairs compared to negative pairs in the layer- 10 LM. This indicates that the model has a measurable, layer-wiss wise spatial consistency for object identity across viewpoints.
Metric
Positive pairs
Negative pairs
Cosine
0.9651
0.9160
Pearson r
0.2750
0.2046
Table 16: Pairwise similarity results for positive and negative pairs.
Stable-dimension search. For each token dimension, we computed the variance across the three views. With only 64 stable dimensions (variance 4.15-6.01), the average cosine similarity across views was 0.068, which is significantly higher than random baselines. These low-variance dimensions act shared neurons that remain nearly constant across different viewpoints. Their high pairwise cosine similarity confirms that the model has learned an invariant representation.
Probing. To further investigate whether the identified stable dimensions truly encode robust, viewpoint-invariint representation, we conducted probing experiment. We trained a simple linear classifier on the activations from the model's 10th LLM layer. The task was to predict the direction of a 90-degree are rotation for given object. The training data consisted of pairs of activations corresponding to the front and left views of the objects, and the target label was either "clockwise" or "counter-clockwisee" This setup forced the classifier to learn the spatial relationship between the two viewpoints.
The classifier was then evaluated held-out test set of objects not seen during training. It achieved an impressive 85% accuracy, significantly outperforming the random chance baseline of 50%. This result provides strong evidence that the stable dimensions identified in previous analysis are not merely statistical artifacts. Instead, they represent a meaningful and generalizable spatial code. The model's ability encode this type of relational information allows simple linear probe to
60

---

Published as a conference paper at ICLR 2026
successfully infer complex spatial transformation entirely new objects, confirming that the latent representations are robust and useful for spatial mental modeling.
F.10 HYPERPARAMETER TUNING RESULTS
We conducted new hyperparameter tuning experiments to further validate our approach. Specifically, we performed a series of experiments to tune the hyperparameters for our Supervised Fine- Munnnn (SFT) settings, as detailed in Table17.
As shown in the table, our original hyperparameter configuration of a learning rate of 10 5, a batch size of 512, and a warmup ratio of 0.03 yielded the highest accuracy 52.28%. The results from these experiments confirm that the hyperparameters used in our initial submission are effective and well-optimized for the task, further substantiating our claims.
Table 17: SFT Hyperparameter Tuning Results
SFT (learning rate, batch size, warmup ratio)
Acc (%)
(1x10-5,512) 0.03) Ours
52.28
(2x10-5,512, 0.03)
51.71
(4x10-5.512, 0.03)
51.52
(1x10-5,256, 0.03)
50.86
(1x10-5.1022 0.03)
51.90
(1x10-5.512, 0.01)
51.81
(1x10-5,512,0.00
50.67
G CAN REMNEORCEMENT LEARNING FURTHER REFINE SPATIAL THOUGHT PROCESSES?
As discussed in the main paper, while Supervised Fine- Fine-Tuning (SFT) establishes strong foundation for spatial reasoning, reinforcement learning (RL) presents an avenue for further optimizing spatial thought processes through outcome-driven feedback. The core inquiry whether guiding VLMMs with rewards can lead to the development of more precise spatial mental models and enhanced reasoning capabilities. This section of the appendix provides a more detailed exposition of the experimental setup employed for the RL phase of our research. Additionally, present case studies to offer qualitative insights into how RL refines the models' spatial representations and reasoning chains.
G.1 DETAILED EXPERIMENTAL SETUP
For the reinforcement learning (RL) phase of our research, we employed the VAGEN framework. The core policy optimization algorithm used was Group Relative Policy Optimization (GRPO). To ensure consistency and allow for direct comparison with earlier stages of our work, key components from the Supervised Fine-Ture Tuning (SFT) experiments were retained. Specifically, the base Vision- Language Model (VLM) for all RL configurations was Qwen2.5-VL- Instructt and evaluations were performed on the MINDCUBE- TINY benchmark. All previously established evaluation met- rics were also retained.
In consideration of computational costs, each distinct RL configuration was trained for a duration of 0.5 epoch. The primary hyperparameters governing the RL training process were follows:
Training Batch Size: 32
Maximum Prompt Length: 1024 tokens
Maximum Response Length: 512 tokens
Actor Learning Rate: 1 x 10
Critic Learning Rate: 1 x 10-5
Number of Trajectories per Rollout: 8
61

---

Published as a conference paper at ICLR 2026
Maximum Turns per Trajectory: 1
As detailed in Section 5.1 of the main paper, investigated three RL task configurations:
RL-FFR (from scratch); The -rren..5VVL-3--Instie
model was trained generate free-form reasoning chains without prior SFT for this specific task format.
2. RL-Aug-CGMM Caa---GMMPPFPP FF--FFFF (from scratch): The model was trained to jointly produce aug- mented cognitive maps and associated free-form reasoning, also starting from the base pre-trained VLM.
RL-Aug-CGAN -ee--CMMM-P--OOU (from SFT): For this configuration, the RL training was initialized using the weights from the strongest performing SFT checkpoint, specifically the Aug-CGMap FFR-Out SFT model.
The reward function was designed to sparse yet directly indicative of desired behaviors. re- ward of +1 was assigned if the model's output was structurally valid (e.g., the generated cognitive map adhered to the predefined schema). A more significant reward +5 was given the model provided the correct final answer to the spatial reasoning question. This dual-component reward was intended to steer the learning process towards generating both well-formed intermediate spatial representations and achieving high end-task accuracy.
G.2 RL REWARD DESIGN ABLATION
A comprehensive reward ablation study can provide more convincing evidence for our approach. Inspired by the Deepseek-RI approachGuo et al. (2025), our reward design incorporates a dual- component structure: small score assigned for valid format generation and a larger score for correct answer generation. This design encourages both structural correctness and factual accuracy in the model's 's output.
To investigate the efficacy of this reward structure, we conducted an ablation study with three dif- ferent reward configurations, as detailed in Table 18. Our baseline, represented by frozen Visual- Language Model (VLM), achieved an accuracy of 40.57% Our proposed setting, which assigns +1 for valid format and +5 for correct answer, achieved accuracy of 50.57%. The "Answer Only" setting, which assigns +5 for correct answer but for format, resulted in a similar accuracy of 50.76%. In contrast, the "Format Only" setting, which provides score of +5 for valid format but 0 for the answer, yielded a lower accuracy of 45.05%.
These results confirm that the reward component for correct answer is the most critical factor in our RL reward design. The high value assigned to the correct answer score (+5) is highly effective improving the model's performance, finding that is consistent with recent work on RL for Large Language Models.
Table 18: RL Reward Ablation Study
RL Reward Setting (format score, answer score)
Acc (%)
Frozen VLM (baseline)
40.57
Our Setting (+1, +5)
50.57
Answer Only (+0, +5)
50.76
Format Only (+5, +0)
45.05
G.3
VLM RESPONSE EXAMPLES AFTER RL FOR CONFIGURATIONS SECTION D.3
Output for FFR: Input Prompt Refer to D.3.2
<think>> From image 4, the white jar is positioned on a surface with a foral-patterned bed sheet. To the left of the jar, there is table with a pink tablecloth featuring heart shapes and a Hello Kitty design. There are also cups on the table. Therefore, the object to the left of the white jar the table with the cups. www./iinnn<nnnerrr>> Table with cups on i/<s//nwer>>
62

---

Published as a conference paper at ICLR 2026
Output for CGMap---Maa-----aa Farr----F---utt (from scratch): Input Prompt Refer to
D.3.8
<cogmap>
"objects": ("name": "white jar", "position": [265, 436], "facing": "left")
l.
"views": [
("name": Wieww/maae 1", "position": [265, 436],
"facing": "left"),
("name": Friew/Image 2", "position": [265, 436],
"facing": "left"),
("name": "View/Image 3", "position": [265, 436), "facing": "left")
("name": SiieewImaaee 4", "position": [265, 436], "facing": "left")
<Your cognitive map>></cogmay
<think>>Th white jar is located in the center of the 10x10 grid, facing the left. The clothes rack is to the left the white jar, as seen in the image 2 view. The bed sheet with floral pattern is located the right of the white jar, as seen in the image view. The white headboard is not visible in any of the provided images.
Based on the provided images, the white jar is located in the center of the 10x10 grid, facing to the left. The clothes rack is to the left of the white jar, as seen in the image 2 view. The bed sheet with a floral pattern is located to the right of the white jar, as seen in the image I view. The white headboard is not visible in any of the provided images. <otsss/////i/>/// <answer> Table with cups on it. </aaswer>>>
Output for RL- RL-Aug CGMap-FFR- -ut (from SFT): Input Prompt Refer to D.3.8
Based my observation, the answer is: <cogmap>
''jjoon
"objects" [
("name": "white jar", "position"; [5, 5]},
("name": "bed sheet with floral pattern",
"position"; [5, 8]),
("name": "white headboard", "position": [2, 5]),
("name": "clothes rack", "position": [5, 2]},
("name": "table with cups it", "position": [8, 5])
},
"views": [
("name": "Image 1", "position": [5, 6], "facing": "up"i,
("name": "Image 2", "position": [4, 5], "facing"; "right"),
("name": "Image 3", "position": [5, 4], "facing": "down"),
("name": "Image 4", "position"; [6, 5], "facing"; "left")
]
</cogmap>> <thnn>> In this scene, I observe four images showing different perspectives. All images feature the white jar as the main object. In image 1, can see white jar in front of the clothes rack. In image 2, can white jar front of the table with cups it. In image 3,
63

---

Published as a conference paper at ICLR 2026
I can white jar in front of the bed sheet with floral pattern. In image 4, can white jar in front of the white headboard. To identify the position change across views, I focus on the main object's 's angle variation. Then, analyze the angles and relative positions of other objects on the platform to back up this observation. understand that: Image is the initial view. Image 2 is captured after 90-degree clockwise rotation from image Image is after another 90-degree clockwise rotation (180 degrees from image 1). Image 4 is after further 90-degree clockwise rotation (270 degrees from image 1). Through analyzing these perspective changes, I can construct complete spatial understanding: when view table with cups on behind white jar in the second view, it implies that in the first view, table with cups on it is the right side of white jar. Similarly, when see white headboard behind white jar in the fourth view, it indicates that in the first view, white headboard is on the left side of white jar. However, I am still uncertain about what lies behind me in the first view. Then, I recognize that can examine the opposite view to find out. The opposite view of the fist view is the third view. As bed sheet with a floral pattern is observed behind white jar in the third view. it means that in the first view, bed sheet with floral pattern positioned behind me. This way, can fully comprehend the spatial relationships of all objects in the entire scene. So, from the perspective of image 4: clothes rack is to the right of white jar, table with cups on of white jar. So the answer is C. Bed sheet with floral pattern. </think <answer>> C. Bed sheet with a floral pattern </answer>
my behind, bed sheet with floral pattern is the left
64


Reviews:
[{"review_id": "JCOXPVDJrF", "reviewer": "Reviewer_DbD3", "summary": "This paper focuses on the VLMs’ capabilities in \"spatial mental modeling\", the ability to imagine environments from a few visual observations. The paper first proposes a benchmark to measure current VLMs’ capabilities, finding that most existing models do not perform well on these tasks. Then, the paper explores methods to improve such capabilities through two approaches: (1) Scaffolds: carefully designed data structures to encourage spatial mental modeling, and (2) Training (SFT and RL). The paper identifies several scaffolds that could benefit spatial mental modeling and observes that combining SFT and RL leads to the best spatial reasoning performance.", "questions": "Please see the weaknesses above.", "limitations": "", "rating": 2, "confidence": 3, "soundness": 2, "presentation": 1, "contribution": 2, "strengths": "1. The paper performs solid and substantial experiments, measuring various models and considering various settings when studying the spatial mental modeling ability.\n2. The paper not only proposes a benchmark, but also proposes several methods to overcome the challenges of current VLMs.\n3. Spatial cognition is an interesting and important problem, and many works have shown that it could be challenging for VLMs.", "weaknesses": "1. Benchmark Motivation: The benchmark aims to evaluate the ability to imagine a full scene from a few views. To me, this seems indirect, since humans and embodied agents usually understand environments through continuous observations or frames rather than 90-degree viewpoint shifts as shown in the proposed benchmark. This raises two concerns: (1.1) Can you provide real-world examples where a VLM must infer spatial relations from four orthogonal views? (1.2) More importantly, how does the proposed benchmark improve upon existing benchmarks that test MLLMs’ spatial cognition abilities given a video, e.g., [1]? The proposed benchmark seems like a special case of video input.\n\n2. Benchmark Settings: The benchmark includes three settings (rotation, around, among). However, the number of “among” samples is significantly larger than the others (18,204 vs. around 1,000). Why is the dataset so imbalanced?\n\n3. Proposed Scaffolds seem trivial: Section 3 introduces three categories of scaffolds that could potentially improve model performance. Based on Figure 3, these are specially designed input structures. How do they differ from traditional chain-of-thought prompting through the system prompt?\n\n4. Clarity and Consistency: Some parts of the paper could be better organized for clarity and consistency. For example, in Table 2, what is the difference between FF-Rsn and FFR? They both seem to refer to free-form reasoning and should not use two different terms. Additionally, it is unclear what distinguishes Aug-CGMap-In from Aug-CGMap-Out, or Aug-CGMap-Out from Plain-CGMap-Out, etc. These distinctions can only be seen clearly through examples the Appendix, but the main text should describe them clearly and concisely.\n\n5. Training: Based on Figure 4, SFT is conducted for around 55 steps. How large is the training dataset? Why does the training converge so quickly?\n\n6. It is not surprising that SFT and RL on the specific tasks could improve the corresponding performance. The conclusion could be useful but the SFT and RL method does not introduce technical novelty.\n\n[1] Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spaces", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "gHclhk0LLg", "reviewer": "Reviewer_ahEo", "summary": "The paper introduces MindCube, a multi-view reasoning benchmark tailored for evaluating whether VLMs can build \"spatial mental models\" of scenes from partial observations. Using MindCube, authors show that state-of-the-art VLMs perform near chance and struggle to maintain cross-view consistency or reason about occluded objects. The paper also explored three structural scaffolds (view interpolation, free-form reasoning, and cognitive maps) and find a synergistic \"map-then-reason\" approach yields the largest gains. Finally, they train models with SFT and reinforce with RL, and find that the joint map-then-reason setup with RL boosts accuracy, indicating that constructing and using internal structured maps substantially improves multi-view spatial reasoning.", "questions": "* I agree that explicit reasoning can help VLM's spatial reasoning. However, it is unclear whether the paper’s \"cognitive map\" is truly explicit: the JSON-like maps are implicitly produced as token sequences by the LLM and only later decoded into a parseable structure. Would it be an feasible option to use a more explicitly constructed representation (e.g., using external tools to draw sketches as in Visual Sketchpad [1] or ViLaSR [2]), so that intermediate maps are drawn and manipulated directly rather than decoded from textual tokens?\n\n---\n\n[1] Visual Sketchpad: Sketching as a Visual Chain of Thought for Multimodal Language Models, Hu et al., NeurIPS 2024\n\n[2] Reinforcing Spatial Reasoning in Vision-Language Models with Interwoven Thinking and Visual Drawing, Wu et al., NeurIPS 2025", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "* The paper addresses an important spatial, cognitive ability for VLMs, which is to understand the coherent 3D structure from partial multi-view observations. It would be essential for VLMs to be integrated into real-world embodied systems.\n\n* The authors propose a novel, well-structured benchmark MindCube tailored for assessing the 3D spatial understanding ability of VLMs.\n\n* The paper delivers a rich set of experiments to evaluate which “cognitive scaffold” best assists VLMs to do accurate spatial reasoning, and how post-training techniques like SFT and RL each enhance the spatial mental modeling process. Moreover, the authors present insights on the behavior of Qwen.2.5-VL on each of these configurations, providing a solid ground for future research on multi-view spatial reasoning.", "weaknesses": "While the paper includes a rich set of experiments, the **experiments setups for different cognitive scaffolds** seems quite naive. That is, the comparisons among different scaffolds may not be fair enough to conclude that \"cognitive map\" is the best choice among those. I outline some of the main concerns below.\n\n* **View Interpolation:** It seems obvious that simply adding more interpolated views would not lead to great improvements. Expanding the visual context can inflate the token budget and increase hallucinations in VLM reasoning. Given that Qwen2.5-VL-3B is relatively low-capacity model compared to larger Qwen variants and other proprietary models, expecting it to improve solely from more views seems unrealistic. A more promising direction could to add a view selection phase to filter redundant perspectives. Additionally, it would be informative to test whether view interpolation becomes effective for higher-capacity models (e.g., > 7B models or proprietary models).\n\n* **Cognitive Map:** The paper presents cognitive maps as a core contribution, but offers little analysis of **which representation formats** most benefit VLMs. Although the paper introduces an augmented version, it is simply addining more information to the original JSON-like format. This format seems to be highly influenced by the cognitive map prompting technique from Thinking-in-Space [1]. Have the authors tried exporing any variants or alternative for the cognitive map formats, rather than following Thinking-in-Space? A broader ablation over map formats would better justify the proposed design and its effectiveness.\n\n* One of the main claims in the paper is that through SFT and RL Qwen2.5-VL can learn to \"generate\" a map and \"reason\" upon it, which leads to notable improvements on MindCube. But recent works in LLM reasoning have shown multiple evidence that the CoT reasoning traces in LLMs may not be logically related to the final output, and in fact they may be separate. In this perspective, I'm curious whether simply generating a map and returning the final output are in fact logically cause-and-effect relationship, or rather they may be mere byproducts similar to the CoT in LLMs [2, 3]. \n\n---\n\n[1] Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spaces, Yang et al., CVPR 2025\n\n[2] Reasoning Models Don't Always Say What They Think, Chen et al., 2025\n\n[3] Reasoning Models Can Be Effective Without Thinking, Ma et al., 2025", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "runJ1cTwFg", "reviewer": "Reviewer_WY9r", "summary": "This paper addresses the critical failure of VLMs in forming spatial mental models: the ability to reason about unseen spaces from limited views. To systematically evaluate this gap, the authors introduce the MINDCUBE benchmark, demonstrating that existing models perform near-randomly on such tasks. The paper's key contribution is a synergistic \"map-then-reason\" approach, which trains a model to first generate an internal cognitive map of an environment and then reason upon it. This method actively constructs and uses an internal spatial representation, especially when refined with reinforcement learning, can significantly boost task accuracy from a baseline of 37.8% to 70.7%.", "questions": "The paper's fine-tuning success is demonstrated exclusively on a single model family, leaving its generalizability as an open question. Have the authors tried other architecturally distinct VLMs?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper originally frames the issue of spatial reasoning through the cognitive science concept of \"spatial mental models\" and introduces MINDCUBE, a high-quality benchmark designed to rigorously diagnose this fundamental capability gap in current VLMs.\n\n2. The \"map-then-reason\" approach is a highly effective solution, significantly outperforming methods that use pre-supplied maps or unstructured reasoning.\n\n3. The study is distinguished by its rigorous and comprehensive experiments, including a critical bottleneck analysis that pinpoints the language reasoning module as the primary weakness, providing a robust foundation for its conclusions and clear direction for future research.", "weaknesses": "1. The proposed cognitive map is a significant oversimplification of the flexible and distorted mental models from cognitive science, potentially teaching the model a brittle form of reasoning that may not generalize to less structured environments.\n\n2. The benchmark's reliance on static images evaluates the one-time synthesis of a mental model but not its crucial ability to dynamically update from a continuous visual stream, which may limit the relevance of the findings for real-world embodied agents that require ongoing spatial awareness.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "oLxTbfkOSn", "reviewer": "Reviewer_tDRP", "summary": "This paper introduces MindCube, a new benchmark for evaluating the ability of VLMs to form \"spatial mental models\" from limited views. The authors show that existing models perform poorly, then propose a \"map-then-reason\" approach, which is to train a model to first generate a structured cognitive map and then reason upon it. This method leads to performance improvement through a combination of supervised fine-tuning and reinforcement learning.", "questions": "1. The interesting finding that view interpolation is ineffective is not deeply analyzed. Could the authors elaborate on the reason for this failure? Is it due to the quality of the interpolated views, the increased context length, or is it further evidence that the primary bottleneck is reasoning, not perception?\n2. Line 391 states a performance gain of \"51.09%\" for FF-Rsn, but this number does not appear in Table 4 (which shows 53.52%). Could the authors please clarify the source of this value?\n\nI'm happy to adjust the scores if the author can address the concerns in the weaknesses and questions.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. High-quality benchmark and problem formulation. The proposed benchmark MindCube is well-designed and challenging, which systematically probes the critical VLM weakness of reasoning about unobserved space from multi-views. \n2. Systematic and rigorous experiments: The authors conduct a thorough investigation into different scaffolding methods with different input and output settings, logically progressing from frozen to trained models via SFT and RL. \n3. The \"map-then-reason\" paradigm yields a huge performance improvement, providing strong evidence for its efficacy.", "weaknesses": "1. Simplified \"Cognitive Map\" Representation: The paper's \"cognitive map\" is a practical choice but is a simplification of the richer cognitive science concept with JSON format, which may limit its generalizability towards more complex scenarios. \n2. Some Shaky Conclusions in the Paper: Regarding experiments, some of the claims remain quite shaky. For example, the conclusion that cognitive maps guide reasoning (L315) in frozen models is based on a marginal performance gain ( from 40.48 to 41.43, less than 1%, 10 data points) that is likely not statistically significant, especially considering the performance variance reported across seeds (in Table 14).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)