SpinBench: Perspective and Rotation as a Lens on Spatial Reasoning in VLMs
Authors: Yuyou Zhang
Radu Corcodel
Chiori Hori
Anoop Cherian
Ding Zhao
Venue: ICLR 2026 Poster (2026)
Keywords: Spatial reasoning
VLMs
benchmark
Primary area: datasets and benchmarks
TL;DR: We propose SpinBench, a cognitively inspired benchmark that decomposes perspective taking into fine-grained tasks, which reveals systematic weaknesses in 43 VLMs and highlights the need for structured diagnostics to advance spatial reasoning.

Abstract: We present SpinBench, a cognitively grounded diagnostic benchmark for evaluating spatial reasoning in vision language models (VLMs). 
SpinBench is designed around the core challenge of spatial reasoning: perspective taking, the ability to reason about how scenes and object relations change under viewpoint transformation. 
Since perspective taking requires multiple cognitive capabilities, such as recognizing objects across views, relative positions grounding, and mentally simulating transformations, SpinBench introduces a set of fine-grained diagnostic categories. 
Our categories target translation, rotation, object relative pose, and viewpoint change, and are progressively structured so that single-object simpler tasks scaffold toward the most demanding multi-object perspective-taking setting.  We evaluate 43 state-of-the-art VLMs, both proprietary and open source. 
Results reveal systematic weaknesses: strong egocentric bias, poor rotational understanding, and inconsistencies under symmetrical and syntactic reformulations.
Scaling analysis shows both smooth improvements and emergent capabilities. 
While human subjects achieve high accuracy (91.2\%), task difficulty as measured by human response time shows strong correlation with VLM accuracy, indicating that SpinBench captures spatial reasoning challenges shared across humans and VLMs.
Together, our findings highlight the need for structured, cognitively inspired diagnostic tools to advance spatial reasoning in multimodal foundation models. 
Our website can be found [here](https://spinbench25.github.io/).

Published as a conference paper at ICLR 2026
SPINBENCH: PERSPECTIVE AND ROTATION AS A LENS ON SPATIAL REASONING IN VLMS
Yuyou Zhang! Radu Corcodel", Chiori Hori', Anoop Cherian", Ding Zhao CCarnegie Mellon University, Mittuubishi Electric Research Labs (yuyouz, di d and cmu edu, {corcodel, chori, cherian]emeri..
ABSTRACT
We present SPINBENCH, cognitively grounded diagnostic benchmark for eval- uating spatial reasoning in vision language models (VLMs). SPINBENCH is de- signed around the core challenge of spatial reasoning: perspective taking, the abil- ity to reason about how scenes and object relations change under viewpoint trans- formation. Since perspective taking requires multiple cognitive capabilities, such as recognizing objects across views, relative positions grounding, and mentally simulating transformations, SPINBENCH introduces tic categories. Our categories target translation, rotation, object relative pose, and viewpoint change, and are progressively structured so that single-object simpler tasks scaffold toward the most demanding multi-object perspective-taking setting. We evaluate 43 state-of-the-art VI.Ms, both proprietary and open source. Results reveal systematic weaknesses: strong egocentric bias, poor rotational understand- ing, and inconsistencies under symmetrical and syntactic reformulations. Scaling analysis shows both smooth improvements and emergent capabilities. While hu- man subjects achieve high accuracy 91.2%) task difficulty as measured by human response time shows strong correlation with VLM accuracy, indicating that SPIN- BENCH captures spatial reasoning challenges shared across humans and VLMs. Together, our findings highlight the need for structured, cognitively inspired diag- nostic tools to advance spatial reasoning in multimodal foundation models. Our website can be found at https:: http:///ppp/////eee//eese github. io/.
of fine- aie-grrined diagnos-
INTRODUCTION
Spatial reasoning is a fundamental component of human cognition and key capability for embodied agents operating in the physical world (Xia et al., 2018). From recognizing object configurations to simulating motion and perspective changes, spatial understanding enables agents to interpret their environment and plan actions accordingly.
Multimodal foundation models, particularly vision-language models (VLMs), have recently achieved impressive progress in visual understanding (Li et al., 2025b; Wang et al., 2025; Qwen et al., 2025; Team et al., 2025; Li et al., 2024b), however their spatial reasoning capabilities re- main poorly understood and underdiagnosed. The demonstrated utility in downstream tasks, such as navigation (Elnoor et al., 2025; Song et al., 2024), manipulation (Yang et al., 2025c; Sermanet al.. 2024), autonomous driving (Xie al., 2025; Pan et al., 2024), and physical commonsense reasoning (Chow et al., 2025) primarily reflects end-to-end performance at the application level, where spatial reasoning is entangled with high-level language and planning objectives. They do not directly test whether models understand geometric primitives, such as rotation, translation, object- relative pose, and viewpoint changes and thus can not expose failures underlies spatial intelligence.
As result, it remains unclear whether VLMs are genuinely capable of spatial reasoning, or whether they rely on dataset biases and shallow pattern matching. Recent benchmarks like MindCube (Yin et al., 2025) and Space (Ramakrishnan et al., 2024) reveal striking failures in mental modeling and spatial generalization, often exposing large performance gaps between models and humans. While efforts such SpaceOm and Spacethinker (Chen et al., 2025a) explore linguistic training enhance-

---

Published as a conference paper at ICLR 2026
Identity matching
Dynamic rotation
Dynamic translation
tear box images shove dibrrniccr
Wilct wny the sted kart Clocsaio oefondinttiio
Whach divctio de the adtoora inaie to the meo
Quaner D-1
du Sitbeall surtardcchote
Porr the parso''s des propection.
Which Beau Frea Imague (A, B. Oishoee a difirrric
Agg, 8
Which dtecdion dd the paatiigg Box chonge postion hore fine &. S bech
Port Fin stewark perpertioon 4 L.3g kn
Object-relation grounding may, glottsl rate
Symaci axmerration
pe oragttatmm eustoont bottle 1gght
ut
191
Iumusterd bottle or du lett op
Irage??
* thn rag futrr the rabals cabe or heras?
Whinh as fytther From wewwr inage?
A. Fort Butata
vag
A. bottle diliiiciuu
7 GROUPS
Mental rotation
Perspective taking
The The 135 segmee codiaiss Witch viw hoor Bw for magan 1A, donu ba rwn uramaiarri
Score Scaction
Snansformation
WWiter wiviing from
the clmpis front and vine whtn winei fooe the bee vieve??
wive B, rosl loep lake ton Fo Nel!
Pastal
wwich obard sppears A. dana auastte
Canonical view selection
360
5PW
Sekcc mage thetber atons Ph staect fom the left
sdeat R, Colk rod livky tike tran the ba5k?
When viewig the lett object
reiut closer and pudting for farther visa frim
No
on Builet?
when
2739 SAMPLES
A read cats G.axtihos ESAE
Figure 1: Overview of SPINBENCH task design across seven task groups. Representative subtasks illustrated for each group with simplified question wording for clarity. In the released benchmark, all queries include explicit frame-of-reference definitions to avoid ambiguity. Human face data are sourced from the Stereo Face Database Fransens al. (2005) and are licensed for research use only.
ments via reinforcement learning, they still exhibit limited transfer of these gains to spatial reasoning tasks (Yin et al., 2025). This calls for a structured diagnosis of: (1) what specifically breaks down in VLMSs spatial reasoning, and (2) how such reasoning can be systematically evaluated.
Our approach is inspired by foundational insights in cognitive science. Early behaviorist theories treated thinking as verbal behavior (Skinner, 1957), but classic mental rotation experiments (Shep- ard & Metzler, 1971) demonstrated that spatial cognition often depends on analog, imagery-based processes contssuuusco imagistic simulations that go beyond linguistic representations. These in- sights motivate the central question: Can VLMs engage in such imagery-based spatial reasoning, or are they limited to symbolic and linguistic associations?
To address this, we introduce SPINBENCH, cognitively grounded, diagnostically structured bench- mark as shown in Fig. 1. Our design is informed by both psychological paradigms and system-level considerations. SPINBENCH emphasizes progressive structure, cognitive fidelity, and controlled variation for diagnostic value. Our progression of tasks reflects increasing spatial complexity and scale (Hegarty et al., 2006); At the low level, we assess single-object perception tasks such as object identity matching, canonical view selection, mental rotation, and dynamic translation/notation:; At the higher level, we evaluate object-elation grounding and perspective taking in cluttered, multi- object scenes. Our most challenging task, multi-object cluttered scene perspective taking, requires models to integrate subskills from all prior tasks, making it a holistic probe of spatial cognition. We include both real-world and photo-realistic synthetic data across diverse domains (e.g., household objects, vehicles, human faces), ensuring validity while maintaining evaluation rigor. Each task type carefully designed to evaluate specific spatial skills and is embedded within a controlled variation regime: we manipulate semme-of-refernce (FoR) (Zhang et al., 2025c), introduce premise- based question structures, apply syntactic and symmetrical augmentations, and vary the number of visual inputs (e.g., single, triplet, quartet). These tasks serve as interpretable bridges from raw perceptual features to fundamental spatial concepts and then to challenging spatial reasoning.
Together, SPINBENCH provides an interpretable and rigorous framework for diagnosing the spatial reasoning capabilities of modern VLMs and for understanding the role of rotation as a window into 3D spatial understanding. Our empirical analysis reveals key failure modes in VLM spatial reasoning: persistent egocentric bias, difficulty with rotation and viewpoint changes, inconsistencies in handling symmetry, and failures in linguistic-only spatial inference. We also observe diverse scaling behaviors across tasks and limited correlation with existing benchmarks, suggesting that SPINBENCH offers novel and complementary diagnostic insights into VLM spatial competence.
2

---

Published as a conference paper at ICLR 2026
2
RELATED WORK
Spatial reasoning benchmarks A wide range of benchmarks have been proposed to evaluate the spatial reasoning abilities. Early diagnostic datasets like CLEVR (Johnson al., 2017) intro- duced synthetic, rendered scenes with simple 3D shapes. Recent spatial reasoning benchmarks for vision-language models have explored diverse aspects of spatial cognition. Some, such as Mind- Cube and VSI-Bench (Yin et al., 2025; Yang et al., 2025b), emphasize cognitive mapping, how models represent and track spatial information across scenes. SpaCE-10. SPHERE, and 3DSR- Bench (Gong et al., 2025; Zhang et al., 2024; Ma et al., 2025a) define range of atomic spatial skills (c.g.. counting, height, orientation). yet often lack controlled variation in perspective, refer- ence frame, multi-frame reasoning. BLINK (Fu al., 2024) highlights perception-level gaps in multimodal models, and NewsSpttal--Bence (Li et al., 2025a) focuses on viewpoint-dependent localization. MulSeT (Zhang al., 2025b) covers distance, occlusion, and viewpoint-dependent localization with synthetic data. Meanwhile, OmniSpatial, 3D-PC and SPACE (Jia et al., 2025a; Linsley et al., 2024; Ramakrishnan et al., 2024) draw from cognitive psychology to design spatial tasks, but sometimes entangle spatial reasoning with functionality and physical commonsense or are limited to abstract 2D plane geometry. Our tasks are carefully designed to isolate spatial reasoning by controlling for distractors, motion dynamics, reference frame shifts, and multi-image input for- mats. We incorporate both real-world and photo-realistic synthetic data to ensure domain diversity and real-world relevance. Instead of emphasizing task comprchensivennss, SPINBENCH offers di- agnostic value by introducing fine-grained control over key spatial factors such as premise structure, symmetry, and syntactic variation. As summarized in Tab. 1, our benchmark uniquely combines pro- gressive task structure, cognitive grounding, and controlled variation. For quantitative evidence that SPINBENCH targets spatial skills that differ from prior spatial VLM benchmarks, we refer readers to Appendix B.3, where we provide full correlation analyses showing weak correlation and task-level orthogonality with prior benchmarks.
Bendwwork
Reforeaee
Preesie Var. Symeeeii
Symorti Var.
Melishengg Yarks
CLEVR at al (207))
cahis ziool lndov
55%
BLINK hour at (2000
3.5t
SpaC 10 Goeget (025))
1050hook atal (3055))
viei
sentat daas ou (204)
X
NaCOCC
Spen La co0che)
36COCO
X
X
Table 1: Benchmark comparison highlighting the controlled structure and diagnostic focus of SPIN- BENCH. Our benchmark supports reference frame variations, premise-based variations, symmetric and syntactic variations, and multi-image spatial reasoning across both real and synthetic domains.
Spatial reasoning models To improve spatial reasoning in VLMs, recent work has explored 31) abstractions and finetuning. Methods like SpatialReasoner (Ma et al., 2025b), SSR (Liu et al., 2025) and APC (Lee al., 2025) use explicit 3D representations for perspective-awars reasoning. Others, such as MetaSpatial (Pan & Liu, 2025), Embodied-R (Zhao et al., 2025), Spatial LLM (Chen et al., 2024), and SVQA-RI (Wang & Ling, 2025), adopt reinforcement learning or large-scale pretraining to enhance spatial understanding across 2D and video data. Despite progress, purely linguistic approaches remain limited, humans rely on structured, often non-verbal representations to reason about space, motivating models that move beyond language-based reasoning alone.
DATASET AND BENCHMARK RECIPES
3.1 DIAGNOSTIC APPROACH TO SPATIAL REASONING
SPINBENCH is designed around the core challenge of perspective taking: reasoning about how scenes and object relations change under viewpoint transformation. Perspective taking is a highly integrative ability as it requires recognizing objects across views, grounding their relative positions, and mentally simulating their transformations. To better diagnose model strengths and weaknesses, SPINBENCH decomposes this advanced reasoning evaluation into a set of targeted diagnostic cat- egories. Each category represents a fundamental spatial reasoning ability that supports perspective taking, such object identity recognition, relation grounding, translation, and rotation. Together,
3

---

Published as a conference paper at ICLR 2026
Task Categories
Data Domains
Task Categories by Data Domain
Irteigen (35.7%) Objects (2.09)))
Fores (2.4M) Cars ( APg
DG
nunbe Surglts
Mental rusatin (60%)
Mently
30
a00
Objech Raiaton Grounding (2.24)
Pesspotive taking (2.00)
EDE
220
Dynami: Retation (12.94)
Somiddy Meotiig
Digut Grensing
porspctiive taking
Cantrical Duleciion
Trnensc Rotalie
Dynomee Yaralllius Rotabron
Figure 2: Distribution of SPINBENCH tasks across seven spatial reasoning categories and four visual domains. Right: Task breakdown by domain.
these tasks allow us disentangle where current vision language models succeed, where they fail, and how these skills compose in the perspective taking setting. To minimize confounds, all tasks are defined in a horizontal 2D plane. Vertical relations (e.g., above/below) and height differences are excluded, and viewpoint changes are restricted to horizontal orbits around the scene.
3.2 TASK CATEGORIES AND DESIGN RATIONALE
The seven categories below are organized so that simpler diagnostic abilities scaffold toward the most demanding task: perspective taking. Representative examples for each category are summa- rized in Figure Task category details are the Appendix A.3.
1. Identity Matching Evaluates whether models can consistently recognize the same object across different viewpoints. This ability a prerequisite for cross-view reasoning, ensuring models can track object identity before more complex spatial inference.
2. Object-Relation Grounding Tests understanding of object-relative configurations within a single static image, including directional relations (left/right, front/behind) or distance relations (near/far) between two objects. This isolates spatial grounding from temporal or multi-viww demands, providing a controlled measure of static scene interpretation.
3. Dynamic Translation Assesses reasoning about linear object displacement over time. Given two temporally ordered frames of the same object, models must identify whether it moved left, right, front, or back relative the viewer. By excluding rotation, this cate- gory isolates translational understanding from other motion cues.
4. Dynamic Rotation Focuses specifically on rotational transformations. Models are given two images of an object before and after in-place rotation and must determine the rotation direction (e.g., clockwise vs. counterclockwise defined from a top-down view). Restricting the task to single rotated object avoids background or displacement confounds, allowing fine-grained analysis of rotational reasoning.
5. Canonical View Selection Examines whether models can map objects across canonical viewpoints. Given reference view (typically the front), models must select the correct candidate from alternative perspectives (left, right, back). This setting avoid the complexity of multi-object scenes.
6. Mental Rotation
Tests whether models can mentally simulate object transformations. Given object and specified degree and direction of rotation, models must select the correct resulting configuration. This requires internal spatial visualization and supports analysis of whether models can simulate transformations beyond what is directly observed.
7. Perspective Taking The centerpiece of SPINBENCH, parkpective-takingg tasks require rea- soning about entire scenes under viewpoint changes. Two subtypes are included: (S) se- lecting the correct scene image from a new perspective, and (T) predicting how object re- lations transform under perspective shifts. This category integrates all diagnostic abilitics and probes compositional spatial reasoning in its most demanding form.
3.3 DATASET COMPOSITION AND DOMAIN COVERAGE
SPINBENCH combines one simulation sinulation-generated synthetic dataset with three real-world datasets, chosen to test spatial reasoning generalization across diverse visual domains and object categories. A
4

---

Allocentric and Egocentric Reference Reference frame ambiguity is a common source of error in pretrained models, arising because natural language often leaves the frame of reference implicit. Humans flexibly switch between defaults (e.g.. egocentric vs. allocentric) depending on context, but models may struggle without explicit cues. Our face rotation tasks directly test this by presenting identical transformations under two interpretations: the viewer's perspective (e.g.. "turn left" as seen by the observer) versus the object's own perspective (c.g., "turn left" as for the person). This contrast Published as a conference paper at ICLR 2026
detailed breakdown of dataset composition, sampling strategies, and annotation pipelines is provided in the Appendix A.
Infinigen Scenes We generate indoor environment table-top multi-object synthetic scenes using Infinigen Raistrick et al. (2024) in the Isaac Sim environment NVIDIA, with objects drawn from the YCB dataset Calli et al. (2015; 2017). Randomized object selection, place- ment, and lighting yield diverse yet controlled settings. Data generated for three task categories: object-relation grounding, dynamic translation, and cluttered scene perspective taking. For the perspective taking, provide occlusion and no-occlusion variants to probe reasoning under visual ambiguity.
ABO Objects We sample household items from the Amazon Berkeley Objects (ABO) dataset Collins al. (2022), which provides high-quality 3D models of real commercial products. Objects include 360 views (72 images at 5 intervals) with diverse geometries and textures. We select geometrically structured objects and exclude highly symmetrical cases to avoid ambiguous rotation or relation judgments.
Cars Vehicle rotation sequences drawn from the Multi-View Car Dataset Ozuysal et al. (2009), which contains 20 cars imaged every 3-4 degrees during a full 360 rotation. Cars are ideal for viewpoint-dependent reasoning due to their strong canonical orienta- tions (front, back, side views). Since degree annotations are not provided, we sample and label images at 45% intervals ensure consistent angular coverage.
Faces Human faces are sourced from the Stereo Face Database Fransens et al. (2005), con- taining 100 individuals captured in 8 distinct poses. Faces pose biologically relevant chal- lenges and require distinguishing viewer- versus object-centered reference frames. Their natural asymmetry (left vs. right profiles) enables unambiguous evaluation of perspective- taking.
3.4 CONTROLLED VARIATIONS
SPINBENCH is designed with fine-grained, controlled variations to evaluate how models handle allocentric and egocentric reference, integrate visual and linguistic information, and model reasoning consistency with symmetric and syntactic variations, providing a diagnostic lens for identifying systematic biases, inconsistency, modality weaknesses. Detailed variations and examples are provided in the Appendix A.2 and A.4.
reveals whether models exhibit systematic biases toward particular frames or can adapt to contextual cues. In domains where objects lack intrinsic orientation, all relations are defined from the viewer's (camera) perspective to ensure consistency.
Consistency via Data Augmentation To probe reasoning stability, we systematically generate equivalent variants of spatial relation tasks using two augmentation strategies: (i) Symmetrical aug- mentation: Logically equivalent variants are created by flipping relations and answers (e.g.. from "Which object on the left?" to "Which object on the right?"). This ensures models maintain consistent reasoning under symmetrical transformations, (ii) Syntactic augmentation: Questions reformulated while preserving meaning (e.g., "Which object is on the left?" - "Is A on the left right of B?"). This tests whether models rely surface phrasing demonstrate robust spa- tial understanding. Augmentations are applied across static (left/right, near/far, front/behind), with combined variants yielding comprehensive test sets for consistency evaluation.
Visual vs. Linguistic Failures variants. In the with premise condition, the spatial relation (c.g., "A is to the right of B in the front view"" is explicitly provided in the prompt, while in the without-premisse condition, models must
To disentangle sources of error, introduce premise- besed task
5

---

Published as a conference paper at ICLR 2026
infer relations solely from the image. Comparing performance across conditions reveals whether failures stem from visual grounding difficulties or from applying geometric reasoning when the premise is known.
EVALUATIONS
4.1
EVALUATION SETUP
Evaluated models We evaluated 43 vision-languns models spanning both proprietary and open- source models to assess spatial reasoning capabilities across diverse model scales and designs. We included 7 proprietary VLMs: GPT-5, 04-mini, GPT-4o, GPT-4.1 OpenAl et al. (2024), Gemini 2.5 Pro Comanici (2025), Claude Sonnet, and Claude 3.5 Haiku. For open-source models, our evaluation covered major model families, model sizes ranging from IB to 38B, resulting in 33 models: Intern VL2.5 (IB-8B) Chen et al. (2025b), Intern VL3 (IB-38B) Zhu et al. (2025), In- Wang al. (2025), Qwen2- VL (2B-7B) Yang et al. (2024a), Qwen2.5-VL (3B-32B) Qwen et al. (2025), Qwen3-VL (4B-30B) Yang et al. (2025a). Gemma-3 models (4B- (7B) Li et al. 27B) Team al. (2025), LLaVA-interleave Li et al. (2024b), LLaVA-OneVVsiie (2024a), Molmo-7B Deitke et al. (2024), MiniCPM-V-2.66 V-2.6 Yao et al. (2024). Phi-3.5-vision Abdin et al. (2024). We also include physical or spatial domain-specific models, including SpaceQwen2.5- VL Jia et al. (2025b), and three spatial reasoning models: SpaceOm Jia et al. (2025b), Space- Thinker Chen et al. (2024), and Cosmos-Reasee Reasonl NVIDIA al. (2025). We included CoT variants for 3 specialized spatial reasoning models (Cosmos-Reasee et al. (2025b), Space Chen et al. (2024)) to assess the impact of explicit linguistic reason- ing on spatial task performance. Proprietary models were evaluated via official APIs. Open-source models implementation details are in Appendix D.
tern VL3.5 (IB-38B)
NVIDIA et al. (2025), SpaceOm Jia
Evaluation metries We employ three complementary metrics assess model performance. Raw accuracy measures the proportion of correctly answered questions in all evaluated questions. Co- hen's kappa (x)) (Cohen, 1960; Coenen, 2014) provides chance-corrected accuracy measure that accounts for varying option cardinality, enabling fair comparisons across different tasks. To evaluate reasoning stability, we introduce Pairwise consistency, which calculates the average of symmetric consistency rates across pairs of questions and their augmentations, measuring whether models pro- duce identical outcomes (both correct or both incorrect) for logically equivalent questions.
4.2 RESULTS
Overall performance Figure 3 presents the overall performance of 43 VLMS across 23 grouped task variants, organized under spatial reasoning categories, and reveals a clear performance gradi- ent across spatial reasoning categories. Object relation grounding emerges as the easiest category, with many models achieving > 0.6, indicating reliable extraction of basic spatial relations (e.g., left/right, front/behind) from single images. Identity matching displays bimodal pattern: smaller models perform near chance, while larger models reach near-perfect accuracy, suggesting emer- gent scaling ability. Dynamic spatial reasoning, especially tasks involving rotation, shows substan- tial difficulty. Mental rotation and perspective taking generally yield the near chance overall scores, with most models performing at or below chance, underscoring the absence of robust internal repre- sentations for rotational transformations, Rankings of model overall accuracy averaged across tasks and model pair-wise consistency are shown on the left side of Figure 4. The top proprietary model is gpt-5, which ranks first in both overall accuracy and consistency, while the top open source model is VLte3VLL-33BB which ranks third in overall accuracy and second in consistency. Notably, the lead- ing model in overall accuracy, gpt-5, and the second strongest model, gemini 2.5 pro, also rank first and second on mental rotation and achieve the second and third highest performance taking. This links overall success to competence on the most challenging tasks and highlights that models excelling in complex, compositional viewpoint reasoming also perform strongly on simpler diagnostic tasks. More detailed results, including raw accuracy and ungrouped performance, are provided in Appendix B. Fig. 32, 33, 34.
perspective
Consistency evaluations As shown in Figure 4, models exhibit severe inconsistencies in logically equivalent spatial queries, revealing fundamental gaps in spatial reasoning. While top performers

---

Published as a conference paper at ICLR 2026
1%
CB
1.0
(randk 200
13
58
321
541
05
533
15
0.8
irentt 5/6
1.1
111
1.6
1.8
442
6.11
1.00
4.5
105
1.
131
4.1
1.3
031
18
011
1.5
513
051
50
1.8
541
634
045
0.1
011
Spa de
041
031
944
Tuiks
Cohen's k(=0; below chance accuracy, D: chance accuracy, 1:09 accuracy)
Figure 3: Performance heatmap of 43 VLMs across 26 grouped task variants, organized under 7 spatial reasoning categories. Cohen's kappa values (x) measure chance-adjusted performance, where indicates chance-level and x = 1 perfect accuracy. 3 chain-of-thouggh (CoT) variants of space reasoning models are included for comparison.
Condizuncy Condution (Prassont - 0.851. p 0.000001
Model farking by overal Accuracc (Co ers based on Corisitonyy %)
100
Jonn
Mode Bancing by Conceimnty Percenaage (Colon baved in Consstency 75)
Ovcral Accuracy (%)
Figure 4: Strong correlation between spatial reasoning accuracy and consistency across vision- language models. Left: Model rankings by overall accuracy (top) and pair-wise consistency per- centage (bottom), with colors indicating consistency levels. Right: Scatter plot revealing robust positive correlation (Pearson r 0.874, p 0.05) between the two metrics.
like gpt-5 achieves 97.1% consistency, most models fail dramatically, with bottom performers be- low 30% consistency. The strong correlation (r = 0.891, < 0.05) between overall accuracy and consistency suggests these failures stem from incompetent spatial reasoning. Models that cannot maintain "A left of B" equals "B right of A" equivalency lack genuine spatial understanding. Al- though overall accuracy and consistency strongly correlate, the differences among top models show that consistency alone is not sufficient. gemini 2.5 pro achieves the second highest overall accuracy but only the fifth highest consistency, whereas Intern VL3-38B attains the second highest consis- tency yet ranks third in accuracy, trailing gemini 2.5 pro by 2.6% points. This non-linearity in the upper right corner of Figure+ 4 demonstrate that high consistency is only the first requirement: once models reach very high levels of consistency, accuracy can still diverge substantially. Strong spatial reasoning therefore requires not only maintaining logical coherence across equivalent queries but also consistently selecting the correct answer. At lower performance levels, better consistency can
7

---

Published as a conference paper at ICLR 2026
Performance improvement from CoT reasoning across models and tasks. Delta reflects the change in Cohen's K score. Bolded values indicate the task with the greatest improvement per model, and gray-highlighted cells indicate negative performance improvement.
Table 2:
SpaceOm((B)
SpaceThinkerrge
Dosmos-Reasont
Task
Baseline
CoT
Baseline
CoT
A
Baseline
CoT
A
Object-relation grounding
0.332
0.493
+0.162
0.185
0.393
+0.208
0.569
0.649
+0.080
Identity mutching
0.088
-0.05
0.217
+0.074
Dynamic
0.000
0.064
+0,064
0.000
0.077
+0.077
0.013
0.449
+0.436
Car canonical view selection (back)
0.775
0,700
-0.075
0.625
0.700
+0.075
1.000
1,000
+0.000
ABO canonical view selection (hack)
0.000
0.167
+0.167
0.076
0.045
-0.030
0.424
0.273
-0.152
Perspective-taking (T) w/ premise (back)
0.050
0,400
+0.350
0.350
0.450
+0.100
0.000
0.600
+0.600
Perspectiv-ttkkngg (T) we premise (back)
-0.250
0,000
+0.250
-0.250
-0.550
-0.300
-0.350
0,300
+0.650
cenpective-Laking (I) w/ premise (L&R)
-0.063
0.102
+0.165
0.075
0.407
+0.331
0.165
0.270
+0.105
Perspective -taknng w/ premise (I.&R)
0.138
0.133
-0,005
0.137
0.066
00071
-0.115
0.016
+0.130
indicate higher accuracy, but among top-performing systems, consistency alone does not guarantee
reliable spatial reasoning. Our findings underscore the difference between stochastic inconsistency and consistent but systematically incorrect behavior. Detailed breakdowns of augmentation strategy analysis, consistency pattern distribution, and comprehensive performance metrics can be found in Appendix B.2.
Table 3: Cohen's kappa (R) values for dynamic rotation tasks in the face domain reveal strong view-centric bias. Models that perform best on the egocentric task (face cotation_viewer perform worst on the allocentric variant
Model
Allocentric (Iace_rctale wetoosrttatioo.ooo Egocentric (face pww.ttttiooooo Lon, viewez)
Gemini 2.5
0.66 (want)
0.94 (best)
Molmo-7u 7B 0-024
0.66
0.94
LL3-38
Qwen2 5-L---22--L----ttttnn
.0:49
0.29
Biased perspective Models exhibit a strong bias toward the viewer's perspective in dynamic rota- tion tasks, even when the question explicitly requires an alternate viewpoint. As shown in Table 3, the oopperforrmnng models the egocentric task are the worst the allocentric version. This asymmetry suggests an inductive bias toward egocentric interpretation, likely influenced by training data dominated by first-perrs first-person visual descriptions. Such bias limits the models' ability to generalize across frames of reference and poses challenges for applications like robotics and navigation that re- quire flexible spatial reasoning. Within the GPT family, we observe a trend of improvement. gpt-4o, gpt-4.1, 1, and especially gpt-5 exhibit steadily stronger performance on the egocentric task, with gpt- additionally showing marked gains in both egocentire and allocentric reasoning. Notably, gpt-5 ranks best on the allocentric task (x = 0.55, 0.78 accuracy) and second-bess on the egocentric task (x = 0.63, 0.814 accuracy). While current vision-language models generally default to egocentric interpretations, gpt-5 shows meaningful step toward reliable reasoning across frames of reference.
Visual failures or linguistic failures Perspective-taking (T) tasks test whether models can reason about how object relations transform under viewpoint shifts. In the premise-based variant, all rel- evant spatial relations are explicitly stated in the prompt, so no visual grounding is required. Yet many models still fail, revealing that errors persist even when the task reduces to purely linguis- tie reasoning over spatial abstractions. As shown in Figure 5 (a), four models (Intern 1.3_5-1B, 202_m-L2255-213 VnternVL3-11 IB, gemma-3-4b-it) consistently select the wrong answer with accu- racy below 0, indicating systematic misinterpretation of reference frames. At the same time, seven models, including gpt- souude-sonnet-4 and several large InternVL variants, achieve near perrect accuracy (>95%), showing that this reasoning is learnable. Overall, 16 of 39 models (41 %) perform below chance, underscoring that even abstracted the linguistic level, spatial concepts are not ro- bustly encoded or manipulated by most VL.Ms.
Does thhoughf- reasoning help spatial reasoning? We evaluate the effect of CoT prompt- ing three models: SpaceOm, SpaceTThinke and Reaunos-ee (Table 2). Results show substantial but heterogeneous gains. Cosmos-Reasee Reason1-7B benefits most, with an average improve- ment of +0.221 across tasks and gains in 7 of 9 categories. Its largest boosts occur on perspective-
8

---

Published as a conference paper at ICLR 2026
Perspectivesaadnnge with Accurney Acorall
Subtype Level: Longer Human Time Lower VLM Accuracy
Pearson 0,7680 0.0000) Spaarman 0.7748 0,0030)
Pearson -0.5399 (p 0.00004), Spearman P -0.5002 (p 0.00054)
0:80
Qu
0.75
Human Reepome Time (encande)
Livey
22
000000
25
23
0.64
25
0.50
0
0.49
0.40
0.5 0. Accumcy
0.4 0. 07
Perspaciie-skkinggT
Accuracy (Avenage scross models)
Figure 5: (a) Scatter plot comparing Perspective-taking[] with premise accuracy against overall accuracy for each model, demonstrating that linguistic spatial reasoning failures are correlated with general model competence. Models are color-coded by Perspective-taking[]] with premise accu- racy. (b) Scatter plot showing the relationship between VLM accuracy (x-axis) and human response time (y-axis) across 51 task subtypes.
taking tasks, +0.650 and +0.650 perspective-taking with and without premise (back), in- dicating that CoT is especially effective for spatial transformations requiring explicit reasoning steps. SpaceOm improves moderately (+0.118 average), particularly on object-relation grounding (+0.162). SpaceThinker shows the weakest effect (+0.052 average), including a sharp drop (-0.300) on perspective-taking without premise (back). Across all models, object-relation grounding consis- tently benefits from CoT, while canonical view selection tasks show mixed results. Overall, CoT prompting provides a more significant advantage for complex, multi-step spatial transformations, with larger models demonstrating more improvement. These patterns suggest that CoT alone may not reliably address errors rooted in perceptual or spatial misinterpretation, which aligns with emerg- ing directions Yang et al. (2025d) that perform reasoning directly within the visual embedding space rather than relying solely on textual COT.
The Effect of Different Inputs To investigate the impact of explicit spatial cues and input image resolution on model performance, we conducted two variations in addition to our standard setting: the Depth Input setting and the High Resolution Input setting. The Depth Input setting provided a depth map, generated by DepthAnything Yang et al. (2024b), which was horizontally concatenated with the original image. This was intended to explicitly supply depth cues to the VLMs without altering the input image number. However, as shown in Figure 6, this modification consistently led to a decrease in overall accuracy across nearly all evaluated models. A possible explanation is that current VLMs are heavily tuned natural RGB images. Also, simply appending depth map changes both the distribution and geometry of the input; without any architectural adaptation or fine-tuning, the models may fail to treat depth as a structured cue. In the High Resolution Input setting, we use the original images instead of the downsampled version used in the main experiments (max size 256). Despite nearly 10x more pixels, we observe no systematic accuracy gains. This is consistent with the fact that most VLM backbones downsample early in the visual encoder and were pre-trained at relatively modest resolutions. In our evaluations, extra pixels provide limited benefit without corresponding changes in architecture or training.
Human response time and VLM accuracy correlation We further validate that SPINBENCH reflects genuine spatial reasoning difficulty by comparing human and model performance. As shown in Figure 5 (b), task subtypes that required longer human response times also elicited lower VLM accuracy, with a significant negative correlation (r = -0.4,, p < 0.05). This alignment indicates that tasks harder for humans also systematically harder for models, supporting that SPINBENCH serves as a diagnostic benchmark whose progressively structured tasks reveal core spatial reasoning challenges. More details the human evaluations setup and results are provided in Appendix C.
Scaling laws and emergent capability Overall performance improves with model scale, but scal- ing patterns differ sharply across task types (Figure 7). Object relation grounding tasks (c.g., left/right, front/behind) improve smoothly and monotonically across model families. In contrast,

---

Published as a conference paper at ICLR 2026
Overall Accuracy Across Settings
Accuracy
0.6
original
0.4
depth high resolution
0.2
InterVVL2 2.11-
Irtemit22 5-28
Interv 5-48
ertamit Internit et
5-20
Internt 11
Internit
incervi 5-1
intervi entemit netemvi 5-3t0
5-18
Ietonv, 5-440
Intem 3-00
Mini
Heino 7D oote
Pric S-wisenn Cuisenn-raind itarrict
Ownn2 qwee ve inaruct
owent 5-42 inarree oweet 5-1 28 irstrved
Vt 7B snstrucd
gomma 12b-a
Demmis 27b-1
germera 2--------
Figure 6: Overall Accuracy Across Settings. We conducted evaluations in 2 additional settings: the depth setting, and the high resolution setting.
Overall Accuracy (x)
Object Relation Grounding
Identity Matching
Dynamic Translation
0.5
140
0.8
528
888
1.0
40
1020
148
:08
0.8
1.0
Cohon's Kappa
0.4
>0
278
129
0.6
20
0.6
125
0.4
0.4
18
0.2
0.2
3p
126
0.2
20/
278
0.0
1ntemVL33
70
122
Owen2 5-M
0.0
0.0
-0.1
10
10
107
10'
10a
10a
10a
10"
Model (B)
Model
Model
Model
Figure 7: Scaling laws across spatial reasoning tasks. Each line shows Cohen's x (chance-adjusted accuracy) with respect to model size for four model families. While overall performance increases gradually with scale, different task types show distinct scaling patterns.
identity matching exhibits clear emergence: smaller models remain at chance, while larger mod- els (7B-8B+) achieve near-perfect accuracy. This non-linear jump suggests that cross-image 3D abstraction only becomes possible once models reach sufficient capacity, consistent with emergent abilities reported in language models Wei et al. (2022). A similar but more gradual emergent trend appears in dynamic translation (e.g., object moving left/right). These distinct scaling behaviors highlight the diagnostic value of our fine-grained benchmark: exposing clear gaps between small and large models and enabling diagnosis of scaling laws in spatial reasoning.
5 CONCLUSION AND LIMITATIONS
We present SPINBENCH, a cognitively grounded diagnostic benchmark for evaluating spatial rea- soning in vision language models through fine-grained, controlled tasks targeting geometric trans- formations and viewpoint changes. By decomposing complex perspective taking into interpretable subskills, SPINBENCH facilitates precise diagnosis of model limitations. Our evaluation 37 VLMs reveals systematic weaknesses, including consistent reference-frame bias, failures in rotation under- standing, and linguistic spatial inference, alongside diverse scaling behaviors and emergent capa- bilities. These findings suggest that different aspect of spatial reasoning are not uniformly learned and often remains underdeveloped even in advanced models. Human evaluation further validates the benchmark, showing strong correlation between human response times and VLM accuracy, suggesting that SPINBENCH captures genuine cognitive difficulty shared across humans and mod- els. SPINBENCH goes beyond scorekeeping by providing diagnostic lens spatial competencies, offering conceptual clarity about what aspects of spatial reasoning VLMs do and do not master, and guiding the development of multimodal foundation models. These diagnostic insights directly reasoning or rotation understanding
actionable for embodied AI, where failures in reference-frann can lead to breakdowns in navigation, manipulation, and other safety-critical tasks, key limitation is that we do not yet cover other important spatial concepts such as containment, support, or vertical relations (e.g.. "in," "on," "above").
10

---

Published as a conference paper at ICLR 2026
Ethics Statement This work includes human evaluations conducted to measure benchmark diffi- culty. All participants were adults who gave informed consent, and their data were collected and analyzed anonymously. The study followed institutional ethics guidelines and posed no foresce- able risks to participants. Beyond this, our research uses only public or synthetic datasets under appropriate licenses. While failures in spatial reasoning can have implications for safety-critical systems, SpinBench is intended solely as diagnostic tool to improve transparency and safety in model development. We affirm full adherence the ICLR Code of Ethics throughout this work.
Reproducibility Statement We have taken steps to ensure that our work can be reproduced. The design of SpinBench, including task categories, dataset composition, and controlled variations, described in detail in Section 3 and Appendix A. Experimental settings, model lists, and evaluation metries are provided in Section 4 and Appendix D, along with additional results in Appendix B and C. All datasets used either publicly available generated using documented pipelines, and details of sampling and annotation are included in the appendix. To further support reproducibility, provide anonymous project website with benchmark resources and plan to release code and data generation scripts in the near future.
ACKNOWLEDGMENTS
This work was fully supported by Mitsubishi Electric Research Labs (MERL).
REFERENCES
Marah Abdin, Jyoti Aneja, Hany Awadalla, Ahmed Awadallah, Ammar Ahmad Awan, Nguyen Bach, Amit Bahree, Arash Bakhtiari, Jianmin Bao, Harkirat Behl, Alon Benhaim, Misha Bilenko, Johan Bjorck, Sébastien Bubeck, Martin Cai, Qin Cai, Vishrav Chaudhary, Dong Chen, Dong- dong Chen, Weizhu Chen, Yen-Chun Chen, Yi-Ling Chen, Hao Cheng, Parul Chopra, Xiyang Dai, Matthew Dixon, Ronen Eldan, Victor Fragoso, Jianfeng Gao, Mei Gao, Min Gao, Amit Garg, Allie Del Giorno, Abhishek Goswami, Suriya Gunasekar, Emman Haider, Junheng Hao, Russell J. Hewett, Wenxiang Hu, Jamie Huynh, Dan Iter, Sam Ade Jacobs, Mojan Javaheripi, Xin Jin, Nikos Karampatziakis, Piero Kauffmann, Mahoud Khademi, Dongwoo Kim, Young Jin Kim, Lev Kurilenko, James R. Lee, Yin Tat Lee, Yuanzhi Li, Yunsheng Li, Chen Liang, Lars Liden, Xihui Lin, Zeqi Lin, Ce Liu, Liyuan Liu, Mengchen Liu, Weishung Liu, Xiaodong Liu, Chong Luo, Piyush Madan, Ali Mahmoudzadeh, David Majercak, Matt Mazzola, Caio César Teodoro Mendes, Arindam Mitra, Hardik Modi, Anh Nguyen, Brandon Norick, Barun Patra, Daniel Perez- Becker, Thomas Portet, Reid Pryzant, Heyang Qin, Marko Radmilac, Liliang Ren, Gustavo de Rosa, Corby Rosset, Sambudha Roy, Olatunji Ruwase, Olli Saarikivi, Amin Saied, Adil Salim, Michael Santacroce, Shital Shah, Ning Shang, Hiteshi Sharma, Yelong Shen, Swadheen Shukla, Xia Song. Masahiro Tanaka, Andrea Tupini, Praneetha Vaddamanu, Chunyu Wang. Guanhua Wang, Lijuan Wang, Shuohang Wang, Xin Wang, Yu Wang, Rachel Ward. Wen Wen, Philipp Witte, Haiping Wu, Xiaoxia Wu, Michael Wyatt, Bin Xiao, Can Xu, Jiahang Xu, Weijian Xu, Ji- long Xue, Sonali Yadav, Fan Yang, Jianwei Yang, Yifan Yang. Ziyi Yang. Donghan Yu, Lu Yuan, Chenruidong Zhang, Cyril Zhang, Jianwen Zhang. Li Lyna Zhang, Yi Zhang, Yue Zhang, Yunan Zhang, and Xiren Zhou. Phi-3 technical report: A highly capable language model locally on your phone, 2024. URL https:/ / arxiv, www.v.org/aa...oa..........oo.tt wwwp/otg////.oooooo///..o/...e/t 2404 1...11411 14219.
Berk Calli, Aaron Walsman, Arjun Singh, Siddhartha Srinivasa, Pieter Abbeel, and Aaron M. Dol- lar. Benchmarking in manipulation research: Using the aale-cmu-berkela object and model set. doi: 10.1109/MRA222015224444
IEEE Robotics & Automation Magazine, 22(3):36 52, 2015.
5,21
Berk Calli, Arjun Singh, James Bruce, Aaron Walsman, Kurt Konolige, Siddhartha Srinivasa, Pieter Abbeel, and Aaron M Dollar. dall-cmu-berkeled dataset for robotic manipulation re- search. The International Journal of Robotics Research, 3603):261-268. 2017. doi: 10.1177/ 5, 5,
1177 000777/2773332 027836491770077 14. URL https: / doii /ttpp//org.......7/..0/.////////
Boyuan Chen, Zhuo Xu, Sean Kirmani, Brain Ichter, Dorsa Sadigh, Leonidas Guibas, and Fei Xia. Spatialvim: Endowing vasion-language models with spatial reasoning capabilities. In Proceedings of the JEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14455-14465, 2024. 3, 6, 67,67,66 69
11

---

Published as a conference paper at ICLR 2026
Hardy Chen, Haoqin Tu, Fali Wang, Hui Liu, Xianfeng Tang, Xinya Du, Yuyin Zhou, and Cihang Xie. Sft or rl? an early investigation into training ri-like reasoning large rision-languee models, 2025a. URL https:tpp/... aawiv..ivvvvv.///ii//..///iaiav/ org/ abs / 2504 11468. 1, 70
Zhe Chen, Weiyun Wang, Yue Cao, Yangzhou Liu, Zhangwei Gao, Erfei Cui, Jinguo Zhu, Shen- glong Ye, Hao Tian, Zhaoyang Liu, Lixin Gu. Xuehui Wang. Qingyun Li, Yimin Ren, Zixuan Chen, Jiapeng Luo, Jiahao Wang. Tan Jiang, Bo Wang, Conghui He, Botian Shi, Xingcheng Zhang, Han Lv, Yi Wang, Wenqi Shao, Pei Chu. Zhongying Tu, Tong He, Zhiyong Wu, Huipeng Deng, Jiaye Ge, Kai Chen, Kaipeng Zhang, Limin Wang. Min Dou, Lewei Lu, Xizhou Zhu, Tong Lu, Dahua Lin, Yu Qiao, Jifeng Dai, and Wenhai Wang. Expanding performance bound- aries of open-source multimodal models with model, data, and test-time scaling, 2025b. URL https: /apps://axaan http//aaxvvv//. org/ als 2412. 05271.
Wei Chow, Jiageng Mao, Boyi Li, Daniel Seita, Vitor Guizilini, and Yue Wang. Physbench: Bench- marking and enhancing vision-language models for physical world understanding, 2025. URL
https: //tp///aavvvv//aa//////ea..a.waa wwwpxiiivrrrveeooooo /ww.////11.. 14111. 66
Frans Coenen. Normalised accuracy. https: /tt.//iccsss cse. www..iivv.on ttaa.............ass///s/ss.aas.
normal aozmaalssecccccrcyy2--1 14- 5- 30. pdf, 2014. 6
Jacob Cohen. coefficient of agreement for nominal scales. Educational and psychological mea-
surement, 20(1):37-46, 1960. 6
Jasmine Collins, Shubham Goel, Kenan Deng, Achleshwar Luthra, Leon Xu, Erhan Gundogdu, Xi Zhang. Tomas F Yago Vicente, Thomas Dideriksen, Himanshu Arora, Matthieu Guillaumin, and Jitendra Malik. Abo: Dataset and benchmarks for real-world 3d object understanding. CVPR, 2022. 5
et al. Comanici, Gheorghe. Gemini 2.5:
Pushing the frontier with advanced reasoning, multimodal- ity, long context, and next generation agentic capabilities, 2025. URL https: http///axvvvv./// org/ abs/ 2507.257.00 06261.
LMDeploy Contributors. Lmdeploy: A toolkit for compressing, deploying, and serving llm. https: /ttp//ggtttttbtt coom/InterIian 1ndeploy, 2023. 67
Matt Deitke, Christopher Clark, Sangho Lee, Rohun Tripathi, Yue Yang, Jae Sung Park, Moham- madreza Salehi, Niklas Muennighoff, Kyle Lo, Luca Soldaini, Jiasen Lu, Taira Anderson, Erin Bransom, Kiana Ehsani, Huong Ngo, YenSung Chen, Ajay Patel, Mark Yatskar, Chris Callison- Burch, Andrew Head, Rose Hendrix, Favyen Bastani, Eli VanderBilt, Nathan Lambert, Yvonne Chou, Arnavi Chheda, Jenna Sparks, Sam Skjonsberg, Michael Schmitz, Aaron Sarnat, Byron Bischoff, Pete Walsh, Chris Newell, Piper Wolters, Tanmay Gupta, Kuo-Hao Zeng, Jon Bor- chardt, Dirk Groeneveld, Crystal Nam, Sophie Lebrecht, Caitlin Wittlif, Carissa Schoenick, Oscar Michel, Ranjay Krishna, Luca Weihs, Noah A. Smith, Hannaneh Hajishirzi, Ross Girshick, Ali Farhadi, and Aniruddha Kembhavi. Molmo and pixmo: Open weights and open data for state- oh-thc-art vision- languae models, 2024. URL https: : //wrxivvvvv///////////////////./ org/ abs / 2499 17146.
Mohamed Elnoor, Kasun Weerakoon, Gershom Seneviratne, Jing Liang, Vignesh Rajagopal, and Di- nesh Manocha. Vi-lad: Vision-language attention distillation for socially-awe aware robot navigation in dynamic euvironments. arXiv preprint arXiv:2503.0982 2025. 1
Rik Fransens, Christoph Strecha, and Luc Van Gool. Parametric stereo for multi-pose face recog- nition and 3d-face modeling. In Wenyi Zhao, Shaogang Gong, and Xiaoou Tang (eds.), Analysis and Modelling of Faces and Gestures, pp. 124, Berlin, Heidelberg, 2005. Springer Berlin Heidelberg. ISBN 778-3-540-32074, 2,5
Xingyu Fu, Yushi Hu, Bangzheng Li. Yu Feng, Haoyu Wang, Xudong Lin, Dan Roth, Noah A Smith, Wei-Chiu Ma. and Ranjay Krishna. Blink: Multimodal large language models can see but not perceive. In European Conference on Computer Vision, pp. 148-166. Springer, 2024. 3, 69
Ziyang Gong, Wenhao Li, Oliver Ma, Songyuan Li, Jiayi Ji, Xue Yang, Gen Luo, Junchi Yan, and Rongrong Ji. Space-10: 10: A comprehensive benchmark for multimodal large language models in compositional spatial intelligence, 2025. URL https: // arxiv. org/ abs 2006000000 07966. 3, 56.6, 69
12

---

Published as a conference paper at ICLR 2026
Mary Hegarty, Daniel R Montello, Anthony E Richardson, Toru Ishikawa, and Kristin Lovelace. Spatial abilities at different scales: Individual differences in aptitude-tess performance and spatial- layout learning. Intelligence, 34(2):151-176. 2006. 2
Mengdi Jia, Zekun Qi, Shaochen Zhang, Wenyao Zhang. Xinqiang Yu, Jiawei He, He Wang, and Li Yi. Omnispatial: Towards comprehensive spatial reasoning benchmark for vision language models, 2025a. URL https::/arxiv...t wwtpv//.rrga/ab//eeeeeeee abs/ 03135. 3,5, 69
Mengdi Jia, Zekun Qi, Shaochen Zhang, Wenyao Zhang, Xinqiang Yu, Jiawei He, He Wang, and Li Yi. Omnispatial: Towards comprehensive spatial reasoning benchmark for vision language models. arXiv preprint arXhv:2506.03135, 2025b. 6, 67
Justin Johnson, Bharath Hariharan, Laurens Van Der Maaten, Li Fei-Fei, C Lawrence Zitnick, and Ross Girshick. Clevr: A diagnostic dataset for compositional language and elementary visual reasoning. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 2901-2910, 2017.
Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and lon Stoica. Efficient memory management for large language model serving with pagedattention. In Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles, 2023. 67
Phillip Lee, Jihyeon Je, Chanho Park, Mikaela Angelina Uy, Leonidas Guibas, and Minhyuk Sung. Perspective-aware reasoning in vision-language models via mental imagery simulation. arXiv preprint arXiv:2504. 2025. 3, 69
Bo Li, Yuanhan Zhang. Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Ziwei Liu, and Chunyuan Li. Llava-onevision: Easy visual task transfer, abss2408 2408 03326. 6
2024a. URL https:/ arxiv. org/ abs/
Dingming Li, Hongxing Li, Zixuan Wang. Yuchen Yan, Hang Zhang, Siqi Chen, Guiyang Hou, Shengpei Jiang, Wenqi Zhang, Yongliang Shen, Weiming Lu, and Yueting Zhuang. Viewspatial- bench: Evaluating multi pertpiierseeetive spatial localization https://arrn
language models, 2025a. URL
21500. 3, 56, 69
Feng Li, Renrui Zhang, Hao Zhang, Yuanhan Zhang, Li, Wei Li, Zejun Ma, and Chunyuan Li. Llateenextimmterleav:: Tackling multi-image, video, and 3d in large multimodal models, 2024b.
URL htttts:://aaxttt org/ abs////3000 22002 07895. 6,6
Zongxia Li, Xiyang Wu, Hongyang Du, Fuxiao Liu, Huy Nghiem, and Guangyao Shi. A survey of state of the art large vision language models: Alignment, benchmark, evaluations and challenges, 2025b. URL https: arxiv. org/ abs/2501. 02189.
Zhenyi Liao, Qingsong Xie, Yanhao Zhang, Zijian Kong, Haonan Lu, Zhenyu Yang, and Zhijie Deng. Improved visual-spatial reasoning via rl-zero-like training. arXiv preprint Sttv:25044008883 2025. 70
Drew Linsley, Peisen Zhou, Alekh Karkada Ashok, Akash Nagaraj, Gauray Gaonkar, Francis E Lewis, Zygmunt Pizlo, and Thomas Serre. The 3d-pc: benchmark for visual perspective taking in humans and machines. arXiv preprint arXiv:2406.0411 2024. 3
Yang Liu, Ming Ma, Xiaomin Yu, Pengxiang Ding, Han Zhao, Mingyang Sun, Siteng Huang, and Donglin Wang. Ssr: Enhancing depth perception in vision-language models via rationale-guided spatial reasoning. arXiv preprint arXiv:2505. / 12448, 2025. 3, 69
Wufei Ma, Haoyu Chen, Guofeng Zhang, Yu-Cheng Chou, Celso M de Melo, and Alan Yuille. 3dsrbench: A comprehensive 3d spatial reasoning benchmark, 2025a. URL wtt.t:t/t//xiiiii
org//bs/222 2412. 07825. 3, 69
Wufei Ma, Yu- Yu-Cheng Chou, Qihao Liu, Xingrui Wang. Celso de Melo, Jianwen Xie, and Alan Yuille. Spatialreasoner: Towards explicit and generalizable 3d spatial reasoning. arXiv preprint arXiv:2504.20022 2025b. 3, 69
13

---

Published as a conference paper at ICLR 2026
NVIDIA. Isaac Sim. URL /tL.
5, 5,
NVIDIA, :, Alisson Azzolini, Junjie Bai, Hannah Brandon, Jiaxin Cao, Prithvijit Chattopadhyay, Huayu Chen, Jinju Chu, Yin Cui, Jenna Diamond, Yifan Ding, Liang Feng, Francesco Ferroni, Rama Govindaraju, Jinwei Gu, Siddharth Gururani, Imad El Hanafi, Zekun Hao, Jacob Huffman, Jingyi Jin, Brendan Johnson, Rizwan Khan, George Kurian, Elena Lantz, Nayeon Lee, Zhaoshuo Li, Xuan Li, Maosheng Liao, Tsung-Yi Yi Lin, Yen-Chen Lin, Ming-Yu Yu Liu, Xiangyu Lu, Alice Luo, Andrew Mathau, Yun Ni, Lindsey Pavao, Wei Ping, David W. Romero, Mísha Smelyanskiy, Shuran Song, Lyne Tchapmi, Andrew Z. Wang, Boxin Wang, Haoxiang Wang, Fangyin Wei, Jiashu Xu, Yao Xu, Dinghao Yang, Xiaodong Yang, Zhuolin Yang. Jingxu Zhang, Xiaohui Zeng, and Zhe Zhang. Cosmos-reason. From physical common sense to embodied reasoning, 2025. org/ abs/ 2503. 1 1555.. 6, 67
URL
OpenAI, Josh Achiam. Steven Adler, Sandhini Agarwal, Lama Ahmad. Ilge Akkaya, Floren- cia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman. Shyamal Anadkat, Red Avila, Igor Babuschkin, Suchir Balaji, Valerie Balcom, Paul Baltescu, Haiming Bao, Moham- mad Bavarian, Jeff Belgum, Irwan Bello, Jake Berdine, Gabriel Bernadett- Saaadddrtt-Shproo, Christopher Berner, Lenny Bogdonoff, Oleg Boiko, Madelaine Boyd, Anna-Luisa Brakman, Greg Brock- man, Tim Brooks, Miles Brundage, Kevin Button, Trevor Cai, Rosie Campbell, Andrew Cann, Brittany Carey, Chelsea Carlson, Rory Carmichael, Brooke Chan, Che Chang. Fotis Chantzis, Derek Chen, Sully Chen, Ruby Chen, Jason Chen, Mark Chen, Ben Chess, Chester Cho, Casey Chu, Hyung Won Chung, Dave Cummings, Jeremiah Currier, Yunxing Dai, Cory Decareaux, Thomas Degry, Noah Deutsch, Damien Deville, Arka Dhar, David Dohan, Steve Dowling, Sheila Dunning, Adrien Ecoffet, Atty Eleti, Tyna Eloundou, David Farhi, Liam Fedus, Niko Felix, Simón Posada Fishman, Juston Forte. Isabella Fulford. Leo Gao, Elie Georges, Christian Gib- son, Vik Goel, Tarun Gogineni, Gabriel Goh, Rapha Gontijo-Lopes. Jonathan Gordon, Morgan Grafstein, Scott Gray, Ryan Greene, Joshua Gross, Shixiang Shane Gu, Yufei Guo, Chris Hal- lacy, Jesse Han, Jeff Harris, Yuchen He, Mike Heaton, Johannes Heidecke, Chris Hesse, Alan Hickey, Wade Hickey, Peter Hoeschele, Brandon Houghton, Kenny Hsu, Shengli Hu, Xin Hu, Joost Huizinga, Shantanu Jain, Shawn Jain, Joanne Jang, Angela Jiang, Roger Jiang, Haozhun Jin, Denny Jin, Shino Jomoto, Billie Jonn, Heewoo Jun, Tomer Kaftan, Lukasz Kaiser, Ali Ka- mali, Ingmar Kanitscheider, Nitish Shirish Keskar, Tabarak Khan, Logan Kilpatrick, Jong Wook Kim, Christina Kim. Yongjik Kim, Jan Hendrik Kirchner, Jamie Kiros, Matt Knight, Daniel Kokotajlo, tukasz Kondraciuk, Andrew Kondrich, Aris Konstantinidis, Kyle Kosic, Gretchen Krueger, Vishal Kuo, Michael Lampe, Ikai Lan, Teddy Lee, Jan Leike, Jade Leung, Daniel Levy, Chak Ming Li, Rachel Lim, Molly Lin, Stephanie Lin, Mateusz Litwin, Theresa Lopez, Ryan Lowe, Patricia Lue, Anna Makanju, Kim Malfacini, Sam Manning, Todor Markov, Yaniv Markovski, Bianca Martin, Katie Mayer, Andrew Mayne, Bob McGrew, Scott Mayer McKinney, Christine McLeavey, Paul McMillan, Jake McNeil, David Medina, Aalok Mehta, Jacob Menick, Luke Metz, Andrey Mishchenko, Pamela Mishkin, Vinnie Monaco, Evan Morikawa, Daniel Mossing, Tong Mu, Mira Murati, Oleg Murk, David Mély, Ashvin Nair, Reiichiro Nakano, Ra- jeev Nayak, Arvind Neelakantan, Richard Ngo. Hyeonwoo Noh, Long Ouyang, Cullen O' Keefe, Jakub Pachocki, Alex Paino, Joe Palermo, Ashley Pantuliano, Giambattista Parascandolo, Joel Parish, Emy Parparita, Alex Passos, Mikhail Pavlov, Andrew Peng, Adam Perelman, Filipe de Avila Belbute Peres, Michael Petrov, Henrique Ponde de Oliveira Pinto, Michael, Pokorny, Michelle Pokrass, Vitchyr H. Pong, Tolly Powell, Alethea Power, Boris Power, Elizabeth Proehl, Raul Puri, Alec Radford, Jack Rae, Aditya Ramesh, Cameron Raymond, Francis Real, Kendra Rimbach, Carl Ross, Bob Rotsted, Henri Roussez, Nick Ryder. Mario Saltarelli, Ted Sanders, Shibani Santurkar, Girish Sastry, Heather Schmidt, David Schnurr, John Schulman, Daniel Sel- sam, Kyla Sheppard. Toki Sherbakov. Jessica Shich. Sarah Shoker, Pranav Shyam. Szymon Sidor, Eric Sigler, Maddie Simens, Jordan Sitkin, Katarina Slama, Ian Sohl, Benjamin Sokolowsky, Yang Song, Natalie Staudacher, Felipe Petroski Such, Natalie Summers, Ilya Sutskever, Jie Tang, Nikolas Tezak, Madeleine B. Thompson, Phil Tillet, Amin Tootoonchian, Elizabeth Tseng, Pre- ston Tuggle, Nick Turley, Jerry Tworek, Juan Felipe Cerón Uribe, Andrea Vallone, Arun Vi- jayvergiya, Chelsea Voss, Carroll Wainwright, Justin Jay Wang, Alvin Wang, Ben Wang, Jonathan Ward, Jason Wei, CJ Weinmann, Akila Welihinda, Peter Welinder, Jiayi Weng, Lilian Weng, Matt Wiethoff, Dave Willner, Clemens Winter, Samuel Wolrich. Hannah Wong, Lauren Work- man, Sherwin Wu, Jeff Wu, Michael Wu, Kai Xiao, Tao Xu, Sarah Yoo, Kevin Yu, Qiming Yuan, Wojciech Zaremba, Rowan Zellers, Chong Zhang, Marvin Zhang, Shengjia Zhao, Tianhao
14

---

Published as a conference paper at ICLR 2026
Zheng, Juntang Zhuang, William Zhuk, and Barret Zoph. Gpt-4 technical report, 2024. URL https:/ /ttpvv/awvvvvv//.a///i//......aa org/ a /tsss/333332 08774. 6
Kun Ouyang, Yuanxin Liu, Haoning Wu, Yi Liu, Hao Zhou, Jie Zhou, Fandong Meng, and Xu Sun. Spacer: Reinforeing milms in video spatial reasoning, 2025. URL https://arxiv.oo abs/2504 01 01805. 70
Mustafa Ozuysal, Vincent Lepetit, and Pascal Fua. Pose estimation for category specific multiview object localization. In 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 5
778-785, 2009. doi: 10.1
Chenbin Pan, Burhaneddin Yaman, Tommaso Nesti, Abhirup Mallik, Alessandro G Allievi, Senem Velipasalar, and Liu Ren. Vlp: Vision language planning for autonomous driving. In Proceedings of the JEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14760- 2024.
Zhenyu Pan and Han Liu. Metaspatial: Reinforeing 3d spatial reasoning in vims for the metaverse.
arXiv preprini arXiv:2503. /8470, 2025. 3, 69
Qwen, An Yang. Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang. Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Mei Li, Mingfeng Xue. Pei Zhang, Qin Zhu, Rui Men, Runji Lin. Tianhao Li, Tianyi Tang, Tingyu Xia. Xingzhang Ren, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yu Wan, Yuqiong Liu, Zeyu Cui, Zhenru Zhang, and Zihan Qiu. Qwen2.5 technical report, 2025. URL https:/ / /arxiv org/ abs/ 2412. 15115. 1,6
Alexander Raistrick, Lingjie Mei, Karhan Kayan, David Yan, Yiming Zuo, Beining Han, Hongyu Wen, Meenal Parakh, Stamatis Alexandropoulos, Lahav Lipson, Zeyu Ma, and Jia Deng. In- finigen indoors: Photorealistic indoor scenes using procedural generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 21783-21794, June 2024. 3, 5,
Santhosh Kumar Ramakrishnan, Erik Wijmans, Philipp Kraehenbuehl, and Vladlen Koltun. Does spatial cognition emerge in frontier models? arXiv preprint arXiv:2410.06466 2024. 1,3,
Pierre Sermanet, Tianli Ding, Jeffrey Zhao, Fei Xia, Debidatta Dwibedi, Keerthana Gopalakrishnan, Christine Chan, Gabriel Dulac-Aroold, Sharath Maddineni, Nikhil J Joshi, et al. Robovqa: Mul- timodal long-horizon reasoning for robotics. In 2024 IEEE International Conference on Robotics and Automation (ICRA), pp. 645-652. IEEE, 2024.
Roger N Shepard and Jacqueline Metzler. Mental rotation of three-dimensional objects. Science, 171(3972):01-773, 1971. 2
Burrhus Frederic Skinner. Verbal behavior. New York: Appleton Appleton-CenntrryCCoe Century-Croffts, 1957. 2
Daeun Song, Jing Liang, Amirreza Payandeh, Amir Hossain Raj, Xuesu Xiao, and Dinesh Manocha. Vlm-social-nav: Socially aware robot navigation through scoring using vision-language models. IEEE Robotics and Automation Letters. 2024.
Gemma Team, Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, Ramona Merhej. Sarah Perrin, Tatiana Matejovicova, Alexandre Ramé, Morgane Rivière, Louis Rouillard, Thomas Mesnard, Geoffrey Cideron, Jean bastien Grill, Sabela Ramos, Edouard Yvinec, Michelle Cas- bon, Etienne Pot. Ivo Penchev, Gaël Liu, Francesco Visin, Kathleen Kenealy, Lucas Beyer, Xi- aohai Zhai, Anton Tsitsulin, Robert Busa-Fekete, Alex Feng, Noveen Sachdeva, Benjamin Cole- man, Yi Gao, Basil Mustafa, lain Barr, Emilio Parisotto, David Tian, Matan Eyal, Colin Cherry, Jan-Thorsten Peter, Danila Sinopalnikov, Surya Bhupatiraju, Rishabh Agarwal, Mehran Kazemi, Dan Malkin, Ravin Kumar, David Vilar, Idan Brusilovsky. Jiaming Luo, Andreas Steiner, Abe Friesen, Abhanshu Sharma, Abheesht Sharma, Adi Mayrav Gilady, Adrian Goedeckemeyer, Alaa Saade, Alex Feng, Alexander Kolesnikov, Alexei Bendebury, Alvin Abdagic, Amit Vadi. András György, André Susano Pinto, Anil Das, Ankur Bapna, Antoine Miech, Antoine Yang, Antonia Paterson, Ashish Shenoy, Ayan Chakrabarti, Bilal Piot, Bo Wu, Bobak Shahriari, Bryce Petrini,
15

---

Published as a conference paper at ICLR 2026
Charlie Chen, Charline Le Lan, Christopher A. Choquette-Choo CJ Carey, Cormac Brick, Daniel Deutsch, Danielle Eisenbud, Dee Cattle, Derek Cheng, Dimitris Paparas, Divyashree Shivaku- mar Sreepathihalli, Doug Reid, Dustin Tran, Dustin Zelle, Eric Noland, Erwin Huizenga, Eu- gene Kharitonov, Frederick Liu, Gagik Amirkhanyan, Glenn Cameron, Hadi Hashemi, Hanna Klimcrak-Plucinske Harman Singh, Harsh Mehta, Harshal Tushar Lehri, Hussein Hazimeh, Ian Ballantyne, Idan Szpektor, Ivan Nardini, Jean Pouget-Abadie, Jetha Chan, Joe Stanton, John Wi- eting, Jonathan Lai, Jordi Orbay, Joseph Fernandez, Josh Newlan, Ju yeong Ji, Jyotinder Singh, Kat Black, Kathy Yu, Kevin Hui, Kiran Vodrahalli, Klaus Greff, Linhai Qiu, Marcella Valentine, Marina Coelho, Marvin Ritter, Matt Hoffman, Matthew Watson, Mayank Chaturvedi, Michael Moynihan, Min Ma. Nabila Babar, Natasha Noy, Nathan Byrd, Nick Roy, Nikola Momchev, Ni- lay Chauhan, Noveen Sachdeva, Oskar Bunyan, Pankil Botarda, Paul Caron, Paul Kishan Ruben- stein, Phil Culliton, Philipp Schmid, Pier Giuseppe Sessa, Pingmei Xu, Piotr Stanczyk, Pouya Tafti, Rakesh Shivanna, Renjic Wu, Renke Pan, Reza Rokni, Rob Willoughby. Rohith Vallu, Ryan Mullins, Sammy Jerome, Sara Smoot, Sertan Girgin, Shariq Iqbal, Shashir Reddy, Shruti Sheth, Siim Pöder, Sijal Bhatnagar, Sindhu Raghuram Panyam, Sivan Eiger, Susan Zhang. Tianqi Liu, Trevor Yacovone, Tyler Liechty, Uday Kalra, Utku Evci, Vedant Misra, Vincent Roseberry, Vlad Feinberg, Vlad Kolesnikov, Woohyun Han, Woosuk Kwon, Xi Chen, Yinlam Chow, Yuvein Zhu, Zichuan Wei, Zoltan Egyed, Victor Cotruta, Minh Giang, Phoebe Kirk, Anand Rao, Kat Black, Nabila Babar, Jessica Lo, Erica Moreira, Luiz Gustavo Martins, Omar Sanseviero. Lucas Gonzalez, Zach Gleicher. Tris Warkentin, Vahab Mirrokni, Evan Senter, Eli Collins. Joelle Bar- ral, Zoubin Ghahramani, Raia Hadsell, Yossi Matias, D. Sculley, Slav Petrov, Noah Fiedel, Noam Shazeer, Oriol Vinyals, Jeff Dean, Demis Hassabis, Koray Kavukeuoglu, Clement Farabet, Elena Buchatskaya, Jean-Barneeeeeeeee SannBaatiste Alayrac, Rohan Anil, Dmitry, Lepikhin, Sebastian Borgeaud, Olivier Bachem, Armand Joulin, Alek Andreev, Cassidy Hardin, Robert Dadashi, and Léonard Hussenot. Gemma 3 technical report, 2025. URL https: : / arxiv org/ abs/ 2200/553..00 19786. 1,
Peiyao Wang and Haibin Ling. Svqa-rl: Reinforcing spatial reasoning in milms via view-consistent reward optimization. arXiv preprint arXiv:2506.01377. 2025. 3, 70
Weiyun Wang, Zhangwei Gao, Lixin Gu, Hengjun Pu, Long Cui, Xingguang Wei, Zhaoyang Liu, Linglin Jing, Shenglong Ye, Jie Shao, Zhaokai Wang. Zhe Chen, Hongjie Zhang, Ganlin Yang, Haomin Wang, Qi Wei, Jinhui Yin, Wenhao Li, Erfei Cui, Guanzhou Chen, Zichen Ding, Changyao Tian, Zhenyu Wu, Jingjing Xie, Zehao Li, Bowen Yang, Yuchen Duan, Xuehui Wang, Zhi Hou, Haoran Hao, Tianyi Zhang, Songze Li, Xiangyu Zhao, Haodong Duan, Nianchen Deng, Bin Fu, Yinan He, Yi Wang. Conghui He, Botian Shi, Junjun He, Yingtong Xiong, Han Lv, Lijun Wu, Wenqi Shao, Kaipeng Zhang, Huipeng Deng, Biging Qi, Jiaye Ge, Qipeng Guo, Wenwei Zhang, Songyang Zhang, Maosong Cao, Junyao Lin, Kexian Tang, Jianfei Gao, Haian Huang, Yuzhe Gu, Chengqi Lyu, Huanze Tang, Rui Wang. Haijun Lv, Wanli Ouyang. Limin Wang. Min Dou, Xizhou Zhu, Tong Lu, Dahua Lin, Jifeng Dai, Weijie Su, Bowen Zhou, Kai Chen, Yu Qiao, Wenhai Wang, and Gen Luo. Internv13.5; Advancing open-souree multimodal models in versa- tility, reasoning, and efficiency, 2025. URL https: http://aaxxvvv//v/e org// abs/ 2508 18265. 1,
Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yo- gatama, Maarten Bosma, Denny Zhou, Donald Metzler, Ed H. Chi, Tatsunori Hashimoto, Oriol Vinyals, Percy Liang, Jeff Dean, and William Fedus. Emergent abilities of large language models, 2022. URL https:/ / arxiv. org/ abs 22062220.022222 07682. 10
Fei Xia, Amir R. Zamir, Zhiyang He, Alexander Sax, Jitendra Malik, and Silvio Savarese. Gibson env: Real-world perception for embodied agents. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2018. 1
Shaoyuan Xie, Lingdong Kong, Yuhao Dong, Chonghao Sima, Wenwei Zhang, Qi Alfred Chen, Ziwei Liu, and Liang Pan. Are vims ready for autonomous driving? an empirical study from the reliability, data, and metric perspectives. arXiv preprint arXiv:2501.04000 2025.
An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, Guanting Dong, Haoran Wei, Huan Lin, Jialong Tang, Jialin Wang, Jian Yang. Jianhong Tu, Jianwei Zhang, Jianxin Ma. Jianxin Yang, Jin Xu, Jin- gren Zhou, Jinze Bai, Jinzheng He, Junyang Lin, Kai Dang, Keming Lu, Keqin Chen, Kexin Yang, Mei Li, Mingfeng Xue, Na Ni, Pei Zhang, Peng Wang, Ru Peng, Rui Men, Ruize Gao,
16

---

Published as a conference paper at ICLR 2026
Runji Lin, Shijie Wang, Shuai Bai, Sinan Tan, Tianhang Zhu, Tianhao Li, Tianyu Liu, Wen- bin Ge, Xiaodong Deng, Xiaohuan Zhou, Xingzhang Ren, Xinyu Zhang, Xipin Wei, Xuancheng Ren, Xuejing Liu, Yang Fan, Yang Yao, Yichang Zhang, Yu Wan, Yunfei Chu, Yugiong Liu, Zeyu Cui, Zhenru Zhang, Zhifang Guo, and Zhihao Fan. Qwen2 technical report, 2024a. URL https: //apps:///rxivv org/ alss 220724007.20 10671. 6
An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang. Jing Zhou, Jingren Zhou, Junyang Lin, Kai Dang. Keqin Bao, Kexin Yang, Le Yu, Lianghao Deng, Mei Li, Mingfeng Xue, Mingze Li, Pei Zhang, Peng Wang. Qin Zhu, Rui Men, Ruize Gao, Shixuan Liu, Shuang Luo, Tianhao Li, Tianyi Tang, Wenbiao Yin, Xingzhang Ren, Xinyu Wang, Xinyu Zhang, Xuancheng Ren, Yang Fan, Yang Su. Yichang Zhang, Yinger Zhang, Yu Wan, Yuqiong Liu, Zekun Wang, Zeyu Cui, Zhenru Zhang, Zhipeng Zhou, and Zihan Qiu. Qwen3 technical report, 2025a. URL https: : / /aa.ivvvvv.../////v//////////vta httaaaaiivrggaaaaaeeee 2505. 09388. 6
Jihan Yang. Shusheng Yang. Anjali W Gupta, Rilyn Han, Li Fei-Fei, and Saining Xie. Thinking in space: How multimodal large language models see, remember, and recall spaces. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 10632-10643, 2025b. 3, 69
Lihe Yang, Bingyi Kang. Zilong Huang, Xiaogang Xu, Jiashi Feng. and Hengshuang Zhao. Depth anything: Unleashing the power of large-scale unlabeled data. In CVPR, 2024b.
Rui Yang, Hanyang Chen, Junyu Zhang, Mark Zhao, Cheng Qian, Kangrui Wang. Qineng Wang. Teja Venkat Koripella, Marziych Movahedi, Manling Li. Heng Ji, Huan Zhang, and Tong Zhang. Embodiedbench: Comprehensive benchmarking multi-modal large language models for vision- driven embodied agents. In Ferty- International Conference on Machine Learning, 2025c. URL https: wtwp//ppeevvieeeeeeeeeeeeeeeeeee iewww. net/ forum? 2 i dd LEBPS..
Zeyuan Yang, Xueyang Yu, Delin Chen, Maohao Shen, and Chuang Gan. Machine mental imagery: Empower multimodal reasoning with latent visual tokens, 2025d. URL https: arrivv.
org/abs/2505 17218. 9
Yuan Yao, Tianyu Yu, Ao Zhang, Chongyi Wang. Junbo Cui, Hongji Zhu, Tianchi Cai, Haoyu Li, Weilin Zhao, Zhihui He, Qianyu Chen, Huarong Zhou, Zhensheng Zou, Haoye Zhang, Shengding Hu, Zhi Zheng, Jie Zhou, Jie Cai, Xu Han, Guoyang Zeng, Dahai Li, Zhiyuan Liu, and Maosong Sun. Minicpm-v: A gpt-4v level milm your phone, 2024. URL https: //twxiaxxvv...//////aa abs/2408. 01800.
Baiqiao Yin, Qineng Wang, Pingyue Zhang, Jianshu Zhang, Kangrui Wang. Zihan Wang, Jieyu Zhang, Keshigeyan Chandrasegaran, Han Liu, Ranjay Krishna, Saining Xie, Manling Li, Jia- jun Wu, and Li Fei-Fei. Spatial mental modeling from limited views, 2025. URL https: / //waxiiv.rrg///aae.. hbtpg//bb/25000
1488. 1, 2, 56, 69, 70
Li Zhang, Youhe Jiang, Guoliang He, Xin Chen, Han Lv, Qian Yao, Fangcheng Fu, and Kai Chen. Efficient mixed-preeision large language model inference with turbomind. arXiv preprint arXiv:2508. 15601, 2025a. 67
Wanyue Zhang, Yibin Huang, Yangbin Xu, Jingling Huang, Helu Zhi, Shuo Ren, Wang Xu, and Jiajun Zhang. Why do milms struggle with spatial understanding? systematic analysis from data to architecture. arXiv preprint arXiv:2509.0235) 2025b. 3, 70
Wenyu Zhang, Wei En Ng, Lixin Ma, Yuwen Wang, Junqi Zhao, Allison Koenecke, Boyang Li, and Lu Wang. Sphere: Unveiling spatial blind spots in vision-language models through hierarchical evaluation. arXiv preprint arXiv:2412.
2024. 3, 69
Zheyuan Zhang, Fengyuan Hu, Jayjun Lee, Freda Shi, Parisa Kordjamshidi, Joyce Chai, and Ziqiao Ma. Do vision-language models represent space and how? evaluating spatial frame of reference under ambiguities. In The Thirteenth International Conference on Learning Representations, 2025c. URL hwwpttttt///ee.cc http:::////peeevieeeee net/ dttp://oommmmdmmdoooeeee
17

---

Published as a conference paper at ICLR 2026
Baining Zhao, Ziyou Wang. Jianjie Fang, Chen Gao, Fanhang Man, Jinqiang Cui, Xin Wang, Xinlei Chen, Yong Li, and Wenwu Zhu. Embodied-r: Collaborative framework for activating embodied spatial reasoning in foundation models via reinforcement learning. arXiv preprint arXiv:2504. 12680, 2025. 3, 70
Jinguo Zhu, Weiyun Wang, Zhe Chen, Zhaoyang Liu, Shenglong Ye, Lixin Gu, Hao Tian, Yuchen Duan, Weijie Su, Jie Shao, Zhangwei Gao, Erfei Cui. Xuehui Wang, Yue Cao, Yangzhou Liu, Xingguang Wei, Hongjee Zhang, Haomin Wang, Weiye Xu, Hao Li, Jiahao Wang, Nianchen Deng, Songze Li, Yinan He, Tan Jiang, Jiapeng Luo, Yi Wang. Conghui He, Botian Shi, Xingcheng Zhang, Wenqi Shao, Junjun He, Yingtong Xiong, Wenwen Qu, Peng Sun, Penglong Jiao, Han Lv, Lijun Wu, Kaipeng Zhang, Huipeng Deng, Jiaye Ge, Kai Chen, Limin Wang, Min Dou, Lewei Lu, Xizhou Zhu, Tong Lu, Dahua Lin, Yu Qiao, Jifeng Dai, and Wenhai Wang. In- ternv13: Exploring advanced training and test-time recipes for open-source multimodal models, 2025. URL https: : / arxiv. org/ abs 25044 10479. 6
18

---

Published as a conference paper at ICLR 2026
TABLE OF APPENDIX CONTENTS
SpinBench
21
A.1
Detailed Dataset Collection Process
21
A.1.1
Simulation
A.1.2
Real-world dataset curation.
................................
21
A.2 Data Annotation Protocol
22
A.2.1
General Guidelines
................................
22
A.2.2
Data Format and Structure
23
A.2.3
Quality Control and Validation
23
A.2.4
Handling Ambiguities
23
A.3 Task Categories and Subtypes
23
A.4
Detailed Task Description with Examples
25
................................
A.4.1
Identity Matching
................................
25
A.4.2
Dynamic Rotation
................................
25
A.4.3
Dynamic Translation
30
A.4.4
Object-Relation Grounding
33
A.4.5
Canonical View Selection
33
A.4.6
Perspective Taking (View Selection)
A.4.7
Perspective Taking (Relative Position Transformation)
41
A.4.8
Mental Rotation
Detailed VLMs Evaluation Results
51
B.1 Raw accuracy and Cohen's kappa
B.2
Detailed Consistency Evaluations
51
B.2.1
Augmentation Types
................................
B.2.2
Performance Metrics
................................
51
B.2.3
Augmentation Strategy Analysis
56
B.2.4
Pattern Distribution Analysis
56
B.3 Correlation Analysis
56
C Human Evaluations
56
C.1
Human Evaluation Tool Design
61
C.I.I
Question Type Detection and Display
Dist.y..........................
C.1.2
Progress Management and Resumption
61
C.1.3
Dataset Curation Integration
C.1.4
Response Collection
61
C.2 Human Performance
C,3
Correlation Analysis
65
19

---

Published as a conference paper at ICLR 2026
D Details the VLM Evaluation Setup
66
D.I Evaluation Configuration
66
D.2 Model Implementations...
67
D.2.1
LMDeploy-Supportee Models
67
D.2.2 Other Models
67
D.3 Prompt for Reasoning Models
67
E
Finetuning VLMs
68
E.1
Cross-task and cross-domain validation.
68
E.2 Training and validation results
68
E.3 GRPO Training Configuration
68
F
More Related Works
69
F1 Spatial reasoning benchmarks
69
F2 Spatial reasoning models
69
G The Use of Large Language Models (LLMs)
70
20

---

Published as a conference paper at ICLR 2026
A SPINBENCH
A.1 DETAILED DATASET COLLECTION PROCESS
A.1.1 Simulation We adopt a synthetic dataset generation pipeline that integrates Infinigen- generated indoor environments Raistrick et al. (2024) with the Isaac Sim simulator NVIDIA. The pipeline fully automated through a custom script built on top of the Infinigen SDG (synthetic data generation) framework. The process can be summarized as follows:
1. Environment loading, A set of nine indoor dining-room scenes are retrieved from the Infinigen asset library. Each scene is instantiated optionally hidden for improved lighting and camera coverage. Colliders are added to all major surfaces (walls, floors, dining table) enable realistic object-surface interactions.
USD stage, with ceiling meshes
2. Object assets. Everyday objects imported from the Yale-CMU-Berkeley (YCB) dataset Calli et al. (2015; 2017). We include 21 distinct items (e.g., banana, soup can, mug, Rubik's cube), each automatically labeled by parsing their USD asset names. Gravity and bedid dynamics are attached using PhysX APIs to support physically plausible placement and falling behavior. Additional assets can be manually labeled with explicit semantic tags.
3. Scene composition. For each scene, objects are sampled and placed in the working area above the dining table. Object poses randomized within bounded 3D ranges (position, orientation, scale). Distractor meshes and primitive shapes are also injected.
4. Lighting. Three movable sphere lights are added per scene and randomized in location, in- tensity (500-2500 lumens), and color balance. Dome lights with HDR textures are random- ized per capture to simulate natural variations in sky illumination (clear, cloudy, evening, night).
5. Cameras. Multiple cameras (default: five per scene) are defined, with randomized intrin- sies and extrinsics. We support both (i) random camera placements on a viewing sphere around target object, and (ii) structured camera orbits with fixed angular increments to capture viewpoint changes.
6. Physics simulation. and allow objects to settle into stable configurations. Captures are taken both after this settling. producing "dropped" views with objects resting on the table.
The scene is stepped forward for several frames resolve collisions
7. Data capture. Render products generated at 480 x 480 resolution using the RTX Path Tracing renderer. For each environment and camera, both RGB images and corresponding semantic pose annotations written to disk through Isaac Replicator writers. On average, we capture 100 frames per environment (500 frames total per scene when multiplied across cameras).
In addition randomized placement, explicitly manipulate object positions to generate trolled spatial displacements. Using custom utility functions, each object is sequentially shifted relative to the initial position:
Left/Right. Objects are translated along the z-axis by fixed increments (e.g., move move.left (di stance=0- 1) and move right (di (distance=0.22 This simu- lates lateral displacements in the viewer's frame of reference.
Near/Far. Objects are shifted along the y-axis (move_near (d ddistance==..1 .1), move_far (distance=C.. simulating depth changes toward or away from the cam- era viewpoint.
This procedure yields a diverse and physically consistent dataset covering static spatial relations, translational dynamics, and multi-view perspective taking (with and without occlusion). The mod- ular design of the script enables controlled variation in object placement, illumination, and camera trajectories, while preserving reproducibility through fixed random seeds.
A.1.2 Real-world dataset curation. To unify diverse real-world sources under a common spa- tial reasoning framework, we implemented multi-dataset curator that standardizes input formats,
21

---

Published as a conference paper at ICLR 2026
view sampling, and question generation. Each dataset is wrapped in dedicated handler class that exposes object discovery, available views, and sample generation routines. The curation pipeline proceeds as follows:
Object discovery. For each dataset, we enumerate object folders (ABO product IDs, car object IDs, and face subject IDs). Only objects with complete view coverage are retained (e.g., 72 views in ABO, consistent rotation sequences in Cars, and multiple head poses in Faces). This ensures all curated objects can support viewpoint-based reasoning tasks.
View normalization. Views are mapped to standardized angular indices. For ABO, we map 72 canonical views 09-355 in steps. For Cars and Faces, we parse angles and normalize them to 09-359". This allows cross-dataset comparison of viewpoint-sensitive tasks.
Task generation. Each dataset supports three primary families of tasks:
1. Object identity. Odd-one-out tasks (triplets or quartets) where two or three views depict the same object/person and one depicts a distractor.
Rotation classification. Pairwise comparisons where an object rotates by a known offset (e-g., 45% 90%), and the model must classify the rotation direction (clock- wise/ccouterclocckwiss)) For Faces, we explicitly test both viewer-centric and object- centric frames of reference.
3. Canonical view selection. Given a front view, models must identify left, right, or back profiles from among candidate images. This directly probes viewpoint reasoning and perspective-taking.
Mental rotation (ABO only). Leveraging ABO's dense 72-view coverage, we generate multiple-choice mental rotation tasks where the model must predict the outcome of rotating an object by 45 -180 in either direction. Distractors are sampled to ensure a minimum angular separation, preventing trivial cues.
Splitting and statistics. After sample generation, the curator splits data into train//alidationntestt sets with dataset-specific ratios (c.g., ABO: 80/10/10; Faces: 70/10/20; Cars: test-only). Statistics such as the number objects, samples per task type, and split sizes are logged for reproducibility.
Query variation. To avoid linguistic bias and encourage genuine spatial reasoning, each task type is associated with multiple natural language templates. For example, odd-one- out task may be phrased as "Which of these three images shows a different object?" or alternatively as "Two of these images show the same object at different views, which one is different?" During dataset generation, a random template is selected from the available pool for each sample, ensuring linguistic diversity across training and evaluation.
Answer option randomization. In addition to varying the textual query, we randomize the ordering of candidate options (A/B/C or A/B/C/D). For odd-one-out tasks, the distractor image can appear in any position; for rotation classification, the labels "clockwise" and "counterclockwise" are shuffled; and for canonical view selection, left/right/back views are permuted across options. This randomization ensures that models cannot exploit positional biases (c.g., always guessing option C) and must instead rely on actual spatial reasoning to succeed.
This unified curation procedure ensures that disparate real-world datasets contribute consistently formatted, balanced tasks, enabling controlled evaluation of spatial reasoning across product-scale objects (ABO), structured geometric entities (Cars), and biologically stimuli (Faces).
A.2 DATA ANNOTATION PROTOCOL
A.2.1 General Guidelines All annotations are designed to probe spatial reasoning while mini- mizing confounds. We adopt the following principles: (i) all questions must be unambiguous under a specified frame reference, (ii) tasks must balance object categories and viewpoints, and (iii) phrasing diversity is required to prevent overfitting single query template.
22

---

Published as a conference paper at ICLR 2026
A.2.2 Data Format and Structure Each annotated instance is serialized JSON with four fields: problem (natural language question), answer (ground truth label, always a single capital letter), images (paths associated views), and met metadata (structured fields such as object IDs, view indices, occlusion condition, task type). This format ensures compatibility with VQA pipelines while retaining rich metadata for controlled analysis. All datasets organized by dataset type (ABO, Cars, Faces, Infinigen), and further by task subtype.
A.2.3 Quality Control and Validation We employ both automated and manual checks: for Infinigen, annotation scripts display candidate images to the curator, who confirms correctness with keystrokes (e.g., pressing "y" validate a generated left/right relation). For real-world datasets, handlers enforce strict view coverage (72 views for ABO, complete rotation for Cars, multi-pose coverage for Faces). Random seeds are fixed during sampling for reproducibility.
A.2.4 Handling Ambiguities To ensure tasks probe genuine spatial reasoning rather than noise, we implement explicit constraints to minimize annotation ambiguíties:
Angular separation. In ABO mental rotation tasks, distractor views are required to differ by least 30 from the target orientation. This prevents trivial confounds where two options appear nearly identical. Car and Face rotation classification restricts rotations to canonical offsets (45", 90", 180") for clearer discriminability.
Visibiliity filtering. In Infinigen, only objects with projected visibility above 0.8 consid- ered valid. Scenes where occlusion prevents reliable labeling are discarded. For occlusion tasks, annotators explicitly tag each scene as no, partial, or full occlusion.
computed from object cuboid centers projected in image space. Objects are required to have distinct x-coordinates to avoid ambiguous ties. Near/far relations based on y-coordinates, requiring minimal vertical separation. In dynamic relation tasks, movement distances are set to non-trivial shifts (0.2 scene units) to guarantee perceptibility.
Positional thresholds. Static left/right judgments
Symmetry control. Centrally symmetric objects (e.g., square stool) excluded from ABO to avoid cases where left/right or rotation cannot be distinguished visually.
Frafenef-reference disambiguation. For face rotation, tasks are duplicated under both object-centric ("the person turned their own head left") and viewer-centric ("the person turned to the viewer's ' right") frames.
These constraints, enforced both in code and manual filtering, ensure that all retained samples are unambiguous and diagnostic of the intended spatial relation.
A.3 TASK CATEGORIES AND SUBTYPES
We provide a comprehensive breakdown of the dataset constitution across major task groups, their subtypes, and the configuration details for each subtype. Table summarizes the complete distribu- tion across all 51 distinct task subtypes.
Table 4: Full task subtype breakdown with configuration details.
Group
Subtype
# #Queries
#Images
#Options
identity
80
0+3
3
car idrinidenttyyyo quartet imagefirst
0+4
4
car dedeentity-quartet -tt.aattt_itterraavaeee jintrrlcaved
0+4
4
car car_identity-quart quaautetteextiiins face.identity identity matching
0+4
4
79
0+3
face identity quartet imagefirst
7
0+4
4
face identity quartet.interlest interleaved
0+4
face face_ident identity quarfet_textffrst
4
0+4
4
object object identity imagefirst
33
0+3
object_identity interlcaved
35
0+3
3
23

---

Published as a conference paper at ICLR 2026
Group
Subtype
# #Queries
#lmages
#Options
object identity quartet imagefirst
42
0+4
object identity quartet interleaved
38
0+4
4
serect_ddetttt..oee -qaattet, object.identity textffrst
22
0+4
37
0+3
object- relation grounding
infinigen_sppaee relation grounding. near
152
1+0
relatal anfinigen.spatiinn relation grounding weffddiinng...t..ooo left right
286
1+0
2
infinigen_spatiall aaanigensspatialrrlatttnnspiii wttporssshntoootoehhinnn
198
1+0
car rotation classification
80
2+0
face rotation classification own perspective face face_rotatin carattion_classffcaatioo
94
2+0
2
dynamic rotation
70
2+0
objecttrotationnccassiii http:/oocccassssfaaatoooaaaaaaie Doject_rrtationn classiffcaatinnnnnnonn interleaved abject. classicionncaassfcaaattoonnoonnt textfiss
35
2+0
47
2+0
2
27
2+0
dynamic translation
htt.://ssppptaaallllltttsoniiiii -ww.mmiic-dnmiccciccommmm aarnamic_frontbbeee brckk
78
2+0
78
2+0
2
canonical www.wviwwwweeeeee.e wwwlccelccnncccc back
19
1+3
wtt.:vwwwwwwcccin.i... warr.aaonicaal.vve.sseecoioooooo
19
1+3
20
1+3
face lice.canonical_vve www.onical.vew.sseeee......eee.e www.wvieeeeeeeinooooooo.o www.s//swwt...eeeeeeeeeeeeeeeee
23
1+2
2
canonical view selection
face canonical view selection own perspective right
19
1+2
face canonical view www.sseection.viieec viewer perspective left
17
1+2
2
face_canone saanonical.veewsseeeeeee viewer, wtwp:eeeppespcciieerieeee
18
1+2
80
1+3
objact.canonical.viee www.w/ewwwwwwecon...... object.cmonnicallvvvv.eeeeeetttd
1+3
57
1+3
3
infinigen_rotationn hatptiioooselccctoohbeeeeee wtw.a//fclllccluusssssse
1+3
infinigen_rotatinn www.rrotttoossttee cerpiinssscccccccchceeeeee cactiion__ac__oooo
laeckkno__ceee pocoocuusion
49
1+3
infinigen www.rttiiioeeeeeiioonn..mmo
wwt.:seeeleeeeccccccccc caction_bac_pppe baock__prtaalocclussoo
47
1+3
infinigen rotation selection left full occlusion
1+3
3
infinigen.rotation wwt.ation.seeeettiii selection.
no occlusion
62
1+3
43
1+3
3
infinigen htt.eeenrrttttooonooomo wtt.rrrtttiossslccttioottoo wwtp:/eeeececccceeeeeeeeee rightonnnintutuuiuoie full occllusion follooouill
1+3
perspective taking
oococlluioon
61
1+3
3
infinigen wwwatiionseleettionntooo
partlll.praaaaaaaaaa paraialloocclusooo Locelusion
40
1+3
infinigen sappssspaaaaaaaaa relaaionnee pra.ttoonttaassoommatiooo premise.back back
33
1+0
2
infinigen spatial relation transformation w premise left
58
1+0
infinigen spatial relation transformation w premise right
53
1+0
2
lefffiigen_ssatiall appatial_relation www.teeonnnnteee iuatiionntansffrmattonn wwo.premise_hate back
36
1+0
2
aufinigen_spatill... www.lreeeennn. www.tiion.tannsorrnnnnaaooo www.sssfronaaiooo.vvvooo wwtprattiiiioooommmmmisse crro-ppreiieeeee
58
1+0
infinigen spatiall htt.sspppaaaalllatioooooooooooo traationntaastoormaatoon wo www.pmmiisc....mmmmmeeommm..eee right
52
1+0
2
mental rotation
boject_mental_nng
78
1+4
4
infinigen_mental rotation menntal
120
1+4
car_mental_rone Protation
20
1+4
4
Task Group Distribution. The dataset contains total of 2739 samples spanning seven major task groups with varying emphasis: Object-Relation Grounding tasks represent the largest category with 636 samples (23.2%), followed closely by Perspective Taking with 613 samples (22.4%). Identity Matching contributes 405 samples ( (14.8%), while Canonical View Selection and Dynamic Rotation each account for approximately 13-14% of the dataset (358 and 353 samples respectively). The smaller categories include Dynamic Translation with 156 samples (5.7%) and Mental Rotation with 218 samples (8.0%).
Dataset Source Distribution. Four distinct data sources contribute to the benchmark: Infinigen provides the majority with 1,525 samples (55.7%), followed by ABO Objects with 617 samples
24

---

Published as a conference paper at ICLR 2026
(22.5%), Faces with 339 samples (12.4%), and Cars with 258 samples (9.4%). Notably, Infinigen exclusively covers Object-Relatiee Grounding, Perspective Taking, and Dynamic Translation tasks, while the other domains span Identity Matching, Canonical View Selection, and Dynamic Rotation tasks.
Task Configuration Details. The image structure varies systematically across task types, decom- posed into reference images and candidate option images. Single reference image tasks (1+0 to 1+4 format) constitute the majority, including spatial relation tasks with text-only options (1+0), canon- ical view selection with 2-3 image options (1+2, 1+3), and mental rotation with 4 image options (1+4). Two-reference image tasks (2+0 format, 509 samples, 19.6%) appear exclusively in rota- tion classification and dynamic relationship tasks with text-only options. Identity matching tasks uniquely employ no-reference format (0+3, 0+4), where all 3-4 images serve as candidate options for comparison.
The relationship between option images and answer choices follows consistent pattern: when the image option count is O, the task employs text-only multiple choice answers; otherwise, the number of image options directly corresponds to the number of answer choices.
Answer Choice Distribution. The benchmark employs a balanced choice structure: binary choices (A/B) represent 39.9% of tasks (1,094 samples), primarily rotation classification and spatial trans- formation tasks. Ternary choices (A/B/C) account for 53.4% (1,463 samples). covering canonical view selection and most identity matching tasks. Four-way choices (A/B/C/D) only appears in quar- tet identity matching and mental rotation tasks. The answer distribution across options shows reasonable balance: option A appears in 41.2% of cases (1,130 samples), option B in 41.2% (1,131 samples), option C in 14.3% (391 samples), and option D in 3.1% (87 samples).
A.4 DETAILED TASK DESCRIPTION WITH EXAMPLES
A.4.1 Identity Matching The identity matching tasks evaluate model's ability recognize whether multiple images depict the same object, person, or vehicle under viewpoint variation. This capability serves as a foundational prerequisite for more complex spatial reasoning, since robust ob- ject identity recognition must occur before reasoning about spatial transformations. Identity match- ing tasks are presented across three domains- cars, faces, and generic objects- wwtt further subdi- visions based on presentation format (triplet vs. quartet, image- first vs. text-first vs. interleaved). Quartet setting compared triplet setting tests whether one more image of the same object increases difficulty by presenting more tokens or decreases difficulty by presenting more views of the same object.
Car identity matching(Fig. 8): The model must decide which image shows a different car, given triplets or quartets of cars photographed from different angles. Subtypes differ by whether the distractor is presented among three images, or within a quartet with either images first, text first, or an interleaved format.
Face identity matching(Fig. 9): Analogous to the car tasks, but using human faces under pose variation. The distractor is a different individual, while the other images depict the same person from different viewpoints. This directly probes human face recognition under multi-view conditions.
Object identity matching (Fig. 10 and Fig. 11): For the triplet form, the model receives three images, two of which depict the same object under viewpoint change, while one shows different object. Subtypes vary by whether images shown first, interleaved with text, or after text. Quartet form is a variation where the model must select the odd one out from four candidate images, again with differences in presentation format. This setting tests whether one more image of the same object increases difficulty by presenting more tokens or decreases difficulty by presenting more views of the same object.
A.4.2 Dynamic Rotation The dynamic rotation tasks evaluate whether models can track the orientation changes of single object across sequential frames. Unlike static relation tasks, these examples isolate rotational transformations with a static camera and a constant background, thereby requiring models reason about in-place turning rather than translation.
25

---

Published as a conference paper at ICLR 2026
Task group: identity matching (car)
Task: car_ Identtyy
Question:
<mage>
<image>
<image>
Two of these images show the sarna car from differant argles. Which one shows diñle vont car?
Only answer with the capital letler from B, C).
Task: car. identity quartet Imagefirst
Question:
<image>
ximage>
<image>
<ir as
Which of these four images B. C. ahows different oor from the other three?
Only anewer with letter C.
Task: car identity__..oooo... quartet__taaoeeee Interleaved
Question:
Look at the fol tollwing four cars:
<image>
<amage>
<image>
Which image shows a different car? Only answer with the capital oeter from (A, B, C, D).
par_identity_geee quarrttt_
Quesfion:
images
<image>
<image>
<image>
D. <image>
Only anower with the cap tal latter from (A. B, C,
Figure 8: Examples of car identity matching tasks. Models must detect the odd out across triplets and quartets, with different presentation styles fimage-first, interleaved, text-first).
26

---

Published as a conference paper at ICLR 2026
Task group: identity matching (face)
Task: face_fccceee sec_iddettty
Question:
<image>
<imege>
<image>
images show person angles. person?
Only answer with the capital letter from (A, C).
jdentitt_quartee, imagefirst
Quesiion:
<image>
<image>
<image>
<image>
Three photos show the same person, one shows someone different. Which is different?
Only answer with the capital letter from D).
Task:
face identity_ Sarpitt__guaee__t_i
interleaved
Question:
Compare these individuale:
A. <imege>
<image>
<image>
simage>
Which the different person?
Only answer with the capital letter from (A. B, C, D).
A
B
Task: seso__dentttttyyeee quartet, quartetttextfirrtt textfirst
Question:
these four images, three show the same person from cifferent poses, but one shows different person. identify the different one.
A. <image>
B. <image>
<image>
<image>
Only enswer with the capital letter from (A, B, C, D).
Figure 9: Examples of face identity matching tasks. The model must identify which image depicts a different individual, under both triplet and quartet setups, with varied presentation orders.
27

---

Published as a conference paper at ICLR 2026
Task group: identity matching (object)
object identity_imaaeeteee
Question:
<image>
<image>
<image>
Which of these (A, different object from the other Only captial C).
Task: object_identity, iddntity_interleaver
Question:
following images:
<image>
<image>
C. <imege>
Which image shows different object?
answer with capéal letter (A,
Task: object Iddently_textfirrs textfirst
Queetion:
tree images, two of them show same object different views, but the other one show a different object, Identiify which show diferent object
A <image>
B <image>
<image>
Only caplial
C
V
Figure 10: Examples object identity matching with triplets. Each row contains three candidate
images; two show the same object under view change, and one shows different object.
28

---

Published as a conference paper at ICLR 2026
Task group: identity matching (object)
Task: seject_identtyy. [denttty_quarte__inneeee Imageettisagefirst
Question:
<imege>
<image>
<image>
Threa of these images show the same objact diffarent views. Which one shows the different abject?
Only anower with the capital lettor from (A, B, C, D).
Task: object_identity identity_quarter interleaved
quartet_interleavee
Question:
following images:
<image>
<image>
C. <image>
<imege>>
Which image different C, D).
Only capital
Task: object_ pecccttdeeettt__t_e quatfatt_tttxtfffirttt textfirst
Question:
these images
object three?
A. <image>
B. <image>
<imega>
<image>
Only answer with the capital letter from (A, B, C. D).
Figure 11: Examples of object identity matching with quartets. Models must identify the image depicting a different object, with task variants controlling text-image ordering.
29

---

Published as a conference paper at ICLR 2026
Car rotation classification(Fig. 12): in place. must decide whether the rotation was clockwise or counterclockwise, with reference to top-down view.
The model sees two sequential views of rotating
Face rotation classification (own perspective vs. viewer perspective) (Fig. 13): These subtypes probe perspective-dependees interpretation. From the human in the image's own perspective, "left" and "right" correspond to their intrinsic body-centered frame. From the viewer's perspective, left/right must be relative the camera's 's position image frame.
Object rotation classification 14): Similar to cars, but applied to generic objects (e.g., furniture). Variants differ in presentation order (image-first, text-first, interleaved).
Task group: dynamic rotation (car)
car_ rotationnn c classiffication
Question:
<Image>
<image>
The car rotated from the front view to the eecond view. Was the rotation clockwise or countercioc A clocwiie, B. courterccckkwle Ony arswer with the captial letter from (A, B) from front top-down view.
The camera stationary the
ccunterdlodwwfee
Task: can_roaationcclaafffcttoonn
<image>
<image>
The first image shows the oer from the front. In which direction did the car rotete to reach the second view? clockwlee B. counterddockwee
Unly answer with the capital latter fram (A, b).
The camera stationary and the rotates in place from the front view view.
Clocksise counterdeckwwsse celined from
Examples of dynamic rotation (car) tasks. The car is shown rotating in place across two images, and the model must determine whether the transformation corresponds to a clockwise or counterclockwise rotation.
Figure 12:
A.4.3 Dynamic Translation The dynamic translation tasks evaluate whether models can detect and interpret translational movements of objects across sequential frames. Unlike rotation clas- sification, the focus here on linear displacement within the viewer's frame of reference while the background and camera remain static. These tasks isolate directional movement (front/back or left/right) from rotational or other spatial transformations.
Front-back translation (Fig. 15): The model observes two frames showing object (e.g., box, canned food) shifted either forward or backward relative to the static camera. It must classify the displacement "front" or "back."
30

---

Published as a conference paper at ICLR 2026
Task group: dynamic rotation (face)
Task: face face_rotatee rww.tttttonncoic iilaaiionccassfiicaaion oww_pesspeeeeeeeeeeeeeee
Question:
<image>
<image>
From the person's own perspeclive, which way did they turn their head? A. right. B. left Only answer with the cepital letter from B). Consider
person's perspective
right).
Task: wotattoo_claasifffeeetooeoeeee oetpppratnnnonnnpprppsttttttte
Queetion:
<image>
<image>
From the person's own perspactive, which way they turn their head? right, B.
Only answer with the capital letter from (A, B).
Consider the direction from person's perspective (their left their right).
Task: face_ wwrrraiionclaasffllalllllteeeene wwwsssifccaioonneiittooo htttiiatttto_vleeeeeeeeeeeee slewer_perssecctve
Question:
<image>
<image>
From the viewer's perspective, which direction did the person's head tumn A. left B. right
Only capital from
Consider direction from viewer's perspective (viewer's 's vs viewer's right).
Task: face faceeeattatioooeaeaaaaaaaaa rwtatiionclaasifiiiiitioi wwtpaasffiiatioonvvieeerrrr
Question:
<image> <lmage>
Looking person from camera's position, they turned right
Only answer with the capital letter from (A, B).
Consider the direction from viewer's perspective (vlewer's left viewerss right).
Figure 13:
Examples of dynamic rotation (face) tasks. The model must classify the direction of a person's 's head turn, either from their own perspective (intrinsic left/rihh) or from the viewer's perspective (extrinsic left/right).
31

---

Published as a conference paper at ICLR 2026
Task group: dynamic rotation (object)
object_ wowpttoottttoonee www.rrrtooncsasssfiiiereoeeeeeee lattssffiicatioommmagfiiii
Question:
<image>
<image>
From stationary viewpoint, in which direction did object rotate between these two views? counterdlckwwiee dockwiee Only answer with the capital leller from (A, B).
The camera stationary and object rotate
and counterclockwise
top-down
Task: Dbject_rotation iiletionclassifiaation interleeavedd
Question:
Looking object
posttion cimagez
<image>
Which rotation direction does the object show/ A dlockeise. B. counterolockwtse
Only anewer with the capital letter from (A, B).
The camera stalionary and the object rotate place.
Clockwise counterolodwwise defined top-down
Task: object_ rolation wtw.rrroooclaasiiiiioeesesssseee classification textfirst
Question:
The viewpoint stati. Which way did the object clockwise counterdlodwwise? <image> <imege>
countarclakwwise.
Only answer the capiaa letier from (A,
The camera la stalionary and the object rolate in place.
Clorkwise and counteroodkwissc are defined from lap down view.
Figure 14: Examples dynamic rotation (object) tasks. Object items are shown before and after rotation, and the model must classify the direction of turn. Subtypes vary by whether the question is posed text-first, image-first, or interleaved.
32

---

Published as a conference paper at ICLR 2026
Left-right translation (Fig. 15): The model observes an object (e.g., scissors, bottle) mov- ing laterally within the scene. It must determine whether the movement occurred toward the left or the right, again from the static camera's viewpoint.
A.4.4 Object-Relation Grounding Object-relation grounding tasks assess a model's ability to infer object-relative spatial configurations from a single image. Each task involves two target ob- jects within the same frame, and models must judge directional relations (e.g., left/right, in front of/behind) or distance-based relations (e.g., near/far). These tasks capture object-relative pose un- derstanding in static scenes. Unlike dynamic or multi-view reasoning tasks, these examples isolate spatial grounding from both temporal reasoning and perspective transformations, serving as a con- trolled evaluation of whether models can interpret scene-centric spatial layouts from static visual input. A key difficulty in these tasks is that the model must correctly identify the correct objects of interest among possibly multiple distractor objects in the scene. This makes the setup closer to an open-world detection problem: even if a model has a strong spatial reasoning ability, focusing on the wrong object will lead to incorrect answers.
For spatial relation tasks with inherent symmetries, we systematically generate equivalent reformu- lations through two complementary augmentation strategies test reasoning consistency: Symmet- rical Augmentation: We create logically equivalent variants by swapping spatial relationships and flipping correct answers. For example, from base query "Which object on the left? A or B," we generate the symmetrical variant "Which object is on the right? A or B," with the correspond- ing answer flipped. This transformation preserves the underlying spatial configuration while testing whether models maintain consistent spatial reasoning across equivalent logical formulations. Syn- tactic Augmentation: We reformulate question structures while preserving semantic content, such as transforming "Which object is on the left? A B," into "Is A the left right of B? A. left B. right." " These variations test whether models rely on specific linguistic patterns or demonstrate robust spatial understanding independent of question phrasing.
Left/Right Relations with Augmentations (Fig. 16): The base task asks whether one ob- ject (e.g., Rubik's 's cube) the left or right of another (e.g.. mustard bottle). Symmetrical augmentation flips the query to its logical equivalent ("Is the mustard bottle to the left or right of the Rubik's 's cube?"), while syntactic augmentation reformulates the phrasing into binary comparisons ("Which object is on the left?" vs. "Is A on the left or right of B?"). To- gether, these variations test whether models preserve consistent reasoning across symmetry and linguistic surface changes.
Near/Far Relations with Symmetry (Fig. 17): The base task asks which of two objects (e.g., marker vs. foam brick) is closer to the viewer. Symmetrical augmentation flips the distance relation by instead asking which object is farther.
Front/Behind Relations with Augmentations (Fig. 18): is in front of the other (e.g., mug vs. Rubik's cube) from a front-view image. Symmet- rical augmentation reverses the relation ("Which object is in the back?"), and syntactic augmentation reformulates the query into pairwise comparisons ("Is the mug in front of or behind the Rubik's cube?"). These augmentations jointly probe whether models generalize depth-order reasoning across logically equivalent but differently phrased prompts.
The base task asks which object
A.4.5 Canonical View Selection The canonical view selection tasks test whether models can correctly identify specified viewpoints of objects, cars, or faces. Unlike dynamic tasks, the images are presented as static alternatives, and the challenge lies in transforming the front-view reference into another canonical perspective (left, right, or back). These tasks isolate perspective transforma- tion without involving temporal dynamics multi-objec relationships.
Car canonical view selection (Fig. 19): Given a front-view reference image, the model must identify which candidate view corresponds to the car viewing from left, right, or back side. This evaluates object-centered perspective reasoning in controlled automotive scenes.
Face canonical view selection (own vs. viewer perspective) (Fig. 20): These tasks in- troduce ambiguity in reference frames. From the person's own perspective, left and right correspond to their intrinsic orientation, whereas from the viewer's perspective, left/right are defined relative the image frame.
33

---

Published as a conference paper at ICLR 2026
Task group: dynamic translation
Task:: hatplgen_ssptillrrltttoonn caclllddddmmm_aaeeeee faapii__foott_eeee
Question:
From static viewer's perspedtive, which direction did the puddligg box change position from the first <image> to second image
Omly single cap leher (A, B).
Task: infinigen_ssatial_ relationship cepatiionsii_ddymmmicc ffontt back
Question:
perspective,
change posttion image <image> image
<image
front B. back
Only answer with sigle capital letter from (A, B).
Task: Saaniignn_spttiine relationship_dyramii_____ loft_right
Question:
From static viewer's perspective, which direction the bleach cleenser change position from first image <image> to the second image
wimape?? laf B. right
Only answer with single capiaal letter from (A, B).
Task: laapliigen_sppaaarrelettte dynamic_leei _efttrightt
Queation:
From Latete vinuace perspective, which direction scissore change position from the first image <image> the second image <image>?
le B. right
Only Srewer
capital iettor from
Figure 15: Examples dynamic translation tasks. Objects undergo front-back (top) or left-right (bottom) displacements while the camera remains fixed. The model must classify the displacement direction.
34

---

Published as a conference paper at ICLR 2026
Task group: static object-relation grounding (left/right)
Task: cacpllrrrrraaanaaeeeeeeeeeeeeeee oeaaril__eeaaoooogouunnnnnenett right
Question:
<image>
From the viewer's peracective, rubiks cube the left right of mustard bottle in the image?
A. left B.
Only sngla capiial letter B).
Symmetrical Augmentation:
Question:
<image>
From the viewer's perspective, is mustard bottle on the left right of jubiks cube in the image?
lef, right
Omly single capital (A, B).
Syntactic Augmentation:
<image>
the perspective, which left image?
A. mustard bottle rubiks cube
Only aniswr with single capital letter from (A, B).
<image>
From viewer's perspective which object the the image?
bottle rubiks
Only anewer with single capital letter from (A, B).
Figure 16: Examples of object-relation grounding left/right relation tasks. The base question asks whether reference object (e.g., Rubik's cube) positioned the left right of another object (e.g., mustard bottle) from the viewer's perspective. Symmetrical augmentation reverses the relation ("Is the mustard bottle to the left right of the Rubik's cube?"), while syntactic augmentation reformulates the question style ("Which object is on the left?" vs. "Is object A on the left or right of object B?").
35

---

Published as a conference paper at ICLR 2026
Task group: static object-relation grounding (far/near)
infinigen infinigen_spatiil_ aarpataa__elationggruuolede Barr___af________e__ee
Question:
<image>
Which Which object closer to the viewer the image?
large marker,
Only answer with single capital letter from (A, B).
Symmetrical Augmentation:
Question:
<image>
Which object is farther from the viewer in the image?
A. large marker, B.foam brick Only single capital lotter
Figure 17: Examples of object-relation grounding near/far relation tasks. The model must deter- mine which two objects (c.g.. marker vs. foam brick) is closer to the viewer within a single image. Symmetrical augmentation inverts the query ("Which object is farther?") while keeping the ground- truth relation consistent. This setup features distance-based reasoning from monocular perspective cues in static frames.
36

---

Published as a conference paper at ICLR 2026
Task group: static object-relation grounding (front/behind)
Task:
behind
<image>
The magge is taken from the front of the scene. Which object is in the front?
rubuiiss cube
answer capital letter (A, B).
Symmetrical Augmentation:
Question:
<image>
image is taken from mug, B.rubik cube Only andwar with a angll capital letter from (A, B).
scene. Which object
Syntactic Augmentation:
Question:
<image>
The image is taken from the front the scene. is mug posiiioned in front of rubiks cube or behind? front of B. behind
Cmy answerwed with single capital letter from (A. B).
Question:
<image>
The image is taken from the front the scene. is rubiks cube positioned in front mug or behind? A. front of B. behind [i]
Only answer with a single capital letler from (A. B).
Figure 18: Examples of object-relation grounding front/behind relation tasks. Given front-facing view, the model must decide which object (e.g., mug vs. Rubik's 's cube) is positioned in front or behind. Symmetrical augmentation flips the depth relation ("Which object is the back?"), while syntactic augmentation reformulates the question ("Is the mug in front of or behind the cube?").
37

---

Published as a conference paper at ICLR 2026
Object canonical view selection (Fig. 21): such as furniture. Models must map the front view to left, right, or back views, testing their ability to reason about viewpoint consistency across diverse shapes.
Similar to cars, but applied to generic objects
Task group: canonical view selection (car)
jrr_canonical_ye latpaa_viee___seeeeeeeeee www.eeeeeeeeooene
Question:
You are shown four images of car order: [1] Front view. [2] View [3] View B. [4] View C.
Select the image that best shows the from the back side.
view: <image>>
A: <image>
B: <mage>>
C: <imaga>
Only answer single capital letter (A, C).
The back alde is defied from the viewer's perspective when looking the front of the car.
Task: car wwa.nnncaaaaaa... wwwp_seeectionlare
You are showr four images of car in order: []] Front view. [2] View A, [3] View B, [4] View C. Select the image that best shows the car from the left side.
Pront view. <image>
A: <image>
B: simage>
<image>
C), perspective looking
Task: car_ wwl_canonical_vvee wwwp_ssllcctoo_eeee _right
Question:
You are shown four irages a car order: (1) Front view, [3] View A, [3] View [4] View Select the image that bast shows the car from the right side.
Front view simegges
A: <imagee
B: <image>
C. <inage>
Only right defined the viewor' perspeztive
front of
Figure 19: Examples of canonical view selection with cars. The model must select the correct left, right, back view given a front-view reference.
38

---

Published as a conference paper at ICLR 2026
Task group: canonical view selection (face)
Task: face canonical view wewectionnnccee perspective, left
Question:
You are shown thee images in der: [1] Fronil view. [2] View A, [9] View B.
Select the image that best shows the left profile (penson's own left).
Front vew: <imago>
<image>
<image>
capital letter (A perspecilue (their right). The calfned from
Task: cat_eccnnnncaaaaaee view www.sceeleecceeeee Beaeen___o_______ caun__espacctvee. right
Question:
You are shown three images in order: Front view, [2] View [3] View D.
Select the image that best shows the right profile (parson's own right).
Front view. simaga>
A <imaga>
<mage>>
single gitt profile
(A
perspectivn right)
Task: face, canonical snto_sslocction_ cetturp_epppcttie_eeeee
Questio:
images
view. View
Select the image shows the prof (vlower's left).
Front view: <imege>
Ac <image>
B: <imege>
Only answor with sirgle copital lottrr (A
The efti ght profile defined from your perspactive as the viewer (your left right whon too dog the parson).
Task: facee www.ncccnnnaacc.coo.oooo fiew_selecction peawerrperspectiveeee right
You are shown three images order: [1] Frent view. [2] View A, [3] View B. Select the image that best shows the right profle (v ewer's right).
Piont aw <image>
A: <image>
B: <image>
Only ansuer with singla capital latter (A
The lefiight profile defired from your perspective viewer (your left right wher looking the
Figure 20: Examples of canonical view selection with faces. Tasks differ depending whether left/right defined from the subject's own perspective or from the external viewer's perspective.
39

---

Published as a conference paper at ICLR 2026
Task group: canonical view selection (object)
Task: @bjjet_ccnoniien www__seeetion_beee back
Question:
imagos
View
Select image that best shows the object from back de.
Front view: <image>
A <image>
B: <image>
4image>Only answer with single capital letter B, 'back side' side that back when looking the object
The back side defined the cameraiviewer'
Task: objjct_ccnonnicat wwwwnnccaoncacccccc sraiccallvieeeeeeeeeeeeee wwww__sllccioo_lofe
shown four images in order Front view, [2] View A, View B, [4] View Select the image that best shows the object from the left side.
Front view: <imaga>
A: <image>
<image?
<image>Orly with single capital letter B,
aide deflied cameraViewer's perspedive:
the appears
looking object
Task: object_canonical canonical wwwpwseletioon_fiee
Question:
You are shown four images in arder: (1) Front vlew, [2] View [3] View B, [4] View Select the image that best shows the object from the right side.
Front view: <image>
A: <image>
B: 4mmag>>
G SmmegeOOrll anowar with single capital latte (A C)
The right side defined from the cameralviewer's perspectiver 'right side' the side that the right when looking the object from front
Figure 21: Examples canonical view selection with objects. The model must determine left, right, or back views of generic objects such as furniture, based on given front view.
40

---

Published as a conference paper at ICLR 2026
A.4.6 Perspective Taking (View Selection) These tasks evaluate whether models can perform perspective taking when selecting the correct viewpoint of a scene, even when parts of objects are occluded. The challenge lies in integrating the reference view with multiple candidate perspectives, reasoning about hidden surfaces, and maintaining consistent spatial relationships. Variants differ in the extent of occlusion.
Full occlusion (Fig. 22): The model sees a reference front view and must choose among candidate views taken from back, left, or right perspectives, where large occluders hide significant portions of the scene. Success requires inferring unseen object sides.
No occlusion (Fig. 23): Similar setup. but with no major occlusion.
Partial occlusion (Fig. 24): Candidate views contain moderate occlusion (e.g.. objects partially blocking others). The model must still identify the correct viewpoint, balancing visible cues with inferred hidden structures.
A.4.7 Perspective Taking (Relative Position Transformation) This task group evaluates whether models can correctly predict how spatial relationships between objects transform under perspective shifts. Unlike view selection tasks, where the goal is to choose the correct viewpoint of scene, these tasks explicitly probe relational transformations: given a reference view, the model must infer how relative positions (e.g., left/right, near/far) are altered when the viewpoint changes to the back, left, right side. In other words, the challenge lies in mentally re-projecting the scene and predicting the new arrangement of objects from different vantage point. To diagnose different failure modes in spatial reasoning, we introduce premise- based variations in spatial transformation tasks. In the with-premise condition, the relevant spatial relationship (c.g., "A is to the right of B in the front view") is provided directly in the prompt along with the corresponding image, allowing the model to reason over an explicit linguistic premise. In the without-premise condition, no such information given, and the model must infer spatial relations from the reference image. This con- trolled comparison does not assume a specific order of grounding and reasoning, but instead helps identify whether failures arise from difficulties in extracting spatial relations from visual input, or from applying geometric reasoning given a known premise.
With premise (Figs. 25, 26, 27): The model is given a linguistic statement of the relative positions (e.g.. "X is closer than Y" or "X is to the left of Y"") alongside the image, and must predict how that relation transforms under a new viewpoint.
Without premise (Figs. 28, 29, 30); The model only sees the reference image and must infer the spatial relations itself before applying the geometric transformation to a new view- point.
A.4.8 Mental Rotation Mental rotation tasks evaluate a model's ability to simulate object trans- formations by imagining how an object's orientation changes under specified rotations. Unlike perspective-taking tasks, which require adopting a different viewpoint, mental rotation requires rea- soning about the intrinsic geometry of single object as it spins place. In these tasks, the model is presented with reference front view of an object and a description of rotation (e.g.. "rotate 135 degrees clockwise"). It must then select the correct image among several candidates that matches the object's new orientation. This requires integrating visual recognition with geometric transforma- tion, a key hallmark of human mental imagery. These tasks are particularly diagnostic because they isolate the ability to track orientation without introducing multi-object relations or cluttered scene grounding.
Object Mental Rotation (Fig. 31): The model is asked to determine the new orientation of a single object after a specified angular rotation. For example, given a chair in its canoni- cal front-facing view, the model must predict which candidate corresponds to 135-degree clockwise rotation. Success requires both accurate angle-tracking and strong spatial imag- ination.
The model is asked to determine the new orientation of the car after a specified angular rotation.
Car Mental Rotation:
Infinigen Mental Rotation:
The model must determine the new orientation of target object (e.g., a clamp) on a cluttered tabletop after it undergoes a specified angular rotation.
41

---

Published as a conference paper at ICLR 2026
Task group: perspective taking (view selection full occlusion)
Task:
infinigen_rotee caciin_rrtttttonote p_t.atioonselllctionnoo back_full_occeeeee pccllsion
Question:
You are shown four imeges order: [1] Front reference view containing the large marker, View [3] View B. View Views C the
three distinct angles, right, back image.
perspeciive
<image> Which of the three candidate views (A <image>, B <image>, or <image>) most Ikkal takon from back side the large marker?
Only snswer with single capilal letter from (A, G).
Task:
infinigen rotation seliictionneciionn_o left_full, full_ occlusion
Question:
images Frant reference containing the from
View
angles, right, From the viewer's perspective in reference image.
<image>
Which of the three candidate views (A simages, <image>, or C <image>) is most likely taken from the left side of the galatin box?
Only single capital
Task: infinigon_rotation, eesaaton_ssllction_iieeeo dcll_occiusion
Question:
shown four order: [1] Front reference view containing the bowl, View [3] View View C. angles, left, right, From the viewer's a perspective in the reference image. <image>
same
Which the three candidate views (A <image>, B <imaga,, <imaga>) is most likey taken from the right side of the bow!?
Only single capital
(A.
Figure 22: Examples of full occlusion perspective taking. Large occluders hide most of the target object, requiring inference about unseen sides.
42

---

Published as a conference paper at ICLR 2026
Task group: perspective taking (view selection no occlusion)
infinigen_roteee catprnoo_ottttotteeeee Baantnbbba_____ occlusion
Question:
Views A, and images order: [1]
containing the [3] order. angles, right,
From the viewer's perspective the reference image.
<image>
Which of the three candidate views (A <image>, B <image>, C <image>) most Ikely taken from the back side of the gelatin box?
Only single letter (A,
V
BLIO
Task: infinigen_rotannnnee rotationn__ooo cectiiio_sslctton___o__ Lentcctoon_et______ occlusion
Question:
You shown four in order. [1] Front reference view containing the marker, [2] View A, View [4] View C.
from dietinct angles, right, From the viewer's perspective in reference image.
<image>
Which of the three candidate views <image>, B <image>, <images) most likely taken from the left side of the large marker?
Only single capital (A,
infinigen_rotatteie swweeceeeeeeeeeeeeeeeeeeeeeeeeee right_no__oehe occlusion
Question:
containing the banana, [2] View [3] B, [4] Views A, B, and C show the same scene from three distinct angles, left, right, and back in random order.
shown images
Front
From the viewer's perspeclive the reference image.
<image>
Which of
candidate
<image>, <image>, <image>) likely from right
Only answer with a single capital letter from (A, B, C).
A
B
V
Figure 23: Examples of perspective taking without occlusion. Models rely solely spatial consis- tency across viewpoints.
43

---

Published as a conference paper at ICLR 2026
Task group: perspective taking (view selection partial occlusion)
Task: ceaniige_rrotttio seeeoctiseeeeeneeeeeeee back_paartal_ooccussinn occlusion
Question:
You shown four images order: Front reference view containing the drill, [2] View A, View View and
same scene From viewer's perspective in the reference image.
angles, right, random
<image>
Which of the three candidate views <image>, B 4images, C <images) most from the back side the dril?
Only answer with single capital letter from (A, B, C).
Task: infinigen_rotateee cotattooortttoo__eee www.rrioo_slllccionn_ooooooo seleccioonleefe partial_occhuar
Question:
You four arder: (1) Front reference containing the [2] View A, View View
distinct angles, right, From viewer's perspective the reference image.
<image>
Which of the three candidate views (A <image>, B kimage>, or O <image>) is most likely taken from the left side of the banana?
with single letter (A,
Task:
infinigen_rotie /tt.atioo_selecttionnnnnoooo _arttial_occuusion occlusioo
Question:
You are shown four images in order: Froni reference view containing the polled mesll can, [2] View A, [3] View B, [4] View C. Views B, and show the same scene from three distinct angles, left, right, and back in random onder.
From viewer's perspective reference <image>
<image>, <image>, <image>) likely
right pottod
Only answer with single capital letter from (A, B, C).
B
Figure 24: Examples of partial occlusion perspective taking. Candidate views contain moderate occluders, requiring reasoning across partially visible cues.
44

---

Published as a conference paper at ICLR 2026
Task group: perspective taking (relative position transformation w/ premise)
Task: infinigen_spaaite ssptsallaaaaapaaaaa relation transformation, pttpprmmss_beee back
Questio:
Kimage>
As pudding box closer and pottoc meat can is farther from the viewer the front vew, then when viewed from the back, which object now closer the viewer?
potted
pudding
Only
single capital letter (A,
tranriootrrassooomttiinnn w_premise
Question:
muslard bottle left mug right given viewed from appoars the new perspective?
mug
Only answer with eingle capital letter from (A, B).
Figure 25: Examples of perspective taking with relative position transformation (with premise), back view.
45

---

Published as a conference paper at ICLR 2026
Task group: perspective taking (relative position transformation w/ premise)
Task: infinigen_spaee seattn_sptttl_taaale relation___nte traniiorttranooottttionn. wt_pprrmseeeel Jeft
Question:
<image>
As polled meal la dloser and pudking box is farther from the viewer the given front view, then when viewed from the viewer's left side, from perspective?
object appears
potted pudding bax
Only answer single captial letter (A, B).
wtt.rrtiionttannfrrreaaaeee Task: infinigen_spetian relation_ transformation, ww_premise_lee Jeft
Question:
<image>
rubika cube is an the left and mug is on the right the given front view, then when viewed from the viewer's left side, which object is closer to the viewer?
A. mug
B. rubika cube
Only anewer with eiglle capital letter from (A, B).
Figure 26: Examples of perspective taking with relative position transformation (with premise). left view.
46

---

Published as a conference paper at ICLR 2026
Task group: perspective taking (relative position transformation w/ premise)
infinigen_ @effniign_epattall___eeea relation transformation, w_ wwwpppeeeeeeppe pemmise_rigtt
Question:
<image>>
large marker and farther from the front viewed from the viewer's side, which lerge the left from the perspective? clamp
Only single capital from B).
Task: papartia__elattootaaaarrtrtttei Bactilio__wwpemme__e____e
Question:
<image>>
mug to
right given
right object
A mug B. rubiks cube
Only answer with single capital letter from (A, B).
Figure 27: Examples of perspective taking with relative position transformation (with premise),
right view.
47

---

Published as a conference paper at ICLR 2026
Task group: perspective taking (relative position transformation w/o premise)
Task: infinigen_span cattioon_patal_____e spatial_relation rwwaaeeo_ttraaaeee tranosortttoooiitonnnommm wo. wtt_prrmmss__eee
back
Question:
<image>
The image showe the front view of scene. Now imagine viewing the same scene from the back. From the new perspective, which object appears the lef??
A. marker
B. clamp
Only single capital letter
Task: infinigen_spaiiiee spatial_relation_____ wwtpiittnnnfrmmtttoowoooooooo httpi/aattiowoopoommmeeee citto_prrmss___eee back
Question:
<image>
The image showe the front view of scene. Now imagine viewing the same scene from back, From the new perspective, which object
cloeer the viewer?
A. large marker
B. large clamp
Only answer with single capital lettor from (A,
Figure 28: Examples of perspective taking with relative position transformation (without premise), back view.
48

---

Published as a conference paper at ICLR 2026
Task group: perspective taking (relative position transformation w/o premise)
Task:
infinigen, sectcn__ptttrattiteeeeee relation, transformation pwwmpseeeeeeeeeeee left
Question:
<image>
The Image shows the front view of scene. Now imagine viewing the same scone from the viewer's left side. From the new perspactive, which
object closer to the viewer?
A. pitcher basa
B. mustard bottle
Only anewer with siigle capital letter from (A, B).
Task: infinigen_sptteiee peatiaalrelation, transformation, wo__ wo_prremise_____ left
<image>
image
imagine viewing
new perspective, wh
object now appears
potted
pudding box
Only answer with single capital letter (A, B).
Figure 29: Examples of perspective taking with relative position transformation (without premise), left view.
49

---

Published as a conference paper at ICLR 2026
Task group: perspective taking (relative position transformation w/o premise)
infinigen, Saaiinnsppaaa_aaaeeeeeee
_spattalreeaationnn relationn transformation wo_ ppepmiemse___mm ritht
Queetion:
<image>
The Image shows the front view of scenc. New imagine viewing the same scone from the viewor's right sido. From the new perspective, which object doser to viewer?
A. luna fish can
B. puodiin box
Ony answer with single capital latter from (A, B).
Task: infinigen_sspeee pat___spaaaaaaatttii relation, transformation plppppppmeetttee right
Question:
<image>
The mage shows the front view of scene. Now imagine viewing the same scene from the viewer's right side. From the new perspective, which objoct now appears on the loft?
A. masser chef can
B. sugar box
Only answer wirra singla capital lotter from (A, B).
Figure 30: Examples of perspective taking with relative position transformation (without premise), right view.
Task group: mental rotation
object_mental___etttt rotation
Question:
<image>
From original vieapoint, the object spins 135 degrees dockwise, view shows new orientation?
<image>
B. <image>
<image>
<image>
captial C,
Figure 31: Examples of mental rotation tasks. The task presents a reference object (c.g., sofa with cushions) and specifies degree of rotation (e.g., 135e clockwise). The model must identify which of the candidate views (A-D) corresponds to the rotated orientation.
50

---

Published as a conference paper at ICLR 2026
B DETAILED VLMS EVALUATION RESULTS
B. RAW ACCURACY AND COHEN'S KAPPA
In addition to the main grouped heatmap reported in the paper. we provide complementary visual- izations support detailed analysis of model performance. Figure 32 reports raw accuracy for the grouped 26 task variants, enabling comparison with the chance-adjusted results in the main text. Figures 33 and 34 further expand to the ungrouped 51 subtype level, presenting both Cohen's k and raw accuracy. Together, these heatmaps give complete view of performance across models, tasks, and evaluation metrics.
B.2 DETAILED CONSISTENCY EVALUATIONS
The benchmark employs two systematic augmentation strategies to probe reasoning consistency:
B.2.1 Augmentation Types
1. Symmetric Augmentation: Logically equivalent transformations that flip spatial relations while maintaining semantic meaning (e.g.. "Which object is on the left?" + "Which object is the right?" with corresponding answer adjustments).
2. Syntactic Augmentation: Surface-level reformulations that preserve semantic content while changing question structure (e.g.. "Which object is on the lefi?" + "Is object A the left or right of object B?").
B.2.2 Performance Metrics
Accuracy (%): Overall correctness rate calculated as:
CCorrect Responses
Accuracy
100%
Total Test Cases
Consistency (%): Average of pairwise consistency across task variants, where consistency is achieved when a pair of question variants yields identical outcomes (both correct or both incorrect).
Perfect Rate (%); Frequency of achieving complete consistency across all four question variants:
Cases with All-Agree Patterns
Perfect Rate
100%
Total Four-variant Cases
This includes both CCCC (all correct) and WWWW (all wrong) patterns.
As shown in Table 5, the InternVL model family continues to demonstrate outstanding stability in spatial reasoning. Nearly all Intern VL3 variants achieve consistency above 90% while maintaining strong accuracy and high perfect rates, making them the most uniformly reliable open-source models in the benchmark. Among proprietary models, gpt-5 remains the strongest overall model, achieving 78.5% accuracy and 97. .%0% consistency, setting a new standard for both competence and stability.
A strong positive correlation between accuracy and consistency evident across the entire model distribution, but the relationship becomes nonlinear higher performance levels. For example, gemini-2.5-pro attains the second-highest accuracy (75.2%) yet ranks noticeably lower in consis- tency (94.4%) compared with the Intern family and gpt-5. These deviations illustrate that once models reach very high accuracy, gains in consistency no longer translate proportionally into im- proved reasoning, and correctness, not merely coherence, becomes the limiting factor. Mid-tier models show significant variability in the achieve 73.5% consistency despite only 51.8% accuracy, indicating systematic but often incorrect reasoning patterns. Conversely, models like Gemma-3-27B maintain 59.8% accuracy but exhibit poor consistency at 53%, suggesting reliance on surface- pattern matching rather than robust spatial understanding, The Perfect Rate metric reveals additional nuances in model behavior. High perfect rates indicate models that, when consistent, tend to be systematically correct or incorrect across variants. Lower perfect rates suggest fragmented reasoning where models may correctly
trade-off. Models like Molmo-7B
51

---

Published as a conference paper at ICLR 2026
U1 18 aorer US0 Lucooos
88 ao 1omee
and
ai aenee ne
22000
di
USS
12
T00 Matching
11
11
160
163
50
12
01
123
13
10
11
11
50
022
02
11
11
123
150
113
69
11
X13
173
-2
-13
11
11
11
153
01
02
11
123
122
11
11
11
042
012
012
04t
55
553
59
590
550
92
24
Dynamic Rotation Dynamic Transiation
072
6.3
177
R2
72
12
113
12
22
24
74
22
74
17
55
153
55
22
13
24
22
1-5
153
155
172
172
113
222
122
22
22
2/4
1
12
650
000
000
20
100
100
300
113
50
100
500
02
LA
200
120
19
101
VLM Benchmark Results: Raw Accuracy
1... Sil aa
L.A
550
100
200
100
150
150
05
00
04
F1
11
112
11
11
11
00
11
11
19
11
112
Canonical View Selection
12
15
00
150
110
150
01
00
02
01
09
raw accuracy
21
04
14
11
12
150
01
04
01
04
04
94
22
Q3
27
04
05
045
12
(2
14
10
10
Object Ralatio Rolation Grounding
de
15
100
12
11
01
23
146
152
12
12
114
12
14e
E12
be
Li
01
120
F2
01
022
02
11
020
000
000
020
077
022
0.22
606
6.2
(1
12
0.00
000
11
011
00
02
0n
077
0x
Lek
110
13
13
000
240
150
130
113
11
113
113
110
123
11
02
Perspective Taking
ii an col ua an male sune
11
11
81
R1
000
100
10
10
20
11
11
15
12
11
11
0.
50
220
240
2:0
110
R20
210
112
11
112
11
15
11
R2
12
12
12
0
122
11
013
12
00
02
00
00
00
22
mm
Mental Rotation 135
22
57
653
123
11
F33
22
11
00
113
113
113
122
11
12
52
11
11
80
52
62
0
57
59
11
F3
113
523
17
82
8
Figure 32: Performance heatmap of 43 VLMs across 26 grouped task variants, organized under 7 spatial reasoning categories. Values show raw accuracy rather than Cohen's x. The 26 columns aggregate 51 fine-grained subtypes by domain or directional symmetry.
52

---

Published as a conference paper at ICLR 2026
11000
400
2-
11
114
1
INT
L
ONT
RAT
HT
Ray
INT
INT
L
SpinBench Evaluations: Cohen's Kappa (Chance-Adiinne Adjusted derfformncc
OP
F
Dreans Retadon Pyswait Inmiintion
120
F
AM
B2
150
F14
Cal
F14
112
F1
F22
F11
F20
F1
F2
F2
F1
11
F1
1
1
F1
LTD
11
LG
A4l
11
TE
AG
LTT
ICE
115
11
11
TUU
F5
4G
LED
SFF
HTT
KM
--
1
F
F6
12
14
F1
12
F1
F1
F1
12
FM
11
11
F-
La
Em
Odpect hedeeeet 1a
1
13
11
L2
+
10
STW
R5
R5
L
S6
28
02
UX URC
OT m m M0 10
I 1
12
F4
IK M5
TT M H
S IKE I
IXS
12
1
IT m
RTS 2
STC
F2
L4
UN m
m
m
HE O
4-
m
D
12
OCT
W
m
ET L E1
W MC
11Y
12
m
25
m
12 11
1
12
OF
M
R
15
11
F2
12
44
25
LS INC
12
Figure 33: Ungrouped subtype-Level heatmap of 43 VLMs showing Cohen's R performance across all 51 fine-grained spatial reasoning subtypes.
53

---

Published as a conference paper at ICLR 2026
Son to moneerartion
1
THE
FRI
1
8%
-
1
WT
WET
HKL
X
MK
+
65
F2
m
35
F4
12
06
18
F4
F8
F4
F8
F4
-
VLM Benchmark desults: Raw Accuracy
1.
11
F
F2
F5
m
15%
1
11
11
1
1
10
15
mm
15
10
1
F2
12
123
F1
F4
FTS
FF
11
10
F1
F9
F8
m
47%
m-
12D
1
1
11
1
m
OFF
15
12
LED
LFS
LED
FFF
4-
4-
Obter
XX
4
R5
85
85
85
4
82
85
W
R3
45 9
A5
85
B5
m
63
85
45
5-
W
+
13
LETT UR
110
S10
TH
KT
VP
AXC
D1C
S:
AXC
110
41C
J1
STC
VHS
2TC
KFC
615
ETT
FFF
FFS
11
1FF
LTT
110
FTS
FFC
AFC
AVC
190
FRI
41
XXR
NTC
MTO
DH
Figure 34: grained spatial reasoning subtypes. This complements Figure 33 by reporting unadjusted accuracy scores for the same set of subtypes.
Ungrouped subtype-level heatmap of 43 VLMs showing raw accuracy across 51 fine-
54

---

Published as a conference paper at ICLR 2026
answer some variants while failing others, indicating incomplete spatial representations. The sub- stantial performance gap between top and bottom models underscores the significant challenges in achieving consistent and correct spatial reasoning.
Table 5: Comprehensive performance ranking of 43 vision-language models spatial reasoning tasks. Accuracy represents overall correctness across all test cases. Consistency measures reasoning stability across question variants, and Perfect Rate indicates the frequency of achieving complete consistency across all four question variants (all correct or all incorrect).
Model
Accuracy (%)
Consistency (%)
Perfect Rate (
gpt-5
78.5
97.0
75.8
OpenGVL.ab V1.3 38B
70.3
95.3
75.1
OpenGVLLab_Inter VL3338B
72.6
95.7
71.1
gemini-2.5-pro
75.2
94.4
67.4
04-mini
71.0
91.1
73.2
OpenGVLab Intern VL3.5.14B
68.3
94.6
68.7
OpenGVLab Internnneenntteee VL3.14B
68.6
91.4
63.7
VL Qwen Qwen3 DL..3.VL.35 8B Instruct
67.4
87.4
70.0
Qwen Qwen3 3L.30B A3B Instruct
65.7
88.1
70.0
Bree__wee2.5__V...0 gpt-4.1
Instruct
65.4
85.7
62.7
67.2
85.9
59.5
Qwen Qwen2.5 Dwwn2.5.VL.33. VL.32B Instruct
67.3
85.7
62.7
OpenGVLab Intern VL.3 .B
65.6
82.4
66.7
OpenGVLab Jnttern__eenntee VL3_5 8B
68.6
83.9
59.7
OpenGVLab Intern VL3.88
66.7
77.4
62.7
BeenGVLab.InternVL..3. gpt-4o
65.8
77.1
61.7
67.8
79.6
51.2
Qwen Owen_Qwen3._ Lwwe3__V_4BB VL.44 Instrrcc
61.4
81.8
53.7
Qwen Qwen2.5 VL 7B Instruct
64.5
63.8
59.2
OpenGVLaab Intern V1.2 8B
61.2
65.9
57.7
claude-oomeet-4222555514
64.8
71.7
42,8
llave-onerrsioon-geenn-------v-
55.6
67.4
58.2
Cosmos-Recason--c00--1
64.1
69.5
41.8
Apenbbb_MiniCP%5
62.1
64.2
45.3
Dven.Qwen2.VL..777 Instruct
61.8
60.2
44.8
OpenGVLab InternVL3 2B
55.7
58.4
50.7
allenai Molmo 7B D 0924
51.8
73.5
35.8
OpenGVL.ab Intern V1.2_554B
55.9
63.4
39.3
OpenGVLab_Inteed 1tLGVLLLaaneeeL3335522222 VL3_5_2B
56.3
59.5
42.3
Ilava.hf interleave qwen.
58.9
65.2
31.3
Owvn.. VL..BB Instruct
55.3
53.4
38.3
google_gentee 3et.eeeemma332230m 3.27b
59.8
53
25.4
Cosmos-Reason1178
58.8
40.9
43.3
googls_gemma.3.122..2
54.3
58.4
25.9
ThaceTThinerr-@we222551-.-1
51.4
53.4
33.3
SpaceOm-cot
53.2
51.6
31.8
microsoft Phi 3.5 vision instruct
57.7
40.5
23,9
OpenGVLabb 2.2 claude-3-S-haiku
50.5
50.5
21.4
57.3
36,9
21.4
47.8
40.1
15.9
OpenGVLab IB
47.2
35.1
22.4
OpenGVLab Littaa__ItterVVee VL2.5_IB
46.4
37.6
19.9
SpaceQwen2.5- V---255-----B----- 3-33V--3--nnttruc
49.4
33.7
16.4
Qwen Qwen2 VL 2B Instruct
48.3
29
17.9
Continued on next page
55

---

Published as a conference paper at ICLR 2026
Table continued from previous page
Model
Accuracy (%)
Consistency (%)
Perfect Rate (%)
OpenGVLLab InternV1.3 1B internlm_intemlmm_en Xtt.ee/lmm.cooopse222..55555....
48.8
28.3
16.9
43.3
36.6
13.9
SpaceOm
49.2
17.9
18.4
Space Thinker--wee225VV133
48.7
11.1
8.5
B.2.3 Augmentation Strategy Analysis In Fig. 36, the augmentation comparison reveals simi- lar consistency rates across different transformation types (66-68%), with symmetric augmentations performing marginally better. The small performance gaps (less than 3 percentage points) suggest that current vision- lision-language models exhibit similar levels of sensitivity to symmetric and syntactic augmentation. The substantial error bars (approximately +8-10%) indicate variance in augmentation sensitivity across models.
B.2.4 Pattern Distribution Analysis The stacked bar chart Fig. 35 reveals several key insights into model consistency behavior. High-performing models (topmost bars) demonstrate substantially larger proportions of perfect consistency patterns (CCCC, all four variants correct), with top models achieving above 60% perfect consistency rates. Conversely, lower-performing models show more fragmented pattern distributions with higher prevalences of mixed consistency patterns and complete failure modes (WWWW). The visualization demonstrates clear correlation between overall accu- racy and consistency stability. Models that perform well on spatial reasoning tasks also maintain more coherent reasoning across question variants.
B.3 CORRELATION ANALYSIS
We calculated correlations between our diagnostic benchmark and four established spatial reasoning benchmarks at both overall and subtask levels (Table 6). Overall correlations between diagnostic benchmark and holistic benchmarks were weak and non-significant: MindCube (r = -0.088, 0.836), NiewSpatial-Bendd (r 0.460, p = 0.299), OmniSpatial (r 0.456, p 0.137), and SpaCE- 10 (r = 0.098, 0.803). These results validate that our approach captures distinct foundational capabilities rather than general spatial intelligence. Subtask correlations revealed targeted diagnostic relationships (Figure 37, 39, 38. 40). Significant correlations emerged between specific diagnostic and benchmark subtasks: dynamic rotation abilities strongly predict 3D reasoning performance in MindCube (r 0.829, p 0.021), identity matching correlates with person-based based perspective taking in ViewSpatial-
= 0.915, p 0.030), and static reasoning predicts object manipulation capabilities in OmniSpatial (r = 0.764, p 0.006). SpaCE-10 10 showed no significant correlations, suggesting it evaluates distinct spatial reasoning components.
These patterns demonstrate that our diagnostic benchmark provides complementary rather than dundant evaluation. While overall performance correlations are minimal, specific subtask relation- ships reveal how foundational spatial deficits contribute to failures in complex holistic tasks, en- abling targeted identification improvement areas.
Table 6: Overall Average Correlation Analysis: Diagnostic Benchmark vs. Holistic Spatial Bench- marks
Benchmark
number of models
Pearson r
p-value
MindCube Yin et al. (2025)
8
-0.088
0.836
ViewSpatial-Bente Li al. (2025a)
7
0.460
0.299
OmniSpatial Jia et al. (2025a)
12
0.456
0.137
SpaCE-10 10 Gong et al. (2025)
9
0.098
0.803
C HUMAN EVALUATIONS
We conducted human evaluation with twelve subjects to establish performance baselines and val- idate task difficulty. One subject completed the full benchmark (2,599 questions), while eleven
56

---

Published as a conference paper at ICLR 2026
Poftern Rate (96)
Consistency Pattem Distribution Across Models (n=46)
5
gengn-25:pr
OpenGVV.vd [=len/i23__0
0o0n@VLa0_innmmMM...5. 866
OpenGVV.ve J=lenki3.. 340
OpenGVLab j_w.noVa_____1.4
@wen__@ena__V... 38 88, Jnssuct
Qwe1, www.oeennnn.. 808 888 Instuct
Qoea, Quen25_VV, 32B
Opend Benndvtat..mmmV.B.... 86
Open@VLat_Jmemn IntenvL3..ee 96
OpenGitaly hw
dbude sonnet 23250614
Qwen, Qw..Qwen22 Inssus
(oureae dq possss) stapop
Qwen__wwent Owwnn VI, 101 Juctus
Qwon, Swont_QWon2VL... VL Instuct
DpendVt ah, Inte/112 5,
Cremos Cremox-Reeentt-o Rm
LAON, H. Iaw, www.reeee ppon, Th. Jr
ricroot Fhy 315 stéon, nemi
halu
Opend ah, 2n 2.5 5,4
Opentituoo Jnlenvi
Qwon, Owen2 38 Insous
pwrgl/_eemmm_____.. _
SpaceCm
Space Thinker Qwer2 SVL 9VL-B cor
ailenal Molmoooooo 70_ 0000
Opendd 2_5
Space Qwan2 5 VL 38 Insous
SpackOm
SpucsTTestee
Osend VLab. intevvi3 18
Qwan, Swon2 28 Insuuct
progie_gemma__3 www.gammmaa..m 3
Opendvtab InneviS 18
OsenCVtab Inter/22 18
CCCO
CCDV
cowe
CoW
CWoc
CWNG
WooN
WONE
WOWw
wwcc
www.
www.e
Figure 35: Comprehensive consistency pattern distribution across all 46 vision-language models, sorted by overall accuracy (top to bottom). Each stacked bar represents the percentage distribu- tion of all 16 possible consistency patterns (CCCC through WWWW) for 4-variant question sets. Green shades indicate patterns with more correct responses (C), while red shades represent patterns with more wrong responses (W). Models with higher accuracy (left) show greater prevalence of all- correct patterns (CCCC, dark green).
57

---

Published as a conference paper at ICLR 2026
Consistency by Augmentation Type (n=46 models)
60
Average Consistency
50
40
30
Symmettic
Syntadic
Augmentation Type
Figure 36: Average consistency rates by augmentation strategy across 46 vision-lanuuage models with 4-variant question sets. Error bars represent standard deviation across models. Symmetric aug- mentations (question reformulations maintaining logical equivalence) achieve slightly higher con- sistency than syntactic (surface-level rephrasing) and cross-augmentation (mixed transformations) approaches.
58

---

Published as a conference paper at ICLR 2026
Correlation Matrix: Diagnostic Subtasks vs MindCube x indicates 0.05)
1.00
dhenbity metchhing
0.268
0.50
Diegnostic Benchmark Subtesio
0.25
dynamiz, dymanic_prismaan prosmatiiic
0.124
0.003
0.216
Jynamic rutary
0.823
0.489
0.776
-00.55
single 0.457
-0.75
mult pafecc_.srapecctss
-1.0
Potation
Amenng
Artund
MindCube Subtasks
Figure 37: Subtask-level correlation matrix between SpinBench components and Mindcube. Rows represent our six diagnostic subtasks, columns represent subtasks from MindCube (n=8). Color intensity indicates correlation strength: red denotes positive correlations, blue denotes negative cor- relations.
Correlation Matrix: Diagnostic Subtasks
ViewSpatial-Bench
Indicates p
3.00
iientiy menthyymatchingg
0.376
Dingrosiic Henehmark
dymami prismatie
0L685
3,007
0.252
0.662
0.545
3.20
cynami_ywsary-
0.054
0.524
-0.22
sigle_oNicct_ee pesseccctpeeeectieeee
0.853
0.801
0.616
muht chject,etccccccc.cc persoective
0.563
40354
0.630
0.086
0.24
-1.00
Camera Olic Del
Bersor Del Dn
Peron Ret Di
Sant
Cemera Ret
ViewSpatial Bench Subrasks
Figure 38: Subtask-level correlation matrix between SpinBench components and ViewSpatial- Bench. Rows represent our six diagnostic subtasks, columns represent subtasks from ViewSpatial- Bench (n=7). Color intensity indicates correlation strength: red denotes positive correlations, blue denotes negative correlations.
59

---

Published as a conference paper at ICLR 2026
Correlation Matrix: Diagnostic Subtasks SpaCE-10
(n 5, indicates < 0.05)
0.75
0.58
x s.matching
-0.511
-0 020
-0.022
0.183
-0.240
00.222
-0.318
-0.01
Pesrson Condiattoo
corromatic
3.721
0.401
0.033
0.288
0.448
0.280
0.150
0.007
0.000
0.043
0.173
0.317
-0.25
serspective
-0.442
0.545
-0.50
perspective -0.622
-0.333
0.021
-0.246
-0.387
-0.293
0.314
-0.231
SpaCE-10 Subtasks
Figure 39: Subtask-level correlation matrix between SpinBench components and SpaCE-10. Rows represent our six diagnostic subtasks, columns represent subtasks from SpaCE-10 (n=9). Color intensity indicates correlation strength: red denotes positive correlations, blue denotes negative cor- relations.
1.00
Correlation Matrix: Diagnostic Subtasks OmniSpatial Indicates p 0.05)
statc
0.764
0.195
0.021
0.725
0.545
0.278
0.240
0.530
0.256
0.135
0.50
0.524
Naasinn Carrel Carwatioo addon
0.,00
amic_rotery
0.175
0.505
40.132
0.3.9
0.300
-0.305
-0.990
0.251
0.082
-034
-0.25
zenpective
0.905
0.125
0.191
0.399
0.208
0.331
0.3/8
0.196
0.231
0.306
zenpective
0.525
0.446
0.155
0.289
0.299
0.180
0.522
0.122
0.147
-0442
Manipulase
Mation, Anadyeta
Rrafie Analpois
Lecate
Geosuatial Strategy
Pettrr Pecognitinn
Grnmetiic measoring
Egg Centrie
Als Centle
Hypattutical
-0.75
OmaiSpatial Subtasks
Figure 40: Subtask-level correlation matrix between SpinBench components and OmniSpatial. Rows represent our six diagnostic subtasks, columns represent subtasks from OmniSpatial (n=12). Color intensity indicates correlation strength: red denotes positive correlations, blue denotes nega- tive correlations.
60

---

Published as a conference paper at ICLR 2026
others completed balanced samples of 200 questions each, with equal representation across all task subtypes.
C.I HUMAN EVALUATION TOOL DESIGN
We developed specialized application for human evaluation. The tool handles diverse question formats automatically, from single reference images with text options to complex multi-image image options mental rotation tasks.
C.1.1 Question Type Detection and Display The system automatically parses question struc- ture using pattern matching to distinguish between reference images and selectable options. For spatial reasoning tasks with text choices (e.g.. "A. mug, B. mustard bottle"), it displays the reference image alongside clearly labeled text options. For mental rotation tasks presenting multiple candidate views, it identifies the initial reference state and labels the four candidate images as selectable op- tions. This smart labeling prevents confusion about which elements are answerable choices versus contextual information.
C.1.2 Progress Management and Resumption The tool implements progress tracking with au- tomatic saving after each response. Subjects can resume interrupted sessions seamlessly. Questions are grouped and sorted by task type minimize cognitive switching costs, with pop-up notifications when transitioning between task categories.
C.1.3 Dataset Curation Integration Beyond collecting responses, the tool also supports real- time dataset quality control. Subjects can flag ambiguous or problematic questions for removal using dedicated key. This dual-purpose design allows human evaluation simultaneously serve both performance benchmark and a dataset refinement process.
C.1.4 Response Collection The interface uses numbered keyboard input (1-4 corresponding to A-D) for efficient response collection, with visual feedback for correctness and validation to prevent invalid inputs. All responses include precise timestamps for response time analysis, with automatic filtering of extended intervals (> 180s) that indicate interruptions rather than genuine decision time. The tool generated detailed logs in JSONL format containing individual responses, tasc-specific performance breakdowns, and timing statistics, enabling comprehensive analysis of human perfor- mance patterns across different visual reasoning categories.
C.2 HUMAN PERFORMANCE
Table 7: Human Performance Statistics by Task Subtype. Accuracy and response time statisties averaged across 12 human subjects, organized by task group categories.
Task Subtype
Accuracy Mean SD
Response Time
Mean + SD
Canonical View Selection
car canonical view selection back car canonical view selection left canonical view selection right face canonical view selection own perspective left face canonical view selection perspective right
1,000 + 0.000
12.5 + 10.7
0.771 = 0.391
11.4 6.7
0.813 + 0.386
6.7
0.750 + 0.369
17.2 10.2
0.833 0.389
6.2
face canonical view selection viewer perspective left
0.771 + 0.391
9.3
face canonical view selection viewer perspective right
0.792 + 0.382
5.3
object canonical view selection back object canonical view selection left object canonical view selection right
0.999 + 0.004
+.4
0.957 0.097
9.7
0.979 + 0.072
5.9
Identity Matching
Continued next page
61

---

Published as a conference paper at ICLR 2026
Table continued from previous page
Task Subtype
Accuracy
Response Time
Mean SD
Mean + SD (s)
car identity
0.999 + 0.004
6.7 + 4.1
car identity quartet imagefirst
0.963 + 0.088
5.9 4.3
car identity quartet interleaved
0,958 + 0.097
10.0 + 5.2
car identity quartet textfirst
1.000 0.000
3.3
face identity
1,000 + 0.000
4.2
face identity quartet imagefirst
1.000 0.000
4.1
face identity quartet interleaved
1.000 + 0.000
+.3
face identity quartet textfirst
0.958 0.097
3.9
object identity imagefirst
1,000 = 0.000
1.5
object identity interleaved
0.979 = 0.072
2.5
object identity quartet imagefirst object identity quartet interleaved
0.998 + 0.007
2.6
1.000 = 0.000
3.3
object identity quartet textfirst
0.979 + 0.072
2.7
object identity textfirst
1.000 + 0.000
+
Dynamic Rotation
car rotation classification
0.999 0.004
19.0 11.1
face rotation classification perspective face rotation classification viewer perspective object rotation classification imagefirst object rotation classification interleaved object rotation classification textfirst
0.806 + 0.220
12.2 + 6.8
0.624 0.390
14.3 12.4
0.917 + 0.207
7.34
1.000 + 0.000
8.5 +.2
0.972 + 0.096
Dynamic Translation
infinigen spatial relationship dynamic front back infinigen spatial relationship dynamic left right
0.792 + 0.351 0.958 0.097
17.4 + 12.9
8.5 + 4.0
Object Relation Grounding
infinigen spatial relation grounding far near infinigen spatial relation grounding left right infinigen spatial relationship front behind
0.810 0.240 0.956 + 0.097
10.5 + 3,8
13.6 + 5.3
0.998 + 0.007
13.9 + 7.1
Perspective Taking
infinigen rotation selection back full occlusion infinigen rotation selection back no occlusion infinigen rotation selection back partial occlusion infinigen rotation selection left full occlusion infinigen rotation selection left occlusion
0,813 + 0.155
33.0 + 15.0
0.873 + 0.167
24.4 14.5
0.825 + 0.225
0.958 + 0.144
2
0.938
6.54
infinigen rotation selection left partial occlusion infinigen rotation selection right full occlusion infinigen rotation selection right no occlusion infinigen rotation selection right partial occlusion infinigen spatial relation transformation w premise back
0
1.45
0.938
6.5
0
15.8
0.938
13.8
0.072
25.0
infinigen spatial relation transformation premise left
0.957
18.6 6.4
infinigen spatial relation transformation w premise right
0.938
17.46.4
infinigen spatial relation transformation wo premise back
0.979
16.6 5.3
infinigen spatial relation transformation wo premise left
0,991 0.021
14.8 - 5.0
infinigen spatial relation transformation wo premise right
0.913 0.161
13.6 + 5.8
Mental Rotation
object mental rotation
0.749 + 0.321
17.2 + 8.8
Overall
0.921 0.091
11.6 + 6.9
Overall Performance. Human subjects achieved high overall accuracy (0.921 + 0.091) across the benchmark, detailed in Tables 7 and demonstrating that while tasks vary significantly in cognitive difficulty. they remain within human capability. Some task groups showed excellent accuracy, with Identity Matching achieving the highest performance (0.988 + 0.052). The primary
62

---

Published as a conference paper at ICLR 2026
Task Group Performance Ranking (by Response Time across Subjects)
Perspective
RG
Menral Rotzaton
Dynamic Transiation
Oitjed
Dynamic Pattation
12.46
Cenorccal Selecion
Hennty Manshing
30
Mean Response Timee (seconds)
Task Group Performance Ranking by Human Response Time. Perspective Taking emerges as the most cognitively demanding task group (19.0s), followed by Mental Rotation (17.2s) and Dynamic Translation (12.9s). Identity Matching tasks show the fastest response times (4.4s), indicating 4-fold difficulty range across major cogmitive categories.
Figure 41:
exceptions were Mental Rotation (0.749 = 0.321), which showed the highest variability and included some of the most challenging scenarios in the benchmark. Response times varied dramatically across tasks, ranging from 2.1 seconds for the fastest subtypes 33.0 seconds for the most challenging. indicating substantial variation in cognitive difficulty.
Table 8: Human Performance Summary by Task Group. Accuracy and response time statistics across 7 major task categories. Accuracy and response time means are averaged across 12 human subjects within each task group. Response time range shows the span of mean response times across different subtypes within each group (fastest to slowest subtype mean).
Task Group
Accuracy Mean + SD
Response Time Mean + SD (s)
Response Time Range (s)
Tasks (n)
Identity Matching
0.988 0.052
4.4 + 2.6
2.1 10.0
14
Canonical View Selection
0.866 + 0.301
9.2 6.3
5.3 17.2
10
Object Relation Grounding
0.921 0.149
1.6
10.5 14.0
Dynamic Translation
0,875 = 0.257
1.6
-..4
2
Dynamic Rotation
0.886 0.205
8.5
8.5-
Mental Rotation
0.749 + 0.321
17.2 + 8.8
17.2
1
Perspective Taking
0.928 0.128
19.0 9.4
13.6 33.0
15
Overall
0.921 0.091
11.6 - 6.9
2.1 -3.0
Task Group Difficulty Ranking. Analysis of the full benchmark results reveals clear difficulty hi- erarchies across major task groups, as shown in Figure 41. By response time, the most challenging groups are: (1) Perspective Taking (19.0s), demanding viewpoint reasoning often under occlusion; (2) Mental Rotation (17.2s), requiring complex 3D spatial transformations; (3) Dynamic Transla- tion (12.9s), involving spatial movement tracking: (4) Object-Relation Grounding (12.78), requiring analysis of spatial relationships between multiple objects; (5) Dynamic Rotation (12.4s), involving rotational movement classification; (6) Canonical View Selection (9.2s), focusing on optimal view- ing angles; and (7) Identity Matching (4.4s), the fastest category involving object recognition across viewpoints.
The most demanding individual subtypes, detailed in Figure 42, include perspective-takinn tasks under full occlusion (infinigen rotation selection back full occlusion: 33.0s), complex spatial trans- formations (infinigen spatial relation transformation with premise back: 25.0s), and partial occlu-
63

---

Published as a conference paper at ICLR 2026
sion scenarios (infinigen rotation selection back partial occlusion: 25.3s). Conversely, the fastest responses occur in identity matching tasks, particularly object identity quartet text-first (2.1s), sug- gesting these tap into rapid visual recognition processes that require minimal deliberative reasoning.
Task Subtypes Performance Ranking (by Response Time across Subjects)
sttaian aduciin
31.0v
anting que sttrinnatic snleccinn buch codtaion
26.36
retrgge spcaal diaiu tnnndormir promiet tack
onisgen sctaiion selictionn cocusion
20.40
sotatin cleatiiation
bock
17.48
vntroce sporse snnettonniio prrsic sight
en roation
dingge sounial rotarin tunakor abon wo pirmas buck
16.00
eigen endeetenrige codtution
15.30
iringgn saaion dian lat partal cocasionn
155s
scssio oottssoo
34.26
pra con di tre boeid
1igh
aagee apulial rolubrr di ight
Buc traaiin cusiai pur parpactie
12.6
cancrica leckoo
18.00
ddject canonical yonu, lecker
severrriiion cluan cllatiicatee enciossd
8.0g
dening
5.76
conmiccal adlectis iga
abvitty quance
5.30
iidery markoard
A.5m
fuce stintty sarn thrkou
3.00
dijc surtly mmgelts
sngect duadity quats imagetsu
Caseriaa Vew
2.36
Transainn
Santic trae--rieez
Maspeeiiee tuking
10
Mean Response Time (seconds)
Figure 42: Detailed Subtype Performance Rankings. The 51 task subtypes ranked by mean hu- man response time, revealing extreme variation from 2. to 33.0s. Perspective-taking tasks under occlusion (dark teal) dominate the most challenging subtypes, while identity matching tasks (yel- low) cluster among the fastest responses. Color coding indicates task group membership.
Task Design Implications. The human performance data validates benchmark's 's difficulty gra- dient and identifies genuinely challenging spatial reasoning scenarios. Tasks combining multiple cognitive demands-such perspective taking under occlusion spatial transformations requiring
64

---

Published as a conference paper at ICLR 2026
premise integration emerge as the most demanding, requiring both extended processing time while generally maintaining high accuracy. The 4.3-fold difference between the easiest (Identity Match- ing: 4.4s) and hardest (Perspective Taking: 19.0s) task groups demonstrates that our benchmark successfully spans a wide range of spatial reasoning difficulties, from rapid visual recognition to complex 3D transformations requiring nearly half minute of deliberation.
C.3 CORRELATION ANALYSIS
Human- VLM Performance Correlation. validate that our benchmark captures genuine spa- tial reasoning difficulty rather than arbitrary task complexity, we analyzed the relationship between human cognitive load and VLM performance across task subtypes. We calculated the correlation be- tween mean human response times (averaged across 12 subjects per task) and mean VLM accuracy (averaged across 37 models per task) for each the task subtypes our benchmark.
Subtype Level: Longer Human Time Lower VLM Accuracy Pearson -0.5399 (p 0.000044) Spearman = -0.5082 (p 0.00014)
Quadratic Linear
Human Response Time (seconds)
Commn Coymaneee Time (6)
Inaniom rocution so soletioon back purtill occission
VLM Accuracy (Average across models)
Figure 43: Hluman-VLM Performance Correlation. Scatter plot showing the relationship be- tween VLM accuracy (x-axis) and human response time (y-axis) across 51 task subtypes. For each task subtype, we computed: (1) mean human response time by averaging individual response times across all human subjects who completed that task, and (2) mean VLM accuracy by averaging performance across all 37 evaluated rision-language models that same task. The correlation anal- ysis treats each of the 51 task subtypes as an independent observation, examining whether tasks that require more human cognitive effort (longer response times) also prove more challenging for VLMs (lower accuracy). Color intensity indicates response time difficulty, with annotations highlighting the most challenging outliers.
Our analysis revealed significant negative correlation between human response times and VLM accuracy (Pearson -0.5399, p < 0.0001; Spearman -0.5082, 0.0001, 51 tasks), as illustrated Figure 43. This moodrrate-t--stroe correlation demonstrates that tasks requiring longer human processing time consistently challenge VLMs more severely, providing empirical evidence that our benchmark captures fundamental spatial reasoning difficulty shared across human and artificial intelligence systems.
65

---

Published as a conference paper at ICLR 2026
Cognitive Load. The correlation analysis reveals that human cognitive load, measured by sponse time, systematically predicts VLM performance degradation. Tasks in the upper-left re- gion of Figure 43 with both long human response times and low VLM uccuracy-represee the most cognitively demanding spatial reasoning scenarios in our benchmark. These include com- plex perspective-taking under occlusion (e.g., infinigen rotation selection back full occlusion: 33.0s response time), spatial transformations with premise integration (c.g., infinigen spatial relation trans- formation with premise back: 25.0s), and challenging mental rotation tasks (17.2s). Notably, while humans maintain high accuracy even on these slow tasks through extended deliberation, VLMs show systematic accuracy degradation on these same challenging scenarios. This divergence suggests that humans can leverage additional processing time to overcome spatial reasoning difficulties, while current VLMs face fundamental limitations.
Benchmark Validity. The systematic relationship between human cognitive difficulty and VLM performance provides strong evidence for our benchmark's construct validity. Rather than testing arbitrary visual challenges, our tasks appear to probe fundamental spatial reasoning capabilities that require significant cognitive resources for both human and artificial intelligence systems. The contrast between human limitations (lower accuracy regardless of computation time) highlights important gaps in current vision-language models' spatial reasoning abilities. This alignment suggests that improvements in VLM performance on our benchmark likely reflect genuine advances spatial reasoning rather than dataset-specific optimizations.
trade-offs (high accuracy with longer processing) and VLM
Benchmark Validity. The systematic relationship between human cognitive difficulty and VLM performance provides strong evidence for our benchmark's construct validity. The negative correla- tion indicates that our tasks probe fundamental spatial reasoning capabilities that require significant cognitive resources across both biological and artificial intelligence systems. Rather than testing arbitrary visual challenges or dataset-specific artifacts, the alignment demonstrates that our bench- mark captures core spatial reasoning demands. The contrast between human adaptive processing (achieving high accuracy through longer deliberation) and VLM limitations (showing lower accu- racy) highlights important gaps in current usioonlanguuge models' spatial reasoning capabilities. This alignment suggests that improvements in VLM performance benchmark likely reflect genuine advances in spatial reasoning.
D DETAILS ON THE VLM EVALUATION SETUP
D. EVALUATION CONFIGURATION
All models were evaluated with consistent parameters to ensure fair comparison:
Temperature: 0.0 (deterministic sampling)
Top-p: 1.0 (no nucleus sampling restriction)
Image preprocessing:
Multi-image inputs were processed by interleaving text and image tokens according to each model's expected format.
We employed robust pattern matching to extract answers (A, B, C, D) from model responses, checking for structured tags first <Kanswer><</anseer followed by stan- dalone letters with word boundaries.
Answer extraction:
Referring to Chow et al. (Chow et al., 2025), during VLM evaluations, we appended an end prompt each question-answer pair. The end prompt follows, depending on the actual option number for each task, as in Tab, 4:
Only answer with a single capital letter from (A, B) Only answer with a single capital letter from (A, B, C) Only answer with a single capital letter from (A, B, C, D).
66

---

Published as a conference paper at ICLR 2026
D.2
MODEL IMPLEMENTATIONS
D.2.1
LMDEPLOY-SUPPORTTE MODELS
For the majority of open-Soure models, we utilized LMDeploy (Contributors, 2023; Zhang et al., 2025a), high-throughput inference engine optimized for large language models.
Models using LMDeploy:
InternVL series: InterVL22.5 (IB-8B) Intern (IB-38B), InterV (IB-38B)
Qwen-Vic VI. series: Qwen2-Vw. (2B-7B) Qwen2.5-VL. (3B-32B)
Gemma seriessgemma--34b-ii.. 3em gemia---
Additional models: Phi-3.S-vision-imsttuuc, MiniCPM-V-2.66 Molmo-7B, llav-interlccave qwen-7b-hf
Configuration: We configured tensor parallelism (TP) settings based on model size: TP=1 for models less than 8B parameters, TP=2 for models less than 16B parameters, and TP=4 for larger models. Backend selection was automatically determined based on model compatibility, with Tur- boMind preferred for supported architectures and Py PyTorch fallback.
D.2.2 OTHER MODELS
For models not supported by LMDeploy or requiring specialized handling, we employed the Hug- gingFace Transformers library with model-specific processors.
LiaVA-OneVision Model: We used official VanV--nneVision implementation with and applied the chat template format for LlavaOnevisi wwwp:/nwwwiiiioooooooooooooo www.rroonnniiiianeeeeeeneeeeeeee multi-image inputs.
Spatial Reasoning Models: For SpaceOm, SpaceThinkerrwwwe2..5--3.3 and SpaceQwen2.55 VL-3B-Instruet. we utilized Qwen2. "ILoorCondiitiona.Geeeettatttt with specialized chat templates supporting structured reasoning formats.
Meanos-Reason17755 Model: we used the official LLaVA-OneVision implementation with vLLM (Kwon et al., 2023) with specialized vision processing utilities handle multi-moda in- puts efficiently.
D.3 PROMPT FOR REASONING MODELS
In section 4.2, we evaluate the impact of CoT prompting across three specialized spatial reasoning models: Cosmos-Reason NVIDIA et (2025), SpaceOm Jia et al. (2025b), SpaceThinker Chen et al. (2024). We provide the prompts for each model: The prompt for Rosmos-Reasonl NVIDIA et al. (2025):
You are a helpful assistant. Answer the question in the following format: "<think> \nyour reasoning'n</thin>>> Lanswer>\nyon answer\n</answer>>
The prompt for SpaceOm Jia et al. (2025b) and SpaceThinker Chen et al. (2024):
You are Vi-Thinking, a helpful assistant with
excellent reasoning ability.
You should first think about the reasoning process and then provide the answer. Use <think>... ..innk...<<think> and
tags.
67

---

Published as a conference paper at ICLR 2026
E FINETUNING VLMS
To further validate whether improvements in foundational spatial reasoning skills transfer to the tar- get perspective-taking task, fine-tune t and Qwen3-VL-8B-Instrue using GRPO for 6 epochs on 5,900 curated samples spanning four core spatial reasoning abilities: mental rotation, canonical-vies selection, identity matching, and dynamic rotation.
E.1 CROSS- TASK AND CROSS-DOMAIN VALIDATION.
Importantly, the validation set drawn from different task type and domain than training. Training samples contain object and face, while validation performed on Seinigen-renderee 3D scenes from the perspective-takinn task in SpinBench (not used in training). This setup evaluates both task- type generalization (from basic transformations to viewpoint reasoning) and domain generalization (from object/face images simulated 3D scenes).
E.2 TRAINING AND VALIDATION RESULTS
Figure 44 shows the GRPO learning curves.
We additionally report the initial validation accuracy reward (Step 0) and the highest validation aceu- racy reward achieved during training. These metrics reflect improvements the target perspective- taking task:
Model
Initial Val Accuracy Reward Highest Val Accuracy Reward
Vwwe3--LL-4B-Inmee Instruct Qwen3-VL-8B-Instee
0.344
0.389
0.302
0.398
Table 9: Validation accuracy reward the perspective-taking task before and after GRPO fine- tuning. 8B models improve noticeably from their initialization, demonstrating transfer from basic spatial skills viewpoint transformation.
Training Reward
Reward
Validation Reward
0.9
0.38
Training Reward
0.36
0.8
0.7
0.32
Validation
0.6
(Train)
Qween3-VL-B-innsrree
Qwen3 WL-B---WL-BB-tttrre (Train)
0.28
LL (Val)
100
150
100
150
Step
Step
Figure 44: Training and validation reward curves for (Wwen3-VL-IB) -nntt---rttttt and Qwen3-VL VL-8B- Instruct under GRPO fine-tuning, The 8B model demonstrates greater improvement on the held-out perspective-taking validation task, consistent with its larger capacity. The 4B model shows slight rise followed by decline in validation reward, which may indicate partial overfitting to the training domain.
E.3 GRPO TRAINING CONFIGURATION
We provide the full GRPO training configuration used for fine-tuning Qwen3-VL-oe BE----LL----InnEe and Wwen3-VL-SB-lInttuue The same configuration is used for both 4B and 8B models unless otherwise noted. During rollout, we generate n samples per prompt with sampling temperature 1.0 and top-p 0.99. Validation uses more deterministic setting with temperature 0.5 and = 1. Training uses global batch size 128, gradient norm clipping at 1.0, Adam optimizer with learning rate 1 x 10-6 and weight decay 1
68

---

Published as a conference paper at ICLR 2026
F MORE RELATED WORKS
F.1 SPATIAL REASONING BENCHMARKS
Beyond traditional vision-language datasets, BLINK (Fu et al., 2024) introduces tasks that humans can solve "within blink," but which remain challenging for multimodal large language models (MLLMs). These tasks highlight persistent gaps between human perception and model capabil- ities- particaaull in spatial reasoning. Recent benchmarks offer complementary perspectives on spatial reasoning: MindCube (Yin et al., 2025) and VSI-Bench (Yang et al., 2025b) focus on how MLLMs construct internal representations of space, a process analogous to cognitive mapping. These benchmarks primarily evaluate advanced, compositional tasks such as object identity tracking across frames, spatial relation grounding within frame, and object motion understanding. How- ever, they do not explicitly isolate or test foundational spatial skills like basic perspective taking or mental rotation. ViewSSptial--enn object localization from different viewpoints. The core task is determining what is visible from a given perspective, foundational problem in spatial understanding. SpaCE-10 (Gong et al., 2025) defines a taxonomy of atomic spatial skills for question answering, including object recognition, lo- calization, spatial relations, size comparison, and counting. However, its reliance scanned indoor scenes limits controlled testing of each skill in isolation. 3DSRBench (Ma et al., 2025a) centers on spatial reasoning in 3D environments, categorizing tasks into height, location, orientation, and multi-object reasoning. While comprehensive, its scope excludes key aspects of human spatial in- telligence, such as perspective-taking and mental rotation. SPHERE (Zhang et al., 2024) proposes a hierarchical evaluation of vision- language models, progressing from single-skill to multi-skill tasks. Single-skill categories include position, counting, distance, and size. However, SPHERE primarily uses single static image as input, limiting its capacity to evaluate dynamic temporally grounded spatial understanding.
(Li et al., 2025a) targets perspective-taking by evaluating
Several recent efforts draw inspiration from cognitive science: OmniSpatial (Jia et al., 2025a) offers tasks rooted in psychological theory, covering dynamic reasoning. complex spatial logic, spatial interactions, and perspective-taking. However, many of these tasks involve commonsense reasoning about motion and function, which are often entangled with spatial cognition, making it difficult to isolate spatial ability. SPACE (Ramakrishnan al., 2024) categorizes spatial tasks into large-scale and small-scale cognition. Large-scale tasks assess environment-leww spatial understanding, while small-scale tasks involve object-level reasoning. However, the object-level data is limited to 2D synthetic shapes, lacking real-world 3D variability and complexity.
In contrast, our benchmark is cognitively grounded and systematically progresses from small- scale to large-scale spatial reasoning tasks. We start from core perceptual challenges (c.g., object identity, canonical view recognition(single object), mental rotation(single object), dynamic trans- lation/rotation object)) and scale up to relational and perspective-taking tasks in complex multi-object scenes. Our tasks are carefully designed to isolate spatial reasoning by controlling for distractors, motion, reference frame shifts, and multi-image input. The use of both real-world and synthetic data enables robust and interpretable evaluations. Our perspective-taking task serves as the most challenging task, requiring integrating of all subskills, making it holistic test of spatial cognition. Existing benchmarks lack this layered structure and often conflate spatial understanding with unrelated reasoning skills.
F.2 SPATIAL REASONING MODELS
One line of work enhances VLMs' spatial reasoning by leveraging explicit 3D abstractions of scenes. SpatialReasoner (Ma et al., 2025b) introduces a large languaeee model that incorporates 3D representations such as object locations and orientations to enable coherent and reliable reason- ing- Similarly, Abstract Perspective Change (APC) (Lee et al., 2025) constructs perspective-awan scene abstractions using vision foundation models for object detection, segmentation, and orien- tation estimation, leading to significant improvements in perspective reasoning. SSR (Liu et al., 2025) transforms raw depth data into structured, interpretable textual rationales to be integrated in VLMs. Another direction relies on continued pre- vee-rrannngg and reinforcement learning post- post-training. MetaSpatial (Pan & Liu, 2025) adopts reinforcement learning framework to iteratively refine scene layouts with physics- aware constraints, generating coherent and realistic 3D arrangements without supervised annotations. Spatial (Chen et al., 2024) introduces large-scale synthetic pre-training
69

---

Published as a conference paper at ICLR 2026
data to equip models with quantitative 3D spatial reasoning, enabling accurate metric distance timation and downstream improvements in VQA and robotics. Embodied-R (Zhao et al., 2025) combines large-scale VLMs and LMs in RL framework that integrates embodied reasoning from video streams, using both fast and slow iterative processes to tackle diverse indoor and outdoor tasks. vsGRPO-7B (Liao al., 2025) employs R1-Zero-like training with GRPO boost visual- spatial reasoning, outperforming baselines and even surpassing GPT-4o on video-based benchmarks. SpaceR (Ouyang et al., 2025) proposes the SpaceR-15 lk dataset alongside spatially-guided RLVR strategy (SG-RLVR), achieving state-of-the -frt results and surpassing GPT-4o by 11.6% VSI- Bench. Likewise, SVQA-RI (Wang & Ling. 2025) extends RI-style reinforcement learning to spatial VQA through Spatial-GRPO, improving accuracy and interpretability without reliance on supervised fine-tuning. More recent efforts such as SpaceOm and Spacethinker (Chen et al., 2025a) attempt to enhance spatial reasoning through RL-driven linguistic fine-tuning, but their improve- ments exhibit limited generalization Yin al. (2025), leaving fundamental questions about VLMs' spatial cognition unresolved. Ultimately, these works underscore that linguistic reasoning alone is insufficient (Zhang et al., 2025b); humans understand physical space through structured reason- ing that does not always translate into words, highlighting the need for models that reason beyond language.
G THE USE OF LARGE LANGUAGE MODELS (LLMs)
Large language models were used only as general-purpose tools to assist with writing clarity and grammar refinement. All technical contributions, benchmark design, and evaluations were devel- oped entirely by the authors themselves.
70


Reviews:
[{"review_id": "xVaudCDo4A", "reviewer": "Reviewer_t7Ct", "summary": "The paper introduces SpinBench, a cognitively grounded benchmark for evaluating spatial reasoning in VLMs. It decomposes perspective-taking into seven diagnostic categories (e.g., identity matching, dynamic rotation, mental rotation). Evaluating 37 VLMs, it identifies systematic biases (egocentric preference, rotation failure, inconsistency under symmetry) and reports correlations between human response times and model accuracy, arguing that SPINBENCH captures cognitively meaningful difficulty.", "questions": "I am curious about the true value of the dataset, as I strongly suspect that it may merely overfit to its own format.\n\nIf a base model (e.g., Qwen-VL) could be fine-tuned on this dataset and subsequently demonstrate performance gains on other benchmarks (such as VSI-Bench, OmniSpatial and SPACE), I would be much more inclined to recognize the dataset’s contribution.", "limitations": "", "rating": 2, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 1, "strengths": "1. **Testing 37 VLMs**—including proprietary and open-source—offers unusually broad coverage.\n\n2. **Controlled Variations**: The introduction of controlled task variations—particularly the \"with-premise\" vs. \"without-premise\" conditions and the \"allocentric vs. egocentric\" reference frames—is a significant strength. These variations allow for a fine-grained disentanglement of visual grounding failures from spatial reasoning failures.\n\n3. **Human-VLM Correlation**: The finding that human response time correlates strongly with VLM inaccuracy (Fig. 5b) provides strong validation for the benchmark's design, suggesting it measures a fundamental and shared aspect of spatial reasoning difficulty.", "weaknesses": "1. Prior work already targets spatial cognition broadly (SPACE), mental modeling (MindCube), and multi-perspective localization (ViewSpatial). Your unique bits are FoR-controlled augmentations and pairwise consistency, but the manuscript underplays direct, quantitative comparisons and ablations demonstrating why SPINBENCH exposes distinct phenomena beyond these. Provide head-to-head overlap/orthogonality analyses, cross-benchmark correlations, and case studies where SPACE/MindCube/ViewSpatial fail to reveal the same error mode.\n\n2. Restricting all relations to a 2D horizontal plane undermines the generalizability of “spatial reasoning.” Without vertical or containment relations, the benchmark measures planar spatial alignment, not full 3D reasoning.\n\n3. The model set omits several state-of-the-art commercial VLMs (e.g., GPT o3/5, Google Gemini 2.5 pro), which weakens the headline claim about “current VLMs.”  They have more robust and powerful ability.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "9PhIk4j9JT", "reviewer": "Reviewer_kk9w", "summary": "The paper introduces SpinBench, a new diagnostic benchmark for evaluating spatial reasoning in VLMs. The benchmark is cognitively grounded, designed around the core challenge of perspective-taking, which is the ability to reason about how scenes and object relations change under viewpoint transformations. SpinBench decomposes this complex skill into seven fine-grained task categories. These tasks are progressively structured, scaffolding from simpler single-object challenges to more demanding multi-object perspective-taking scenarios. A key contribution is the use of controlled variations to probe model robustness. The authors evaluate 37 state-of-the-art VLMs. The results reveal systematic weaknesses in current models, such as a strong egocentric bias, a poor understanding of rotation, and high levels of logical inconsistency. The study also includes a human baseline (91.2% accuracy) and finds that task difficulty for humans strongly correlates with VLM accuracy, validating the benchmark's focus on fundamental cognitive challenges.", "questions": "1. I am concerned about the low sample size for some of the 51 subtypes. How did you ensure the statistical reliability of the results for these specific subtypes?\n\n2. The scaling analysis in Figure 6 is interesting, particularly the emergent capability in Identity Matching. Did you observe similar emergent patterns in the most complex tasks, like Perspective Taking and Mental Rotation? Or did performance on those tasks scale more smoothly, even if starting from a very low, chance-level baseline?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 4, "contribution": 3, "strengths": "+ While many spatial reasoning benchmarks exist, SpinBench is novel in its cognitively grounded approach, systematically decomposing the complex, high-level skill of perspective taking into a progressive hierarchy of 7 diagnostic sub-skills. The explicit testing of allocentric vs. egocentric frames, visual grounding vs. linguistic inference, and logical consistency provides a multi-dimensional, surgical tool for diagnosing model failures, which is a unique combination not present in prior work.\n\n+ The benchmark is well-motivated by foundational work in cognitive science. The data curation is thoughtful, drawing from four diverse domains (synthetic indoor scenes, real-world objects, cars, and faces).\n\n+ The findings are specific, insightful, and actionable: the discovery of a strong egocentric bias, the fact that models fail even at purely linguistic spatial reasoning, and the high degree of logical inconsistency all point to fundamental gaps in current models. \n\n+ The paper is overall well presented and clearly formulates the motivation.", "weaknesses": "+ The benchmark only contains 2599 samples in total, distributed across 51 distinct task subtypes. This distribution means that some of the fine-grained subtypes have a very small number of samples. The reliability of the performance metrics on such small test sets is questionable. A larger sample size for these specific subtypes would strengthen the conclusions. For example, the Mental Rotation task is presented as a key cognitive inspiration for the benchmark in the introduction. However, this category seems underdeveloped compared to the others. It contains only 78 samples.\n\n+ Despite being evaluated with diverse VLMs, it remains unclear what the upper bound performance is on SpinBench for existing VLMs, that is, SOTA models like GPT-5 or Gemini 2.5 Pro are not tested.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "IZWcKHrrwq", "reviewer": "Reviewer_SReC", "summary": "The authors introduce SpinBench, a spatial reasoning benchmark for VLMs. SpinBench measures the perspective-taking abilities of VLMs through seven tasks, six subtasks representing skills required to accomplish the seventh perspective-taking task. They conducted extensive experiments on their benchmark using 37 VLMs, finding that VLMs exhibit egocentric bias in spatial reasoning and showing that the difficulty of each SpinBench problem correlates with human difficulty in solving the same problem.", "questions": "* What could be the reasoning behind the inconsistent impact of CoT reasoning: Cosmos-Reason1-7B and SpaceOm show improved performance whereas SpaceOm shows degraded performance on perspective-taking.\n\n* In Figure 4 and L317-323, the authors show that consistency is strongly correlated with accuracy. However, this correlation may only imply that strong spatial reasoning leads to strong consistency which is obvious. It would be more interesting if the other direction were true: strong consistency leads to stronger spatial reasoning. Is there evidence on whether stronger consistency leads to stronger spatial reasoning?\n\n* In Figure 3, most models perform much better on the car_view_back task than the other canonical view selection tasks. Is there any possible explanation why this task is particular easy for most VLMs? \n\n* Why do the mental rotation tasks consist of only images rendered from ABO Objects?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "* The paper provides a comprehensive evaluation of 37 VLMs of varying sizes, including two reasoning models.\n\n* The paper shows that VLMs fail in perspective-taking when performing spatial reasoning tasks.\n\n* The consistency metric reveals an interesting spatial reasoning bias in VLMs, and the connection between consistency and spatial reasoning accuracy is interesting.\n\n* A detailed and comprehensive description of the construction of the benchmark is included.", "weaknesses": "* While the observation that VLMs exhibit egocentric bias in spatial reasoning is interesting, the general observation that VLMs exhibit egocentric bias is not a novel observation, as observed by [1,2].\n\n* Related to the previous point, it is unclear what is gained by developing the seven categories used in SpinBench. The authors motivate the seven categories based on six basic spatial reasoning abilities which scaffold towards perspective-taking. However, it is unclear what useful conclusions can be drawn by devising the benchmark in this way. Experiments showing that improving the six simpler diagnostic abilities leads to better perspective-taking ability would emphasize the usefulness and necessity of the benchmark. As it stands, I am not convinced of the practical usefulness of SpinBench.\n\n* The results on CoT reasoning are inconclusive: Cosmos-Reason1-7B significantly benefits from CoT, whereas SpaceOm only enjoys moderate benefits, and SpaceThinker shows degraded performance on perspective-taking without premise.\n\n* While the proposed benchmark shows some interesting observations and correlations, my main concern is that the benchmark does not indicate the direction of causality: the six simpler reasoning tasks may lead to improved perspective-taking abilities or perspective-taking abilities may lead to improved performance on each spatial reasoning task. Fine-tuning the model based on the six simpler tasks and seeing if it leads to improved perspective-taking would show a useful and more interesting observation which can be used to improve existing VLMs.\n\nMinor Weaknesses:\n\n* In Figure 2, the legend “Static Inter-Object” does not match the description “Object-Relation Grounding” in the text.\n\n---\n\n[1] Seeing through their eyes: Evaluating visual perspective taking in vision language models, Góral et al., 2024\n\n[2] 3DSRBench: A Comprehensive 3D Spatial Reasoning Benchmark, Wufei et al., ICCV 2025", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "fZw0ws7WfR", "reviewer": "Reviewer_cvAh", "summary": "This paper proposes SpinBench, a cognitively grounded benchmark designed to evaluate spatial reasoning in vision-language models, focusing on perspective taking and viewpoint transformation. It includes seven structured task types across 2,599 samples and 51 tasks. Evaluating 37 VLMs, the study finds consistent weaknesses such as egocentric bias, poor rotation understanding, and low consistency under symmetry and phrasing changes. Human-model correlations further confirm that SpinBench effectively captures genuine spatial reasoning challenges.", "questions": "I have no more questions to the author. The paper clarifies most benchmark details well.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "1. SpinBench offers a standardized, fine-grained, and vision-centric framework for analyzing the spatial reasoning capabilities of existing methods. Unlike previous benchmarks that primarily focus on semantic recognition, SpinBench introduces tasks such as perspective taking to effectively evaluate whether models can comprehend camera movements and exhibit true spatial modeling ability.\n2. The experiments conducted on 37 VLMs are comprehensive and include an in-depth analysis of their performance.\n3. The paper is well-written and easy-to-follow. The task definitions are reasonably motivated.", "weaknesses": "1. Although SpinBench includes a diverse range of question types, its overall data scale is relatively small compared to existing spatial reasoning datasets such as VSI-Bench and ViewSpatial-Bench.\n2.  SpinBench primarily evaluates VLMs using image inputs. A valuable extension would be to include baselines where VLMs take both images and their corresponding depth maps as input (e.g., generated by DepthAnything). This addition would help determine whether poor model performance stems from insufficient spatial information in the input data or from the model’s limited spatial reasoning ability.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "F9HJx8i5PQ", "reviewer": "Reviewer_fu7D", "summary": "This paper introduces SpinBench, a high-quality diagnostic benchmark for evaluating spatial reasoning in VLMs, with a focus on perspective-taking and mental rotation. Its design is cognitively-grounded and features a progressive structure that isolates specific cognitive abilities. Extensive experiments uncover systematic failures like egocentric bias and inconsistent reasoning, along with detailed discussions from various aspects. The benchmark's difficulty is validated against human studies, showing a strong correlation between VLM accuracy and human response times.", "questions": "1. In your scaling law analysis (Figure 6), you show that Identity Matching exhibits a sharp emergent capability, whereas Object Relation Grounding improves smoothly. This is a fascinating contrast. What do you believe is the underlying reason for this difference? Does it imply that cross-view object abstraction is a fundamentally different and more complex type of computation that only \"activates\" at a certain scale, compared to in-scene relation extraction?\n2. Have you considered evaluating the impact of different image resolutions or aspect ratios on model performance? Given that spatial relations can be sensitive to visual details and object boundaries, it would be interesting to know how robust these models are to such perceptual variations.\n\nMissing . in L288", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Benchmark design. SpinBench is cognitively-grounded and diagnostically powerful with its progressive structure (from simple perception to complex perspective-taking) and controlled variations (e.g., consistency checks, frame-of-reference shifts), allowing for precise identification of model failures.\n2. Comprehensive and insightful discussion. The large-scale evaluation of various VLMs along with the analysis yields significant findings in various aspects, including the exposure of systematic biases (egocentrism), inconsistent reasoning, and distinct scaling patterns (emergent vs. smooth) across different spatial skills.\n3. Strong validation with human studies. The strong correlation between VLM accuracy and human response time provides compelling evidence that the benchmark measures genuine cognitive difficulty, justifying its credibility.", "weaknesses": "1. Lack of architectural analysis. While the evaluation is extensive, the paper misses an opportunity to connect observed performance differences to specific architectural choices in the VLMs (e.g., vision encoders, fusion methods). Such analysis could provide more direct guidance for future model design. (This is minor. )", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)