SpatialViz-Bench: A Cognitively-Grounded Benchmark for Diagnosing Spatial Visualization in MLLMs
Authors: Siting Wang
Minnan Pei
Luoyang Sun
Cheng Deng
Yuchen Li
Kun Shao
Zheng Tian
Haifeng Zhang
Jun Wang
Venue: ICLR 2026 Poster (2026)
Keywords: Multimodal Large Language Models
Spatial Reasoning
Primary area: datasets and benchmarks

Abstract: Humans can imagine and manipulate visual images mentally, a capability known as \textit{spatial visualization}. 
While many multi-modal benchmarks assess reasoning on visible visual information, the ability to infer unseen relationships through spatial visualization remains insufficiently evaluated as a spatial skill. 
This reliance on publicly sourced problems from IQ tests or math competitions risks data contamination and compromises assessment reliability.
To this end, we introduce \textbf{\textit{SpatialViz-Bench}}, a comprehensive multi-modal benchmark for \textit{spatial visualization} with \emph{12} tasks across \emph{4} sub-abilities, comprising \emph{1,180} programmatically generated problems, a scalable framework that allows for expansion to ensure fair and continuously reliable evaluations. 
Our evaluation of \emph{27} Multi-modal Large Language Models (MLLMs) reveals wide performance variations, demonstrates the benchmark's strong discriminative power, and uncovers counter-intuitive findings: Chain-of-Thought (CoT) prompting paradoxically degrades accuracy on open-source models.
Through statistical and qualitative analysis of error types, SpatialViz-Bench demonstrates that state-of-the-art MLLMs exhibit deficiencies in \textit{spatial visualization} tasks, thereby addressing a significant lacuna in the field.

Published as a conference paper at ICLR 2026
SPATIALVIZ-BENCE: A COGNITIVELY-GRUNDDER BENCHMARK FOR DIAGNOSING SPATIAL VISUALIZA- TION IN MLLMS
Siting Wang 12.3 Minnan Pei1.2 Luoyang 123 Cheng Deng*
Yuchen Li
Kun Shao Zheng Tian
Haifeng Zhang
Jun Wang?
'The Key Laboratory of Cognition and Decision Intelligence for Complex Systems
Institute of Automation, Chinese Academy of Sciences
University of Chinese Academy of Sciences
3 *AI Lab, The Yangtze River Delta
*Huawii Noah's Ark, UK
Shanghai Jiao Tong University
Schooo of Creativity and Art, ShanghaiTech University
7 University College London
ABSTRACT
Humans can imagine and manipulate visual images mentally, a capability known spatial visualization. While many multi-modal benchmarks assess reasoning on visible visual information, the ability to infer unseen relationships through spatial visualization remains insufficiently evaluated as spatial skill. This liance on publicly sourced problems from IQ tests or math competitions risks data contamination and compromises assessment reliability. To this end, we introduce a comprehensive multi-modal benchmark for spatial visual- ization with 12 tasks across 4 sub-abilities, comprising 1,180 programmatically generated problems, scalable framework that allows for expansion to ensure fair and continuously reliable evaluations. Our evaluation of 27 Multi-modal Large Language Models (MLLMs) reveals wide performance variations, demonstrates the benchmark's 's strong discriminative power, and uncovers counter-intuitive find- ings: Chain-of-Thought (CoT) prompting paradoxically degrades accuracy on open-source models. Through statistical and qualitative analysis of error types, Spatial Viz-Bech demonstrates that state-of-the-art MLLMs exhibit deficiencies in spatial visualization tasks, thereby addressing a significant lacuna in the field. The benchmark data and code are publicly available.
INTRODUCTION
Large Language Models (LLMs) have demonstrated strong capabilities in complex reasoning, and the integration of Vision Transformers (ViTs) has given them "eyes", extending these abilities into the multimodal domain. While many tasks focus on visible information, real-world challenges in fields like architectural design and medical-image-assisee surgery often demand the ability to mentally construct and manipulate unseen structures, a capability in which existing MILMs still struggle. To bridge this gap, spatial visualization must be abstracted and assessed through targeted evaluations that isolate it from confounding factors, like a well-designee physics exam tests fundamental principles. However, current evaluations rely heavily web-sourced problems, risking data leakage and inconsistent formulations, underscoring the need for a procedurally generated, standardized benchmark to ensure fair and reliable assessment.
This cognitive faculty for mental manipulation is known as spatial visualization, which was first identified by Thurstone in his work primary mental abilities (Thurstone 1938). Successfully
'Correspondence to Cheng Deng Halfeng Zhang Jun Wang
Data: pat / h/wgggiing ing Face. co/ dat aset e M- Tean /Spat 1a 1011----00 sual 12at! or Renchmark
Code: LLLppt/7gil com/ www.wwagsssso..... U1 Lal1 bench

---

Published as a conference paper at ICLR 2026
Human
(b)
Tasks in SpatialViz-Bench
Proviously Evaluated
After folding the not into cube. what the relationship between the purple
Preciounly Unavalttted
Spatial Perception
Spetial Momorization
Spatial Visualization
bille Houua
supple face
Discem impliit inforation tnrough roascring
memorry MLLM
The purple onange hoow are sopaicled green
Montal Rotation
Mental Folding
Visual Panetration
Mental Animation
They opposita faces the
Gemind
3D-Gammeryy
High
MARVEL
G0-Geomrty
Nodiun
ENIMA
2D Trana/ommatinn
Desoritive Geomctyy
Trans
Spatal Mecharical
Vista
Puzzies
Speaal Honceng
Figure 1: Overview of SpatialViz--eent (a) presents a representative task instance. (b) unfolds the reasoning behind (a): perceiving visible cues to infer unseen relationships via iterative visualization and memorization. The table highlights a systematic gap: unlike perception, spatial visualization remains a largely unassessed blind spot in prior benchmarks (indicated by lighter colors). (c) displays zero-shot accuracy revealing significant gaps against human performance.
performing spatial visualization tasks relies on two other fundamental spatial abilities: Spatial perception (Thurstone [950), which aims perceive external spatial information and relationships, and spatial memorization (Della Sala et al. 1999), which requires temporarily storing transformation information mentally without accessing physical objects.
Despite their importance as dedicated spatial-reasoning challenges, spatial visualization tasks are often buried under broader categories like mathematical logical reasoning, appearing as multimodal puzzles or 3D geometry problems. This categorization obscures the evaluation of spatial visualization as a distinct capability and focuses on "solving" a problem rather than driving research toward core spatial abilities. Moreover, most examples are drawn from publicly available sources, online IQ tests, administrative exams, and math contests, which risks overlap between training and evaluation data and undermines reliability. The scarcity of items per subskill also magnifies random error, while heterogeneous formats make hard to distinguish true reasoning failures from misinterpretation. Consequently, with potential pretraining exposure, performance remains poor. State-of the-art systems score just 27.64 on 3D Geometry in MM-IQ (Cai et al. 2025) and 26.00 on Descriptive Geometry in Vision (Wang et al., 2024). Beyond task difricutty, the modern paradigm of pretraining on vast, scraped internet data fundamentally challenges evaluation validity (Wu et al.) 2025). a problem exacerbated by proprietary datasets that make auditing for contamination impossible. This fundamental challenge calls for a new generation of benchmarks with dynamically updatable test banks ensure persistent evaluation integrity (Ni al. 2025).
To address these shortcomings, we introduce Spatial Bench, a novel benchmark designed to formally evaluate the spatial visualization capabilities of MLLMs, comprising a framework of key sub-abilities(mmntaa rotation, mental folding, visual penetration, and mental animation) from which 12 targeted tasks are designed for comprehensive assessment. Inspired by benchmarks like CLEVR (Johnson et al., 2017), diagnostic benchmark for spatial perception, which uses Blender (Blender Online Community 2016) for data generation, we developed a pipeline that integrates Python with FreeCAD (FreeCAD Team 2025) for the programmatic generation of novel test cases, enabling scalable task expansion while effectively preventing data contamination by dynamically updating the test bank through randomized generation. We employ standardized question templates to minimize errors arising from varied instructions. Furthermore, programmatic generation allows control task difficulty precisely and to create distractors with explanations systematically.
2

---

Published as a conference paper at ICLR 2026
Mental Rotation
2D Rotation
3D Rotation
Three-Vine Vieee Projection
Orginel
laceere
Froet
To
Tue
Orggna
Orgea
Mental Folding
Unfolding
Paper Folding
Operation
Operatend
Visual Penetration
Cross Section
Cube Counting
Cube Assembly
Rold
Mental Animation
Moving
System
Moving
Fiust Stats
Figure 2: Overview of Tasks in Spatial Bez-Bech.. SpatialViz-Bende evaluates 4 spatial sub-abilities, mental rotation, mental folding, visual penetration, and mental animation, via tasks each (12 tasks total). Each task has 2-3 difficulty levels of 40-50 cases, yielding 1,180 question-answw pairs.
Models with strong spatial visualization skills can serve as an efficient internal world model, providing foundational capability for various downstream applications. This allows model to fast, lightweight internal "what-if" scenarios (e.g.. "what happens rotate this object?", "if this gear turns clockwise, which way will the connected gear move?") to predict the outcome of actions. This far more efficient than the current alternative invoking large, diffusion- based video generation models to explicitly render future state.
The main contributions our work can be listed as follows:
We introduce PpatialViz-Bench. the first benchmark formally establish comprehensive and challenging evaluation framework for spatial visualization, a core yet long-overlooked cognitive ability. grounded in cognitive science and assesses 4 key sub-abilities through 12 distinct tasks, resulting in total of 180 examples across
difficulty levels.
We establish scalable and trustworthy programmatic generation methodology for // of our tasks. This approach not only enables continuous expansion of tasks but also sets new standard for fair evaluation by preventing data contamination through dynamic updates to the test bank.
We systematically evaluate 27 MILMs, with top scores from Gemini-2.5-pp (44.66%) and ol (41.36%). These results demonstrate the benchmark's challenge and high discriminative power, revealing a significant capability gap to human performance.
We conduct a diagnostic analysis revealing that model failures stem primarily from fundamental Perceptual and Spatial Transformation deficits, rather than from high-level reasoning, which offers a clear direction for future improvements.
3

---

Published as a conference paper at ICLR 2026
Input
Algorithm Pool
Question: Which of the aptions the right cannot be obtained by rotating the original cube stack?
?
Output Data
PLA
Uendker The peape doeas be eetian
+
Task Name
Input Image
enth boubrped by treernng mi ergeel daub. Faace
Reference Image
Task
Input Text
C.C. Mo shore Repkention
Parameters
Positive Samples
Ground Truth
0B Question
Rotatett
Answer
ees aile vea cogiinal ciil now was
Template
Negalive Samples
Explanation
dackaoll unar oligiaal stail
Removall
cdt
Figure 3: The programmatic generation pipeline of a data instance. We constructed the dataset using an programmatic generation system that integrates Python with FreeCAD, enabling precise control of difficulty, systematic generation of distractor options, and programmatic recording of explanations for incorrect choices.
2 RELATED WORKS
Current Landscape in Spatial Reasoning Benchmarks The evaluation of spatial reasoning in MLLMs has largely concentrated on abilities tied to directly observable information. Benchmarks for spatial perception, the ability identify and interpret spatial relationships from visual input, are the most established. Existing benchmarks like What's (Kamath et al.] [203). Blink (Fu jet al. 2024), and SpatialRGPT-bench (Cheng et al.) 2024) assess how models understand object- or camera-centric relationships, relative distances, sizes, and positions. Progress has also been made in evaluating spatial memorization. with video- based benchmarks like VCBench (Li al.) 2024) and VSI-bench (Yang et al. 2024b) challenging models to track objects in dynamic scenes. These efforts have built a foundation for assessing a type of spatial reasoning that relies on explicit visual information and applies a model's world knowledge to interpret what is perceived. However, they largely neglect the advanced capability of spatial visualization, the ability infer implicit visual-spatial information through transformation of structures derived from visible inputs, leaving significant gap in the current evaluation landscape.
Evaluation of Spatial Visualization Evaluating spatial visualization presents challenges regarding data contamination, obscured categorization, and narrow task coverage. A primary concern is contam- ination from public sources (Xu et al.) 2025b), risk programmatic generation seeks to mitigate, as seen in the LEGGO-Puzzess benchmark (Tang et al.) 2025). Furthermore, spatial visualization is often subject to obscured categorization, subsumed under broader domains like mathematical or logical reasoning in general benchmarks (e.g.. MM-IQ (Cai et al., 2025). Math (Wang et al. 2024)), which diverts focus from
core ability. Concurrently, specialized datasets exhibit narrow task coverage, focusing on single sub-skills like mental rotation (SPARE3D (Han et al. 2020). CLEVR- MRT (Beckham et al., 2023)) or specific tasks like paper folding (SRBench (Stogiannidis al.. 2025 > Yin et al. (2025) also assess mental modeling, utilizing distinct organizational frameworks, such as relative spatial perspectives.
3 SPATIALVII-BENCH
3.1 SPATIAL VISUALIZATION
Spatial visualization is core component of human cognitive systems and critical capability for deployment downstream applications. Research into this ability began with Thurstone (Thurstone) 1938). who defined it as performing mental operations visual images and identified it as one of the key spatial factors: spatial perception, spatial visualization, and mental rotation (Thurstone) 1950).
Building on this foundation, establish cognitive framework that decomposes spatial visualization tasks into two phases: observing visible information and discerning implicit information. The former requires basic spatial perception, while the latter demands alternation between spatial visualization (mentally manipulating images to find implicit information) and spatial memorization (temporarily storing visuospatial information) (Della Sala et al. 1999).
Our benchmark's 's design is guided by 4 core sub-abilities: 1) mental rotation: Mentally representing and rotating objects while maintaining their features; 2) mental folding: Mentally folding 2D patterns
4

---

Published as a conference paper at ICLR 2026
Table 1: A Compact Summary of Spatial Reasoning Tasks.
Category
Tisk Nume
Core Objective
Negative Sumples
Diflcully Scaling
20 Rotziion
Edcatiiyy everas 2D retatiion
internal pateen rotation
Minocing:
Nas-ccninlly symmnctric gatterss
Mental Rotation
5D Rotziion
Idcasify cverast 3D retsticnn
cube reneval
View minaing
Largr avecnisiiss
Three View Projectoo
Select left view from projections
Wrong view substintion: view fipping:
Real enginectins Dars (DeepCAD Wa et 2001
line deletion
Puper Foiding
Predict undolded hole pattern
Hole mirmding, additioo, deletion, or relocation
More folds:
larger grd::
more holes
Mental
Fobding
Cube Undolding
Select correct 20 net from view
Swapping face colone retating imeemal putterns
A Agmmettiollot patters on faces
Cube Resoustruction
Select view from net: opposite face
Minocing the caroec siew
Follows Clube Unfobding
Guss-Sectins
Ideatify croee scction solid
Allorel genmataic propoctions
3-sclid composites. obliqqe slicing
Visual Pemetration
Cabi Counting
cabe count from views
Options miniinsss bounds
larger ussemblles
Cube Assembly
complementrr part split stack
Add cubes corret part
Larger 3-pert splins
Moxing
Predict final
Incorrect endpoint from same stur
Multiple anowe:
muvement sequener
Mental Animation
Moving
Predict findl gravity
Insvenen finsil
Highe complesity longer sequessess
Mochanical System
Understaral motion propagation
Insvenec mitiion oslovancs
Mire system modaless
into 3D objects or vice versa (Glass al. 2013); structure of an object from its external features (Titus & Horsman 2009); 4) mental animation: Mentally visualizing the motion of components within a system (Sims & Hegarty 1997).
3) visual penetration: Imagining the internal
3.2 OVERVIEW OF SPATIALVIZ-
Stemming from availability driven collection, current web-sourced benchmarks containing spatial visualization tasks lack standardization and a cognitive theory basis, resulting in inconsistent tasks and incomplete coverage. We counter this with systematic, ability-centric methodology: we use hierarchical framework based on cogmitive principles to guide new task design and employ a unified input format with standardized templates to reduce confounds and enable fine-grained error analysis.
Based on our cognitive framework, we propose Spatial Vizeech to comprehensively evaluate the spatial visualization capabilities of MILMs. It is organized around 4 core sub-abilities mental rotation, mental folding, visual penetration, and mental animation wwith 3 assessment tasks designed for each, totaling 12 tasks. Each task includes 2 to difficulty levels, with each level containing 40 or 50 test cases, comprising 1, 180 ceeestionnansweer pairs in total, mostly with image-based options to focus on visual reasoning. Further details on the dataset characteristies are provided Appendix C
3.3 CONSTRUCTION OF MITTIALVI--BENEE
SpatialViz-Bere Bench is constructed through a combination of programmatic generation and manual design. For 11 of the tasks, used programmatic system integrating Python with FreeCAD (FreeCAI) Team 2025) (see Figure 3). By explicitly utilizing cognitive load parameters rather than heuristics, such as aligning rotational complexity (global object vs. internal pattern rotation) with mental transformation steps (Shepard & Metzler 1971). our programmatic framework ensures precise difficulty control, while employing controlled randomness enhance diversity and generate distractor options with explanations for deep diagnostics. Notably, the Three-Viee Projection task (Level I) uses fixed DeepCAD (Wu et al., 202T) models. but we programmatically generate novel distractors (e.g., random line deletion, view flipping) to ensure novelty. Conversely, the Mechanical System task generation was technically
(1/12) was manually designed, as programmatic, physically-consiste difficult. Using representative public simulations a reference, experts designed all questions from scratch. These visual-based questions probe dynamic motion propagation (c.g., rotational dynamics from single image), testing visual simulation rather than caption recall theoretical derivation.
This combined methodology, leveraging both programmatic generation and the vast pool of public simulations for expert-driven driven question design, supports dynamically updated test bank that proac- tively mitigates data contamination. A task summary is presented in Table I, with detailed generation processes, algorithmic pseudocode, and illustrative examples deferred to Appendix B. B.4 and
5

---

Published as a conference paper at ICLR 2026
Table Comparison of open-source model performances. Tasks: 2D Rotation (2DR). 3D Rotation (3DR), Three- View Projection (3VP), Paper Folding (PF), Cube Unfolding (CU), Cube Reconstruction
(CR), Cross-Section (CS), Cube Counting (CC), Cube Assembly (CA), Arrow Moving (AM), Block
Moving (BM), Mechanical System (MS). The first and second highest accuracy of MLLMs are marked in red and blue, with open- open-source and closed-source models marked separately.
Model
Overall
Mestal
Roceioo
Mental Fulding
Viuaal Perstration
Mental Anlisation
wib Eal
wrrrr
2011
2016
Ang
CU
Ang
CA
Ang
AM
MS
Arg
Hareas
$2:46
10.00
19.16
87.50
83.56
19.75
75.00
72,92
90.56
72.00
70.83
$2.50
75.22
90.00
$7.50
$7.50
88.33
Raskon
25:06
23.75
27.50
31.00
27.00
10.17
20,00
25.83
21.07
30.00
25.00
30.00
28.12
26.75
16.25
25.00
23.39
Qvend 728 (ostaal
15.86
15.00
35.00
15000
21,7
1657
20.02
20.00
45,00
25.00
30,00
24.39
Open Soarce MILLSs
39.32
24.15
22.50
22.50
22.00
22.31
20.00
27.50
20.00
22.50
24.17
26.67
32.50
27.19
21.25
25.00
21.50
24.58
3rwmVV13-33 3-28
76.19
16.25
35.35
11.00
25.87
24.44
20:00
10.85
10.00
20.50
18.75
10.00
29.39
11:35
12.00
22.30
21.00
22.09
21.87
20.83
19.07
21.56
20333
22.30
19.75
18:75
21.25
25.00
21.87
Quees2 5 VL dutraci
30.17
26.30
20.00
15.75
21,00
20.00
25.00
25.83
21,67
24.17
25.85
25.33
30,00
25.04
35.00
30.00
42,90
33.83
Queen 5-VI Titteee
30.76
37.81
25.00
16:75
75.00
7335
14.17
31.67
30.00
75.61
16:67
36.67
75.75
77.99
32.50
25.75
51.75
$7.91
S-Cmmi--1 SAIL-VL-11--8 1..0VL33.8
29.15
25.00
15.75
21.25
25.00
21.12
26.35
25.00
15.33
23.00
21.67
19.37
23.75
21:25
25.00
35.00
45,00
25.00
30:55
20.00
18.75
23.33
23.56
38.75
51.25
Kimi- VlL-A-l--tmmmmiM
12.37
23.90
16.25
34.00
15.00
25:15
25.85
20.00
36:67
24.17
21:67
5.00
24:25
17.19
15:00
31.50
27:92
Kimi VL- chiniina (8a)
16.14
13.75
20.00
25.00
20.00
23.33
24.11
25,87
04.12
25.00)
36.57
25.00
20.58
30.00
45.75
41.50
Decsieck (12o
25.17
25.17
31.25
16:25
26.00
24.62
22.50
25,00
26.67
04.72
9:17
35.00
35.00
25.31
26.25
23.75
25.75
26.25
Desssecl- N12227B)
30088
25.31
25.00
33.75
30.00
20.62
31,67
25.00
21.50
26.59
19.33
30.17
28.75
28.75
26:25
30.00
31.25
20.17
Qwen? ltsttet
13.03
31.25
31.00
25.00
36.67
43.75
28.75
70
29:75
30.34
22.50
33.75
29.00
25.46
20.83
20.17
30.83
26.94
21.67
32.50
41.25
30.63
25.00
30.00
55.25
37.16
V- 118-lastreet ponice
35,00
$5.31
25.75
31.25
25.00
20.25
22:50
20.00
30.00
24.17
30.00
41.67
48.75
39.66
27.50
40.00
63.75
43.75
21.25
30.00
31.00
19.17
22.50
25.00
43.75
20.58
TrunVIl3-00
12:28
79:31
25.00
25.00
14.00
19:17
25.00
23.90
70.85
00.00
93.25
11.35
41:71
Lianta Mavenick 178-1248-1nammat
31.75
20.0
40.30
4000
33.35
20:31
23.07
25.00
19.17
35.00
41.30
32.19
35.00
40.50
41,30
31.17
LEamr 4 Scou: 17B. IIE fiotrdt
34.24
32.50
35.00
43,00
37.51
16.07
32.50
36.67
28.0
17.50
37.50
53.75
34.00
28.75
40.00
50.00
17.58
Closed Source MLL
4P
30.76
31.00
32.50
27.50
33.00
31.15
20.17
15.83
30.00
25.00
19.17
40.53
40.00
32.50
22.50
32.50
60.00
34,33
Charde S-ceeett
26.96
32.54
51.25
25.00
45.00
$4.02
20135
22.50
31.07
25.00
22.50
35.85
45.25
33.64
37.50
31.25
$1.50
411-2
Clode:37
33.90
30.50
36.35
54 08
26.67
24.72
24.17
43.75
$1.50
65
26.75
43.75
Gerin-7 S-fach
16.35
10.30
15.00
15.77
26.67
30.30
40:81
9110
16.35
25:75
37.81
48.75
91.81
Gemm-22-pp S-poo
64.66
32.30
43.35
31.57
16.00
53.35
55.00
35.25
42.19
15.10
38.75
62
Doebas S-iiinngoo
37.54
33.31
1.50
38.00
45.00
30.58
23.33
29.17
23.06
30.00
55.85
35.00
22.50
47.50
Qwenn VLiraa
36.16
33.03
23.75
26.25
33.00
20.06
24.17
17.50
31.67
24.44
26.67
47.50
42.50
38.44
36.25
36.25
55.00
31.17
EVALUATION
4.1 EVALUATION SETUP
Models We conducted comprehensive experiments on a diverse range of MLLMs, including 8 closed-source and 19 open-source models. For closed-source MLLMs, we evaluated models from
5 major providers, including OpenAl series (GPT-40 (Hurst et al. 2024). ol (Jaech et al.) 2024)),
Gemini series (Gemini-2.5-flashh Gemini-2.5-pro (Deepmind 2025)), Claude series (Claude-3.5-
sonnet (Anthropic 2024).
(Anthropic ), Qwen-VL-may (Bai et al. 2023).
and Doubao- 1.s---iii-vvi-e-ee pro (ByteDance 2025). For open-source MLLMs, we assessed Qwen2.5-VL
series (Bai ct al. 2025), QvQ (Qwen Team 2024). Qwen-Omni (Xu et al. 2025a), Intern VL-3
series (Zhu et al. 2025). Deepseek- VL.2 series (Wu al. 2024), SAIL-VL series (Dong et al. 2025)
Kimi-VL-AA3 series (Team et al., 2025) and LLama-4 series (Meta Al 2025). For text-only LLM,
we used 10ven2..-722BInnttruu
(Yang et al. 2024a).
Setting For a rigorous evaluation, all experiments were performed in a zero-shot setting (Hao et al. 2025] Wang et al., 2024), comparing model performance under two prompting schemes: (1) CoT, where prompts were designed to encourage models to output their reasoning process before the final answer, and (2) Direct Answering (non-CoT), where prompts solicited the answer directly (see Appendix E.2. This methodology enabled us to not only assess the accuracy of responses but also gain deeper insights into the models' underlying reasoning mechanisms across our benchmark tasks.
Metric Design To evaluate models handling multimodal inputs and generating textual outputs, with most options presented as images, we formatted all tasks as Multiple-Chi Chhicee Answer (MCA) with one correct answer. Option and reference images were integrated into unified visual input. For

---

Published as a conference paper at ICLR 2026
60 Oveeall Model Perfocmance 100%
Do Moddet Seealibily Plahributiin
Took Seastlivlly vy, Average Accunecy
65%.05
Meaa
Cos Amouditail
Nareebi ITB
Percestape (%)
Coeat of Siguifcamn Changes Meecoss 14ftically Lenok)
Cross Local Ntorage Accuracy (i)
Figure 4: Statistical Analysis of Model Performance, Difficulty Sensitivity, and Task Discrim- inability. (a) presents the overall model performance with 95% Wilson confidence intervals. (b) shows the distribution of model sensitivity to difficulty gradients. (c) provides a task-centered analysis of difficulty sensitivity, revealing how difficulty levels differentiate model capabilities across tasks.
questions where answers could be expressed as simple text, we also provided a text-based answer format (detailed in Appendix (.4). Model performance was assessed using accuracy, based on the match between predicted and ground-truth answers. This standardized approach ensures consistent evaluation across tasks and enables fair comparison of multimodal understanding across models. A comparative analysis of performance on both formats is provided in Appendix F.3
Human Baseline Our human baseline established with 8 graduate students from mechanical engineering and computer science, selected for their strong spatial reasoning backgrounds. Each participant solved 72- 72-problem subset under strict conditions designed to be analogous MILM evaluation: no external aids (e.g., scratch paper) were allowed, but time was unlimited. This protocol isolates intrinsic spatial visualization abilities for a fair comparison.
4.2 EVALUATION RESULTS
This section first establishes the performance gaps between different models and then, through a CoT ablation study, investigates the impact of explicit reasoning to identify the core abilities required for advanced spatial reasoning.
4.2.1 MAIN RESULTS
Tasks in Spatial Bench are Vision-Dependent and Reasoning-Intensive As the textual input alone insufficient, visual input is essential for problem-solving. making the benchmark highly vision-dependenn. We empirically validated this claim by evaluating a powerful text-only LLM 1a1.2.55-72--Instruct As detailed in Table 2] the text-only model achieved a total accuracy of 25.86%, which is negligibly different from the random-chance baseline (25.08%). quantitatively proving that the visual modality is indispensable. Most options are image-based, requiring precise visual analysis rather than simple matching. thereby increasing reasoning complexity. For both humans and MLLMs, these tasks demand multi-step spatial transformations and inferences that mirror complex CoT processes.
Performance Gaps Reveal a Statistically Validated Hierarchy of MLLMS All evaluated models performed well below the human baseline (82.46%), underscoring the benchmark's 's difficulty. Our analysis, now supported by 95% Wilson confidence intervals (CIs) (as shown in ??), confirms this performance hierarchy statistically robust. The top performer, Gemini-2.5-pro (44.66%, Cl: [41.85%, 47.51 %)),, demonstrates capabilities irrefutably above the random baseline (25.08%, CI: [22.69%, 27.64%]), () as their CIs do not overlap. More importantly, this analysis provides solid statistical backing for the critical capability gap between proprietary and open-source models. The CI for Gemini-2.5-pp (34.24%, Cl: [31.58%, 36.99%].. confirming this 100% performance delta is significant. Conversely, the Cls help group statistically similar models into "performance tiers"; for example, the Cls for 41aMA-4-Scom and V-VI--7B--Inntrru (35.00%, Cl: [30.67%, 36.04%]) highly overlap,
shows no overlap with that of the top open-source model, L.LMMA--SScom
7

---

Published as a conference paper at ICLR 2026
Table 3: Robustness analysis of CoT performance. (a) Performance remains stable across different CoT prompt templates. (b) The significant performance gap between CoT and non-CoT persists across extraction rules, ruling out parsing failures as the cause of performance drops.
(a) Sensitivity to Prompt Variations (Aceuracy %)
(b) Sensitivity to Extraction Rules (Ace. Drop%)
Model
CoT'
CoT
A
Model
Rule !
Rule B
Dwen2.5-VL-725
33.31
31.19
-2.12
SAIL- VL-1.--BBB
-8.22
-7.29
+0.93
GPT-4o
31.10
30.81
-0.29
Prepseek-VL2-33
-5.18
-5.01
+0.17
Claude-3.5-sonnet
32.54
28.31
-4.23
Kimi-VL-16B
-8.47
-9.66
-1.19
making their performance statistically indistinguishable. This statistically validated discriminative power highlights significant room for improvement.
Core 3D Visualization Tasks Reveal Common Model Failures Models with higher overall accuracy generally perform well across individual tasks. Most models show near-random accuracy on core 3D tasks like 3D Rotation, Cube Unfolding & Reconstruction, indicating common and severe perceptual and visualization limitations in 3D space. Both proprietary models perform well on the Arrow Moving task, with Gemini-22 2.55ppo even surpassing human performance, while most of open-source models perform at near-random levels. This suggests that, despite its relatively low visual complexity, the task requires advanced reasoning-suee as understanding object-centered motion-which open-source models still lack. In most cases, model performance matched our expected difficulty levels, though some discrepancies with human perception offer valuable insights for refining task design and guiding future research. Additional evaluation results and task-specific analysis are provided in Appendix F.
Difficulty Collapse Only Visible in Top-Tier Models We first validated our intended difficulty gradient (DG) against human performance and hypothesized models would show similar scaling. However, data reveals a widespread "performance floor" at LO; 10 models showed < significant DG, while the top-performing Gemini-2.5-pp was most sensitive (7 DGs) (Figure 4J)). From task- centric perspective (Figure 4|c), three tasks induced a significant DG in or more models. Notably, the stark DG contrast between CubeReconstructinn
(12 models) and its symmetric counterpart CubeUnfolding (1 model) suggests models better reason about symmetry from unfolded views. Conversely, BlockMoving (0 DGs) proved challenging at both levels, rendering any drop statistically invisible. Critically, on 3DRotation, the only two models exhibiting a DG were the top-two performers (Gemini-2.5-pa) ol). This confirms core claim: only top-tier models achieved non-random LO accuracy, and thus were the only ones capable of showing statistically significant collapse at LI.
4.2.2 COT PROMPTING ABLATION STUDY
For the non-CoT evaluation, excluded models designed for extended reasoning (e.g., o1, Gemini- 2.5 series) or those unable to adhere to the format (e.g.. Intern VL3-2B), proceeding only with models that could reliably provide a single-letter answer (detailed in Appendix E.2).
Our ablation study on Chain-of-Thought (CoT) prompting confirms a "CoT paradox," a phenomenon also noted by EMMA (Hao et al. 2025): CoT benefits high-performing closed-source MLLMs but often paradoxically degrades their open-source counterparts. We provide new statistical validation for this. As shown in Table in Appendix F2 the impact significantly positive for claude-3.5-somme but significantly negative for several leading open-souree models.
Crucially, our analysis pinpoints where this degradation occurs. The performance loss for these open-source models is not uniform but is highly concentrated in "pure-visual" spatial tasks (e.g.. 3 VVieeProeection, 3DRotation). This strongly supports our hypothesis: for these models, the mandate to generate explanatory text (CoT) interferes with their native visual-spatial judgment, acting as a cognitive distraction rather than an aid. In contrast, top-tics tier closed-source models demonstrate superior resistance to this interference, likely due to specialized RL-based reasoning training, allowing them to leverage CoT effectively.
4.2.3 ROBUSTNESS TO PROMPTING AND EXTRACTION STRATEGIES
To rule out the possibility that the observed CoT degradation is artifact of specific prompt engineering or parsing failures, we conducted a sensitivity analysis in Table 3) First, we tested models
with alternative CoT prompt template (detailed in Appendix E.2 As shown in Table 3(a), the
8

---

agreement. Disagreements were resolved Published as a conference paper at ICLR 2026
Erer Types
Paceptal
Teae
THE
Teat I2
Spenil Mhrnorsston
Inth Follwing
Calculi Reaseriy
23.4
SUN
38%
17
Figure 5: Comparison of error type distributions, with chart (a) showing the overall breakdown
and charts (b-e) detailing results for specific MLLMs: (b) Gemini-2.5, (c) ol, (d) Qwen2.5-VL-72BB and (e) Qwen2.55VVL-775 Errors are classified into six categories: Perceptual, Spatial Transformation, Methodological, Instruction Following, Spatial Memorization, and Calculation & Reasoning.
performance trends remained consistent, with Qwen2.5-VI--77. still underperforming compared to its non-CoT baseline (35.00%). Second, we compared two distinct answer extraction rules (truncated letter matching as Rule A vs. full-format regex matching as Rule B, detailed in Appendix [..]. Table 3)b) reveals that the discrepancy between rules is negligible (< 1.2%), confirming that the negative impact CoT (ranging from -5% to -9%) is genuine reasoning failure, not parsing error.
4.3 ERROR ANALYSIS
This section first presents a statistical error analysis across several representative models to identify common failure modes, followed by detailed case study of Gemini-2.5-pro illustrate its specific reasoning processes.
4.3.1 STATISTICAL ERROR ANALYSIS
This evaluation conducted primarily through manual review human annotators), utilizing Gemini-2.55pp pro as an assistive tool based on 6 manually defined error categories, including perceptual, spatial transformation, spatial memorization, instruction following, methodological, and calculation & reasoning етror (detailed in Appendix [.662)). 'To account for diversity in developers, model sizes, and open/closed-soure pro and ol (the top-performing model), and its smaller counterpart, Qwen2.5-VL-7B, To ensure the reliability our error taxonomy, two annotators independently annotated subset of 100 errors. We calculated the Cohen's Kappa coefficient (x = 0.85), indicating strong through discussion with third expert.
paradigms, we selected models for deeper analysis: Gemini-2.5- source models), Qwen2.5-VIL-72B (a leading open-source
Perceptual and Spatial Transformation Errors Dominate Failures The dominance of Perceptual and Spatial Transformation errors, which collectively account for nearly 60% of all failures, quantita- tively supports our central hypothesis that the primary MILM bottleneck stems from fundamental failures in visual perception and transformation, not from high-level reasoning deficits. contrast, the low frequency of Calculation & Reasoning and Instruction Following errors confirms the bench- mark's effective isolation of spatial deficits. Methodological errors, the third-largest category at over 23%, indicate that models often adopt suboptimal problem-solving strategies. This highlights a clear direction for future improvements: enhancing spatial visualization capabilities by augmenting the training data with more correct solutions.
Model Scaling Fails to Resolve Core Spatial Deficits model's absolute error count correlates with its performance rank: Gemini-2.5-pro had the fewest errors (204), followed by ol (236), Qwen2.5- VL-72B (272), and Qwen2.5-VL-7B (328). Although top models show similar error profiles, Gemini- 2.5-pro's lower rate of Methodological errors partly explains its superior performance. The limits of model scaling become evident when comparing tenfold parameter increase, their eгто patterns remained strikingly similar, with Perceptual and Transformation errors still dominant. While the 72B model nearly eliminated Spatial Memorization and Calculation errors, it made only limited gains these most critical error types. This reveals
1ien2.55-VLT7 and Qwen2.5-VL-722. Despite a

---

Published as a conference paper at ICLR 2026
Meanil Filling Cole Recosrrectin-lee
Mesnil Blakk Mosing
Mental Animation Mechasiral
Visaal Prostration Avorilly L
of
Guniie se an le necen aloem prict RE 101 Chilenr 0.3 0.6
cenetiee daoc witeen riga
File Be poller puom poa caaaeiii dia-.
dkuui cotlnce
mnje co gre wile an
and Kamaa ai cove Thrapaaee
saed"
demii aaic Bo mik caree
peoee de oow
à E.C.
Tiaakig Fedtirt
th feem snd lele palre
Thakig
Tieekng
1111--0 13 1.
corae pare dinai at
aa pudtac fett Coloag Aitappaaa
Dag Coralax
phic god
Theking
all Roce Cole an 1i' sest of
Soart D.
and Ve did ane nesile up-l and
(ffe entess
kailt blisin toe
Slace
Theedaie oe anruciioo e plown and
bdiating Fan koluma cahos
right beltee puokios.
peaasee dal casiii bervinn Goo afher filling
-Mining Medd and th paddiios of blodo winee
hereer taaddiidtyyyy thallag vibier tieential derivatie
That "eoorne cilciatta seched
Figure 6: Case study of 2:5pro--..-ppro's reasoning in different tasks.
crucial insight: scaling alone does not resolve fundamental spatial reasoning deficits. True progress will likely require innovations in training paradigms, such as (DeepSeek-Al et al. 2025), rather than merely increasing model size.
4.3.2 ANALYSIS OF TEST CASES
To complement the statistical analysis, we conducted qualitative case study of Gemini-2.5 reasoning processes. The model exhibited strong reasoning, following logically coherent and complete processes, validating the effectiveness of our evaluation results. This analysis reveals a significant gap between its abstract reasoning capabilities and its visuospatial processing abilities, reinforeing that the primary bottleneck is not high-level logic but fundamental perception and visualization.
Deficiencies Found in Both Perception and Visualization qualitative case study of Gemini- 2.5-pro's reasoning reveals errors occur at two distinct stages: perceiving visible information and reasoning about unseen spatial relationships. In processing visible information, the model exhibited deficiencies in 2D tasks like color recognition and complex pattern identification (Figure 6).) These perceptual failures were more pronounced in 3D space, where it struggled to accurately identify the quantity, position, and spatial relationships of stacked cubes (Figure 6lb). This difficulty is quantified by a stark performance drop, with accuracy plummeting from 95% on the 2D Arrow Moving task to just 35% on analogous 3D tasks. The model's primary struggles, however, emerged when reasoning about unseen information. It consistently failed tasks requiring mental manipulation. such as accurately inferring the structure of cube nets or the symmetrical relationships between faces after folding.
Pre-training Biases Drive Non-Simulative Problem Solving The case study also uncovered strong pre-training biases that shape the model's 's -ooliin approach. For Mechanical System tasks, which were designed to be solvable via pure spatial visualization, Gemini-2.55pm often defaulted to applying theoretical physics formulas instead of mentally simulating the motion (Figure 6/c). This behavior diverges sharply from human strategies and reveals a critical misalignment between the model's problem-sole soollmm-oolvigggoeo approach and genuine spatial intelligence, suggesting its internal world model is more analytical than simulative. These qualitative examples directly illustrate the types of Methodological failures identified in statistical analysis, forming a cohesive picture of current MLLM limitations.
5 CONCLUSION
We introduce SpatialViz-Benchh cognitie--cienne-inssiee for testing spatial visualization in MLLMs, designed for continuous task expansion while ensuring fair evaluation by preventing data contamination via a dynamic test bank. It comprises 12 tasks (1.180 problems) across 4 core sub- abilities: mental rotation, mental folding, visual penetration. and mental animation. Its results show strong discriminative power, revealing the primary limitation in models is visuospatial acquisition over logical reasoning, guiding targeted optimizations in spatial skills.
10

---

Published as a conference paper at ICLR 2026
6 ACKNOWLEDGEMENT
This work was supported by the National Natural Science Foundation of China (NO.62572471).
ETHICS STATEMENT
Data Licensing The SpatialViz-Bencil benchmark released under the MIT license to promote academic and non-commercial research. Its licensing fully complies with all third-party assets used in its creation, which include materials governed by the LGPL (e.g.. FreeCAD), MIT (e.g.. DeepCAD), CCO 1.0 Universal Public Domain Dedication (e.g.. assets from public websites), and default licenses from websites that are known to support non-commercial fair use (e.g., assets from various video websites). For SpatialViz-Bench, including such use by- ... scholarship. or research, is not an infringement of copyright", where fair use determined by "the purpose and character of the use, including whether such use is of a commercial nature or is for nonprofit educational purposes" and "the effect of the use upon the potential market for or value of the copyrighted work."
abide by Fair Use $107 "the fair use of copyrighted work,
Labor Practices All manual data processing and annotation adhered to fair labor practices. Data review for the Mechanical System task was performed by non-author members of our research group, who were compensated for their work via research stipends. All other manual processes, including initial task creation, data verification, and model error analysis, were conducted by the author team as part of their standard research responsibilities.
8 REPRODUCIBILITY STATEMENT
To ensure the full reproducibility of our research, we have made all necessary materials available. The supplementary materials include the complete source code used for data generation (11 of 12 tasks) and model evaluation. Due to submission size constraints, we have provided a "mini" version of our benchmark data, which is sufficient to verify our experimental setup and replicate the core results. Furthermore, facilitate clear understanding of our methodology. Appendix B.4 provides detailed pseudocode for each key algorithm.
REFERENCES
Anthropic. Claude 3.5 Sonnet, 2024.
URL httpss//wwww.rrhroppl...cm///ss
c claude- - 3- 5- sonnet
Anthropic. Claude 3.7 Sonnet, 2025.
URL https//www.atchooolcc.oooo/
claude 3-7-sonnet.
Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. Qwen-vl: A versatile vision-language model for understanding, localization, text reading, and beyond. arXiv preprint arXiv:2308. 1296, 2023.
Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. Qwen2. 5-vl technical report. arXiv preprint 233953 2025.
Christopher Beckham, Martin Weiss, Florian Golemo, Sina Honari, Derek Nowrouzezahrai, and Christopher Pal. Visual question answering from another perspective: Clevr mental rotation tests. Puttern Recognition, 136: 109209, 2023.
Blender Online Community. Blender a 3d modelling and rendering package, 2016.
ByteDance, ByteDance Releases Doubao Large Model 1.5 Pro, Performance Surpassing GPT-4o and Claude3,,Sonnet, 2025. URL htttps://ww/t a aiaase. www.aaa...../eeeeeeaaeeeeeee www a ba com/
news news/11931
Huanqia Cai, Yijun Yang, and Winston Hu. Mm-iq: Benchmarking human-like abstraction and reasoning in multimodal models. arXiv preprint arXiv:2502.00698, 2025.

---

Published as a conference paper at ICLR 2026
Boyuan Chen, Zhuo Xu, Sean Kirmani, Brain Ichter, Dorsa Sadigh, Leonidas Guibas, and Fei Xia. SpatialvIm: Endowing rision-languaee models with spatial reasoning capabilities. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), pp. 14455- 14465, June 2024.
An-Chieh Cheng. Hongxu Yin, Yang Fu, Qiushan Guo, Ruihan Yang, Jan Kautz, Xiaolong Wang, and Sifei Liu. Spatialrgpt: Grounded spatial reasoning in vision-language models. In NeurIPS, 2024.
Google 2025.
Deepmind.
Gemini 2.5: Our most intelligent Al model, March
https://bles hwwpb/ooggggggl//teeceeeeoe 1 ogy/ / googleooollloooooooooooooooooo deepmind/ geming ni- modellthinke wwwpnniiinguudttteeeeeeeee march -
URL
DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, Xiaokang Zhang. Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, Aixin Liu, Bing Xue, Bingxuan Wang, Bochao Wu, Bei Feng, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, Damai Dai, Deli Chen, Dongjie Ji, Erhang Li, Fangyun Lin, Fucong Dai, Fuli Luo, Guangho Hao, Guanting Chen, Guowei Li. H. Zhang, Han Bao. Hanwei Xu, Haocheng Wang, Honghui Ding, Huajian Xin, Huazuo Gao, Hui Qu, Hui Li, Jianzhong Guo, Jiashi Li, Jiawei Wang. Jingchang Chen, Jingyang Yuan, Junjie Qiu, Junlong Li, J. L. Cai, Jiaqi Ni, Jian Liang, Jin Chen, Kai Dong, Kai Hu, Kaige Gao, Kang Guan, Kexin Huang, Kuai Yu, Lean Wang. Lecong Zhang. Liang Zhao, Litong Wang. Liyue Zhang, Lei Xu, Leyi Xia, Mingchuan Zhang, Minghua Zhang, Minghui Tang. Meng Li, Miaojun Wang, Mingming Li, Ning Tian, Panpan Huang, Peng Zhang, Qiancheng Wang, Qinyu Chen, Qiushi Du, Ruiqi Ge, Ruisong Zhang, Ruizhe Pan, Runji Wang, R. J. Chen, R. L. Jin, Ruyi Chen. Shanghao Lu, Shangyan Zhou, Shanhuang Chen, Shengfeng Ye, Shiyu Wang. Shuiping Yu, Shunfeng Zhou, Shuting Pan, S. S. Li, Shuang Zhou, Shaoqing Wu. Shengfeng Ye, Tao Yun, Tian Pei, Tianyu Sun, T. Wang, Wangding Zeng, Wanjia Zhao, Wen Liu, Wenfeng Liang, Wenjun Gao, Wenqin Yu, Wentao Zhang, W. L. Xiao, Wei An, Xiaodong Liu, Xiaohan Wang, Xiaokang Chen, Xiaotao Nie, Xin Cheng, Xin Liu, Xin Xie, Xingchao Liu, Xinyu Yang, Xinyuan Li, Xuecheng Su, Xuheng Lin, X. Q. Li, Xiangyue Jin, Xiaojin Shen, Xiaosha Chen, Xiaowen Sun, Xiaoxiang Wang, Xinnan Song, Xinyi Zhou, Xianzu Wang, Xinxia Shan, Y. K. Li, Y. Q. Wang. Y. X. Wei, Yang Zhang, Yanhong Xu, Yao Li. Yao Zhao, Yaofeng Sun, Yaohui Wang. Yi Yu, Yichao Zhang, Yifan Shi, Yiliang Xiong, Ying He, Yishi Piao, Yisong Wang, Yixuan Tan, Yiyang Ma, Yiyuan Liu, Yongqiang Guo, Yuan Ou, Yuduan Wang. Yue Gong, Yuheng Zou, Yujia He, Yunfan Xiong, Yuxiang Luo, Yuxiang You, Yuxuan Liu, Yuyang Zhou, Y. X. Zhu, Yanhong Xu, Yanping Huang, Yaohui Li, Yi Zheng, Yuchen Zhu, Yunxian Ma Ying Tang, Yukun Zha, Yuting Yan, Z. Z. Ren, Zehui Ren, Zhangli Sha, Zhe Fu, Zhean Xu, Zhenda Xie, Zhengyan Zhang, Zhewen Hao, Zhicheng Ma. Zhigang Yan, Zhiyu Wu, Zihui Gu, Zijia Zhu, Zijun Liu, Zilin Li, Ziwei Xie, Ziyang Song, Zizheng Pan, Zhen Huang, Zhipeng Xu, Zhongyu Zhang, and Zhen Zhang. reepscek-rl: Incentivizing reasoning capability in Ilms via reinforcement learning, 2025.
URL wttttt:ss/ttttteee org/ abs /2011 12948
Sergio Della Sala, Colin Gray, Alan Baddeley, Nadia Allamano, and Lindsey Wilson. Pattern span: tool for unwelding visuo-spatial memory. Nearopsychologia, 37(10):1 189-1199, 1999.
Hongyuan Dong, Zijian Kang, Weijie Yin, Xiao Liang, Chao Feng, and Jiao Ran. Scalable vision language model training via high quality data curation. arXiv preprint arXiv:2501.05952 2025.
FreeCAD Team. FreeCAD: Official source code of freecad, a free and opensource multiplatform 3d parametric modeler. Accessed on 2025-08-15.
http://www.auoeeeeeeeeeeeeeee/ee 2025. Version 1.0;
Xingyu Fu, Yushi Hu, Bangzheng Li, Yu Feng, Haoyu Wang, Xudong Lin, Dan Roth, Noah Smith, Wei-Chiu Ma, and Ranjay Krishna. Blink: Multimodal large language models can see but not perceive. arXiv preprint arXiv:2404.122 12390, 2024.
Leila Glass, Frank Krueger, Jeffrey Solomon, Vanessa Raymont, and Jordan Grafman. Mental paper folding performance following penetrating traumatic brain injury in combat veterans: a lesion mapping study. Cerebral Cortex, 23(7): 16731 2013.
12

---

Published as a conference paper at ICLR 2026
Wenyu Han, Siyuan Xiang, Chenhui Liu, Ruoyu Wang, and Chen Feng. Spare3d: A dataset for spatial reasoning on three-view line drawings. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11690-14699, 2020.
Yunzhuo Hao, Jiawei Gu, Huichen Will Wang. Linjie Li, Zhengyuan Yang, Lijuan Wang, and Yu Cheng. Can milms reason in multimodality? emma: An enhanced multimodal reasoning benchmark. arXiv preprint arXiv:2501.05444 2025.
Aaron Hurst, Adam Lever, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Os- trow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card. arXiv preprint arXiv:2410.21276, 2024.
Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Camey, al. Openai ol system card. arXiv preprint arXiv:2412.16720, 2024.
Dongzii Jiang, Renrui Zhang, Ziyu Guo, Yanwei Li, Yu Qi, Xinyan Chen, Liuhui Wang, Jianhan Jin, Claire Guo, Shen Yan, et al. Mme-cot: Benchmarking chain-of-thought in large multimodal models for reasoming quality, robustness, and efficiency. arXiv preprint arXiv:2502.0m 09621, 2025.
Yifan Jiang, Jiarui Zhang, Kexuan Sun, Zhivar Sourati, Kian Ahrabian, Kaixin Ma, Filip Ilievski, and Jay Pujara. Marvel: Multidimensional abstraction and reasoning through visual evaluation and learning. arXiv preprint arXiv:2404. 13591, 2024.
Justin Johnson, Bharath Hariharan, Laurens Van Der Maaten, Li Fei-Fei, C Lawrence Zitnick, and Ross Girshick. Clevr: A diagnostic dataset for compositional language and elementary visual reasoning. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 2901-2910, 2017.
Amita Kamath, Jack Hessel, and Kai- Chang. What's "up" with vision-language models? investigating their struggle with spatial reasoning. In EMNLP, 2023.
Chenglin Li, Qianglong Chen, Zhi Li, Feng Tao, and Yin Zhang. Vebench: A controllable benehmark for symbolic and abstract challenges in video cognition. arXiv preprint arXiv:2411.09105, 2024.
Fangyu Liu, Guy Edward Toh Emerson, and Nigel Collier. Visual spatial reasoning. Transactions of the Association for Computational Linguistics, 2023.
Meta AI The Llama herd: The beginning of a new era of natively multimodal Al innovation, 2025.
URL hittps://a/ http://aaaaaaa..e///ee/e.. .we.a.ammmmmmmmmmmmmm....mm..... com boeoon//oo////i aaa.loo//lmaaaaaaammmmmaaaaaaala mu timtmallll--- - -nte 11 gence/
Shiwen Ni, Guhong Chen, Shuaimin Li, Xuanang Chen, Siyi Li, Bingli Wang, Qiyao Wang, Xingjian Wang. Yifan Zhang, Liyang Fan, Chengming Li, Ruifeng Xu, Le Sun, and Min Yang. A survey on large language model benchmarks, 2025. URL httth::tt//aaxtttt org/ abs/ 2508 15361
Jiahao Nie, Gongjie Zhang, Wenbin An, Yap-Peng Tan, Alex C Kot, and Shijian Lu. Mmrel: A relation understanding benchmark in the milm era. arXiv preprint arXiv:2406.0912), 2024.
Minnan Pei, Gang Li, Junwen Si, Zeyu Zhu, Zitao Mo, Peisong Wang, Zhuoran Song, Xiaoyao Liang, and Jian Cheng. Gec: A 3dgs inference architecture with gaussian-wise and cross-stage conditional processing. In Proceedings of the 58th IEEE/ACM International Symposium on Mieroarchitecture, pp. 1834-1837, 2025.
Qwen Team. Qvq: To the world with wisdom, December 2024. URL https:: //ee://////emmmmmmm github.
wttpb/ic/////.... htto/llo//vq-2700 72b-preview
Roger N Shepard and Jacqueline Metzler. Mental rotation of three-dimensional objects. Science, 171 (3972):701-703, 1971.
Fatemeh Shiri, Xiao Yu Guo, Mona Far, Xin Yu, Reza Haf, and Yuan-Fang Fang Li. An empirical analysis on spatial reasoning capabilities of large multimodal models. In Proceedings of the 2024 Conference Empirical Methods in Natural Language Processing, pp. 11440-21455. 2024.
13

---

Published as a conference paper at ICLR 2026
Valerie K Sims and Mary Hegarty. Mental animation in the visuospatial sketchpad: Evidence from dual-task studies. Memory & Cognition, 25:321-332, 1997.
Yueqi Song, Tianyue Ou, Yibo Kong, Zecheng Li, Graham Neubig, and Xiang Yue. Visualpuzzles: Decoupling multimodal reasoning evaluation from domain knowledge, 2025. URL https
//axxiv.org/ab///5533 10342
Ilias Stogiannidis, Steven McDonagh, and Sotirios A Tsaftaris. Mind the gap: Benchmarking spatial reasoning in vision- vision-language models. arXiv preprint arXiv:2503. 19707, 2025.
Kexian Tang, Junyao Gao, Yanhong Zeng, Haodong Duan, Yanan Sun, Zhening Xing, Wenran Liu, Kaifeng Lyu, and Kai Chen. Lego-puzzles: How good are milms multi-step spatial reasoning? arXiv preprini arXiv:2503. 19990, 2025.
Kimi Team, Angang Du, Bohong Yin, Bowei Xing, Bowen Qu, Bowen Wang. Cheng Chen, Chenlin Zhang, Chenzhuang Du, Chu Wei, et al. Kimi-vl technical report. arXiv preprint arXiv:2504.07491, 2025.
Louis Leon Thurstone. Primary mental abilities: Psychometric monographs no. 1. The measurement of intelligence, pp. 131-136. Springer, 1938.
Louis Leon Thurstone. Some primary abilities in visual thinking. Proceedings of the American Philosophical Society, 94(6):117-521, 1950.
Sarah Titus and Eric Horsman. Characterizing and improving spatial visualization skills. Journal of Geoscience Education, (744))2442-244 2009.
Ke Wang, Junting Pan, Weikang Shi, Zimu Lu, Houxing Ren, Aojun Zhou, Mingjie Zhan, and Hongsheng Li. Measuring multimodal mathematical reasoning with math-vision dataset. In The Thirty-eight Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2024. URL htttts:://ppereeieeeeee net wwt.:/oodoowaddoooooo000020
Mingqi WI, Zhihao Zhang, Qiaole Dong, Zhiheng Xi, Jun Zhao, Senjie Jin, Xiaoran Fan, Yuhao Zhou, Huijie Lv, Ming Zhang, Yanwei Fu, Qin Liu, Songyang Zhang, and Qi Zhang. Reasoning or memorization? unreliable results of reinforcement learning due to data contamination, 2025. URL
https::/arxx www.o//a/asa.. 2507. 10532
Rundi Wu, Chang Xiao, and Changxi Zheng. Deepead: A deep generative network for computer- aided design models. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). pp- 6772-6782. October 2021.
Zhiyu Wu, Xiaokang Chen, Zizheng Pan, Xingchao Liu, Wen Liu, Damai Dai, Huazuo Gao, Yiyang Ma, Chengyue Wu, Bingxuan Wang, et al. Deepseek-v12: Mixture-of-experts vision-language models for advanced multimodal understanding. arXiv preprint arXiv:2412.10302 2024.
Yijia Xiao, Edward Sun, Tianyu Liu, and Wei Wang. Logicvista: Multimodal 11m logical reasoning benchmark in visual contexts, 2024.
wwwp/axiivvvi........a.a.aaaaaaa wbtp:///////////..//////////// 04973
Zhinan Xie, Peisong Wang. Shuang Qiu, and Jian Cheng. Hivis: Hiding visual tokens from the drafter for speculative decoding in vision-language models. arXiv preprint arXiv:2509..23928 2025.
Jin Xu, Zhifang Guo, Jinzheng He, Hangrui Hu, Ting He, Shuai Bai, Keqin Chen, Jialin Wang, Yang Fan, Kai Dang. et al. Qwen2. 5-omni technical report. arXiv preprint arXiv:2503.20215, 2025a.
Wenrui Xu, Dalin Lyu, Weihang Wang, Jie Feng, Chen Gao, and Yong Li. Defining and evaluating visual language models' basic spatial abilities: A perspective from psychometrics. arXiv preprint arXiv:2502.11859 2025b.
An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang. Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Mei Li, Mingfeng Xue, Pei Zhang, Qin Zhu, Rui Men. Runji Lin, Tianhao Li, Tingyu Xia, Xingzhang Ren, Xuancheng Ren, Yang Fan. Yang Su, Yichang Zhang, Yu Wan, Yuqiong Liu, Zeyu Cui, Zhenru Zhang, and Zihan Qiu. Qwen2.5 technical report. arXiv preprint 1rX1v:2412.15115, 2024a.
14

---

Published as a conference paper at ICLR 2026
Jihan Yang, Shusheng Yang, Anjali W Gupta, Rilyn Han, Li Fei-Fei, and Saining Xie. Thinking in space: How multimodal large language models see, remember, and recall spaces. arXiv preprint
arXiv:2412.14171, 20244.
Baiqiao Yin, Qineng Wang, Pingyue Zhang, Jianshu Zhang, Kangrui Wang, Zihan Wang, Jieyu Zhang. Keshigeyan Chandrasegaran, Han Liu, Ranjay Krishna, Saining Xie, Manling Li, Jiajun Wu, and Li Fei-Fei. Spatial mental modeling from limited views, 2025. URL https: pettps://arxix
org/ abs/ 2506 21458
Jinguo Zhu, Weiyun Wang, Zhe Chen, Zhaoyang Liu, Shenglong Ye, Lixin Gu, Yuchen Duan, Hao Tian, Weijie Su, Jie Shao, et Internv13: Exploring advanced training and test-time recipes for spen- multimodal models. arXiv preprint arXiv:2504.1047, 2025.
15

---

Published as a conference paper at ICLR 2026
APPENDIX
Detailed Related Works
17
A.1
Current Landscape in Spatial Reasoning Benchmarks
17
A.2
The Inadequate Evaluation of Spatial Visualization
17
Data Curation Details
18
Task Construction
18
B.2
Programmatic Data Generation Pipeline
19
B.3
Manul Design for Mechanical System Task
20
4
Pseudocode
21
Dataset Characteristic
39
Data Examples
40
Evaluation Details
46
Models
46
Prompts for Response Generation
46
Zero-shot Setting
46
Methods for Answer Extraction
46
5
Human Performance
47
E,
Error Analysis
48
Detailed Results
49
Intra-Category Comparisons Across Levels
49
Statistical significance analysis of Co' prompting
51
3
Performance Comparison between Different Question Format
51
Test Cases
56
16

---

Published as a conference paper at ICLR 2026
A DETAILED RELATED WORKS
A.1 CURRENT LANDSCAPE SPATIAL REASONING BENCHMARKS
Spatial reasoning foundational to embodied intelligence, supporting critical tasks like navigation, interaction, and scene understanding. The evaluation of this ability in MILLMs has historically focused on two primary areas: spatial perception and spatial memorization, both of which rely on interpreting directly observable, explicit visual information.
Spatial Perception, the ability to interpret spatial relationships from static visual input, is the most established area. Early benchmarks targeted perceptual level understanding, such as monocular depth estimation and object localization. With the rise of MLLMs, this has shifted to visual question answering formats. For instance, datasets like VSR (Liu et al. 2023) and What's (Kamath et al.. 2023) benchmark models' comprehension of object-centric spatial relationships. Others, including SpatialVLM (Chen et al., 2024). Spatial-MM (Shiri al. 2024), and MMRel (Nie et al. (024). further expand this evaluation to include relative distances, camera-object perspectives, and object size comparisons. More advanced benchmarks like Blink (Fu
2024), with its Multi-view Reasoning task, and OpatialRGGTT-enee (Cheng et al. 2024), which incorporates world knowledge and multi-hop reasoning, have pushed the boundaries but remain centered on interpreting what explicitly perceived.
Spatial Memorization, the ability to track objects and their relationships in dynamic scenes, has been increasingly addressed by iideo-hased benchmarks. VCBench (Li et al., 2024) evaluates this through tasks like Flash Grid and 3D Navigator, which test a model's capacity to retain 2D spatial positions and predict trajectories in 3D space. Similarly, VSI-bench (Yang et al. 2024b) focuses on skills essential for navigation, such as egocentric-to-alle tololeccet transformation and perspective-shifting
While these efforts have built strong foundation, they predominantly assess reasoning based on explicit visual cues. They largely neglect the more advanced capability of spatial visualization the mental manipulation of shapes and inference of implicit spatial information-leanee significant gap in the current evaluation landscape.
A.2 THE INADEQUATE EVALUATION OF SPATIAL VISUALIZATION
Despite its importance, the evaluation of spatial visualization is fraught with challenges, including obscured categorization general benchmarks, high risk of data contamination, and a lack of diagnostic depth.
Obscured Categorization Spatial visualization is often not recognized as a distinct spatial skill. Instead, it is frequently subsumed under broader domains like mathematical logical reasoning within general-purpont MLLM benchmarks. Examples are widespread: appears as the 3D-Geometry category in MM-IQ (Cai et al. category in EMMA (Hao et al. 2025) 3D Shapes in Logie Vista (Xiao et al. 2024), IQ-Test in Blink (Fu al. 2024), and Descriptive/ Descripiive/Tranffonaatti 2024). While VisualPuzzles (Song et al. 2025) correctly situates it under spatial reasoning, this is an exception. This common miscategorization diverts focus from developing and evaluating spatial visualization as a core ability, treating it merely as type of puzzle.
2025) and MARVEL (Jiang et al., (2024). the 3D Spatial Simulation
Geometry in Math-Vision (Wang al.
Risk of Data Contamination The difficulty of designing novel spatial visualization tasks means that existing benchmarks often source questions from public materials like IQ tests, administrative exams, and math contests. This practice creates a high risk of data contamination, as these materials are likely part of the massive web-scraped datasets used for pretraining MI..Ms. For example, work by Xu et al. (2255) collects data entirely from online psychological tests. Consequently, model's high performance such benchmarks may not reflect true reasoning capabilities but rather memorization from the training data, compromising evaluation validity.
Non- Non-Diagnostic Evaluation Current evaluations are often caught between two non-diagnostic ex- tremes. On one hand, the heterogeneous, mixed-format questions in general benchmarks make difficult to isolate and diagnose emrors in spatial visualization specifically. On the other hand, special- ized datasets are often too narrowly focused on a single sub-skill. For example, SPARE3D (Han et al.
2020) and CLEVR-MRT (Beckham al. 2023 concentrate on mental rotation, while SRBench (Sto-
17

---

Published as a conference paper at ICLR 2026
giannidis al. 2023) uses only paper folding tasks to assess the entire ability. This narrow scope fails to provide a comprehensive assessment model's 's overall spatial visualization proficiency.
In contrast to these prior works, our benchmark is designed to be systematic and diagnostic. It structured around 4 core sub-skills of spatial visualization identified cognitive psychology. with curated tasks targeting each ability. By employing procedural generation for most tasks, benchmark ensures greater reliability, reduces the risk of training-set overlap, and enables scalable data creation for both evaluation and future training. Furthermore, by summarizing the essential phases of spatial visualization, our framework allows for more granular analysis to identify the root causes of reasoning errors.
B DATA CURATION DETAILS
B. TASK CONSTRUCTION
Mental Rotation
2D Rotation Task. A colored grid pattern with red corner marker is rotated by 909/180'/200* generate positive samples. Negative samples involve horizontal/vertical mirroring. We further replace symmetric color fills with non-centrally symmetric patterns. Negatives include mirror flips and internal rotations of pattern components, increasing spatial reasoning difficulty. As shown in Alggortt I
3D Rotation Task. connected cube stack is rotated along x/y/z axis to form positives. Negatives are created by removing one cube or mirroring the isometric view, ensuring no simple rotation can reproduce them. Spatial complexity increased by enlarging assembly dimensions, requiring enhanced 3D rotational reasoning. As shown in Algorithm 2 and Algorithm 3
This task has two categories. Firstly, given isometric, front, and top views of a connected cube stack with marked reference cubes, the task is to select the correct left view. Negatives involve altering reference cube positions or substituting the right view. We further introduce real engineering parts from the DecpCAD dataset (Wu et al. (202I). rendered into standard projections via FreeCAD. Negatives are crafted through random internal lines deletion, view flipping/rotation, transformations on unseen views. As shown in Algorithm 4) and Algorithm 5
Three-Vier Projection Task.
2. Mental Folding
Paper Folding Task. A Python-based based pipeline generates grid patterns undergoing sequential folds perricca/horizzontal/laagggo
followed by hole-punching and unfolding. The task requires identifying the correct unfolded hole distribution. Negative samples generated by mirroring, deleting, adding, or relocating holes to violate fold-induced symmetry. Task difficulty increases with more folds, larger grids, and denser hole placements. As shown in Algorithm aand Algorithm
Cube Unfolding Task. Given a cube with six uniquely colored faces and a view from a corner (three visible faces), the task is to select the correct 2D net (11 possibilities as shown in Figure 7). Positives can be crafted either by using different cube nets of the same cube by fixing the mapping of visible faces while randomly shuffling the remaining faces. Negatives are crafted by swapping visible face colors flipping visible-opposite face pairs. We further replace solid colors with non-centrally symmetric patterns. View angles prioritize faces with asymmetric patterns. Internal rotations of pattern components
introduced to further increase the reasoning difficulty. To push the difficulty even further, all six faces feature random colored-dot patterns on 3x3 grid. As shown in Algorithm 8 Algorithm 9 and Algorithm 10
Cube Reconstruction Task. Cubes have six uniquely colored faces. Two task variants exist: (1) select the correct vertex view of a cube when given its net pattern, with negative samples created by mirroring the correct view; (2) identify the color of a face opposite to a given colored face. Difficulty progression follows the cube unfolding tasks. As shown in Algorithm 8 and aangorlggi 11
3. Visual Penetration
Cross-Section Task. Nine basic geometric solids (e.g., triangularrrcctanuullrciiiu prisms/pyra- mids/frustums) are combined in pairs with conical shapes on top. Cross-sections are generated by slicing the composite shapes using planes parallel the XY/Y7/XZ planes. Negative samples are
18

---

Published as a conference paper at ICLR 2026
Bk
Dt
BR
Sk
1-4-1-0
1-4-1-1
1-4-1-2
1-41-3
1-+14
1-411-5
Bt: Bottom
T: Top Front
DI
BI
Di: Bark
Right
L: Left
2-3-1-0
2-3-1-1
2-3-1-2
2-2-2
Figure 7: The eleven unfolded patterns of cube with their corresponding numbered names. Assuming the square in row I, position 0 represents the bottom face, and position represents the right face, the corresponding arrangement of the remaining faces can be determined, facilitating the rotation of the cube.
constructed by adjusting the relative geometric proportions within the composite. Task complexity increased by introducing composites with three solids, which often produce disconnected cross- sections that demand enhanced visual reasoning. Additional complexity is introduced by generating oblique cross-sections at /5"/ As shown in Algorithm 12]
Cube Counting Task. The task requires inferring the total cube count of connected cuhe stack based on two orthogonal projection views. The minimum and maximum counts are mathematically derived guide the construction of answer options. Constraints increase to three orthogonal projection views, reducing the number of possible solutions while increasing view integration complexity. Task difficulty further increases by expanding the spatial dimensions of the cubic assemblies. As shown in Alloriihm 2 and Algorithm 13
Cube Assembly Task. A pyramid-like cube stack is split into two connected parts. Tasks require identifying the complementary piece that fits the reference part. Negative samples are generated by modifying the correct piece through the addition or removal of cubic units. The difficulty further increased by enlarging the spatial dimensions and dividing the structure into three parts instead of two. As shown in Algorithm 14 and Algorithm 15
4. Mental Animation
Arrow Moving Task. For the easy version, an arrow with random initial position and orientation in a 3x3 grid operates by ego-centric rules: movement occurs in 4 directions horward/backwar//efft///// with "forward" always indicating the arrow's 'ss current orientation. The arrow reorients to the movement direction after each movement. Valid operation sequences samples share the same initial state but yield incorrect endpoints. For the hard version, multiple colored arrows are introduced with extended rules: empty positions allow direct entry: occupied positions trigger object exchanges while maintaining Level movement principles. Tasks include predicting final states from sequences, inferring correct sequences from state pairs. As shown in Algorithm 16 Algorithm 17 Algorithm 18 and Algorithm 19
algorithmically generated; negative
Block Moving Task. Colored cube stack combines directional movement with gravity simulation. Cubes move along six directions with unsupported cubes falling until reaching support and swapping positions as same as Arrow Moving Task. Increased spatial complexity and longer sequences elevate reasoning difficulty. As shown in Algorithm and Algoriitm 21
Mechanical System Task. We use open-souree mechanical system simulations, classifying com- plexity by module quantity and designing appropriate questions. These tasks assess advanced mental animation abilities, particularly to understand how the motion of one component affects others.
B.2 PROGRAMMATER DATA GENERATION PIPELINE
FreeCAD, an open-source Computer- Aided Design (CAD) software, provides deep integration with Python programming language, enabling parametric model construction through programming. We leveraged the synergy between FreeCAD and Python successfully automate the generation of
19

---

Published as a conference paper at ICLR 2026
9 spatial visualization tasks: 2DRotation, 3DRotation, 3ViewProjecttoon, CubeFolding, CubeRe- construction, CrossSection, CubeCounting, CubeAssembly, and BlockMoving. Additionally, two tasks PasssPolppoo and ArrowMoving- calSystem task, due to its complexity and specific requirements, we employed precise manual design methods. To supplement the task overview presented in Section [3.3] the following sections provide detailed pscudocode for each programmatically generated task, offering more systematic and in-depth technical insights.
implemented solely using Python. For the Mechani-
Mental Rotation Tasks. Algorithm I presents the pseudocode for the 2D Rotation Task. For the 3D Rotation Task, Three- View Projection Task, Cube Counting Task, and Block Moving Task, we need to construct connected cube stacks, with the core functions detailed in Alggorithhm 2 Algorithm 3) demonstrates the complete implementation process of the 3D Rotation Task. The method for generating three-view projections of marked cube stacks is elaborated in[Algorithm 4 [Algorithm 5 describes the process of importing models from the DeepCAD dataset and generating their three-view projections.
Mental Folding Tasks. Algorithm 6jimlleeents Paper class for simulating the dynamic processes of paper folding, holes punching, and unfolding. Based this simulation framework, Algorithhm 7 constructs the data for the Paper Folding Task. AAgorithm 8 presents the core functions for trans- forming 11 standard cube nets (as shown in Figure 7p into three-dimension transformation functions, while Algorithm demonsttaates how different unfolding patterns can produce the same cugorrign and Algorithnn 11 provide the complete pseudocode implemen- tations for the Cube Unfolding Task and Cube Reconstruction Task, respectively.
cubes. Utilizing these
Visual Penetration Tasks, Algorithm 12 details the implementation pseudocode for the Cross- Section Task. Algorithm 13 comprehensively presents the data generation procedure as well as the mathematical calculation process guide the construction of answer options in the Cube Counting Task. |Algorithm 14 contains the core functions for decomposing a complete cube stack into multiple connected parts. Building upon these functions, Allgrrithm 13 provides the complete construction pseudocode for the Cube Assembly Task.
Mental Animation Tasks. Algorithm 16 implements an ArrowPath class for simulating the move- ment process of an arrow centered on itself. Algorithm jijmplmmentss an ArrowMap class that inherits from the ArrowPath class, designed to simulate movement and exchange operations in multi-arrow environments. Based on the ArrowPath class, Algorithm details the data construction process for the single-arrow version of the Arrow Moving Task. Correspondingly. using the ArrowMap class, Algorithm 19 clucidates the data construction process for the multi-arrow version of the Arrow Moving Task. |Algorithm 20 implements a Block class for simulating the movement and exchange processes of blocks that follow gravitational rules. Building upon this Block class, Allgorithm 21 presents the complete pseudocode implementation the Block Moving Task.
B.3 MANUL DESIGN FOR MECHANICAL SYSTEM TASK
To ensure the objectivity and quality of the Mechanical System task, we first collected simulation materials from open-source platforms. The question-answe pairs were designed by members of the author team, who strictly followed a standardized template based on the observable and deterministic animations (e.g.. "If component A rotates clockwise, how does component B move?"). This structured process was designed to minimize subjectivity and focus the evaluation specifically on model's is ability to infer causal dynamics from visual input. To verify the accuracy of these question-answer pairs, we recruited two graduate student annotators from our research group, who received compensation for their contributions. They first performed independent reviews of each sample and then discussed their findings to resolve any discrepancies and reach a final consensus. This rigorous process ultimately produced 80 validated data samples.
20

---

Published as a conference paper at ICLR 2026
B.4 PSEUDOCODE
Algorithm 2D Rotation Task
Input: Color(Pattern) set C. grid size (H, W), unit length s, marker length s', task mode m
2: Initialize binary matrix M E {0, with random values
3: Initialize empty lists posilive satpliiesssmmmeeseeeeeeeee negalieeessmppes
4: function
C, H, W, 8', record = list())
5: for i < to 1 do
6: for j 0 to W --W--- do
7:
pos + [j 8, (H 1 8, 0)
8:
square <- FreeFeDe www.maakePllaa elneess, 8, (pos, 09))
9:
M then
if record is empty then:
11:
Randomly select e E C' and assign e to square at pos Append to record
13:
else
Assign rota rotatt (PPopattePoo( 0), 90%) to square at pos end if end end for
15:
16:
17:
18:
end for
19:
Randomly select corner |'top left", "top right", "bottomm_eff" "bottom "bottom_right")
20:
posmarker - Sereeeearee______________
W, 8, s', corner)
21:
FreeCAD FreeCAD.makepe makeeallaaaeee maee 8', (po8marker, 0*)) with red color
22:
img +- FreeCAD, smaeee
return img, record
24: end function
25: ref_img, record + Crw.Grr....ttthaarrrrr..C.CC.C.
26: if m = "mpattreern" then
27:
record <- DrawGridwittharkerr C,H, W, reood))
28: Append transform_ime forrriigmmmgmmmmooo to negative_sample
29: end if
30: for angle E (90%, 1800, ) do
31: img f- row.tteeeeeee............
angle)
32: Append img to positive_samples
end for
E 34: for { ["horizontal", "vertical") do
img + flip(ref laatrreffmgg.aaeee faip.flip_dii [lip
36: Append img to negative stattieeessapessssseeeeeeessesse
37: end for
38: samples + [positive_sampless reggatieesamplees
39: Shuffle samples to assign [A, B, C, D] and record answer _id
40: data create arreate_dta_eee alre fimg, samples, question, onswer_ _
21

---

Published as a conference paper at ICLR 2026
Algorithm 2 Fucntions for Creating Cubes with None-isolated Regions
1: Input: Spatial size (X, Y, Z), cube size
2: Initialize zero value 3D tensors placement E (0)2xXxX empty list cubes
3: function CREATECUBE((, y, 2)
4: cube FrereCCAD ma makeebox(ss, (I, v)) and append cube to cubes
5: placemend[=][]][]]
6: end function
7:
function CREATECUBES(K, Y, Z)
8:
for 4 to 1-1 do do
9:
for y to
10:
for x + 0 to X-1
11:
= 0 or placement falaccement_spccele-eeee [p then
12:
With 50% probability CreateCube (z, v,
13:
end if
14:
end for
end for
16:
end for
end function
18:
function CONNECTISOLATEDCUUBSSS Y)
19:
cubesuy + {(x: y) placement(() [v] 1}
20:
Initialize empty set visited, empty list regions directions <-
21:
22:
for all (x, y) E cubes2y do
if (z, y) & visited then Initialize empty list region, empty queue queue Add (x, y) visited, add (x,, y) queue while queue is not empty do
24:
25:
26:
(cz, cy) < -popLeft(queee) Append (cx,cy) to region for all (dx, dy) E directions do
28:
30:
(næ, +- (ex + dx, cy + dy)
31:
0 S næ < X and 0 S ny < Y and (nz, nyy $ visited
and placement (0] [nt]] = 1 then
32:
Add (næ, ny) to visited, add (nz, ny) to queue
33:
end if end for end while
34:
35:
36:
Append region to regions end if end for
38:
39:
if |regions| > then
40:
for i <- 0 to |egiooss| - 2 do
41:
Find (x1, (I2,, y2) with min L, distance between regions[i] and regionsli + T1, 6- 1/1
42:
43:
while ( x2) or (y + y2) do
44:
if r + z2 and
1/2 then
45:
x x x
46:
else if * 32 then
47:
x + x =+
48:
else if y + y/2 then
49:
y/ + y 1
end if if placement (icemmentsppcce[]llla = then CreateCube CreateChubeplacemenn -, x, end if end while end for
51:
52:
53:
54:
55:
56:
end
57:
end function
22

---

Published as a conference paper at ICLR 2026
Algorithm 3 3D Rotation Task
1: Input: Spatial size (X, Y, Z), cube size
2: Initialize zero value 3D tensors placement [0]*XXxX, empty list cubes
3: Initialize empty lists pnsilive_samplee
negalive_samipn
4: Update placement, cubes with CreateCubes( Y, Z)
5: Update placement, cubes with Connect Ielaeedbee,,,, Y)
6: ref_img + www.eCADD.aaeeaaeeeeeeee
7: for + 1 to 4 do
8: Randomly select oris E {x, y, 2] and angle {90", 180", 270")
9: Mitated_cubes < rotate(cubes, azis, angle)
10:
rotated feaate_img-- t- FreeCAD save Image brotated_cubert
11:
Append rotated img
12: end for
13: cubes' Raandomly remove a cube from cubes and rotate the left cubes as above ptaaee__rmmovd__ing +- FreeCAD, saveemageecuuees 15: Append rotated_removedd
to megative_samplee
16: for Flip_din ["horizontal", "vertical") do
17: Randomly choose sample from positive_samppea
18: img f- flip( isamsmlee, flip_ dir)
19: Append img to regaaive_samplee
20: end for
samples + (positive sattesssampsssssseeeee negative samples)
22:
Shuffle samples to assign [A, B, C, D] and record answer id
data +- www.teeeeeeeeee [rreate_datarref _img, samples, question, onswer_ id)
23

---

Published as a conference paper at ICLR 2026
Algorithm 4 Three- Projection Task with Marked Cube Stack
1: Input: Spatial size (X, Y, Z), cube size
2: Initialize zero value 3D tensors placement & [0)2xXXX empty list cubes
negaliveesamples 4: Update placement, cubes with CreateCubes(X, Y, Z)
3: Initialize empty lists posilive_sampple
5: Update placement, cubes with Commectssoaaeeduuesset
Y)
6: function COLORVISIBLEFACCSSC Y, Z, colored_numi
7: cubes +- Find cubes that can be seen from front or top left view
8: Randomly color min (eaoooooeee_uuu (cubesl) cubes in red
9: end function
function (ceeVeESS(cubes)
11: Initialize empty list views
for all view ("Isometric", "Top", "Front", "Left") do
13: img F-FreeCDD saveView(view) and append img to views
end for 15: return views
end function
17: Update cubes with ColorVissibleeFcee((X Y, Z, colored_nume
18: views SaveViews (cubes)
19: Select le.L_ from views to posilive_sample
20: Select right_riew from views negalive_sample
Cleaer all colors and update cubus with colorviisiblePacees() Y, Z, colored colored_num) num) as above
22: new_vieut < SaveViews (cubes)
23: Select left_view and right_vie _iew from nee_wiews to negative_samplle
24: samples
petratiee_sapless
25: Shuffle samples to assign [A, B, C, D] and record answer _id
26: ref_ing (isometric_vien) view, top_view, (ront_view)
27: dala + -r-eeteedata(ree _img, samples, question, answer_idl _i))
24

---

Published as a conference paper at ICLR 2026
Algorithm 5 View Projection Task with Models from DeepCAD Datasets
1: Input: step file path pth
2: Initialize empty lists posilive_samples. negalive_samppes
3: shape O-een
4: views + SaveViews (shape)
5: function CREATTENCCORECCTEEEWWW.WO.
mode)
6: if mode = 0 then
7: img' Extract all internal lines and randomly delete line
8: else if mode = 1 then
9:
img' t 90%)
10: else if mode = 2 then
img' fl flie "horizontal" or "vertical")
12: end if
return img" end function
14:
15: ref_wiew CChoos view from views with max area
16: (questioned_ wiie other_wiee _iter_vwwww t- Randomly assign views except for ref_view
Append questioned view positive_samples 18: for mode + 0 to 2 do incorrect_vrieu -crrattetnoorrrrevveeeeearrraede other_vien mode) 20: Append incorrect_ wiew to negative_samplee
end for
22: samples + (positive arositive_samplle, negative 3
23: Shuffle samples to assign [A, B, C, DJ and record answer_id iid
24: ref_img + (isometric, view, top_ sie_vee, front_view)
25: data create
ctcccreeeeatrreeee dataareefaaaaaeaaddda _ing, samples, question, onswer_id) _id)
25

---

Published as a conference paper at ICLR 2026
Algorithm 6 Simulation for Paper Folding, Punching and Unfolding
1: Class Paper
2: Attributes:
3: grid, complele_ggid: 2D arrays representing current and complete paper states
4: Tigginl_rouss original _cols: initial dimensions
5: current_ron rows, current_cod current dimensions after folding
6: folds: list of fold operations
7: function FoL.(direction, line diagoonalpoiniee
8:
if' direction is horizontal then
9:
Calculate folded area
10:
Update complete_gre
by marking folded area as -1
Create new grid with updated dimensions
12:
else if direction is vertical then
Similar to horizontal but for columns else if direction is diagonal then
14:
15:
Calculate diagonal line equation Mark appropriate triangular area -1 17: end
16:
18: Record fold operation in folds
19:
end function
20: function PUNcH(points)
for each (æ, y) in points do
22: Set grid(ar [y] 1
Set corresponding complete_griee position to
24: end for
25: Record punch operation in folds
26:
end function
27:
function UNFOLD
28:
for each foold in reverse folds do
if fold horizontal then
30:
Mirror grid about fold line
else fold is vertical then
32:
Mirror grid about fold line
else fold is diagonal then
34:
Mirror grid about diagonal line
35:
end
Update current dimensions of paper 37: end for
36:
38: Clear folds list
39:
end function
40: function CRESTELLNOOREECTTVEEWWYdmaee
Create incorrect variant by: if mode "row" then
42:
43:
Either remove a row of holes, add extra row, or swap rows
44:
else if mode = "col" then
45:
Either remove column of holes, add extra column, or swap columns
46:
else
47:
Combine row and column errors
48:
end if
49:
Update paper with above changes
end function
26

---

21: end for Published as a conference paper at ICLR 2026
Algorithm 7 Paper Folding Task
1: Input: Dimensions of paper (rows, cols), number of folds steps, number of holes punches
2: Initialize paper with dimensions rows x cols
3: Initialize empty lists ref ref_imgs, posilice_sanpless negaliveesampless
4: for step < to steps do
5: if' step steps then
6: direction t "diagonal"
7: else
direction + Randomly select direction E ["horizontal" "vertical"] 9: end
8:
"vooriionaal""vvvricaaal"
10: if direction= "horizontal" then
11:
line + randomInt (1, pwppp..ppaaaee wwwrrruurrrertrueeee 1)
12:
paper.Fold(ddrrctionn line)
13:
else direction "vertical" then
14:
line rwndomInn.n.(..m.mm.. paper.current_comm cols 1)
15:
paper.Fold line)
else if direction "diagonal" then
17:
diagonall_oiive + Randomly select one set of 45-degree line endpoints Daper.Fold iipperPolddddrrctionn,, d )
19: end if
img seadraa_paper(peee and append img to ref_imgs
22: points Randomly select punches positions
23: paper.Punchhppiints))
24: img wdwawwapaaeeeeppppppppp and append img ref_imge
paper.Unfold()
26: img htttrrraaaappepppee and append img to jespplee_samples
27: Initialize paper' with same dimensions paper
28: paper'-grie + paper.grid to copy the state of unfolded paper
29: Determine the incorrect view mode
30: for ++1 to do
31: Update paper' with paper' wttp://rrreeeeenorreeteeeeeeeeee
32: img +-draw (paper') and append img to negative_samples
end for
samples + (positioe_samples negative_samples)
35: Shuffle samples to assign [A, B, C, D] and record answer_id _id
36: dala +- create_dataaaeeeeeeeeeeeeeeeeeee _imgs, samples, question, onswer _id)
27

---

Published as a conference paper at ICLR 2026
Algorithm 8 Functions for Reconstruting Cube from 11 Kinds of Cube Nets
1: Input: cube size $
2: Define rotation operators:
3: (0): Rotation about X-axis by 0 degrees
4: (,)() Rotation about Y-axis by 9 degrees
5: (0): Rotation about Z-axis by 0 degrees
6: function NET2CUBF(pllmee_aate _nae map; view, rof)
7:
Initialize placement dictionary planes
8:
planes "Bottom" "daanes""Bttom""" - ((=/2, p
planes["Top"]]
(1809))
9:
Rr(0)
10:
planes "Right") - (s8,
(-9-))))
planes "Left")
< [Left") (0.s/22s/)) Ry (90%) R. (90%))
12:
planes["Backe < s.//2,,. Rz(90%)
if plane_name name is "2-2-2" then
14:
plames["Top <- (s/2, s), R. (180%) R2 (-
15:
else plane_ plane_name name "1-4-1" then
16:
planes["Left") + s/2)), (90%)
17: end
18:
if plane_ plane_name nome E "=1-4-1-0","2-3--100"""" then
19:
planes("Front" <- ((s/2.0.s/2), {(ss//20.ss/),.R,[ R,(-909)
20:
else if plane_nanee
[-1-4-1-1", """"=""""""""""""""""""""""""""" "2-2-2") then
21:
planes["Front" <- ((s/2,0,s/2), R, R, ( R=(-90))
else plane name
"3-3"" then
23:
planes (Frontt" - ((s/2,0,s/2).
else plane_name name "1-4-1-3" then
25:
"Faons""Front"" < s//)),,,//2, Rx(-90%) (90%)))
26:
end
27:
if plane_naree E 1"1-4-1-4", "1-4-1-5" then
28:
"lanks["Back")) + ((s/2, 8, 8/2), (90%) e R2 (90%)
29:
end if
30:
Form cube by:
31:
for all face name E planes do
face_name placement + plones (ammeccee_aaa]ae____________ name]
33:
square +FreeCAD makeP (a, placement) mapiface, name] if rot is true then
35:
36:
Assign rotate(e, 90%) to square at placement else
37:
38:
Assign c to square at placement
39:
end if
40:
end for
41:
img Freeeeee www.:/...aaaaewwweeeaeo.a
42:
return img
43:
end function
function DRAWNET(net, map, s, rot)
45:
for face_nand pacee E nel do i, +- nellfare nelefarennamee naccee_nameea_ pos < 88 (H - i) 0)
47:
48:
square F www.maellaaaa waw.mmaaeellasss,....mmaa s, (pos, 0%))
49:
c mmapfaac__name] name]
50:
rot true then
51:
Assign rotate(c, 90%) square pos else
52:
53:
Assign e square at pos
54:
end if
end for
56:
img +- FreeCAD Savemggee)
return img
58:
end function
28

---

Published as a conference paper at ICLR 2026
Algorithm 9 Functions for Unfolding Cube to 11 kinds of Cube Nets
1: Using the same parameter definitions as those in Alggorithm 8 function DrwwNErWiPPvoryllann..cco
2:
nel, map, 8, rol)
3:
mame <- "1-4-1-0"
4: Initialize rotation dictionary planes
5: if plane_nanan nome |"1---1-1", "1-4-1-4", "2-3-1-1", "2-2-2") then
6: planes["Front" <- (909)
7:
else if plane_name --1-4-1-2", "1-4-1-5", "2-3-1-2", "3-3"] then
8: planes[ <- R R=100)))
9: else if plane_namee "1-4-1-3" then
10:
planes["Froni < (2
end
12:
if plane
plane_name name E -"1-4-1-4, ""144-1-4","1"44--""""" "1-4-1-5"] then ( (-90)))
planes[ end if
14:
if plane plane_name name E 1'2-3-1-0", "2-3-1-1", "2-3-1-2", "3-3", "2-2-2") "2--22" then
16:
planes["Left"" -- (-90))))
17:
end
18:
if plane_name
nome is "2-2-2" then Mlanes""Top' < Rg (-90))
19:
20:
end if
Create which can form the same cube with pivot plane:
22:
for face_name E net do
f- facefaae__eeeee name| pos < 8, 8, (H square F-FreeCAD makem mankePlane(s, 8; (pos, 0%))
24:
i s. 0)
25:
26:
if rot is true then
27:
Assign rotate(c, 90%) to square at pos
28:
else
29:
Assign e to square at pos
30:
end if
31:
if plane_name mainnennmmmeeee + "1-4-1-0" then
32:
if face_name planes then
33:
rotation + wwwpppllolseeeeaannaae square.Placement.RReee Rotation rotation end if end
35:
37:
end for
38:
img f- FreeCAD ImagIm)
39:
end function
29

---

Published as a conference paper at ICLR 2026
Algorithm 10 Cube Unfolding Task
1: Input: Color(Pattern) set C, unit length 8, task mode m
2: Initialize 11 cube nets
nets: : {face_nene
(4.j))[ace_namee E {"Top", "Bottom", "Right, "Left", "Back", "Front"]]
Initialize empty lists posttite_samples, negples
4: map: {faace_ {faacenamee name c/c E C) Randomlly shuffle set C' and assign it to six faces
5: Randomly select a view E8 corner views a cube
6: mivol_nel_nanne
<- "1-4-1-0"
7: ref_img Neett
map, view, rot false)
8: for to do
9: plane_ none_nnme,, net + Randomly select net from nets
10: img <-rawwee Wwtiwiviooooooooo (plone_ name, net, map, 8, rot false)
11:
Append img to positive_sample if m "pattern" then
13:
Append img' to (opaiiottppleepnne name, net, map, 8, rol true)
irng' DDra-eettiiiiP
15:
end if
end for
17:
map' Fix the mapping of face_nane _ome_aameae view, and random shuffle the others for to 2 do
19:
plane_namee net +- Randomly select net from nets
img tDgg==-WeeeiiPP www.eeiiiiiooooeeeeeoeeeeeeeeeee name, net, map, 8, rot false)
21:
Append img to positive_sample
22: end for
23: map' Swapp the colors(patterss of a randomly selected face view with its opposite face
24: plane_name, net <- Randomly select net from nets
(plane_name sateee_mmeenece net, map', rot false) 26: Append img to srngative_sampllee
25: img -DrawNet
27: samples + (positive_samples, negative_samples)
Shuffle samples assign [A, B, C, D] and record answer
29: dala (reeat_dataare__ sumples, question, onswer_id _
Algorithm 11 Cube Reconstruction Task
Input: Color(Pattern) set C. unit length s, task mode
2: Initialize 11 cube nets
"Right,, "Left", "Back", "Front")] nets: {face_ {face_name: nome: (fce))lacee_nan)) E ("Top". "Bottom", "Right",
negative_sampnn 4: map {face name c/c E C} Randomll shuffle and assign six faces Randomly select net from nets
3: Initialize empty lists
5: net {0, 1 1
6: ref_img + Draawee (net, map, 8, rot false) and append img to positive_sample
7: for <
do
8: view +- Randomly select view from 8 comeer views of a cube
9: img <-Nett (nel, map, view, rol false)
10: Append img to positive_sampll
end for
12: for flip dir E ["horizontal", "vertical") do
Randomly choose sample from positive_sample 14: img + flipe flip(sample fip__ir)
Append img to negativeesamples
16: end for
17: samples + (positive coositive_samples.. negative negative_samples)
18: Shuffle samples to assign [A, B, C, D) and record answer íd
19: dala aataate_datarre __mng, samples, question, answer_ie _id)
30

---

Published as a conference paper at ICLR 2026
Algorithm 12 Cross-Section Task
1: Input: Number of objects reum, number of sections per mode k; whether rotate the slicing plane rol.
2: Initialize candidate objects list objects, empty list selected objects
Initialize empty lists
negative_ _aapleesss k, plane)
4: function (ETSECTTONSSCOmpoune
5: Initialize empty list imgs
6: Determine coordmin and coordmar from bounding box
7: step + (coordmar - coord /
8:
for i +- 1 to k do
9:
of fet + coordmin + x step
10:
normal_vector vector + unit vector normal to plane
11:
section <- FreeCAD, s ce(compound, normal_vector of off sse) Rotate section for better visualization
12:
13: img + FreeCAD. sava Iavaemagessectio and append img to imgs
end for
15: return imgs
end function
17:
function CorTorr oris, center)
18:
cris_vectorrooo t- Corresponding unit vector of aris
19:
plane t Parallel to azis
20:
for angle {45%, 135%) do
21:
arir aris_wector' +- rotate angle, plane) center
of fset section + F F-eeCDDD s slice(compound, aris_vecton of set)
23:
24:
Rotate section for better visualization
25:
img + FreeCAD, sava Imageesetaissii and append img to imgs
end for
27: return imgs
end function
29: selected -erjettddobjectstse Randomlyy select num objects from objects
30: Randomly assign sizes objects in selected _bbjcctosssssssseeeeee
31: compound +- Create objects in FreeCAD and compound objects
center Obtain the center of compound object
33: for plane E ""XY",XZ"' "YZ"} do
35: Append imgs to (compound, k, plane)
imgs + Sect sons
36: end for
37: if rol is true then
38:
for aris E ("x", "y", "z") do
39:
for angle E (45", 135") do imgs t- ee Append imgs positive_samples end for
40:
oris, center)
41:
42:
43:
end for 44: end if
compound' Randdomly alter the relative ratios objects in compound
46: imgs + Use any of the above approaches to obtain cross-sections of compound'
48: Append imgs to negative_samples
49: samples + (positive_samplees, regaatieesamplest
50: Shuffle samples to assign [A, B, C, D] and record answer __d
51: data create. wwwateeedttarreeeee_eeeeeeaaaaae data _img, samples, question, onswer id)
31

---

Published as a conference paper at ICLR 2026
Algorithm 13 Cube Counting Task
1: Input: Spatial size (X, Y, 2), cube size number of constraint views num
2: Initialize zero value 3D tensors placement [0]2x xxx, empty list cubes
3: Initialize empty list samples
4: function DETECTGRID((riee)
row_nut col_num) 5: contours <-iind contours in view
6: Initialize grid matrix of size row__aum x col_nvum
7:
for contour contours do
8:
(z. w, fi) + Bounding rectangle of contour row < y/ /h, col <- x/Ww
9:
10:
if row and col within bounds then grid[row]|[ool <-
12: end if
13: end for
14:
return grid
15:
end function
16:
function GETCUBEANSWER(fTom) top, left, num)
sum_froot col Column sums of front sum_top_cod + Column sums of top
18:
mar_2view mar 2view <- sum www__foot__eee col sum fomm_to_cot col
20:
min fin__vieww- t- cum(sum_top_en)
+ sum cam_ffont_col)
21:
if num = then
22:
return (mar__it 2vir__, min mrin_wview)
23:
end
24: sum sum_lef Lese_eeloooo < Column sums of left
25:
Initialize answer matrix with the same dimension top Hxw for row + to H do
26:
for col + to V do if top[row]]cco = then cossroow]col t
28:
colla_c__ooool,__ sum wwwssslmmmttteeeee left__oolffooul
30:
end if end for end for
31:
32:
33:
mar 3view + sum(ans)
34:
ham_top_rouut Row sums of top
35:
min_3view < (rax(sum(ssuu (ssu_tto__owwrr - sum_left_col), min__vvieu)
36:
return 1maz_3vieet min_3view)
37:
end function
38: Update placement, cubes with CreateCubes(X, Y, Z)
Update placement, cubes with Connect Colaeetlola Y)
40: (front_view, lop_view, lefl_viens +- SaveViews (cubes)
41: Tront_mas lop_mat, left_mal +- DetectGrialfront view), DetectGria(top view), Dettct&@iidllet..rcee
42: if nuu then
43:
ref_img (top_views front_vieu (max__vee view, min_view) + GetCubeenswerrfront top_mat, left_mat, 2)
44:
45: else if nam 3 then
46:
ref_img + (top_view, front_view, left,iew) (mar_vien min_view) < GetCubeenswer(Froll. top_mat, le left_mna,, 3)
47:
48: end if
49: samples Generate correct and incorrect nums based on the min_wiew maz_view range
Shuffle samples assign [A, B, C, D) and record answer
_immmmg 51: data cwweceeeeeeeeeee aatate_dattaae a(ref aamef samples, question, answer_ia iid)
32

---

end if Published as a conference paper at ICLR 2026
Algorithm 14 Functions for Splitting Cube Stack into Several Connected Parts
1: function GETNEIGHIORR((cuee_
cubes)
2: (x, 2) & cube_pos
3: Initialize empty list neighbours
4:
for dr E (-1100011 0, do
5:
for dy E (-1,0,1)) do ( (-1,0, do for dz E
6:
7:
if da) [dyl da then neighbor pos + (x + dæ, dy, 2 + dz)
6-connected neighborhood
8:
9:
if neighbor_pon pos cubes then Append neighbor _ootorrposstt to neighbours
10:
end if end if 12: end for
14: end for
15: end for
16: return neighbors
17:
end function
18:
function REGIONGROWING(eubes,, max_cubes)
Initialize empty set part, empty list queue
20:
start_ _orrt_pos_eeee + Randomly select a position from cubes and append start_ _os to queue while queue not empty and |part] < max_ _ubes do current_pos <- pop(queue, 0) if currenf_pos & part then
22:
24:
Add current_po to part neighbors + seetNeeggborrsuurrrr
25:
cubes)
26:
Extend 'n E neighbors & part) to queue
27: end
28:
end while
29:
return part
30:
end function
function ISCONTINUOUS(part)
32:
Initialize empty set part, empty list queue
start_pon post_pos____ t part(0) [0] and append start_pos to queue
34:
while queue not empty do current_pos + pop(gueue, 0) if current_pos visited then
35:
36:
37:
Add current_po to visited
38:
neighbors Get ghbors(eurreen_pp... part)
39:
Extend (n E neighbors n E part and n & visited] to queue end if
41: end while
42: return Whether visitedd] |part]
43:
end function
44: function SPLITCUBES(cubes, maaz_cubes, num_parts)
partl + Reg.ooooooiiww mar_raaaaoo mna__cuess) 46: if IsContinucus then
45:
47:
remaining +- Remove partl from cubes end if if
48:
49:
IsCont cenntiinouusrremmnn (remainnn)))))))) then if num__prts 2 then
51:
return sort ([partl remaining)) by size else numm_part 3 then
53:
Similarly find part2 from remaining cubes as above par13 f- Remove part2 from remaining return wowtt(prrrr.,....o.oo por(2, by size
54:
55:
end
57:
58:
end function
33

---

Published as a conference paper at ICLR 2026
Algorithm 15 Cube Assembly Task
1: Input: Spatial size (X, Y, Z), cube size s, number of splitting parts &
2: Initialize zero value 3D tensors placement & (0) [0)2xYXX, empty list cubes
3: Initialize empty lists ref jings, aosilive_sampllee negalive_sampll
4: function CREATTCCUBSPPRAAIIIXX Y, 2)
5: Initialize num
6: for - 0 to Y 1 do
7:
num randomInt.num, min(y + 2, X))
8:
for < 0 to num do CreateCube(R, y:
9:
10: end for
11: end for
12: for < 1 to Z - 2 do
13:
Initialize num for y to Y do num = randomInt. max(num, sum(placementfz 1[())))
14:
15:
for 0 num do CreateCube(r, y, z)
17:
end for end for end for
19:
21:
for
< to Y 1 do
for x < 0 to do
23:
With 50% probability CreateCube(r, y, Z -
24:
end for 25: end for
26:
end function
27: Update placement, cubes with CreateCubesPyramin Y, 2)
cubes img < FreeCAD Saaeemaae and append cubes img to ref imggs 29: parts t SplitCubesscubees max cubes, num cum_parts)
30: for part E parts(: -1] do
31:
part_img + FreeCAD save save Image(part) and append part_ima to ref ref_imgs
32: end for
33: part_ima
FreeCAD F -anoCDDD_avemmmgeeerrrralll]] and append part_ing to
positive_samples 34: for do
35: part' Randomly remove 1 cube from part[-1]
36:
part'_img negative_samples
FreeCAD. save Tmage( and append part_ing
end for
38: samples + (positive settiiveesampless,ee sengativessamplen
39: Shuffle samples to assign [A, B, C, D) and record onswer idd
40: data + arreate_dtaarre _imgg samples, question, answer_io id)
34

---

Published as a conference paper at ICLR 2026
Algorithm 16 Simulation for Arrow Moving
1: Class ArrowPath
2: Attributes:
3: W, H, k: Map width, height, and step count
4: mas_slep
min(x;)) y)
5: directions + ((0,1)(1 (000))((11000--)x111110))
b up, right, down, left
6: path: Initialize with empty list to record relative moving direction and steps
7: states: Initialize with empty list to record pos and orientation during transformation
8: function INTTIALIZESTATE
9: Reset path, states
orient id randomInt (0,33
11: pos {(r,,) 3)} +- Randomly select a position in the map
12: Append (orient_id, pos) to states
13:
end function
14: function TOTTRELAATVVDDEEECONNNOrnnnn
15:
forrardd &- directions (rrinlt
backward + (- forward(0], forward(1) left + directions (orient id 1) mod 4]
17:
right t directions (orient,/oorrtnr_nire +d+11 1) mod 4]
19:
return ('forrard".forwardd "backward" backwward "left"lleft, "right":right
20:
end function
21: function UPDATEORIENTID(Te. ddr, orient_id)
22:
if rel dér is "backward" then
23:
orient_id +- (orient_id + 2) mod 4 dir is "left" then 24: else if rel_dir
25:
orient_ id + (orient id - 1) mod 4
else if rel dir rel_dir is "right" then
27: orient_id id + (orient_id id + 1) mod 4
end
29: return orient_ia id
30:
end function
31: function Move(state, rel_dir, steps)
32:
pos, orient id +- state
33: move_dir <- GetRe veDi rect ionooriett_ddiifrreee
34:
new poe_ppos_e [pos[0] move (ir steps, pos[1] [1] move_dir(1] steps)
35:
if new_pos is invalid then return false
37:
end if
Append (rel_ (rel_dir, steps) to path
39:
Append (Ipcatt@rrienttdddelddi Id(relddirr,, orient_id), new_pos) to states
40:
return true
41:
end function
42:
function GENERATEPATHÜ' ond_sttte=NNne)
43:
for < to 1: do repeat
45:
Randomly select rel_dir E {"forward", "backward", "left", "right") steps randomInt maax_step
46:
47:
valid flag t- Move Move(states) - -1]. rel_dir, steps)
end state is not None and k then volid_flay +- valid_flag & state[-1] & end_state
49:
50:
end if until valid_ flag is true
51:
52:
end for
53:
end function
35

---

Published as a conference paper at ICLR 2026
Algorithm 17 Simulation for Arrows Moving
1: Class Arrow.Map(nnerii from Class ArrowPath)
2: Attributes:
3: colors: Color set
4: path: Initialize with empty list to record arrow position, relative moving direction and steps
5: states: Initialize with empty list to record map during transformation
6: function INTIIALIZESTATE
7: Initialize empty matrix state
8: for <-1 to do
9:
for <- 1 to W do
10:
With 50% probability:
Randomly select color E colors
12:
Randomly get orient_id t- randomInt (0,3) state[pos] Record color and orient_ pos(2; y) 14: end for
end for
16:
Append state to slales
17:
end function
18: function Move(state, arrow_pos, rel_dir, steps)
19:
curr_pos +- arrow_pos
20:
curr_orient_id, curr_color + state|z]]y]
move_dir +-
reniiiecurinnurrr...moo. connurrooriint_dee _rrioottiiirrree___eteee dirl
22:
new + [pos[0] [ + move dir(0) x steps, pos[1] 1] + move_ __r[[_drr]]]]]_ x steps)
if new _ltepppos___ invalid then
24:
return false
end
26: new _orienllidd 4- UwddateOrient.drreeeeee.ee orient_id)
27:
if new_pos- = curr surr_pos and new_orient id curr curr_oriend _orrrrreettteee _id then
28:
return false
29:
end
30:
Append arrow_por vel_dis steps path if stale moddeenew_poss is None then statelcurr_pos None
31:
32:
33:
else
target_color, karget_orient id + rongggot
35:
target gree_move__tte dir + movee dir
target_rel rel_diircttoons t- Get
37:
taget_rel_at siget_rel_dir-e +- Find (key jarget_rel_drrections
directions value target_move. dir}
38:
new cew__trrgetoorirne statelcurr_pon
<- wataattereeennn((taaeeaee pee_eett_ee_iirr _dir, target_orient_id,
39:
tattttt_rooole and new new_ler wargetrrgeeeooe largeetorriett____
40:
end
41:
state _aopuuppppp______ f- curr culo_color and curr purr_orient. id
42:
return true
43:
end function
function GENERATEPATHCK; end stote=None)
45:
for i t to do repeat
47:
Randomly select arrow_pos E {pos state[pos] is not None}
48:
Randomly select rel_dir ("forward", "backward", "left", "right") sleps <- randomInt [1,mar_slep
49:
50:
valid flag t- Move (state, arrow _osp rel_dir, steps) if end state is not None and = & then walid_flag &- valid_flag & state[-1] + end_state
51:
52:
53:
end if until valid_ is true
54:
55:
end for
56:
end function
36

---

Published as a conference paper at ICLR 2026
Algorithm 18 Arrow Moving Task in Easy Version
1: Input: Dimension of map (W,H), step count k.
2: Initialize empty lists prailive_samples negpleseessmmpes
3: Initialize arrow parow with dimension W x H
4: Initialize state with arrow prrow_poth.Tae Inth.Initial zeState() and record as initial state
5: Update path, states with arrow wwwwwwpprt..aeeeeeeeeee www....eeeeaeeeeatticccccc
6: Append path to positive_samples
7: ref_img -- drawddawaaaa sarra__mppstates|[] states|-t
8: Record and_sstte t states
9: From the same initial_sttod
10: for 4- to do
11: Update path' with arrow www.:...eeeatteeatttl.... sta_e)
12: Append poth' to negalive-samples
13: end for
samples + (positive sactiiieessappess..eee negative _amppiesssmmps
15:
Shuffle samples to assign [A, B, C, DJ and record onswer jid
data f- create _aaattd__atrreefeeeeeeee _img, samples, question, onswer id)
Algorithm 19 Arrow Moving Task Hard Version
1: Input: Dimension of map (W, H). step count k, task mode m
2: Initialize empty lists
negative paaplesssssesssseeeeeeeee 3: Initialize arrow m_mpppppp with dimension W x H
4: Initialize state with arrow perow_map.Ine map.Initiallzettttee and record as initial state
5: Update path, states with arrow_
6: Append path to posilive_samplee
7: if m = "state" then
8: ref_img- + draw www.ddrwwwwwataaeeeee map(states(0)
9: Append states[-1] to positive_sampleee
10: else if m = "path" then
11:
ref_img -arreddddadddddeeee (aw__aa(stttes()))pppppaaaaaap_ statel-1]]
draww
12:
Append path to positive_samples
end if
14: Record end_stale + states[-1]
15: From the same initial-ssan state
for 4- do
17: Update path', states' with arrow_rraaae httrooommp..eeeeeateaaattooeeo/e end_state)
18: if' m = "state" then
19:
Append states'[-1] to negative_sample
20:
else m "path" then
21: Append path' to Segative_samples
22:
end
23:
end for
samples + (positive (positivessappees negative_sanaeeeeeeee samples)
25:
Shuffle samples to assign [A, B, C, D) and record answer ji
data +- cweeeteedattrreeeeeeeeeeeeeeaeee (img samples, question, answer_ie id)
37

---

Published as a conference paper at ICLR 2026
Algorithm 20 Simulation for Block Moving
1: Class Block
2: Attributes:
3:
K.Y.Z,k Spatial size and step count
4: directions; directions
5: colors: Color set
6: jubes_infoe Initialize with empty list to record positions and colors cube objects
7: transformation: Initialize with empty list record transformations
8: function INTTIALIZESTATE
9: Update cubes with CreateCubest
10: Assign randomly selected colors to cubes and record their colors and positions in cubes_in in
end function
function HASSUPPORT(C; y/ 2) 13: if 2 0 or there is cube at (z, V, 2 1) then
14:
return true
15: end if
16: return flase
17:
end function
function DROPCUBES
19: Sort cubes info by of pos in ascending order
20:
for cube cubee_ii fileesinfo do (z.y,, Acquire position of cube from cubes_in, while HasSupport y, =) flase do
21:
Change the position of cube to (æ, v. 1) and update 24: end while
23:
25: end for
26:
end function
27:
function CHECKMoVE((rom
to_pos)
if (to_por pos invalid) or (HasSupport (to pos) is false) or (there no cube at from_ poo) or (there is no cube at to po_ and to popos is on top of from_pos pos) then
return false 30: end if
31: return true
32:
end function
33:
function MoVECUBE(From
to_pos)
34:
if there is no cube at to_pos then
35:
Update cubes_in fo with changing the position of cube at from__ps to to_pos
36:
else
Update cubes cubes_infe with swapping the cube from_pos and to_pos end if
38:
39:
DropCubes()
40:
Append (from_pos, to_pos from_pos) to trans transformation orrmaioo
41:
end function
42:
function for 6 to ls do
43:
44:
Initialize empty list possible_mooes for all cube E cubes in do for all direction directions do pamm_pos__ to_pos) is true then
46:
pos t The position of cube from pos + direction if CheckMove(from Append (from_pos, direction, to_pos) to possible_moves
48:
50:
end if end for end for
51:
52:
53:
Randomly select (from_pol pos, direction, to_pos) E possible_man Mroveuube(from_pol lo_pos)
54:
55:
end for
56:
end function
38

---

Published as a conference paper at ICLR 2026
Algorithm 21 Block Moving Task
1: Input: Spatial size (X, Y, Z), step count k
2: Initialize empty lists ref rirgs, posilive_samples, negalive_sample
3: Initialize block with size (X, Y, 2)
4: Initialize with book. Tni maliilli zeState() and record as initial cicioll_ubesiiii
5: img < FreeCAD save Image (initial _cteles__ueeeeee and append img to ref ref_imgs
6: Update trans comnsoormaion, cubes _in_f with block. titec..teeeeattttraaaffoommttttt
7: Append trons formation fomation to positive-samples samples
8: Record f'inal_cubes ceueecuueeuueeeeeeeeeee -n after transformation
9: img < FreeCAD saveImage (final (f and append img to ref_ _imgs
From the same initial ciattial_cube_jii_
11: for 4-- 1-to to 3 do
13: repeat
cubes innesssf'o'' with bbeeff.eenerateerraasfftttttto until cubes_in.fo * final_cubes_ii
Update
15: Append trans formation to negalive_samples
end for
17: samples + poositioe_sapplee negativnn negative_samplest
18: Shuffle samples assign [A, B, C, D] and record answer id
19: dala + cweateedaaaareee
samples, question, onswer_ _idd
C DATASET CHARACTERISTIC
Option Modality & Format A significant majority of questions (818) feature image-based options emphasize visual reasoning. The choice formats are intentionally varied, including standard A/B//DD choices (508 questions), options with A/B/C/AAlI three other options are incorrect' (310 questions), and unique text (242 questions) or numeric (120 questions) answers to prevent models from overfitting to a single question style. For the numeric answers, we additionally provide direct numerical responses, and in F.3 we present a comparative analysis of model performance across different question format.
The answer distribution well-balanced across options A (26.5%), B (27.5%), and C (28.5%). The lower frequency of option D (17.5%) is deliberate design choice enhance the rigor of the evaluation. For many complex tasks, option serves the distinct role of "All three other options are incorrect". This asymmetrical design is critical for two reasons. First, it acknowledges the difficulty of generating multiple high-quality distractors for complex 3D tasks, ensuring all visual options remain challenging. Second, it compels models the most similar". Instead, this approach demands eliminative reasoning, requiring the model to rule out every other option to prove genuine understanding of the spatial rules being tested.
Answer Distribution
beyond simple heuristics like "pick
39

---

Published as a conference paper at ICLR 2026
D DATA EXAMPLES
We present exemplars of varying difficulty levels for all tasks, with each sample containing an image, question, options, answer, and explanation.
Mental Rotation 2DRotation: Figure 8
3DRotation:
Figure 9
VieewProjection:
Figure 10
Mental Folding PaperFolding:
Figure 11
CubeUnfolding:
Figure 12
CubeReconstruuttoo::
ure 13
Visual Penetration CrossSection: Figure 14 CubeCounting: Higure 15
CubeAssembly: Figure 16
Mental Animation Arrow.oovnng Figure 17 BlockMoving:
Figure 18
MechanicalSystem: Fig
ure 19
2D Rotation Task-Levd
2D Rotation Tusk-Levil
Q uestion: jett image shows a colored goi with square marking one comer W hich grid can be obtained by rotating the left gnd only, without flipping or other changes? Please answer from options Choices:
The image shows colored grid with a zed square marking cne comner hoch grid can be obtained by rotating the left grid only, without flipping other changes?
Qusstion:
Please from ophins B, Choices:
A.A
A.A
B.
B.B
C.C
C.C
A. Uhen olber options are incurnet
D.ALl duee olher optinns incurect
Explanation:
Explanation:
A: Option A is correct becouse was obtained by
A: Option is incomect becanse was obtaiied by
rotating the original image 270 degrees.
rotating the oriiinal image 270 degrees and then flipping bortzontally.
B: Option B is incorrect because it obrained rotating the original image degress and flipping bortzontally.
Option
because obtatned
rotating original image degrees.
C: Opéion C is incorret bacause it was obtained by cutating the original image: 100 degness and then flipping it.
C: Option incoment becaese was obtained by
rutating the mmmtric pollers the image.
vertically.
Figure 8: 2D Rotation Task.
Rotation Task-L.vvll
3D Rotation Task-L
Question:
Question: The shows Uhe original cube slack made equal sized small cubes. Which of the options right
made equal sized small caless Wh Which the options right cannot ubtained by robsting original slark? Please unswer from opions A, B, or C Choices:
rotating the original cube slack? Please answer from ogtions A, B, C, Choicess
A.
A.A
All thme other options are incureett.
D. three other uptions are incormest
A nswwr::
Amswer: A
Explanation:
Explanation:
Oprion A is incorrest because the cube stock
A: A: cornet because the rube stack
mibting original degrees.
axis by
by rolaling original stack
270 degrzes.
B: Optica B is inccerect because the cube stack can be
B: Option is incomrett because it vertically
obtained by rotating the ortginal stack around the y-axis by
mitroged version the original cube stack.
90 degrees.
Option
obtained
C: Option is correct because was removing one small cabe from the original stack.
by
mmaving one
from original
Figure 9: 3D Rotation Task.
40

---

Published as a conference paper at ICLR 2026
Three Vinw Projection Task-1 evel Cuhes
Three-Vieee View Projection Leve CAD Model
The cube stank as made of equal sizeed small cubes, mostl grny with a few red ones. The top row shows its isametric front stack? Please scower from options A, B, C hoicess:
O Question:
Q Qesstion: The cow shows the isumetaic view (lefi) and the top new (ighht) of 3D model W Which image in the hothom the left view the model? Plezse options A,
the from left to night.
imagie
butlom
cube
Choices:
A.
B.
D
All thrse other options are incomect
A nswer:
Answer: C
Explanation:
Explanation:
A: Option A incoreest because the shows the lef
A: Option incureet. becouse the internal outlines
right the
missing.
B: Opticn B is inceerect because the shape matches
Option incomeet because the internal outlines
right view Lasteed of the left view, and the red cubes are not in the correct position.
829 missing.
C: Option incorrett because the red cubes contect position
Figure 10: Three-view Projection Task.
Paper Folding Task-L
Paper olldnng Task-Level
Paper I Folding Task-Level
Operation
Operation
Punch
Operation
Operation
Punch Holes
Operation
Operation
D
Q Quuestio::
The original papec 3 grid papee I'ee images in the top row show the results of
Question: The original poper is a 4-+4 grid paper.
Question: odiginal paper is 5x5 grid papec:
The images the top row show the results of
The images in the top row show the results of
conseculive lolding operaLions the grid paper. Folding operations include tolding along bortzontal, vertical, or 45-degrse direction. The tightmost image in the top row shows the result after punching holes the folded paper. lich paper aller untolding the punched paper? Black solid clrcles represent gid cells with holes, Please answer from options B, C,
conseculive Folding opecations Folding operations include folding along horizontal, vectical, or 55-degree direction. rightmost image in the top zow shows the result afher punching holes the fulded papee. W hich image
the grid paper.
consecutive lolding operalions
the grid paper.
Folding opecations include lolcing along bortzocial, vertical, or 55-degeee direction. The iig rightmest image in the top row shows the result efter panching holes in the tolded image represents appearance alt unfolding the punched paper? Black solad circles represent grid cells with holes. Please from options C,
hich peper
image represents appearacce
appearance the pager after unfolding the sucched paper? Black solid circles represent grdd cells with boles. Please enswer from options A, B, Choicess
Choices: A.A.
Choices: A.A
A.A
B
B.
B.
C.
C.C
C.C
A nosu er: C
Answer:
Answer:
Explanation:
Explanation:
Explanation:
Option incommet because extra holes
B. Option B incumest because extra holes
A. Option incoreet because holes missing.
appear
eppear
Option B incormet bocause holes in
C: Option C is incomeet bocause holes
B. Option incormet because holes that.
column are missing.
row 4 are missing.
should appear in row 4 appear in row
Oplion D is incorrect because holes thet.
D: Oplion D incorrest because holes that
D: Opppiio incourect because extra holes
should kppear appear
appear
appoor
appear
Figure 11: Paper Folding Task.
41

---

Published as a conference paper at ICLR 2026
Cube Unfolding Leve-
Cube Unfolding Task-LLvell
Unfolding Tack-Levvl
The left image shows colored cube from particular viewing angle. The options nets (unfolded patterus) folded upward form the cube. Which net, when folded, cannot farm the cube shown in the left image? Please
Question:
Question: The left image shows a cube with different patterns particular viewing angle. The options nets (unfolded potterne) the cuhe, which are folded upward to form the cube. Which net, when folded, form the cube shown the left image? Plense answer from options A, Cholees:
Question: The left image shows a cube with different pattems
faces from
faces from
are
particular viewing angle. The options (unfolded partems) the cahe, which are folded upward to form the cube. Which net, when folded, form cube shown the left imsaee? Please answer from options A, or Choices: A.
from options B,
Choices: A.A
A.
B.
D.D
D.D
13, D
Answer: C
Antwer:
Answer: B
Explanation:
Explanation:
Explanation:
Option A/T/R incorrect because
A:Oplion incurrett hocnuse the squares
A/C:D: Option A/C is incorrect hecause
this not could be valid mot for the given cubc,
with asymmettic pailums have boen rotated.
the rolative positions of three faves maich the
the positions red, pink, and blue malch the
B: Option B is incurrect bocnuse the
cuhe shown the left image.
shown ouhe.
soymmetric pattems
been rotated. C: Option C incurrecl bocause (wo Gives
faces have swapped positions, so cannol form the
Opticn B correct
Option curnect because this net
cannot be valid net for the given cube, as the
have swapped posilions.
cube shown the left imagee
positions of yellow and pink reverse..
D: Oplion correct because the rolative
posilions three the left imagee
Figure 12: Cube Unfolding Task.
Cube Reconstruction Task-LLevel
Cube Reconstruction T ovel
Cube Reconstruction Task-LLevel
Question: As shown, this is the not (unfolded pottom) cube, with faces colomd colors. The is cube, Which color face apposita to the green faoo? Please answer from options A, R, C, or D.
The left image shows the not (unfolded patherm) of cubn, with forns having different, pattems, upward to form cibe. From an axonometric (3D) viewing angle of the cube, which combination adjacent patterns possible see? Please answer from options B, C, or
Question:
The left image shows the net (unfolded pattrr) cube, with faces having different palierns, The upward to form a cube. From an aronometric (3D) viewing angile of the cube, which combination of acent patterss possible see? Please answer from options A, C, Choices: A.A
Question:
upward
folded
C hoirres:
Choices: A.A
A. yellow
B.
B.
C.All three other options are incocrect
C.C
C.C D.D
D. red
D.
A Answer:
Answer:
A nswerr A
Explanation:
Explanation:
Explanation:
A/B/C/D: ssuming bottom face is
sssuming bottom the cell the second zow of the net, and the right face is
A Assuiigg bottom face is first in the second zow of the net, and the right Face is the cel Its nghtt
first cell in the secoad zow of the net, then after folding, the front face is red, the back face is green, the left face blue, the right face is cyan, the top lace yellow, the bottom lace is pink.
the cell to its right,
Option incorret because is
Option corract because shows the
vectically minooced version of back top- right view.
back-top-riglt view.
B: Optin is incomret because it includes
B: Option 3 is inconect because includes
rolated non symmelicc faces.
noiz- symmetii faces.
Option
because it
C: Option C is correct because it shows the
horizonlally minored venson of the front-
froul- moul-beliom-- tight view.
beltom- right view.
D:
Option D incorrect because it
Optiinn D is incorrect because includes
borizontaaly mirrored version
log-
non sy mmetric
left view.
Figure 13: Cube Reconstruction Task.
42

---

Published as a conference paper at ICLR 2026
Cross-Section Task-Level
Cross-Section Task-Level
Cross-Section Task-Leves
The top row shows the combined shape Viewad from two different angles. The shape consista
Question:
The top row shows the combined shape vieeee from two different angles The shape consists triangular trustum, ey linder and cizcular frustum from top bottom. Whichh the following images cannot crees section of shape? Please answer from options A, B, C, D. Choices:
Question:
The top row shows combined shape viewed from two diffecent angles. The shape coosiits of squure
Question:
of square frustum. the following images cannot section of shape? Please answee from options A, B, C, D.
and cylinder from top bottom. Which the following images camoot cross-section the shape? Please answer from options B, C, Choices:
Choices:
B.B
B.
B.
C.C
C.C
C.
D
D.D
D.
Explanation:
Explanation:
Explanation:
A: Option A is incemtel because is the
A: Oplion A is incorrec because is the
A: Oplica incomect bes ause il the
crees section of the stape made by plane paraile.
cross- section of the shape made by plane parallel
cross-section, of the shape made by plane parallel
the XY
the
X
Oplion is incorvel because il is srsss-eetiion.oo of the stape made by a plane paralle.
Oplion B is cornect because
B: Oplion incomeet because it the
corvepooding cross- section does not match the
cross-sction of the shape made by plaze parallel
the XZ plane.
shape shown the reference
to XY plane.
Oplicn correct
Option
because
coresponding cross section does not match the
cross section made by plane perpendicular to
comesonniin seceiinction does not match
shape shown in the reterence image.
X2 plane and rotated. 45 degrees around the y-axts,
shage shown in the refermnce image. D: Optica incorret because it the of shape made by place parallel
Option D is iacoreec because is the
D: Oplion is incourect because is
section the stape by a plane paralle
cross section the X plane.
shape by place parallel
the XY plane.
to X plane.
Figure 14: Cross-sectionn Task.
Cube Counting Task-Level L
Cube Counting Task-L evel
C Cube C Counting Task-1 evel
Front
Top
Front
Top
Left
vidwg what minimum number of cubes required satisty the constraints shown in the images? Please from oplious A, C, or
Question: three what maximum number cubes required to salisly the coustraints shown in the images? Please from oplions A, C, or
many
cubes could be necded salitl the constraints shown ln the images? Please from oplions B, D.
Choices:
Choices:: A.
Choices:
A. 5
A.All thrse other options are incomret.
All three other ogtions are inconeet
B. 11
B.7
C.7
C.
C.
D.9
A usswr::
Answer:
Answer:
Explanation:
Explanation:
Explanation:
A/B/C/D: Given views, al least at cubes are required sabisfy the constmints.
A/B/C/D: Given three views, al least are required satisty the constraints.
A/B/C//D :Givn thrre views, at least 11 cabes most 12 cubes required satisfy the constraints
Figure 15: Cube Counting Task.
43

---

Published as a conference paper at ICLR 2026
Cube ssemmbly Task-Level
Cube A ssembbl Task-Leve
Q Question: The top left image shows the ortginal complete cube stock combining
Question: The left image shows the original complete formed cube stacks on night with one of options below. W hich ophion completes the original cube stack? Please answer from options A, B, C, or Choloes A.A
cobe sized cubes. be formed nigt 1) with of the options below: W Which uption completes the original cube stack? Please answer from options A, or D. Cholces:
cube stack made equal sized cubes. combrring
A.
C.
C.C
D. A.A thrse other options are incomrect
D.AJJ thrse other optices are incourect
A nsweer:
Answer: A
Explanation:
Explanation:
Option A/ll incoment bocause one missing nsalling an incervest cube stack shape.
B/C: Option
because
missing, reallnng an incoreet cube stack shape,
Figure 16: Cube Assembly Task.
Arzow Moving Task-1 lask-Lee evel
Arrow Moving Task-Level I(v1)
Arrow Moving Task- I(V2)
Question: In the diagram, the mol arrow is the the
The image shows the initial state. Arrow
Questton:
The left image shows the initial state, and right image shows the finel state. can move in four directions (forward, barksward, left, right), where "Forward' always relers the current direction the arow pointing. A fler each direction updated to tomptt posi tion empty, the arow con move there directly: otherwise, it needs to swap with the arow the movements should satisfy aforementioned requirements. Which the following paths can transform the ond from the unitial state the dimal state?
Question:
initial
the finel arow The aroo can move four directions (forvard, backward, left, right), where forward! always refers to the current direction the aroo is arow's movement.
in four dimetioos (forward,
backward, night), where forward' always refors to the curment dimection the amow pointing. Fler each movement, the aroo's direction updoted the direction movement. torpet pos tion is empty, arrove move there directiy; otherwise, needs swap with the aroow the target position, and both should satisfy
pointing ffer
movement, movement.
updated the
Which of the following paths can makee the arrow move from the starting position to the ending position? Please from ogrions B, D.
torget position, and both
aforementioned requiremeots, After
mansformatioss ((0, 1) Right. ul (2.22 2)
Forward. uniti-IfL 21 Left 2 units! which
optins can teached?
Please
options
answver from ogtions A, B, C, or...
C Choices:
Choices: A.A
Choices:
A. (Left umits)-(L (ogg)-(Leet unit)
A. ((1. Backoaard, unit)-((1, 2)
(Fowwrd. unill- (uill--BBadeen
Backward,
C. (.oowwdrd. ualde-Baacwwardd units)
C.C
B. (11, Left. unit)-((1, 1) Forward,
D. (Porward, Lmit--LLftt oft unit)
D.D
unit)
Auswer:
Answer:
C. C.1
unit)((1) 1) Left, uni)) ((1. 2) Forward, mit))((0.2 2)
Explanation:
Explanation:
A/B/C: Option A/B/C incomret because
C: Opton correct because the initial
Backward, mntt
the lattial anoow cannot be transformed into the tinal artow.
state can be transtommed into the target state.
Answer: A
A/B/D: Opion incorrect because
Explanation:
Option correc bec the initial
vans/ocmed
A. Oplica contec
Broow can be transoomed Lato final artow.
target state.
state can bansformed into the target state.
B/C/D. Option B/C/D is incormect because
the inilial state cannol be transfomeed inlo the stage.
Figure 17: Arrow Moving Task.
44

---

Published as a conference paper at ICLR 2026
Blocks Task-L
Blocks Moving Task-Level
Rotated
State
Rotated
The top images bottom row shows state atbe different views the final state after traneformation. During the transtomatoon process, blocks can move oue unit in any direction (forward, backward, left, nihht up. down), the position empry, the block block, they swap places.
Question:
of images bottom shows state cabe stack, different views the final state afte transfomationn During the transforaatio process, blocks can move one unit in any direction (forward, bac backward, left, right, down), the the black block, there directly: swap places.
Question:
them directly;
if target position almedd
the target position alrendy Blocks cannut Iloat in sir. If block is muved eway fiom position, any block above it will Jall down uniil reachlngg supporting surface. The xy2 are shown the diagramt. blook's coondinates (x.y2).
Blocks cannot float in the Il bluck is muved aw from position, any Rock above will fall down until meartling supporting surface. The and block's position be precisely adeetified using coordinates (x.y)). W Which sequeneos can change the cube stack from the iitial state to the Emnll state show in the diagram? Please answer from options B, C, or Choices:
shown the diagram
precisely idectified using following transfomation sequences can change the cabe stack from mitial statn to the final state shown in Uhe diegram? Please answer from options A, o.
following tranformation
Cholers:
y+
y y
1)
B. (1.0 y+ 12 1, y+ 0,
B. f1. x+ (1. y+
C. 22.22. 01 x- 12.0----00000 12
0. 0) - (1. y+ 1244--12000 12.0
0) (2. 0) (2.7--22.22)
D.10, 0)
(0. 11
Explanation:
Explanation:
B/C/D: Option BCD incorret berzuse the inttaal
A/B//D Option A/B/D incomeed becaze the initial
statee cannol be transoouee inlo the final state.
stole canmol be Laansforme into Uhe final state.
Figure 18: Block Moving Task.
Mechanical System Task-LLevel
Mechanical System Task-Lovel
Question: When the red shaft commected the green rod claclewise,
the the gear fixed concentric shaft, while yellow pink goors on their own shafts If the green gear rolaees clockowise the given vies, what ure the resuling mullons of the yellow piak gous? Please choose from options B, Choioes:
Question:
the contrally blue gear? Pleasee choose from options A, B, C,
Choliee::
Rotatas clockwise
A. Rotates clockwise, rotates clockonse
R. Rotates counterclockwrien
B. Rotates clockwise,, rotates coumarmioocwwiie
C. Dois not rolat
C. Rubates counbrcluckwwsse; rotates mountertlockwwsse
D. Translates the right
D. Rotates counterclockwnse, rotates clockrvise
Answer:
Explanation:
Explanation:
A/D/C/D: The gruen rod crives the blue year lu rokale
A/B/C/D The yullow gear dimelly meshues with
only in the clockwise direction.
green gear and lbus rolates the opposèle disection wlile
the pizk gear driven through two meshing steps and the direction
green
Figure 19: Mechanical System Task.
45

---

Published as a conference paper at ICLR 2026
E EVALUATION DETAILS
E. MODELS
For the Deepseek series, In.ern series, In.ernVL.3 series and SAIL-VL series, deployed these models on H100 servers and used the officially provided code to load the pre-trained models for inference. For all other models, we employed API calls through OpenAI's client service for inference. All closed-source models accessed via API in this study were used with specific, identifiable versions ensure consistency and reproducibility. Specifically, used the following model versions:
cpt-40-2024408-009 for GPT-4o
o1-2024-12-17for oo
-laude-3-5-sonness 20240620 for Claude-3.5-Somie
clande-3-7-ssome--2225222 for Claude-3. 3.ade-3.---onett
Semt:al-2.5-11aasshpeeeeeeee 17 for Gemini-2.5-flash
for Gemini-2.5-p
for Doubao- Saabao-----vision -5-viision-pro
gwen-vl-max-0400 for Swen-VVL-man
E.2 PROMPTS FOR RESPONSE GENERATION
We use the prompt template follows:
1) Original CoT Prompt A from DeepSeek-RR (DeepSeek-Al al. 2025): "You should first provide a reasoning process, then provide single option (A, B, C or D) as the final answer. The reasoning process and the answer are enclosed within <think>-/thinke and <answer></answer> tags, respectively, i.e., <think>reasoning process</think>, <answerannserrc/sswer. AnQuetion: <question here> cnAcoopiion A here> \nB.<option (ere>>,nC<opttinn here> .nD.ooption D here>\n"
2) Variant CoT Prompt B from EMMA et al. 2025 "Answer with the option's 's letter from the given choices and put the letter in one "A boxed". Please solve the problem step by step. Quuustion: <question here>\nA. AnA.coption A here> \nB.coption here> <oCtooiioo C here> > AnD. D here>\n"
3) Non-CoT Prompt:
"Answer with a single option letter (A, B, C, or D), enclosed within the canswer></answer> tag. For example: Aanswer>A//asswerr
Ensure that your output contains only the final answer, without any intermediate reasoning or additional content. \nuussion: <question here coption A Anre>AnB.ootiioo B here> \nC.coption C h \nD.coption here>>
E.3 ZERO-SHOT SETTING
Our decision to focus exclusively on the zero-shot evaluation setting is grounded in both method- ological precedent and practical considerations. This approach aligns with the standards set by many recent,
benchmark papers, such as Math- Vision (Wang et al. 2024). MM-IQ (Cai et al. 2025), and EMMA (Hao et al., 2025). all of which centered their evaluations on the zero-shot setting to assess novel reasoning capabilities. While we considered few-shot prompting, we concluded its utility is limited in our context of complex spatial reasoning. For these intricate visualization tasks, providing examples with only the final answer offers minimal effective guidance. On the other hand, creating effective chain-of socin-of-thous thought examples that include complete, multi-step reasoning would be prohibitively expensive for comprehensive benchmarking.
E.4 METHODS FOR ANSWER EXTRACTION
To ensure robust evaluation and minimize parsing errors, we employ a hierarchical, two-stage rule-based approach for answer extraction.
Stage 1: Coarse Extraction with Boundary Enforcement.
Adopting the strategy from MME-CoT (Jiang al. 2025), first attempt to locate the answer
46

---

Published as a conference paper at ICLR 2026
segment by scanning for comprehensive set of standard identifiers, including XML-style tags (e.g., <answer></aneeeeeeeeee answer>) and natural language markers (c.g.. "<answer>", "Answer:", "Final answer". "final answer", "Final Answer", "the answer is", "The answer is", "correct answer", "Correct answer", "Correct Answer", and "correct path"). The text following these markers isolated and truncated at the first subsequent period delimiter. Critically, to prevent false positives where common words starting with option letters (c.g., "All", "Backward") are mistakenly identified as answers, we enforce strict word boundary constraints. We utilize the regular expression Äb DD]) Ab accept only standalone option letters.
Stage 2: Prioritized Fine-Grained Matching.
In instances where the coarse extraction fails to yield valid option, we trigger secondary, high- precision extraction routine. This process iterates through prioritized list of compiled regular expression patterns designed to handle specific formatting variations (e.g., tagged encapsulated outputs, boxed answers) and semantic fallback structures. The patterns are applied in the following order:
CoT Prompt A with tags:
r " "<answer>'s (2P<value>> ,?) "
CoT Prompt with boxes:
r" r"AM1, 2/xxeed (?: (?:\itex) / rm) () (PP<value> [A-D])*
Other common answer formats:
r "Knsswwr>> lww.swerrrroootiooo..e. (?P<value> (A-D]) (?=</answer>")
r" (?: (tw.ec///coorrrrcc \stanswerls= (P:1s)) \s* (?roption\s=) ? (2P<value> (A-D]) \b"
r "ooptionss- (2P (2p<value>>>vvv [A-D]) \b"
t"choose\s+ (ww<value> (A-D]) \b"
This dual-layer approach ensures high recall for compliant responses while maintaining precision against hallucinated or verbose outputs. Even with these rules, 100% parsing success isn't guaran- teed, as models can still flexibly produce outputs in non-standard formats. For the purpose of our comparative analysis, we designate the baseline coarse extraction method (excluding strict boundary enforcement) as Extract Rule A, and the comprehensive dual-stage strategy described herein as Extract Rule B.
For multiple-choice questions, response is considered correct if and only if the extracted result contains exactly one uppercase option letter (A, B, C, or D) matching the standard answer. For non- choice questions, perform direct string matching between the extracted result and the reference answer. This hybrid rule-based evaluation ensures consistent and fair judgment across both option- based and open-form tasks.
E.5 HUMAN PERFORMANCE
To establish robust human baseline analogous the tested MILMs, we recruited 8 graduate students (4 Ph.D., M.S.; aged 22-27) from mechanical engineering and computer science. All participants possessed strong backgrounds in geometry and physics, confirmed through their academic curriculum, and reported familiarity with spatial reasoning tasks. This selection criterion was chosen because it mirrors the specialized knowledge domains inherent in the models' training data. Participants were compensated at the standard rate for graduate research assistants.
To ensure data quality and minimize the impact of cognitive fatigue and time constraints, we curated a representative subset of the benchmark for the evaluation. Specifically, we randomly sampled 6 problems from each of the task categories, resulting in a total of 72 problems per participant. Before commencing each task type, participants were briefed on the rules and completed several practice trials for familiarization. The evaluation protocol required participants to solve problems without the use of external aids (e.g., scratch paper, calculators), and they were allowed unlimited time per question. This approach was designed to emphasize and assess their intrinsic spatial visualization and mental manipulation capabilities, creating evaluation condition comparable assessing model's 's internal reasoning processes without external memory aids. The reported human performance the mean accuracy across all participants.
external
47

---

Published as a conference paper at ICLR 2026
E.6 ERROR ANALYSIS
E. MODEL SELECTION FOR DIRECT ANSWER (NON-COT) EVALUATION
Our Direct Answer evaluation tests model accuracy without induced reasoning chains. We excluded specific models based on 2 criteria:
1.
Paasoning-Centrii Architectures: Models explicitly designed for extended reasoning (e.g., ol, Gemini-2.5, Kimi-thinking thinkiggn Llama-4 series) were excluded, as inhibiting CoT contradicts their core design principles.
2. Instruction Adherence: Models unable to suppress reasoning traces despite strict formatting prompts (specifically VL3-2B) were excluded. This failure reflects a limitation in instruction following rather than reasoning capability.
Consequently, we retained only models capable of strictly adhering to the single-letter answer format. This exclusion iriteria-based format compliance rather than performance- the baseline remains representative and uninflated.
E.6.2 ERROR TYPES
Perceptual Error: Failure to perceive fundamental visual properties, such as color, shape, or pattern structures.
2. Spatial Transformation Error: Failure deduce correct spatial states after transformation. This includes:
(a) Rotation/Flippinng Errors angle or axis; confusing rotation with flipping.
(b) Folding/Unfonn Unfolding: Incorrect mapping between 2D nets and 3D cubes; confusing adjacent or opposite faces.
(c) Spatial Relationships: Misjudging object composition, internal structure, or occlusion.
Spatial Memorization Error: Forgetting or misremembering object positions or relationships acTOss sequence of operations.
4. Instruction Following Error: Misunderstanding textual instructions, such as task rules (e.g.. negation) or required output formats.
Methodological Error: Adopting flawed or suboptimal problem-solving strategy, such as using a rigid or unnecessarily complex reasoning path.
6. Calculation and Reasoning Error: Errors in non-spatial logic or mathematical calculations.
E.6.3 INTER-ANE AENORATTO AGREEMENT ANALYSIS
To ensure the reliability and reproducibility of our error taxonomy (detailed in Appendix E.6.2 we conducted a rigorous inter-annotator agreement study.
Table 4: Inter-Annotator Agreement, Cohen's 's i calculated via binary decomposition for multi-label error classification.
Category Perc.
Trans.
Meth.
Instr.
Memo.
Calc.
Avg.
Cohen's x
0.90
0.81
0.75
0.96
0.89
1.00
0.88
Methodology Since our error analysis involves multi-label classification task (i.e., a single failure case may stem from multiple error sources simultaneously), the traditional global Cohen's 's not directly applicable. Instead, adopted standard binary decomposition approach for multi-label agreement. Specifically, we decomposed the multi-label task into 6 independent binary classification tasks, treating each error category as "Yes/No" decision.
Calculation We randomly sampled 100 failure cases from the evaluation set. Two authors inde- pendently annotated these cases based the defined taxonomy. We then calculated Cohen's ' separately for each error category. The results, presented in Table demonstrate high reliability. The Methodological category showed substantial agreement (x 0.75), while all other categories
48

---

Published as a conference paper at ICLR 2026
achieved almost perfect agreement (k 0.81), with Calculation & Reasoning reaching perfect consensus (k = 1.00). The macroscopic average Cohen's K across all categories is 0.8847, indicating an almost perfect level of inter-annotator consistency.
F DETAILED RESULTS
In this section, we provide more evaluation results and test cases from Gemini-2.5-pro for each task.
F.1 INTRA-CATEGORY COMPARISONS ACROSS LEVELS
To provide deeper insight into the spatial visualization reasoning capabilities of Multi-modal Large Language Models (MLLMs), this section presents comprehensive experimental results that comple- ment the aggregate performance assessment in Section 4.2) This analysis details the accuracy of each evaluated model across the four core sub-abilities mental rotation, mental folding, visual penetra- tion, and mental unimation-defiie task type and difficulty level. This granular performance breakdown reveals specific strengths and weaknesses of the models when confronting various spatial reasoning challenges, offering targeted insights guide future model improvements.
the Spatial
benchmark, with results stratified by
F.1.1 MENTAL ROTATION
Table documents model performance on 3 sub-tasks within the mental rotation category- 2D Rotation (2DR), 3D Rotation (3DR), and 3-View Projection (3VP)-across different difficulty levels.
In the 2D Rotation (2DR) task, several models demonstrate foundational capabilities at Level O, with ol (72.5%) and Gemini-2.5-pon (62.5% achieving notable results. As difficulty increases Level 1, most models show performance decline, though leading models maintain relatively high accuracy (ol: 52.5%, Gemini-2.5-pre 42.5%).
For 3D Rotation (3DR), performance degradation with increased difficulty is more pronounced. At Level ol (42.5%) and Gemini-2.5 (45.0%) perform adequately, but their accuracies decrease substantially to 15.0% and 20.0%, respectively. at Level 1. Many open-source models perform at or below random chance (25%-30%) at this higher difficulty level, highlighting the challenge of mental rotation in complex 3D space.
Interestingly, the View Projection (3VP) task reveals a different pattern: when transitioning from Level 0 (cube stacks) to Level (DeepCAD engineering models), some top-tier models like ol (improving from 40.0% to 58.0%) and Gemini-2.5-pp (increasing from 28.0% to 66.0%) demonstrate enhanced performance. This suggests certain Level I image features may be more amenable to these models' processing mechanisms, despite the presumed increase in complexity. Nevertheless, many other models show decreased performance from Level 0 to Level in this sub-task. Overall, mental rotation tasks reveal a clear performance gradient across dimensions and geometric complexity while highlighting significant capability variations among model families.
F.1.2 MENTAL FOLDING
Table 8 documents model performance on 3 sub-tasks within the mental folding category-Paper Folding (PF), Cube Unfolding (CU), and Cube Reconstruction (CR) at varying difficulty levels. These tasks assess models' capacity for continuous reasoning and dynamic visualization of 3D information throughout transformation processes.
In the Paper Folding (PF) task, as folding steps and hole-punching complexity increase (Level 0 to Level 2), most models perform near random chance, indicating significant challenges tracking multi-step geometric operations and performing subsequent spatial reasoning.
The more complex Cube Unfolding (CU) and Cube Reconstruction (CR) tasks proved challenging for all models. These tasks require understanding the correspondence between 2D nets and 3D cubes, while also assessing the ability to mentally execute folding operations and continuously reason about transforming 3D structures. Even at Level 0, most models demonstrate low accuracy, often below random chance. In the CU task, Gemini-2. 2.5 2.5 scored 37.5% (LO), 27.5% (L1), and 30.0% (L2), while ol achieved 37.5% (L.0), 37.5% (L.I), and 27.5% (I.2).
49

---

Published as a conference paper at ICLR 2026
For CR, Gemini- 2.5-pro performed at 45.0% (LO), 10.0% (LI), and 35.0% (L2), and ol at 42.5% (LO), 12.5% (L1), and 25.0% (L2), both experiencing significant performance drops at Level 1. However, the surprising performance improvement at Level contradicts human intuition, as Level 2 patterns objectively more complex for humans. Analysis of sample solutions reveals that models approached these tasks by employing clear textual descriptions define patterns composed of differently colored dots, representing their positions in matrix form. Conversely, line patterns proved more challenging for models describe, and internal rotations could not be easily represented through matrix transposition operations, which This insight provides valuable direction for designing more challenging tests that effectively evaluate model limitations. The overall results reveal a severe deficiency in reasoning and visualization capabilities when finer-grained correspondence and transformation tracking are required. The introduction of asymmetric patterns further challenges models' ability to maintain precise visual perception and topologcclll understanding. These results highlight current MLLMs' core weaknesses in handling spatial tasks involving geometric correspondence, topological transformations, and dynamic 3D reasoning.
F.1.3 VISUAL PENETRATION
Table 9 documents model performance on 3 sub-tasks within the Visual Penetration category -Cross- Section (CS), Cube Counting (CC), and Cube Assembly (CA) at varying difficulty levels. This ability requires models to infer internal object structures from visible external features.
In the Cross-Section (CS) task, which requires models to visualize sectional shapes produced by cutting composite geometric solids with various planes, Gemini-2.5-pro and ol maintained relatively stable performance across Levels 0, I, and 2, while most other models performed near random chance.
For the Cube Counting (CC) task, increasing constraints from two-view (Level 0) to three-view (Level 1), and subsequently expanding spatial dimensions (Level 2), progressively challenged models' view integration and counting inference capabilities. Gemini-2.5 -po'' accuracy declined sharply from 80.0% (LO) to 52.5% (LI) and 32.5% (L2). Interestingly, ol's performance followed a pattern of 45.0% (L.0), 32.5% (L1), and 45.0% (L.2), recovering at Level 2 to match its Level 0 score. Most models struggled effectively integrate multi-view information in this task.
The Cube Assembly (CA) task, which assesses the ability to identify complementary parts forming a complete structure, showed increasing difficulty as structures enlarged and constituent parts increased (Level 0 to Level 1). For example, Gemini-2.5-pp 2.5r-'''s accuracy dropped from 45.0% (LO) to 27.5% (L1), and ol's from 35.0% (L.) 32.5% (L1). Collectively, these results reveal current models' limitations in inferring global internal structures and spatial occupancy from local surface information.
F.1.4 MENTAL ANIMATION
Table dodcumeetts model performance on 3 sub-tasks within the Mental Animation category-Arrow Moving (AM), Block Moving (BM), and Mechanical System (MS)-at varying difficulty levels. These tasks assess understanding of dynamic state changes and causal propagation among system components.
In the Arrow Moving (AM) task, which requires understanding ego-centric movement rules and tracking state changes, the transition from simple single-arrow movements (Level 0) to multi-arrow environments involving swaps (Level 1) increasingly challenges models' rule comprehension and state tracking. A notable performance disparity exists between dlosed-oource models (e.g.. Gemini-2.5-pru and ol) and open-source counterparts: the former maintain high accuracy across both difficulty levels (almost 100% accuracy by 2e-5ni-22.5-pro), while most open-source models perform significantly worse (near random), particularly in complex multi-arrow Level I scenarios. This suggests capability gap, potentially stemming from differences in architecture or training data, when precise instruction following and multi-step dynamic spatial reasoning required.
The Block Moving (BM) task combines directional movement with gravity simulation, increasing spatial complexity and operational sequence length, thereby challenging models' intuitive physics and 3D dynamic spatial reasoning. Gemini-2.5-pro. accuracy declined sharply from 95% to 35%, showing the difficulty in dealing with 3DD scene.
50

---

Published as a conference paper at ICLR 2026
For the Mechanical System (MS) task, which evaluates understanding of motion transmission and component linkage in complex mechanical systems, questions were designed to minimize reliance formal physics formulas while emphasizing comprehension through observation and spatial imagination. Interestingly, some open-source models performed better than expected based on their performance in other 3D imagination tasks. This suggests these models may transform such problems into more formalized reasoning processes similar to physical rule application, rather than relying solely on intuitive 3D mental simulation. While this strategy may yield relatively good scores in certain instances, it potentially deviates from the primary goal of assessing pure spatial visualization capabilities. Overall, mental animation tasks especialll those involving complex dynamic interactions and implicit physical laws-nontinue to pose significant challenges for current MLLMs, with models exhibiting considerable diversity performance strategies and capabilities.
F.2 STATISTICAL SIGNIFICANCE ANALYSIS OF COT PROMPTING
The impact is significantly positive for claude-3.. 3.5 3.5-sonnet but significantly negative for several leading open-source models.
Table 5: Statistical significance analysis of CoT prompting impact (p < 0.05).
Model
Source
CoT Impact
Significant 0.05)
p-valuc
Sami-VL-AABB-Istee
Open
Negative
Yes
0.0192
Crepseek-VL22-ingg
Open
Negative
Yes
0.0463
Internvi2.5-78%
Open
Negative
Yes
0.0368
Owen2.550mmni-7
Open
Negative
Yes
0.0216
Sail-VL VL-1.6-8B
Open
Negative
Yes
0.0479
Claude-3.5-sonnet
Closed
Positive
Yes
0.0007
F.3 PERFORMANCE COMPARISON BETWEEN DIFFERENT QUESTION FORMAT
This benchmark primarily uses MCAs, a deliberate and justified design choice. MCAs are particularly effective for tasks with complex answers (e.g., 3D Rotation Task) that difficult to express textually or match automatically. Moreover, well-crafted distractors can increase task difficulty and test model's fine-grained discrimination.
Our rationale for using the MCA format is threefold:
MCAs align with human qualitative intuition. Humans often rely on estimation rather than precise calculation in spatial reasoning. This format assesses a model's grasp of core transformation logic ("qualitatively correct" reasoning) without penalizing minor deviations.
Converting some tasks to a direct-answer format is technically challenging. For instance, in 3D Rotation and Paper Folding, the answers complex images. Requiring models generate these images is a frontier research problem beyond the scope of current multimodal evaluation.
We quantitatively measured the difficulty gap. When the Cube Counting task was converted fill-intheebbann format, all models showed significant performance drop. As shown in Table 6. GPT-4o's accuracy dropped by 32.50%, while even the top-performing Gemini-2.55pro's declined by 14.17%. This indicates the direct-answer format is independent reasoning, even with options like "All three other options are incorrect" to reduce guessing. Consequently, for a comprehensive assessment, we provide both formats for the Cube Counting and parts of the Cube Reconstruction tasks. This performance gap demonstrates that MCA options provide clues or "error-correction" " opportunities, helping models select best- fit answer. In contrast, the direct-answer format more authentically exposes deficits in precise reasoning.
demanding of model's
51

---

Published as a conference paper at ICLR 2026
Table 6: Performance Drop on Cube Counting: Multiple-Choice vs. Fill-intthe-Blank, The "Perfor- mance Drop" column quantifies the accuracy degradation when switching from the discriminative (Multiple-Choie) to the more challenging generative (Fill-in-the- Blank) task format.
Multiple- (%)
Fill-i-tth--aank
(%)
Avg Performance
Model
L00
L2
L0
L1
L2
Drop (%)
Open Source Models
Owen2.5-VI-7BB. VI--7B-Isttmet
32.50
50.00
27.50
15.00
2.50
0.00
-30,83
Owen2.5-VL--22--meee
32.50
50.00
42.50
25.00
32.50
5.00
-20.83
Cloxed Source Models
GPT-4o
40.00
45.00
37.50
10.00
12.50
2,50
-32.50
45.00
32.50
45.00
20.51
22.50
10.00
-23.16
Geminn-2.55pro
80.00
52,50
32.50
55.00
52.50
15.00
-14.17
Table 7: Comparison of model performances Mental Rotation tasks. The first and second highest accuracy of MLLMs are marked in red and blue, with open open-source and closed-source models marked separately.
2DRotation
3DRotation
3VjewPronontton
Model
Overall
L.0
L1
Avg
L.0
L1
Avg
L.0
Avg
Human
92.50
87.50
90.00
83.33
75.00
79.17
91.67
83.33
Random
27.69
25.00
22.50
23.75
25.00
30.00
27.50
30.00
32.00
31.00
Open Source MLLMs
SAIL-VL--1.522
22.31
20.00
25.00
22.50
17.50
27.50
22.50
20.00
24.00
22.00
IntemVL3-28
27.31
12.50
20.00
16.25
32.50
35.00
33.75
24.00
38.00
31.00
Byppeckk-L2--iny3BB
10.00
17.50
20.00
22.50
22.00
32.00
D.v.2..5VVL.3B.
VI.3B Instruct
20.00
25.00
15.00
20.00
15.00
22.50
18.75
16.00
26.00
21.00
Qwen2.5-VL-7B-Instruc
23.85
25.00
25.00
25.00
20.00
12:50
16.25
14.00
44.00
29.00
Qwen2.5-Ommi-78
24.23
32.50
12.50
22.50
25.00
15.00
20.00
22.00
36.00
29.00
SAIL-VL-L66-B IntemVVL3-85
28.85
22.50
17.50
20.00
35.00
42.50
38.75
18.00
38.00
28.00
Kimi VL-ABB Instruct(16B)
28.08
15.00
17.50
16.25
32.50
27.50
30.00
24.00
48.00
36.00
Ximi-VL-A33B-hinninngl00
20.00
10.00
17.50
13.75
17.50
22.50
20.00
20.00
30.00
25.00
Deepeck-V[2-small((6))))
24.62
40.00
22.50
31.25
10.00
22.50
16.25
22.00
30.00
26.00
328
Deepseek- VL2(2BB)
29.62
20.00
30.00
25.00
35.00
32.50
33.75
20.00
40.00
30.00
Qwen2.5-VL-32B--nstuu IntemVLL3-38B
35.00
35.00
27.50
31.25
32.50
37.50
35.00
22.00
54,00
38.00
28.46
25.00
20.00
22.50
32.50
35.00
33.75
22.00
36.00
29.00
72B
Qwen2.5-VLL777 20..5--V--72B--mme- QuQ-72B-preview
29.23
25.00
32.50
28.75
40.00
22.50
31.25
22.00
34,00
28.00
15.00
27.50
21.25
27.
32.50
30.00
30.00
InterVVL3-78B
28.46
20.00
30.00
25.00
25.00
25.00
25.00
20.00
48.00
34.00
108B
Llama-t-Maverick-178 178-128--nnttruttt
25.00
15.00
LLaama 4 Scout- 17B- 16E. Instruct
37.31
32.50
32.50
32.50
32.50
37.50
35.00
28.00
58.00
43.00
Closed Source MILMS
GPT-4o
31.15
20.00
45.00
32.50
30.00
25.00
27.50
20.00
46.00
33.00
46.92
72.50
$2.50
62.50
42.50
15.00
28.75 75
40.00
58.00
49.00
Claude-3.5-oonet
Claude-3.7
38.08
40.00
25.00
32.50
40.00
32.50
36.25
34.00
$4.00
44.00
Gemini 2.5-flash
Gemini 2.5 pro
44,23
62.50
42.50
52.50
45.00
20.00
32.50
28.00
66.00
47.00
Doubao-1 Douhao-1-S-vision- J-5-vision-pro
7.50
7.50
7.50
00
62.00
Qwen- VL-max
28.08
12.50
35.00
23.75
30.00
22.50
26.25
22.00
44.00
33.00
52

---

Published as a conference paper at ICLR 2026
Table 8: Comparison of model performances on Mental Folding tasks.
PaperPooding
CabeUnfooding
CubeReconttrutton
Model
Overall
1.0
L1
1.2
Avg
1.0
1.1
1.2
Avg
LD
L1
1.2
Avg
Human
80.56
100.00
93.75
87.50
93.75
87.50
75.00
62.50
75.00
81.25
75.00
62.50
72.92
Random
21.67
17.50
20.00
20,00
19.17
15.00
27.50
17.50
20000
30.0
2500
22,50
25.83
Open Source
SAIL -1-1--V1-
22.50
12.50
25.00
22.50
20.00
30.00
27.50
25.00
27.50
22.50
2000
17.50
20.00
Intem VL3-2B
24.44
25.00
27.50
15.00
22.50
35.00
12.50
30.00
25.83
35.00
22.50
17.50
25.00
Deepseck-Vi. (122))n)(3))
20.56
27.50
17.50
20.00
21.67
20.00
25,00
17.50
20.83
15,00
20.00
22,50
19.17
24.17
20.00
37.50
17.50
25.00
25.00
25,00
27.50
25.83
25.00
32.50
7.50
21.67
D-en2.5-VL-7B--mme
Qwen2.5- 5-0mii-78
24.17
27.50
30.00
17.50
25.00
32.50
37.50
12.50
27.50
17.50
27.50
15.00
20.00
SAIL-VL-1.6-BB
23.89
35.00
17.50
32.50
28.33
25.00
30.00
20.00
25.00
17.50
25.00
12.50
18.33
IntemV1.3388
25.56
25.00
20.00
40.00
28.33
25.00
20.00
25.00
23.33
25.00
27.50
22.50
25.00
16B
Kimi VL-A3B Instruct((8B)
24.17
27.50
22.50
27.50
25.83
22.50
15.00
22.50
20.00
15,00
27.50
37.50
26.67
Kimi VL-A3B shinking
24.72
10.00
25.00
35.00
23.33
20.00
20.00
32.50
35,00
17.50
27.50
26.67
Derpsook VL2-smalk(16))
Deepseck-VL.22 V122(77))
26.39
22.50
35.00
37.50
31.66
32.50
15.00
27.50
25.00
17.50
30.00
20.00
22.50
Qwen2 1onn22.-VL-3B--Insnme
24.72
15.00
37.50
12.50
21.67
17.50
35.00
22.50
25.00
30.00
10.00
42.50
27.50
Intera IntemVL33 VL3 38B
26.94
22.90
20.00
20.00
20.83
25.00
35.00
27.50
29.17
22.50
32.50
37.50
30.83
72B
Qwen2.5-VV --niseettt
QvQ-72B preview
108B
Llama- M-aeer---Mericc---- Benewrick-----1222 778-128E-Instrun
25.00
15.00
17.50
17.50
16.67
30.00
25.00
32.50
29.17
30.00
32.50
35.00
29.17
28.61
15.00
17.50
17.50
16.67
35.00
32.50
30.00
32.50
42.50
32.50
35.00
36.67
Closed Source
GPT-4o
25.00
35.00
25.00
C..ome-..-woomett 5-somnet
25.00
7.50
20.00
20.83
25.00
17.50
25.00
32.50
Cludee 3.7oomc. 7-sonnet
24.72
20.00
20.00
15.00
18.33
32.50
25.00
22.50
32.50
17.50
37.50
29.17
Gemini-2, f.5-lahhh
32.50
15.00
37.50
27.50
26.67
32.50
30.00
27.50
30.00
55,00
27.50
40.00
40.83
Seminii2,55-jro 2,5-pro
35.00
57.50
40.00
32.50
43.33
37.50
27.50
30.00
31.67
45.00
10.00
35.00
30,00
Doubao-F 1-5- --visioo-pro
28.06
25.00
37.50
32.50
31.67
22.50
22.50
25.00
23.33
45.00
17.50
25.00
29.17
Qwen- VL-max
24.44
27.50
25.00
20.00
24.17
12.50
15,00
25.00
17.50
42.50
22.50
30.00
31.67
53

---

Published as a conference paper at ICLR 2026
Table Comparison model performances on Visual Penetration tasks.
CrossSection
CubeCounting
CubeAssembly
Model
L0
Avg
Arg
Avg
Human
75.42
75.00
75.00
68.75
72.92
81.25
75.00
56.25
70.83
87.50
75.00
82.50
Random
28.12
32.50
27.50
30.00
30.00
30,00
20.00
25.00
25.00
22.50
37.50
30.00
Open Source
3B
-ALL-VL-1.5-2B
Ia3emVL332B
26.56
22.50
22.50
15.00
20.00
22.50
32.50
37.50
30.83
27.50
32.50
30.00
Deepieak-VI V1. V1.2-inn((R)
20.94
17.50
25.00
20.00
20.83
25.00
25.00
17.50
22.50
17.50
20.00
18.75
Qwen2. Qwen2.5.VI..33 5-l-3-3--
25.94
25.00
25.00
27.50
25.83
35.00
23.33
30.00
30.00
30.00
Swem2.5-VL-7B-Insse
27.19
12.50
12:50
25.00
16.67
32.50
50.00
27.50
36.67
35,00
22.50
28.75
Qwen2.5-Ommi-77
27.50
SAIL-VL-LL6-8B
21.25
17.50
22.50
25.00
21.67
22.50
17.50
17.50
19.17
30.00
17.50
23.75
30.94
17.50
15.00
15.00
15.83
25.00
45.00
52,50
40.83
45.00
32.50
38.75
Kimi-VI VI-AA3BB-nssttcr(11f5
17.19
17.50
25.00
22.50
21.67
7.50
2.50
5.00
5.00
27.50
30.00
28.75
Kimi- VL-AAB--hisskiigg166
29.38
27.50
17.50
30.00
25.00
45,00
40.00
25.00
36.67
20,00
30.00
25.00
Deepseek VL2-mmal(16)))
32.50
32M
Kimi-VLa VL--VVL-ABBBB VL-AA-----nIitttr11333
17.19
17.50
25.00
22.50
21.67
7.50
2.50
5.00
5.00
27.50
30.00
28.75
Deepeesk- VI 2-2smll((16))
25.31
7.50
12.50
7.50
9.17
30.00
32.50
42.50
35.00
30.00
40.00
35.00
72R
Owen2.5-VL-72. -i---i22--
39.06
27.50
40.00
22.50
30.00
32.50
50.00
42.50
41.67
55.00
42.50
48.75
QvQ-T2B-proone
27.81
32.50
30.00
27.50
30.00
35.00
25.00
7.50
22.50
40.00
25.00
32.50
Intern V1.37888
35.00
17.50
25.00
20.00
20.83
52.50
30.00
40.00
42.50
55.00
48.75
108B
Llama-4 Miveerick- 1ww.errickk1...A8-nnnnntte Insttrrttttt
32,19
27.50
15.00
15.00
19.17
27.50
47.50
30.00
35.00
52.50
42.50
47,50
34.06
17.50
17.50
17.50
17.50
35.00
47.50
30.00
37.50
50.00
$7.50
53.75
Closed Source
GPT-4o
32,50
25.00
25.00
7.50
19.17
40.00
45.00
37.50
40.83
52.50
27.50
40.00
32.50
33.75
Claosle3..-sommer
33.44
35.00
20.00
12.50
22.50
35.00
45.00
27.50
35.83
47.50
45.00
46.25
Claude-3.7 3.7-soneet
31.56
20.00
35.00
17.50
24.17
30,00
32.50
30.00
30.83
40.00
47.50
43.75
Gemini-22.5faan S-flahhh
32.81
32.50
35.00
22.50
30.00
52.50
32.50
30.00
38.33
30.00
27.50
28.75
Gemini-2.5-pp -.5
42.19
32.50
35.00
32.50
33.33
80.00
52.50
32.50
55.00
45.00
27.50
36.25
Doubso-1-S-ai --siiiioo-ppro
39.69
35.00
30.00
25.00
30.00
62.50
65.00
40.00
55.83
42.50
17.50
30.00
Qwen- -m-max
38.44
32.50
20.00
27.50
26.67
62.50
22.50
47.50
50.00
35.00
42.50
54

---

Published as a conference paper at ICLR 2026
Table 10: Comparison of model performances on Mental Animation tasks.
ArrowMoving
BlockMoving
MechanicalSystem
Model
Overall
L0
L1
Avg
10
L1
Avg
L0
L1
Avg
Human
88.33
92.50
87.5
90.00
95.83
79.16
87.5
87.50
87.50
87.50
Random
23.33
32.50
25.00
28.75
10.00
22.50
16.25
30.00
20.00
25.00
Open Source
3B
SAII-VI--155228
24.58
15.00
27.50
21.25
22.50
27.50
25.00
35.00
20.00
27.50
InternVL33-28
22.50
15.00
18.75
27.50
32.50
25.00
35.00
Despsek-VLL2-tnyye VL2-tiny(3B)
21.67
25.00
12.50
18.75
25.00
17.50
21.25
25.00
25.00
25.00
Qwen2.5-VL-3B-Ime
35.83
35.00
35.00
35.00
32.50
27.50
30.00
$7.50
27.50
42.50
7B
Qwen2.5-VL-7P
Owen2.5-VL-7B-Insstrut
32.50
22.50
22.50
22.50
25.00
23.75
67.50
35.00
51.25
Qwen2.5-0mmi-78
35.42
27.50
35.00
31.25
32.50
27.50
30.00
67.50
22.50
45.00
SAII-V1L-166-8
35.00
12,50
37.50
25,00
37,50
32.50
35.00
52.50
37.50
45.00
IntemnLL3-8B
37.08
30.00
30.00
30.00
30.00
30.00
30.00
62.50
40.00
51.25
27.92
17.50
12.50
15.00
35.00
31,25
57.50
17.50
37.50
40..2
22.50
37.50
30.00
35.00
52.50
43.75
62.50
32.50
47.50
Deepseek-VV.. V1.2-small(16B)
26.25
25.00
27.50
26.25
25.00
22.50
23.75
47.50
10.00
28.75
32B
Despsck-VL2((7B)
VL2(27B)
29.17
20.00
32.50
26.25
35.00
25.00
30.00
40.00
22.50
31.25
Qwen2.5-VL-32B-Inssi
22.50
35.00
28.75
27.50
27.50
27.50
62.50
47.50
55.00
VL3-38B
37.08
25.00
25.00
25,00
25.00
35.00
30.00
65.00
47.50
56.25
72B
Owem2.5-VV-772--nnsmt
43.75
27.50
27.50
27.50
45.00
35.00
40.00
67.50
60.00
63.75
QvQ-77BB-preveww IntemVVL3-78
25.00
35.42
25.00
22.50
23.75
47.50
41.25
55.00
27.50
41.25
108B
Llama Maveerick- 17B- 178-1120 28E--Instrue
39.17
35.00
35.00
35.00
40.00
40.00
40.00
45.00
40.00
42,50
amam--4--9 4-outtt 17B-1678666 (7B-16F-Inan Instruct
39.58
15.00
42.50
28.75
47.50
32.50
40.00
57.50
42.50
50.00
Closed Source
GPT-4o ol
38.33
32.50
12.50
22.50
25.00
40.00
32.50
62.50
57.50
60.00
57.50
75.00
60.00
67.50
50.00
55.00
52.50
62.50
42.50
52.50
Claude33.-sommet
40.42
42.50
32.50
37.50
25.00
37.50
31.25
$7.50
47.50
52.50
Claude-3.7-somee
72.50
60.00
66.25
25.00
32.50
28.75
55.00
32.50
43.75
Gemini-2.5-flush
50.00
82.50
52.50
67.50
32.50
35.00
33.75
62.50
35.00
48.75
Gemini-2.5-po
62.92
97.50
92.50
95.00
27.50
42.50
35.00
75.00
42.50
58.75
Doubao-I-5-visionn 1-5-vision-pro
35.83
22.50
22.50
22,50
32.50
42.50
37.50
$7.50
37.50
47.50
Qwen-VL-mmx VL-mmax
39.17
25.00
27.50
26.25
35.00
37.50
36.25
45.00
55.00
55

---

Published as a conference paper at ICLR 2026
F.4 TEST CASES
This section presents test cases from Gemini-2.5-ppm across various tasks, helping us identify error sources and provide direction for model improvement.
Mental Rotation-2D Rotation-Level
Prompt: You should first provide reasoning process, then provide single option(A, B, the final The reasoning piocess and the answer are enclosed within <think></thinke> and <answer><(answer>> tags, respectively, Lee, <hhinkvrasoring proces-/thinko,, <answerrannwrraansvee
Question:
The left image shows pattermed grid with red square marking one comer: Which grid can obtained by rotating the from options A. D.
left grid only? Please
Choices:
Original
A.A B,
C All three other options are incorrect
Ground Truth Answer:
Thinking Process:
1. nalyze the Original Grid:
The grid is 3x3. Note the poltern cach call and the position of the red square marker the bottom- left corcer
Top row: Blank, Diagonal, Blank
Middle row: Circle, -saaae,, Diagonal
Bottom Vertical lines, Blank, Diagonal
Red square: Bottom-left.
A malyye ze O Option A:
Wrong incorrect pattern details
The grid pattem A identical to the Original grid. The red square also in the bottom-left This corresoonds
degree (or 360-66600000 360-degree) rotation. While technically rotation, it doesn't 't change the orientation.
3. nalyze Option B:
square is in the top- right corner. This position corresponds to a 180-degree rotation from original bottom- leet position. Let's check the pattern matches 180-degree rotation: Rotating Original grid 180 degrees means the cell (row,
col) moves to ro-rowtl, 3-col+1) (4-row, 4-col), and the content of the coll is also rolated 180 degrees.
Original (3.1) [Vertical lines] Rotated (1,3) [Vertical lines] (Matches
Original (3,2) [Blank] Rotatod (1,2) [Blank] (Matches
Original (3.3) [Diagonal Rotatee
Rotated (1,1) [Diagonal ] (Matches
Original (2,1) [Circle] -2 Rotated (2,3) [Cirdle (Matches B)
Original (2,2) [X [X-saape) Rotated (2,2) -happp (Matches
Original (2,3) [Diagonal -- Rotatot (2, [Diagonal (Matches
Original (1,1) [B [Blank] Rotatod (3.3) [Blank] (Matches B)
Original (1,2) [Diagonal 1-> Rotated (3,2) [Diagonal (Matches B)
Original (1.3) [Blank] -> Rotated (3.1) [Blank] (Matches
The patterm in B perfectly matches a 180-degree rotatoon of the Original grid.
nalyee Option C:
square is in the top- right corner, also suggesting a 180-degree rotation. However, pattern does not match expected 180 d0--dgrree totation detived above. For example, the top left cell C is Blank, but the expocted call Diagonal The
top right cell in C has Horizontal lines, but the expected cell has Vertical lines. Grid C not rotation of the Original.
Conclusion:
rotation. The question asks which grid can be obtained by rotating. Both 0-degree and 180 degree rotations are rotations. However, usually, such questions imply change in orientation (non z0r0 rotation). Grid B represents a non trivial rotation (180 dogroes) of the original grid.
Grid
O-degree rotation (identical), and Grid B d80-de rotation. Grid
Final Answer:
Figure 20: Test case of 2D Rotation Task from Gemini-2.5-pro..
56

---

Published as a conference paper at ICLR 2026
Mental Rotation-3D Rotation-Level 0
Prompt: You provide a reasoning process, provide a single option( A, B, or as the answer The
reasoning process and the answer enelosed within <tink-<</hink>> and <unswer>=</nsswr> tags, respctivvlyy <thinl-reasoning process</hinkk> Sanswrr-answerr//nswwee
Question: The image shows original cube slack made of equal-sized small cubes. Which the options on the right cammot obtained by rotating the original cube stack? Please from options A,
Choices: A.A. B. B
Original
options are
Ground Truth Answer:
Thinking Process: - Confusing description
Anaalyye Original Shape:
The original shape consists of equal-sized cubes. Visual inspection suggests central cube connected four other cubes. Given the 3D isometric view, it's reasonable to ussume the central cube (which might be hidden) has cubes attached to four of its faces. assume the structure central
(N), South (S), East (E), Top (T), This structure, JC, S, E, TJ, inherently 3-dimensional (nom nomm-aanar). The central cube C has connections (neighbours), and the other cubes (N, S, E. T) each have only connection (to C).
neighbours
Ananly Shape
Shape A consists of cubes. appeurs planar. Let's determine connectivity (number neighhours for each cube). visual inspection, it seems to have a "backbone" of cubes and two 'legs'. The central cube of the bone appears connected to 3 other cubes. No cube appears connected to others. The connectivity pattern seems to be (1 cabe with 3 neighbours, I cube with 2 acighbbuurs, cubes with neighbour). This differs from Original's 's pattern (1 cube with 4 neighbours, cubes with neighbour). Therefore, A cannot be obtained by rotating the Original.
3. Analyze Shape
appears non-planar. Let's determine connectivity. looks like L-shape base cubes, with two cubes stacked vertically on two of the base cubes (one on the comeer one on an end). By visual inspection, the maximum number of neighbours any cube has (the corner cube the 1. and the two cubes adjacent to the L... The connectivity neighbour). This differs from the Original's patlern (1x4, 4xl).
Shape consists of'
cubes Therefore, cannat be obtained by rotating the Original.
puttern scems
neighbours,
4. Amalyze ze Shape C:
Shape consists cubes, This shape appears to planar, meaning cubes in a single layer. forms shape known pentomino (a 2x2 square with cube attached to the side). A planar shape consists cubes whose centers all lie
the same plane.
5. Compare Shapes with Original:
different than the cubes. rigid transfomnation, It cannol change non plon-planar object into a planar object. Since the Original shape is non non-plama planar and Shape C is planar, Shape C cannot be obtained by rolating the Original shape. be oblained cotating Original (due different counectivity patterns), difference in planarity between the Original and C more fundamental geometric property preserved under rotation. Given that
The Original shape (C, N, S, TJ non-planar because the Top cube
Shape C is planar. Rolation
While analysis also suggested A and B
usually only one option is correct in such problems, the planarity difference provides the strongest reason.
6. Conclusion:
Shape C cannot be obtained rotating the original cube stack because the original stack nea-planar and shape C is planar.
Final Answer: C
Figure 21: Test case of 3D Rotation Task from Gemini-2.5-pra
57

---

Published as a conference paper at ICLR 2026
Mental Rotation-3V ieew Projection-Level
You should first provide reasoning process, then provide single optionIA, B, C or D) as the final answer The ressoning process and the answer are enclosed within <think><<think>> and <answer=4answer> tags, respodively, i.c. <hinn--rasooinng prones<<thinks, ansserransseeeeeaswe....
Prompt:
Question: The cube stack is made of equal-sized small cubes, mostly gray with a few ones. The top row shows isometric view, front view, and top from left right. Which image in the bottom is the left view of the cube stack? Please answer from options A, B, C, or D.
Choices:
C.C
Ground Truth Answer: A
Thinking Process:
nayye the Isometric View:
The object is composed of stacks of cubes on a 3x3 base grid (viewed from top- down, let's Jabel rows as Frunt, Middle, Back
columns Left. Middle, Right),
Front- Left stack: cubes high (Bottom Red, Middle Grey, Top Red).
Front Middle stack: cubes high (A]l Grey).
Front- Right stack: cube high (Grey).
Middle-Left stack: cubes high (All Grey).
Middle-Middi stack: 4 cubes high (3 Grey base + 1 Red on top).
Middle-Rigg stack: cube high (Grey).
Back-Les Left stack: cubes high (A Grey, implied by structure).
Back Middle stack: cubes high (A Grey, implied base for MM rod cube).
Back-Right stack: cube high (All Grey, implied by structure).
Determine Left View:
from left side. The view will have columns representing the depth (Back, Middle, Front) and Jows representing the beight (Bottom, Middle, Top). We need to determine the color of the cube face visible at each position in this grid. Since the options are 3x3, we donsider boights up to
left view shows object
Looking from the left:
We see the lefunost faces of the stacks in each depth layer (Front, Middle, Back).
Front layer (Closest depth to the right in the left view): The leftmost stack is the Front-Lei stack. It high (Bottom Red,
Middle Grey, Top Red). Its face is visible. So,
left view is R (Top), (Middle), R (Rottom). Middle layer (Middle depth): The leftmost stack is the Middle Left stack. is high (All Grey). Its left face visible. So,
rightmost column
the middle column of the left view
(Top). G (Middle), G (Bottom). Back layer (Furthest depth to the
the left view): The leftmost stack is the Back-Left stack. is high (A Grey), Its left view is (Top), (Middle), (Bottom).
face is visible. So, leftmost column
4. Construct the 3x3 L. eft View Grid:
Row 3 (Top): Grey (Back), Grey (Middle). Red (Front)
Row (Middle): Gray (Back), Grey (Middle), Grey (Fmat) Row (Bottom): Grey (Back), Gray (Middle), Red (Front)
5. Compare with 0 pptions:
OptionA: Top Row (G, G, R), Middle Row (G, G, G), Bottom Row (G, G, R). This matches the dorived left view.
Option B: Top Row (R,
Middle Row (G,
Row (R, G, G). Mismatzh.
Option C: Top Row (G, G, R), Middle Row (G, G), Bottom Row (G, G. G). Mismatch.
Option D: Top Row (R, G, G), Middle Row (G, G, G), Bottom Row (G, G, G). Mismatzh.
Note on discrepancies: The provided Front and Top views in the question appear inconsistent with the Isometric view (e.g..
the height of the Front Right stack, the position of the topmost Red cube, the empty space in the Top view). However, the question asks for the left view of the cube stack shown in the isometric view. T Therefore, we derive the left view from the isometric drawing. The derivation based on the isometric view matches Option A. Final Answer:
Figure 22: Test case of Three- View Projection (Level 0) Task from Gemini-22 2.5-prro.
58

---

Published as a conference paper at ICLR 2026
Mental Rotation-3V iew Projection-Level
You should first provide reasoning process, then provide single optionIA, B, C or DJ as the final answer The reasoning process and the answer are enclosed within <think><(think>> and <anseer><answer>>>>>>> tags, respodively, i.c, <hinnk>eessonnng prones<ttinnk,, susserrensserreanssee.
Prompt:
Isometric
Top
Question: The top row shows the isometric view (left) and the top view (right) 3D model. W hich image in the bottom row is the front view of the model? Please
from options C,
Choices:
A. B. C.C D.All throe other options are incorroct
Ground Truth Answer:
Thinking Procoss:
1. Understand the Task:
The goal identify correct frunt view abject, given isometric view and top view.
nalyze Isometric View:
The isometric view shows 3D representation of the object. It consists of main rectangular prism (or block) and cylinder
attached to one of the larger lacos of the prism. The cylinder extends outwardss from this face.
Analyze the Top View:
shape. The wider horizontal part of the "T" corresponds to the top surface of the reclangular prism. The narrower vertical part of the "T" corresponds to the tup view of the sylinder. This confirms that the cylinder centered horizontally on the face it's attached to. also shows the relative width primm and the diameter of the cylinder, well the length the cylinder protrudes.
The top view shows object
from directly above. It appears
Determine the O rientation for the Front View:
The isometric view typically shows the front, top, and one side (usually right). The top view given. The front view would bo looking face from which cylinder pro trorudss.
Visualize the Front View:
Imagine looking directly at the face of the rectangular prism where the cylinder is attached. You would see the rectangular cylindor, which circle, superimposed onto roctangular fooc. From the isometric view, the cylinder uppears centered vertically the face.
outline of face. yu would
see
From the top view, the cylinder is centered horizontally on the face.
Therefore, front view should rectangle with circle located in its center:
Compare with the Options:
Option A: Shows rectangle with circle inside, centered horizontally and vertically. This matches the visualization of the front view derived from the isometric and top views.
Option B: Shows 'T" shape lying on side. This does not match the front view. resembles top view rotated.
Option C: Stows only roclangle. This would the back view the objoct, the front view if the cylinder absent.
7. Concluston:
Optin A the correct represenation of the front view of the given 3D object
Final Answer:
Figure 23: Test case of Three- View Projection (Level 1) Task from Gemini-2.5-pre
59

---

Published as a conference paper at ICLR 2026
Mental Tolding-Paper Folding-Level 0
Prompt: You should first provide reasoning process, then provide single optionlA, B, C D) as the final answer: The reasoning process and the answer are enclosed within <hink>></hinkk>>> and wawpwerr-s/wwwerreeeeeee tags, respectively, i.c. shinnigcsoninn process/think>, ousserennseerraane
Operation Operation
Punch Holes
Question: The original peperis is 3x3 grid paper: The images in the top row show the results of 2 consecutive folding operations on the grid paper Folding operations include folding along horizontal, vertical,
degree direction.
The righimost image in the top row shows the result after punching holes the folded paper. W hich image represents the appearance of the paper after unfolding the punched paper? Black solid circles reprsennt grid calls with holcs. Please answ from options A, B, C, Choices:
A.A B. C. D.
Ground Truth Answer:
Thinking Process:
Step nalyze the starting paper.
paper 3x3 grid. Let's the cells by (row, column) starting from top-lcft as (1,1).
(1.1) (1.2) (1,3); (2,1) (2.2) (2.3); (3.1) (3.2) (3.3)
Step Analyze Operation 1.
horizontal fold made across the middle the original 3x3 grid. The diagram shows bottom third (row being
folded upwards, lying undernatt the middle third (row The fold line is between row and row
The state after Operation 1 is a 3x2 grid shape (rows and are visiblc).
Row contains cells (1,1), (1,2), (1,3) (single lay er).
Row (visible) contains cells (2, 1), (2,2), (2,3) top er, cells (3.1), 1), (3,2), (3.3) bottom layer, folded undemeath.
Step 3: A Analye Operation 2.
Starting from the 3x2 shape after Operation A.A diagonal is made within right 2x2 section (columns 3). The fold line goes from the bottom- Jeft comer cell (2, to the nightghht of cell (1,3). The top right triangular part (involving cells (1,2). (1,3), (2,3)) is folded down/left onto the bottom- left trtangular part (involving oells (1,2), (2,2), (2,3)).
The state after Operation 2 is shown. Note the lay crs:
The leftmost column (original cells (1,1) and 12,1 is unaffected by this fold.
Cell (1,1) is single layer. Cell (2,1) is the top layer, (3,1) is the bottom layer:
- -The right part is now folded.
Step Analyze Punch Holes.
single hole punched. Its location the bottom-let left square of the shape shown after peration 2. This corresponds to
the coll location (2,1) in the configuration after Operation 1. Since this location (2,1) had two layers of paper from pprrtion I (original cell (2,1) on top and original cell (3,1) undemeath), punching a hole here creates holes through both layers. This area was
affected the fold Operation
Step 5: Unfold 0 Opertiio 2.
We reverse Operation 2 by unfolding the diagonal fold. Since the punched hole was in the part not involved in this fold ((2, area), unfolding Operation 2 does change the location the or croate new ones. The paper returns to the state after Operation (the 3x2 shape). We have holes through both layers at position (2,1). That means original cell (2,1) has hole, and original cell (3,1) has bolc.
Step Unfold Operation
Operation by unfolding the battom third 3) back down. The fold line between and 3. The holes are currently in cell (2,1) and cell (3. Unfolding separates these layers. The final unfolded 3x3 paper will have hole in cell (2,1) Imiddle row, left columa) and a hole in cell (3,1) (bottom row, Jeft column).
We
Step Compare with options.
Option shows holos (2.1) and (3,1). This matches result.
Option B shows holes at (1,1), (2, (3.1). Incorrec..
Option C shows holes at (1,1) and (3,1). Incorrect.
Option D shows hole (3,1). Incorrect
Therefore, the correct unfolded pattem is shown in Option A.
Final A nnser: A
Figure 24: Test of Paper Folding Task from Gemini-2.5-pro.
60

---

Published as a conference paper at ICLR 2026
Mental Folding- Cube Unfolding-Level 0
You should first provide a reasoning process, then provide single optionIA, B, or as the answer. The reasoning process and the answer enclosed within <hink>></hink>> and <answer><<answeer lags, respectively, l.e. <think>rosoring process<(think> <unswer>answer wr
Prompt:
The Left image shows colored tube from particular viewing angle. The options are nets (unfolded patterns) of the cubc, which are folded upward form cube. Which net, when folded, cannot form the cube shown left image? Please answer from options A, B, C, or D.
Question:
C Choices:
Cobo
A.A B.B C.C D.
Ground Truth Answer:
T Thinking Process:
1. Analyze the Target Cubee
The image shows cube with three visible faces meeting vertex: Red (Top), Blue (Frent), and Green (Right). This means
Red, Blue, and Green must he adjacent faces in any valid net representation. W Wrong color detection
2. Analyze the Nets (A, B, C, D):
Each not represents an unfolded cubo. To dotermine a can form the target cube, need chock the required faces
(Red, Blue, Green) are adjacent and if their spatial rel relattonship matches the target cube when folded. A key property of cube nets
is identifying opposite faces, Faces are opposite if they are separated by one face in straight line in the net, or by other standard
rules deperading on the net's shape.
3. Check Net
Identify opposite faces: Trace paths or visualize folding. In Net Teal is opposite Blue, Red opposite Y Yellow, and Green Is opposite Purple. -> W Wrong opposite faces
Check required adjacencies: The target cube requires Red, Blue, and Green to adjacent. Since none of these are opposite
pairs in Net (Red/Y ellow, Blue/Teal, Green/Purple) they can he adjacent.
Visualize folding: It's possible to fold Net A such that Red is Top, Blue is Front, and Green is Right, (For example, place Red top. Yellow bottom. Arang neighbours Teal, Puple, Green around Red. Blue opposite Teal. One possible fold results Top:Red, Bottom:Ycllon Front: Purple, Right:Groon, Back:Blue, Left.Taal. Rotating this gives the target view.)
Thus, A can form the cube.
4. Check Net B:
Identify opposite faces: In Net Yellow opposite Red, Teal opposite reen, Purple opposite Blue,
Check required adjacencies: The target cube requires Red, Blue, and Green to be adjacent. None of these are opposite pairs in
Not B (Red/Ycllow, Bluc/Purple, Green/Tral), so they can be adjacent.
Visualize tolding: It's possible fold Net to match target cube. (For example, one fold gives Top:Blue, Bottom:
Front:Green, Back:Teal, L Yellow Reght:Red. This can be rotated match the target view: Rop. Blue- Front, Green- Right).
Thus, Net B can form the cube.
5. Check Net C:
Identify opposite faces: In Net Yellow opposite Blue, Red opposite Purple, and Green opposite Teal.
Check required adjacencies: The target cube requires Red, Blue, and Green to be adjacent. None of these are opposite pairs in
Net C (Red/Purplee, Blue/Yellon ellow, Green/Teal so they can be adjacent.
target cube. (For example, one fold gives Top:Toal, Bottom: Front:Red, Back:Purple Left:Yelow,, Blue,. This configuration can be rotated into the target view Red- Top, Blue-Front, Creen-Right).
Visualize folding: possible fold Net
Thus, can form the cube.
6. Check Net D:
Idenlify opposite faces: In Not D. Blue is opposite Red, Purple is opposite Teal, and Green is opposite Yellow,
Check required adjacencies: The target cube shows Red, Blue, and Green meeting a vertex, which requires Red and Blue to
adjacent. However, Net Red and Blue are opposite faces.
Conclusion:
Opposite faces cannot mest at a vertex or share an edge, Since Net D places Red and Blue opposite each other, is into cube shown on left where Red Blue are adjacent Thus, Not cannot form the cube.
impossible fold
Final Answer:
Based on the analysis, Nets A, B, and C can form the target cube, while Net D cannot because It places two adjacent faces from the target cube (Red and Blue) in opposite positions.
Final nsser:
Figure 25: Test case of Cube Unfolding Task from Gemini-2.5-pra
61

---

Published as a conference paper at ICLR 2026
Mental Folding- Cube Unfolding-Level2
You should first provide reesoning process, provide single option(A, B, as the final answer: The rossoting process and the answer caclosed within <thin>>/think>>> and <unswor></nswerr>>> lags, respectively, Lc., <hhink>rasoning thicss,-/
Prompt:
The image shows a cube with different pattems on its faces from particular viewing angle. The options are nots (unfolded. pattems) of the cube, which are folded upward to form the cubo. Which not, when folded, cannot form the cube shown in the left image? Please answer from options,, B, C, or D. Choices:
Question:
A.A. B. B C. D.
Ground Truth Answer: A
Thinking Process:
Let's analyze the visible faces the cube:
Top face (T); Pattorn of Blue, Purple, Green dols. Spocifically, (8 B B P/G G G). The bottom- right comer dot is Green
(G).
Front (F): Pattem of Purple, Green Red, Blue dots. Specifically, (P
The top-right. comer Green (G).
Right face (R): Pattem of Green, Purple, Black, Blue, Red dots. Specifically, (G P./ G BI B/ B R). The top-left comer dot.
is
Green (G).
The common vertex where these three faces (T, E, R) moot a Green dot from oach face. So, this vertex is (G, G, G).
Now let's examine each net:
Net
Let's identify the pattems T, F, in Net if they exist.
Face of Net.A matobes T. (B B PP. P/G G.G)
Face 3 of NetAA matches F. (P G R B)
Now, we need to find face R: (G P./G B1 B
Let's list the pattems of the other faces NetA:
Face 1: (B G M R. / BB
G M .) Face 6: (P (M=Magenta/Puplly Face (BI B B.P.B.P........... /.G. Face 5: (P BI./ /
G G M B R.) Comparing R. with Faces 1, 4, 5, 6: None of them match the pattern of R. Since Face R is not present in NetA,
Net.A cannot form the cube shown.
Let's verffy other nets, assuming this is the primary criterion. If other nets contain T, R and can be arranged correctly, then
A is the answer.
Net
Face T. Face 3 Face 4 is R. These faces aranged such that is above F, and R is the right F. This is the standord T, R. configuration.
check common verter: (face 2): hottom-right right dot is G. (face 3): top-right dot R (face 4): top-left dot G. vertex (G,G,G). This is consistent with the cube. Thus, Net B can form the cube.
So, the
Net C:
Face T. Face is F. Face is R. These faces are arranged such that is above F, and R is to the right R. This the standard F, configuration. Let's check the cummon vertex:
T (face 1): bottom right dot is G. F (face 3): right dot G. R. (face 4): left dot G. So, the common vertex is (G, (G,G,))
This is consistent with the cube. Thus, Net C can form the cube.
Net
The topmost face R. The face below is T. The face to the left of T So, R, are present. mangement when folded:
is Top. I is Front, is Left.
Lot's find the common vertex for this configuration: (Top): bottom- Jeft dot (from R). T (Front): top-Jeft dot
(from B B R). (Left): rightight dot is (from G). So, the common vertex is (R, B, G). For this match the (G,G,G) vertex of the cube; we would need R=G and B=G, which is false as the colors are distinct. Therfore, Net D cannot form the cube shown with this specific comer.
However, the primary reason for Net being unable to form the cube absence of face R. This issue than correrr matching if all faces are present If a face is missing. absolutely cannot fomm the cube.
fundamenter
If we assume that there's 's only one incorrect option, and the "missing face" criterion for A is decisive, then A the answer. The with D (and potential edge matching issues with and C) might be secondary based on misinterpretations of "forming the cube". But missing face definitive failure, Let's reconfirm the "missing face in Net A argument. R (G P./ G Bl B R R) NotA, Face (B G R Bl
match (c.g. G lop-left of B lop- left of Face
NetA, Face (B1 /RRPP P G P)- No match (e.g.. G at top- of R, Bl top-left Face 4)
NetA, Face 5: (P Bl BBBBBBBBBBB G M) No match (e.g. G at top-left of R, Pat top-left of Face 5)
NctA, Face GPG M R) No match (e.g. G lop-left of R. P top-Jeft Face 6)
conclusion that face R is not present in Net seems correct. Therefore, N cannot form the cube. The final answer Silboxe((A}S.
Final Answer:
Figure 26: Test case of Cube Unfolding Task from Gemini-2 2.5-pro.
62

---

Published as a conference paper at ICLR 2026
Mental Folding- C ube Reconstruction-Leeel
Prompt: You should first provide roasoning process, then provide single option(A, B, the final answer. The reasoning process and the answer are caclosed within <think></think>> and <unswer></answer>> lags, respectively, Le., <think>ressoiing process//tiink>, ausawerrasswrr/aasweee
The left image shows not (unfolded pattem) of cube, with faces baving different pottems. The folded upwand form cube. From an axonometric (3D) viewing angle of the cube, which combination of adjacent patterns is possible to see? Please answer from options A, B, C, D.
Question:
Choices:
A.A B. C.C D.D
Ground Truth Answer: A
Thinking Process:
1. Identify the faces and their opposite pairs:
Label the faces in the net based on the number of dots: Top row: 4; Middle row: 3, 6; Bottom row: 1, (where below 3,
and 5 below 6) - Wrong pattern detection
Detemine opposite faces folding mentally noting faces separated by one other face straight line (or around comers):
The face with dots is opposite the face with 1 dot.
The face with dots is opposite the face with 6 dots.
face with dots is opgosite the face with 5 dots.
Evaluate the options based opposite
A cube view only show faces that are adjacent to each other. Opposite faces can never be seen together: Option A: Shows faces with 4 dots, dots, and dot. Since 4 and are opposito faces, this view impossible. opposite faces, this opposite faces, this view impossible. Option C: Shows faces with 2 dots, dots, and dots. Nome of these pairs (2-4, 2-3, 4-3) are opposite pairs (Opposites: 2-5, combination adj adjacent faces possible.
Opt on R: Shows
with 5 dots, dots,
dots, Since 2 and
impossible.
Option D: Shows faces with 3 dots, 6 dots, and dots. Since 3 and
4-1 1, 3-6). Therefore,
Verify Option C with adjacency and orientation (Optional but recommended):
Consider the face with 2 dots (let's call it F2) as the front face.
In the net, the face with dots (F4) is attached to the top edge of (relative the net layout shown). When folded,
bocomes the top face.
In the net, the face with 3 dots (F3) is attached to the left edge of F2. When folded, F3 becomes the left face.
So, the faces 2, 4, and 3 can indeed meet at the top- sop-frunt-left vertex.
Check ontentations shown C:
Front (2 dots): The dots diagonally from top-lett bottom- right. This matzhes the orientation the net relative the
edges connected to F4 and F3.
Top (1 dots): The dots form square. This matches the orientation on on the net relative to the edge connected to F2, vertically. middle dot orientation on the net relative to the edge connected to F2.
Left (3 dots);
edge shared with F2 (the front face). This matches the
The adjacencies and orientations in Option C are consistent with the net.
Conclusion:
Option and impossible bocause they show opposito faces.
Option shows three faces that are adjacent in the net and whose orientations are consistent with folding.
Final A nswer:
Figure 27: Test case of Cube Reconstruction Task from Gemini-2.5-pm
63

---

Published as a conference paper at ICLR 2026
Visual Penetration- Cross Section-Level 0
You should first provide reasoning process, provide single option(A, B, or the final answer The reasoning process and the answer are enclosed within <hink>-4think>> and <answer><</nnseet tags, respectively, i.e. <think rresoning (hinks,< caswerrnnswerrranse
Prompt:
The top row shows the combined shape viewed from bwo different angles he shape consists of a cone on top of square frustum. Which of the following images cannot cross section shape? Please answer from option A, B, C, D. C Cholces:
Question:
A.A B.B C.C D.
Ground Truth Answer:
Thinking Process:
n.Anal the Shape:
The object consists of cone positioned inside circular depression the top face square frustum. The base of the cone is circular and lies on the plane of the top face of the frustum. The frustum has a square base and tapers upwards to a smaller square top face. The circular depression (and thus the cone's base) is contained entiraly within this top square fare.
A Anayze Option A (Circle):
horizontal cross- section plane passing through the cone part (above the frustum's top face) will intersoct the form
circle. This is a possible cross section.
Analyze Option D (Square):
horizontal cross section plane passing through the frustum part below the Level of the cone's base (and below the
de depression) will intersect the frustum to form square. This is possible cruss- section..
4. Analye Option C (Triangle on Trapezoid):
vertical cross section plane pessing through central axis of both cone and the frustum, and parallel two opposite
side faces of the square frustum, will intersect the cone to form a triangle and the frustum to form isosceles trapezoid. The triangle will sit atop the trapezoid. This is e possi ble cross-section.
5. S.Analyze Option B (Curve on Trapezoid): - Correct and detailed analysis
soction Jormed by plane cutting through both the and the frustum. The curved part reprosents the intersection with the cone. the plane is vertical and off center, this curve is typically hyperbola segment, If the plane is ulted, could be an ellipse or parabola segment. The trapezoodal bottom part repreeent the intersection with the frustum. side face) trapezoid (if cutting diagonally
This shape represents
A vertical plane cutting the frustum
result rectangle parallel angle relative to the side faces). A tilted plane could also result in a trapezoid.
The key feature to examine the junction between the curved top and the trapezoidal base. This junction line represents the intersection of the cutting plane with the plane containing the cone's base (the top surface of the frustum). The width of the curved corresponds the length of the chord formed width of the top side of the trapezoid corresponds to the length of the line segment formed by the cutting plane intersecting the fristum's 's top square face. Since the cone's 's circular base strictly smaller than and contained within the frustum's top square fice in Views and 2), any cutting plane that intersects both the circle and the square will create cherd acmos the circle that is strictly shorter than the segment across the square (unless the plane is merely tangent to the circle). T Therefore, the base width of the curved (cone) section must be strictly narrowr than the top width the trapezoidal (frustum) section. However, in option B, base width of the curve appears be oqual the top width given that the cone's base circle lies inside the top square face the frustum.
section its
cutting plane intersecting the cone's circular base. The
trapezoid. This configuration is goometrically impossible
6. C onclusion: - Incorrect identification of the relative length relationship of options
Options A, and D represent possible cross sectiiss the described 3D object
Option depicts configuration where width the section matches the width of the frustum section the joining plano, which contradicts the goometry shown in Views 1 and 2 where the cone base is smallor than the top face of the frustum. Therefore, B cannot be cross-section.
Final Answer:
Figure 28: Test case of Cross-Section Task from Gemini-2.5-pro.
64

---

Published as a conference paper at ICLR 2026
Visual Penetration- Cube Counting- Level 1
You should first provide reasoning proces, then provide single optionAA, B, or D) as the final answer: The reasoning process and answer are enclosed within <hink><4think>> and <answer><<answer lags, respectively, l.c. <think Preasoning process<(hink>, <änswerzanswer canssrransseeraansssee
Prompt:
Front
Top
Left
Question: Given three views, what is the minimum number of cubes required satisfy the constraints shown in the images? Please answer from options
C,
C holces:
All three other options are incorrect B, 9
C. 11
Ground Truth Answer:
Thinking Process: -> Incorrect details, but based on the wrong details, the analysis is correct
Understand Goal:
The objective is to find the minimum number of unit cubes required build 3D structure whose orthographic pmjections
(Front, Top, Lett views) match the given diagrams.
Analyze the Top View:
The Top view shows the layout the cubes the base level (or the projection of all cubes onto the base). It occupies 2x3
grid area, with the cell in the ni-riiht comer (row column 3) and the bottom-rigg right cell (row 2, column 3) being potentially different.
The cells occupied Top row (back): (1,1), (1,2), (1,3); Bottom (front): (2,1), (2,2); Cell (2,3) empty. This means there must be at least one cube stacked each of the locations (1,2), (1,3), (2,1), and (2,2).
This establishos a minimum of cubes based on the footprint.
Analyze the Front View: W Wrong order
The Front view shows the maximum height. of the stacks in each column (left: righ).
Column (left): Height is 3. Corresponds to lorations (1,1) and (2, So, max(Heightt) 11, Height(2, []] =3
Column (middle): Height is Corresponds locations (1,2) (2,2), So, max Height( 22,), Height/2,2)) =3.
Column (right): Height Corresoonds location (1,3) (since (2,3) empty according the view). Height(
4. Analyze the Left View: The Left view shows
height the in each nw (front-to-backk viewed from the left).
Back Row (left side of the Left view): Height 3. Coresponds to locations (1,1), (1,2), (1,3). So, max(Height((.1))
Height(1,2), Height =3.
Front Row (right side
Left view): Height is 1. Corresponds to locations (2,1), (2,2). So, max(Height Height(2.2))
5. Combine Information and Minimize Cubes:
Let Hir, <) denote the height of the stack at row 'r' and column 'c". From Top view: H)],])=m], H(1,2)>=1, H(1,3)==1, H(2,1)>=1, H(2,2)>=1.
From vien: H,)g =1.
From Left view: max(H(2,1), H(2,2)) H(2,2) =1
Since we know H[2,1)>=] and H(2,2)==1, this forces H(2,1) H(2,2) =1.
Now the Front view constraints again:
max(H((,1)) H(2,1)) =3, Since =(2,1)=1, 1, we must have H(1 = to satisfy the maximum beight and minimize cubes.
(Hx,2) (1,2),, H(2,2)) =3 Since H(2,2) 1, we must have H(1,2) 3 satisfy the maximum height and minimize cubes.
Check the remaining Left view constraint: max(H(1,1), H(1.2), H(1,3)) max(3, =3.
This is satisfied by our dotermined heights.
6. Calculate Total Cubes: - T error does not affect the final numerical result determined the minimum required heights
stack: H(1,1) H(1.2) H(1.))( H(2,1 H(22)
Total number of cubes = H[1,1) H(1,2) H(1,3) H(2,1) H(2.2)
Total cubes =3 + +
Compare with Options:
The calculated minimum number of cubes This matchos option
Final nsswer: B
Figure 29: Test case of Cube Counting Task from Gemini-2.5-pm
65

---

Published as a conference paper at ICLR 2026
Visual Penetration- Cube ssembly-Level 0
You should first provide reasoning process, then provide single optionlA, B, C or D) as the final answer The reasoning process and answer are enclosed within <hink></think>> and <answer><danswer> tags, respectively,
Prompt:
Comslette Stack
Port
<thinkoreasoning
process<(hinnk> sanwernanswer<aassww
Question: The top left image shows the original complete cube stack made of equal sized cubes.
be formed by combining small cube stack on the pight 1) with one of the options below. W hich option completes the original cube stack? Please answer from options A,
B, C, or D. Choices:
A.A B. C.C D.All three other options are incorroct
Ground Truth Answer:
Thinking Process:
Most of the identifications of the number of cube stacks are incorrect
1.Analyze the Complete Cube Stack:
Visualize stack layer by layer or column by column. Let's use columns from back-left front right (like reading order
viewed from slightly above).
Back- Left Column: 2 cubes high.
Back- Right Column: cubes high.
Middle-Left Column: 2 cubes high.
Middle-Right Column: cube high (only bottom layer).
Front-Let eft Column: cube high (only bottom layer). Front-Righ Column: cube high (only bottom loy er).
Total Cubes = + + +1
cubes.
2. nalyze Part
Using same column description:
Back-Left Column: cubes high.
Back- RacktRight Column: cubes.
Middle-Leff Column: cube high (only bottom layer).
Middle-Righ Right Column: cube high (only bottom layer).
Pront-Lef eft Column: cube high (only bottom layer).
Front-Right Column: cubes.
Total Cubes +
cubes.
Determine the Missing Part (Complete Stack Part Incorrect approach to calculation
Subtract the cubes in Part from the Complete Stack for each column position:
Back-Left Column: (Complete)- (Part 1) = cubes.
Back Raght Column: (Complete)- (Part 1) =2 cubes (column high).
Middle-Left Column: 2 (Complete) (Part =1 cube (must be the top cube).
Middle-Righe Right Column: (Complete) 1 (Part 1) = cubes.
Pront-Leff Column: (Complete)- (Part
cubes.
Pront-Right Column:
(Complete) (Part 1) = cube (must be the hottom cube).
Total Cubes in Missing Part = + +
cubes.
Structure of Missing Part: A column of cubes in the back-right position, a single cube in the middle-left top position, and tront-right right bottom pesition.
single cube the front:
nalye the 0 ppiions:
Option A: Has column of 2 in the back-left,
mefd (bottom), in front-lof (bottom). Total 4 cubes. Doos not
match the required structure.
back-right (bottom), 1 in middle-lefs left (hottom), in middle right (bottom). Total 5 cubes, Incorrect number of cubcs and structure.
Option B: Has column of 2 in back-
Option C: column of 2 in the back-right position, single cube that aligns with middle-left left top position, and single cube that aligns with the front right bottom position. Total cubes. This matches the required structure derived step 3.
5. Conclusion:
Option missing part that combines with Part form Complete Cube Stack.
Final Answers sewer:
Figure 30: Test case of Cube Assembly Task from Gemini-2.5-pro.
66

---

Published as a conference paper at ICLR 2026
Mental Animation-Arrow Moving-Leva
You should first provide reasoning process, then provide single option(A, B, or D) as the final answer The reasoning process and the answer are enclosed within <hink><4think>> and <answer><danswer tags, respectively, Le., <hink>ressoning
Prompt:
proeess<thing <thinkk,, antsweranswerr/annee
In the diagram, the red arrow is the initial arrww and the green arrow is the final arrow. The arow can move in four directions (forward, backward, left, right), where 'orward' always refers to the current direction arrow is pointing After movement, arrow's s direction updated to the direction of movement. W Wich of the following paths make the amow move from the starting position to the ending position? Please answer from options A, B, C, or D. Cholces:
Question:
A. (Left, 2 units)--(Led (Leeft, unit)
B. (Forward, 1 uni))(BBcckvel (Backward, 1 unit)
C. (Forward, (mit)-((ackwart units)
(Forwarl, unit)-(Left, unit)t
Ground Truth Answer:
Thinking Process:
malyzz Initial State:
The rod arrow (start) is Jocated in the top left quadrant. Its center appears to be coortinates approximatoly (0.5, 1.5).
The direction of the red arow is polinting downwards (South).
nalyee Final States
The green arrow (end) located in the bottom-nige right quadrant. Its center appears be coordinates approximstely (1.5, 0.5).
The direction of the green arrow is pointing to the right (East).
Define Movement Rules:
Forward, Backward, Left, Right relative curent orientation. Atter each the arrow's orientation changes to the direction of the move. Cardinal Directions relative to current orientation:
facing North (Up): Forward=Upp Backward=Down, Left=West, Right=East
facing South (Down): Forward-Down Backward-Up, Left=East, West
If facing East (Right): Forward-Right, Backward-Let, Left=Up, Right=Down.
facing West (Lett): Forward-Left, Backward-Right Lefb=Down, Right=Up. Evaluate Option (Leff, 2 units) (Left, unit)
~ Fully comprehend the rules
Start: Position (0.5, 1.5), Direction Down.
Move 1: Left,, 2 units: Relative to Down, Left is East (Right). Move 2 units East
New Position: (0.5 + 1.5) (2.5, 1.5). New Direction: East (Right).
Move Left, unit. Relative to East (Right), Left North (Up). Move unit North.
New Position: (2.5, 1.5 +1) (2.5, 2.5).
Now Direction: North (Up).
Result: Position (2.5, 2.5), Direction Up. not match final
5. Evaluate Option B: (Forward, unit) (Backward, 1 unit)
Start: Position (0.5, 1.5). Direction Down.
Move 1: Forward, unit. Relative to Down, Forward Down. Move unit Down.
New Position: (0.5, 1.5 (0.5, 0.5). New Diroction: Down.
Move Backward, unit. Relative to Down, Rackward Up. Move unit Up.
New Position: (0.5, 0.5 +1) (0.5, 1.5). New Direction: Up.
Result: Position (0.5. 1.5), Direction Up.
not match the final 6. Evaluate Option C: (Forward, 1 unit) -- (Backward, units)
Start: Position (0.5, 1.5). Direction Down.
Move Forward, unit. Relative
Forward Down. Move unit Down.
New Position: (0.5, 1.5
(0.5, 0.5). New Direction: Down.
Move 2: Backward, 2 units. Relative to Down, Backward is Up. Move units Up.
New Position: (0.3, 0.5 +) (0.5, 2.51. New Direction: Up.
Result: Position (0.5, 2.5). Direction Up. Does not match the [inal state.
7. Evaluate Option D: (Forward, 1 unit) - (Left, 1 unit)
Start: Position (0.5, 1.5). Direction Down.
Move Forward, unit. Relative Down, Forward Down. Move unit Down.
New Position: (0.5, 1.5 1) = (0.5, 0.5). New Direction: Down.
Move 2: Left, unit. Relative to Down, Left East (Right). Move unit East
New Position: (0.5 +1 0.5) (1.5, 0.5). New Diroction: East (Right).
Result: Position (1.3) 5, 0.5). Direction East (Right). Matches the final state (green arow).
8. Conclusion: Optio D corrccly descrtbes the path from the rod anow to the green anow.
Final nswerr:
Figure 31: Test case of Arrow Moving Task (Level 0) from Gemini-2.5-pra
67

---

Published as a conference paper at ICLR 2026
Mental Animation-Arrow Moving-Level
Prompt: You should first provide reasoning process, then provide single optioni,, B, or the final answer: The ressoning process and the answer are caclosed within <hink>><think>>>> and <answor></answer>>> lags, respectively, I.c.
<think>xresoning prooese-think,, ausseeraasserrdanweer.
Question: The left image shows initial state. rrows can move four directions (forward, backward, right), where forward' always refers the current direction the anow is pointing, A fler each movement, the arro''s direction is updated to the direction of movement. the target position is empty, the arrww can move there directly; otherwise, it needs to swap with the arrow at target position, and both arrows' movements should satisfy aforementioned requirements, ffte transtoraations ((O, Left, units)-((11, Right, I unit), which state from the options be reached? Please from options A, B, C, D.
C Choices:
S
C. Ground Truth Answer:
t
T Thinking Process:
Analyze Initial State:
Grid coordinates range from x=2 and y=0 y=2 (implicitly, based on positions).
Arrow 1: Red at (0, 0) Orientation: Right (East).
Amw Green at 0), Orientation: Down (South).
Anoo Pink (2, 0), Orientation: Right (East).
Anow 4: Red at (0, 2), Ortentation: Up (North).
Analyze ze Transformation I: ((0, Lett, 2 units)
Identify the arow:
srrow starting (0, is mow (Red, faciigg Right).
Determine
diroction: 'Left' relative
's orientation (Right). facing Right, 'Left' is Up (North).
Determine the target position: Move 2 units Up from (0, 0). Target is (0, 0 + =0, 2).
Check target position: Position (0, corupied by Anow 4 (Red, facing Up).
Execute swap: Since the tarpet occupied, edd now and now
swap positions.
Arrow I moves from (0, 0) to (0, 2). Movement direction: Up. New orientation for nrow
Artow 4 must move from (0, 2) to (0, 0). Movement diroction: Down.
Check if Arrow 4's move valid: Arrow 4 facing Up.
Moving Down 'Backward', This valid relative move.
New orientation for now 4: Down (direction of movement).
mow Red at (0, 0), Orientation: Down. A nrow 2: Green at (1, 0), Orientation: Down. moo Pink 0), Orientation: Right A
State after Trarsfommtion
Red at 2), Orientation: Up.
3. Analyee Transformation 2: ((1, 0) Right, 1 unit)
Identify the arrow The artow currently at (1, 0) Amoww 2 (Green, facing Down).
Determine movement diroction: 'Right relative to the arrow's orientation (Down). facing Down, 'Right' is Left (West).
Determine the target position: Move unit Left from (1, 0). Target is (1 1, 0) (0, 0).
Check the target position: Position (0, 0) occupled by A now 4 (Red, facing Down).
Execute swap: Since the target occupled, now and rrow must swap positions.
Artow 2 moves from (1, (0, Movement direction: Left. New orientation for Anow Left.
Arro 4 must move from (0, 0) (1, 0). Movement direction: Right.
Check if Arrow 4's move valid: Arrow 4 is facing Down. Moving Right is "Right". This is valid relative move.
New orientation
now 4: Right (d of movement).
State after Transtomation (Final Statc): Arrow Green (0, 0). Orientation: Left. Arrow 4: Red at Orientation:
Right. A mow 3: Pink at (2, 0), Orientaion: Right. A now 1: Red at (0, 2), Orientation: Up.
Compare Final State with Options:
Option A: Incorrect wrwwsspositons/oriennaii
Option B: Incorrect wwtwspposiionssootetaaaiio
Option C: Groon at (0, 0), facing Loft (Match); Red at (1, 0), facing Right (Match); Pink at (2, 0), facing Right (Match);
Red (0, 2), facing Up. (Match). This matches the derived final state.
Option D: Incorrect wwwssppoiiionssooinnttioo
5. Conclusion: The state reached after the transformations matches Option C.
Final Answer:
Figure 32: Test case of Arrow Moving Task (Level 1) from Gemini-2.5 1amini-2255-po...
68

---

Published as a conference paper at ICLR 2026
Mental A nimation-Block Block Moving-Leved 1
Prompt: You should first provide roasoning process, then provide single option(A, B, C or D) as the final answer. The reasoning process and the answer are enclosed within <hink><4thnkk> and <ansver><lasswer>>> tägs, respectively, <thinkoresoning thinks, sanswrrpnnswerde
Initial State
Rotated Initial State
Question: The top row of images shows ditferent views of the initial state of cube stack, while the bottom row shows different views of the final state after transformation. uring transformation process, blocks can move one unit any direction (forward, backward, left, right, down). the target position is empty, the block can move there direclly; if the target position already has block, they swap places. Blocks cannot flost in the air. If a block moved away from posiiion any block above it will fall down until reaching supporting surface. The axes are shown in the diagram, and each block's position can be procisely identified using coordinates (x1,y1,z1), Which of the tollowing transformation sequences can change the cube stack from initial state final state shown the diagram? Please answer from options C, or Choices:
Final State
Rotated Final State
A. (1. x- (0, x+
0)
1, x+- 1)z-
C. [1. 1) 12, x-- [1, 1)x+
(0 0, x+- (2, x-
Ground Truth Answer:
Thinking Process:
Analyze States:
Identify block positions (R, G, DB, P. C) in Initial and Final states using consistent coordinate system (e.g. 0.0,0).
Initial: R,0,0,0), G(1,0,0), (B(0,1,0) 1(1,1,0) 1(1,11 Missing block and the positions of blocks are wrong
Final: R(0,0,0), DB(1,0,0) G(0,1,0), P(1,100,, C(1.1.1)
The change G swaps with DB.
2. A naay ze Rules:
Moves are 1 unit Target empty = move. Target occupied swap. Gravity applies if support is removed.
Evaluate Options:
A: (1x1,1)x- moves to (0,1,1). (0,0,0)x+ swaps R and
Result: G(O,0,0), R(1,0,0), DB(0,1,0), 1,0), P(1.1.0), C(0,1,1). Incorrect state.
(1,1,0)x- swaps P and DB. (1,1,0)x moves B(oow at 1,1,0) to (2,1,0), causing C(1,1,1) to fall to (1,1,0). (1,11)2-
attempts
from empty (1,1,1). Invalid.
C: (1,1,1]x+ moves ([1,1,1) to (2),, causing C to fall to 12,1, (2,1,0)x- swaps C(2,1,0) and P(1,1,0). (1,1, 1]x+ attempts
to move from empty (1.1,1). Invalid.
D: (0.0000x++ swaps R(0,0,0) and G(1,0,0). State: G(0,0,0), R(1,0,0), DB(0,110), 11,1,0), C(1,11)) (2,1,0 attempts move
from empty (2,1,0). Invalid.
4. Conclusion:
All options appear iny alid incorrect on the rules. B, C, reference empty coordinates. leads to the wrong state.
Reassess:
The problem likely contains eroor However, if forced to choose, D's first move is valid. The required transformation (swapp swapping 6 and DB) needs multiple steps. found a 3-step sequence: (,,y, (0,0,0)x+ (0,0,0) nother 3 step sequence starting like D: (0,0,0)x+, (0,1,0)x+, (0,0,0)y+ Since Option has only steps the second invalid, cannot correct as has the most "reasonable" Option D starts with a valid swap. Without clarification, choosing D assumes it's the intended answer despite the error.
written. But multiple-choice questions with flaws, sometimes the intended
structure.
Final Answer:
Figure 33: Test case of Block Moving Task from Gemini-2.5-pp
69

---

Published as a conference paper at ICLR 2026
Mental Animation echanical System-Level
Prompt: You should first provide reasoning process, then provide single optionlA, B, C D) as the final answer. The reesoning proces and the answer are enclosed within <thin=><think>> and <answ </asswrr tags, respectively, i.c. <chnnk>rasoming prooes-/think>, <answ www.eeeraawweeesssseeee Question: In the coaxial system shown, the orange gear fixed, and the yellow and green gears are identical in shape. If the yellow gear rotates clockwise, what is the motion of the green gear relative to the orange gear? Please choose from options C, Choices:
A. Clockwise rotation, clockwise rovolution
Clockwise rotation, counterclockwise revolution
C. No rotation, clockwise revolution
D. No rotation, counterclockwise revolution
Ground Truth nswer:
Thinking Process:
Can't intuitively thinking without theoretical derivation Define the sy stet and motion:
epicyclic gear train. The orange gear (Sun) is fixed (co o The yellow gear (Planet 1) meshes with the orange O). The green (Planet 2) meshes with the yellow gear, and is identical to it (N g= N y). The yellow and green gears are cartied by an implicit arm that revolves around the center of the orange gear. We need to find the fixed orange gear.
We
gear and rotates clockwise
absolute rotation (spin) and revolution (orbit) the green gear relative
Determine the arm's 'ss revolution (o) arm):
The reletionship between the sun, planet, and arm is given by the Willis equation: (oo o am) / (oo 0 to am) = =-N_o N y. Substituting u o O: (w to-y-- o. arm) / (-w arm) = o/ N y. Simplifying: to co, arm = (-oo am) (-N o/ //N y) = am o/ y). Rearanngng for oo o
o y). We are given that eo clockwise (negative). Since N and N are positive tooth counts, the term (1 + N o/ O is positive. T Therefore, for to be nogative, o arm must also be negative. This moans the arm, carryngg the yellow and green goars, revolves clockwise around the
amm oo (N N y) = arm
orange gear.
Determine the green gear's rotation (co g):
Now consider the meshing between the yellow and green gears. They are carried by the same amm The relative motion formula applies: (to g o. arm) (o y- o. am) =.N y y/N Since the yellow and green gears are identical, N N_y=N = g. The (wo to_g- g--e co arm) (m_y co arm) =1 __g wo =-(eo y- arm) =-m_y + arm, Rearranging for
ratio -N //N g =-1.
-0-00 +22
4. Express ei_ in terms of o_y and gear ratio:
Substituee the expression for arm from stop 2 too arm (1 o/ N y)] into the equation for o g_gg-aggg y=
[o y/O1 +N o/N v),. Factor out in y: ==o [-I+2/(1 N of N. vII. Combine temms inside the bracket o g=o
(1 +N N)
o_g (cI-NooNNy +2) + +Noo/ N 5)). 0_g =oy [(1 N/
o/NN v)]
5.Analyze the rotation based relative
Case 1: If N > (Orange gear larger than Yellow, as suggested by the image). Then -N o/ y) is negative. Since
o is pegative (clockwise), 0 g (negative) (negative/ positive) = positive. Green gear rotates counter clockwise. Motion: CCW rotation, CW revolution. (Not option).
Case 2: 1f N <N (Orange gear smaller than Yellow). Them (1 N o/ /N is positive. g (negative) (pesitive/ positive) = negative. Green geer rolates clockwise. Motion: CW rotation, CW revolation. (Option
Case 3: If N o y (Orange gear same size as Yellow). Then (1 N a/ y) = O. w__ g = y (0 positive) 0. Green gear no rotation (absolute rotation zem).. Motion: No rotation, CW revolution. (Option
6. Select the most plausible answer:
Although the image suggests N >N_ y, this loads to a result not listed in the options. Goar train problems in multiple choice format often rely on specific configuration, sometimes simplifying assumptions (like equal gear sizes not explicitly stated otherwise) that lead to one the provided answers, ssuming the problem intends for the case where = =N_y allows match option C.
Final Motion (assuming N o =NN yyy):
Rotation green gear [u_g):000000 matatio). Revolution of green gear (co arm): Clockwise
Final Answer:
Figure 34: Test case of Mechanical System Task from Gemini-2.5-pro.
70


Reviews:
[{"review_id": "MpeWjGTC6b", "reviewer": "Reviewer_XXzx", "summary": "This paper introduces SpatialViz-Bench, a benchmark that isolates and evaluates spatial visualization in multimodal LLMs across 4 sub-abilities mental rotation, mental folding, visual penetration and mental animation. This is introduced using 12 tasks and 1,180 programmatically generated problems with controllable difficulty. Eleven tasks are created with a Python + FreeCAD pipeline to enable scalable expansion and minimize contamination; one task (mechanical systems) is manually curated. The authors evaluate 27 MLLMs (8 closed-source, 19 open-source) in zero-shot settings with both Direct Answer and CoT prompting. Results show a large gap to humans and reveal an interesting finding that CoT hurts many open-source models, widening the closed vs. open-source gap. The diagnostic error analysis finds failures dominated by Perceptual and Spatial Transformation errors, suggesting limits in visuo-spatial processing rather than high-level logic. The benchmark aims to provide a stable, expandable, and contamination-resistant testbed to improve progress on visuo-spatial capabilties in MLLMs.", "questions": "1. How sensitive are CoT results to prompt variants, answer extraction rules, and output-length limits? Reporting accuracy across a prompt suite and per-model variance bars would provide more insights.\n2. Is possible to also publish 95% CIs per task/model and paired significance tests across prompts (CoT vs. Direct), and across difficulty levels to substantiate the key conclusions.\n3. Do improvements on SpatialViz-Bench correlate with performance on downstream visuospatial tasks or questions, VQA on CAD/DeepCAD?\n4. Was there any ablations on image resolution, number of views, and rendering style (lighting, line thickness) to determine sensitivity to low-level vision vs. spatial reasoning.\n5. Was there an analysis on controlled difficulty and its relation to performance of the models to identify where does it break?\n6. For the six error categories, is it possible to report inter-annotator agreement (κ)?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "* Centers spatial visualization (not just perception or generic “spatial reasoning”) and grounds it in cognitive science, decomposing ability into 4 sub-abilities with 12 targeted tasks.\n* Programmatic generation (Python + FreeCAD) for 11/12 tasks yields controllable difficulty, systematic distractors, and an expandable test bank to mitigate contamination; standardized templates reduce instruction confounds.\n* Broad coverage (rotation, folding, penetration, animation) plus error taxonomy (perceptual, transformation, memorization, instruction following, methodological, calculation/reasoning) surfaces where models fail rather than just how much.\n* Clear hierarchy among models with large human–model gap and near-random performance on core 3D tasks for many models; CoT ablation highlights that enforced reasoning text can degrade open-source model accuracy.", "weaknesses": "* Lacks uncertainty estimates (CIs/SEs) and significance testing across models/tasks.\n* The “CoT hurts” result may be prompt and format-sensitive. More prompts, temperature sweeps, and output-length controls are needed to ensure the effect isn’t an artifact of prompt adherence or extraction errors.\n* Pure multiple-choice framing may enable elimination strategies viz. no free-form diagram/pose prediction, 3D pose estimation, or procedural rollouts that might better stress genuine mental simulation.\n* There is limited evidence that gains on SpatialViz-Bench transfer to robotics, CAD, surgical planning, or navigation\n* The error analysis is “primarily manual” with model assistance; inter-annotator agreement (e.g., Cohen’s κ) is not reported.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ZXBEJQ364J", "reviewer": "Reviewer_GoVB", "summary": "This paper introduces SpatialViz-Bench, a new benchmark designed to evaluate the critical but under-tested skill of spatial visualization in Multi-modal Large Language Models (MLLMs). The authors argue that existing benchmarks fail to adequately assess an MLLM's ability to mentally manipulate unseen objects (e.g., mentally folding a net into a cube) and often rely on publicly sourced problems, risking data contamination.\nTo address this, SpatialViz-Bench offers a comprehensive suite of 1,180 problems across 12 tasks, which are programmatically generated to ensure fairness, scalability, and resistance to data leakage. The tasks are grounded in cognitive science, covering four core abilities: mental rotation, mental folding, visual penetration, and mental animation. Through an extensive evaluation of 27 MLLMs, the paper reveals several key findings: 1. A significant performance gap exists between all current models, including state-of-the-art systems like Gemini-2.5-pro, and a human baseline, highlighting the difficulty of the tasks. 2. In a counter-intuitive discovery, Chain-of-Thought (CoT) prompting paradoxically degrades the accuracy of many open-source models. 3. Error analysis indicates that model failures stem primarily from fundamental deficits in visual perception and spatial transformation, rather than high-level logical reasoning.\nUltimately, SpatialViz-Bench provides a robust, diagnostic tool for the research community, identifying a crucial weakness in modern MLLMs and guiding future efforts toward improving their core visuospatial intelligence.", "questions": "*   **The CoT Prompt:** The paper makes a strong claim about CoT, but the specifics of the prompt used are relegated to an appendix.\n    *   Can you give examples of the CoT prompt?\n    *   Why was a single, standardized prompt chosen for all models? It is well-known that prompt effectiveness is highly model-dependent. The observed negative effect might be an artifact of a suboptimal prompt for certain models, not an inherent issue with CoT itself.\n\n\n*   **Use of External Datasets:** For the \"Three-View Projection\" task, Table 1 mentions \"Real engineering parts (DeepCAD).\" It's unclear if DeepCAD is another procedurally generated source or a fixed dataset. If it's a fixed dataset, how does this align with the goal of creating a dynamically updatable benchmark?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "### 1. Addresses a Critical and Under-Evaluated Research Gap in MLLMs\nThe paper's strength is its focus on **spatial visualization**, a specific and advanced cognitive skill that has been largely overlooked by existing MLLM benchmarks. While many benchmarks test reasoning on *visible* information (e.g., \"What color is the car on the left?\"), SpatialViz-Bench is one of the first to systematically evaluate the ability to reason about *unseen* spatial relationships through mental manipulation. This is a crucial capability for real-world applications in fields like engineering (CAD models), medicine (interpreting 3D scans), and robotics (planning object manipulation), making this benchmark both timely and highly relevant.\n\n\n### 2. Diagnostic Analysis Beyond Simple Accuracy Metrics\nThe statistical error analysis (Figure 4) and the qualitative case studies (Figure 5) provide deeper insights into the models' failure modes. The conclusion that **\"Perceptual and Spatial Transformation errors dominate failures\"** is an important finding. It suggests that the primary bottleneck for MLLMs is not high-level logic or instruction following, but rather fundamental deficits in visual perception and mental manipulation. This provides a clear and actionable direction for future research and model development.\n\n\n\n\n### 3. Methodologically Robust and Trustworthy Benchmark Construction\nThe authors have designed a benchmark that directly confronts the most pressing challenges in modern AI evaluation, particularly data contamination and reliability. The core strength here is the **programmatic generation** of 11 out of the 12 tasks. This approach provides several key advantages:\n*   **Mitigates Data Contamination:** By generating novel, synthetic problems, the benchmark ensures that models are tested on their reasoning abilities rather than their ability to recall answers to similar problems scraped from the internet (e.g., from public IQ tests or math competitions).\n*   **Ensures Scalability and Longevity:** The programmatic framework allows for the continuous generation of new test cases. This creates a \"dynamic test bank\" that can be updated over time, making the benchmark a sustainable and long-lasting resource that is resilient to future training data leakage.\n*   **Enables Controlled Difficulty and Systematic Analysis:** The generation process allows for precise control over task difficulty (e.g., by increasing the number of folds or the complexity of an assembly). It also enables the systematic creation of distractor options based on common error types, which makes the evaluation more diagnostic.", "weaknesses": "**1. The \"Cognitively-Grounded\" Claim is Overstated**\n\nThe paper's implementation of its central claim to be \"cognitively-grounded\" is shallow. True cognitive grounding goes beyond simply adopting high-level task categories from historical psychology literature; it requires integrating principles of human cognition into task design, particularly concerning difficulty and error analysis.\n\n*   **Absence of Parametric Difficulty Scaling:** The paper's approach to scaling difficulty, as outlined in **Table 1 (lines 218-236)**, relies on intuitive and qualitative heuristics such as \"**Larger assemblies,**\" \"**More folds,**\" or \"**More system modules.**\" This approach fails to incorporate research on spatial reasoning. For example, a deeply grounded mental rotation task would parametrically vary the angle of rotation, as human reaction time and error rates are known to increase linearly with this variable. Similarly, a paper folding task's difficulty could be systematically controlled by the number of folds or the complexity of the resulting hole symmetry. By not grounding its difficulty levels in these quantifiable cognitive-load metrics, the benchmark cannot guarantee that \"Level 1\" is consistently and measurably more difficult than \"Level 0\" in a way that reflects cognitive processes.\n\n*   **Implication:** This lack of parametric control weakens the benchmark's diagnostic power. We cannot be certain that a model's performance drop from Level 0 to Level 1 is due to a failure in a specific cognitive step or simply an inability to handle more visual \"clutter.\" The benchmark measures performance on harder problems but doesn't provide the fine-grained control needed to diagnose *why* they are harder from a cognitive standpoint.\n\n\n\n**2. Issue with manually designed \"Mechanical System\" task**\n\nOne of the paper's arguments is its use of programmatic generation to create a scalable, dynamic, and contamination-resistant benchmark.  However, this rigor is abandoned for one of the 12 tasks, creating inconsistency.\n\n*   **Contradiction of Core Philosophy:** The authors state in **Section 3.3 (lines 265-267)** that the \"**Mechanical System task was manually designed**\" using \"**representative public simulations.**\" This contradicts their own critique of other benchmarks that \"**rely heavily on web-sourced problems, risking data leakage**\" (**line 096**). This task is now vulnerable to the exact problems the rest of the benchmark was designed to solve. The \"public simulations\" may very well be part of the pre-training corpora of the models being evaluated, turning a test of reasoning into a test of memorization.\n\n*   **Implication:** This inconsistency compromises the internal validity of the benchmark's results. Performance on the Mechanical System task cannot be reliably compared to the other 11 tasks. As seen in **Table 2 (line 328)**, many open-source models score surprisingly high on this task relative to their poor performance on other 3D tasks like Cube Unfolding or 3D Rotation. Is this because they possess a distinct \"intuitive physics\" capability, as the authors suggest (**lines 2534-2537**), or is it because they have been exposed to the source material during training? The manual design of this task makes it difficult to disentangle these possibilities, undermining the clarity of the evaluation.\n\n\n\n\n**3. The \"Chain-of-Thought Paradox\"**\n\nThe paper's finding that CoT prompting degrades performance on open-source models is a thought provoking conclusion. However, the analysis supporting this claim is not robust enough to rule out simpler, confounding explanations.\n\n*   **The Flaw of a One-Size-Fits-All Prompt:** The entire CoT analysis rests on a single, standardized prompt template applied to all models, detailed in **Appendix E.2 (lines 2347-2351)**. This is a methodological flaw. Different MLLMs are instruction-tuned on vast and varied datasets, leading to high sensitivity to prompt phrasing, structure, and keywords. A prompt that works well for one model family may be confusing or suboptimal for another. The observed performance degradation may not reflect a fundamental cognitive \"interference\" but rather a failure of prompt engineering—the models may simply be failing to adhere to an unfamiliar instruction format. A robust study should  include a prompt sensitivity analysis, testing several CoT variations to ensure the effect was not an artifact of their chosen template.\n\n*   **Selection Bias in the Comparison Group:** The paper introduces a selection bias that skews the comparison. The authors explicitly state they \"**excluded models... unable to adhere to the format (e.g., InternVL3-2B)**\" from the non-CoT evaluation (**lines 368-370**). This means the group of open-source models in the non-CoT test is a pre-filtered subset of the most capable or compliant models. The weaker models that fail even simple formatting are removed. This inflates the baseline non-CoT performance for the open-source category, making the subsequent drop when CoT is enabled appear larger than it may actually be.\n\n*   **Speculative Explanation:** The authors hypothesize that CoT \"**interferes with their intrinsic visual judgment**\" (**line 395**). The paper provides no qualitative analysis of the generated CoT content itself. A rigorous analysis would involve manually inspecting the reasoning steps to see *how* they fail. Do the models make perceptual errors in their text (e.g., \"the orange face is next to the green face\" when it is not)? Or are the errors logical, showing a flawed transformation process? The error analysis in **Figure 4 (line 390)** categorizes final outcomes but does not correlate specific error types with the use (or non-use) of CoT. Without this evidence, alternative hypotheses—such as the models simply generating irrelevant, distracting text—are equally plausible.\n\n\n\n**4. A Foundational Claim Lacking Empirical Verification: \"Vision-Dependence\"**\n\nThe paper rightly positions its benchmark as highly \"vision-dependent,\" stating that \"**textual input alone is insufficient**\" for problem-solving (**line 309**). While intuitively true, this is a foundational, empirical claim that is asserted rather than proven. The necessity of the visual input could have been easily and powerfully demonstrated with a simple control experiment. By feeding the text-only components of the prompts to a powerful text-only LLM (like GPT-4) and recording its performance, the authors could have provided definitive quantitative evidence for their claim. A resulting accuracy at or near the random-chance baseline (**25.08%**, noted in **Table 2**) would have formally established the visual modality as indispensable for this benchmark, thereby strengthening the paper's core motivation and contribution.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "C7bnzXVB2l", "reviewer": "Reviewer_9vwG", "summary": "The paper introduces SpatialViz-Bench, a cognitively-grounded, programmatically generated benchmark designed to evaluate spatial visualization abilities in Multi-modal Large Language Models (MLLMs). Covering 12 tasks across four sub-abilities (mental rotation, mental folding, visual penetration, and mental animation), the benchmark comprises 1,180 problems and enables scalable, contamination-resistant evaluation. Experiments on 27 MLLMs reveal significant performance gaps with humans and highlight that current models struggle primarily with perceptual and spatial transformation tasks, not high-level reasoning.", "questions": "As covered in the weaknesses, there is need for some validations on the claims and observations made", "limitations": "", "rating": 8, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "- Novelty & Relevance: Proposes a comprehensive, cognitively-motivated benchmark specifically targeting spatial visualization in MLLMs, addressing a clear gap in current evaluation practices. The \"gap\" identified here - whether LLMs can infer unseen relationships through\nspatial visualization - is a novel and crucial problem, not intensively researched\n\n- Methodological Rigor: The eye for detail in the error analysis as well as ensuring scalability are good grounds to cover in a benchmark paper. The idea of randomizing generations and dynamically updating the test bank is a valid thought for benchmark's robustness.\n\n- Insightful Analysis: Provides detailed error analysis and ablation studies, revealing that model failures are rooted in perceptual and transformation deficits rather than logic. The fact that CoT degrades model performance for open-source artefacts is an interesting call out and calls for further investigation", "weaknesses": "- CoT Degradation Analysis can be fleshed out more : \n   - This is a crucial and rather counter-intuitive observation made by the paper. It would be good to provide more information on the details of the framework for this specific study to establish trust on these results. \n   - The ablation study is limited to accuracy comparisons with and without CoT, and the explanation for the degradation is largely speculative (e.g., CoT text may distract models not fine-tuned for long-form reasoning). There is a need for more in-depth, systematic root cause analysis—such as controlled experiments isolating prompt length, reasoning style, or model architecture effects.\n   - The study does not explore whether prompt engineering, alternative CoT formats, or model-specific tuning could mitigate the observed degradation. Need to ablate on the CoT settings to ensure this is not a framework bias\n   - The paper does not provide error breakdowns or qualitative examples specifically illustrating how CoT leads to errors in open-source models.\n\n- Data Contamination solutions needs more validation : the idea of dynamically updating the test bank with randomized generations (while maintaining standardized templates for fair comparisons) is a sound hypothesis to mitigate data contamination issues, but quantification/testing to confirm the validity of this hypothesis is lacking\n    - There is a need for basic adversarial or retrieval-based analysis to test if models are “memorizing” or “recalling” similar problems from pretraining.\n    - The effectiveness of dynamic test bank updates is asserted, but not validated with concrete evidence or contamination metrics. There is no empirical study or audit quantifying the risk or presence of contamination in current models.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "iBErmrzXIz", "reviewer": "Reviewer_nVip", "summary": "This paper introduces SpatialViz-Bench, a new benchmark for diagnosing spatial visualization abilities in MLLMs. The authors address limitations and potential data contamination in existing benchmarks by creating a scalable framework that programmatically generates 1,180 novel problems across 12 distinct tasks. Their evaluation of 27 MLLMs reveals wide performance variations and shows that even state-of-the-art models struggle with these tasks. The diagnostic analysis suggests that these failures stem from fundamental deficits in perception and spatial transformation rather than high-level reasoning.", "questions": "1. The methodology for labeling error categories is unclear. Are these categories mutually exclusive (uni-label), or can a single failure case be assigned multiple labels (multi-label)? If it is a uni-label system, clarification is needed on how the \"critical error\" is determined or prioritized over other potential errors in a single instance.\n\n2. Regarding the counter-intuitive finding that CoT prompting degrades performance on open-source models: have the authors investigated whether this could be attributed to issues such as parsing failures in the CoT responses?", "limitations": "", "rating": 4, "confidence": 5, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1. The use of a synthetic dataset is a significant strength, as it effectively mitigates the risk of data contamination and ensures comprehensive coverage across a wide range of tasks.\n2. The benchmark demonstrates strong discriminative power, effectively highlighting performance differences across a broad spectrum of MLLMs.\n3. The statistical error analysis, which breaks down failures into fundamental perceptual and spatial transformation deficits, provides valuable insights into the core limitations of current models.", "weaknesses": "1. Figure 1 is dense and difficult to parse due to information overload. The upper-right legend, in particular, is unclear: the meaning of the \"error mark\" and the reference to \"existing evaluation\" are not immediately evident.\n2. The Related Work section requires revision. It currently reads more like an extension of the Introduction rather than an objective survey of the field. The tone is heavily opinion-based, with most existing work being cited primarily to emphasize their limitations. Furthermore, the section appears to neglect several of the most recent or concurrent benchmarks focused on spatial reasoning.[1-6]. \n\n\nReferences:\n\n[1] Ramakrishnan, Santhosh Kumar, et al. \"Does Spatial Cognition Emerge in Frontier Models?.\" arXiv preprint arXiv:2410.06468 (2024).\n\n[2] Stogiannidis, Ilias, Steven McDonagh, and Sotirios A. Tsaftaris. \"Mind the gap: Benchmarking spatial reasoning in vision-language models.\" arXiv preprint arXiv:2503.19707 (2025).\n\n[3] Xu, Wenrui, et al. \"Defining and Evaluating Visual Language Models' Basic Spatial Abilities: A Perspective from Psychometrics.\" arXiv preprint arXiv:2502.11859 (2025).\n\n[4] Li, Chengzu, et al. \"11plus-bench: Demystifying multimodal llm spatial reasoning with cognitive-inspired analysis.\" arXiv preprint arXiv:2508.20068 (2025).\n\n[5] Tang, Kexian, et al. \"LEGO-Puzzles: How Good Are MLLMs at Multi-Step Spatial Reasoning?.\" arXiv preprint arXiv:2503.19990 (2025).\n\n[6] Yin, Baiqiao, et al. \"Spatial Mental Modeling from Limited Views.\" arXiv preprint arXiv:2506.21458 (2025).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)