pySpatial: Generating 3D Visual Programs for Zero-Shot Spatial Reasoning
Authors: Zhanpeng Luo
Ce Zhang
Silong Yong
Cunxi Dai
Qianwei Wang
Haoxi Ran
Guanya Shi
Katia P. Sycara
Yaqi Xie
Venue: ICLR 2026 Poster (2026)
Keywords: Multi-Modal LLMs
Spatial Reasoning
3D Vision
Primary area: foundation or frontier models, including LLMs
TL;DR: We introduce pySpatial, a visual programming framework that flexibly composes spatial tools (e.g., 3D reconstruction, camera movements, and novel view synthesis) to enable MLLMs to explicitly reason in 3D space for diverse spatial reasoning tasks.

Abstract: Multi-modal Large Language Models (MLLMs) have demonstrated strong capabilities in general-purpose perception and reasoning, but they still struggle with tasks that require spatial understanding of the 3D world. To address this, we introduce pySpatial, a visual programming framework that equips MLLMs with the ability to interface with spatial tools via Python code generation. Given an image sequence and a natural-language query, the model composes function calls to spatial tools including 3D reconstruction, camera-pose recovery, novel-view rendering, etc. These operations convert raw 2D inputs into an explorable 3D scene, enabling MLLMs to reason explicitly over structured spatial representations. Notably, pySpatial requires no gradient-based fine-tuning and operates in a fully zero-shot setting. Experimental evaluations on the challenging MindCube and Omni3D-Bench benchmarks demonstrate that our framework pySpatial consistently surpasses strong MLLM baselines; for instance, it outperforms GPT-4.1-mini by 12.94% on MindCube. Furthermore, we conduct real-world indoor navigation experiments where the robot can successfully traverse complex environments using route plans generated by pySpatial, highlighting the practical effectiveness of our approach. Our project website will be available at https://pySpatial.github.io.

Published as a conference paper at ICLR 2026
PYSPATIAL: GENERATING 3D VISUAL PROGRAMS FOR ZERO-SHOT SPATIAL REASONING
Zhanpeng Luo Ce Zhang!
Silong Yong'
Cunxi Dai' Qianwei Wang!.
Haoxi Ran' Guanya Shi'
Katia Sycara'
Yaqi Xie!
I Carnegie Mellon University
University Pittsburgh
Uniiversity of Michigan
Query
Views
From the viewpoint presented image what is to the loft of the office chair with blue seat!
microwave
C. Blue rubbish bin
D. Kitchenetoe
Spatial Mental Models
pyšpatial (Ours)
First, construct cognitive
Let us consider directly 3D space...
New I execute the visual program...
map to conceptualize the scene...
3D Reconstruuciin
Cognitive Map
Imagine the scene...
To find what is to the left, we can simulate
by rotating the
camera to the left from the viewpoint of
looking this novel view, one can identify the objects to left office chair.
Q at view 3
corresponding program:
</> 3D Visual Program
novel, view
Novel Vivel Synthesis
Looking left from the chair's cell. the nearest object along that side Blue rubbish bin.
monelunnee prastaki antternee aedeedtton
acusted canns pos biaers remuetec)
From this synthesized novel view. answer
mimelly
Figure 1: Comparing our pySpatial with spatial mental models for multi-view spatial reason- ing tasks. Unlike spatial mental models (Yin et al., 2025), which rely on the implicit imagination of MLLMs construct a 2D cognitive map, we introduce pySpatial, a visual programming frame- work that flexibly composes spatial tools (e.g., 3D reconstruction, camera movements, and novel view synthesis) to enable MLLMs to explicitly reason 3D space for diverse spatial reasoning tasks.
ABSTRACT
Multi-modal Large Language Models (MLLMs) have demonstrated strong capabilities in general-purpose perception and reasoning, but they still struggle with tasks that require spatial understanding of the 3D world. To address this, we introduce pySpatial, visual programming framework that equips MLLMs with the ability to interface with spatial tools via Python code generation. Given image sequence and natural-language query, the model composes function calls to spatial tools including 3D reconstruction, camera-pose recovery, novel-view rendering, etc. These operations convert raw 2D inputs into explorable 3D scene, enabling MLLMS to reason explicitly over structured spatial representations. Notably, pySpatial requires gradient-based fine-tuning and operates in fully zero-shot setting. Experimental evaluations on the challenging MINDCUBE and OMNI3D-BENCH benchmarks demonstrate that our framework pySpatial consistently surpasses strong MILM baselines; for instance, it outperforms GPT-4.1-minl by 12.94% on MINDCUBE. Furthermore, we conduct real-world indoor navigation experiments where the robot can successfully traverse complex environments using route plans generated by pySpatial, highlighting the practical effectiveness of our approach. Our project website is available at https:://// http://yy/pppllappe github.ic io.
Equul contribution.

---

Published as a conference paper at ICLR 2026
INTRODUCTION
Multi-modal Large Language Models (MLLMs) have achieved remarkable success across diverse tasks such image captioning (Bucciarelli et al., 2024; Zhang et al., 2025b), referring ground- ing (Kazemzadeh et al., 2014), video understanding (Zeng et al., 2025; Fu et al., 2025), and robust reasoning (Mathew et al., 2021; Zhang et al., 2025a). However, this progress has not translated into robust 3D spatial reasoning: recent studies (Wu et al., 2025; Chen et al., 2024a; Chang et al., 2025) reveal that MILLMs still struggle with challenges spanning from hasic tasks such as judging relative object positions or estimating depth in single image (Liu et al., 2023; Cheng et al., 2024) to more complex reasoning over egocentric motion and multi-view relations (Yin et al., 2025; Yang et al., 2025). Such limitations pose a substantial barrier to their reliable deployment in safety-critical applications including robotics, augmented reality, and embodied intelligence, where tasks such as navigation, manipulation, and numan-roboo interaction depend on precise spatial understanding (Li et al., 2024; Duan et al., 2024; Song et al., 2025; Qiao et al., 2025).
While recent efforts (Chen et al., 2024a; Cheng et al., 2024) have primarily targeted improving spa- tial understanding from a single image (e.g.. "Is the stool in front of the oven?") ("), in this work we fo- cus on the more challenging problem of 3D spatial reasoning, where the environment is only partially observed with limited views and models must reason across perspectives to answer queries such as "Where should I move from view I to view 2?"_a setting in which ofate-of-the-a art MLLMs perform only slightly above random guess (Yin et al., 2025). Recent studies (Chen et al., 2024a; Ma et al., 2025) suggest that this weakness largely stems from the training data: although MLLMs are pre- trained on internet-scale image-caption pairs, explicit 3D supervision is sparse and costly, making it difficult to learn reliable correspondences between language and 3D spatial structures and thereby constraining models' ability to reason effectively in 3D space. More recently. Yin et al. (2025) explores the use of data structures such as 2D cognitive maps, where the model encodes object posi- tions in top-down view to mentally simulate spatial layouts, as shown in Figure 1. However, these approaches still rely on implicit "imagination" mechanisms and offer only limited effectiveness.
These limitations motivate our central research question: how can we equip MLLMs with explicit reasoning capabilities in 3D space? A natural first step toward this goal is to obtain an explicit geometric foundation on which such reasoning can take place. Recent advances in feed-forward 3D reconstruction (Wang et al., 2024b; 2025a) makes this feasible by recovering scene geometry directly from sparse 2D views, including camera parameters, depth maps, and scene-level point clouds. Such representations transform limited 2D views into an explorable 3D scene, within which models can perform spatial transformations (hereafter referred to as spatial tools) such camera translation, rotation, and viewpoint shifts to enrich visual context and build interactive reasoning chains. For instance, given the query "what is behind me ifl am al view 3," the model could rotate the virtual camera by 180 at the specified viewpoint within the reconstructed seene, thereby uncovering previously occluded regions and grounding its reasoning in geometric evidence.
However, how to enable MLLMs to flexibly compose spatial tools and seamlessly interact with 3D environments in a context-aware manner remains critical challenge. To address this, inspired by pioneering works on visual programming (Gupta & Kembhavi, 2023; Surís et al., 2023), we intro- duce pySpatial, a framework that employs MLLMs like GPT-4o as Python code generation agents to invoke function calls for tools such as 3D reconstruction, natural language description of move- ments, and novel view synthesis. As illustrated in Figure 1, pySpatial leverages a well-defined API to automatically select and compose the appropriate tools to solve diverse spatial reasoning tasks. Notably, pySpatial operates fully in a zero-shot setting and serves as a plug-and-play framework applicable to both open-source and closed-source MLLMs, offering interpretable solutions and reli- able responses that make it well-suited for diverse real-world tasks.
We evaluate the effectiveness of approach the MINDCUBE and OMNI3D- -BNCH bench- marks, where results demonstrate that pySpatial consistently outperforms strong MLLM basclines by substantial margins (e.g, achieving 12.94% improvement over GPPT-4.1-min on MINDCUBE). Qualitative analyses further verify that our approach can generate high-quality executable and in- terpretable visual programs that can effectively solve complex spatial reasoning tasks in a zero-shot manner. Furthermore, we apply pySpatial to real-world indoor navigation, where it successfully enables quadrupedal robot to traverse complex environments using generated route plans.
Our contributions can be summarized as follows:
2

---

Published as a conference paper at ICLR 2026
We present pySpatial, novel zero-shot framework that enables MLLMs reason explicitly in 3D space by generating and executing visual programs that leverage various spatial tools in a structured, compositional manner to solve diverse spatial reasoning tasks.
We evaluate pySpatial on MINDCUBE and OMNIID-BENCE where it demonstrates superior performance over strong MLLM baselines. Qualitative analysis validates that pySpatial reliably generates executable and interpretable visual programs for diverse spatial reasoning tasks.
We further assess the practical effectiveness of pySpatial on indoor navigation tasks, showing that it can generate route plans that enable quadrupedal robot traverse complex environments, demonstrating strong potentials for practical use cases.
2 RELATED WORK
MLLMS for Spatial Reasoning. Recent MILMS have demonstrated remarkable performance on multi-modal tasks (Wan et al., 2025; Alayrac et al., 2022; Zhang et al., 2024; 2026). However, stud- ies have shown that these models exhibit significant limitations in interpreting spatial relations (Yu et al., 2024; Kamath et al., 2023; Wang et al., 2024a; Tong et al., 2024), a critical precursor to a wide range of practical applications, including robotic manipulation (Huang et al., 2022; Shridhar al., 2023) and embodied navigation (Qiao et al., 2025; Huang et al., 2023). To address this, recent works such as SpatialVLM (Chen et al., 2024a) and SpatialRGPT (Cheng et al., 2024) typically propose scalable data synthesis and curation pipelines to strengthen single-view spatial reasoning capabilities through large-scale pre-training- Despite these advances, more recently, Yin et al. (2025); Huang et al. (2024); Ma et al. (2023) demonstrates that current MLLMs and such approaches still strug- gle with geometric understanding and perspective-takinn in multi-view settings. In this work, we among the first to tackle this challenge, and we propose novel zero-shot visual programming framework called pySpatial that systematically combines and applies various spatial tools, enabling models to explicitly reason in 3D and solve diverse spatial tasks.
3D Reconstruction. Classical 3D reconstruction methods, such as Structure-from-Motion (Schon- berger & Frahm, 2016), typically involve multiple stages and often rely on time-consuming optimization pipelines. More recently, feed-forward 3D reconstruction approaches such as DUS(3R (Wang al., 2024b), MAS13R (Leroy et al., 2024), CUT3R (Wang et al., 2025b) and VGGT (Wang et al., 2025a) leverage large-scale 3D pre-training and vision transformers to directly predict tixel-aligned 3D point maps. These data-driven methods demonstrate strong generalizabil- ity, even in scenarios without overlapping views. Building on this progress, subsequent works have extended feed-forward 3D reconstruction to applications in neural rendering (Charatan et al., 2024), SLAM (Maggio et al., 2025), and dynamic reconstruction (Lin et al., 2025; Luo et al., 2025).
Modular Visual Reasoning. To enhance compositional multi-moda. modall understanding, recent ad- vances treats vision specialists (such GroundingDINO (Liu et al., 2024) and SAM (Ravi et al., 2025)) as symbolic operators and composes them to solve complex reasoning problems. Represen- tative works such as Visual ChatGPT (Wu et al., 2023), MM-REACT (Yang al., 2023), and Hug- gingGPT (Shen et al., 2023) follow this direction by integrating LLMs with predefined toolchains to process multi-modal inputs. Building on this idea, VisProg (Gupta & Kembhavi, 2023) and ViperGPT (Surís et al., 2023) introduce visual programming that extends this paradigm by prompt- ing MILMSs to generate executable Python programs that call set of visual parsers through pre- defined APIs. More recently. VADAR (Marsili et al., 2025) introduces the visual programming paradigm for single-view spatial reasoning tasks with an adaptive API design. In contrast, our pySpatial introduces framework explicitly designed for multi-view spatial reasoning, equipping models with compositional 3D tools to handle diverse and complex spatial scenarios.
3 METHOD
In this section, we present pySpatial, a visual programming framework that enables MLLMs to reason explicitly in 3D space by generating and exccuting visual programs that orchestrate multiple spatial tools to address diverse spatial reasoning tasks. We also describe the framework design, including the pySpatial API signatures and the spatial tools it employs.
3

---

Published as a conference paper at ICLR 2026
3.1 PROBLEM FORMULATION
We consider setting where an MLLM M provided with an image sequence where each view has resolution H x W and captures partial observations of a 3D scene, along with a natural-language query q concerning spatial relations between objects or camera movements. The objective is to produce the correct response r* from the answer space A that answers the query.
{I, J-
As introduced in Section 1, we convert the limited 2D views into an explorable 3D scene via feed- forward reconstruction. This yields consistent depth estimates D, camera intrinsics K E [R3x3, and extrinsics G E SE(3) for each frame. Together, these quantities define a point cloud P world coordinates, which serves the geometric basis for downstream reasoning.
In addition, we adopt a program of an image sequence and a query (I, q), code agent J generates Python program that invokes a set of spatial tools through a well-defined API. The program is executed by an interpreter & to produce an intermediate output O, which may take the form of text, single image, a list of images depending on the program 2. This output provides direct visual evidence to support answering the query. For instance, when the query asks, "what is behind me if am at view 3," the program renders a new view by rotating the camera 180 at the specified viewpoint. Finally, the MLLM M integrates both the original visual inputs and the program outputs to generate the final response r E A.
persseeetieesseeetiiveeee following Surís et al. (2023). Given an input
3.2 SPATIAL TOOLS AND API
To guide the MILMs to explicitly reason in 3D space, we introduce various spatial tools such as 3D reconstruction, camera description, and novel view synthesis. We provide the pySpatial API signatures Code 1 and the details of each tool are described in the following sections.
3D Reconstruction. We adopt two feed- forward reconstructions depending on the specific task requirement. For metric-scale scenes, we use CUT3R (Wang et al., 2025b), which returns depth in real-world units. When relative distance in normalized unit space suffices, we adopt VGGT (Wang al., 2025a) for its generalizability.
class pySpatial:
""pySptial interface for 30 vieion tools.""
def reconstruct (aceze: Scene):
8 33 rosonctruction from eceze insges.
def catersbeooacrronotiiooo Roconetraction):
# Describe cazera motion fron reconstraction resulte.
Baabbsiie__ovel_iirrre sev carerrss a Gemerate novel view syathesis froz. reconstruction results.
Reconotraction
Formally, each pixel Pa IR? in a view I, with predicted depth D, (p:) is back-projected into the camera coordinate system using the intrin- sics K, and then transformed into world coor- dinates via the estimated pose G2 SE(3):
def righte_right (extrinsic, anglo=(S): # Rotate casera pose to the right, rotate 45 degreo default
X; = - (p., (P)),, (1)
def rotate letelelfffeetttae (extrinsic, angle=(5):
a Rotate canera pose to the left rotate 45 degree by default
where -- denotes the back projerrojettion from image coordinates to the 3D point in the camera frame. We get the point cloud P in the world space by concatenating X; for all pixels in all frames.
def nove foreard (extriasic, distance=0.3). a Nove canera pose forward, default dictance provided
def nove baw.baaacwdddeaeee (extrinsic, distasce=0.2 a Nove canera pose backvard, default distance provided
Camera Description. We translate raw camera pose matrices into natural language labels to make egocentric motion interpretable the language model. Each pose represented by an extrinsic matrix G R t E 33x4 which maps world points into the camera frame. The corresponding camera center in world coordinates C -R. Given two poses (R,, t1) and (R2, (R,2t2), t2). the displacement coordinates is AC,, C2 Cj. We then express this displacement in the first camera's frame as AC, = R₁ AC Restricting the displacement to the horizontal plane, we compute the yaw angle
def turn_areund (extrincic):
a Turn canera pose around 180 degreua
Code I: pySpatial API signatures.
world
(aan2(d,,4 d.) 180/n, where (dz, dz) the æ and components of AC, The angle is discretized into eight canonical motion categories (forward, backward, left, right, and four diagonals), yielding compact natural- natural-language description of egocentric movement.
Novel View Synthesis. To facilitate active exploration of the reconstructed 3D scene, we enable the agent to render novel views from arbitrary camera poses. Given point cloud P and
4

---

Published as a conference paper at ICLR 2026
transformation G R t| R3x4, we rasterize P into RGB image with respect to G and the corresponding camera intrinsics K. The agent can then issue high-level actions such as sscate_leff and turn_ around, which are implemented as yaw rotations about the world y-axis by angle Φ. The updated camera pose G' is obtained by applying the rotation to the camera-to-world transform and inverting back world-to-camera form. This operation provides interactive visual feedback that supports explicit spatial reasoning.
3.3 3D VISUAL PROGRAMMING
Program Generation. Given a query q. the code agent F synthesizes a Python program : = F()) that composes function calls specified the pySpatial API. By default, use GPT-4o, strong MLLM baseline that has demonstrated effectiveness in code generation, as it has been trained on Internet-scale Python code data. Note that the agent interacts only with the public interface (e.g.. reconstruct, rotate_right, synthesize _aateee__el_____eeeeeeeee_eeeeee novel_view) and has no access to internal imple- mentation details such as model weights, file I/O, or rendering backends. This abstraction separates high-level reasoning from low-level execution. We also provide default parameters for public inter- face regarding rotation and movement, i.e. 45 for rotation, and 0.3 for movement, as specified in Code 1. We guide program synthesis using in-context examples, where the prompts include inter- face documentation and query-code pairs without ground-truth answers. In addition, we leverage structured outputs to first enable free-form natural language reasoning, followed by the synthesis of Python code. The generated Python code, or visual program, acts as an explicit intermediate repre- sentation that encodes a sequence of tool invocations. It is inherently interpretable, researchers can readily inspect, debug, or modify the generated program, and composable, enabling seamless integration with additional tools or downstream reasoning modules. Once constructed, the program is executed by the interpreter to produce concrete spatial operations.
Program Execution. At execution time, the synthesized program = is executed by Python inter- preter& & over the input image sequence I, yielding an intermediate output O = T), Depending on the query, the output may take the form of text, single image, or sequence of rendered views. This intermediate output provides an explicit grounding of the program's 's reasoning steps in observable evidence. In the final stage, a MILM M integrates the original image sequence I, the program output O, and the natural language query to generate the final response r = =M[T, O q)-
EXPERIMENTS
In this section, we assess the effectiveness of pySpatial on MINDCUBE (Yin et al., 2025) and MANI3D-BENCK (Marsili et al., 2025), comparing it with existing state-of-the-as approaches.
4.1 EXPERIMENTAL SETTINGS
Benchmarks. We mainly evaluate our framework on the MINDCUBE (Yin et al., 2025), which is designed to probe the spatial reasoning capabilities of MLLMs under limited views. Specifically, MINDCUBE contains over 21,000 spatial question-answer pairs grounded in 3,268 multi-view in- door scenes, spanning three canonical camera motion types: rotation, around, and among. We also evaluate on MINDCUBE- k, a subset of MINDCUBE with 1,050 questions, specifically designed for evaluation purposes. In addition, following prior work (Marsili et al., 2025), we also evaluate our framework on OMNI3D-BEL -ENCI,, a single-view spatial reasoning benchmark, to examine whether our visual programming approach can generalize beyond multi-view settings.
Baselines. We compare the performance of pySpatial against four categories of existing baselines: (1) weinht linlte-imageMeee MLLMs, such as LLaVA-OneVisionn Vi (Li et al., 2025) and Qwen2.5- VL--3B-Instruct (Bai et al., 2025); (2) proprietary MI.LMs. including GPT-4o, GPT-4.1-min
and
Hlaud-4-Sonnet,, (3) specialized spatial models, such as Space-Qwar (Chen et al., 2024a) and VLM-3R (Fan et al., 2025), and (4) prior visual programming approaches such ViperGPT (Surís et al., 2023), VisProg (Gupta & Kembhavi, 2023), and VADAR (Marsili et al., 2025).
Implementation Details. By default, we follow prior visual programming work (Marsili et al., 2025) to leverage GPT-40 as the code agent to generate Python programs and produce final responses to queries. We use VGGT (Wang al., 2025a) as 3D reconstruction model on the MINDCUBE and OMNI3D-BENCH benchmarks. For real-world navigation, we use CUT3R (Wang et al., 2025b), which provides metric-scale reconstructions rather than normalized outputs. For point cloud rasteri-
5

---

Published as a conference paper at ICLR 2026
Table 1: Performance comparison on the full MINDCUBE (Yin al., 2025) dataset. The best results are shown in bold, and the second-best are underlined. Note that we implement pySpatial using GPT-4. 1-min as the code agent for this dataset due to budget constraints.
Method
Reference
Overall
Rotation
Among
Around
Baseline
Random (chance)
32.35 33.02
36.36 38.30
32.29 32.66
30.66
Random (frequency)
35.79
Open- Open-Weight Multi-Image Models
ALaVA-OneVision
(2025)
47.43
36.45
48.42
44.09
LLaVA- Si-VA-Vidoo--we--11 mPL.UGG-OOW13-B--241100
Zhang et al. (2025c)
41.96
35.71
43.55
30.12
ct al. (2025)
44.85
37.84
47.11
26.91
InternVL2.5-80
Chen et al. (2024b)
18.68
36.45
18.20
13.11
Brren..5-Vi-77--nsttue
Bai et al. (2025)
29.26
38.76
29.50
21.35
Bai et al. (2025)
33.21
37.37
33.26
30.34
DeepSeel-VLL--Small
Lu et al. (2024)
47.62
37.00
50.38
26.91
Proprietary Models
GPT-4o
OpenAI (2024)
38.81
32.65
40.17
29.16
GPT-4.1-min I-mini
OpenAl (2025)
45.62
37.84
47.22
34.56
Claude-4-Soman
Anthropic (2025)
44.75
48.42
44.21
47.62
Specialized Spatial Models
RoboBrain
et al. (2025)
37.38
35.80
38.28
29.53
SpaceMantis
Chen ct al. (2024a)
22.81
37.65
21.26
29.32
Spatial-MLLM
Wu et al. (2025)
32.06
38.39
20.92
32.82
Space-Qwen
Chen et al. (2024a)
33.28
38.02
33.71
26.32
VI.M-RR
Fan et al. (2025)
42.09
36.73
44.22
24.45
pySpatial (Ours)
58.56
43.20
60.54
48.10
Performance comparison on the MINDCUBE-IK (Yin et al., 2025) dataset. The eval- uated mental models (Yin et al., 2025) are based on Bwen2.5-VV-3B--nnse VADAR w/ Recon. denotes that we implement VADAR with our 3D reconstruction module. The best results are highlighted in bold, and the second-best are underlined.
Table 2:
(Bai al., 2025).
Method
Reference
Overall
Rotation
Among
Around
Baseline Models
Owen2.5-VL-3B-Instue
Bai et (2025)
37.81
36.00 43.00
45.20
GPT-4o
OpenAl (2024)
42.29
35.00
46.40
Spatial Mental Models
Chain-of- Chain-of-Thought
40.48
32.00
36.00
58.00
View Interpolation
Yin (2025)
37.81
36.50
Cognitive Map
41.43
37.00
41.67
44.40
Visual Programming Approaches
ViperGPT
Suris et (2023)
36.95
20.50
41.00
40.40
VADAR
Marsili (2025)
40.76
40.67
VADAR w/ Recon.
35.62
31.00
36.83
36.40
pyspatial (Ours)
62.35 +: 1.18
41.83 :: 2.34
64.89 =: 2.60
72.67 +:: 3.30
zation, we use Open3D (Zhou et al., 2018) to render novel views. All experiments are conducted on single NVIDIA A6000 Ada GPU. We provide full implementation details of pySpatial, along with the prompts used, in Appendix B and C. Code will be made publicly available upon acceptance.
4.2 QUANTITATIVE RESULTS
We first perform comprehensive evaluations of pySpatial the chal- lenging MINDCUBE benchmark rigorously assess its effectiveness multi-view spatial reason- ing. Table summarizes the results in comparison with baseline approaches. Overall, pySpatial achieves a clear performance margin over all categories of baselines. Specifically, reaches an overall accuracy of 58.56%, outperforming the best open-weight model DeepSeek-VL2-Small by 10.94%, and surpassing the strongest proprietary model GPT-4.1-min by 12.94%. On the Among category, which requires reasoning over how the central object relates to all surrounding objects, pySpatial achieves 60.54%, substantially outperforming all baselines, none which exceed 50%. Remarkably, pySpatial also outperforms VLM-3R (Fan al., 2025), which leverages
Results on MUNDCUBE.

---

Published as a conference paper at ICLR 2026
Table 3: Performance comparison on OMNI3D-BENCH. Following VADAR (Marsili et al., 2025), We report mean relative accuracy (Yang et al., 2025) for the numeric (other) and accuracy for the other category. The best results are shown in bold, and the second-best are underlined.
Method
Reference
numeric (ct)
numeric (other)
y/n
multi-choice
Total
Baseline Models
GPT-4o
OpenAI (2024)
28.1
35.5
66.7
57.2
42.9
Claude3.5-Sonie S-Sonett
Anthropic (2024)
22.4
20.6
62.2
50.6
32.2
Llama-3.2
Meta (2024)
24.3
19.3
47.5
27.4
25.6
Geminil.5-
Google (2024)
25.2
28.1
46.2
37.6
32.0
SpaceMantis
Chen ct al. (2024a)
20.0
21.7
50.6
48.2
30.3
Visual Programming Approaches
ViperGPT
Surís et al. (2023)
20.0
15.4
56.0
42.4
26.7
VisProg
Gupta & Kemihhvi (2023)
2.9
0.9
54.7
25.9
13.5
VADAR
Marsili et al. (2025)
21.7
35.5
56.0
57.6
40.4
pySpatial (Ours)
22.9
38.6
72.0
54.7
44.2
CUT3R (Wang al., 2025b) as the 3D encoder and fine-tuned on synthetic spatial reasoning data, by 16.5%, despite operating entirely in zero-shot setting. These results demonstrate that pySpatial generalizes well across diverse task categories on MINDCUBE. By explicitly decom- posing spatial reasoning into modular tool calls, our approach provides stronger inductive bias than both open- open-weight and proprietary MILMs, including those specialized for spatial reasoning.
Results MINDCUBE-IK. Table compares pySpatial against approaches based implicit mental modeling (Yin et al., 2025) (e.g., chain-of-thought reasoning, cognitive maps) and prior visual programming agents (e.g., ViperGPT, VADAR) MINDCUBE-Ik. We have the following key observations: (1) Spatial mental models (Yin et al., 2025), which rely on the implicit imagina- tion mechanisms of MLLMs for spatial reasoning, yield only limited performance gains, whereas pySpatial outperforms each of them by roughly 20%; (2) Our pySpatial substantially outper- forms existing visual programming approaches, achieving, for example, 21.9% improvement over VADAR. Notably, pySpatial also surpasses VADAR w/ Recon., where we re-implement VADAR using our 3D reconstruction module. This result demonstrates that even when equipped with 3D information, VADAR's s adaptive API design remains unreliable and lacks robustness for reasoning 3D space. These results validate the superior effectiveness pySpatial over existing baselines, demonstrating the advantages of enabling explicit 3D reasoning for multi-view spatial reasoning.
Results on OMNI3D-BENCH. We further evaluate pySpatial on the recent single-view spatial reasoning benchmark 3MNI3 D-BENCH, demonstrating that our framework generalizes effectively to single-view settings and provides consistent improvements across task categories. Table 3 shows results OMNI3D-BENCH, BENCH, where follow the evaluation protocol of VADAR (Marsili et al., 2025): mean relative accuracy (MRA) is reported for the numeric (other) subtask, and standard ac- curacy is used for the remaining categories. Our pySpatial outperforms prior visual programming approaches, achieving gains of 3.8% over VADAR and 17.5% over ViperGPT, and sets a new over- all state-of-the-arr on DMNI3D-BENCH! Notably, pySpatial also surpasses GPT-4o on the total score, underscoring that our visual programming framework provides benefits even over advanced proprietary MLLMs. This result highlights the broad generalizability of pySpatiall. even single- view settings where geometric cues are less apparent, explicitly invoking 3D functions through the code agent continues enhance spatial reasoning.
4.3 QUALITATIVE RESULTS
To further illustrate the capabilities of our pySpatial framework, we conduct qualitative experi- ments on representative examples from the MINDCUBE benchmark. As shown in Figure 2, each query paired with the generated 3D visual program, the reconstructed 3D scene, the program out- puts, and the final response produced by pySpatial. These examples highlight how pySpatial enables MLLMs to reason explicitly within an explorable 3D scene reconstructed from sparse 2D inputs. By synthesizing executable and interpretable visual programs that perform operations such as camera translation, rotation, and novel view synthesis, the framework provides interpretable out- puts that ground the reasoning process in geometric evidence. Across diverse spatial reasoning tasks, pySpatial produces responses that closely align with ground annotations, highlight- ing the effectiveness of our approach. It is worth noting that the generated 3D visual programs
7

---

Published as a conference paper at ICLR 2026
Query: which direction did move from the view to the second view!
Ground Truth: Diagonally forward and left.
3D Reconstruction:
Program Ourput:
def program Arpur poam: Sours:: 15
Euages
From image to Image 2, the camera move dizgoraly forward and left.
ansesaurtion nysportal morredrrutt tepue sceee)
stop 2: decortoe conece teturne the nuo stoue astag eutertantin
Artunn the deceriptiin lio aniic civur sorvt coe of the acour chatces
ranens
Final Response: The visual due explictly states the camera moved from pose to pose 2 diagonally forward and left. Therefore, the direction of the
movement is diagonally forward and left.
Query: From the viewpoint 2, what on to the right of the blakk chair!
Ground Truch: Gray Sofa.
out program apert Somnls
3D Reconstruction:
from
Program Output:
recoeste etlant prdential recenstracii tepet, Joene)
vimppiins, recostractioouu.o (1) right
blach chete
nosal, alequdot ponpatia,, inttttttttaaae wig.er/tgtteeeeeeeeooooo Al
renural
nevel, ster aySpecial. syrthalie, rovel, ylu rnporttrccttdd, eewe.ytemerrt) to ccaat the paectiss return revel_
tespection
Final Response: From the rendered point cloud image, we a large grey sofa. Its shape, cushion structure, and placerment along the make it clearly identifable a sofa rachor a table, cabinot, office setup.
Query: From viewpoitt 4, I turn left and forward, will closer
two single safas?
Ground Teuch
oragren text Sommes
from Creat Suoger
Reconstruction:
reclnctracte Laaee pahpat recuetttionnn Lepect
kengringuurt recantracinn www. antrannii steestre
prdpettal rotirs, Jeft hare, steoetet:
Auai daaand four
Lift vitappto
Systrentze rowes vito for the turned ceft and cored forved risupeints
Program Outpuc
Evewer rndeuital tut qestecting seel, sine sooedim meteed sinee wne Fewwid) decare foo teoqus for rusal conpartion thick closer aofics
Leage, tumn Hbrst.
Image, wove, forward
/anward
Final Response: The second view shows the two single sofas prominendy and larger, suggesting cleser distance.
Query: From veepoint 3, what right the blue bag?
Ground Truch: Black sofa.
prog Recosstract the as acone froe the four Smages
3D Reconstruction:
reconstractel.ce somee astgatial insorstirntt Lopre
canere axcronste vdeupstet_ recenstracted, comn, actrinstess 2) do tst right or the olse bay
Lmage (Teder
Program Output:
Syethecton the abent vies from the notered conero soce
reves yonr asdpetixi ayrtlayttti anod. mepamsdrnciedd yome, viaapiin__iipt.pp LM the ahir the zight of Alue nog
trupecttion iikecity
Final Response: The novel view synthesized by rotating the camera to the right of the viewpoint 3 shows a black sofa point cloud, The shape, color, and texture in that alig well with the appearance of blck sofa rather than window, plush toy, display shelves.
Figure 2: Qualitative results on four representative examples from MINDCUBE. We show that pySpatial enables MILMs to explicitly reason within the reconstructed explorable 3D scene, effectively addressing diverse spatial reasoning tasks through interpretable and executable 3D visual programs. Figure Al further illustrates that pySpatial is capable of composing executable 3D visual programs with control flow constructs for.o.)or-oopss. allowing it to robustly address a wide range of spatial reasoning tasks. Best viewed when zoomed in.
include well-structured comments that capture the reasoning process of pySpatial, thereby pro- viding transparency and interpretability that researchers can readily verify, debug, or modify.
4.4 REAI.-WORLD ROBOT NAVIGATION
To test the potential of real-world deployment using purely MLLMs, we employ a quadrupedal robot with a velocity-tracking controller in 50 m² two-room laboratory. In this setup, the MLLM gener- ates high-level position commands, which are manually converted into temporal velocity targets that the controller tracks, enabling the robot to navigate from an initial pose room toy). From limited 2D views, pySpatial reconstructs an explorable 3D scene, infers camera poses via visual programming. and generates a structured motion plan for the robot to execute.
target object (a mush-
As shown in Figure 3, our pySpatial successfully guides the robot through doorways, make correct turns, and finally toward the correct goal location. Notably, the MLLM baseline GPT-4.1 struggles to resolve relative direction such as left-right and fails to provide absolute metric distance estimates, leading to navigation errors. In contrast, our agent outputs precise rotations and translations that align with real-world execution, resulting in reliable task completion. This experiment demonstrates that our approach not only produces coherent spatial reasoning in question answering benchmarks, but also transfers effectively to physical robotic platforms for complex indoor navigation tasks.
8

---

Published as a conference paper at ICLR 2026
Query: Given quadrupedal robot initialized the pose corresponding image generate specification plan navigading the mushroom toy Image 4.
Input Views
Reconstructed 3D Scene & Motion Plan
Goal
Real-World Robot Navigation
Ours GPT4.1
Proposed Mation Plan from GPT-4.1
Proposed Mation Plan from pyšpatial (Ours)
Move forward 2.2m the office through doorway larger room. Then turn left 35 to directly face the white table. toward the near edge the table (canter position), and then turn left 40 to align with the long left wall.
Go through the door into next room (straight 3. 3.09m).
Move forward
Turn right 78.8", move alongside the white table toward the back corner
brown box (4.23m).
Move diagorally forward-lett 2.6m along the cable's side, stapping the far by the whitaboards and then right 30 face the mushroom toy on the chair.
From there, tumn left 69 move past the black chair with mushroom toy, straight to
(2.23m).
Figure 3: Qualitative results on real-world robot navigation. We deploy pySpatial on a Unitree- Gol robot navigate toward target object (mushroom toy) using limited views input. Compared to the GPT-4.1 baseline, which fails due to an incorrect initial turn and produces a collision-pront trajectory, pySpatial generates a geometrically consistent plan that successfully reaches the goal.
4.5 DISCUSSIONS
Ablation Study the Code Agent. To ab- late the effect of our code agent, we con- duct experiments the MINDCUBE-1k bench- mark by comparing the performance of vari- ous MLLM baselines with and without integra- tion of pySpatial. As summarized in Table 4, augmenting models with pySpatial consis- tently leads to substantial improvements across all tested MILMs, including GPT-4o, GPT-4.1-
Table 4: Ablation study on the code agent. We
report the accuracy on the MINDCUBE- 1k dataset.
Method GPT-4o
Overall
Rotation
Among
Around
42.29
35.00
43.00
46.40
pySpatial
62.67
41.00
65.00
66.33
GPT-4.1-mini
43.34
36.00
45.00
44.80
pySpatial
58.19
37.50
62.00
65.60
GPT-4.1
44.67
35.50
45.33
mini, and GPT-4.1. For instance, GPT-4o im-
pySpatial
62.35
41.83
64.89
72.67
proves from 42.3% to 62.7% overall accuracy, indicating that pySpatial generalizes across differ- ent MLLMs and effectively enhance spatial reasoning.
Failure Case Analysis. From MINDCUBE benchmark, we se- lect representative subset of about 100 samples and conduct manual analysis to identify the underlying sources error in cases where the final response incorrect. As shown in Figure 4, among the 39% of failure cases, only 6% are attributable to incorrectly gen- erated visual programs that fail to address the query, validating the effectiveness of our overall programming pipeline. Beyond this, 20% of errors arise from the MLLMs at the final reasoning step, while 13% stem from limitations in the 3D reconstruction models, where the generated visual programs are correct but the program outputs do not provide useful information. These results also sug- gest that advances 3D reconstruction and code generation models hold the potential to further enhance our performance.
13%
61% %
20%
Correct
6%
Progran am Error
Res ponse Error
Recon. Error
Figure 4: Failure case study. We manually examine the er- ror sources in about 100 sam- ples from MINDCUBE.
Remarks on Efficiency. Our pySpatial completes the MINDCUBE-IK benchmark 2.17 hours on single GPU using single thread for 1,050 queries, averaging 7.45 seconds per query. As the breakdown, code generation requires 2.41 seconds, program execution 2.14 seconds, and answer generation 2.90 seconds. For comparison, VADAR (Marsili et al., 2025) requires 17.25 seconds per query on average. These results demonstrate that our visual programming framework enhances the spatial reasoning capabilities of MILMS while remaining efficient deploy without excessive cost.

---

Published as a conference paper at ICLR 2026
5 CONCLUSION
this work, we present pySpatial, a visual programming framework that enhance spatial rea- soning capabilities of MILMs through zero-shot Python code generation. By composing functions such as 3D reconstruction and novel-view synthesis, pySpatial converts 2D image sequences into explorable 3D scenes, enabling explicit reasoning in 3D space. Experiments the MINDCUBE and OMNI3ID-BENCE benchmarks demonstrate that pySpatial consistently outperforms strong MILMM baselines, with gains of up 12.94% MINDCUBE compared GPT-4. Beyond benchmarks, real-world indoor navigation experiments further validate its practicality, showing that robots can successfully traverse complex environments using route plans generated by pySpatial.
ACKNOWLEDGEMENN
This work has been funded part by the Army Research Laboratory (ARL) award W91 2N--23-2- 0007 and W91 IQX-24-F-0049, DARPA award FA8750-23-2-1011 and ONR award N00014-23-1 2840. The author would like to thank Dr. Xiang Yue for valuable discussions during the project.
REPRODUCIBILITY STATEMENT
We are committed to ensuring the reproducibility of our results. All the spatial tools used this work are open-sourced, and the benchmark datasets we evaluate are publicly available. We have provided detailed descriptions of our experimental setup and implementation details in Section and Appendix to facilitate reproducibility. Code publicly available at https://gil /tte:////hh.../////// com/ Zhanpeng1202/ 2rampennt2022/y$patiii. SpStati.l.
REFERENCES
Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, lain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katherine Millican, Malcolm Reynolds, et al. Flamingo: a visual language model for few-shot learning. Advances in Neural Information Processing Systems, 35:23716-
23736, 2022.
Anthropic. Claude 3.5 sonnet, October 2024. claude- -- 5- sonnet.
URL
Anthropic. Claude 4 somnet system card, May 2025. URL 6be924522667770887744aaaadddd 32ed995. pdf.
anthropic.com com/
Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. Qwen2.5-vl technical report. arXiv preprint arXiv:2502. 2025. 5,6
Davide Bucciarelli, Nicholas Moratelli, Marcella Cornia, Lorenzo Baraldi, and Rita Cucchiara. Per- sonalizing multimodal large language models for image captioning: an experimental analysis. In European Conference on Computer Vision, pp. 351-368. Springer, 2024.
Chun-Peng Chang, Alain Pagani, and Didier Stricker. 3d spatial understanding in milms: biguation and evaluation. In IEEE International Conference Robotics and Automation, pp- 13537-13544, IEEE, 2025. 2
Disam-
David Charatan, Sizhe Lester Li, Andrea Tagliasacchi, and Vincent Sitzmann. pixelSplat: 3d gaus- sian splats from image pairs for scalable generalizable 3d reconstruction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 19457-19467, 2024.
Boyuan Chen, Zhuo Xu, Sean Kirmani, Brain Ichter, Dorsa Sadigh, Leonidas Guibas, and Fei Xia. Spatialvim: Endowing vision-language models with spatial reasoning capabilities. In Proceedings of the IEFF/CVF Conference on Computer Vision and Pattern Recognition, pp. 14455-14465, 2024a. 6,
Zhe Chen, Jiannan Wu, Wenhai Wang, Weijie Su, Guo Chen, Sen Xing, Muyan Zhong, Qinglong Zhang, Xizhou Zhu, Lewei I Lui et al. Internvl: Scaling up vision foundation models and aligning
10

---

Published as a conference paper at ICLR 2026
for generie visual- visual-linguisti tasks. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 24185-2411 24198, 2024b.
An-Chich Cheng. Hongxu Yin, Yang Fu, Qiushan Guo, Ruihan Yang, Jan Kautz, Xiaolong Wang, and Sifei Liu. Spatialrgpt: Grounded spatial reasoning in vision-language models. Advances in Neural Information Processing Systems, 37:135062-13509, 2024. 2,3
Jianguo Duan, Liwen Zhuang, Qinglei Zhang, Ying Zhou, and Jiyun Qin. Multimodal perception- fusion-control and human-robar collaboration in manufacturing: A review. The International Journal of Advanced Manufacturing Technology, 132(3):1071-1093 2024.
Zhiwen Fan, Jian Zhang. Renjie Li, Junge Zhang, Runjin Chen, Hezhen Hu, Kevin Wang. Huaizhi Qu, Dilin Wang, Zhicheng Yan, et al. Vlm-3r: Vision-language models augmented with instruction-aligned 3d reconstruction. arXiv preprint arXiv:2505.20279, 2025. 5,6
Chaoyou Fu, Yuhan Dai, Yongdong Luo, Lei Li, Shuhuai Ren, Renrui Zhang, Zihan Wang, Chenyu Zhou, Yunhang Shen, Mengdan Zhang, al. Video-mme: The first-ever comprehensive evalu- ation benchmark of multi-modal Ilms video analysis. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 24108-24118, 2025.
Google. Our next-generation model: Gemini 1.5, February 2024. URL https://blogg / technology/ a hatp//aa/gggoooeoe google- geeii-gmiiiii-ii next- generation www-mmmddllddbrrry.--ee wew.uarrrrre00--- 20044/
Tanmay Gupta and Aniruddha Kembhavi. Visual programming: Compositional visual reasoning
without training. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14953-14962, 2023. 2, 3,5,7
Chenguang Huang, Oier Mees, Andy Zeng, and Wolfram Burgard. Visual language maps for robot navigation. In IEEE International Conference on Robotics and Automation, pp. 10608-10615. IEEE, 2023. 3
Jiangyong Huang, Silong Yong, Xiaojian Ma. Xiongkun Linghu. Puhao Li, Yan Wang, Qing Li, Song-Chun Zhu, Baoxiong Jia, and Siyuan Huang. An embodied generalist agent in 3d world. In International Conference Machine Learning, pp.
PMLR, 2024. 3
Wenlong Huang, Pieter Abbeel, Deepak Pathak, and Igor Mordatch. Language models as zero-shot planners: Extracting actionable knowledge for embodied agents. In International Conference on Machine Learning, pp. 91 18-9147. PMLR, 2022. 3
Yuheng Ji, Huajie Tan, Jiayu Shi, Xiaoshuai Hao, Yuan Zhang, Hengyuan Zhang, Pengwei Wang, Mengdi Zhao, Yao Mu, Pengju An, al. Robobrain: A unified brain model for robotic manipu- lation from abstract to concrete. In Proceedings of the Computer Vision and Puttern Recognition Conference, pp. 1724-1734, 2025. 6
Amita Kamath, Jack Hessel, and Kai- Rei Chang. What's "up" with vision-language models? in- vestigating their struggle with spatial reasoning. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, pp. 9161-9175, 2023. 3
Sahar Kazemzadeh, Vicente Ordonez, Mark Matten, and Tamara Berg. Referitgame: Referring to objects in photographs of natural scenes. In Proceedings of the Conference Empirical Methods in Natural Language Processing, pp. 787-798, 2014. 2
Vincent Leroy, Yohann Cabon, and Jérôme Revaud. Grounding image matching in 3d with mast3r. In European Conference Computer Vision, pp. 71-91. Springer, 2024.
Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang. Kaichen Zhang, Peiyuan Zhang, Yanwei Li. Ziwei Liu, and Chunyuan Li. LLa -onovviiinn Easy visual task transfer. Transactions Machine Learning Research, 2025. URL https: h/tteprr///pnrevieer net/forum? id=zKv8qULV6n. 5,
Xiaoqi Li, Mingxu Zhang, Yiran Geng, Haoran Geng, Yuxing Long. Yan Shen, Renrui Zhang, Jiaming Liu, and Hao Dong. Manipllm: Embodied multimodal large language model for object- centric robotic manipulation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 18061-18070, 2024. 2

---

Published as a conference paper at ICLR 2026
Chenguo Lin, Yuchen Lin, Panwang Pan, Yifan Yu, Honglei Yan, Katerina Fragkiadaki, and Yadong Mu. Movies: Motion-aware 4d dynamic view synthesis in one second. arXiv preprint arXiv:2507.10065, 2025. 3
Fangyu Liu, Guy Emerson, and Nigel Collier. Visual spatial reasoning. Transactions of the Associ- ation for Computational Linguistics, 11:635-665 651, 2023. 2
Shilong Liu, Zhaoyang Zeng, Tianhe Ren, Feng Li, Hao Zhang, Jie Yang, Qing Jiang, Chunyuan Li, Jianwei Yang, Hang Su, et al. Grounding dino: Marrying dino with grounded pre-training for open-set object detection. In European Conference Computer Vision, pp. 38 38-55. Springer, 2024. 3
Haoyu Lu, Wen Liu, Bo Zhang, Bingxuan Wang, Kai Dong, Bo Liu, Jingxiang Sun, Tongzheng Ren, Zhuoshu Li, Hao Yang. et al. Deepseek-vl: towards real-world vision- langule understanding. arXiv preprint arXiv:2403.05525 2024. 6
Zhanpeng Luo, Haoxi Ran, and Li Lu. Instant4d: 4d gaussian splatting in minutes. In Advances in Neural Information Processing Systems, 2025. URL https://openreviiii het/forum?id= HTLa6Ao0jG. 3
Wufei Ma, Yu-Cheng Chou, Qihao Liu, Xingrui Wang, Celso M de Melo, Jianwen Xie, and Alan Yuille. Spatialreasoner: Towards explicit and generalizable 3d spatial reasoning. In Advances in Neurul Information Processing Systems, 2025. URL https: haaps://opepee http:///ppeeeeeeee//eeee net/ tatt/forum?? id=
hFaXVjRFHI. 2
Xiaojian Ma. Silong Yong, Zilong Zheng, Qing Li, Yitao Liang, Song-Chun Zhu, and Siyuan Huang. Sqa3d: Situated question answering in 3d scenes. In International Conference on Learning Rep- resentations, 2023. URL
ftre:/forummict 1tte://foooomm21-10000//833 3
Dominic Rosario Maggio, Hyungtae Lim, and Luca Carlone, VGGT-SLAM: Dense RGB SLAM optimized on the SL(4) manifold. In Advances in Neural Information Processing Systems, 2025.
URL https: htt:://oppp//vvee wwt.:nettooouioe formmidddddddddooeo matt/Eorrmriidd33.jCCmm... j.CMMEE.
Damiano Marsili, Rohun Agrawal, Yisong Yue, and Georgia Gkioxari. Visual agentic ai for spatial reasoning with dynamic api. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 19446-19455, 2025. 3, 5, 6, 7,7
Minesh Mathew, Dimosthenis Karatzas, and CV Jawahar. Docvqa: A dataset for vqa document images. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp.
2021. 2
Meta. Llama 3.2: Revolutionizing edge
and vision with open, cus-
tomizable models, September 2024. 1lama-3-2 2-nnecnneett-----ee-ooe--e---e-e www.iiiieeieeooiiiiiooeooeeeeeee
URL https://ai. hatp:://aa..mta..... com/blogt
OpenAl. Hello gpt-4o, August 2024. URL http::p://ppeaai.coooo/eeee/////
6,7
OpenAl. Introducing gpt-4.1 in the api, April 2025. URL https:://opeaai.com/ gpt-4-1/..
Yanyuan Qiao, Haodong Hong, Wenqi Lyu, Dong An, Siqi Zhang, Yutong Xie, Xinyu Wang, and Qi Wu. Navbench: Probing multimodal large language models for embodied navigation. In Advances in Neural Information Processing Systems, 2025. URL /ttps://openreviie net/ borun/id-nf8PKKk-20 2,
Nikhila Ravi, Valentin Gabeur, Yuan-Ting Hu, Ronghang Hu, Chaitanya Ryali, Tengyu Ma, Haitham Khedr, Roman Rädle, Chloe Rolland, Laura Gustafson, Eric Mintun, Junting Pan, Kalyan Va- sudev Alwala, Nicolas Carion, Chao-Yure Yuan Wu, Ross Girshick, Piotr Dollar, and Christoph Fe- ichtenhofer. SAM 2: Segment anything in images and videos. In International Conference on Learning Representations, 2025. URL https: http::////ppeeeviee//e net/ffrruurt htt./forrmmmLiHaaRRRRMMMMMaa
Johannes L Schonberger and Jan-Michael Frahm. Structure-fromt from-motion revisited. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition, pp. 4113-4113, 2016.
12

---

Published as a conference paper at ICLR 2026
Yongliang Shen, Kaitao Song, Xu Tan, Dongsheng Li, Weiming Lu, and Yueting Zhuang. Hugging- gpt: Solving ai tasks with chatgpt and its friends in hugging face. Advances Neural Information Processing Systems, 36:38155 50-5844-0000-- 2023. 3
Mohit Shridhar, Lucas Manuelli, and Dieter Fox. Perceiver-actor: A multi-task transformer for robotic manipulation. In Conference on Robot Learning, pp. 785-799. PMLR, 2023.
Chan Hee Song, Valts Blukis, Jonathan Tremblay, Stephen Tyree, Yu Su, and Stan Birchfield. Ro- bospatial: Teaching spatial understanding to 2d and 3d vision-language models for robotics. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 15768-15780, 2025. 2
Dídac Surís, Sachit Menon, and Carl Vondrick. Vipergpt: Visual inference via python execution for reasoning. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 11888-1 1189,, 2023. 3, 4, 5, 7
Peter Tong, Ellis Brown, Penghao Wu, Sanghyun Woo, Adithya Jairam Vedagiri IYER, Sai Charitha Akula, Shusheng Yang. Jihan Yang, Manoj Middepogu, Ziteng Wang, et al. Cambrian-ls I: fully open, vision-centric exploration of multimodal Ilms. Advances in Neural Information Processing Systems, 37:87310-87356, 2024. 3
Zifu Wan, Ce Zhang, Silong Yong. Martin Q Ma, Simon Stepputtis, Louis Louis-Philippe Morency, Deva Ramanan, Katia Sycara, and Yaqi Xie. Only: One-layer intervention sufficiently mitigates hallu- cinations in large vision-language models. arXiv preprint urXiv:5507.08895 2025.
Jianyuan Wang, Minghao Chen, Nikita Karaev, Andrea Vedaldi, Christian Rupprecht, and David Novotny. Vggt: Visual geometry grounded transformer. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 3294-5306, 2025a. 2,3,4,5
Jiayu Wang, Yifei Ming. Zhenmei Shi, Vibhav Vineet, Xin Wang. Sharon Li, and Neel Joshi. Is picture worth thousand words? delving into spatial reasoning for vision language models. Advances in Neural Information Processing Systems, 37:75392-75447 2024a.
Qianqian Wang, Yifei Zhang, Aleksander Holynski, Alexei A Efros, and Angjoo Kanazawa. Con- tinuous 3d perception model with persistent state. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 10510-10522, 2025b. 3, 4, 5,
Shuzhe Wang, Vincent Leroy, Yohann Cabon, Boris Chidlovskii, and Jerome Revaud. Dust3r: Ge- ometric 3d vision made easy. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 20697-20700 2024b. 2,3
Chenfei Wu, Shengming Yin, Weizhen Qi. Xiaodong Wang, Zecheng Tang, and Nan Duan. Vi- sual chatgpt: Talking, drawing and editing with visual foundation models. arXiv preprint arXiv:2303.04677 2023.
Diankun Wu, Fangfu Liu, Yi-Hsin Hung, and Yueqi Duan. Spatial-ML.LM: Boosting MILM ca- pabilities in visual-based spatial intelligence. In Advances in Neural Information Processing Systems, 2025. URL https: / /ttp///pppeeeeeeeeeeeeeeeeeeeeee
aK4K.. 2, 6
Jihan Yang. Shusheng Yang, Anjali W Gupta, Rilyn Han, Li Fei-Fei, and Saining Xie. Thinking in space: How multimodal large language models see, remember, and recall spaces. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 10632-10643, 2025. 2,
Zhengyuan Yang, Linjie Li, Jianfeng Wang, Kevin Lin, Ehsan Azarnasab, Faisal Ahmed, Zicheng Liu, Ce Liu, Michael Zeng, and Lijuan Wang. Mm-react: Prompting chatgpt for multimodal reasoning and action. arXiv preprini arXiv:2303: 1381, 2023. 3
Jiabo Ye, Haiyang Xu, Haowei Liu, Anwen Hu, Ming Yan, Qi Qian, Ji Zhang, Fei Huang, and Jingren Zhou. mPLUG-ow13: Towards long image-sequence understanding in multi-modal large language models. In International Conference on Learning Representations, 2025. URL https: // //openreview. wwapenrriiii.nee/foor/....ee. net.fforumidde::37abullt 6
13

---

Published as a conference paper at ICLR 2026
Baiqiao Yin, Qineng Wang, Pingyue Zhang. Jianshu Zhang, Kangrui Wang. Zihan Wang, Jieyu Zhang, Keshigeyan Chandrasegaran, Han Liu, Ranjay Krishna, et al. Spatial mental modeling from limited views. arXiv preprint arXiv:2506.21458 2025. 2,
Weihao Yu, Zhengyuan Yang, Linjie Li, Jianfeng Wang, Kevin Lin, Zicheng Liu, Xinchao Wang, and Lijuan Wang. Mm-vet: Evaluating large multimodal models for integrated capabilities. In International Conference on Machine Learning, pp. 17730-57754 PMLR, 2024. 3
Xiangyu Zeng, Kunchang Li, Chenting Wang, Xinhao Li, Tianxiang Jiang, Ziang Yan, Songze Li Yansong Shi, Zhengrong Yue. Yi Wang, Yali Wang, Yu Qiao, and Limin Wang. Time- suite: Improving MLLMs for long video understanding via grounded tuning. In International Conference Learning Representations, 2025. URL https: nAVejJURqZ. 2
Ce Zhang, Simon Stepputtis, Katia Sycara, and Yaqi Xie. Dual prototype evolving for test-time generalization of vision-language models. Advances in Neural Information Processing Systems, 37:32111-32136, 2024. 3
Ce Zhang, Simon Stepputtis, Katia Sycara, and Yaqi Xie. Enhancing vision- vision-language few-shot adap- tation with negative learning. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 5905-5915, 2025a. 2
Ce Zhang, Zifu Wan, Zhehan Kan, Martin Q. Ma, Simon Stepputtis, Deva Ramanan, Russ Salakhut- dinov, Louis-Phillippe Morency, Katia P. Sycara, and Yaqi Xie. Self-correcting decoding with gen- erative feedback for mitigating hallucinations in large vision-language models. In International Conference on Learning Representations, 2025b. htLthttss///pemttt tTBXePRKSx. 2
www.wwew/e..o...e..eeeeeeeeeeeee nnt/ fent/forum?idd
Ce Zhang, Kaixin Ma, Tianqing Fang, Wenhao Yu, Hongming Zhang, Zhisong Zhang, Haitao Mi, and Dong Yu. Vscan: Rethinking visual token reduction for efficient large vision-language mod- els. Trunsactions on Machine Learning Research, 2026. URL forum?id-K2YhyiIPut.
net/
Yuanhan Zhang, Jinming Wu, Wei Li, Bo Li, Zejun MA, Ziwei Liu, and Chunyuan Li. LLaVA- video: Video instruction tuning with synthetic data, Transactions on Machine Learning Research, 2025c. URL https: /esttss:/oppeee http:://oopeeeveeeeeeeeeee www.:eeffooooeoe /tttt
Qian-Yi Yi Zhou, Jaesik Park, and Vladlen Koltun. Open3d: A modern library for 3d data processing. arXiv preprint arXiv: 2018. 6
14

---

Published as a conference paper at ICLR 2026
Explicit 3D Spatial Reasoning via Program Generation Appendix
In the appendix, we provide additional experimental results and implementation details of our pySpatial framework. The appendix is organized follows:
Section A provides more experimental results.
Section B shows the API specification for pySpatial.
Section C presents the prompt implementation for agent.
Section D disclosures the use of large language models.
A MORE EXPERIMENTAL RESULTS
A. RESULTS ON MMSI-BENCH
We further evaluate our approach on MMSI-Bench and report the results in Table A1. We observe that pySpatial improves the overall MMSI-Bench performance by 6.4% on average, further demon- strating the effectiveness of our approach.
Table Al: Evaluation results on MMSI-Bench. Our pySpatial is based on GPT-4o.
Positional Relationship
Attribute
Motion
MSR
Models
Avg.
Cam--CumDty Reg-Reg Cam.OOij Ohj-Rog. Cam-Rog
Appr.
Com.
Ohj
Proprictary
GPT:5
43.0
35.1
32.1
48.8
42.4
51.8
60.9
36.4
32.4
36.8
42.0
41.9
03
45.2
39.4
37.0
44.2
47.1
62.6
54.7
28.8
31.1
32.9
34.9
41.0
GPT-4.5
34.4
29.8
39.5
51,2
47.1
55.4
39.1
33.3
41.9
40.8
36.4
40.3
GPT-4.1
36.6
26.6
27.2
29.1
36.5
27.7
37.5
24.2
36.5
32.9
28.8
30.9
GPT-4o
37.6
35.1
36.8
Gemini-2.5-Pro
39.7
31.9
39.5
45.3
35.2
43.3
51.5
21.2
36.4
30.2
34.3
36.9
Open-source
IntemMVL3-788
34,4
23.4
32.1
12.8
37.6
26.5
37.5
19.7
28.4
31.6
29.3
28.5
VL..5ML
29.1
1ren2.5-VL.-72B
25.8
34,0
34.6
23.3
34.1
36.1
45.3
27.3
27.0
30.3
27.3
30.7
LLaVA-One Visionnn
43.0
31.9
33.3
30.2
37.6
38.6
28.
19.7
13.5
32.9
15.7
28.4
Baseline
4P
27.7
36.8
pySpatial (Ours)
$1.6
28.7
27.2
20.9
41.2
38.6
46.9
39.4
46.0
38.2
36.4
37.3
A.2 MORE ABLATION STUDIES
Code Agents. In Table A2, compare the performance of our pySpatial framework when paired with different code agents. Across all categories, pySpatial consistently improves upon the base GPT-4o model, regardless of the underlying L.MM used for code generation. Among the evaluated agents, GPT-4o achieves the strongest overall performance, reaching 62.67% accuracy, while Qwen3-Coder and DeepSeek-v3 deliver comparable results 62.10% and 61.05% respec- tively. Notably, Qwen3-Coder performs best on the Around category, whereas GPT-4o provides the most balanced improvements across all task types. These findings indicate that our framework is robust to the choice of code agent and that most of the performance gains stem from the 3D visual programming paradigm rather than the specific code LLM used.
3D Reconstruction Backbones. In Table A3, we compare the impact of different 3D reconstruc- tion backbones on the performance our pySpatial framework. All three backbones, including VGGT, Pi3, and CUT3R, lead to substantial improvements over the GPT-4o baseline, indicating that our 3D visual programming paradigm is robust to the choice of reconstruction method. Among them, Pi3 achieves the best overall performance (63.33% with notable gains in the Rotation and Among categories. VGGT provides similarly strong results, while CUT3R performs slightly lower but still significantly surpasses the base model. These findings suggest that pySpatial can effec- tively leverage a range of modern reconstruction backbones, and its spatial reasoning improvements not tied to specific reconstruction architecture.
15

---

Published as a conference paper at ICLR 2026
Table A2: Performance comparison of pySpatial framework with different code agents.
Method
Overall
Rotation
Among
Around
GPT-4o
42.29
35.00
43.00
46.20
pySpatial w/ GPT-4o
62.67
41.00
66.33
71.20
pySpatial w/ Qwen3-Coder
62.10
40.00
64.50
74.00
pySpatial w/ DeepSeek-v3
61.05
40.50
65.33
64.80
Table A3: Performance comparison pySpatial using different 3D reconstruction backbones.
Method
Overall
Rotation
Among
Around
GPT-4o
42.29
35.00
43.00
46.20
pySpatial w/ VGGT
62.67
41.00
66.33
71.20
pySpatial w/ Pi3
63.33
43.50
65.66
72.33
pySpatial w/ CUT3R
61.05
40.50
64.33
69.60
Table A4: Comparison of pySpatial with different numbers of in-context learning examples.
Method
Overall
Rotation
Around Among
GPT-4o
42.29
35.00
43.00
46.20
pySpatial w/ examples
53.62
32.50
55.16
66.80
pySpatial w/ 2 examples
62.67
41.00
66.33
71.20
pySpatial w/ 4 examples
63.14
46.00
64.50
73.60
In-Context Learning Examples for Code Agents. In Table A4, we examine how the number of in-context learning examples influences the performance of our pySpatial framework. We observe a clear upward trend: providing even a small number of examples substantially improves performance across all categories. Using 0 examples already offers strong boost over the base GPT-4o model (53.62% vs. 42.29%), demonstrating that pySpatial can operate effectively even without demonstration guidance. Adding examples leads significant further gain, reaching 62.67% overall accuracy. Increasing to 4 examples yields the best performance (63. 14%). with notable improvements particularly the Rotation and Around categories. These results suggest that pySpatial benefits from additional examples, but even minimal in-context supervision is sufficient unlock strong spatial reasoning capabilities.
A.3 ADDITIONAL QUALITATIVE RESULTS
We present more qualitative results from MINDCUBE in Figure A1. Beyond the linear program flow demonstrated in Figure 2, pySpatial is also capable of executing more expressive control-flow operations, including for-loops, conditionals, and lambda-style functional compositions, allowing to construct complex multi-step 3D operations. These richer program structures enable pySpatial to iteratively explore candidate viewpoints, selectively apply geometric predicates, and adapt its reasoning based scene-dependent constraints. Across the presented examples, the 3D visual programs generated by our code agent consistently guide pySpatial toward responses that closely align with the ground-truth spatial relationships, further validating the correctness, interpretability, and robustness our 3D reasoning framework.
A.4 ADDITIONAL REAL- WORLD EXPERIMENTS
We also include an additional qualitative example challenging dynamic scene, where the views captured at different times and person is moving through the environment, shown in Fig- ure A2. Despite the temporal inconsistency and the presence of dynamic elements, pySpatial remains effective: the reconstruction module robustly integrates the multi-view observations and preserves the stable structural cues needed for accurate spatial reasoning. In contrast, GPT-5 contin- ues to struggle this scenario, failing generate safe or correct navigation trajectory and often producing instructions that are incompatible with the underlying scene geometry. These extended real-world experiments further validate the robustness and practical effectiveness of our approach.
16

---

Published as a conference paper at ICLR 2026
Query: From image then right and forward, get
door!
Ground Truth: Yes.
Pecomddentt the stome free (he unper omayc
3D Reconstruction:
base recestrrattes entrenetceii store Jren the sane poes leage
inntartal moture right, ("aestors sxppl/4ll turetia eight indeptted aove fonerrt ('dlstance", 0.3)), snberty flrward
Clues
recty recation serveret to cimuuete The plyciical tuminglaattee
Program Outpur: them
Losn
viseptort eninammune,, Mangll
censrote claser)
compore
change is appearance
cluee. onserrtal, syithectrv soeet xlew recostraatre, virepator))
Response: program's right turn larger and more prominent while the nearby dark chair-lkke object also appears bigger, indicating reduced distance.
transformation resulting synthesized images, doorway becomes
Query: viewpoint presented image 3, to
ball?
Ground
progren ICEN: JORNT Scemets Recono incontenctel potre spdpetted incorsrrctt www.ttettttittteeeeeeeee base viceette
Reconstruction:
free the forr proviscd
3- left rntectin Inansforntion limes (Promend Hew Jett", Lotto tellinate hel. tow tedeattlleeee
rinpulet.
fww.eely. eniop A.A1)
sheee pramested vieus kww.
Aplla Trans forettit in Leap.
compre one vinqulatt
Program Output:
Synthartte eavil viee free that troncforiad conare poes.
"our Jehe
"ranend thes Jere
Jert
growecry
uegge J's p0i6.
ipt worthealeddyyiee appentivion vice!.
retamn wyetwstsed, stee
Response: synthesized viewpoints display yellow-franme framed window on the wall.
object the (from viewpoint image 3)
Figure Al: More qualitative examples from MINDCUBE. We show that pySpatial enables MILMS explicitly reason within reconstructed, explorable 3D scene, allowing the model not only to interpret spatial structure but also to compose executable 3D visual programs with control flow, such for-loops robustly solve diverse spatial reasoning tasks.
Query: Suppose you looking the viewpoint corresponding to Image generate task specification and motion plan for navigating to the goal destination at Image 4.
Inpuc Limited Views
Reconstrurted 3D Stene Motion Plan
Ours GPT-5
Proposed Motion Plan from pySpatial (Ours)
Proposed Motion Plan from GPT-5
Tum lak 37.6", orianting yoursalf sightl toward the faceball cable and pingponn
white bartoooss then forward 7.54
Turn 45' to your right and walk meters forward toward the space beside the focsball table.
your
Turn right 84.2" so thet you face soward the red-fet pool rabl beyond the focaball area, then walk forward 7.01 my moving between the focaball
walk near the red pool table.
Turn 35" your
forward toward the
robiee and the couch area.
3.
Turn 80" your right and walk 2 meters forward toward the area with the guitar and chairs.
Turn right 72.8" algning view with the yallow ostoman, then wulk
forward 4.75 arriee
Figure A2; An additional qualitative real-world example in challenging dynamic scene.
17

---

Published as a conference paper at ICLR 2026
B IMPLEMENTATION FOR PYSPATIAL
complete description of the API interface provided in Code
import
import glab
from typing import List, Union
from pathlib import Path
taol.
import recomatruct_ad
from .eggnent inport segnent_inage; segnent_.utonatic
eetheteeiintee import eetimate_depth.
from tool cam.raa import analyze_camera. trajectoryy
from tool gove_forward, nove_backward,
import novel_riev.eyatteeis rotate_right, rotate_left,
import re
def ...init. (self, point_cloud, extrinsics, intrinsics, colors-Kone): self Saaf.point_clouee point_cloud
self extrinnitissice extrinsics list of +4 numpy array felffintrinsice intrinsics colorc
attribute
class Scene:
Simple scene class that holds inage data.
daf initt._ (self, (allf, path_to_inagas: Union [atr, Liat[str]] queation: str "", Xone):
scene
self self.question questton questioa self.econe_id scene_id
self colf.oriigial__ptt self.image. laff___odd_iaageeee imagge_)_
path_to__nages # Store original path for reconstraction
self.reconstraction Reconstruction Nane sode.code stx None
velf.visual.clue
imad__i (self, path_to_inagen aettoad image paths from diractory list.""s
(str, [str]]) List[str]:
if
isinstance (pinstan taab__to_inaee,,, str):
1t..ptt..dddrrpatt..t. # Check if this is demo directory (contains glb files) peth_path Path(path_ to_inge)))
glb_files list www.ooopth..glbb.... glob(("..bbb.)))
if glb_files
This is a deno diractory, load inages from color/ subdirectory deno_path "color"
if color_dir exiete
image Lmage_extenssone ['e.pag', 't.jpg', "t.jpeg") images = oraga_xxtansiona:
for inages. globlos (atr (color_dir), ext)))
images. egtend
return sorted sorted (inages)
print (f"Varning: Demo directory detected but no color/ subdirectory found apath_to_inaaeeeee raturn []
elae:
Regular directory, load all inages fron directory
inaga_extension. ['e.png', 'x.jpg', '*.jpeg inages
for ezt in inage extensions:: images. extend glob(es. path. pottttto ext)))
return sortad
else:
Single inage file return [path_to_ to__gess__
List of inage paths tpatt_to_inagee
class pyšpatial:
simpimll interface for 3D vision tools."*a nox
disable
18

---

Other tensor types Published as a conference paper at ICLR 2026
reconstruct Scene):
3D reconstruction from scane images."
Check if this is demo directory (contains gib files)
isinstance (scene. original_path. str) and os.path, indir is.ee... orrpiiesecc..iiiinl.p.a.... demo_path Path (scene. original_path)) gib_tiles list (demo-path... wet.hpppah..bb(ptte
glob("s.glb1)) "..n..pth..gooo(".ggbbb1
if gib_tilea:
This is deno directory, pass the directory path for demo data loading wwwsseee..orgiinll.............. originel_path, scene_id-scene. scene_id))
result Ceconetruct_ddeeeee
clse:
Regular reconstruction with image patha
result (scenee
acene_id-scom, cenn__ie-econ..ccoe...d acene_id)
Regular reconstruction with paths
result irconstruct_3ddeeee images, scene_id-scene ade
Convert the ras result dictionary to Recoastruction object point_claud
get g'po'nnta'', Nona)
cameras result ('ameers Kone)
result, get colore',
result
Convert point cloud to numpy if it's 8 tensor
if point_cloud is not None: if hasattr (point_cloud, 'cpu'): PyTorch tensor point_cloud point_cloud. cpu numpy
elif hasattr (ponntt_poottt "numpy' )); Other tensor types point_cloud point_cloue numpy
Convert colors to nunpy if it's tensor if colors None:
if aaaatt((colore 'cpu'): PyTorch tensor colore. () nunpy c
elif Masatir(colorz 'aumpy ) (auppy
colors.nunpy()
Extract extrinsics and intrinsics from cameras if available extrinsics None
intrinsica None
if cameras is not None:
extrinsice
cpp muuupy () if hasattr(caneras, 'cpu') else
Create and Reconstruction object with colors
reconstruction Reconstruction (point_ cloud, extrinsice, intrinsice, colors)
Store the rav result for debugging reconstruction.__rau. reautl
retura reconstruction
@staticmethed
describe_canee caarbee_aaaar_aaeeee namers_notion Reconstruction):
Describe canera notion fron reconstruction results. Arga: uns
extrinsics recan extrinsics trajecettoryrjjjoooorroooooo
@staticmethod
def seatheeiza_novel__leeeeree
Reconstruction, nav_camera_poat vidth=512, height
=512, out path
un Generate novel view synthesis fron reconetruction results.
Arga:
Reconetruction object
point_cloud, extrinsice, intrineics
cer_camers_pose: 3x4 or 4x4 extrinsic natriz for the nev viespoint vidth: output inage width (default: 512)
height: output inage height (default: 512)
out_path: output inage path (default: Bone, returns image object not
provided)
Returns:
str image: path rendered image out_path provided, othervise
image object
return vovel (recon, new_canera_poae sidth, height, out_path)
Dataticnethod rotate_right (extrinsic, angle-None):
19

---

Published as a conference paper at ICLR 2026
un Rotate canera pose right reture rithte_right (extrinsic, angle)
@staticnethod
def rotate_left (extrinsic, angle=None):
un Rotattee raturn lextt_ii angle)
pose to the left**"
dstaticmethod
def nove. forward (extrinsic, distance-None);
au Hove
pose forward""e
return gov__oorwardoxxtrinn distance)
estaticnethed
nove fackvard(xttrinall distance-Nene):
an Hove canera backvard"'s
return wwe_baackerd((ettinni. distance)
@staticmathed
turn_around (extrinsic):
Tur canera pose around 130 degreeg***
class Agent:
daf inint.. (self, api_kay str = None): key api_key getgete ('OPEMA__AILKKE')
generate_ coderate_odd(self scene: Scene):
from agent. codedgentt, import senerate_code_rt (rondee_rrruuerr return conerate_codo..roo_uuee (scene, self. self. api_key)
daf LLM one Extracts the first pythen code block '"'python "*) from tezt. string,
acane: Scane, reaponse: atr):
not found.
ans
from agent. codedgent. exocute inport parse_LLX_respann code parze_LiM__rsppnne (reppon))))))))) scene. code code
execute (self, scene: Scene):
Exacute a code string with scene and return the visual clue result. uus
try: agent codedgent execute import execute_code
progran execute_code (scene.code)
vieual_clue progran(scon))
return visual_cius
except Exception
import traceback
erzor_detaile f"Execution
(etrce)))atracebac:: (traceback.
(armat_oxc()
Store error detailed reporting
self paat_eexccttioneree error_detaile
return ththere is an error during code generation, no visual clue provided. (e))
agent. codedgent......o.c execute inport execute_code progran cocutt_ccdeesccme code)
visual_clue progran (scene) return visual_clue
def (sele, scene:
Scene, visusl_clue): question from agent-anzer inport ansver
Set the vieusl clue in the scene.
viaual_clue
Call the ansver function with API key return ansver (acane, self apt_ky)
Code Full pySpatial API specification.
20

---

Published as a conference paper at ICLR 2026
C IMPLEMENTATION DETAILS OF THE AGENT PROMPT IN PYSPATIAL
present prompts used our experiments box below.
AGENT PROMPT IN PYSPATIAL
task faak_description aun
You are now asked to solve spatial reasoning related problem. The input are image(s) and a natural langugae question that specifically designed to test your spatial reasoning ability. It is not trivial to solve these tasks directly as a vision
langugae model. However, You have access to the following Python API UnH
api upi_specification
In the PySpatial API, we explicitly introduce We provide a Scene class that contains the image (s) and a question. Further, we also provide used to generate a 3D point cloud and camera parameters.
inductive bias.
reconstruction process that can
class Reconstruction:
def __init__ (self, point_cloui extrinsics, intrinsics): self self-point_clour point_cloud belf.extrinse extrinsics self intrinsics intrinsics
class Scene:
"Simple scene class that holds image data." def (senf, sath_to_images: Union [str, [str],
question: str ""):
self question question
self self. images self._load loaad_imageessseeeeeeeeessasseee (path soatth_tt__iie ) images)
self relf.reconstruction Reconstruction None
def @oad_images
Union[str, [isr]])
-> List[str]:
"Load image paths from directory list."
if isinstance (path_ to images, str): if os. (
# Load all images from directory imagg_extensioon images = [
['*.png', '+.jpg', 'x.ee'']
for extend woooollloboss....o glth(os.path..oon path.join( images
sange__xtenssoos::
path_to to sech_to_image,, ext)))
return sorted(images, (images)
else:
# Single image file return (path_to__magee
else:
# List of image paths
return list (path_ imagges
class pySpatial:
"Simple interface for 3D vision tools." we disable other function for now
21

---

Published as a conference paper at ICLR 2026
@stati
def reconstruct (scene: Scene)
"3D reconstruction from scene images." "
return Reconstruct__ddseeeee images)
@staticmethod
def aesoribe_camerammoiionn
Reconstruction):
"Describe camera motion from reconstruction results. Args:
extrinsics = recon. recon.extrimaaa extrinsics return describe_ cat.erramooooiiineeriiineee
@staticmethod
(recon def prnthesize_novel_vie (recon: Reconstruction,
new_camera_pose)
"Generate novel view synthesis from reconstruction results.
Args:
return novel
# methods to manipulate camera pose
def rotate_rigtt(eextiinttt oagle=nppi//9
rotate_left iggl=np.pi./ pi/2):
def move.forvarc(extrinsiic,, distance=0.15 1):
def move_backwardlexxr.nniiii,, distance=O. .1):
def turn_aroum(exttrinss))))
@staticmethod def estimate_des _eteeeddephhthee (image): return estimate_depal deteeddeepppipiiinie (image)
in-context learning exmaples
Problem 1:
Question: "Based on these two views showing the same scene: in which direction did I from the first view the second view?
Diagonally forward and left
B. Directly right
Directly left
D. Diagonally forward and right"
How to solve this problem?
Step 1: we can easily find the ansewr with camera extrinsics. should first reconstruct the scene, and then use the camera extrinsics to find the answer. Step 3: it is still not trivial to directly get the answer from extrinsic matrix.
Step 2: therefore,
Step 4: can use the
22

---

Published as a conference paper at ICLR 2026
to get the answer.
Next, write python code within the pySpatial API provided, then agent will automatically collect the code I wrote and execute it.
python
def reconstruction3D Scene)
(intt camera_motion pyšpatial ctt.orrrbe_cmmmrr_mmtti.... reconstruction3) return camera_motion
Step 5: After get the visual clue from execution, I can easily match the answer:
Problem 2:
Based on these four images (image 1, 2, 3, and 4)
showing the pink bottle from different viexpoints (front, left, back. and right), with each camera aligned with room walls and partially am standing at the same spot and facing the same direction as shown in image 1, then I turn right and move forward, will get closer to the pink plush toy and headboard?
capturing the surroundings:
since do not have the way to compare distance in 3D space, we can render two images, and use these two images as visual clue. "'pyhhon
def program (pogram(input__scene Scene)
reconstructed_scent pySpatiial reconstruct (inputt_nnnnntttte base_vid viisevveennntt reconstructed_scee _sceedes_eeeeeeeeeeeeeeeee extrinsics [0]
# the image indicates the 0th index in the array
viespoint
(base_viewel viieevviewnnnt
dlespointtmove__eel
pySpatial.move movve_forward (viespoint_tum_rriitt))
image_right
yySpttial.sycchasse...ooaa...
secconstruuctted_cceeee
Image_forward Segpaatial.ssmttesiiie...oo
dicconstructe_sccet tlespoint_mvve_for
# we should compare these two images, check if the object exists and the distance closer.
visual_clue = [image_right, image_forward] return visual_clue
code ceaie-gnnration_promeeee f"""
Now please utilize the PySpatial API and write python function to solve the problem. Noted that you first do reasoning and then write the code. But the code should be wrapped in the ""python block. Write compact code block
23

---

Published as a conference paper at ICLR 2026
Also, the function written should be named program and the input parameter should be a Scene object. for example, "pyyhhn
def programiinput_aceo Scene)
return
try add simple comments the code to explain your logic.
Make sure to first reasoning, why we write program like this, becuase we have pySpatial API that allows us to explore the 3D space, please first do a reasoning like (I want to know what is to the right of something, therefore just render novel view from that).
Prompt template for ReAct: ReAct: Synergizing Reasoning and Acting in Language Models
answer_backge auswer_background = f"""
We are now solving a spatial reasoing problem.
It is not trivial to solve these tasks directly as vision language model.
However, We have access to the following PySpatial API: eapi_epecification)
We generate a python code based on the PySpatial API to solve this problem.
answer_proom
Based on the code and the visual clue from the execution, answer the question.
unn
Prompt for the answer without visual clue wi ttchout_visual_clle cuugbuigouuueebbckuuuuuu =
Solve this spatial reasoning problem based on the question and the image input.
First, analyze the question, extract useful information from the question description, then try to answer it based on the useful visual information.
Give your best guess you cannot find the best answer.
D THE USE OF LARGE LANGUAGE MODELS (LLMs)
We employed LLMS solely auxiliary tool polish the writing of this manuscript. They were used to improve grammar, clarity, and readability, but no LLMs were involved in ideation, data analysis, experiment design, or result interpretation.
24


Reviews:
[{"review_id": "YksDcF01pd", "reviewer": "Reviewer_ZMiG", "summary": "This paper presents pySpatial, a visual programming framework that enables MLLMs to perform explicit 3D spatial reasoning. The core contribution is a Python-based API that allows MLLMs to compose spatial tools—including feed-forward 3D reconstruction, camera pose estimation, and novel view synthesis—through code generation. Given sparse image views and natural language queries, the system generates executable programs that transform 2D observations into explorable 3D representations, enabling geometric reasoning rather than relying on implicit mental models.\nThe framework operates in a zero-shot setting without gradient-based fine-tuning. Experimental results demonstrate substantial improvements over MLLM baselines on MindCube (+12.94% over GPT-4.1-mini) and Omni3D-Bench benchmarks. The authors also provide qualitative validation through real-world robot navigation experiments. The interpretability of generated programs and modular tool composition are key practical advantages.", "questions": "1. Your main results use GPT-4o/GPT-4.1-mini. Can you report results using fully open-source code generation models (e.g., CodeLlama-34B, DeepSeek-Coder-33B, or recent Qwen-Coder models) for both program synthesis and final answer generation? This would: (a) Improve reproducibility for researchers without API access; (b) Clarify how much performance depends on proprietary model capabilities; (c) Identify minimum model requirements for the framework to be effective. Even negative results would be valuable for the community.\n\n2. Please clarify the technical pipeline for camera parameters: (a) For VGGT (up-to-scale), how do you handle scale ambiguity in Equation 1 and the camera motion description (Section 3.2)—is scale normalized per scene, or are only relative directions used? (b) How are camera intrinsics K determined—estimated by reconstruction models, from metadata, or assumed? (c) For navigation with CUT3R, what is the sensitivity to metric scale errors (e.g., ±10-20%)? (d) How are camera extrinsics aligned to a consistent world frame for scenes with limited overlap? This is critical for reproducibility.\n\n3. Can you provide: (a) Confidence intervals or bootstrapped standard errors for the main benchmark results (Tables 1-3)? (b) Statistical significance tests comparing pySpatial against key baselines (e.g., GPT-4.1-mini, VADAR)? (c) Ablation on prompt design—how sensitive is performance to the choice of in-context examples, API documentation verbosity, or structured output formatting? (d) Results with multiple random seeds or temperature settings to quantify variance? This would increase confidence that reported improvements are robust.\n\n4. Given that 13% of failures stem from reconstruction quality (Section 4.5), can you provide systematic analysis of performance under varying input conditions? Specifically: (a) Performance curves as a function of number of input views (e.g., 2, 4, 6, 8 views); (b) Impact of view overlap percentage on reconstruction and final task accuracy; (c) Comparison between VGGT, DUSt3R, and CUT3R on the same MindCube subset; (d) Scene characteristics that most correlate with reconstruction failures (texture, lighting, clutter level)? This would help users understand when pySpatial is reliable.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1. Novel and Well-Motivated Problem Formulation\n\nThe paper addresses a clearly identified limitation in current MLLMs regarding 3D spatial reasoning from limited views (Section 1, lines 54-65). While recent works like SpatialVLM and SpatialRGPT focus on single-view spatial understanding, this work tackles the more challenging multi-view setting where models must reason across perspectives. The visual programming paradigm is well-suited to this problem, allowing flexible composition of spatial tools through a clean Python API (Code 1, Section 3.2).\n\n2. Strong and Consistent Empirical Results Across Multiple Benchmarks\n\nOn the challenging MindCube benchmark (Table 1), pySpatial achieves 58.56% overall accuracy, substantially outperforming the best open-weight model DeepSeek-VL2-Small by 10.94% and the strongest proprietary baseline GPT-4.1-mini by 12.94%. The results are particularly impressive on the \"Among\" category (60.54%), where all baseline approaches fail to exceed 50%—this category requires reasoning about how a central object relates to all surrounding objects, a challenging spatial task. Notably, pySpatial outperforms VLM-3R by 16.5% while operating entirely zero-shot. On MindCube-1k (Table 2), pySpatial with GPT-4o reaches 62.67%, outperforming spatial mental models by ~20%. The framework also generalizes to single-view settings, achieving state-of-the-art on Omni3D-Bench among visual programming approaches (+3.8% over VADAR, Table 3) and even surpassing GPT-4o on total score.\n\n3. Zero-Shot Operation with Broad Generalization\n\nA key practical advantage is that pySpatial requires no gradient-based fine-tuning (lines 47-48, 106-107). The ablation study (Table 4, Section 4.5) demonstrates that augmenting different MLLMs with pySpatial consistently yields substantial improvements: GPT-4o improves from 42.29% to 62.67%, GPT-4.1-mini from 43.34% to 58.19%, and GPT-4.1 from 44.67% to 63.42% on MindCube-1k. This plug-and-play nature makes the framework broadly applicable without requiring expensive task-specific training, and the consistent gains across different code agents validate the robustness of the approach. The framework also successfully transfers from multi-view (MindCube) to single-view (Omni3D-Bench) settings, demonstrating versatility.\n\n4. Interpretability, Transparency, and Reproducibility\n\nUnlike black-box MLLM approaches, pySpatial generates executable Python programs (Figure 2) that are human-readable and include well-structured comments explaining the reasoning process (e.g., \"# Step 2: rotate the camera to the right from viewpoint 2 to see what is on the right side of the black chair\"). This interpretability allows researchers to inspect, debug, and modify generated programs. The API is thoroughly documented (Code 1 in main paper, full specification in Appendix A Code 2), and prompts are provided in Appendix B (lines 892-1079). The modular design separates high-level reasoning from low-level execution (lines 220-223), and the paper commits to releasing code upon acceptance (line 266). Implementation details are comprehensive, including specific model choices (VGGT, CUT3R), rendering backend (Open3D), and computational setup (single A6000 GPU, Section 4.1).\n\n\n5. Comprehensive Experimental Evaluation and Analysis\n\nThe evaluation is thorough and multi-faceted: (1) Benchmark diversity: Tests on both multi-view (MindCube) and single-view (Omni3D-Bench) benchmarks; (2) Baseline breadth: Compares against open-weight MLLMs, proprietary models, specialized spatial models, and prior visual programming approaches (Tables 1-3); (3) Qualitative analysis: Figure 2 shows diverse examples with generated programs, 3D reconstructions, and outputs; (4) Ablation study: Table 4 validates effectiveness across different code agents; (5) Failure analysis: Manual examination of ~100 samples (Section 4.5, Figure 4) attributes errors to program generation (6%), 3D reconstruction (13%), and final reasoning (20%), providing transparency about system limitations; (6) Efficiency analysis: Reports 7.45s per query on MindCube-1k, competitive with VADAR's 17.25s (Section 4.5); (7) Real-world validation: Indoor robot navigation experiment (Figure 3, Section 4.4) demonstrates practical applicability.", "weaknesses": "1. Limited and Non-Rigorous Real-World Validation\nWhile Section 4.4 presents robot navigation as evidence of practical effectiveness, the evaluation is limited: (a) Qualitative only: No quantitative success rates, path efficiency metrics, or safety margins are reported; (b) Single environment: Testing appears confined to one 50m² two-room laboratory; (c) Manual intervention: High-level position commands are \"manually converted into temporal velocity targets\" (lines 375-377), reducing the autonomy claim; (d) Limited comparison: Only one baseline (GPT-4.1) is tested, and failure modes are not systematically analyzed. The paper states the robot \"successfully traverses complex environments\" (line 46), but Figure 3 shows only one trajectory. To strengthen claims of \"practical effectiveness,\" the authors should provide: success rates over multiple trials/environments, collision avoidance validation, automated planning-to-control integration, and discussion of failure recovery mechanisms.\n\n\n2. Reconstruction Quality as Performance Bottleneck with Insufficient Analysis\nThe failure analysis (Section 4.5, Figure 4) reveals that 13% of errors stem from 3D reconstruction quality, and the paper acknowledges this dependency. However, the analysis lacks depth: (a) No sensitivity studies: How does performance vary with 2 vs 4 vs 8 input views? What about different view overlap percentages or baseline distances? (b) Scene characteristics unexplored: Which scene properties (low texture, reflective surfaces, repetitive patterns, dynamic objects) most affect reconstruction quality? (c) Backend comparison incomplete: VGGT is used for most experiments, CUT3R for navigation, but no systematic comparison between VGGT/DUSt3R/CUT3R on the same multi-view tasks; (d) Mitigation strategies absent: Are there ways to detect reconstruction failures (e.g., uncertainty estimation) or fallback mechanisms when reconstruction quality is poor? The statement \"advances in 3D reconstruction... hold the potential to further enhance our performance\" (lines 469-470) is true but sidesteps the question of current robustness limits.\n\n\n3. Heavy Reliance on Proprietary Models Limits Reproducibility\nMain results depend critically on GPT-4o/GPT-4.1-mini (Table 1 uses GPT-4.1-mini \"due to budget constraints,\" line 273; Table 2 and most analyses use GPT-4o). While Table 4 shows the framework works with different code agents, all are GPT-4 series models. The paper lacks: (a) Open-source code generation results: No experiments with CodeLlama, DeepSeek-Coder, or other fully open models for program synthesis; (b) Reproducibility concerns: Proprietary model APIs can change behavior across versions, making exact reproduction difficult; (c) Cost analysis: No discussion of API costs versus open alternatives; (d) Sensitivity to model choice: Beyond the three GPT-4 variants shown, how much does performance degrade with weaker but open code generators? This dependency limits the \"zero-shot\" and \"plug-and-play\" accessibility claims for researchers without API access or budgets.\n\n\n4. Insufficient Statistical Rigor and Evaluation Details\nThe quantitative results lack statistical validation: (a) No confidence intervals: Tables 1-3 report point estimates without error bars or standard deviations; (b) No significance testing: Claims of improvement (e.g., \"+12.94% over GPT-4.1-mini\") are not tested for statistical significance; (c) Limited failure analysis scale: Only \"about 100 samples\" manually examined (line 460), representing <1% of MindCube and <10% of MindCube-1k; (d) Baseline parity unclear: No discussion of whether baselines use identical prompts, temperature settings, or number of trials; (e) Prompt sensitivity unstudied: The approach relies heavily on in-context examples (Appendix B), but no ablation on prompt variations or example selection is provided. Given that results are based on sampling from language models (which can be stochastic), statistical rigor is essential for verifying claimed improvements are robust rather than within noise margins.\n\n\n5. Underspecified Calibration and Scale Handling\nCritical details about camera calibration and scale are missing or unclear: (a) VGGT scale ambiguity: VGGT produces \"up-to-scale\" reconstructions (line 182), but Section 3.2's camera description assumes computing actual displacements (equations 1, line 191). How is scale determined or normalized for motion categorization? (b) Intrinsics estimation: Camera intrinsics K appear in Equation 1, but it's unclear whether these are estimated by the reconstruction model, extracted from metadata, or assumed as default values; (c) Coordinate frame alignment: How are multiple views aligned into a consistent world frame, especially for scenes without significant overlap? These technical details are important for understanding when the approach will succeed or fail and for enabling reproduction.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "VQYG9AAZlW", "reviewer": "Reviewer_Rgza", "summary": "This paper proposes pySpatial, a zero-shot visual programming method to perform multi-view spatial reasoning techniques. The model calls a set of pre-existing APIs best on the reasoning of MLLM to output the final answer for spatial reasoning tasks.", "questions": "Please refer to the weakness section for my main concerns. I would like to further understand the limitations of VADAR in terms of the complexity of the programs that it can generate, as well as more benchmarks to show the range of 3D spatial reasoning tasks it can perform.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "In general the paper is written quite clearly. The method is well described, and Figure 1 and is quite clear in terms of describing the differences with most spatial mental models and pySpatial. The model is compared on several datasets (MindCube, Omni3D-Bench) to show the effectiveness.\n\nThe problem of spatial reasoning with MLLMs is also an important and relevant task in the community.", "weaknesses": "I have some questions on the actual effectiveness of the visual program set up. From my understanding, and from all the results shown in the paper, pySpatial lists out the procedures of calling external APIs. It does not do additional complex actions (e.g., loops, if/else, etc) beyond a sequence of API calling. Are there cases where the question answer requires more than a linear sequence of API calling and if so can we see several of these examples? If not, this makes me question whether explicitly converting it to program language code really helps with the set up or just a list of action items is fine.\n\nIn addition, one of the main differences between VADAR and pySpatial is that VADAR generates the APIs required for a particular task, while in your case a set of pre-defined APIs are set. I wonder if the set of API limits the type of spatial reasoning tasks you can perform. For example, I wonder if the model would still perform better on tasks such as absolute metrics such as distances and volumes estimation, where the reconstruction itself isn’t explicitly providing more information. It would also be great to see the results on more benchmarks, e.g., MMSI-Bench, All-Angle Bench, etc. It would also be great if you can highlight a bit more on the architectural difference of VADAR and pySpatial.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "1J2Sab5EqS", "reviewer": "Reviewer_Xvqg", "summary": "The paper introduces pySpatial, a visual programming framework designed to enhance the 3D spatial reasoning abilities of multimodal large language models (MLLMs). While existing MLLMs perform well on general perception and reasoning tasks, they struggle with understanding spatial relationships in three-dimensional environments. pySpatial addresses this limitation by allowing MLLMs to generate and execute Python programs that call spatial tools to transform 2D image inputs into explorable 3D scenes. This enables models to reason explicitly over geometric structures without additional fine-tuning, operating entirely in a zero-shot manner. The method builds on recent advances in feed-forward 3D reconstruction and visual programming, integrating them into a unified, interpretable API for spatial reasoning. Experiments on the MINDCUBE and OMNI3D-BENCH benchmarks show that pySpatial outperforms strong MLLM baselines like GPT-4.1-mini. The framework also demonstrates practical effectiveness in real-world indoor navigation tasks, where a robot can successfully generate and follow route plans derived from pySpatial’s reasoning outputs.", "questions": "1. Could pySpatial be integrated with or further enhance the spatial reasoning capabilities of specialized spatial MLLMs (e.g., VLM-3R or Space-Qwen)? Specifically, would combining explicit program-based reasoning with models already trained on 3D-augmented data yield additional improvements?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The paper clearly identifies a relevant gap between current MLLMs’ implicit, imagination-based spatial reasoning and the need for explicit geometric grounding. The motivation is reasonable and reflects an active research direction in improving spatial understanding for embodied and multi-view settings.\n\n2. The proposed visual programming framework is well-structured and methodologically sound. Its modular API design offers a clear and interpretable mechanism for integrating 3D reasoning tools with MLLMs. This contributes to transparency and reproducibility, though the overall paradigm follows existing visual programming approaches.\n\n3. The framework demonstrates consistent zero-shot performance gains on both multi-view (MINDCUBE) and single-view (OMNI3D-BENCH) benchmarks. These improvements, achieved without any fine-tuning, indicate that the method is generally effective as a plug-and-play enhancement for existing MLLMs.\n\n4. The experimental evaluation is thorough, with comparisons spanning open-weight models, proprietary systems, and prior visual programming baselines. Additional ablation and failure analyses help clarify the contribution of each component and provide a more complete understanding of the system’s strengths and limitations.\n\n5. The experimental evaluation is thorough, with comparisons spanning open-weight models, proprietary systems, and prior visual programming baselines. Additional ablation and failure analyses help clarify the contribution of each component and provide a more complete understanding of the system’s strengths and limitations.", "weaknesses": "1. While the integration of 3D spatial reasoning within a visual programming framework is well-executed, the core concept of using generated Python code as an intermediate reasoning layer is not entirely novel. Prior works such as VisProg, ViperGPT, and VADAR have explored similar paradigms for visual reasoning. The novelty here primarily lies in extending this paradigm to 3D tools rather than introducing a fundamentally new reasoning mechanism. Consequently, the conceptual contribution may be perceived as incremental in scope.\n\n2. The real-world navigation experiment, though interesting, is relatively limited in scale. It consists of a single scenario under controlled conditions. Additional experiments across varied environments, different robot embodiments, or more complex navigation tasks would significantly strengthen the claims regarding real-world applicability and robustness.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "GwvKaNoQn8", "reviewer": "Reviewer_FeNP", "summary": "The paper proposes an agentic prompting method to improve spatial understanding tasks with explicit 3D representations and achieves suprior performance than existing MLLMs.", "questions": "The explicit 3D modeling method for spatial understanding may be limited to static environments. While in real-world robot navigation tasks, how much will changes in the environment affect the overall performance?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "- The paper presents a novel agentic method, incorporating state-of-the-art 3D foundation models for spatial understanding tasks.\n- The proposed method achieves suprior performance comparing to other methods.\n- Real world application demonstrates the effectiveness of the whole system.", "weaknesses": "- Since the paper requires the model to explicitly reconstruct the 3D scenes, render novel views, and navigates inside the reconstructed scene. Is it sensitive to the reconstruction methods, *i.e.* incorporating 3D foundational models other than VGGT? \n\n- Is the proposed method sensitive to the rendering quality? In Figure 2 we see holes in the rendered images. Is VLMs sensitive to those kind of inputs? Is there anyway to compensate the visual gap? Would gaussian splatting be a better choice? \n\n- Is the method sensitive to the code agent choice? Table 4 mainly compares the performance of the GPT family. What about other VLMs, will coding VLMs perform better than general purpose VLMs?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)