Reasoning in Space via Grounding in the World
Authors: Yiming Chen
Zekun Qi
Wenyao Zhang
Xin Jin
Li Zhang
Peidong Liu
Venue: ICLR 2026 Poster (2026)
Keywords: 3d spatial reasoning
3d visual grounding
Primary area: foundation or frontier models, including LLMs
TL;DR: 3D visual grounding is one of the cornerstones of spatial reasoning

Abstract: In this paper, we claim that 3D visual grounding is one of the cornerstones of spatial reasoning and introduce the $\textit{Grounded-Spatial Reasoner (GS-Reasoner)}$ to explore the effective spatial representations that bridge the gap between them. Existing 3D LLMs suffer from the absence of a unified 3D representation capable of jointly capturing semantic and geometric information. This deficiency is manifested either in poor performance on grounding or in an excessive reliance on external modules, ultimately hindering the seamless integration of grounding and spatial reasoning. To address this, we propose a simple yet effective \emph{dual-path pooling} mechanism that tightly aligns geometric features with both semantic and positional cues, constructing a unified image patch-based 3D representation that encapsulates all essential information without extra tokens. Leveraging this holistic representation, GS-Reasoner is the first 3D LLMs that achieves autoregressive grounding entirely without external modules while delivering performance comparable to state-of-the-art models, establishing a unified and self-contained framework for 3D spatial reasoning. To further bridge grounding and spatial reasoning, we introduce the $\textit{Grounded Chain-of-Thought (GCoT)}$ dataset. This dataset is meticulously curated to include both 3D bounding box annotations for objects referenced in reasoning questions and step-by-step reasoning paths that integrate grounding as a core component of the problem-solving process. Extensive experiments demonstrate that GS-Reasoner achieves impressive results on 3D visual grounding, which in turn significantly enhances its spatial reasoning capabilities, leading to state-of-the-art performance.

Published as a conference paper at ICLR 2026
REASONING IN SPACE VIA GROUNDING IN THE WORLD
Yiming Chen 122,3
Zekun Qi
Wenyao Zhang
Xin Jin 6
Li Zhang
Peidong Liu'-
Westlake University
Shanghai Innovation Institute
*Zhejiang University
"Tsinghua University
Shanghai Jiao Tong University
"Eastern Institute of Technology Fudan University
Project Page
Github Code
Huggingface
ABSTRACT
In this paper, claim that 3D visual grounding is one of the cornerstones of spatial reasoning and introduce the Grounded -Saittaa-paaaall Reasoner Res-oeas))))) explore the effective spatial representations that bridge the gap between them. Existing 3D LLMs suffer from the absence of a unified 3D representation capa- ble of jointly capturing semantic and geometric information. This deficiency is manifested either in poor performance on grounding or in an excessive reliance on external modules, ultimately hindering the seamless integration of grounding and spatial reasoning. To address this, we propose simple yet effective dual- path pooling mechanism that tightly aligns geometric features with both semantic and positional cues, constructing unified image patch-based 3D representation that encapsulates all essential information without increasing the number in- put tokens. Leveraging this holistic representation, ES--easonner is the first 3D LLM that achieves autoregressive grounding entirely without external modules while delivering performance comparable to tate-of-the--rt models, establishing unified and self-contained framework for 3D spatial reasoning. To further bridge grounding and spatial reasoning, we introduce the Grounded Chain-of ofTThhggh (GCoT) dataset. This dataset is meticulously curated to include both 3D bounding box annotations for objects referenced in reasoning questions and step-by-step rea- soning paths that integrate grounding as core component of the problem-solve process. Extensive experiments demonstrate that GS-Reasoner achieves impressive results on 3D visual grounding, which in turn significantly enhances its spatial reasoning capabilities, leading to state-of-the-an performance.
INTRODUCTION
Visual-spatial intelligence encompasses the capability to perceive, interpret, and reason about 3D spaces, including the spatial layouts, object sizes, positions and their potential interactions. This skill is fundamental to various domains, such embodied intelligence and autonomous driving. Accurately linking 3D objects with textual descriptions, a task known as 3D visual grounding, is prerequisite for effective spatial reasoning. This aligns with human cognitive processes, where identifying relevant objects is a fundamental step before reasoning about their spatial relationships. Despite recent advancements in 3D large language models (LLMs) (Cheng et al., 2024; Cai et al., 2024; Zhou et al., 2025; Zheng et al., 2025; Wang et al., 2025b; Hong et al., 2023a; Chen et al., 2024a; Huang al., 2023b; 2024; Zhu et al., 2024b), 3D LLMs still rely on pretrained 3D detectors or external decoders for grounding. This reliance not only limits their ability to fully understand 3D scenes but also impedes the cohesive integration of grounding and spatial reasoning. Therefore, critical question arises: How can we enable 3D LLMs to perform natural and effective grounding in an autoregressive manner, thereby enhancing their spatial reasoning capabilities?
We identify two primary challenges in grounding enhanced spatial reasoning. The first challenge arises from the inherent complexity of 3D data. Unlike 2D images, point cloud-based 3D scenes encode rich spatial relations and depth cues that are difficult capture and align with the semantic space of LLMs, especially given the scarcity of large-scale 3D datasets. Moreover, representing
"Corresponding authors.

---

Published as a conference paper at ICLR 2026
VSI-Bench
Relative Spatial Reasoning
Absolute Spotial Reasoning
of standing by the feont-Sert frestiright, sece-loff,, or
Viatt is the dimension largth
certiester
8.43,-3 15 as Lengled di (..
colcako The boueding toens relevant meaitor (-2.2277.23 21, 16, 2.31 2.31,7 7.M0
the Iergest conertee lengthiz Mie.. Conertt to 2hice
lecal coortinte postan are censwayy rea right
Wan te dletance betwees treak zin and
General 3D Tasks
wiich there of abjects tretos, sin) is the closest to
ctttato boanding boans relevaet 48 2011
5-86, 1,32), poaker (-2.31,2 22.30
200 2313
<1ocako bounding trees relevent reast (6.10,-) 1t. 1.......... b.85)
The mirdnun dietance benweer trach bis sertiger gert), sheret greaact trast atruete nees,, pissil trast atn of pacter Jaxcs, 7:25,
The sallleet dietanose par category anes teash briedd do. bn con cludky masot his smillest distance to plart.
121100
Figure I: We propose GS-Reasoner, which integrates visual grounding as an intermediate chain-of- thought for spatial reasoning. All the bounding boxes shown above are autoregressively derived by GS- ES-Reasoner the reasoning process. Notably, the showeased video is captured in the wild without sensory 3D inputs, highlighting the strong generalization capability of our model.
such fine-grained structures often requires substantially larger number of tokens, further increasing modeling cost. Previous works (Hong et al., 2023a; Chen et al., 2024a) compress point cloud features with Q-former, while others (Fu et al., 2025; Huang al., 2025b) adopt voxel-based representations to better preserve structure. However, these methods typically trade geometric fidelity for token efficiency, and the extracted point cloud features contain only limited semantic information, making accurate grounding and reasoning difficult. More recent approaches (Zheng et al., 2025; Zhu et al., 2024b) encode 3D positional cues into video-based semantic features from vision foundation models, showing promising 3D reasoning benefits from visual LLM pretraining. Nevertheless, the geometric cues derived solely from 3D position encodings are weak, which constrains grounding performance. The second challenge lies in the lack of high-quality datasets that integrate grounding as an intermediate step for spatial reasoning. Existing 3D VQA datasets (Azuma et al., 2022; Ma et al., 2022) provide only short answers without grounding annotations reasoning steps, making the combination of grounding and reasoning impossible. Additionally, these datasets fail to capture the contextual richness and structural complexity required for comprehensive spatial reasoning, further limiting progress toward robust 3D LLMs.
In this work, we propose novel approach to address the identified challenges by introducing comprehensive 3D seene representation and a GCoT dataset for spatial reasoning. Our 3D scene rep- resentation integrates semantic features from vision foundation models, geometric features encoded by a point cloud encoder, and 3D positional information. The key idea is to unify these heterogeneous signals within an image patch-based representation. Specifically, we pool the geometric features of point maps in a -ual-path way to align them with the corresponding semantic feature and 3D position of the image patch, and subsequently fuse them into a unified hybrid representation. This hybrid representation preserves the strong generalization ability of LLMs gained from visual-semantic pretraining, while the incorporation of geometric information significantly strengthens its 3D scene comprehension. As result, -S-Reasoner can accurately locate objects without relying on any exter- nal modules, which provides a natural intermediate step for spatial reasoning. To train models capable of handling both tasks, we construct the GCoT dataset. It includes precise 3D bbox annotations for objects mentioned in reasoning questions, along with step-by-step reasoning paths that embed grounding as core component of problem solving. By structuring the tasks in this way, the dataset encourages models to first identify relevant objects before addressing complex spatial reasoning, yielding more interpretable and cognitively aligned approach to learning spatial reasoning.
We propose comantic-geometric hybrid 3D scene representation that endows LLM with strong geometric priors, firstly enabling LLM to autoregressively perform 3D visual grounding with impressive results.
2

---

Published as a conference paper at ICLR 2026
We introduce the GCoT dataset, which bridges the gap between grounding and spatial reasoning, enabling Re-Reasoner to first ground objects and then reason about their spatial relationships in manner aligned with human cognition.
We demonstrate the effectiveness of GS-Reasonee through extensive experiments, showcasing its remarkable performance in both 3D visual grounding and spatial reasoning tasks.
2 RELATED WORK
3D Large Language Models for 3D Understanding. Recent advances in MILMS have enabled 3D LLMs that integrate 3D information for tasks such as 3D VQA, visual grounding, and captioning. Early work 3D-LLM (Hong et al., 2023a) introduces Q-Former to align point cloud features with LLMs, followed by studies (Chen et al., 2024a;b; Zhu et al., 2024a; Deng et al., 2025) constructing 3D representations with controllable token lengths. Voxel-based approaches (Fu et al., 2025; Huang et al., 2025b) balance token efficiency and geometric fidelity, while object-centric methods (Huang al., 2024; 2025b; Yu al., 2025) improve 3D scene understanding but lack global context. Recent works (Zheng et al., 2025; Zhu et al., 2024b; Wang et al., 2025b) propose encoding 3D positional information into visual features extracted by vision foundation models, achieving promising results on 3D tasks while maintaining the generalization ability of visual LLMs. Despite these advances, existing 3D L.LMs still struggle to jointly capture semantic and geometric information from 3D) scenes, limiting performance on 3D visual grounding or forcing reliance on external modules.
Video-Language Models for Spatial Reasoning. The goal of visual- spatial intelligence is to equip video MLLMs with the ability to understand and reason about 3D spatial structures directly from video data. While Video-Language Models (VLMs) (Lin et al., 2023; Li et al., 2024; Bai et al., 2025; Liu et al., 2024; Chen et al., 2024c) perform well on video-language tasks, they still show limited results on recent spatial reasoning benchmarks (Yang et al., 2025). Spatial- MLLM (Wu et al., 2025a) and VLM-3R (Fan et al., 2025) enhance spatial reasoning by incorporating geometric features from recent developed visual geometry models (e.g.. VGGT (Wang et al., 2025c)) and constructing large-scale spatial reasoning QA pairs for training. However, the constrained answer formats, such as single-choice selections or short numerical responses, potentially limit the ability of MLLMs to fully exploit the rich 3D information encoded in the geometric features of visual geometry models.
3 GS-REASONER FRAMEWORK
3.1 OVERVIEW
Given a sequence of N RGB images {1; E R3xHXW Ji=1 of a 3D scene and spatial reasoning query Q. our goal is to build a model that can first identify all objects potentially relevant to Q and then perform step-by-step spatial reasoning in autoregressive manner to derive the final answer. Depth maps (D; E R HW i. camera intrinsics E R3x3 and extrinsics {T; E IR [ fi=1 are assumed available or be estimated using visual geometry methods (Maggio al., 2025).
As illustrated in Fig. 2 (a) and (b), the proposed GS- ESS-Reasoner framework comprises three main components: semantic encoder, geometric encoder, and video LLM. The semantic encoder extracts rich semantic features from the input RGB images using a pre-trained vision foundation model. Meanwhile, the depth maps are back-projected into point maps {P. E R J.N which are subsequently transformed into geometric and 3D positional features. Specifically, the geometric encoder processes the aggregated sensor point cloud P = U P,, where P IR M x denotes M 3D points, to capture structural information of the scene. Since the geometric features are permutation- invariant and thus lack explicit positional cues, we further position-encone points. Finally, the semantic, geometric, and positional features are fused into unified semantic- geometric hybrid 3D scene representation. This hybrid representation, together with the text query Q. is fed into the video LLM to perform autoregressive object grounding and spatial reasoning, ultimately producing the final answer.
the 3D coordinates of
We format the output of GS- CS-Reasoner in Chain-of- Chain-of-Thought (CoT) manner. All intermediate reason- ing is enclosed within the "<think>. </think>" block: the model first analyzes the query, and then lists the 3D bounding boxes of all relevant objects in the following format "ORJECT _AMEE
3

---

Published as a conference paper at ICLR 2026
depths
images
Grounded-Spatio Reasoner
Trainable
umproject
Frozen
Aelaptive CrosAtth
Pooling
No mesarting fro the clneat of thnu okgects col',, pland) the closast the
Interpolin
xy,, rgh
pland
Semantic-Geometri Fusion
Enodeer
Geometric
Semantic
Video
unpooling
Paua opped
000
plavs
so0v 1Ma, leartiry relioans cijacts
be bimes first
86
Encoding
Position
Projector
Geumetric
Projector
benwen each cofect and Joan select the obrect alts mallert soddes sistanos
2 pr0), The Goillet Rictasee casegiry ane sart wint, canader mare bu10y
the mallintt distanee to the doar, dlong options
Patch based Fuston
GS Reasoner Overview
Semantic-Goometric
Patch- based Fusion
Fusion
Figure 2: Overview of GS-Reasoner framework. Our method builds semantic geommetri hybrid 3D scene representation, enabling 3D LLM to perform 3D visual grounding autoregressively, which allows grounding to be integrated as chain-of of-thought within the spatial reasoning process.
If object bound- ing boxes are considered unhelpful for answering during question analysis, they are omitted. Each tuple " (x1, yi, 21, x2, y2, 22) " denotes the coordinates of two opposite corners of an axis-aligned 3D bounding box expressed in the world coordinate frame (units: meters). After ground- ing, the model carries out step-by-step spatial reasoning using all available information. Finally, emits a concise final answer enclosed in "<answer>. output format improves interpretability while remaining flexible, enabling GS-Reasoner to be applied to various 3D visual grounding and spatial-reasoning tasks without changing the architecture.
SOJECT_COUNT <bbox> (x1, yl, z1, x2, y2, z2) </bbb//bb.....
</ s/answer>" This autoregressive
3.2 SRMANTIC-GEOMETRRR HYBRID 3D SCENE REPRESENTATION
In this section, we describe the construction a comprehensive 3D scene representation that seamlessly integrates semantic and geometric information. Building on Video LLM, goal is to enhance its spatial understanding capabilities by incorporating richer geometric cues, without increasing the input token count compromising its language comprehension. Inspired by recent works (Zheng et al., 2025; Zhu et al., 2024b) that augment image patch features with 3D positional encoding, we design our 3D scene representation using image patches the basic building block.
Geometric Feature Extraction. The first challenge arises when extracting per-patch geometric features from point maps. Instead of processing points independently within each aatch-which often contain very few points and thus provide limited context for effective feature learning-we process the point cloud as a whole. Specifically, we first partition the point maps {P, x Ji=1 into patches of size x p, aligning with the image patch size used in the semantic encoder. To reduce computational cost, we uniformly sample K points from each patch, resulting subsampled point maps denoted as (Pruu E 3x xW y. where H' Hf and W
The collection of all
sampled points across patches forms the input point cloud P. which is subsequently processed by a point cloud encoder to extract geometric features. We adopt the encoder-only Point Transformer v3 (PTv3) (Wu et al., 2024; 2025b) as our point cloud encoder owing to its efficiency and effectiveness. Given a point set M (P, where F R denotes point attributes (e.g., position, color), PTv3 first serializes the input point cloud with space-filing curves and partitions the points into subsets (M1, M2..., Mar] according to their serialization order. The serialized subsets then processed
4

---

Published as a conference paper at ICLR 2026
by U-Net-like encoder-only architecture, where each layer employs serialized attention to capture both local and global context. Between layers, each subset M, = Fi). is pooled as follows,
(I) where (p', f() denotes the position and features of pooled point aggregated from subset M,, and U E Rexe is a linear projection. Collecting pooled points from n' subsets yields the point set M' = f: for the next stage of encoding. Unpooling is performed by preserving mapping relationships through the pooling layers, which allows point features to be projected back to the original resolution and concatenated with features from the previous encoding stage as,
f [f, If E F.)), p = MeanPool (p, P, E P:)),
f. cconaa( fi. fj"),
if (p:, fi) E Mj.
By progressively unpooling across layers and mapping the features back to point maps, we obtain the final geometric feature maps (G, E RC Kx xW" " which are spatially aligned with the inputs.
Dual-Path Pooling, With extracted geometric feature maps {G, RC J J==1 and point maps (P E g3 x J,y Ji=1. straightforward strategy for deriving per-patch representations is to apply max pooling within each patch on the geometric feature maps and mean pooling on the point maps, following the design in PTv3. However, we observed that this naive approach results in poor grounding performance, which can be attributed to two key issues: (1) misalignment. While processing point cloud as a whole enhances the receptive field and enables more accurate geometric feature extraction compared to treating points within each patch independently, it also leads to misalignment between the geometric features and semantic features in each patch, as the 3D points in patch can interact with almost all the points in point cloud, whereas the semantic features are constrained to the information visible in the current image. The geometric features pooled by max pooling emphasize the most salient features without considering the semantic context of misalignment. Traditional point
the patch, exacerbating this misalignment. (2) position-geomettee cloud encoders typically group points using KNN (Qi et al., 20;7;;b; Zhao et al., 2021; Wu et al., 2022) or serialization (Wu et al., 2024; 2025b), ensuring that points within a group are spatially close in 3D space. This spatial proximity allows naive pooling strategies to effectively preserve geometric information within the group. In contrast, 3D points within an image patch do not necessarily satisfy this condition, particularly when a patch contains both foreground and background elements, which can lead to large spatial distances among points. Consequently, directly applying max pooling to the geometric features within the patch may introduce geometric inconsistencies, while mean pooling the 3D points can produce positions that are far from both foreground and background objects. These issues can negatively impact the accuracy of predicted 3D bounding boxes.
To address these challenges, propose simple yet effective dual-path feature fusion module that aligns semantic, geometric, and positional information at the patch level. To mitigate semantic- geometric misalignment,
construct semantic-aligne geometrie features via a lightweight cross- attention network. Each patch's 's semantic feature serves as the query, while the K geometric features within the patch serve keys and values. The attention mechanism allows the network to selectively integrate the geometric features most relevant to the patch's 's semantic context. For the position- geometric misalignment, we directly sample the 3D point corresponding to each patch's 's center pixel for position encoding, and then interpolate the geometric features based on the position of the 3D point to obtain position-aligned geometric feature. This simple strategy ensures consistency between positional and geometric information: if the sampled point is on the foreground, the interpolated features mainly come from foreground points, and vice versa. Finally, the semantic-aligned and position-aligned geometric features are concatenated and projected to produce the final patch-level geometric features, which are then combined with the projected semantic feature and sampled 3D point positional encoding to obtain the final patch-level hybrid feature.
4 GCoT: GROUNDED CHAIN-OF-THOUGH DATASET
Recent works (Wu et al., 2025a; Fan et al., 2025; Ouyang et al., 2025) attempt to improve the spatial reasoning ability of MLLMs by constructing large-scale QA pairs with 3D object annotations. However, the answers in these datasets are typically restricted to single choices or short numerical values. Such limited supervision narrows the learning space of MLLMs, thereby reducing their learning efficiency and resulting in less interpretable outcomes. In fact, spatial reasoning is largely
5

---

Published as a conference paper at ICLR 2026
BEV
21%
GoT
Chain of Thought
Room Size (2.38)
588)
(156k)
Asselute Distanoe (R
Relative Directio
vasir the antee serrent (4)) Lenter pooiiin) Seitin ateettetiie
teing mnjost
75%
Object Sire (U,8))
Objecct
1.11 2010
Leral contteenc Driettie dering divended)
Q&A
Scarhet
Seerte afrecte fuard volen bate 148
Sarnhet-
pritiimn Juny 1-8 110 Lend conntwrre
(155k)
(k, 8) atviing
7236
Figure 3: Overview of Grounded Chain-of-Thought (GCoT) Dataset. We first construct spatial QA pairs without CoT, and then prompt GPT-44o to generate CoT paths based on the bird's-eye view, object information, and QA pairs.
grounded in the locations and size relationships of relevant objects, indicating that identifying objects and reasoning about their geometry are fundamental steps, which is essentially a 3D visual grounding task. Introducing grounding as an intermediate step in spatial reasoning not only provides richer supervision but also improves interpretability, which motivates the construction of the GCoT dataset.
Fig. 3 presents an overview of the GCoT dataset. We first generate spatial reasoning QA pairs without CoT by following the dataset construction pipeline of (Yang et al., 2025; Fan et al., 2025), while preserving the object bounding box information used during generation. Leveraging the QA pairs, object bounding boxes, and bird's-eye views of the scenes, we then prompt GPT-4o (OpenAl et al., 2024) to produce coherent CoT reasoning paths that lead to the final answers. The resulting dataset consists of 1.56k QA pairs, among which 79% contain CoT annotations. We omit CoT construction for the Appearance Order, Object Counting, and Room Size Estimation tasks, as these tasks do not require complex spatial reasoning. Additional details are provided in the Appendix A.
5 EXPERIMENTS
We first describe the implementation details in Section 5.1 and report results on 3D visual grounding in Section 5.2. Since grounding forms the basis for spatial reasoning, we then evaluate our framework on spatial reasoning tasks in Section 5.3. Section 5.4 presents additional results on general 3D tasks, and Section 5.5 provides zero-shot evaluation and ablation studies analyze the contributions of individual model components and the proposed GCoT dataset.
5.1 IMPLEMENTATION DETAILS
Model Architecture. OS-Reasonerrs is developed on top of LLa Video 7B (Zhang et al., 2024), an open-source video LLM based on Qwen2-7B (Team, 2024). For semantic encoding, we adopt SigLIP (Zhai et al., 2023), a vision transformer pre-trained on large-scale image- pairs through contrastive learning. For geometric encoding, we employ Sonata (Wu al., 2025b), an efficient point cloud encoder built upon PTv3 (Wu et al., 2024) and pre-trained in self-supervised manner on large-scale point cloud datasets. We adopt sinusoidal positional encoding (Vaswani et al., 2017) to encode the 3D positions of image patches.
Training. GS-Reasoner is trained end-to-end for next-token prediction. We first pretrain on subsets of 3D visual grounding datasets, including ScanRefer (Chen al., 2020), Multi3DRef (Zhang et al., 2023), SR3D, and NR3D (Achlioptas al., 2020), to warm up object grounding, and then finetune on proposed GCoT dataset, the remaining grounding data, and other 3D tasks (ScanQA (Azuma et al., 2022), SQA3D (Ma et al., 2022), Scan2Cap (Chen et al., 2021)). Data augmentation is important for training Re-Reasoner and provide more details in Appendix B.1.
Inference. Unless otherwise specified, we uniformly sample 32 images from each scene as the model input during inference. For the 3D visual grounding task, ground-truth depth maps and camera

---

Published as a conference paper at ICLR 2026
Table Evaluation Visual Grounding. GS- Reasoner achieves performance comparable to LLMs using mesh proposals or external grounding, without any external components.
ScanRefer
Multi3DRef
SR3D
NR3D
Methods
2c
Acc@
F1
F1
Acc
Acc
Expert Models
3D VisTA (Zhu et ul., 2023)
$1.0
46.2
56.5
47.7
PQ3D (Zhu et al., 2024c)
56.7
51.8
62.0
UniVLG (Jain
2025)
63.5
56.4
73.0
56.3
Locate 3D (Mc et al., 2025)
61.1
50.9
68.2
56.1
30 M + Proposals from Mesh PC
Chat- (Huang al., 2024)
55.5
50.2
57.1
52.4
Inst3D-LMM (Yu al., 2025)
57.8
51.6
58.3
53.5
Video-3D LLM (Zheng et al., 2025)
58.1
51.7
58.0
52.7
ROSS3D (Wang al., 2025b)
61.1
54.4
59.6
54.3
SceGround (Li al., 2025b)
44.1
39.4
3D External Grounding Module
Grounded 3D-LLM (Chen et al. 2024b)
48.6
44.0
44.7
40.8
ReGround3D (Zhu et al., 2024a)
53.1
41.2
LLaVA-3D (Zhu 2024b)
54.1
42.2
54.3
47.2
3D LLMs
3D-LLM (Hong al., 2023b)
30.3
GS- Reasoner
60.8
42.2
61.7
45.3
56.7
50.0
Table 2: Evaluation on VSI-Bench. ES-Reasoner achieves state-of-the-ar performance most tasks, with further gains using more accurate (ground-truth) depth.
Numerical Question
Multiple-Chodl Chhtip Question
Methods
Rank. Avg.
Obj Cut. Abs. Dist. Obj. Size Room Size Rel. Dist.
Rel. Dir.
Route Plan Appr. Order
Buseline
Chance Level (Random)
25,0
36.1
28.3
25.0
Chance Level (Frequency)
34.0
62.1
32.0
29.9
33.1
25.1
47.9
28.4
25.2
VSI-Bench Perf. (+= Tiny Set)
Level
1Gemini- 1,5 Flash
45.7
50.8
33.6
56.5
45.2
48.0
39.8
32.7
59.2
1Gemini- 1.5 Pro
48.8
49.6
28.8
58.6
49.4
46.0
48.1
42.0
68.0
iGemini-2N Flash
45.4
30.6
24.5
Proprictary Models (API)
GPT-4o
34.0
46.2
5.3
37.0
31.5
28.5
Gemini-1.5 1.5 Flash
2
42.1
49.8
30.8
53.5
54.4
37.7
41.0
31.5
37.8
Gemini-1. 1.5 Pro
45.4
56.2
30.9
64.1
43.6
51.3
46.3
36.0
34.6
Open-sourced VLMs
V1.emVV.2-408
LongVILA-8B
9
21.6
29.1
9.1
16.7
0.0
29.6
30.7
32.5
25.5
VILA-I15-40B
7
31.2
22.4
24.8
48.7
22.7
40.5
25.7
31.5
32.9
LongVA-7B
25.4
VLVVA-NeXT-Video-72
4
35.6
48.5
14.0
47.8
24.2
43.5
42.4
34,0
30.6
LLaVA-NcXT-VVdoo-228
40.9
48.9
22.8
57.4
35.3
42.4
36.7
35.0
48.6
QWen2.5VL-7B
14.8
29.8
LLaVA OOVV--Oneiisonn> 7B
32.4
47.7
20.2
47.4
12.3
42.5
35.2
29.4
24.4
2
40.2
43.5
23.9
$7.6
37.5
42.5
39.9
32,5
44.6
Specialized Spatial Reasoning Models
Spatial-MI MM4BB
48.4
65.3
34.8
63.1
45.1
41.3
46.2
33.5
46.3
VLM-3R-7B
60.9
70.2
49.4
67.1
45.4
40.1
GS- RS--eesonner (pred dep.)
64.7
69.1
61.9
70.0
65.7
65.4
88.9
44.3
52.3
GS- GSS-Reasoner (gi dep.)
70.1
70.9
73.6
77.8
81.8
70.6
90.5
42.8
52.6
parameters are provided to ensure a fair evaluation. For the spatial reasoning task, depth maps and camera parameters are estimated using VGGT-SLAM (Maggio et al., 2025), followed by metric alignment with Moge2 (Wang et al., 2025e). More details are provided in Appendix B.2.
5.2 EVALUATION ON 3D VISUAL GROUNDING
We evaluate model on four widely used 3D visual grounding benchmarks: ScanRefer, Multi3DRef. SR3D, and NR3D. For single-object grounding (ScanRefer, SR3D, NR3D), we report Ace@22 25 and 50c@500, 50, where prediction is correct if its Intersection over Union (IoU) with ground truth exceeds

---

Published as a conference paper at ICLR 2026
Table Evaluation on General 3D Tasks. GS- Re-Roes outperforms state-of-the-an 3D LLMs on Scan2Cap and achieves comparable results on ScanQA and SQA3D.
Scan2Cup
ScunQA
SQA3D
Methods
B-4 Rouge
CIDEr
Meteor
B-4
Rouge
CIDEr
Meteor 1 EM
EM
3 MilaamnM()amnggoo (Hong et 2023a)
7.2
32.3
12.2
20.4
3D-LLMIBLIP2- (Hong et al., 2023a)
12.0
35.7
69.4
14.5
20.5
LL3DA (Chen et al., 2024a)
36.8
55.1
65.2
26.0
13.5
37.3
76.8
15.9
Chat-3Dv2 (Huang
2023a)
LEO (Huang al., 2023b)
36.9
57.8
68.4
27.7
13.2
49.2
101.4
20.0
24.5
50.0
Sceno-LLM (Fu et al., 2025)
12.0
40.0
80.0
16.6
27.2
54.2
ChatSceme (Huang et all, 2024)
36.3
58.1
77.2
28.0
87.7
21.6
LLaVA-3D (Zhu et al., 2024b)
41.1
63.4
79.2
30.2
14.5
50.1
91.7
20.7
27.0
55.6
Video-3D LLM (Zheng ct al., 2025)
42.4
62.3
83.8
28.9
16.2
49.0
102.1
19.8
30.1
58.6
ROSS3D (Wang et al., 2025b)
43.4
81.3
30.3
17.9
50.7
107.0
20.9
30.8
GS-Reasoner
47.6
69.2
101.0
32.1
16.2
49.2
102.6
19.8
29.9
59.9
0.25 or 0.5, respectively. For multi-object grounding (Multi3DRef), we use the F1 score computed ToU thresholds of 0.25 and 0.5. For fair comparison, we group the baselines into four categories: (I) Expert Models, specifically designed for 3D grounding and trained with both bounding box and mask supervision; (2) 3D LLMs + Proposals from Mesh Point Cloud, which select from proposals generated by detectors such as Mask3D (Schult et al., 2022); (3) 3D LLMs External Grounding Module, which pair 3D LLM with an auxiliary grounding module; and (4) 3D LLMs, which directly predict bounding boxes autoregressively without relying any external modules proposals.
As shown in Tab. 1, our model achieves superior performance compared with 3D-LLM (Hong et al., 2023b). Among other 3D L.LM-based methods, it achieves state-of-the-art F1 @25 Multi3DRef, even surpassing methods that rely on proposals or external grounding modules. Moreover, on ScanRefer Acc@50 50 and Multi3DRef FI @ 50, GS-Reasoner matches the performance of 3D LLMs with external grounding modules, despite using only noisy, incomplete sensor point clouds rather than high-quality mesh inputs. However, GS- -e-aoesoner still lags behind 3D LLMs with proposals from mesh point clouds on these two metrics. We attribute this to two factors: (1) mesh point clouds are more complete and less noisy; and (2) conventional 3D detectors (e.g.. Mask3D) are commonly trained with mask supervision, which is more conducive to precise object localization than bbox supervision, as also reported in (My et al., 2025; Jain et al., 2025). An interesting observation is that -S-Reasoneer achieves comparable results expert models on ScanRefer but falls behind on SR3D and NR3D, suggesting LLM-based methods are better at complex queries (as in ScanRefer), while expert models excel in precise localization for simpler descriptions (as in SR3D and NR3D).
5.3 EVALUATION ON SPATIAL REASONING
We evaluate "S--Resonner's spatial reasoning capability on VSI-Bench (Yang et al., 2025), which comprises over 5,000 QA pairs from egocentric videos in ScanNet (Dai et al., 2017), ScanNet++ (Yesh- wanth et al., 2023), and ARKitScenes (Baruch et al., 2021). VSI-Bench provides multiple choice (MCA) and numerical (NA), and covers eight tasks spanning spatial and temporal reasoning. We follow the official VSI-Bench evaluation protocol for metric computation. The results in Tab. 2 demonstrate that RS-Reesoner achieves impressive performance, particularly on the Relative Direction and Absolute Distance tasks, which require complex spatial reasoning and precise 3D object localization. It also attains mate-of-the-art results on the Appearance Order task, indicating that our semantic- geoomtric hybrid features effectively preserve temporal information from the original video while providing additional spatial cues. Moreover, performance consistently improves with more accurate depth input, with the average score exceeding 70 and yielding nearly a 10-point gain over the previous state of the art
answer formats,
5.4 EVALUATION ON GENERAL 3D TASKS
We further present results on three established 3D vision-language understanding tasks: Scan2Cap, ScanQA, and SQA3D, following the official protocols and reporting performance in terms of CIDEr, BLEU, METEOR, ROUGE, and exact-match (EM) accuracy. The results in Tab. show that GS- Reasoner sets new state of the art for 3D dense captioning, achieving the best results on Scan2Cap across all metrics and significantly surpassing the previous leading method, ROSS3D (Wang et al.,
8

---

Published as a conference paper at ICLR 2026
Table 4: Zero-shot 3D visual grounding. We train the models exclusively on ScanNet and evaluate them on ScanNet++ and ARKitScenes for visual grounding, reporting all results in Acc@25, Note that GPT-4o is prompted to do 2D visual grounding and then back-project 3D via depth map. Locate 3D expert model.
ScanRefer
LX3D
Methods
ScanNet
Scan
ARKitSeenes
GPT-4o VLM
59.9
60.5
26.8
Locate 3D
61.1
56.7
46.2
GS-Reasoner
51.0
45.6
Table 5: Ablation Study Data Aug. and 3D Representation. We train models to autoregressively predict
3D bounding box coordinates using ScanRefer and Multi3DRef, and report results on ScanRefer.
Methods
Data Aug.
Pos. Enc.
Geo.Pool.
Acc@25
Acc@50
LLaVA-NeXT
X
X
0.0
0.0
Video-3D 3D LLM
Avg
15.4
3.5
Avg
53.2
29.8
Avg
Max
57.5.4.3
35.7-5.9
Avg
Cross-Altn
$8.9
38.6.
Sample Sample
Interpolate
59.3
40.2..0.4
GS- GS-Reasoner
Dual-Path
60.8,7.6
42.2,12.4
Table 6: Ablation Study Grounded CoT Mechanism. We report results only for tasks in the GCoT dataset that include CoT annotations, highlight the effectiveness of grounded CoT.
Numerical Question
Multiple-Choice Question
Methods
Avg.
Abs. Dist.
Obj. Size
Rel. Dist.
Rel. Dir.
Route Plan
LLaVA-NeXTI Veoodeo ft (wlo CoT)
52.3
45.1
64.3
58.9
60.7
32.5
GS Reasoner ft (wio CoT)
57.7.s.4
50.8.5.7
65.7-1.4
62.3...4
79.3
30.4-2.1
GS-Reasoner ft (Full)
66.1.13.a
61.9.16.a
70.0.s.7
65.4...5
88.9.2a.2
44.3,11
2025b), We attribute these gains to explicitly predicting coordinates for 3D visual grounding, which forces the model better capture geometric and positional cues, thereby improving dense captioning performance. However, GS- GS-Reasoner does not achieve leading results on ScanQA and SQA3D. We believe the main reasons are the presence of many ambiguous questions in these datasets that do not clearly specify the target object, as well as a strong bias in answer distribution. These factors encourage the model to overfit textual patterns instead of effectively exploiting 3D tokens. Recent studies (Huang et al., 2025a; Li et al., 2025a) have also shown that finetuning LLMs without 3D input can yield results comparable reconstruction constraints in the output (as done in ROSS3D (Wang et al., 2025b)) may help encourage the model to utilize 3D tokens, and we leave this for future research.
the state of the art on ScanQA and SQA3D, Incorporating
5.5 ANALYSIS AND ABLATION STUDIES
Zero-shot Generalization. We evaluate the zero-shot generalization of GS-Reasoner unseen 3D scenes. The model is trained solely on ScanNet data (ScanRefer, SR3D, etc.), and tested on the ScanNet++ and ARKitScenes validation splits of the Locate3D dataset (Mc al., 2025) without finetuning, As shown in Tab, 4, GS-Reasoner achieves performance comparable to SOTA expert models on ARKitScenes and demonstrates strong results novel scene spatial reasoning (Fig. 1).
Effectiveness of Data Augmentation and Geometri Hybrid 3D Representation. We conduct ablation studies to assess the effectiveness of our data augmentation strategies and the proposed semantic-geome geometric hybrid 3D representation, using the 3D visual grounding task as the evaluation benchmark. We believe this task directly reflects the model's 's ability to jointly leverage semantic and spatial information from the input 3D scene. The results in Tab. 5 show that the model fails to accurately predict 3D bbox coordinates when only image input provided (LLaVA-NexI), Incorporating average position encoding (as in Video-3D LLM) still results in poor performance due to overfitting. Data augmentation brings notable improvements, yet the model continues to struggle with precise object localization, indicated by the low Acc @ 50. Finally, by introducing geometric features from the geometric encoder and employing Dual-Path Pooling to progressively fuse position- aligned and semantic-aligned geometric features, achieve substantial gains in both @cc@ 25 and

---

Published as a conference paper at ICLR 2026
Table 7: Computation Cost Comparison. values are reported in milliseconds.
Methods
2D Vision Enc.
3D Vision Enc.
Dual-Path
Total
LLa NeXXTViddo
429
470
GS- C.S-Reasoner
430
204
41
737
Table 8: Ablation Study on Input Frames. We investigate the impact of different numbers of input frames and sampling strategies 3D visual grounding performance.
ScanRefer
Multi3DRef
Frames Num
Sampling Strategy
Acc@25
Acc@ 50
@1@2
@
32
uniform
60.8
42.2
61.7
45.3
48
uniform
61.7
44.5
62.2
46.9
64
uniform
61.2
45.1
62.5
48.0
cdviews 61.1
43.4
61.9
46.2
50. These results demonstrate that the proposed hybrid 3D representation strengthens the model's understanding of 3D scenes and enables more accurate visual grounding.
Effectiveness of GCoT Dataset. We also ablate the impact of incorporating grounding into the thain-of-thought process spatial reasoning performance. Specifically, remove the CoT part from the answers in the GCoT dataset and train the model to directly predict the answer from the 3D scene and question. We report results for five tasks that incorporate grounding within the CoT process Tab. 6. The results show that integrating grounding as part of the CoT process substantially improves performance across all tasks, particularly in object absolute distance, object relative direction, and route planning. This highlights the importance of not only providing the model with grounding capabilities but also guiding it to leverage grounding effectively to support spatial reasoning, demonstrating the necessity of the proposed GCoT dataset.
Computation Costs Evaluation. GS Reasone''s'ss computational overhead compared the backbone LLaVA-Nesa Video primarily arises from the construction of visual tokens. Specifically, GS- Reassonn requires encoding point clouds using Point Transformer and integrating them into image patch features. Therefore, we conduct comparison of computational overhead during the visual token construction phase, using a single 4090 GPU with 32 input images. As shown Tab. 7, GS- GS-Reasoner introduces some additional computational overhead. However, since all visual tokens only need to be constructed once per response, this overhead is acceptable relative to the overall inference time. Furthermore, during subsequent reasoning, the semantic- geometric hybrid 3D representation constructed by GS- Reasoner does not increase the number of visual tokens, so the inference time remains comparable to that of VA--Nee Video.
blation Study on Input Frames, We conduct an ablation study to investigate the impact of different numbers of input frames and sampling strategies 3D visual grounding performance. As shown in Tab. 8, model performance consistently improves with an increasing number input images, and the improvements are more pronounced in Acc@ 50 (Fl @50) compared Ace@2: (FI @2)), indicating that providing denser point clouds can indeed help enhance localization accuracy. Moreover, employing context-award sampling strategies such as cdViews (Wang al., 2025a) also yields certain improvements.
6 CONCLUSION
In this work, we present CS-Reasoner, a novel framework that integrates grounding into spatial reasoning as a chain-of-thoughh process. Built upon hybrid semantic-geometre tation, GS- (eS-Reasoner performs grounding without requiring any external modules, making it a natural intermediate step for spatial reasoning. The GCoT dataset further strengthens the model's ability to handle both tasks effectively.
scene represen-
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGMENTS
This work was supported in part by New Generation Artificial Intelligenee-Nationa Science and Technology Major Project (2025ZD0123004), Ningbo grant (2025Z038) and National Natural Science Foundation of China (Grant No. 62376060).
ETHICS STATEMENT
Our work introduces new dataset generated entirely using LLMs. As the dataset is synthetically generated, does not contain any personally identifiable information or sensitive human data. Nevertheless, synthetic data may inherit biases present in the underlying LLM,, and could potentially be misused for harmful or mislcading purposes. To mitigate these risks, the dataset is intended solely for academic research, and will be released with clear guidelines responsible usage. Users are encouraged to consider ethical implications when employing the dataset for downstream tasks.
REPRODUCIBILITY STATEMENT
To facilitate reproducibilityy we will release the full dataset, preprocessing scripts, and detailed documentation upon acceptance. All experimental code, pretrained models, and evaluation protocols will also be made publicly available. The datasets used in our experiments are either publicly accessible or will be released as part of this work. We provide complete hyperparameter settings, training schedules, and random seeds in the paper and supplementary materials.
REFERENCES
Panos Achlioptas, Ahmed Abdelrcheem, Fei Xia, Mohamed Elhoseiny, and Leonidas Guibas. Referit3d: Neural listeners for fine-grained 3d object identification real-world scenes. In European conference computer vision, pp. 422 440. Springer, 2020. 6, 18
Daichi Azuma, Taiki Miyanishi, Shuhei Kurita, and Motoaki Kawanabe. Scanqa: 3d question answering for spatial scene understanding. In proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 19129-19139, 2022. 2, 6,
Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, al. Qwen2. 5-vl technical report. arXiv preprint arXiv:2502. 2025.
Gilad Baruch, Zhuoyuan Chen, Afshin Dehghan, Tal Dimry, Yuri Feigin, Peter Fu, Thomas Gebauer, Brandon Joffe, Daniel Kurz, Arik Schwartz, et al. Arkitscenes: diverse real-world dataset for 3d indoor scene understanding using mobile rgb-d data. arXiv preprint urXiv:2111.0889) 2021. 8, 16
Wenxiao Cai, Iaroslav Ponomarenko, Jianhao Yuan, Xiaoqi Li, Wankou Yang, Hao Dong, and Bo Zhao. Spatialbot: Precise spatial understanding with vision language models. arXiv preprint arXiv:2406. 13642, 2024.
Dave Zhenyu Chen, Angel X Chang, and Matthias Nießner. Scanrefer: 3d object localization rgh-d scans using natural language. 16th European Conference Computer Vision (ECCV), 2020. 6, 18
Sijin Chen, Xin Chen, Chi Zhang, Mingsheng Li, Gang Yu, Hao Fei, Hongyuan Zhu, Jiayuan Fan, and Tao Chen. LI3da: Visual interactive instruction tuning for omni-3d understanding reasoning and planning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 26428-26438, 2024a. 2.
Xingyu Chen, Yue Chen, Yuliang Xiu, Andreas Geiger, and Anpei Chen. Ttt3r: 3d reconstruction as test-time training, arXiv preprint arXiv:2509.266 26645, 2025. 19
Yilun Chen, Shuai Yang, Haifeng Huang, Tai Wang, Runsen Xu, Ruiyuan Lyu. Dahua Lin, and Jiangmiao Pang. Grounded 3d-llm with referent tokens. arXiv preprint (tt::2405.10370 2024b. 3,
11

---

Published as a conference paper at ICLR 2026
Zhe Chen, Weiyun Wang. Yue Cao, Yangzhou Liu, Zhangwei Gao, Erfei Cui, Jinguo Zhu, Shenglong Ye, Hao Tian, Zhaoyang Liu, et al. Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling. arXiv preprint arXiv:2412.05277/ 2024c.
Zhenyu Chen, Ali Gholami, Matthias Nießner, and Angel Chang. Scan2cap: Context-aware dense captioning rgb-d scans, In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 3193-3203, 2021. 6, 18
An-Chieh Cheng, Hongxu Yin, Yang Fu, Qiushan Guo, Ruihan Yang, Jan Kautz, Xiaolong Wang, and Sifci Liu. Spatialrgpt: Grounded spatial reasoning in vision-language models. Advances in Neural Information Processing Systems, 37:125062-135093, 2024.
Angela Dai, Angel X Chang, Manolis Savva, Maciej Halber, Thomas Funkhouser, and Matthias NieBner. Scannet: Richly-annotated 3d reconstructions of indoor scenes. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 5828-5839, 2017. 8, 16
Jiajun Deng, Tianyu He, Li Jiang, Tianyu Wang, Feras Dayoub, and lan Reid. 3d-llava: Towards generalist 3d Imms with omni superpoint transformer. Proceedings of the Computer Vision and Pultern Recognition Conference, pp. 3772-3782, 2025.
Rumpei Dong, Zekun Qi, Linfeng Zhang, Junbo Zhang, Jianjian Sun, Zheng Ge, Li Yi, and Kaisheng Ma. Autoencoders cross-modal teachers: Can pretrained 2d image transformers help 3d representation learning? arXiv preprint arXiv:2212.08320, 2022. 19
Zhiwen Fan, Jian Zhang, Renjie Li, Junge Zhang, Runjin Chen, Hezhen Hu, Kevin Wang, Huaizhi Qu. Dilin Wang, Zhicheng Yan, Hongyu Xu, Justin Theiss, Tianlong Chen, Jiachen Li, Zhengzhong Tu, Zhangyang Wang, and Rakesh Ranjan. Vim-3r: Vision-language models augmented with instruction-alignes 3d reconstruction, 2025. htLpsts::/// http://axvvvv... abs/ 2505 20279. 9.
3, S, 6, 16, 17
Rao Fu, Jingyu Liu, Xilun Chen, Yixin Nie, and Wenhan Xiong. Scene-llm: Extending language model for 3d visual reasoning. In 2025 /EEE/CVF Winter Conference on Applications of Computer Vision (WACV), pp. 2195-2206. IEEE, 2025. 2, 3,
Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross B. Girshick. Masked autoencoders are scalable vision learners. 2022. 19
Yining Hong, Haoyu Zhen, Peihao Chen, Shuhong Zheng, Yilun Du, Zhenfang Chen, and Chuang Gan. 3d-llm: Injecting the 3d world into large language models. Advances in Neural Information Processing Systems, 1-220422224942 2023a. 1, 2,
Yining Hong, Haoyu Zhen, Peihao Chen, Shuhong Zheng, Yilun Du, Zhenfang Chen, and Chuang Gan. 3d-llm: Injecting the 3d world into large language models, 2023b. 7,8
Haifeng Huang, Zehan Wang. Rongjie Huang, Luping Liu, Xize Cheng, Yang Zhao. Tao Jin, and Zhou Zhao. Chat-3d v2: Bridging 3d scene and large language models with object identifiers. CoRR, 2023a.
Haifeng Huang, Yilun Chen, Zehan Wang, Rongjie Huang, Runsen Xu, Tai Wang, Luping Liu, Xize Cheng, Yang Zhao, Jiangmiao Pang, al. Chat-scene: Bridging 3d scene and large language models with object identifiers. Advances in Neural Information Processing Systems, 37:1 13991-
114017, 2024. 3, 7,8
Jiangyong Huang, Silong Yong, Xiaojian Ma, Xiongkun Linghu, Puhao Li, Yan Wang, Qing Li, Song-Chun Zhu, Baoxiong Jia, and Siyuan Huang. An embodied generalist agent in 3d world. arXiv preprint 2...v:2311.128771 2023b. 1,8
Jiangyong Huang, Baoxiong Jia, Yan Wang, Ziyu Zhu, Xiongkun Linghu, Qing Li, Song-Chun Zhu, and Siyuan Huang. Unveiling the mist over 3d vision- langualanguage understanding: Object-centric evaluation with chain-of- analsss.. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 24581, 2025a. 9,
12

---

Published as a conference paper at ICLR 2026
Jiangyong Huang, Xiaojian Ma, Xiongkun Linghu, Yue Fan, Junchao He, Wenxin Tan, Qing Li, Song-Chun Zhu, Yixin Chen, Baoxiong Jia, et al. Leo-vl: Towards 3d vision-language generalists via data scaling with efficient representation. arXiv preprint arXh:22506.09933 2025b. 2,
Ayush Jain, Alexander Swerdlow, Yuzhou Wang, Sergio Arnaud, Ada Martin, Alexander Sax, Franziska Meier, and Katerina Fragkiadaki. Unifying 2d and 3d vision-language understanding. arXiv preprint arXiv:2503. 2500: 2025. 7,8
Moo Jin Kim, Karl Pertsch, Siddharth Karamcheti, Ted Xiao, Ashwin Balakrishna, Suraj Nair, Rafael Rafailov, Ethan Foster, Grace Lam, Pannag Sanketi, et al. Openvla: An open-source vision- aaaoon-lnguage-cttionn model. arXiv preprint arXhv:2406.092466, 2024. 20
Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Ziwei Liu, et al. Llava-onevision: Easy visual task transfer. arXiv preprint arXiv:2408. 033266, 2024. 3
Haoyuan Li, Yanpeng Zhou, Yufei Gao, Tao Tang, Jianhua Han, Yujie Yuan, Dave Zhenyu Chen, Jiawang Bian, Hang Xu, and Xiaodan Liang. Does your 3d encoder really work? when pretrain-sft from 2d vlms meets 3d vims. arXiv preprint arXiv:2506.05318, 2025a. 9, 19
Rong Li, Shijie Li, Lingdong Kong, Xulei Yang, and Junwei Liang. Seeground: See and ground for zero-shot ppen-vocabulary 3d visual grounding. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 3707-3717, 2025b. 7
Bin Lin, Yang Ye, Bin Zhu, Jiaxi Cui, Munan Ning, Peng Jin, and Yuan. Video-llava: Learning united visual representation by alignment before projection. arXiv preprint arXiv:2311.10122. 2023. 3
Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee, Improved baselines with visual instruction tuning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 26296-26306 2024. 3
Xiaojian Ma, Silong Yong, Zilong Zheng, Qing Li, Yitao Liang, Song-Chun Zhu, and Siyuan Huang. Sqa3d: Situated question answering in 3d scenes. arXiv preprint arXiv:2210.07474. 2022. 2, 6, 18
Dominic Maggio, Hyungtae Lim, and Luca Carlone. Vggt-slam: Dense rgb slam optimized on the sl
(4) manifold. arXiv preprint arXiv:2505.12555 2025. 3, 7, 7,718,, 18, 19
Paul Ma Sergio Amaud, Ada Martin, Arjun Majumdar, Krishna Murthy Jatavallabhula, Phillip Thomas, Ruslan Partsey, Daniel Dugas, Abha Gejji, Alexander Sax, et al. Locate 3d: Real-world object localization via sulf-supe learning 3d. In Forty-second International Conference on Machine Learning, 2025. 7. 8,
OpenAI, Aaron Hurst, Adam Lerer, Adam P. Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, al. Gpt-4o system card, 2024. URL htt.s: htRp:://s:///ere //arxiv.. org/abs/220 2410 21276. 6, 17
Kun Ouyang, Yuanxin Liu, Haoning Wu, Yi Liu, Hao Zhou, Jie Zhou, Fandong Meng, and Xu Sun. Spacer: Reinforcing milms in video spatial reasoning. arXiv preprint arXiv:2504.01800 2025. 5
Charles R Qi, Hao Su, Kaichun Mo, and Leonidas J Guibas. Pointnet: Deep learning on point sets for 3d classification and segmentation. In Proceedings of the IEEE conference computer vision and pattern recognition, pp. 652-660, 2017a. 5, 19
Charles Ruizhongtai Qi, Li Yi, Hao Su, and Leonidas J Guibas. Pointnet++: Deep hierarchical feature learning on point sets in metric space. Advances in neural information processing systems, 30, 2017b. 5,
Zekun Qi, Runpei Dong, Guofan Fan, Zheng Ge. Xiangyu Zhang, Kaisheng Ma, and Li Yi. Contrast with reconstruct: Contrastive 3d representation learning guided by generative pretraining. In International Conference Machine Learning, pp. 28223-28244 PMLR, 2023a. 19
13

---

Published as a conference paper at ICLR 2026
Zekun Qi, Muzhou Yu, Rumpei Dong, and Kaisheng Ma. Vpp: via voxel-point progressive representation. Advances in Neural Information Processing Systems, 36:26744- 26763, 2023b. 19
Efficient conditional 3d generation
Zekun Qi. Rumpei Dong, Shaochen Zhang, Haoran Geng, Chunrui Han, Zheng Ge, Li Yi, and Kaisheng Ma. Shapellm: Universal 3d object understanding for embodied interaction. In European Conference on Computer Vision, pp. 214-238. Springer, 2024. 19
Zekun Qi, Wenyao Zhang, Yufei Ding, Rumpei Dong, Xinqiang Yu, Jingwen Li, Lingyun Xu, Baoyu Li, Xialin He, Guofan Fan, et al. Sofar: manguugge-grunddee orientation bridges spatial reasoning and object manipulation. arXiv preprint arXiv: 13143, 2025. 20
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision. volume 139 of Proceedings of Machine Learning Research, pp. 8748 $748-8763. PMLR, 2021. 19
Jonas Schult, Francis Engelmann, Alexander Hermans, Or Litany, Siyu Tang, and Bastian Leibe. Mask3d: Mask transformer for 3d semantic instance segmentation. arXiv preprint arXiv:2210.03100 2022. 8
Qwen Team. Qwen2 technical report. arXiv preprint arXiv:2407.1067 2024. 6
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017. 6
Fengyun Wang, Sicheng Yu, Jiawei Wu, Jinhui Tang, Hanwang Zhang, and Qianru Sun. 3d question answering via only 2d vision-language models. arXiv preprint arXiv:2505.22143. 2025a.
Haochen Wang, Yucheng Zhao, Tiancai Wang, Haoqiang Fan, Xiangyu Zhang, and Zhaoxiang Zhang. Ross3d: Reconstructive visual instruction tuning with 3d-awareness. arXiv preprint arXiv:2504.01901, 2025b. 3, 7, 8, 9, 19
Jianyuan Wang, Minghao Chen, Nikita Karaev, Andrea Vedaldi, Christian Rupprecht, and David Novotny. Vggt: Visual geometry grounded transformer. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. $294-5306, 2025c. 3, 19
Qianqian Wang, Yifei Zhang, Aleksander Holynski, Alexei A Efros, and Angjoo Kanazawa. Continu- ous 3d perception model with persistent state. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 10510-10522, 2025d. 19
Ruicheng Wang, Sicheng Xu, Yue Dong, Yu Deng, Jianfeng Xiang, Zelong Lv, Guangzhong Sun, Xin Tong, and Jiaolong Yang. Moge-2: Accurate monocular geometry with metrie scale and sharp details. arXiv preprint arXiv:2507.00 02546,, 2025e. 7, 18, 19
Diankun Wu, Fangfu Liu, Yi-Hsin Hung, and Yueqi Duan. Spatial-mllm: Boosting milm capabilities in visual- based spatial intelligence. arXiv preprint arXiv:2505.23747, 2025a. 3,5
Xiaoyang Wu, Yixing Lao, Li Jiang, Xihui Liu, and Hengshuang Zhao. Point transformer v2: Grouped vector attention and partition-based pooling. Advances in Neural Information Processing Systems, 35:33330-3333 33342,, 2022. 5, 19
Xiaoyang Wu, Li Jiang, Peng-Shus Wang, Zhijian Liu, Xihui Liu, Yu Qiao, Wanli Ouyang, Tong He. and Hengshuang Zhao. Point transformer v3: Simpler faster stronger. In Proceedings the IEFE/CVF conference on computer vision and pattern recognition, pp. 1840-4851) 2024. 4, 5, 6, 19
Xiaoyang Wu, Daniel DeTone, Duncan Frost, Tianwei Shen, Chris Xie, Nan Yang, Jakob Engel, Richard Newcombe, Hengshuang Zhao, and Julian Straub. Sonata: Self-superviset learning of reliable point representations. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 22193-22204, 2025b. 4, 5.5.6 6, 19
14

---

Published as a conference paper at ICLR 2026
Jihan Yang, Shusheng Yang, Anjali W Gupta, Rilyn Han, Li Fei-Fei, and Saining Xie. Thinking in space: How multimodal large language models see, remember, and recall spaces. In Proceedings of the Computer Viston and Pattern Recognition Conference, pp. 10632-10643, 2025. 3, 6, 8, 16, 20
Chandan Yeshwanth, Yueh-Cheng Liu, Matthias NieBner, and Angela Dai. Scannet++: A high- fidelity dataset of 3d indoor scenes. In Proceedings of the IEEE/CVF International Conference on
Computer Vision, pp. 12-22, 2023. 8, 16
Hanxun Yu, Wentong Li, Song Wang, Junbo Chen, and Jianke Zhu. Inst3d-Imm: Instance-aware 3d scene understanding with multi-modal instruction tuning. In Proceedings of the Computer Vision and Pattemn Recognition Conference, pp. 14147-14157, 2025. 3,7
Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. Sigmoid loss for language image pre-training. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 11975-11986. 2023. 6
Wenyao Zhang, Hongsi Liu, Zekun Qi, Yunnan Wang, Xinqiang Yu, Jiazhao Zhang, Runpei Dong, Jiawei He, He Wang, Zhizheng Zhang, et al. Dreamvla: a with comprehensive world knowledge. arXiv preprint arXiv:2507.0444/ 2025. 20
action model dreamed
Yiming Zhang, ZeMing Gong, and Angel X Chang. Multi3drffer: Grounding text description to multiple 3d objects. In Proceedings of the IEEE/CVF International Conference Computer Vision, pp. 15225-15236 2023. 6, 18
Yuanhan Zhang, Jinming Wu, Wei Li, Bo Li, Zejun Ma. Ziwei Liu, and Chunyuan Li. Video instruction tuning with synthetic data. arXiv preprint arXiv:2410.02713, 2024. 6
Hengshuang Zhao, Jiang, Jiaya Jia, Philip HS Torr, and Vladlen Koltun. Point transformer. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 16259-16268, 2021. 5, 19
Duo Zheng, Shijia Huang, and Liwei Wang. Video-3d Ilm: Learning position-awar aware video represen- tation for 3d scene understanding. In Proceedings of the Computer Vision and Pattern Recognition
Conference, pp. 8995-9006, 2025. 1, 2, 3, 4, 4,7,
Enshen Zhou, Jingkun An, Cheng Chi, Yi Han, Shanyu Rong, Chi Zhang, Pengwei Wang. Zhongyuan Wang, Tiejun Huang, Lu Sheng, et Roborefer: Towards spatial referring with reasoning in vision-language models for robotics. arXiv preprint 2506v:2506.044
2025.
Chenming Zhu, Tai Wang, Wenwei Zhang, Kai Chen, and Xihui Liu. Scanreason: Empowering 3d visual grounding with reasoning capabilities. In European Conference on Computer Vision, pp- 151-168. Springer, 2024a. 3,7
Chenming Zhu, Tai Wang, Wenwei Zhang, Jiangmiao Pang, and Xihui Liu. Llava-3d: simple yet effective pathway to empowering Imms with 3d-awareness. arXiv preprint arXiv:2409.18122 2024b. 1, 2, 3, 7,
Ziyu Zhu, Xiaojian Ma, Yixin Chen, Zhidong Deng, Siyuan Huang, and Qing Li. 3d-vista: Pre-trained transformer for 3d vision and text alignment, 2023.
Ziyu Zhu, Zhuofan Zhang, Xiaojian Ma, Xuesong Niu, Yixin Chen, Baoxiong Jia, Zhidong Deng, Siyuan Huang, and Qing Li. Unifying 3d languae understanding via promptable queries. ECCV, 2024c.1 7
15

---

Published as a conference paper at ICLR 2026
A ADDITIONAL DATASET DETAILS
Grounding Chain-of-Thougee Dataset plays crucial role in our training, guiding the model to learn how to incorporate 3D visual grounding as an intermediate step in spatial reasoning. Here, we provide additional details on the dataset construction.
A.1 INSTRUCTION QA GENERATION
We first construct spatial reasoning QA pairs without ffaiin-ff-thuugh annotations, following the dataset generation pipeline of (Yang et al., 2025; Fan et al., 2025). All data are sourced from existing large-scale 3D datasets, including ScanNet (Dai et al., 2017), ScanNet++ (Yeshwanth al., 2023), and ARKitScenes (Baruch et al., 2021). To ensure consistency across datasets, we perform the following preprocessing steps for each scene:
Point Cloud. We directly use the raw point cloud provided by each dataset. Since ScanNet++ and ARKitScenes do not guarantee alignment of the global coordinate system with the physical room structure (e.g.. the XY plane may not align with walls or floors), we further apply axis alignment. Specifically, we estimate the gravity direction and compute principal components of the point cloud to align the axes, yielding transformation matrix for each scene.
Sampled Frame Data. uniformly sample 50 RGB frames, which serve the basis for constructing frame metadata. These frames provide a consistent visual context for generating spatial reasoning questions and ensure coverage of diverse viewpoints within the scene.
Based on the preprocessed data, we further construct detailed metadata for each scene, consisting of the following components:
This metadata is used for all spatial reasoning questions construction. We extract the axis-aligned bounding boxes (AABBs) of all object instances, either directly from mask annotations or by converting from oriented bounding box (OBB) annotations. In addition to bounding boxes, this metadata also includes global scene statistics such the number of objects and room dimensions, which are later used to formulate numerical reasoning questions.
Scene Metadata.
Frame Metadata. This metadata is used specifically for appearance- temporal reasoning questions. For each object, we determine its appearance time by recording the first frame in which its 2D mask area exceeds a given threshold. Consequently, the frame metadata of each scene contains the appearance time all objects, enabling the construction of reasoning questions grounded in temporal visual evidence.
These two types of metadata provide the necessary information to generate a diverse set of spatial and temporal reasoning questions. Following the predefined question templates in (Yang al., 2025), we iterate over the scene metadata to construct a large pool candidate questions and their corresponding answers. The detailed procedures follows:
Spatial Reasoning QA.
For each object category with at least two instances in the scene, we generate counting questions by directly querying the number of instances.
Object Count.
Absolute Distance. We randomly select pairs of objects that appear only once in the scene and compute their Euclidean distance, which serves as the basis for absolute distance queries.
Object Size. For objects with single instance, compute the object size using the diagonal length of their AABB, and use this value to construct size-related questions.
Room Size. We estimate the overall room size of each scene using the alpha-shape algo- rithm applied the scene point cloud, allowing us ask questions about scene-level spatial dimensions.
Relative Distance. We randomly select set of N objects (3 < N < 5), compute all pairwise distances, and identify the closest pair of objects. This enables the construction of questions that require comparative spatial reasoning.
16

---

Published as a conference paper at ICLR 2026
Relative Direction. We randomly select three objects with unique instances and compute their relative directions based on the centers of their AABBs. The resulting orientation relations form the basis of direction-biis based reasoning questions.
Temporal Reasoning QA. For temporal reasoning (i.e., appearance order), we randomly select four objects from each scene and determine their order of appearance using the frame metadata.
Route Planning QA. For route planning questions, we follow the procedure in VLM-3R (Fan et al., 2025) and employ the Habitat simulator to generate diverse navigation trajectories between two predefined points each scene. The turning direction at each step is determined by computing the angle between consecutive anchor points along the trajectory. To identify relevant objects, calculate their proximity to the trajectory by measuring the distance between anchor points and the 3D bounding boxes of scene objects provided in the scene metadata. Finally, construct the QA pairs using predefined templates consistent with those in VLM-3R (Fan et al., 2025), where each question is grounded in the trajectory's turning direction and nearby objects.
A.2 CHAIN-OF- GENERATION
Building upon the generated QA pairs and their associated 3D object bounding boxes, we further create CoT annotations to guide the model in exploiting 3D visual grounding for spatial reasoning. Specifically, we employ GPT-4o (OpenAI et al., 2024) to generate CoT reasoning paths for each QA pair. To mitigate potential hallucinations from the LLM, we construct bird's-eyy s-eye-view map of each scene using the preprocessed point cloud. On this map, explicitly annotate the 3D bounding boxes of relevant objects, with different colors denoting distinct object categories, consistent with the color keys used in the input bounding box information. The BEV map, together with the question, answer, and relevant object annotations, then provided to GPT-4o to generate coherent and detailed CoT paths. For efficiency, (e.g., Room Size, Object Count, Appearance Order) from CoT generation, as these can be accurately answered without the need for step-by sep-by-step step grounding-bard based reasoning.
exclude tasks that do not require complex spatial reasoning grounding
A.3 DATA FORMAT
To train a model that can determine when to ground relevant objects and perform step-by-step reasoning autonomously, we prepend a question analysis part each answer. This part explicitly guides the model to decide whether grounding is required before initiating the reasoning process. An example the final data format for QA pairs with CoT shown below:
QA w/ CoT Example
Question: [preprompt] am standing by the radiator and facing the table, is the telephone the left or the right of the table? Options: A. left B. right
Answer: <think>The question is asking for the relative direction of the telephone with respect to position (at the radiator) while facing the table. To solve this, I can identify all relevant objects and their bounding boxes first.
radiator <bb0x>(-1.9165, -1.0266, 0.0798, -1.6415, 0.9513, 0.6104)</bbxx>, table 1 <bb9xx(-0.9128. -1.7861, 0.0150. -0.5043, -1.0341, 0.5050)</bbox>, telephone <bbox>(-1.4182, 0.9637, 0.9736, -1.1277, 1.2378, 1.0825)</bbo>>
To determine the telephone's position relative to the table, I analyze spatial relation- ships through three steps:
Step 1: Observer Position and Orientation
Standing at the radiator's centroid:
(-1.9165 + - -1641111/ = (-1.0266 0.9513)/2 -0.038
Facing the table, whose centroid:
The telephone lies to the left of the table. Option A is correct. </think>
canswer>A</anwwer
17

---

Published as a conference paper at ICLR 2026
And example of the final data format for QA pairs without CoT is shown below:
QA w/o CoT Example
Question: [preprompt] What is the size of this room (in square meters)? If multiple rooms are shown, estimate the size of the combined space.
Answer: <think>-The question is asking for the room size in square meters. To solve this, can rely on the overall spatial information present in the video without identifying any object bounding boxes.
From the video, the room size is about 11.0 m² </think>> <answer> 1.0</answer>>
By structuring the data in this way, the model learns to autonomously decide when to ground relevant objects and perform step-by-stey reasoning, without the need for additional prompting.
B ADDITIONAL IMPLEMENTATION DETAILS
B.1 TRAINING DETAILS
GS- Reasoner is trained end-to-end with cross-entropy loss for next-token prediction. We first pretrain on subsets of 3D visual grounding datasets, including ScanRefer (Chen al., 2020), Multi3DRef (Zhang et al., 2023), SR3D, and NR3D (Achlioptas et al., 2020), to warm up ob- ject grounding, and then finetune on our proposed GCoT dataset, the remaining grounding data, and other 3D tasks (ScanQA (Azuma et al., 2022), SQA3D (Ma et al., 2022), Scan2Cap (Chen et al., 2021)). All parameters are learnable except those the vision encoder. The LLM learning rate is fixed at 1e-0, while other modules use 1e during pretraining and le during finetuning. We use batch size of 16 for pretraining and 256 for finetuming, set K = 64 in all experiments, and uniformly sample N E [16, 48] images per scene during training. Data augmentation GS-Reasoner, as the autoregressive objective tends to overfit object grounding under limited 3D data. We avoid conventional point cloud augmentations (e.g.. jittering elastic distortion) already covered in Sonata's 's pretraining, and instead focus on decoupling geometric and positional cues. Specifically, we apply Z-axis rotations of [90", 180" 270%). random scaling within [0.75, 1.25], and translations within 1, 1] meters, which alter bounding box positions and scales, forcing the model to exploit both cues for accurate predictions.
crucial for training
B.2 INFERENCE DETAILS
We develop a pipeline recover metric depth and camera parameters from multi-view images, enabling spatial reasoning without any input beyond images. Specifically, we first use VGGT- SLAM (Maggio et al., 2025) to reconstruct dense depth maps and relative camera intrinsics and extrinsics from the multi-view images. We then apply MoGe-2 (Wang et al., 2025e) to estimate absolute-scale depth maps and per-image camera intrinsics independently. Since the intrinsics from these two methods may not be aligned, Instead, we project all points into the camera coordinate system and compute a global scale factor such that the scaled VGGT-SLAM point maps align with the corresponding MoGe-2 point maps across all views. Formally, is obtained by solving the following optimization problem:
avoid direct scale estimation in the depth dimension.
Mc
arg mim
|s
PE.
SLAM
MoGe-2 P.j
1|2
where p) VGGT-SLAM and p
MoGe-2 denote the j-th point in the i-th view from VGGT-SLAM and MoGe- 2, respectively, M, is the number of valid points in view and N is the total number of views. Furthermore, we compute a per-scene axis-alignment transformation matrix based on the estimated camera poses and reconstructed point clouds.
18

---

Published as a conference paper at ICLR 2026
Table 9: Ablation study on the impact of predicted depth maps and poses on spatial reasoning.
Numerical Question
Multiple-Choice Question
Recon Methods Align Methods
Avg.
Obj. Cat.
Abs. Dist.
Obj. Size
Room Size
Rel. Dist.
Rel. Dir.
Route Plan
Appr. Order
70.9
70.6
42.8
VGGT-SLAM VGGT-SIAM
MoGe-2
64.7
69.1
61.9
70.0
65.7
65.4
88.9
44.3
52.3
GT-Deetth
67.8
69..4
68.9
76.5
78.8
65.2
87.6
43.4
$2.9
VGGT
MoGe-2
59.6
66.8
55.6
65.5
63.1
58.3
77.3
34.0
CUT3R TTT3R
56.8
66.8
55.3
60.2
50.7
59.8
76.6
34.0
50.7
58.0
67.5
57.6
60.2
46.5
62.8
82.9
35.1
51.5
C ADDITIONAL RELATED WORK
Point Cloud Representation Learning. Point cloud representation learning has been extensively studied for 3D understanding. Early works like PointNet (Qi et al., 201 20;7;;)) use MLPs and sym- metric functions to extract global features from point clouds. More recent methods, such as Point Transformer (Zhao et al., 2021; Wu et al., 2022; Qi et al., 2023b; Wu et al., 2024; 2025b), leverage attention mechanisms to capture local geometric structures and point relationships. ACT (Dong et al., 2022) pioncers cross-modal geometry understanding through 2D or language foundation models such as CLIP (Radford et al., 2021) or MAE (He et al., 2022). RECON (Qi et al., 2023a; 2024) further proposes learning paradigm that unifies generative and contrastive learning. Despite architectural differences, these methods share a common pipeline: points are grouped based on spatial distribution, features are extracted per group, and then aggregated into a global representation. The resulting sparse features can be upsampled to the original resolution for tasks such as semantic segmentation.
D ADDITIONAL EXPERIMENTAL ANALYSIS AND RESULTS
D.1 ANALYSIS ON GENERAL 3D TASKS
As shown in Tab. 3, GS-Reasoner does not achieve leading results on ScanQA and SQA3D. We believe the main reasons are the presence of many ambiguous questions in these datasets that do not clearly specify the target object, as well as a strong bias in answer distribution. These factors encourage the model to overfit to textual patterns instead of effectively exploiting 3D tokens. Recent studies (Huang al., 2025a; Li al., 2025a) have also shown that finetuning LLMs without 3D input can yield results comparable the state of the art ScanQA and SQA3D, Incorporating reconstruction constraints the output (as done in ROSS3D (Wang et al., 2025b)) may help encourage the model to utilize 3D tokens, and we leave this for future research.
D.2 ABLATION STUDY ON PREDICTED DEPTHI MAPS AND POSES
We further evaluate the impact of different depth and pose estimation methods on the final spatial reasoning performance. Specifically, we consider the following geometry reconstruction methods: VGGT-SLAM (Maggio et al., 2025), VGGT (Wang et al., 2025c), CUT3R (Wang et al., 2025d), and TTT3R (Chen et al., 2025). For relative geometry reconstruction methods (VGGT-SLAM, VGGT), we use MoGe-2 (Wang et al., 2025e) for absolute scale recovery. We also experiment with using ground-truth depth for absolute scale recovery. The results are shown Tab. 9, where the first row corresponds to using ground-truth depth and poses. From the experiments, we summarize two key factors that influence performance:
Accuracy of metric scale. As observed, the most significant performance drop compared to using ground truth depth occurs tasks related to absolute size estimation, such as room size and object absolute distance. During training, GS- Reasoner utilizes ground truth depth, which leads the model to prioritize 3D features in the input for these tasks. Consequently, inaccuracies in metric scale have substantial impact on the results of such tasks. Notably, when using ground truth depth instead of MoGe2-predicted depth for metric alignment, there is significant performance improvement.
Accuracy of pose. The accuracy of pose affects the relative positional distribution of object point clouds within the scene, thereby influencing the model's 's understanding of spatial relationships between objects. In the point clouds estimated by CUT3R, observed significant pose errors,
19

---

Published as a conference paper at ICLR 2026
Question
I standing by window and facing the crytoand, the nefrigenator my front-left, front-right, back-left, or back-night? directions refen to the quadrants of cartessan plane (3f an standing at the origin and racing along the positive y-axis).
back-ieft
-eft
back-right
D. front-right
<thinlo The cuestion asks for the refrigerator's relative direction from my pasition (st the window) while facing the keyboard, i can find these objects as follows first: window cecoro 1.3284, 2. 2,8585, 8.7850, 0.6388, 225888, 1. 1.7580) keyooard 1
(bbort-1, (22222,, 1.3885, 6.685e, -1.8988, 1.7478, a. / refrigerator cbbox 1.8540, -a, -8.2900, -b, 8648, -1.0200, 8.3830, e, c/bo))
The relative direction can be deternined by local coordinate of refrigerator, which can be calcul ated in steps:
Step 1: observer frame
Mindow centroid (observer): (-8.345, -2.999), keyboard centroid: (-1-209, 1. 1.524), Facing vector: (-8.364, 4.413), nomallzed forward (-8.192, 0.981), Right vector (0.98), 0.192)
Stop 2: Refrigerator position Befrigenaton controld: (-1,337, 8,887), Vector (abserver rofrigerstar): (-8.992, 2.896)
Stup 3: Projection to local coordinates
local -0.465, y.locll 5.857, yocal correct.c/thies c/thinnk
front; x_local left. option
consuersBl//asae /anaseer>>>>
Figure 4: Qualitative results on VSI-Bench.
often resulting in completely incorrect scene layouts. This leads to a noticeable decline GS- Reasoner's performance on tasks such object relative direction. TTT3R improves upon CUT3R by reducing pose error accumulation during long-sequence inputs, resulting in performance gains.
Beyond these two factors, other aspects have relatively minor impact on spatial reasoning. For instance, the quality of depth details (e.g.. sharpness) does not significantly affect -S-Recsoner.
D.3 MORE QUALITATIVE RESULTS
We present qualitative results of GS-Reasoner on VSI-Bench (Yang et al., 2025) as follows:
E
FUTURE WORK
Spatial reasoning is key aspect of robotics and embodied reasoning, especially for the vision- language-action (VLA) models (Kim et al., 2024; Qi al., 2025; Zhang et al., 2025). Leveraging the strong spatial reasoning ability of GS-Reasoner in robotic tasks can substantially enhance the generalization and robustness of embodied reasoning. Future directions include jointly fine-tuning with GCoT data and action data, and employing GS Reasoner as an embodied brain for planning and task decomposition.
F THE USE OF LARGE LANGUAGE MODELS (LLMS)
In this work, we leverage LLMs to facilitate the construction of our Grounded Chain-of Chain-of-Thoughr (GCOT) dataset. Specifically, the generation of CoT paths for spatial reasoning tasks is performed using LLMs, which allows us to capture rich intermediate reasoning steps that go beyond simple question-answer pairs.
20

---

Published as a conference paper at ICLR 2026
Question
back left, back facing Cartesian front-left, front- The directions refer to along positive y-asis).
standing
front-right
65 Reasoner
<thinko cuestion IV's direction objects follnes cB. sofa -b.2078)//bbox>, (dm00(3).931, 7948, 0.0661, -1.8902, 2.5420, c/2600)
while chioss(-1.a00) <bbox>(0.2260
deternined
coordinates the TV,
stops: Observen
Stove centroid (observer): (-1.685, -1.353), Sofa
(0.238, 1.996), Facing (2.312, 3,339), normallzed forward (0.576, 8.822), Right vector (0.822, -8.594)
2: positian
centrold: 6-2, 2-100), Vector (observer TV): (-9.250, 3-523)
Step 3: Profection to local coondinates
x x_local -2,265, y_ y_Jocal 2.760, yLlocal fronts x_local left, Among options A is correct. </thisko conswersAc/answer>
Figure Qualitative results on VSI-Bench.
Question
Keasuring
of
nonitor,
clock
ReRRsennnn
<hhinloThe cuestion asks which object (clock, nonitor, trash bin, window) is closest the coor. can find these objects as follows first:
coor dbboxsf-e 9539522, -1. 11.7866, -0,0199, -a 8.829,, -1.7058, 1. c/bboe, clock tthoro-1.4200
-8.2518, 2.asen, -1-1655, a. a.nse8, (aass)c/booss, sonitor <booso (- 1, 188,, e, 3505, B.a65a, -1.1558, 8.878,, 1-2751)</hbmr>, trash bin (-1-2309,, e, 9848, -8 6389, -8 9180, 1,2350, 8.3458)</bbox>, vindow 1 (1b,30-1..348, 1.7945, 8,7688, 1 1.2428, 251588, 2. 2.5025)</ebor>
The closest object can be deteraineed by the wintmun distance to the door, calculated as: eistance sart(ge"2 gy"2 22"2) colculatioss:
Monitor 6.418,
Clock
1.474, 82 9,178 dist 1.548
Trash bin 8.282, 2.855,
0.308, 2.689, 0.080 dist 2.855
0,000 dist 2.089
kindaz 8,008, 5.490, 0.008 dist 3.490
Conclustio: clock closest the door. Aeong aptions, correct. </thinks <answerskc/anseer>
Figure Qualitative results on VSI-Bench.
21

---

Published as a conference paper at ICLR 2026
Question
What is the length of the longest divension (length, width, or height) of the bed, neasured in centineters?
Re-Reasoner
<thinksThe question askingg bounding box
this, identiff
bed eaboxo cabax--11. (-1.4882, -B. 3987, -0.0079, -8.3592, 1.93,,, 1.8011)</2box>
The boundiig box the bed is given neters. Calculating dinenstons:
Length (xeaxts): -0.3591 [-1 (-1.4892)
1,1211
kidth (y-axis); 1,9181 (-8.3567) 2.2668
Helght (2 axts): 1.6811 (-8.6879) 1.0998
The longest dimension is width (2,2668 n). Convert to cs: 2.2668 188 226.58 227
ce</thisko
aanseer>2224aanswer
Figure 7: Qualitative results VSI-Bench.
22


Reviews:
[{"review_id": "t6GMwkkQne", "reviewer": "Reviewer_ckhy", "summary": "This paper proposes a 3D large language model, GS-Reasoner, and introduces a new dataset, GCoT. The GS-Reasoner constructs a semantic–geometric hybrid representation of 3D scenes through a dual-path pooling mechanism. The model achieves strong performance on both 3D visual grounding and spatial reasoning tasks without relying on pretrained 3D detectors or external decoders.", "questions": "1. The proposed GS-Reasoner shows a relatively larger performance gap between Acc@25 and Acc@50 on ScanRefer, and between F1@25 and F1@50 on Multi3DRef, compared with other methods. Could the authors clarify the reason behind this discrepancy? Additionally, what strategies might help narrow this gap?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1. Clear motivation addressing key limitations of 3D LLMs. The work is both meaningful and timely, as it explores how to empower 3D LLMs with spatial reasoning and visual grounding capabilities without depending on pretrained 3D detectors or external decoders.\n2. Well-presented methodology and thorough validation. The paper provides detailed methodological descriptions and extensive experiments. The results are competitive on 3D visual grounding tasks and achieve state-of-the-art performance on spatial reasoning and general 3D benchmarks.", "weaknesses": "Weaknesses: \n1. Performance gap with state-of-the-art baselines. The proposed GS-Reasoner still lags behind ROSS3D on several key benchmarks, including ScanRefer (Acc@50), Multi3DRef (F1@50), ScanQA, and SQA3D.\n2. Dependence on external modules for geometry estimation. The proposed GS-Reasoner still relies on VGGT-SLAM to estimate depth maps and camera parameters, which introduces additional dependencies and may limit the model’s end-to-end autonomy.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "VUSqm7R2Pq", "reviewer": "Reviewer_2vrB", "summary": "This paper introduces a unified representation of geometry and semantics, and proposes a method called Dual-Path Pooling to address misalignment issues in deriving per-patch representations. Furthermore, the authors present the GCoT dataset, which incorporates grounding as an intermediate step in spatial reasoning to enhance the spatial reasoning capabilities of MLLMs.", "questions": "The issues identified and the proposed solutions appear reasonable, and the introduced dataset makes a meaningful contribution. While a few minor points remain unclear, I believe they can be adequately addressed during the rebuttal phase.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The two identified misalignments are a valid concern, and the proposed Dual-Path Pooling offers a simple yet effective solution to mitigate them. In particular, the approach of directly sampling 3D points and subsequently interpolating their geometric features is both elegant and effective.\n\n2. I agree that grounding is essential for spatial reasoning, and the proposed GCoT appears to be of high quality.\n\n3. The experiments are sufficient to demonstrate the effectiveness of the proposed method and dataset.", "weaknesses": "1. The paper claims that the lack of a unified 3D representation leads to reliance on external modules. However, as shown in Table 1, while the proposed method significantly improves the Acc@25 metric on ScanRefer, it does not outperform LLMs equipped with external grounding modules on Acc@50. This suggests that although the method enhances the model’s spatial reasoning capabilities, it does not improve its ability to perform precise object localization—and thus does not fully eliminate the need for additional localization modules. Accurate localization may still require such external modules. \n\n2. The efficiency of the proposed model is unclear; it would be beneficial to include an analysis or empirical results demonstrating its computational efficiency.\n\n3. Since GCoT is built upon GPT-4o, it is important to clarify how the authors ensure the correctness of the generated data. Providing a explanation of the validation or filtering mechanisms would help substantiate the quality of the proposed dataset and, in turn, strengthen its overall contribution.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "2RgxHJvtyK", "reviewer": "Reviewer_L2UB", "summary": "This paper proposes GS-Reasoner, and a GCoT dataset, to tackle 3D visual grounding. Specifically, GS-Reasoner encodes image and 3D representations with a semantic geometric fusion model before feeding into a Video LLM for finetuning. GCoT dataset, on the other hand, construct QA pairs, and then augment them with CoT paths with GPT-4o based on the information of the bird’s eye view, object information, and Q&A. GS-Reasoner, when fine-tuned with GCoT, performed better on multiple visual grounding datasets, including ScanRefer, Multi3DRef, and spatial reasoning benchmarks like VSI-Bench, etc.", "questions": "The main questions I have derived from the Weaknesses.\n\n1. Is GS-Reasoner set up purely for visual grounding? Or does it actually help with more generalized spatial reasoning to other scales and other types of questions?\n\n2. Does fine tuning on the GCoT dataset help with visual grounding? Does it help with other spatial reasoning benchmarks?\n\n3. Does CoT within the GCoT dataset help with visual grounding and other benchmarks?\n\nOverall, I think the current stage of the experiments are not comprehensive for me to accept just yet. I hope the authors can shed light on some of the questions listed above.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "I think the architecture itself is a great contribution. Fusing depth and image seems like a straightforward but valid approach to improve spatial reasoning. I do wonder whether other information like normal maps could potentially give similar results.\n\nFor Table 2, I think showing the results of predicted and GT depth is a great addition, as it allows us to understand the upper bound of the current architecture and training.\n\nBaselines are strong, spanning across closed and open-source models, as well as expert VLMs, with recent state-of-the arts such as VLM-3R-7B.", "weaknesses": "One of the main claims of this paper is that 3D visual grounding is the cornerstone of spatial reasoning. This suggests that improvement on spatial grounding would generally improve Spatial VQA, and from this perspective I find just the set of benchmarks tested is slightly lacking given the large variety of spatial reasoning benchmarks these days. To my understanding, SQA3D, Scan2Cap, and ScanQA questions are still majorly descriptors/grounding of 3D objects in a scene in similar scale. VSI-Bench is also heavily based on ScanNet and ScanNet++. The full story of spatial reasoning may not be told with these datasets alone. Some additional benchmarks that could be helpful (not asking for all evaluation but some additional ones with larger domain shift): SPAR-Bench, All-Angles Bench, MMSI-Bench, etc. This would give us insights on better visual grounding that translates to general 3D understanding. \n\nThe same questions also apply to training with/without GCoT and with/without CoT within the GCoT dataset. I believe understanding these would make the paper more comprehensive and solidify the claim that visual grounding is highly correlated with the other spatial reasoning tasks with larger domain change.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "i6exx98UAQ", "reviewer": "Reviewer_TRwm", "summary": "This paper introduces GS-Reasoner, a novel spatial reasoning framework that enhances vision-language models' spatial understanding by incorporating 3D visual grounding as an intermediate reasoning step. Its core contributions include: a semantic-geometric hybrid 3D scene representation that aligns and fuses semantic features, geometric features, and 3D positional information via a dual-path pooling mechanism; the GCoT dataset, which provides 3D bounding box annotations and chain-of-thought reasoning paths to integrate grounding into spatial reasoning; and an autoregressive grounding capability that, enables end-to-end 3D localization without external detectors or modules, achieving competitive performance across multiple benchmarks.", "questions": "See Weakness.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1.The paper is well written and easy to follow\n2.The dual-path pooling mechanism effectively mitigates misalignment issues between semantic-geometric and position-geometric features.\n3.The GCoT dataset fills a critical gap in existing resources for integrated \"grounding-reasoning\" tasks, and extensive evaluations across 3D grounding, spatial reasoning, and general 3D tasks provide comprehensive validation.", "weaknesses": "1.The training stage is mainly focus on 3d reasoning,  I'm curious about whether the performance of GS-Reasoner on general-purpose benchmarks would decline after being trained on such a large amount of 3D-related data, and if so, by how much.\n2.The construction pipeline of this representation method is somehow complex. It requires parallel execution of a 2D vision encoder, a 3D point cloud encoder, and an additional dual-path pooling fusion module, I wonder what is the. compute cost of these modules.\n3.The \"geometric\" component of the entire representation heavily relies on the quality of depth maps and point clouds generated by preceding steps. In real-world scenarios, depth estimation (e.g., via VGGT-SLAM or MoGe-2) is inherently imperfect, I think it will be better to show how these noise will affect the performance of model.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)