SSR: Enhancing Depth Perception in Vision-Language Models via Rationale-Guided Spatial Reasoning
Authors: Yang Liu
Ming Ma
Xiaomin Yu
Pengxiang Ding
Han Zhao
Mingyang Sun
Siteng Huang
Donglin Wang
Venue: NeurIPS 2025 poster ()
Keywords: Visual-Language Model
Primary area: applications

Abstract: Despite impressive advancements in Visual-Language Models (VLMs) for multi-modal tasks, their reliance on RGB inputs limits precise spatial understanding. Existing methods for integrating spatial cues, such as point clouds or depth, either require specialized sensors or fail to effectively exploit depth information for higher-order reasoning. To this end, we propose a novel Spatial Sense and Reasoning method, dubbed SSR, a novel framework that transforms raw depth data into structured, interpretable textual rationales. These textual rationales serve as meaningful intermediate representations to significantly enhance spatial reasoning capabilities. Additionally, we leverage knowledge distillation to compress the generated rationales into compact latent embeddings, which facilitate resource-efficient and plug-and-play integration into existing VLMs without retraining. To enable comprehensive evaluation, we introduce a new dataset named SSR-CoT, a million-scale visual-language reasoning dataset enriched with intermediate spatial reasoning annotations, and present SSRBench, a comprehensive multi-task benchmark. Extensive experiments on multiple benchmarks demonstrate SSR substantially improves depth utilization and enhances spatial reasoning, thereby advancing VLMs toward more human-like multi-modal understanding. Project page: [https://yliu-cs.github.io/SSR](https://yliu-cs.github.io/SSR).

SSR: Enhancing Depth Perception in Vision-Language Models via Rationale-Guided Spatial Reasoning
Yang Liu lit Ming Ma Xiaomin Yu* Pengxiang Ding ..2.f.8 1.ng..2.t.8 Han Zhao! Mingyang Sun Siteng Huang", Donglin Wang"* 2* Westlake University, ZZhejiang University. Harbin Institute of Technology, 'The Hong Kong University of Science and Technology (Guangzhou), Shanghai Innovation Institute (1iuyang67, wangdongl in) @westlake edu. cn
Abstract
Despite impressive advancements in Visual-Language Models (VLMs) for multi- modal tasks, their reliance RGB inputs limits precise spatial understanding. Existing methods for integrating spatial cues, such as point clouds or depth, ei- ther require specialized sensors or fail effectively exploit depth information for higher-order reasoning. To this end, we propose a novel Spatial Sense and Reasoning method, dubbed SSR, novel framework that transforms raw depth data into structured, interpretable textual rationales. These textual rationales serve as meaningful intermediate representations significantly enhance spatial reasoning capabilities. Additionally, we leverage knowledge distillation to compress the generated rationales into compact latent embeddings, which facilitate resource- efficient and plug-and-play integration into existing VLMMs without retraining. To enable comprehensive evaluation, we introduce new dataset named SSR -CO, a million-scale visual-language reasoning dataset enriched with intermediate spa- tial reasoning annotations, and present SSRBENCH, comprehensive multi-task benchmark. Extensive experiments multiple benchmarks demonstrate SSR substantially improves depth utilization and enhances spatial reasoning, thereby advancing VLMs toward more human-like like multi-modal understanding. Project page:
cs. github. io/SSR.
Introduction
VLMs represent pivotal advancement in bridging the gap between image and natural language. demonstrating astounding capabilities across myriad multi-modal tasks [1-7]. Nevertheless, relying solely on RGB inadequate for accurately capturing spatial information such as relative positions and distances, which presents inherent limitations in capturing precise spatial relationships, thereby constraining the capacity of VLMs comprehend complex scenes. Consequently, enhancing the ability of VLMs to understand and reason about spatial relationships is essential for critical real-world applications, particularly robotics.
Recent advancements in VLMs have catalyzed research on explicitly incorporating spatial information enhance model performance. While some methods leverage point cloud data for improved spatial understanding [8-10], they typically rely on specialized sensors (c.g.. LiDAR) that are impractical in scenarios restricted to monocular RGB images. In this context, monocular depth estimation has emerged as a compelling alternative, particularly with the proliferation of generative methods [11, 12]. These methods enable the acquisition of high-quality depth images from standard 2D images through
"Corresponinng author. 'Equal contribution. SProjcct lead.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

imsago
Inage
BSRBh
VLM
Dhipth
VLM
Amsiituue
Coary
Guary
Faliue Eeirarre
Cemtral
SSR-Col
Clapst
Performance Compar son
Spulial
Ques2 5-VL
358
Figure 1: Unlike conventional VLMs, SSR integrates depth perception to enhance spatial reasoning. We introduce a curated dataset SSR-COT and benchmark SSRBENCII, demonstrating significant improvements in spatial reasoning tasks.
various pre-trained models [13-16], eliminating additional hardware requirements. By leveraging visual encoders pretrained on RGB images, depth features can be efficiently encoded and seamlessly integrated into VLMs, offering a promising pathway for enhancing spatial awareness.
However, a critical limitation of current methods lies in their superficial utilization of depth informa- tion [8-10, 17-20J. Unlike humans, who intuitively employ depth as an integral component within broader reasoning processes, existing methods incorporate depth explicitly without capitalizing on its inferential value [17]. Consider query such Are objects A and B far apart? Human cognition naturally analyzes the spatial relationship between objects and then leverages this understanding to inform subsequent reasoning. This implicit reasoning process underscores the necessity for more sophisticated integration of depth information into VLMs, not merely as supplementary input, but as fundamental component that facilitates complex spatial reasoning. Developing methodologies that emulate this human-like implicit utilization of depth could substantially enhance VLM capabilities.
To this end, we propose SSR, novel paradigm designed to redefine the integration of depth informa- tion within VLMs. Specifically, SSR translates raw depth data into a structured rationale language, providing interpretable intermediate representation that bridges low-level depth perception and higher-level reasoning. This rationale-basee based language facilitates VLMs in generating outputs that both more accurate and contextually appropriate, while also enabling the previously underuti- lized inferential capabilities inherent to spatial depth information. By converting modality-specine depth data into semantically rich and inherently aligned representations, SSR effectively overcomes the interpretability limitations associated with traditional approaches. Consequently, this method significantly enhances the utilization of depth information, laying the groundwork for more robust and human-like spatial reasoning capabilities within contemporary VLMs. To further enhance the efficiency of rationale-language utilization, we transform depth information into compact latent strategy to compress rationale-languae representations into concise latent embeddings [21]. Dissimilar vanilla
embedding. Specifically, we apply a knowledge-distilant
(CoT) methods [22, 21, 23, 24] that rely primarily on textual explanations, our distillation strategy signifi- cantly reduces computational overhead while preserving the depth and inferential richness inherent to rationale- based representations 25-27J. Importantly, this module can be seamlessly integrated into existing VLMs via a training- ffree mechanism, highlighting the flexibility and broad applicability of the proposed framework. To achieve SSR, we first curate SSR-CoT, million-leve rision-languge spatial reasoning dataset that facilitates depth-aware reasoning and provides a robust foundation for developing sophisticated spatial reasoning models. To validate our approach, we perform extensive experiments across multiple benchmarks. Specifically, we also evaluate on our benchmark SSR- BENCH, which comprises six distinct tasks spanning both general and spatial domains. Extensive

---

experiments and analysis demonstrate that SSR substantially enhances spatial reasoning capabilitiess across diverse tasks, highlighting the effectiveness and broad utility of our proposed method.
Overall, our principal contributions in this paper are illustrated in Figure 1 and summarized as follows:
We propose an efficient VI.M, dubbed SSR, capable of simultaneously performing depth perception and spatial reasoning, and generating answers based on implicit reasoning rationales.
We introduce SSR-CoT, million-scale visual-language reasoning dataset enriched with interme- diate spatial reasoning annotations, and present SSRBENCII, a comprehensive multi-task benchmark.
Extensive experiments and solid analysis across various benchmarks demonstrate our SSR can efficiently and dramatically enhance the spatial understanding of existing VLMs.
2 Related Work
2.1 Visual-Language Models
LLMs [24, 28-38] have led to major advancements in Natural Language Processing (NLP) tasks, and also have incited interest in developing VLMs. Building a unified LLM with visual inputs for visual language tasks thus remains one of the most important desiderata for VLMs. Over the last few years, VLMs achieved significant performance improvements in multi-modal tasks by integrating a pre-trained visual encoder and projecting the feature into semantic space into LLMs as well as training on large-scale multi-modal question-answering pairs [1, 2, 39-43]. This straightforward method can work well for general tasks, but expecting the model deduce answers for more complex tasks without deep reasoning can be daunting.
2.2 Multi-Modal Reasoning
LLMs display an emergent capability for step-by-step reasoning through in-context learning, a phenomenon referred to as CoT reasoning. Such reasoning significantly enhances the performance of LLMs on complex reasoning tasks [22, 44, 45]. Concurrently, notable advancements have also occurred in multi-modal CoT research, paradigm appealing due its similarity to human problem- solving behaviors [46, 47]. Current research efforts in multi-modal CoT primarily emphasize the construction of intermediate reasoning rationale datasets to train image-text reasoning models. Many existing studies adopt rich textual captions and detailed descriptions as intermediate rationales [48- 51]. Beyond text-based rationales, recent approaches have leveraged mouti- rationales for more comprehensive reasoning [52-56]. However, existing multi-modal CoT methods primarily focus on tasks involving code generation, mathematical problem solving, and general question answering, which require sophisticated reasoning to achieve accurate responses. In contrast, this paper introduces an efficient CoT method that leverages depth images to enhance the performance of VLMs, particularly by improving spatial understanding.
2.3 Spatial Intelligence
Spatial reasoning is an essential capability for VLMS and has therefore been included in several Visual Question Answering (VQA) benchmarks [57-60]. However, the majority of existing VLMs [7, 61-64] are primarily trained on two-dimensional images paired with textual data, a setting that inherently lacks comprehensive spatial information. Consequently, these models exhibit limited performance in spatial reasoning tasks. To overcome this limitation, recent works such as Spatial VLM [10], SpatialRGPT [65] and RoboRefer [66] have sought to improve the spatial reasoning capacity of VLMs by compiling specialized ppatiall--orientee question- guestion-answer answer datasets and fine-tuning models accordingly. Nevertheless, despite these advancements, prior approaches largely neglect the integration of language-based based reasoning capabilities within the spatial reasoning framework. This omission hampers the effectiveness of existing VLMs in addressing more complex tasks, particularly those requiring intricate multi-step reasoning processes.
3

---

Visual- Language Model
Lange Langagg
image
Deoth
MIDI
mor and moe
Thaiiin Cojectivv Spudial Fiblaeed CoT Tuticralee
Thaiiin Ohacsivvs draveing
Visual-Language Model
Large Languagg
quago
Lisert Projador
Litarn Projuctor
Efficient Language Model
Elficient Larguagge Modill
Effizerti L Langaage Maxch
MIDI
What tr
Ciooder
Shocoder
Srubs
Figure 2: Schematic of SSR framework. (a) Overall pipeline. (b) Full architecture of SSR, comprising the MIDI module followed by the VLM. (c) Two training stages of the SSR. In the stage 1, the LLM provides alignment supervision for the MIDI module, whereas the stage is optional.
3 Methodology
3.1
Architecture
The primary goal of our proposed SSR effectively leverage the reasoning capability of efficient language models effectively enhance the depth understanding and spatial reasoning capability for existing VLMs. The overall framework is illustrated in Figure 2.
3.1.1 Image-Depth Interpreter
To achieve comprehensive spatial understanding via depth interpretation, we propose a simple yet effective plug-and-pp -lay module entitled Mamba-based Senge-Depth Interpreter (MIDI). MIDI generates enriched depth-aware latent token representations, providing essential spatial reasoning information before feeding these tokens into the VLM.
Given an input image Xy Hx x3 and a corresponding textual query XT. we first utilize pretrained monocular depth estimation model, Depth Pro [15], to produce depth Xp E R #xWx x the image. Subsequently, image features Zy and depth features ZD are extracted from Xy and Xp. respectively. Specifically, we employ pre-trained CLIP ViT-L/14 [67, 68] as the visual encoder Ey, and SigLIP [69] as the depth encoder Ed: Ha = (X.),, a E (V, D). Then we apply Multi-Layer Perceptron (MLP) modules, comprising two fully connected layers with GELU [70] activation, as projectors Φv and &D, transforming these visual features into the semantic embedding space compatible with the subsequent efficient language model: 2. d. (Ha), e DJ. To jointly encode visual and depth information conditioned on the textual query, we introduce an intermediate reasoning module implemented as the Mamba-based language model [71], denoted as fLM. This module produces latent tokens representing intermediate spatial rationales: Specifically, we uniformly insert several additional special tokens into
HR fiM the rationales to facilitate the knowledge distillation process and encode textual tokens into latent representations [21]. Finally, similar to previous steps, we apply an additional latent projection module ΦR to map these latent rationale tokens into another semantic embedding space, matching the dimensionality of the word embeddings used in the subsequent VLM: ZR QR(HR).
Hence, our proposed MIDI module generates a sequence of spatial-aware latent tokens Z These tokens can easily be plugged into the query sequence for existing VLMs, effectively injecting depth-based spatial reasoning information and significantly enhancing the spatial understanding capabilities.

---

Table 1: The mixture detail of SSR-COT dataset. SSR-COT consist over million image-depths answer pairs, where the rationale containing rich spatial-related knowledge the
question-rationan -anseee
enhance Visual- Models (VLMs).
Dataset
Source
Size
Dataset
Source
Size
ShareGPT4V [72]
31.3k
Flickr30k [73]
136k
CharQQ [74]
17.2k
GQA [57]
88k
A-OKVQA [75]
16.1k
Visual7W [76]
43k
Visual-CoT [52]
A12D [77]
11.4k
OpenImages [78]
43k
GeoQA+ [79]
11.4k
Birds-200-2022 [80]
10k
LLaVA-CoT [51]
ScienceQA [81]
5.6k
VSR (82]
DocVQA [83]
4.0k
SCREENED TOTAL
289k
PISC [84]
1.0k
GQA [57]
72k
CLEVR [85]
0.5k
VoCoT (53)
LLaVA-Instrun [86]
6k
CLEVV- [87]
0.5k
LVIS [88]
TOTAL
98k
SCREENED ONE-TURN
TURN TOTAL
317k
SpatialQA [17]
Bunny [89]
695k
OXE (90]
7.5k
SCREENED TOTAL
501k
3.1.2 Spatial Sense and Reasoning
Our proposed MIDI module fully leverages spatial information derived from the depth images and generates a latent representation ZR. which encodes intermediate reasoning rationales essential for producing the response. Subsequently, image Xy and textual question X7, into an existing VLM fVLM to generate the answer YA in an auto-regressive manner: YA fVLM (Xv, Zn, Xr)-
input these latent tokens Zn. alongside the original
3.2 Training Paradigm
For training the proposed SSR, we adopt a two-stage procedure, as illustrated on the bottom-right side of Figure 2. In Stage 1, we train the underlying MIDI module to generate rationale latent tokens and project them into the language semantic space. In Stage 2, we conduct joint training of the MIDI module and existing Vision- Vision-Language Models (VLMs) to further enhance performance. Notably, Stage is optional due to the modular and plug-and- and-play nature of our MIDI module, enabling straightforward integration into existing VLM frameworks.
3.2.1 Stage 1: Reasoning and Alignment
At the initial stage, we aim to train an efficient language model within the MIDI module, enabling to generate and encode coherent thought processes represented by sequence of features consistently aligned with the natural language semantic space. To this end, each training sample at this phase includes detailed and accurate rationale YR the ground truth. After feeding latent tokens produced by the MIDI module into the subsequent LLM, we require the LLM to reconstruct the original textual rationale solely from these latent representations. Training the LLM for precise rationale recovery depends not only on accurate reasoning capabilities of the MIDI module itself, but also on successfully projecting latent tokens into semantic space consistent with the frozen-state L.M..
The learning objective for Stage is defined by the standard causal modeling loss, given by:
L (0) =
2n.
E!'il [f
(all log Po(YR, Xv,Xn, Xr, ZR, YR,cs)
(1)
Following this training stage, latent tokens Z generated by the MIDI module can be readily integrated into existing VLM image-text sequences, thereby enhancing their spatial understanding capabilities.
3.2.2 Stage Co-Training
To further enhance the performance of SSR, we jointly train the MIDI module along with existing VLMs. Similar to instruction-tuniin we discard intermediate rationales in the second training stage and allow the VL. to directly generate the final answer. this setting, accurate answer generation by the VLM requires not only effective reasoning from the MIDI module but also the capacity of
5

---

Image Toxt
Table 2: Quality evaluation for SSR- CoT dataset. We conduct the evalua- tion based on the powerful visual- language model Qwen2.5-VL-7B. Instruct [91].
SpataROFT
GFT-60
Ratinalss
Net dverr Dasry
Partaxx
Bound ng Box
Rationale Accuracy
Score
Exolution
Sondling
67.80
3.6721
79.42
4.1289
BBR:DoT
Samples
Cacha Fool
(*11622
(10.4568)
Figure 3: Schematic of SSR-CoT annotation pipeline.
Image & Depth
Quastion
Wha a puated pessen
Wisl cky
peabed
perzee as awuncyyk peatcamed ereisnennn casthoe materyyc sxnaxing retorycke theetees entoryyl that pesson
leatel condzaa 423 The
(Io beee Eootthed posies queine The rauch for the objen bearth the dighart x thes kronsh vieen olpelann hhrckee dictten saning
olyphae decembed lego ane Ulioss olphani, with
Ratoona
the penss
223 03335-2225
Arawar
exkmayde
mal
Figure 4: Illustrative samples of SSR-CoT dataset.
VLM comprehend and utilize the reasoning information. Specifically, the second-stage learning objective is formulated as standard cross-entropy loss for auto-regressive generation of the final answer Yx:
(0) Exxy.Ao.
E/ IYal ==
log Po(Ya. Xv. XT. YA.s))
(2)
M
Since the rationale serving as the ground truth for supervised learning is omitted during Stage training, we can incorporate additional VQA pairs to expand the dataset, thereby enhancing the generalization capability of the model. Furthermore, training during this stage is optional due to the modular houg-and-play nature of the MIDI module.
Experimentation
4.1 SSR-CoT Collection
There is scarcity of langaale CoT datasets with detailed reasoning processes annotations to train the SSR model for depth perception and spatial understanding. Therefore, we curate a new dataset from existing VQA datasets, resulting over a total of million image-depth- question-rational--anwe pairs. There are four dataset sources we integrated: (1) LLaVA-CoT [51]: Systematic and structured reasoning risual-language CoT datasct, including general and science- targeted VQA data source. (2) Visual-CoT [52]: Multimodal CoT dataset that takes the bounding box as an intermediate thinking step. including general, relation reasoning and fine-grained science- targeted VQA data source. (3) VoCoT [53]: Fine-grained image-text CoT dataset that rationale provides detailed relationships between various objects with bounding box, including general and relation reasoning VQA data source. (4) SpatialQA [17]: Spatial QA dataset for sufficient utilization, including depth-ree related and robotic-related VQA data sources.
To generate visual- llnguage reasoning data enriched with spatial information, we follow a multi-step process, as shown Figure 3. First, we extract depth estimations from raw images using Depth Pro [15]. For the LLaVA-Cor CoT [51] source, this is the only preprocessing step performed. Second, for

---

General (432)
Spatial (357)
Nho
[0] many benk parked the
Chinass 1007
Goa'tiy (40
[A,
(C) Aur Eee claci
[Q] How
seria th
[A]TK mnncriago.
[Q] korto
la peaat sulking sukliigpppee dadging (A]
point de ndua? righ
Actua 1577
Posli Uid
[A]
[QJ calor ofgght
[A]R
What caler eatatilattic
[RU Wartsacc bout?
A Sprce
Arribat 1160
sood and horve is calle
dlipes 1996
veneey harm evntert: buluaa
knywuad hore onl
[A] wiliess kaukes nalioo wedd hevve meeees
Figure 5: Examples for each task within the benchmark SSRBENCH.
datasets such as VoCoT [53] and SpatialQA [17], we refine long-form conversations by extracting concise, one-turn question-anseen answer pairs. Third, leverage SpatialRGPT [65] comprehensively mine precise spatial attributes within images, such as object size, distance, and relative positioning, based on intermediate reasoning steps, including bounding box annotations from Visual-CoT [52] and VoCoT [53]. Finally, we employ GPT-4o [92] to integrate all extracted information, generating detailed reasoning processes that enhance spatial understanding. Notably, we also incorporate cache pools and perform sampling quality checks within iterative loops to ensure the high quality of the generated data. Specifically, similar to the quality-assessmenn protocol shown Table 1, we randomly draw 10% of the cached samples and evaluate VQA accuracy both with and without their generated rationales. Rationales that improve accuracy are retained and incorporated into the final SSR-COT dataset; those that degrade performance are discarded, and the all samples in cache are re-submitted for re-annotation. Overall, we compile upproximately 1.2 million preprocessed data samples into SSR-CoT dataset. Figure 4 illustrates several data samples from the SSR- -OR dataset. Each data instance within SSR-CoT comprises the original image, an associated question-answe pair, the corresponding estimated depth information, and rationale. The rationale incorporates fundamental reasoning steps used in duestion-answerr tasks and provides detailed spatial reasoning to support accurate answer generation.
To evaluate the quality of SSR-COT, we conducted an assessment based on the performance of the Owen2.5-VL-7B-BB
[91] on the VQA task. This evaluation was carried out on randomly selected subset comprising approximately 1% of the full dataset, corresponding roughly to 10k samples. Performance metrics include accuracy as well as quantitative score ranging from 0 to 5, both are produced using the LLM-Assistant powered by the Qwen2.5-14B 1-n.tttttt----stuuttt------ IM [38, 93]. Further methodological details regarding the evaluation process described in Appendix D. As presented in Table 2, responses generated with intermediate rationales demonstrate an accuracy improvement of more than 10% compared direct question-answering This finding indicates that the intermediate reasoning rationales annotated in our dataset are of high quality and effectively enhance the question-ane -anwweiinn performance of VLMs.
4.2 SSRBENCH Construction
Currently, there are established benchmarks specifically designed for evaluating spatial under- standing and reasoning capabilities on image-text pairs. To address this gap, we propose SSRBENCH, a novel evaluation benchmark created from the SSR-CoT dataset. Importantly, the data incorporated into SSRBENCH will be fully removed from SSR-COT to prevent overlap. SSRBENCH consists of two primary categories, general understanding and spatial understanding, allowing simultaneous evaluation VLM performance in both general question answering and spatial reasoning tasks. Each category contains three distinct evaluation tasks, with detailed sample sizes provided in Appendix E.
7

---

Table 3: Performance comparison on SpatialBench [17] and our SSRBENCH. SSRBENCH and SSRBENCH $ denote general and spatial tasks, respectively.
Method
Size
SpatialBench [17]
SSRBENCH
SSRBENCH
Mamba
VLM
Position
Existence
Counting
Size
Existence
Attribute
Action
Counting
Position
Object
GPT-4o-minl (92]
Clande3355 3.5 (99)
N/
NIA
53.9
65.0
72.2
26.7
51.4
52.2
43.7
42.0
34.1
38.3
OruN-SouncH
Sp [10, 100)
N/A
3B
$2.9
80.0
77.1
28.3
31.7
38.3
63.7
31.7
55.8
65.4
LLaVA-1.5 [2]
N/A
7B
44.1
45.0
82.8
30.0
81.3
64.3
66.9
43.9
63.6
63.6
MaVA-NeXT
-LaVA-NeXT/BR
75.0
SpatialBBot [17]
80.0
86.7
75.7
61.3
67.5
39.0
74.1
61.7
Ema3 [101]
NIA
47.1
20.0
10.0
25,0
38.9
35.7
37.6
19.5
51,9
37,0
Qwen2 5-VL [91]
N/A
38
55.9
80.0
76.4
25.0
66.4
58.9
63.1
34.1
60.5
51,9
V-LL VL.
NIA
78
61.8
80.0
87.1
30.0
75.7
62.5
70.1
43.9
61.7
55.2
SSR (Ours)
130M
80.0
74.7
SSR (Ours)
85.0
Table Performance improvement of SSR compared to the backbone model. SpatialBench [17]. SSRBENCH, and CV-Bench [102] report average.
Method
Size
SpatialBench [17]
SSRBENCH
CV-Bench [102]
VSR (82)
What's Up [103)
Mamba
VLM
General
Spatial
Random
Zero-Shot
Qwae2. [91]
59.3
62.8
67.0
85.4
SSR (Ours)
(1.5
(120.9
(15.6)
(7.9
(12.))
Quen2 (91)
64.7
SSR (Ours)
130M
7B
$2.1 (1166)
76.1 (222.5)
73.3 (10.3)
NAA
N/A
NAA
We illustrate the process to construct SSRBENCH as shown in Appendix E.2. First, we define 6 distinct task categories. Then, we randomly sampled image-text pairs from SSR-CoT, proportionally retaining the distribution of its original data sources. These samples were independently classified into task categories by GPT-4o [92] and Gemini-2.5-PP [94]. Only instances for which both models agreed the assigned category were included in SSRBENCH; instances with disagreement were returned to SSR-COT.
In recent years, LLMs have demonstrated significant advancements in language understanding, reasoning, and text generation, exhibiting strong perceptual and comprehension capabilities through the implicit world knowledge they encapsulate. Therefore, LLMs have increasingly been used as assessors to evaluate generation performance in question-answeer answerins tasks [7. 1, 95, 96]. Consistent [38, 93],
with our approach in data quality evaluation, we employ the powerful LLM, to evaluate the performance of VLMs this benchmark.
4.3 Implementation Details
In this paper, we utilize Mamba [71] as the lower-level efficient language model for reasoning, Qwen2.5 [38] as the LLM for alignment the first training stage, and Qwen2.5-VL [91] as the VLM supporting multi-modal comprehension in the second training stage. During Stage 1, we exclusively train the MIDI component on our proposed SSR-COT dataset. Stage 2, we jointly train the SSR using both the SSR-CoT dataset and the LLaVA--nstruee 150K dataset [1]. Leveraging the efficiency of LoRA [97] and Fully Sharded Data Parallel (FSDP) [98], training SSR requires approximately 19 hours for Stage 1 and 48 hours for Stage 2, using single Nvidia 8-H800 GPU node equipped with 80GB VRAM. Detailed hyperparameter configurations are provided in Appendix A.
4.4 Main Results
Table 3 presents the comparative performance of the SSR against its backbone and state-of-the-ar baselines on SpatialBench [17] and SSRBENCH. As shown by the results, our SSR in 3 billion parameters can achieve comparable even higher results than targe-scale baseline models, including closed-source and backbone models. Our larger variant, comprising 7 billion parameters, yields the best performance on most tasks across the two benchmarks. Compared to the top-performing baselines in each benchmark, SSR exhibited notable improvements in the average question answering accuracy, achieving maximum enhancement of 13.6 and an average improvement of 6.77. Moreover, we also provide a detailed analysis of the performance improvements compared the backbone model across additional benchmarks in Section 5.

---

Table 5: Performance comparison general VQA benchmarks
Size
Method
VQAv2[104)
Te [105]
POPE [106]
MMBencc [107]
GQA [57]
Mamba
VLM
Owen2.5-VL.pp9
N/A
3B
72.5
57.0
84.4
75.9
56.2
SSR (Ours)
130M
(16.)
(14.3)
86.0 (11.6)
78.3 (12.4)
(17.4)
Table 6: Performance comparison among the backbone model, the SSR with/without the second training stage. PAP indicates that the MIDI module was employed in a plug-and-play manner.
Method
Size
Spatialleneh [17]
SSRBESCn
SSRBENCH
Mamba
VLM
Position
Existence
Counting
Size
Existence
Atteibute
Action
Counting
Position
Object
V. (91)
NIA
3B
55.9
$0.0
76.4
25.0
66.4
58.9
63.1
34.1
60.5
51.9
63.7
SSRP
130M
(18.8)
(0.0)
(33.3)
(15.0)
(13.7)
(10.6)
(10.6)
(12.5
(10.6)
(1.3)
SSR
130M
38
(18.3)
64.7
0.0
80.0
(6.5)
82.9
(16.7)
31.7
(116.8)
83.2
(23.2)
82.1
(19.5)
72.6
(17.0
$1.2
(122.8)
83.3
(122.8)
74.7
Analysis
5.1
Performance Improvement
We present additional experimental results Table 4, demonstrating the improved performance of SSR compared to the backbone model across the five benchmarks shown in Table 11 at varying model scales. Specifically, across the three benchmarks reporting average values, SSR models of different sizes demonstrated average improvements of 11.2 and 9.4 compared to the backbone model. The most significant improvements were observed in the space task of the benchmark, where the enhancements reached 20.9 and 22.5, respectively. This result exceeds the improvements reported in Table 2, indicating that our SSR effectively reasons about information highly relevant to multi-modal VQA tasks without introducing significant additional noise. Furthermore, the training paradigm of SSR enhances performance not only on two evaluation datasets closely related to the training data but also on the out-of-domain CV-Bench [102], VSR [82], What's Up [103] and multiple general VQA benchmarks [57, 104-107] shown in Table 5. These findings indicate that our training approach effectively further improves the generality and generalization capability of the SSR in addition to enhanced spatial understanding performance.
5.2
Ablation Studies
As reported in Table 6, these experiments illustrate the performance of the MIDI module when integrated in a plug-and-play manner without second training stage, leading to improved spatial understanding. Specifically,, this approach achieves average performance gains of 4.4 and 1.6 on different benchmark datasets. On certain tasks, the plug-and and-play approach achieves performance improvements of up 8.8, demonstrating the effectiveness of this usage. In addition, after the second stage training, the performance of the complete SSR model will be significantly improved on this basis, achieving average performance gains of 5.7 and 18.7 on different benchmark datasets. Moreover, we provide case studies in Appendix F.
5.3 Efficiency
We evaluate the Qwen2.5-VL [91] after fine-tuning on SSR-COT dataset, overly protracted and convoluted textual intermediate reasoning chains not only increase the risk of erroneous conclusions but also impose prohibitive computational costs that undermine inference efficiency, which can better reflect the importance of latent reasoning method in CoT application. As illustrated in Table 7, the results demonstrate that, although SSR introduces a modest absolute latency per generated token, its latent-reasoning paradigm dramatically curtails the number of CoT tokens needed to reach a final response. Consequently, under the CoT-based evaluation framework, the overall end-to-end inference speed is substantially improved.

---

Table 7: Inference efficiency comparison SpatialBench [17].
Model
Size
SpattalBench
Sample Token Second
Inference Time Per Sample
Qwen2.5.VL (911 (w/ SET on SSR-CoT)
51.3
437.25
18.33
23.16s
SSR (Ours)
3B
64.8
2,62
8.18
0.32x
Table 8: LLM-Assistant Evaluation Performance comparison on SSRBENCH, evaluated using different L.Mss. (For each result, the left column presents the original Qwen-based evaluation score, while the right column reports the corresponding GPT judgment).
Model
Size
Existence
Attribute
Action
Counting
Position
Object
Qwen2.5-VL [91]
3B
66.4 58.9
58.9 59.5
63.1 /7.55
34.1 /3.11
60.5 58.6
$1.9 52.6
SSR (Ours)
3B
83.2 81.3
82.1 79.8
72.6 73.3
51.2/51.2
83.3 80.8
74.7 / 74.7
LLM-Assistant Evaluation
To mitigate potential biases arising from employing models of the same family as judges, re- assessed the SSRBENCII results with GPT-4o-mini [92]. the outcomes are reported Table High inter-model agreement between the evaluation scores assigned by different LLM judges in the table suggests that simple answer-comparison tasks largely resist bias within model series.
5.5 Rationale Embedding
To analyze whether MIDI effectively captures depth information and conducts spatial reasoning guided by rationale, we visualize the co- sine similarity between latent tokens, both with and without rationale. Figure visualizes the cosine similarities between the latent tokens produced in two different paradigms: x-axis: latent tokens inserted inside the rationale, y-axis: latent tokens inserted immediately after the question and used to start the answer generation. Diagonal cells represent these two states of the same sample. High values on the diagonal indicate that the model has learned to map the rationale to the latent representation, confirming that successfully distills the spatial knowledge embedded in the rationale. Low off- aff-diagonal values indicate that the latent tokens remain sample-specis and do not collapse to a generic representation.
Figure 6: Cosine similarity matrix of reasoning latent to- kens with/without rationale.
Conclusion
In this paper, we propose novel VLM SSR with an important module named MIDI to interpret depth for enhancing the depth perception and spatial reasoning capabilities of existing VLMs. MIDI can even be efficiently integrated into existing VLMs seamless, plug-and-plas manner. To enable effective training and evaluation, we curate multi-modal CoT dataset SSR-CoT and present comprehensive benchmark SSRBENCH. Extensive experiments conducted across four distinct benchmarks demonstrate that SSR consistently achieves state-of-the-arr performance enhancements over existing approaches, particularly excelling in spatially-oriente visual question answering tasks.
Broader Impacts. Our proposed SSR demonstrates that spatial reasoning capabilities can be incrementally enhanced without adversely affecting existing VLM functionalities. This provides an innovative avenue for research communities to integrate additional capabilitiess into VL.Ms.
Limitation and Future Works. Although SSR shows astounding performance, this study limited to the Qwen/Qwen- VL series; future work will broaden the VLM scope to test generalizability.
Acknowledgments and Disclosure of Funding
This work was supported by the National Science and Technology Innovation 2030 Major Project (Grant No. 20222D0208800), and NSFC General Program (Grant No. 62176215)
10

---

References
[1] H. Liu, C. Li, Q. Wu, and Y.J. Lee, "Visual Instruction Tuning," in Proc. of NeurlPS, 2023.
[2] H. Liu, C. Li, Y. Li, and Y. Lee, "Improved Baselines with Visual Instruction Tuning," in Proc. of CVPR, pp. (6286-26296 2024.
[3] Y. Yuan, W. Li, J. Liu, D. Tang, X. Luo. C. Qin, L. Zhang, and J. Zhu, "Osprey: Pixel Understanding with Visual Instruction Tuning," in Proc. of CVPR, pp. 28202-28211, 2024.
[4] H. A. Rasheed, M. Maaz, S. S. Mullappilly, A. M. Shaker, S. H. Khan, H. Cholakkal, R. M. Anwer, E. P. Xing, M. Yang. and S. Khan, "GLaMM: Pixel Grounding Large Multimodal Model," in Proc. of CVPR. pp. 13009-13018 2024.
[5] Kirillov, E. Mintun, N. Ravi, Mao, C. Rolland, I.. Gustafson, T. Xiao, Whitehead, A. C. Berg. W. Lo, P. Dollár, and R. B. Girshick, "Segment Anything," in Proc. of ICCV, pp. 3992-4003, 2023.
[6] Y. Liu, H. Ping, D. Zhang. Q. Sun, S. Li, and G. Zhou, "Comment-aware multi-modal heterogencous pre-training for humor detection in short-form videos," in Proc. of ECAI, pp. 1568-1575, 2023.
[7] Y. Liu, Ding. S. Huang, M. Zhang. H. Zhao, and Wang, "PīTe: Texporleporal Alignment for Large Video-Language Model," in Proc. of ECCV, pp. 160-176, 2024.
[8] H. Zhen, X. Qiu, Chen, Yang, Yan, Du, Y. Hong, and C. Gan, "3D-VLA: A 3D Generative World Model," in Proc. of ICML, 2024.
Vision-Language-AAee
[9] Y. Hong, H. Zhen, P. Chen, S. Zheng, Y. Du, Chen, and C. Gan, "3D-LLM: Injecting the 3D World into Large Language Models," in Proc. of NeurlPS, 2023.
[10] Chen, Z. Xu, S. Kirmani, B. Ichter, D. Sadigh, L. J. Guibas, and F. Xia, "SpatialVLM, Endowing Vision-Language Models with Spatial Reasoning Capabilities," in Proc. of CVPR, pp. 14455-14465, 2024.
[11] J. Song, C. Meng, and S. Ermon, "Denoising Diffusion Implicit Models," in Proc. of ICLR. 2021.
[12] J. Ho, A. Jain, and P. Abbeel, "Denoising Diffusion Probabilistic Models," in Proc. of NeurIPS, 2020.
[13] L. Yang, B. Kang, Z. Huang, X. Xu, J. Feng, and H. Zhao, "Depth Anything: Unleashing the Power of Large-Scale Unlabeled Data," in Proc. of CVPR, pp. 10371-10381, 2024.
[14] L. Yang, B. Kang, Z. Huang, 7. Zhao, X. Xu, J. Feng, and H. Zhao, "Depth Anything V2," in Proc. of NeurlPS, 2024.
[15] A. Bochkovskiy, A. Delaunoy, H. Germain, M. Santos, Y. Zhou, S. Richter, and V. Koltun, "Depth Pro: Sharp Monocular Metric Depth in Less Than Second," in Proc. of ICLR, 2025.
[16] X. He, D. Guo, H. Li, R. Li, Y. Cui, and C. Zhang, "Distill Any Depth: Distillation Creates a Stronger Monocular Depth Estimator," CoRR, 2025.
[17] W. Cai, Ponomarenko, Yuan, X. Li, W. Yang, H. Dong, and Zhao "SpatialBot: Precise Spatial Understanding with Vision Language Models," in Proc. of ICRA, 2025.
[18] S. Chen, X. Chen, C. Zhang, M. Li, G. Yu, H. Fei, H. Zhu, Fan, and T. Chen, "LL3DA: Visual Interactive Instruction Tuning for Omni-3D Understanding. Reasoning, and Planning." in Proc. of CVPR, pp. 26418-26423 2024.
[19] J. Zhang. M. Cai, T. Xie, and J. Lee, "CounterCurate: Enhancing Physical and Semantic VinuusL.ig Compositional Reasoning via Counterfactual Examples," in Proc. of ACL. Findings, pp. 15481-15499 2024.

---

[20] W. Yuan. J. Duan, V. Blukis, W. Pumacay, R. Krishna, A. Murali, A. Mousavian, and D. Fox, "RoboPoint: Vision-Language Model for Spatial Affordance Prediction for Robotics," CoRR, 2024.
[21] B. Lee, C. W. Kim, B. Park, and Y. M. Ro, "Meteor: Mamba-basee Traversal of Rationale for Large Language and Vision Models," in Proc. of NeurlPS, 2024.
[22] J. Wei, X. Wang, D. Schuurmans, M. Bosma, B. Ichter, F. Xia, E. H. Chi, Q. V. Le, and D. Zhou, "Chain-of-Thoughh Prompting Elicits Reasoning in Large Language Models," in Proc. of NeurlPS, 2022.
[23] A. Jaech, A. Kalai, A. Lerer, A. Richardson, A. El-Kishky, A. Low, A. Helyar, A. Madry, A. Beutel, A. Carney, A. Iftimie, A. Karpenko, A. T. Passos, A. Neitz, A. Prokofiev, A. Wei, A. Tam, A. Bennett, A. Kumar, A. Saraiva, A. Vallone, A. Duberstein, A. Kon- drich, A. Mishchenko, A. Applebaum, A. Jiang, A. Nair, B. Zoph, B. Ghorbani, B. Rossen, B. Sokoloosky, B. Barak, McGrew, B. Minaiev, B. Hao, B. Baker,
Houghton, B. McK-
inzie, B. Eastman, C. Lugaresi, C. Bassin. C. Hudson, C. M. Li, C. de Bourcy, C. Voss, Shen, C. Zhang, C. Koch, C. Orsinger,
Hesse, C. Fischer, C. Chan, D. Roberts, D. Kap- pler, D. Levy, D. Selsam, D. Dohan, D. Farhi, D. Mely, D. Robinson, D. Tsipras, D. Li, D. Oprica, E. Freeman, E. Zhang, E. Wong. E. Proehl, E. Cheung, E. Mitchell, E. Wallace, E. Ritter, E. Mays, F. Wang, F. P. Such, F. Raso, F. Leoni, F. Tsimpourlas, F. Song, F. von Lohmann, F. Sulit, G. Salmon, G. Parascandolo, G. Chabot, G. Zhao, G. Brockman, G. Leclere, H. Salman, H. Bao, H. Sheng. H. Andrin, H. Bagherinezhad, H. Ren, H. Lightman, H. W. Chung, Kivlichan, I. D'Connell, I. Osband, C. Gilaberte, and I. Akkaya, "OpenAl ol System Card," CoRR, 2024.
[24] Dee,Seek-AI, D. Guo, D. Yang, H. Zhang, Song, R. Zhang, R. Xu, Q. Zhu, S. Ma, P. Wang, X. Bi, X. Zhang. X. Yu, Y. Wu, Z. F. Wu, Z. Gou, Z. Shao, Z. Li, Z. Gao, A. Liu, B. Xue, B. Wang, B. Wu, B. Feng, C. Lu, C. Zhao, C. Deng, C. Zhang, C. Ruan, D. Dai, D. Chen, D. Ji, E. Li, F. Lin, F. Dai, F. Luo, G. Hao, G. Chen, G. Li, H. Zhang, H. Bao. H. Xu, H. Wang,
H. Ding, H. Xin, H. Gao, H. Qu, H. Li, Guo, J. Wang, J. Chen, Yuan, J. Qiu, J. Li,
J. Cai, J. Ni, J. Liang. J. Chen, K. Dong, K. Hu, K. Gao, K. Guan, K. Huang, K. Yu, L. Wang.
L. Zhang, L. Zhao, L. Wang. L. Zhang, L. Xu, L. Xia, M. Zhang. M. Zhang. M. Tang, M. Li,
M. Wang, M. Li, N. Tian, P. Huang, P. Zhang, Q. Wang. Q. Chen, Q. Du, R. Ge, R. Zhang, R. Pan, R. Wang, R. J. Chen, R. L. Jin, R. Chen, S. Lu, S. Zhou, S. Chen, S. Ye, S. Wang, S. Yu, S. Zhou, S. Pan, and S. S. Li, "DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning," " CoRR, 2025.
[25] S. Hao, S. Sukhbaatar, D. Su, X. Li, Z. Hu, J. Weston, and Y. Tian, "Training Large Language Models to Reason in a Continuous Latent Space," CoRR, 2024.
[26] J. Zhang, Y. Zhu, M. Sun, Y. Luo, S. Qiao, L. Du, D. Zheng, H. Chen, and N. Zhang, "Ligh(Thinker: Thinking Step-by- Compression," CoRR, 2025.
[27] S. Feng, G. Fang, X. Ma, and X. Wang, "Efficient Reasoning Models: A Survey," CoRR, 2025.
[28] A. Radford and K. Narasimhan, "Improving Language Understanding by Generative Pre- Training," 2018.
[29] A. Radford, Wu, R. Child, D. Luan, D. Amodei, and Sutskever, "Language Models are Unsupervised Multitask Learners," 2019.
[30] T. B. Brown, B. Mann, N. Ryder, M. Subbiah, J. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, S. Agarwal, A. Herbert- Voss, G. Krueger, T. Henighan, R. Child, A. Ramesh, D. M. Ziegler, J. Wu, C. Winter, C. Hesse, M. Chen, E. Sigler, M. Litwin, S. Gray, B. Chess, J. Clark, C. Berner, S. McCandlish, A. Radford, I. Sutskever, and D. Amodei, "Language Models are Few-Shot Learners," in Proc. of NeurlPS, 2020.
[31] L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. L. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Ray, J. Schulman, J. Hilton, F. Kelton, L. Miller, M. Simens, A. Askell, P. Welin- der, P. F. Christiano, Leike, and R. Lowe, "Training language models to follow instructions with human feedback," in Proc. of NeurlPS, 2022.
12

---

[32] OpenAI, "ChatGPT: Optimizing Language Models for Dialogue," 2023.
[33] H. Touvron, T. Lavril, G. Izacard, X. Martinet, M. Lachaux, T. Lacroix, B. Rozière, N. Goyal, E. Hambro, F. Azhar, A. Rodriguez, A. Joulin, E. Grave, and G. Lample, "LLaMA: Open and Efficient Foundation Language Models," CoRR, 2023.
[34] H. Touvron, L. Martin, K. Stone, P. Albert, A. Almahairi, Y. Babaei, N. Bashlykov, S. Batra, P. Bhargava, S. Bhosale, D. Bikel, L. Blecher, C. Canton-Ferrer, M. Chen, G. Cucurull, D. Esiobu, J. Fernandes, J. Fu, W. Fu, B. Fuller, C. Gao, V. Goswami, N. Goyal, A. Hartshorn, S. Hosseini, R. Hou, H. Inan, M. Kardas, V. Kerkez, M. Khabsa, I. Kloumann, A. Korenev, S. Koura, M. Lachaux, T. Lavril, J. Lec. D. Liskovich, Y. Lu, Y. Mao, X. Martinet, T. Mihaylov, Nie, Poulton, Reizenstein, R. Rungta, K. Saladi, A. Schelten, R. Silva, E. M. Smith, R. Subramanian, X. E. Tan, B. Tang, R. Taylor, A. Williams, J. X. Kuan, P. Xu, Z. Yan, Zarov, Y. Zhang, A. Fan, M. Kambadur, S. Narang, A. Rodriguez, R. Stojnic, S. Edunov, and T. Scialom, "Llama 2: Open Foundation and Fine- Chat Models," CoRR, 2023.
P. Mishra, I. Molybog,
[35] J. Bai, S. Bai, Y. Chu, Z. Cui, K. Dang, X. Deng, Y. Fan, W. Ge. Y. Han, F. Huang, B. Hui, L. Ji, M. Li, J. Lin, R. Lin, D. Liu, G. Liu, C. Lu, K. Lu, J. Ma, R. Men, X. Ren, X. Ren, C. Tan, S. Tan, J. Tu, P. Wang. S. Wang, W. Wang, S. Wu, B. Xu, J. Xu, A. Yang, H. Yang, J. Yang, Yang. Y. Yao, B. Yu, H. Yuan, Z. Yuan, Zhang, X. Zhang, Y. Zhang, Z. Zhang,
C. Zhou, J. Zhou, X. Zhou, and T. Zhu, "Qwen Technical Report," CoRR, 2023.
[36] A. Yang. B. Yang, B. Hui, B. Zheng, B. Yu, C. Zhou, C. Li, C. Li, D. Liu, F. Huang, G. Dong,
H. Wei, H. Lin. J. Tang, J. Wang, J. Yang. J. Tu, J. Zhang. J. Ma, J. Yang. J. Xu, J. Zhou,
J. Bai, He, J. Lin, K. Dang. K. Lu, K. Chen, K. Yang. M. Li, M. Xue, N. Ni, Zhang,
P. Wang, R. Peng. R. Men, R. Gao, R. Lin, S. Wang, S. Bai. S. Tan, T. Zhu, T. Li, T. Liu,
W. Ge. X. Deng, X. Zhou, X. Ren, X. Zhang, X. Wei, X. Ren, X. Liu, Y. Fan, Y. Yao, Y. Zhang.
Y. Wan, Y. Chu, Y. Liu, Z. Cui, Z. Zhang, Z. Guo, and Z. Fan, "Qwen2 Technical Report," CoRR, 2024.
[37] A. Dubey, A. Jauhri, A. Pandey, A. Kadian, A. Al-Dahle, A. Letman, A. Mathur, A. Schelten, A. Yang, A. Fan, A. Goyal, A. Hartshom, A. Yang, A. Mitra, A. Sravankumar, A. Korenev, A. Hinsvark, A. Rao, A. Zhang, A. Rodriguez, A. Gregerson, A. Spataru, B. Rozière, B. Biron, Tang, B. Chern, C. Caucheteux, C. Nayak, C. Bi, Marra, McConnell, C. Keller, C. Touret, C. Wu, C. Wong, C. C. Ferrer, C. Nikolaidis, D. Allonsius, D. Song, D. Pintz, Livshits, Esiobu, D. Choudhary, D. Mahajan, D. Garcia-Olano, D. Perino. D. Hupkes, E. Lakomkin, E. AlBadawy, E. Lobanova, E. Dinan, E. M. Smith, F. Radenovic, F. Zhang, Synnaeve, Lee, G. L.. Anderson, Nail, G. Mialon, G. Pang, G. Cucurell, H. Nguyen, H. Korevaar, H. Xu, H. Touvron, I. 7arov, I. A. Ibarra, I. M. Kloumann, I. Misra, I. Evtimov, J. Copet, J. Lee, J. Geffert, J. Vranes, J. Park, J. Mahadeokar, J. Shah, J. van der Linde, J. Billock, J. Hong. J. Lee, J. Fu, J. Chi, J. Huang, J. Liu, J. Wang, J. Yu, J. Bitton, J. Spisak, J. Park, J. Rocca, J. Johnstun, J. Saxe, J. Jia, K. V. Alwala, K. Upasani, K. Plawiak, K. Li, K. Heafield, K. Stone, and et al., "The Llama 3 Herd of Models," CoRR, 2024.
[38] Yang, B. Yang. B. Zhang, Hui, Zheng, B. Yu, C. Li, D. Liu, Huang, H. Wei, H. Lin, J. Yang, J. Tu, J. Zhang, J. Yang, J. Yang, J. Zhou, J. Lin, K. Dang, K. Lu, K. Bao, K. Yang, L. Yu, M. Li, M. Xue, P. Zhang, Q Zhu, R. Men, R. Lin, T. Li, Xia, X. Ren, X. Ren, Y. Fan, Y. Su, Y. Zhang, Y. Wan, Y. Liu, Z. Cui, Z. Zhang, and Z. Qiu, "Qwen2.5 Technical Report," CoRR, 2024.
[39] J. Alayrac, J. Donahue, P. Luc, A. Miech, I. Barr, Y. Hasson, K. Lene, A. Mensch, K. Millican, M. Reynolds, R. Ring, E, Rutherford, S. Cabi, T. Han, Z. Gong, S. Samangooei, M. Monteiro, J. L. Menick, S. Borgeaud, A. Brock, A. Nematzadeh, S. Sharifzadeh, M. Binkowski, R. Bar- reira, O. Vinyals, A. Zisserman, and K. Simonyan, "Flamingo: a Visual Language Model for Few-Shot Learning." in Proc. of NeurIPS, 2022.
[40] A. Awadalla, I. Gao, J. Gardner, Hessel, Hanafy, W. Zhu, K. Marathe, Bitton, S. Gadre, S. Sagawa, J. Jitsev, S. Kornblith, P. W. Koh, G. Ilharco, M. Wortsman, and L. Schmidt, "OpenFlamingo: Open-Source Framework for Training Large Autoregressive Vision- Language Models," CoRR, 2023.
13

---

[41] J. Xu, X. Zhou, S. Yan, X. Gu, A. Arnab, C. Sun, X. Wang, and C. Schmid, "Pixel Aligned Language Models," in Proc. of CVPR, pp. 13030-13039, 2024.
[42] D. Zhu, J. Chen, X. Shen, X. Li, and M. Elhoseiny, "MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models," in Proc. of 2024.
[43] H. Zhao, M. Zhang, W. Zhao, P. Ding, S. Huang, and D. Wang, "Cobra: Extending Mamba to Multi-Modaa Large Language Model for Efficient Inference," in Proc. of AAAI, pp. 10421- 10429, 2025.
[44] K. Cobbe, V. Kosaraju, M. Bavarian, M. Chen, H. Jun, L. Kaiser, M. Plappert, J. Tworek, J. Hilton, R. Nakano, C. Hesse, and J. Schulman, "Training Verifiers to Solve Math Word Problems," CoRR, 2021.
[45] M. Geva, D. Khashabi, E. Segal, T. Khot, D. Roth, and Berant, "Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategies," Trans. Assoc. Comput. Linguistics, pp. 346-361, 2021.
[46] Z. Chen, Q. Zhou, Y. Shen, Y. Hong, H. Zhang, and C. Gan, "See, Think, Confirm: Interactive Prompting Between Vision and Language Models for Knowledge- based Visual Reasoning," " CoRR, 2023.
[47] Z. Zhang, A. Zhang, M. Li, H. Zhao, G. Karypis, and A. Smola, "Multimodal Chain-of- Thought Reasoning in Language Models," Trans. Mach. Learn. Res., 2024.
[48] B. Luan, H. Feng, H. Chen, Y. Wang, W. Zhou, and H. Li, "TextCoT: Zoom In for Enhanced Multimodal Text-Rich Image Understanding." CoRR, 2024.
[49] Z. Jia, J. Liu, H. Li, Q. Liu, and H. Gao, "DCoT: Dual Chain-of-Thought Prompting for Large Multimodal Models," " in Proc. of ACML, pp. 1064-1079, 2024.
[50] T. Gao, P. Chen, M. Zhang, C. Fu, Y. Shen, Y. Zhang, S. Zhang, X. Zheng, X. Sun, L. Cao, and R. Ji, "Cantor: Inspiring Multimodal Chain-of-Thought of MLLM," Proc. of ACM MM, pp. 9096-9105, 2024.
[51] G. Xu, P. Jin, H. Li, Y. Song. L. Sun, and L. Yuan, "LLaVA-CoT: Let Vision Language Models Reason Step-by-Step." CoRR, 2024.
[52] H. Shao, S. Qian, H. Xiao, G. Song. Z. Zong. L. Wang, Y. Liu, and H. Li, "Visual CoT: Advancing Multi-Modal Language Models with Comprehensive Dataset and Benchmark for Chain-of-Thought Reasoning," in Proc. of NeurIPS, 2024.
[53] Z. Li, R. Luo, J. Zhang, M. Qiu, and Z. Wei, "VoCoT: Unleashing Visually Grounded Multi-Step Reasoning in Large Multi-Modal Models," in Proc. of NAACL, 2025.
[54] W. Wu, S. Mao, Y. Zhang, Y. Xia, L. Dong, L. Cui, and F. Wei, "Mind's Eye of LLMs: Visualization-o--Thuuge Elicits Spatial Reasoning in Large Language Models," in Proc. of NeurlPS, 2024.
[55] F. Meng, H. Yang, Y. Wang, and M. Zhang, "Chain of Images for Intuitively Reasoning," CoRR, 2023.
[56] C. Li, W. Wu, H. Zhang, Y. Xia, S. Mao, L. Dong, I. Vulic, and F. Wei, "Imagine while Reasoning in Space: Multimodal soralization-oo-Thuglge CORR, 2025.
[57] D. A. Hudson and C. D. Manning, "GQA: A New Dataset for Real-World Visual Reasoning and Compositional Question Answering," in Proc. of CVPR, pp. 6700-6709, 2019.
[58] K. Yi, C. Gan, Y. Li, Kohli, J. Wu, A. Torralba, and J. B. Tenenbaum, "CLEVRER: Collision Events for Video Representation and Reasoning," in Proc. of ICIRR, 2020.
[59] C. H. Song, V. Blukis, J. Tremblay, Tyree, Y. Su, and Birchfield, "RoboSpatial: Teaching Spatial Understanding to 2D and 3D Vision-Language Models for Robotics." CoRR, 2024.
14

---

[60] J. Yang, S. Yang, A. W. Gupta, R. Han, L. Fei-Fei, and S. Xie. "Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spaces," CoRR, 2024.
[61] X. Chen, X. Wang, S. Changpinyo, A. Piergiovanni, P. Padlewski, D. Salz, S. Goodman, A. Gryener, B. Mustafa, L. Beyer, A. Kolesnikov, J. Puigcerver, N. Ding, K. Rong, H. Akbari, G. Mishra, L. Xue, A. V. Thapliyal, J. Bradbury, and W. Kuo, "PaLI: A Jointly-Scaled Multilingual Language-Iman
Model," in Proc. of ICLR, 2023.
[62] J. Li, D. Li, S. Savarese, and S. C. H. Hoi, "BLIP-2: Bootstrapping Language-Image Pre- training with Frozen Image Encoders and Large Language Models," in Proc. of ICML, pp. 19730-19742, 2023.
[63] Driess, Xia, M. M. Sajjadi, C. Lynch, A. Chowdhery, B. Ichter, A. Wahid, Tompson, Q. Vuong, T. Yu, W. Huang. Y. Chebotar, P. Sermanet, D. Duckworth, S. Levine, V. Vanhoucke, Hausman, M. Toussaint, Greff, A. Zeng, I. Mordatch, and Florence, "PaLM-E: An
Embodied Multimodal Language Model," in Proc. of ICML, pp. 8469-8488, 2023.
[64] S. Wu, H. Fei, L. Qu, W. Ji, and T. Chua, "NEXT-GPT: Any Multimodal LLM," in Proc. of ICML, 2024.
[65] A. Cheng, H. Yin, Y. Fu, Q. Guo, R. Yang. J. Kautz, X. Wang, and S. Liu, "SpatialRGPT: Grounded Spatial Reasoning in Vision-Language Models," in Proc. of NeurlPS, 2024.
[66] E. Zhou, J. An. C. Chi. Han, S. Rong. C. Zhang, P. Wang. Z. Wang, Huang. Sheng, and S. Zhang, "RoboRefer: Towards Spatial Referring with Reasoning in Vision-Language Models for Robotics." " CoRR, 2025.
[67] A. Radford,
Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, G. Krueger, and I. Sutskever, "Learning Transferable Visual Models From Natural Language Supervision." in Proc. of ICML. pp. 8748-878 8763, 2021.
[68] A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. De- hghani, M. Minderer, G. Heigold, S. Gelly, J. Uszkoreit, and N. Houlsby, "An Image is Worth 16x16 Words:
Transformers for Image Recognition at Scale," in Proc. of ICLR, 2021.
[69] X. Zhai, B. Mustafa, A. Kolesnikov, and L. Beyer, "Sigmoid Loss for Language Image Pre- Tre-Trainn Proc. of ICCV, pp. 11941 11952, 2023.
[70] Yu and Z. Su, "Symmetrical Gaussian Error Linear Units (SGELUs))" CoRR, 2019.
[71] A. Gu and T. Dao, "Mamba: Linear-Time Sequence Modeling with Selective State Spaces," in Proc. of CoLM, 2024.
[72] L. Chen, J. Li, X. Dong, Zhang, C. He, J. Wang, F. Zhao, and D. Lin, "ShareGPT4V: Improving Large Multi-modal Models with Better Captions," in Proc. of ECCV, pp. 370-387, 2024.
[73] Plummer, Wang. C. M. Cervantes, J. C. Caicedo, J. Hockenmaier, and Lazeb- nik, "Flickr30k Entities: Collecting Region-to- tP-raasse Correspondences for Richer Image-to- Sentence Models," in Proc. of ICCV, pp. 2641-26499 2015.
[74] A. Masry, D. X. Long, Q. Tan, S. R. Joty, and E. Hoque, "ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoning," in Proc. of ACL Findings, pp. 2263-2279, 2022.
[75] D. Schwenk, A. Khandelwal, C. Clark, K. Marino, and R. Mottaghi, "A-OKVQA: Bench- mark for Visual Question Answering Using World Knowledge," in Proc. of ECCV, pp. 146- 162, 2022.
[76] Y. Zhu, O. Groth, M. S. Bernstein, and L. Fei-Fei, "Visual7W: Grounded Question Answering in Images," in Proc. of CVPR, pp.
2016.
[77] Kembhavi, M. Salvato, E. Kolve, M. J. Seo, H. Hajishirzi, and A. Farhadi, Diagram Worth Dozen Images." in Proc. of ECCV, pp. 235-251, 2016.
15

---

[78] A. Kuznetsova, H. Rom, N. Alldrin, J. R. R. Uijlings, I. Krasin, J. Pont-Tuset, S. Kamali, S. Popov, M. Malloci, T. Duerig, and V. Ferrari, "The Open Images Dataset V4: Unified image classification, object detection, and visual relationship detection at scale," CoRR, 2018.
[79] J. Cao and J. Xiao, "An Augmented Benchmark Dataset for Geometric Question Answering through Dual Parallel Text Encoding," in Proc. of COLING, pp. 1511-1520, 2022.
[80] C. Wah, S. Branson, P. Welinder, P. Perona, and S. Belongie, "The caltech-uesd birds-200-2011 dataset," Tech. Rep. 200-TR-0011-001, California Institute of Technology, 2011.
[81] P. Lu. S. Mishra, I. Xia, L. Qiu, K. Chang, S. Zhu, O. Tafjord, P. Clark, and A. Kalyan, "Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering," " in Proc. of NeurIPS, 2022.
[82] F. Liu, G. Emerson, and N. Collier, "Visual Spatial Reasoning," Trans. Assoc. Comput. Linguistics, pp. 635-651, 2023.
[83] M. Mathew, D. Karatzas, and C. V. Jawahar, "DocVQA: A Dataset for VQA on Document Images," in Proc. of WACV. pp.
2021.
[84] J. Li, Y. Wong, Q. Zhao, and M. S. Kankanhalli, "People in Social Context (PISC) Dataset," 2017.
[85] J. Johnson, B. Hariharan, L.. van der Maaten, I. Fei-Fei, C. L. Zitnick, and R. B. Girshick, "CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reason- ing," in Proc. of CVPR, pp. 1988-1997, 2017.
[86] H. Liu, C. Li, Y. Li, B. Li, Y. Zhang, S. Shen, and Y. J. Lee, "Llava-next: Improved reasoning, ocr, and world knowledge," January 2024.
[87] A. D. Lindström and S. Abraham, "CLEVR-Math: A Dataset for Compositional Language, Visual and Mathematical Reasoning," in Proc. of IJCLR, pp. 155-170, 2022.
[88] A. Gupta, P. Dollár, and R. B. Girshick, "LVIS: A Dataset for Large Vocabulary Instance Segmentation," " in Proc. of CVPR, pp. 5356-5364, 2019.
[89] M. He, Y. Liu, Wu, J. Yuan, Y. Wang, T. Huang, and B. Zhao, "Efficient Multimodal Leaming from Data-centric Perspective," CoRR, 2024.
[90] A. O'Neill, A. Rehman, A. Maddukuri, A. Gupta, A. Padalkar, A. Lee, A. Pooley, A. Gupta, A. Mandlekar, A. Jain, A. Tung, A. Bewley, A. Herzog, A. Irpan, A. Khazatsky, A. Rai, A. Gupta, A. E. Wang, A. Singh, A. Garg, A. Kembhavi, A. Xie, A. Brohan, A. Raffin, A. Sharma, A. Yavary, A. Jain, A. Balakrishna, A. Wahid, B. Burgess-Limerick,
Kim, B. Schölkopf. B. Wulfe, B. Ichter, C. Lu, C. Xu, C. Le, C. Finn, C. Wang. C. Xu, C. Chi, C. Huang, C. Chan, C. Agia, C. Pan, C. Fu, C. Devin, D. Xu, D. Morton, D. Dricss, D. Chen, D. Pathak, D. Shah, D. Büchler, D. Jayaraman, D. Kalashnikov, D. Sadigh, E. Johns, E. P. Foster, F. Liu, F. Ceola, F. Xia, F. Zhao, F. Stulp, G. Zhou, G. S. Sukhatme, G. Salhotra, G. Yan, G. Feng. G. Schiavi, G. Berseth, G. Kahn, G. Wang, H. Su, H. Fang, H. Shi, H. Bao, H. B. Amor, I. Christensen, H. Furuta, H. Walke, H. Fang, H. Ha, I. Mordatch, I. Radosavovic, I. Leal, J. Liang, J. Abou-Chakra, J. Kim, J. Drake, Peters, J. Schneider, J. Hsu, J. Bohg, J. Bingham, J. Wu, J. Gao, J. Hu, J. Wu, J. Wu, J. Sun, J. Luo, J. Gu, J. Tan, J. Oh, J. Wu, J. Lu, J. Yang. J. Malik, J. Silvério, J. Hejna, J. Booher, J. Tompson, J. Yang. J. Salvador, J. J. Lim, J. Han, K. Wang, K. Rao, K. Pertsch, K. Hausman, K. Go, K. Gopalakrishnan, K. Goldberg, K. Byrne K. Oslund, K. Kawaharazuka, K. Black, K. Lin, K. Zhang, K. Ehsani, K. Lekkala, K. Ellis, K. Rana, K. Srinivasan, K. Fang, K. P. Singh, K. Zeng, K. Hatch, K. Hsu, L. Itti, L. Y. Chen, L. Pinto, L. Fei-Fei, L. Tan, L. J. Fan, L. Ott, L. Lee, L. Weihs, M. Chen, M. Lepert, M. Memmel, M. Tomizuka, M. Itkina, M. G. Castro, M. Spero. M. Du, M. Ahn, M. C. Yip. M. Zhang, M. Ding, M. Heo, M. K. Srirama, M. Sharma, M. J. Kim, N. Kanazawa, N. Hansen, N. Heess, N. J. Joshi, N. Sünderhauf, N. Liu, N. D. Palo, N. M. M. Shafulllh,, O. Mees, O. Kroemer, O. Bastani, P. R. Sanketi, P. T. Miller, P. Yin, P. Wohlhart, P. Xu, P. D. Fagan, P. Mitrano, P. Sermanet, P. Abbeel, P. Sundaresan, Q. Chen, Q. Vuong, R. Rafailov, R. Tian, R. Doshi, R. Martin-Martín, R. Baijal, R. Scalise, R. Hendrix, R. Lin, R. Qian, R. Zhang, R. Mendonca, R. Shah, R. Hoque, R. Julian, S. Bustamante, S. Kirmani, S. Levine,
16

---

S. Lin, S. Moore, S. Bahl, S. Dass, S. D. Sonawani, S. Song, S. Xu, S. Haldar, S. Karamcheti,
S. Adebola, S. Guist, S. Nasiriany, Schaal, S. Welker. S. Tian, S. Ramamoorthy, S. Dasari,
S. Belkhale, S. Park, S. Nair, S. Mirchandani, T. Osa, T. Gupta, T. Harada, T. Matsushima, T. Xiao, T. Kollar, T. Yu, Ding. TT. Davchev, TT. Z. Zhao, T. Armstrong. Darrell, T. Chung, V. Jain, V. Vanhoucke, W. Zhan, W. Zhou, W. Burgard, X. Chen, X. Wang. X. Zhu, X. Geng, X. Liu, L. Xu. X. Li, Y. Lu, Y. J. Ma, Y. Kim, Y. Chebotar, Y. Zhou, Y. Zhu, Y. Wu, Y. Xu,
Y. Wang. Y. Bisk, Y. Cho, Y. Lee, Y. Cui, Y. Cao, Y. Wu, Y. Tang, Y. Zhu, Y. Zhang,
Y. Jiang, Y. Li, Y. Li, Y. Iwasawa, Y. Matsuo, Z. Ma, Z. Xu, Z. J. Cui, Z. Zhang, and Z. Lin, "Open X-Embodiment: Robotic Learning Datasets and RT-X Models: Open X-En X-Embodiment Collaboration," " in Proc. of ICRA, Pp. 6892-6903 2024.
[91] S. Bai, K. Chen, X. Liu, J. Wang, W. Ge, S. Song, K. Dang, P. Wang, S. Wang, J. Tang, H. Zhong, Y. Zhu, M. Yang, Z. Li, J. Wan, P. Wang, W. Ding, Z. Fu, Y. Xu, J. Ye, X. Zhang, T. Xie, Z. Cheng, H. Zhang. Z. Yang, H. Xu, and J. Lin, "Qwen2.5-VL Technical Report," 2025.
[92] A. Hurst, A. Lerer, A. P. Goucher, A. Perelman, A. Ramesh, A. Clark, A. Ostrow, A. Welihinda, A. Hayes, A. Radford, A. Madry, A. Baker- Whitcomb, A. Beutel, A. Borzunov, A. Carney, A. Chow, A. Kirillov, A. Nichol, A. Paino, A. Renzin, A. T. Passos, A. Kirillov, A. Chris- takis, A. Conneau, A. Kamali, A. Jabri, A. Moyer, A. Tam, A. Crookes, A. Tootoonchian, A. Kumar, A. Vallone, A. Karpathy, A. Braunstein, A. Cann, A. Codispoti, A. Galu, A. Kon- drich, A. Tulloch, A. Mishchenko, A. Baek, A. Jiang, A. Pelisse, A. Woodford, A. Gosalia, A. Dhar, A. Pantuliano, A. Nayak, A. Oliver, B. Zoph, B. Ghorbani, B. Leimberger, B. Rossen, B. Sokolowsky, B. Wang. B. Zweig, B. Hoover, B. Samic, B. McGrew, B. Spero, B. Giertler,
B. Cheng, B. Lightcap, B. Walkin, B. Quinn, B. Guarraci, B. Hsu, B. Kellogg, B. Eastman,
C. Lugaresi, C. L. Wainwright, C. Bassin, C. Hudson, C. Chu, C. Nelson, C. Li, C. J. Shern, C. Conger, C. Barette, C. Voss. C. Ding, C. Lu, C. Zhang. C. Beaumont, C. Hallacy, C. Koch, C. Gibson, C. Kim, C. Choi, C. McLeavey, C. Hesse, C. Fischer, C. Winter, C. Czarnecki,
C. Jarvis, C. Wei, C. Koumouzelis, and D. Sherburn, "GPT-40 System Card," CoRR, 2024.
[93] A. Yang, B. Yu, C. Li, D. Liu, F. Huang, H. Huang, J. Jiang, J. Tu, J. Zhang. J. Zhou, J. Lin, K. Dang, K. Yang, L. Yu, M. Li. M. Sun, Q. Zhu. R. Men, T. He, W. Xu, W. Yin, W. Yu, X. Qiu, X. Ren, X. Yang, Y. Li, Z. Xu, and Z. Zhang, "Qwen2.5-1M Technical Report," CoRR, 2025.
[94] M. Reid, N. Savinov. D. Teplyashin, D. Lepikhin, T. P. Lillicrap, J. Alayrac, R. Soricut, Lazaridou, O. Firat, J. Schritwieser, I. Antonoglou, R. Anil, S. Borgeaud, A. M. Dai, K. Millican, E. Dyer, M. Glaese, T. Sottiaux, B. Lee, F. Viola, M. Reynolds, Y. Xu, J. Mol- loy. J. Chen, M. Isard, P. Barham, T. Hennigan, R. Mcllroy, M. Johnson, J. Schalkwyk, B. Collins, E. Rutherford, E. Moreira, K. Ayoub, M. Goel, C. Meyer, G. Thornton, Z. Yang, H. Michalewski, Z. Abbas, N. Schucher, A. Anand, R. Ives, J. Keeling, K. Lenc, S. Haykal, S. Shakeri, P. Shyam, A. Chowdhery, R. Ring, S. Spencer, E. Sezener, and et al., "Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context," CoRR, 2024.
[95] Y. Niu, M. Ning, M. Zheng, B. Lin, P. Jin, J. Liao, K. Ning, B. Zhu, and L. Yuan, "WISE: A World Knowledge-Informn Semantic Evaluation for Text-to-Image Generation," CoRR, 2025.
[96] W. Yu, Z. Yang, Li, J. Wang, K. Lin, Z. Liu, X. Wang, and L. Wang, "MM-Vet Evaluating Large Multimodal Models for Integrated Capabilities," in Proc. of ICMI. 2024.
[97] J. Hu, Y. Shen, Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, and W. Chen, "LoRA: Low-Rank Adaptation of Large Language Models," in Proc. of ICLRR 2022.
[98] Zhao, A. Gu, R. Varma, Luo, C. Huang, M. Xu, L. Wright, H. Shojanazeri, M. Ott, S. Shleifer, A. Desmaison, C. Balioglu, P. Damania, B. Nguyen, G. Chauhan, Y. Hao, A. Math- ews, and S. Li, "PyTorch FSDP: Experiences on Scaling Fully Sharded Data Parallel," Proc. VLDB Endow., pp. 3848-3860 2023.
[99] Anthropic "Claude 3.5 Haiku." "
4-5-models-ande www.andddoouer-uuuuuuau 2024.
17

---

[100] remyxai, "VQASynth," 2024. GitHub repository.
[101] X. Wang, X. Zhang. Z. Luo, Q. Sun, Y. Cui, J. Wang, F. Zhang, Y. Wang, Z. Li. Q. Yu, Y. Zhao, Y. Ao, X. Min, T. Li, B. Wu, B. Zhao, B. Zhang, L. Wang, G. Liu, Z. He, X. Yang, J. Liu, Y. Lin, T. Huang, and Z. Wang, "Emu3: Nexi-Token Prediction is All You Need," CoRR, 2024.
[102] P. Tong, E. Brown, P. Wu, S. Woo, A. Iyer, S. Akula, S. Yang, J. Yang, M. Middepogu, Z. Wang, X. Pan, R. Fergus, Y. LeCun, and S. Xie, "Cambrian-I: 1: A Fully Open, Vision-Centric Exploration of Multimodal LLMs," in Proc. of NewrIPS, 2024.
[103] Kamath, J. Hessel, and K. Chang, "What's "up" with vision-language models? Investigating their struggle with spatial reasoning," in Proc. of EMNLP, 9161-9175, 2023.
[104] Y. Goyal, T. Khot, D. Summers-Stay D. Batra, and D. Parikh, "Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answering," in Proc. of CVPR, pp. (325-6334, 2017.
[105] A. Singh, V. Natarajan, M. Shah, Y. Jiang, X. Chen, D. Batra, D. Parikh, and M. Rohrbach, "Towards VQA Models That Can Read," in Proc. of CVPR, pp. 8317-8326, 2019.
[106] Y. Li, Y. Du, K. Zhou, J. Wang, W. X. Zhao, and J. Wen, "Evaluating Object Hallucination in Large Vision-Language Models," in Proc. of EMNLP, pp. 292-305, 2023.
[107] Y. Liu, H. Duan, Y. Zhang, B. Li, Zhang, W. Zhao, Y. Yuan, J. Wang, C. He, Liu, K. Chen, and D. Lin, "MMBench: Is Your Multi-modal Model an All-Around Player?," in Proc. of ECCV, pp. pp. 216-233,20 2024.
[108] I. Loshchilov and Hutter, "Decoupled Weight Decay Regularization," in Proc. of ICLR, 2019.
[109] I. M. Alabdulmohsin, X. Zhai, A. Kolesnikov, and L. Beyer, "Getting VIT in Shape: Scaling Laws for Compute-Optimal Model Design," in Proc. of NeurIPS, 2023.
18

---

A Hyperparameters
Detailed hyperparameter configurations provided in Table 9.
Table 9: Training hyper-parameteen of our proposed SSR.
Configuration
Stage
Stage
Configuration
Stage
Stage
Vision Encoder
Clip- Clip-ViT-Land WrT-LLagg-Paihhee Patchh143366 [67]
Optimizer
AdamW [108]
Depth Encoder
Siglip-Sot00M-Patchh444 [69, 109]
Learning Rate
0,00002
Mamba
Mamba 130M [71]
Numerical Precision
BF
LLM
Qwen2.5 3B [38]
N/A
Epoch
2
1
VLM
N/A
Qwen2.5-VL VL (91]
Global Batch Size
32
32
Question Length
256
Learning Schedule
Cosine Decay
Rational Length
1024 N/A
N/A
Warm-up Ratio
0.02
Answering Length
256
Number of Latent Tokens
B SSRBENCH Results in Score Metrics
The evaluation metrics employed for SSRBENCI include both accuracy and quantitative score ranging from 0 to 5. Quantitative results are presented in Table 10, with detailed descriptions of the assessment methodology provided in Appendix D. These scores are generally consistent with the accuracy trends presented in Table 3.
Table 10: Score performance comparison on SSRBENCH. SSRBENCH G and SSRBENCH S denote general and spatial tasks, respectively.
Size
SSRBENCH
SSRBENCH
Method
Mamba
VLM
Existence
Attribute
Action
Counting
Position
Object
PROPRIETARY
GPT-4o-mini [92]
N/A
N/A
4.05
2.95
3.46
3.12
2.87
2.66
Claude-3.5-Haiku [99]
N/A
N/A
3.48
2.99
2.71
2.75
2.56
2.31
OPEN-SOURCH SOURCE
SpatialVI.M [10, 100]
N/A
3B
2.34
3.32
3.55
2.34
3.56
3.24
LLAVA-1.5 [2]
N/A
7B
4.17
3.72
3.66
2,71
3.87
3.56
LLaVA-NeXT [86]
N/A
7B
4.23
3.59
3.79
2.66
3.69
3.41
LLaVA-NeXT
N/A
13B
4.30
3.82
3.79
2.76
3.78
3.12
SpatialBot [17]
N/A
3B
3.97
3,47
3.82
2.66
3,96
3.47
Emu3 [101]
N/A
8B
3.07
2.35
2.39
1.71
3,04
2.28
Qwen2.5-Vi 5-L [911
N/A
3B
3.56
3.42
3.56
2.41
3.43
3.00
Qwen2.5-Vi VL [91]
N/A
7B
4.07
3.55
3.71
2.85
3.50
3.16
SSR (Ours)
130M
3B
4.44
4.28
3.95
3.17
4.40
4.02
SSR (Ours)
130M
7B
4.65
4.17
4.10
3.71
4.43
4.16
Discussion with Meteor. Meteor [21] is approach similar to ours, designed to compress ratio- nales using efficient large language models. However, unlike our method, Meteor does not separate the reasoning module from the large language model during response generation. Due to this tight coupling, Meteor must trained end-to-end from scratch, a process that demands extensive datasets and significant computational resources. In contrast, method specifically focuses enhancing the spatial awareness and reasoning abilities of Liaguu-Le Models (VLMs), leveraging their approach substantially reduces the
inherent capabilities to greater extent. Consequently, plexity and resource requirements related to training VLMs from scratch. Moreover, we focus on directionally enhancing the depth perception and spatial reasoning capabilities of existing VLMs in this paper. Therefore, our comparative analysis primarily emphasizes evaluating model performance before and after applying these enhancements.
C SSR-COT
As detailed in Section 4, SSR-COT constructed from four distinct data sources, with spatially- aware CoT rationales generated for each data sample. Representative samples shown in Figure 4.
19

---

Additionally, a comprehensive description of the rationale-generation pipeline specific to each data source presented in Appendix C.2, C.I, and C.3.
C.1 Visual-CoT
For Visual-CoTT [52], each data sample includes bounding box that serves as CoT rationale to guide the generation of the corresponding answer. We utilize this bounding box, which is closely related the target answer, an intermediate step to query spatial information about the selected object using SpatialRGPT [65], a spatial puestion-annsweiigg model tailored for vertical domains. Subsequently, we aggregate the obtained spatial question-and-anse LLM such GPT-4o [92]. The resulting text serves as the CoT rationale for the Visual-CoT data source within SSR-COT.
information using powerful
Spatial Query for Visual-CoT
1. What is the object in [bbox]? Think step by step, and avoid repetition.
2. Can you estimate the height and width of [bbox]? Think step by step, and avoid repetition.
3. What the object to the left of [bbox], and what its height and width? Think step by step, and avoid repetition.
4. What is the object to the left of [bbox], and what is its distance to [bbox]? Think step by step, and avoid repetition.
5. What is the object to the right of [bbox], and what is its height and width? Think step by step, and avoid repetition.
6. What is the object the right of [bbox], and what is its distance [bbox]? Think step by step, and avoid repetition.
7. What is the object front of [bbox], and what is its height and width? Think step by step, and avoid repetition.
8. What is the object in front of [bbox], and what is its distance to [bbox]? Think step by step, and avoid repetition.
9. What is the object behind [bbox], and what is its height and width? Think step by step, and avoid repetition.
10. What is the object behind [bbox], and what is its distance to [bbox]? Think step by step, and avoid repetition.
11. What is the object below [bbox], and what is its height and width? Think step by step, and avoid repetition.
12. What is the object below [bbox], and what is its distance [bbox]? Think step by step, and avoid repetition.
13. What is the object above [bbox], and what is its height and width? Think step by step, and avoid repetition.
14. What is the object above [bbox], and what is its distance to [bbox]? Think step by step, and avoid repetition.
Rationale Generation for Visual-CoT
Please generate an image description in continuous paragraphs using these strict guidelines:
Coordinate Usage Rules:
1. ONLY use coordinates that are explicitly defined in this mapping table:
RRegion [O]: [bbox]
Do NOT create or infer any new coordinates
3. Each coordinate only be used ONCE in the description
20

---

4. Coordinates must be written in [xl.y 1.x2,y2] format without spaces
Content Rules:
Place coordinate immediately after describing its corresponding object
2. Integrate coordinates naturally within complete sentences
3. Include all provided measurements and spatial relationships
4. Maintain narrative flow while incorporating technical details
5. Focus on visual elements and their relationships
6. Embed coordinates from the mapping table immediately after their corresponding region
objects (e.g., "a dog [xl.y Ix2.y2]")
Maintain paragraph continuity by integrating coordinates within complete sentences
8. Preserve strict aigion-coordinate mapping from the provided table
Use only [x2.2] format without spaces
10. Exclude technical metadata and region index numbers from final text
11. Automatically resolve spatial contradictions using coordinate data
12. Ensure coordinate annotations flow naturally after object nouns
Input Data:
Spatial Query and Response for Visual-CoT
C.2 VoCoT
VoCoT [53] includes multiple bounding boxes per data sample, more than Visual-CoT [52], to clearly outline reasoning paths involving multiple objects within image. Similar the process used for Visual-CoTT, we perform spatial queries on each object associated with a bounding box. Additionally, capture the relative spatial relationships between every pair of objects to comprehensively utilize available spatial context and support accurate reasoning. Finally, we aggregate this spatially derived question-and-ansten information using robust language model, such as GPT-4o [92].
Spatial Query for VoCoT
Query for each bounding box:
1. What is the object in [bbox]? Think step by step, and avoid repetition.
2. Can you estimate the height and width of [bbox]? Think step by step, and avoid repetition.
3. What is the object to the left of [bbox], and what is its height and width? Think step by step, and avoid repetition.
4. What is the object to the left of [bbox], and what is its distance to [bbox]? Think step by step, and avoid repetition.
5. What is the object to the right of [bbox], and what is its height and width? Think step by step, and avoid repetition.
6. What is the object to the right of [bbox], and what is its distance to [bbox]? Think step by step, and avoid repetition.
7. What is the object in front of [bbox], and what is its height and width? Think step by step, and avoid repetition.
8. What is the object in front of [bbox], and what is its distance to [bbox]? Think step by step, and avoid repetition.
9. What is the object behind [bbox], and what is its height and width? Think step by step, and avoid repetition.
10. What is the object behind [bbox], and what is its distance [bbox]? Think step by step, and avoid repetition.
11. What is the object below [bbox], and what is its height and width? Think step by step, and avoid repetition.
21

---

12. What is the object below [bbox], and what is its distance to [bbox]? Think step by step, and avoid repetition.
13. What is the object above [bbox], and what its height and width? Think step by step, and avoid repetition.
14. What is the object above [bbox], and what is its distance to [bbox]? Think step by step, and avoid repetition.
Query for every two bounding box:
Which one is higher between [bbox [] and [bbox2]? Think step by step, and avoid repetition.
2. Can you estimate how far apart [bbox 1] and [bbox2] are? Think step by step, and avoid repetition.
3. What direction is [bbox2] in relation to [bbox 1]? Think step by step, and avoid repetition.
4. How far [bbox1] from [bbox2] horizontally? Think step by step, and avoid repetition.
5. Does [bbox have larger size compared to [bbox2]? Think step by step, and avoid repetition.
6. Does [bbox1] have lesser width compared to [bbox2]] Think step by step, and avoid repetition.
Rationale Generation for VoCoT
Integrate all measurements values and spatial information from the conversation into answer to get detailed reasoning rationale with spatial details. Then, extract the direct question and answer from question and answer respectively.
Content Rules:
Place coordinate immediately after describing its corresponding object first time, make sure each coordinate appear only once.
2. Avoid introducing other coordinates that do not appear in answer.
Add all provided measurements values and spatial relationships from the conversation the rationale detailedly.
Ensure the rationale contains all the information from each sentence the conversation, especially the measurements values and spatial relationships. Automatically resolve spatial contradictions using coordinate data based the image.
Output in the following json template:
"question": <question>>>>
"rationale": <rationale>
"answer": <answer>
Question: Question
Answer: Answer
Conversation: Spatial Query and Response for VoCoT
C.3 SpatialQA
Unlike Visual-CoT [52] and VoCoT [53], the SpatialQA [17] dataset does not provide intermediate CoT reasoning steps or bounding boxes for object identification. Therefore, we leverage GPT-4o [92], powerful multi-modal large language model, to generate detailed synthetic rationale data. These
22

---

synthetic rationales supply the necessary intermediate reasoning processes to enable accurate answer generation.
Rationale Generation for SpatialQA
I have an image and a question that want you answer. I need you to strictly follow the format with four specific sections: summary, caption, reasoning, and conclusion. It is crucial that you adhere to this structure exactly outlined and that the final answer in the conclusion matches the standard correct answer precisely. To explain further:
In summary. briefty explain what steps you'll take to solve the problem.
In caption, describe the contents of the image, specifically focusing on details relevant to the question.
In reasoning, outline step-by-step thought process you would use to solve the problem based on the image.
In conclusion, give the final answer in direct format, and it must match the correct answer exactly. If it's a multiple choice question, the conclusion should only include the option without repeating what the option is. Finally, integrate these sections into a natural thinking paragraph.
Here's the question and answer: Question: Question
Answer: Answer
D LLM-Assistant Evaluation
As discussed in Section 4, we utilize the LLM-Assistant evaluation method to assess the data quality
of SSR-CoT and measure the performance of SSRBENCH [7, 1, 95, 96]. Evaluation metrics include accuracy and quantitative score ranging from 0 to 5; both metrics are determined by the LLM-Assistant powered by the Qwen2.5-1444 1---5-111-IInttuee LM [38, 93].
Prompt for LLM-Assistant VQA Evaluation
You are an intelligent chatbot designed for evaluating the correctness generative outputs for cesisiion-nsswer pairs.
Your task to compare the predicted answer with the correct answer and determine if they match meaningfully. Here's how you accomplish the task:
##INSTUCCTIONS
Focus on the meaningful match between the predicted answer and the correct answer.
Consider synonyms paraphrases as valid matches.
Evaluate the correctness of the prediction compared to the answer.
Please evaluate the following maaee-based question-ansee answer pair:
Question: question
Correct Answer: answer
Predicted Answer: response
Provide your evaluation only as yes/no and score where the score an integer value between O and 5, with 5 indicating the highest meaningful match.
Please generate the response the form of a Python dictionary string with keys 'pred' and 'score', where value of pred'' is string of yess or 'no" and value of 'score' is in INTEGER, not STRING.
DO NOT PROVIDE ANY OTHER OUTPUT TEXT OR EXPLANATION, Only provide the Python dictionary string.
For example, your response should look like this: ('pred': 'yes', 'score': 4.8).
23

---

E
Benchmarks
E.1 Benchmark Employed
We evaluate our method using various benchmarks: SpatialBench [17], CV-Bench [102], VSR []. What's Up []. proposed SSRBENCH, and multiple general VQA benchmarks 'aable 11 summarizes the statistics for the all benchmarks. Tables 3 and 6 present comparisons between SSR and baseline methods, along with comprehensive ablation studies conducted on SpatialBench and SSRBENCI, Furthermore, Table 4 summarizes the performance improvements observed across all spatial-related benchmarks.
Table 1I: Statistics of benchmarks utilized in this paper.
Benchmark
Task
Size
Benchmark
Task
Size
Position
34
Count
788
Existence
40
Relation
SpatialBench [17]
CV-Bench [102]
Counting
20
Depth
600
Size
40
Distance
600
TOTAL
114
TOTAL
2638
Existence
107
Counting
41
General
Aurribute
Spatial
Position
162
SSRBENCH
Action
157
Object
154
TOTAL
789
Random
1222
Zero-Shol
2195
VSR [82]
TOTAL
3417
What's Up [103]
TOTAL
820
VQAv2 [104]
TOTAL
107k
Tex(VQA [105]
TOTAL
5k
POPE [106]
TOTAL
9k
MMBench [107]
TOTAL
3k
GQA [57]
TOTAL
12k
E.2 SSRBENHH
To construct the SSRBENCH dataset, we first filter data samples from SSR-CoT. Subsequently. we feed each filtered sample into the multi-modal large language models GPT-4o [92] and Gemini-2.5- Pro [94], using the following prompt to classify the task category. As mentioned in Section 4 and shown Figure if the classification results from both models consistent, the sample is added to SSRBENCH; otherwise, it is returned to SSR-Coi.
Chos by
sSR-CoT
Image Text Pair
Task Calegory
SSRBench
Gemini
Retox
Class
Figure 7: Schematic of SSRBENCH construction pipeline.
Task classification for SSRBENCH
You are an expert in image- based question classification. You need to classify each input question into specific task type based on the following
24

---

taxonomy:
Task Categories:
Spatial:
Explanation: Involve identifying and understanding the position, size, shape, and relative relationships of objects in an image.
Subtasks:
Count: Counting objects in the image (e.g.. questions like "How many ...?").
Relative Position Recognition: Determining spatial relations like "to the left of", "above", or "on the right".
Position Based Object Recognition: Identifying an object based on its spatial relation
another object (e.g.. "What the object to the left of the dog?").
General:
Explanation: Involve classifying, recognizing, reasoning about visual content without necessarily focusing on spatial relations.
Subtasks:
Determining whether an object or feature is present (e.g.. "Is there cat?"). Attribute Recognition: Identifying attributes like color, texture, size, state (c.g.,
Existence:
"What color is the apple?").
Action Recognition: Recognizing what action or activity is occurring (e.g., "What is the man doing?")..
For each input question:
First determine whether the question belongs to the spatial general category. Then classify it into one of the three subtasks under that category. If the question does not match any of the subtasks under either category, return None.
Output format:
("category": "spatial" or "general", "subtask"; "subtask name" or "None"]
Example Input: "Is there bicycle in the image?"
Example Output: ("category": "general", "subtask": "existence")
Now, let's begin classification. Here's the question: Question: Question
Case Studies
SpatialBench
SSRBench
the away banune?
the
[A] Yes
(A) Tablecioth
Claude
Nothing
X
GPT-4o mini
No
x
Owon2. VL-9B
No
X
GPT-4g mini
None X
5wen2.5
Clair
X
SparialBot
(Ours)
SpatisiBot
Plant
(Ours)
Figure 8: Two examples illustrating question- answering performance by baseline models compared to our SSR are presented.
To further illustrate the effectiveness of our proposed SSR, we provide two example cases in Figure 8,
comparing the performance of SSR against five baseline models: Claude-3.5-Haite [99], GPT-4o- mini [92], SpatialBot [17], LLaVA-NeXT-7B [86], and the backbone model Qwen2.5-VL [91].
25

---

As shown, our SSR consistently produces correct answers, whereas all baseline models fail to provide accurate responses.
In the left example, the images depict only people and bananas. Consequently, the model must abandon conventional assumptions and carefully reason about the spatial relations explicitly present in the image to answer accurately. In the right example, complex relationships among numerous objects are depicted, and relevant features for answering the posed question are not immediately obvious. In this case, the model must thoroughly comprehend the correspondence between each object and the given question, as well as understand intricate spatial relations among these objects, to produce correct response. These examples clearly demonstrate that our SSR effectively enhances the spatial awareness and reasoning capabilities of lision-language models, thereby significantly improving their ability to understand complex spatial relationships.
26

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The claims presented in the abstract and introduction clearly stated and fully aligned with the contributions this paper.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
We discuss the limitations in the Section 6..
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
27

---

Justification:
None of theoretical assumptions.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: We provide the implementation details to reproduce the main experimental results in Section 4.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
28

---

Answer: [Yes]
Justification: Instructions for reproducing our experiments are provided in Section 4. We will publicly release the data and code once they have been finalized and prepared.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.c cc/
for more details.
While we encourage the release of code and data, we understand that this might be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to reproduce the results. See the NeurlPS code and data submission guidelines (https: //nips.. "nips.cc/pppii cccppblic/guu getggguiieccsssai wwweeesummssiioPoooo ssionPolicy) for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Answer: [Yes]
Justification:
The experimental setup required for our study is described in Section 4.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined, or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [No]
Justification: Conducting experimentfor statistical significance when training large-scale models typically demands an exponential increase computational resources. To ad- dress this concern, we performed comprehensive multi-angle analyses on SSR, and our experimental results consistently validated its effectiveness.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
29

---

The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report 2-sigma error bar rather than state that they have 96% CI, if the hypothesis of Normality of errors not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g., negative enror rates).
If error bars reported in tables plots, the authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: Details regarding the computational resources used for all experiments are presented in Section 4.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers, CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs, as well as estimate the total compute.
The paper should disclose whether the full research project required more computing than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
9. Code of ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics https: http:::///euueeee http///nurrrpppeee httpp//pbbbccttt..cccccuueeeeeea
Answer: [Yes]
Justification: The research presented in this paper fully complies with the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
We discuss impacts the Section 6.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
30

---

Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), faireess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does paper describe safeguards that have been put place for the responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: We used publicly available visual instruction tuning dataset and pre-trained visual foundation models and large language models.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example, requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but encourage authors to take this into account and make best-faith effort.
12. Licenses for existing assets
Are the creators original owners of assets (e.g., code, data, models) used in the paper properly credited, and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification:
We cited models and datasets we deal with in this paper.
Guidelines:
The answer NA means that the paper does use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
31

---

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. If this information is available online, the authors encouraged to reach out the asset's s creators.
13. New assets
Question: Are assets introduced in the paper well documented, and is the documentation provided alongside the assets?
Answer:
[Yes]
Justification: We propose novel model SSR, curate a dataset SSR-COT, and construct benchmark SSRBENCH.
Guidelines:
The answer NA means that the paper does not release assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
We do not include human subjects this paper.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
We do not include human subjects in this paper.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
32

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigor, or originality of the research, the declaration is not required.
Question:
Answer: [Yes]
Justification: We utilized LLM for data synthesis and the reporting of performance indicators, which are comprehensively described throughout this paper.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to our LLM policy ptttp://nnerrigee http://uurpp...cooeeeeeeeeeeeeee / for what should or should not be described.
33


Reviews:
[{"review_id": "hzmDvgUZLn", "reviewer": "Reviewer_karK", "summary": "To enhance spatial reasoning, SSR translates raw depth data into structured textual rationales as intermediate representations. These rationales are distilled into latent embeddings via the MIDI module and then passed as tokens into a vision-language model (VLM) to answer spatial reasoning questions. Experiments show that SSR significantly improves spatial reasoning performance across multiple tasks while preserving general VLM capabilities.\n\nIn addition, the authors introduce SSR-COT, a million-scale vision-language dataset with spatial annotations, and SSRBENCH, a comprehensive benchmark for evaluating spatial understanding. Together, these resources support robust evaluation and training of spatial reasoning models.", "questions": "1. Could the authors provide qualitative examples or a detailed analysis of common failure cases to better understand when and why the method underperforms and outperforms baseline models?\n\n2. How does the proposed MIDI module affect efficiency? Can the authors report the computational or memory overhead compared to a standard vision-language model during training and inference?", "limitations": "Limitations have been discussed.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 4, "strengths_and_weaknesses": "Strengths\n1. Novel depth integration approach: The transformation of raw depth information into interpretable textual rationales offers a more meaningful way to incorporate spatial cues compared to direct depth feature fusion, enabling better spatial reasoning.\n\n2. Comprehensive evaluation framework: The introduction of both a large-scale dataset (SSR-COT with over 1 million samples) and the SSRBENCH benchmark provides valuable resources for the community to evaluate spatial reasoning capabilities.\n\nWeaknesses\n\n1. Limited analysis of failure modes: The paper lacks a qualitative discussion of when and why the proposed approach fails, or how it improves upon baseline models in specific scenarios.\n\n2. Unclear computational overhead: While the paper introduces the MIDI module, it does not provide a clear analysis of inference-time computational cost or memory usage compared to a vanilla vision-language model.", "was_revised": true, "final_justification": "**Response to rebuttal**\n\nI thank the authors for their effort and response. My concerns have been addressed. Overall, I believe the paper makes a significant contribution to improving the spatial reasoning capabilities of VLMs by incorporating depth information and chain-of-thought reasoning. I will keep my original positive score.", "rebuttal": "Thanks for your careful and valuable comments. We will explain your concerns point by point.\n\n**W1 & Q1**: Due to conference policy, we are unable to provide full image cases. However, we have analyzed error cases and can describe key phenomena verbally. For *counting* tasks, most failures stem from ambiguous, unclear, or boundary-indistinct target objects (e.g., two overlapping giraffes being misidentified as one), indicating the requirement for more fine-grained, object-oriented visual understanding. Similarly, failure in other tasks arise from issues like small, unclear objects (e.g., a child’s shoe in the image being misrecognized as a sandal due to its pattern), highlighting the need for VLMs to receive and comprehend higher-resolution images.\n\n**W2 & Q2**: We added inference time and memory usage statistics when supplementing general task experiments for Reviewer aoD6's Q2. Across all evaluated tasks, the computational consumption remains highly consistent; a representative result for POPE benchmark is summarised in the table below:\n\n| Model      | Size | Accuracy  | Token Per Second | Memory Usage |\n| ---------- | ---- | --------- | ---------------- | ------------ |\n| Qwen2.5-VL | 3B   | 56.18     | **12.08**        | **20.46**    |\n| SSR        | 3B   | **63.58** | 8.29             | 34.32        |\n\nMoreover, we counted the relevant resource consumption data when fine-tuning Qwen2.5-VL with SSR-CoT for Reviewer JmB8's W3, which can better reflect the importance of Latent Reasoning method in CoT application:\n\n| Model                          | Size | SpatialBench | Token Per Sample | Token Per Second | Inference Time Per Sample |\n| ------------------------------ | ---- | ------------ | ---------------- | ---------------- | ------------------------- |\n| Qwen2.5-VL (w/ SFT on SSR-CoT) | 3B   | 51.28        | 437.28           | **18.88**        | 23.16s                    |\n| SSR                            | 3B   | **59.52**    | **2.62**         | 8.18             | **0.32s**                 |\n\nThe results demonstrate that, although SSR introduces a modest absolute latency per generated token, its latent-reasoning paradigm dramatically curtails the number of CoT tokens needed to reach a final response. Consequently, under the CoT-based evaluation framework, the overall end-to-end inference speed is substantially improved.\n\nWe appreciate your careful review and insightful suggestions. Accordingly, the aforementioned experimental results will be incorporated into Section 5."}, {"review_id": "famWQVxhVk", "reviewer": "Reviewer_5EXr", "summary": "This paper aims to address a common limitation in existing vision-language models (VLMs), where they rely primarily on RGB images and thus lack accurate spatial understanding capabilities. To alleviate this limitation, the paper introduces the new Spatial Sense and Reasoning (SSR) approach, which leverages depth information to generate structured and interpretable rationales in text. Additionally, the approach also uses knowledge distillation to compress the abovementioned rationales into latent embeddings. To train the resulting model, the authors curate the new SSR-CoT dataset as well as introduce SSRBENCH which is a benchmark that evaluates on multiple downstream tasks.", "questions": "Please look at the abovementioned weaknesses. It will be much more insightful to include empirical comparisons to more VLMs that are targeted at spatial reasoning and more ablations.", "limitations": "Yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strength 1 - In terms of clarity and quality of the paper, the paper is relatively well-written and motivated. The model figures are informative and especially helpful in helping the reader to understand the proposed SELongVLM approach.\n\nStrength 2 - With regards to the significance of the work, this paper introduces an interesting method to integrate depth information, which is well-aligned with the latent language embedding space of the base LLM. This can be especially useful and practical since the introduced Mamba-based Image-Depth Interpreter (MIDI) modoule can be integrated into existing VLMS without having to retrain them. This helps to save time and cost for adapting such VLMs to new tasks or improving their performance directly on benchmarks. \n\nStrength 3 - This work also introduces a new large-scale training dataset SSR-CoT and the SSRBENCH evaluation benchmark. The former can be useful for the research community to integrate into their existing data curation and training pipelines for training VLMs that are more capable of spatial reasoning. \n\nWeakness 1 - The proposed SSR approach somewhat lacks novelty. While integrating depth information for spatial reasoning in VLMs effectively is an important research problem, there are already numerous recent works such as SpatialVLM, SpatialBot and SpatialRGPT that have explored adding depth and spatial information to VLMs. It is unclear how different SSR is from the abovementioned three works.\n\nWeakness 2 - More importantly and related to Weakness 1, the empirical results included in the paper do not compare to some VLMS that are specifically targeted at spatial reasoning, such as SpatialVLM and SpatialRGPT among many others. It will be much more insightful if such empirical comparisons are included. \n\nWeakness 3 - Finally, while the reported performance of SSR across multiple benchmarks appear to be strong, it is difficult to understand the relative importance of the different components proposed in the paper.  For instance, the paper does not include any detailed ablation on whether using depth information alone could provide substantial performance gains without generating the intermediate chain-of-thoughts rationale.", "was_revised": true, "final_justification": "The authors have addressed my concerns.", "rebuttal": "Thanks for your careful and valuable comments. We will explain your concerns point by point.\n\n**W1**: A key differentiator between our proposed SSR framework and existing approaches (SpatialVLM, SpatialBot, SpatialRGPT) lies in the portability of spatial understanding capabilities. Specifically, our MIDI module enables zero-shot transfer of spatial reasoning abilities to diverse VLMs without requiring retraining. This is achieved by translating spatial information into latent tokens within the semantic space, allowing seamless integration. Notably, as demonstrated in our response to Reviewer JmB8's W2 iii, when integrated with latest powerful VLMs like InternVL3, the MIDI module enables performance exceeding Qwen2.5-VL. In contrast, adapting the existing methods to newer, more capable VLMs necessitates complete re-adaptation and retraining, which is a process demanding significant computational resources.\n\n**W2**: Thank you for suggesting these spatial understanding baselines. SpatialVLM has not released official code or checkpoints; we therefore evaluated the community-recommended third-party re-implementation version:\n\n| Model                  | Size | SpatialBench | SSRBench$^{\\text{G}}$ | SSRBench$^{\\text{S}}$ | CV-Bench  |\n| ---------------------- | ---- | ------------ | -------------- | -------------- | --------- |\n| SpatialVLM (3rd Party) | 3B   | 57.66        | 51.23          | 50.97          | 61.53     |\n| SSR                    | 3B   | **59.52**    | **79.30**      | **69.73**      | **68.90** |\n\nWe will incorporate this results into Table 3 in the final version. Regarding SpatialRGPT, we note that it is an object-centric approach that expects one or more target objects to be supplied with bounding boxes or segmentation masks in every query. Designing a protocol that fairly aligns this requirement with our setting is non-trivial, and we were unable to complete such an adaptation within the current submission deadline. We will explore appropriate evaluation methods and update the evaluation results in time.\n\n**W3**: Thanks for your suggestions, to comprehensively evaluate our SSR method, we conducted supplementary ablation studies focused on depth information. Specifically, we established two baselines: (1) directly integrating depth data into the backbone model, and (2) removing depth information from the plug-and-play setting. We then rigorously compared the performance of SSR against these baselines:\n\n| Model                          | Size | SpatialBench | SSRBench$^{\\text{G}}$ | SSRBench$^{\\text{S}}$ |\n| ------------------------------ | ---- | ------------ | -------------- | -------------- |\n| Qwen2.5-VL                     | 3B   | 53.80        | 62.80          | 48.83          |\n| Qwen2.5-VL (RGB + Depth)       | 3B   | 52.46        | 62.57          | 49.03          |\n| SSR$_{\\text{PAP}}$ (w/o Depth) | 3B   | 55.98        | 63.50          | 49.07          |\n| SSR$_{\\text{PAP}}$             | 3B   | 57.20        | 64.37          | 50.97          |\n\nAs shown in above table, directly integrating depth information into the backbone model impairs its understanding capability, leading to performance degradation. However, spatial-related CoT learning systematically enhances the model's spatial reasoning capacity. Subsequent incorporation of depth information further amplifies this spatial understanding, demonstrating the complementary effect of depth data when processed through the CoT framework."}, {"review_id": "cZXHQl3AwA", "reviewer": "Reviewer_JmB8", "summary": "This paper focuses on enhancing and evaluating spatial understanding leveraging depth images and reasoning traces or rationales. First of all, authors present two datasets, SSR-CoT and SSR-Bench, which are built semi-automatically from existing datasets, such as LLaVA-CoT, Visual-CoT, VO-CoT and SpatialQA. Both datasets are VQA datasets, where rationales are added together with the answers. SSR-CoT is built for training purposes, and SSR-Bench is directly extracted from SSR-CoT. As the second contribution, authors present a new framework to build VLMs with spatial understanding. The main idea is to leverage RGB and depth images together with rationales to train a new module called MIDI. MIDI is designed to produce latent tokens that aggregate the information from RGB, depth and answer rationales, which can be used as the input for a standard VLM for general VQA tasks. Authors evaluate their approach on various datasets, including their SSR-Bench showing improvements.", "questions": "1. In line 20, authors state: \"relying solely on RGB is inadequate for accurately capturing spatial information such as relative positions and distances\". I do not agree with this statement. Indeed, the depth information used by your system comes from RGB images, which means that in principle, that information could be represented by VLMs in their latent representations. Do you have any evidence to support this claim?\n2. From line 37 to line 43: I understand this text is written to motivate your work. However, I think it is too vague. You should better specify what VLMs are missing for spatial understanding. By the way, what does this sentence mean? \"existing methods incorporate depth explicitly without capitalizing on its inferential value\". Could you clarify it please?\n3. From line 47 to 53: similar to the previous. I think the text is too vague. You should better motivate the usage of language-based rationales. \n4. Typos: lines 77 and 78: there are two \"has\" that sould be \"have\".\n5. Line 113: Please, rewrite the sentence. \n6. Line 155: \"an efficient language within the MIDI\" -> \"an efficient language model within the MIDI\"?\n7. Equations 1 and 2 are the same loss function, aren't they? Why do you use two different names to refer to them? \"Standard causal modeling loss\" and \"standard cross-entropy loss for auto-regressive generation\". Did I miss anything here?\n8. Line 200: authors mention that they perform sampling quality checks. Can you explain how you do so?\n9. Line 245: detailed hyperparameters in Appendix A, but how did you set those hyperparameters? I couldn't find any information about the procedure. \n10. Table 4: I don't know MME in detail, but it's very weird to me to see the employed performance metric (MME reports summation). It is very difficult to understand the differences in performance between different systems. Could you clarify this point better, please?\n11. Table 5: It is not clear to me what PAP means here. I think you refer to the usage of latent tokens generated by MIDI as the result of the Stage 1 training, directly as the input of VLMs, but without any Stage 2 training. Is that right? In that case, please make it more clear in the text. \n12. Figure 6: I don't understand this figure and its motivation. Legends for the axes are missing and the text in Section 5.3 is not very descriptive. It may be my fault, but I read it three times and I don't understand what this is showing us.", "limitations": "Yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strengths:**\n1. The SSR-CoT and SSR-Bench datasets are interesting.\n2. Leveraging depth images estimated from monocular RGB images is also an interesting idea.\n3. The plug-and-play nature of the latent tokens learnt by MIDI, which could be used for any VLM (although this is not proven in the experimentation).\n\n**Weaknesses:**\n1. The writing of the paper can be improved in my opinion. The paper lacks clarity at many points and makes some claims that are very arguable in my opinion (I list some of those points in the Questions section). \n2. Many of the design decisions are not justified. Examples: i) the selection of vision encoders for RGB and depth images (why CLIP for one and SigLIP for the other?), ii) the usage of Mamba for the MIDI module (why not a standard transformer-based LM?), iii) why do they use an LLM to align MIDI latent tokens semantically and not the VLM itself (which has an LLM that could be used and seems a more natural option), iv) why do they use Qwen2.5 as the alignment LLM and not another LLM? I'm not saying the design decisions are wrong, but I think those decisions should be justified in the paper. \n3. The experimental methodology has many flaws from my point of view. \n- I think some baselines are missing to support the main claims of the paper, which are, to the best of my understanding, that depth images and language-based rationales are key to improve spatial understanding of VLMs. One of the baselines is Qwen2-5-VL (their base VLM) directly fine-tuned in SSR-CoT, without the MIDI component, but with the rationales provided by the dataset. Another important baseline could be their whole system but without the depth images. I think those baselines are needed to really understand the contribution of depth images and rationales in the results. \n- The selection of evaluation datasets is not justified and is arguable. I think that some spatial understanding datasets are missing, such as Visual Spatial Reasoning [1] and What's Up [2]. On the other hand, I don't understand why MME is used, when the dataset is not designed for spatial understanding (and I have no evidence spatial understanding is key for improved results; I may have missed something here). \n- The evaluation protocol is weak. They use a judge LLM, but they do not provide any human correlation information. They blindly believe the output of the judge, which is not a good practice. On the other hand, the selection of the judge LLM is not suitable. In general, LLMs tend to evaluate better their own outputs, and as far as I know, it is not recommended to use the same LLM family to evaluate competing LLMs. In this case, their system is based on Qwen2.5-VL, whose backbone LLM is Qwen-2.5, and the judge is also a Qwen2.5 LLM (the 14B variant).\n\n[1] Liu, Fangyu, Guy Emerson, and Nigel Collier. \"Visual spatial reasoning.\" Transactions of the Association for Computational Linguistics 11 (2023): 635-651.\n\n[2] Kamath, Amita, Jack Hessel, and Kai-Wei Chang. \"What’s “up” with vision-language models? Investigating their struggle with spatial reasoning.\" In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp. 9161-9175. 2023.", "was_revised": true, "final_justification": "I raised my score to 4, because the authors show that their approach actually improves over strong baselines. The evaluation methodology is now clear and correct, and the experiments performed are significant. Besides, authors also contribute an interesting dataset for multimodal spatial reasoning. However, I still have concerns about the clarity and writing of the paper. The authors promised to improve it, but even in their rebuttal text they have many typos. That is why I score the paper with a 4.", "rebuttal": "Thanks for your careful and valuable comments. We will explain your concerns point by point.\n\n**W1**: We will revise the paper per your suggestions and carefully review all expressions in the paper.\n\n**W2 i)**: Consistent with the prevailing paradigm in modern VLMs (e.g., LLaVA), we adopt CLIP as the RGB encoder. For depth modalities, we follow the SpatialBot design and employ SigLIP.\n\n**W2 ii)**: As shown in Table 3(a) of the original Meteor paper, the Mamba model has inherent computational efficiency based on linear complexity and strong long sequence modeling capability. With the same parameter count, BERT, XLNet, and GPT all require roughly twice the computation of Mamba, highlighting its clear advantage in throughput.\n\n**W2 iii)**: We align the latent tokens with the LLM rather than with the feature space of a particular VLM, because our objective is to situate the tokens within a universal semantic space, like CLIP, thereby maximizing cross-model generalizability. Consequently, after Stage 1 training, MIDI can be seamlessly transferred to diverse VLMs in a plug-and-play fashion:\n\n| Model                           | Size | SpatialBench | SSRBench$^{\\text{G}}$ | SSRBench$^{\\text{S}}$ | CV-Bench |\n| ------------------------------- | ---- | ------------ | -------------- | -------------- | -------- |\n| InternVL3                       | 8B   | 66.15        | 68.90          | 58.90          | 75.43    |\n| MIDI + InternVL3$_{\\text{PAP}}$ | 8B   | **72.73**    | **75.97**      | **71.73**      | **83.8** |\n\nAs illustrated in the table below, when MIDI is combined with InternVL3-8B without any task-specific adaptation, it achieves markedly superior performance, underscoring both the generalization capacity and the broad applicability of our approach.\n\n**W2 iv)**: The majority of experiments in this paper employ models from the Qwen series, which have emerged as one of the most widely adopted open-source large language models in the research community. Their extensive usage, like InternVL series, offers a well-established and readily available foundation, thereby facilitating the seamless integration and evaluation of our proposed approach.\n\n**W3 a)**: Thanks for your suggestions, we supplement these contrastive experiments to provide a clearer understanding of the role of depth information:\n\n| Model                          | Size | SpatialBench | SSRBench$^{\\text{G}}$ | SSRBench$^{\\text{S}}$ |\n| ------------------------------ | ---- | ------------ | -------------- | -------------- |\n| Qwen2.5-VL (w/ SFT on SSR-CoT) | 3B   | 51.28        | 38.40          | 28.60          |\n| SSR$_{\\text{PAP}}$ (w/o Depth) | 3B   | 55.98        | 63.50          | 49.07          |\n| SSR$_{\\text{PAP}}$             | 3B   | 57.20        | 64.37          | 50.97          |\n\nWe also provide the performance indicators of the qwen2.5-vl base model after adding depth information. Please refer to the response for reviewer 5EXr's W3, where the analysis description will also be provided.\n\n**W3 b)**: Both Visual Spatial Reasoning and What’s Up were constructed prior to the era of LLM; owing to time and resource constraints, we report zero-shot performance instead of task-specific fine-tuning on these datasets in the table below:\n\n| Model      | Size | VSR$_{\\text{Random}}$ | VSR$_{\\text{Zero-Shot}}$ | What's Up |\n| ---------- | ---- | --------------------- | ------------------------ | --------- |\n| Qwen2.5-VL | 3B   | 72.98                 | 76.43                    | 85.37     |\n| SSR        | 3B   | **78.63**             | **82.90**                | **87.93** |\n\nAs for MME benchmark, our primary objective is to demonstrate that the proposed method directionally strengthens the VLM’s spatial-reasoning capabilities without degrading its performance on general vision–language tasks, please refer to the supplementary experiments we provided to Reviewer aoD6's Q2 for more results.\n\n**W3 c)**: As mentioned in lines 231–236, the LLM-Assistant evaluation only requires a straightforward textual comparison between the model’s response and the ground-truth answer, which is a easy operation for contemporary, highly-capable LLMs. To mitigate potential biases arising from employing models of the same family as judges, we re-assessed the SSRBench results with `GPT-4o-mini`, the outcomes are reported in the table below (For each result, the left column presents the original Qwen-based evaluation score, while the right column reports the corresponding GPT judgment):\n\n| Model      | Size | Existence   | Attribute   | Action      | Counting    | Position    | Object      |\n| ---------- | ---- | ----------- | ----------- | ----------- | ----------- | ----------- | ----------- |\n| Qwen2.5-VL | 3B   | 66.4 / 58.9 | 58.9 / 59.5 | 63.1 / 67.5 | 34.1 / 34.1 | 60.5 / 58.6 | 51.9 / 52.6 |\n| SSR        | 3B   | 83.2 / 81.3 | 82.1 / 79.8 | 72.6 / 73.3 | 51.2 / 51.2 | 83.3 / 80.8 | 74.7 / 74.7 |\n\nThe close agreement between the evaluation scores assigned by different LLM judges in the table indicates that simple answer-comparison tasks are largely immune to bias arising from the use of models within the same series.\n\n**Q1**: RGB images alone provide insufficient spatial information for robust spatial understanding, as evidenced by recent studies (e.g., SpatialVLM, SpatialBot, SpatialRGPT). To address this limitation, existing work focuses on enriching VLMs with explicit spatial cues. While depth information is typically derived from RGB data, it offers a more effective spatial representation by distilling critical distance relationships. Conversely, raw RGB data may introduce ancillary visual noise that obscures spatial reasoning. This principle parallels the role of optical flow in video understanding field: though computationally extracted from RGB sequences, optical flow provides essential motion semantics that significantly enhances neural network performance, which demonstrating how processed geometric representations can surpass raw pixel data in conveying structural information.\n\n**Q2**: We consider that what existing VLMs lack is the ability to perceive spatial information and reasoning. Spatial perception indicates identify the position relationship, real distance between different objects and so on, which are ambiguous to obtain from rgb image. The reasoning ability denotes the integrate and extract the portion of this spatial information that is beneficial to answering the question. The sentence means input depth information directly into VLMs without any adaptation like SpatialBot, we argue that this compromises the efficient use of depth information. We will correct the misquote of this sentence in the final version, thank you for your careful reading.\n\n**Q3**: Thanks for your insightful suggestion. In the final version, we will expand the corresponding paragraph to elucidate why and how SSR converts raw depth maps into a structured latent representation within in semantic space. In addition, we will explicitly highlight the SSR’s superior generalization capacity and computational efficiency supported by supplementary experiments (Response to W2 iii and Response to Reviewer KarK’s Q2).\n\n**Q4 - Q6**: We sincerely appreciate your meticulous suggestions; all typos will be corrected in the final version.\n\n**Q7**: The two equations are indeed the same mathematical loss (causal, next-token cross-entropy), but they are applied to different model inputs and targets, and this difference is the key point. Equation 1 is computed on the reasoning chain.  Input: $Z_{R}, Y_{R,<i}$, Target: $Y_{R,i}$, Goal: train the lightweight LLM to (i) perform implicit reasoning correctly and (ii) keep its latent representations aligned with the frozen LLM’s semantic space. Equation 2 is computed on the final answer tokens. Input: $Y_{A,<j}$, Target: $Y_{A,j}$, Goal: train the VLM to interpret the latent tokens emitted by the LLM and generate the correct textual answer. Thus, the loss are identical in form but operate on disjoint data streams and serve different training objectives in the two-stage pipeline.\n\n**Q8**: Similar to the quality-assessment protocol shown in Table 2, we randomly draw 10 % of the cached samples and evaluate VQA accuracy both with and without their generated rationales. Rationales that improve accuracy are retained and incorporated into the final SSR-CoT dataset; those that degrade performance are discarded, and the all samples in cache are re-submitted for re-annotation.\n\n**Q9**: Owing to the high computational costs inherent in training LLMs, hyper-parameters were configured based on a comprehensive synthesis of prior empirical experience and existing papers.\n\n**Q10**: The MME score shown in Table 4 is indeed an aggregate (sum) of multiple sub-task accuracies. To verify that the targeted spatial-enhancement does not harm general capability, we evaluate the SSR model on more general visual-language tasks, please see our detailed response to Reviewer aoD6's Q2.\n\n**Q11**: PAP indicates the MIDI module is directly applied to the pre-trained VLM in a plug-and-play manner after the training stage 1, without co-training. We will add a clearer explanation to the final version.\n\n**Q12**: Figure 6 visualizes the cosine similarities between the latent tokens produced in two different paradigms: x-axis: latent tokens inserted inside the rationale, y-axis: latent tokens inserted immediately after the question and used to start the answer generation. Diagonal cells represent these two states of the same sample. High values on the diagonal indicate that the model has learned to map the rationale to the latent representation, confirming that it successfully distills the spatial knowledge embedded in the rationale. Low off-diagonal values indicate that the latent tokens remain sample-specific and do not collapse to a generic representation. For improved clarity in the final version, axis labels and enhanced descriptions will be added."}, {"review_id": "IVUB9QdKlT", "reviewer": "Reviewer_aoD6", "summary": "This paper introduces SSR (Spatial Semantic Reasoning), a novel paradigm that redefines depth integration in vision-language models (VLMs) through interpretable and structured rationale-language representations. Furthermore, the proposed framework transforms raw depth data into latent embeddings via knowledge distillation, thereby reducing computational costs. SSR provides a plug-and-play method to improve the spatial reasoning capabilities of existing VLMs. Finally, for a comprehensive evaluation, the author constructs a large-scale vision-language dataset (SSR-COT) and a comprehensive multi-task benchmark (SSR-BENCH).", "questions": "If the authors can address all of my major concerns, I will raise my rating; otherwise, I will lower the current rating. Please correct me if I've misunderstood any aspects of the work.", "limitations": "yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\n1. SSR demonstrates superior comprehension of depth maps and effectively enhances the spatial reasoning capabilities of the VLM.\n\n2. The collection, annotation, and quality evaluation pipeline for the large-scale vision-language dataset (SSR-CoT) are well-designed, providing high-quality spatial Chain-of-Thought (CoT) reasoning data.\n\n3. The SSRBench benchmark thoroughly assesses VLM spatial reasoning performance.\n\nMajor Weaknesses:\n\nQ1. My primary concern regards the training-free SSR_{PAP} module, which demonstrates only marginal performance gains on certain tasks, particularly on the proposed SSRBench. While this limited improvement is understandable given that pretrained VLMs have never seen similar depth latent features before, could the authors explain why the SSR method can operate training-free and be directly plugged into a pretrained VLM model?\n\nQ2. Following Q1, if SSR undergoes Stage 2 training, would this lead to pretrained knowledge forgetting or a degragation in inherent reasoning capabilities? For example, how would SSR-7B perform on other general- or common-sense reasoning benchmarks compared to its backbone, Qwen2.5-VL?\n\nQ3. For the method part, if SSR does not convert depth information into latent features but directly generates language representations as additional text input for the VLM, how would this affect spatial reasoning performance?\n\nMinor Weaknesses:\n\nQ4. In Table 3, while the 7B backbone (Qwen2.5-VL) outperforms the 3B version on all metrics, why does SSR-3B surpass SSR-7B on certain metrics? Is this due to random variability, or does the effectiveness of the proposed method (SSR) diminish as model parameters scale up?\n\nQ5. In Figure 1, the right-side VQA example contains an error: the question is multiple-choice, yet the model responds \"No.\"", "was_revised": true, "final_justification": "Although the authors have addressed my major concerns, some minor weaknesses remain. I hope the authors can conduct additional experiments to provide a more reliable estimate of variance in the final version, which would also further enhance the overall quality of the paper. Therefore, I will maintain my original positive rating.", "rebuttal": "Thanks for your careful and valuable comments. We will explain your concerns point by point.\n\n**Q1**: The latent tokens were aligned into semantic space of LLM through training stage 1, thus VLM based on LLM as backbone can understand the meaning of token to a certain extent.\n\n**Q2**: Limited by time and computational resource constraints, we provide supplementary experiments for the following general VQA tasks based on the SSR-3B model:\n\n| Model      | Size | VQAv2     | TextVQA   | POPE      | MMBench   | GQA       |\n| ---------- | ---- | --------- | --------- | --------- | --------- | --------- |\n| Qwen2.5-VL | 3B   | 72.54     | 57.01     | 84.44     | 75.85     | 56.18     |\n| SSR        | 3B   | **79.04** | **61.33** | **86.00** | **78.26** | **63.58** |\n\nOur proposed SSR model outperform Qwen2.5-VL on multiple general benchmarks, this demonstrates that the pre-training knowledge of the Qwen2.5-VL will not be forgotten, but more common knowledge will be learned during training.\n\n**Q3**: We evaluate the Qwen2.5-VL after fine-tuning on SSR-CoT dataset, overly protracted and convoluted textual intermediate reasoning chains not only increase the risk of erroneous conclusions but also impose prohibitive computational costs that undermine inference efficiency:\n\n| Model                          | Size | SpatialBench | SSRBench$^{\\text{G}}$ | SSRBench$^{\\text{S}}$ | CV-Bench  |\n| ------------------------------ | ---- | ------------ | -------------- | -------------- | --------- |\n| Qwen2.5-VL (w/ SFT on SSR-CoT) | 3B   | 51.28        | 38.40          | 28.60          | 56.73     |\n| SSR                            | 3B   | **59.52**    | **79.30**      | **69.73**      | **68.90** |\n\n| Model                          | Size | SpatialBench | Token Per Sample | Token Per Second | Inference Time Per Sample |\n| ------------------------------ | ---- | ------------ | ---------------- | ---------------- | ------------------------- |\n| Qwen2.5-VL (w/ SFT on SSR-CoT) | 3B   | 51.28        | 437.28           | **18.88**        | 23.16                     |\n| SSR                            | 3B   | **59.52**    | **2.62**         | 8.18             | **0.32**                  |\n\n**Q4**: We thank the reviewer for this perceptive observation. We attribute SSR-3B's occasional superiority over SSR-7B on SpatialBench metrics primarily to two factors: (1) training both models on the identical dataset may inadequately support the larger model's capacity, and (2) SpatialBench's limited size increases susceptibility to statistical fluctuations. Finally, owing to computational constraints we could not perform multiple random restarts to obtain a reliable variance estimate. These factors jointly confound any firm conclusion about a occasional drop in SSR’s effectiveness as parameters scale. We will explicitly discuss these limitations in the final version and investigate them more thoroughly in future work.\n\n**Q5**: The erroneous example depicted in Figure 1 will be amended, with the corrected version also provided in Figure 5."}]

Author rebuttal:

Meta-review: This paper was reviewed by four experts who universally recommended the paper for acceptance.  This paper presents an approach that incorporates depth information into a VLM to boost spatial reasoning performance.  While this has been studied in prior work, results in the rebuttal demonstrate that the proposed approach better utilizes this information by aligning the depth information with text tokens rather than directly using the depth information.  While there are some areas of concern that prevent a higher recommendation, such as missing error bars or a more extensive comparison to closely related work that was raised in the rebuttal, the paper provides enough evidence for their approach that it merits acceptance.  The authors are strongly encouraged to consider reviewer comments for clarity and requests for empirical evidence that can further bolster their contributions for their camera ready.
Decision: Accept (poster)