Title: SpatialReasoner: Towards Explicit and Generalizable 3D Spatial Reasoning
Venue: NeurIPS 2025
Accepted: yes
Keywords: 3D Spatial Reasoning

Vision-Language Models
Abstract: Despite recent advances on multi-modal models, 3D spatial reasoning remains a challenging task for state-of-the-art open-source and proprietary models. Recent studies explore data-driven approaches and achieve enhanced spatial reasoning performance by fine-tuning models on 3D-related visual question-answering data. However, these methods typically perform spatial reasoning in an implicit manner and often fail on questions that are trivial to humans, even with long chain-of-thought reasoning. In this work, we introduce SpatialReasoner, a novel large vision-language model (LVLM) that addresses 3D spatial reasoning with explicit 3D representations shared between multiple stages--3D perception, computation, and reasoning. Explicit 3D representations provide a coherent interface that supports advanced 3D spatial reasoning and improves the generalization ability to novel question types. Furthermore, by analyzing the explicit 3D representations in multi-step reasoning traces of SpatialReasoner, we study the factual errors and identify key shortcomings of current LVLMs. Results show that our SpatialReasoner achieves improved performance on a variety of spatial reasoning benchmarks, outperforming Gemini 2.0 by 9.2% on 3DSRBench, and generalizes better when evaluating on novel 3D spatial reasoning questions. Our study bridges the 3D parsing capabilities of prior visual foundation models with the powerful reasoning abilities of large language models, opening new directions for 3D spatial reasoning.


Peer reviews:
comments: SUMMARY:
The paper introduces SpatialReasoner for 3D spatial reasoning. SpatialReasoner employs structured 3D representations. The model understands the space with transformations among different space coordinates. The model is trained in two stages: SFT and RL to enhance adaptability and generalization. Experimental results across multiple benchmarks show that the model achieves excellent performance.

STRENGTHS AND WEAKNESSES:
Strength:

1. The work is solid with a generated 3D-aware training data. Also, SFT and RL are both adopted and analyzed.

2. Comprehensive experiments are done on various baseline models and benchmarks.

3. The paper is well written and easy to follow

Weakness:
1. Please provide more visual examples of your dataset, and comparasion of model outputs.

2. What is the accuracy of estimation 3D positions? (On-demand 3D parsing in Figure 2)

3. The images in the article does not appear to be vector images.

4. Please provide more metadata. e.g. the size of training dataset. For line 207~208, the authors may want to provide more details.

QUESTIONS:
1. How would the model perform on robotics setting? Please consider some spatial understanding datasets on robots:

[1] Song, Chan Hee, et al. "Robospatial: Teaching spatial understanding to 2d and 3d vision-language models for robotics." Proceedings of the Computer Vision and Pattern Recognition Conference. 2025.
[2] Li, Yun, et al. "Sti-bench: Are mllms ready for precise spatial-temporal world understanding?." arXiv preprint arXiv:2503.23765 (2025).

I don't have further questions. The paper is good to be accepted to NeurIPS.

LIMITATIONS:
Yes, the authors have discussed the limilations of their method.
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 4
DATE: 2025-06-03

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper explores the integration of explicit 3D representations into the outputs of large vision-language models (LVLMs). The motivation is that 3D representations offer interpretable and debuggable structures, enabling clearer reasoning and computation over visual content. The proposed approach demonstrates performance improvements on both in-domain objects and, to a lesser extent, on out-of-domain objects.\n\nSTRENGTHS AND WEAKNESSES:\n# Strengths\n\n- The use of explicit 3D representations in LVLM outputs is a novel and compelling idea\n- Comprehensive evaluation across both in-domain and out-of-domain objects\n\n\n# Weakness\n- Section 3.4 lacks sufficient detail on data synthesis. It's unclear how the dataset is constructed, its size, or the distribution of question types, making reproducibility difficult.\n- The method for computing the 3D coordinates in Figure 2 is not clearly explained. It's also unclear what units are used and whether the representation can handle distant objects, such as cars in street scenes.\n- The presentation of experimental results is difficult to follow. Key findings are scattered, and understanding the full picture often requires digging into the appendix. For example, the analysis in Table 6 on spurious correlations in CVBench-3D depth is insightful, but easy to overlook.\n\nQUESTIONS:\n- **Dataset**: What's the size of the SFT and RL dataset? Any numbers showing the distribution of different question types?\n\n- **Results**\n  - SpatialReasoner-Zero outperforms on novel objects. Could this suggest that explicit 3D representations may actually hurt performance in some cases? For instance, requiring the model to generate 3D outputs might increase hallucinations.\n  - In L248, what does “multi-object-closer-to” mean? \n\n- **Training**: Line 179: It’s unclear how the 3D-aware process reward is defined or explored. More detail would help.\n\nLIMITATIONS:\nThe paper does not explicitly discuss its limitations. Given that its core contribution is the integration of explicit 3D representations into reasoning traces and the proposal of a data synthesis method, the lack of clarity around the data construction process is a significant issue. The writing should be improved to make the data synthesis pipeline and its characteristics more transparent.", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 4, "CLARITY": 1, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-06-28"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper aims to impart explicit 3D reasoning to VLMs. Specifically, the paper collects and pseudo-annotates images with 3D annotations and chains of thoughts, and uses this to fine-tune a model to make it 3D-aware. The paper explores doing this via, RL-only, SFT-only, SFT+RL and achieves good improvements over prior SOTA. The paper also ablates several design choices and studies failure modes of their models.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n- The paper is well-written and easy to follow\n- Making VLMs have explicit 3D reasoning is an interesting direction, and the paper achieves good results by doing so\n- I liked the ablations in the papers, very thorough and cover a lot of important baselines. I also like the failure analysis on the CVD-bench datasets as well as analysis on which among the 3D perception and computation contributes to the most errors\n\n\nWeaknesses:\n- I strongly recommend moving the implementation details section to the main paper. In my first reading, I was lost on a) what is the base VLM the paper starts with b) what is the training data? What exactly is the SFT and RL model are trained on. These are crucial details to understand the experiments, and I think they need to be in the main paper\n-  Spatialreasoner-zero is not defined and was misleading for me initially. I think this is following the deepseek’s terminology; however “-zero” to mean only RL is non-standard in the literature and I initially confused it with “zero-shot”. Please either introduce this term or instead use spatialreasoner-RL-only which is perhaps much more clear\n- Another crucial detail from the implementation detail section was that the SFT model is trained on one set of dataset, and then an additional portion of dataset is used for RL fine-tuning. Naturally, one thinks of an SFT baseline which uses both the datasets and authors include it as SpatialReasoner-SFT (+ HQ SFT) [which was unclear to me before reading implementation details]. This baseline underperforms the model without the additional fine-tuning. \n    - I think a more stable baseline would be to concatenate both datasets together and train an SFT baseline on them jointly instead of sequential training\n    - In the sequential training, what checkpoint did the authors take? Since the second-stage SFT data is tiny, it is likely that a very early checkpoint would lead to the best performance, while the latter checkpoints would be quite overfitted [that’s why joint training would actually be more stable for getting a stronger SFT baseline. Unlike RL training, SFT training does not need two stages, and hence it should be allowed to see all the data together]\n- The paper does not contain the details on how the pseudo-annotations were created. I would highly recommend the authors to add those details at least to the supplementary. \n- Additional Baselines: Did the authors consider fine-tuning the “specialist” baselines like spatial-rgpt on their additional collected data? \n\n\nI think this paper has good contributions and detailed results, and most of the weaknesses I listed are fixable. Hence, I am recommending acceptance for now, and looking forward to author rebuttal.\n\nQUESTIONS:\nI think this paper has good contributions and detailed results, and most of the weaknesses I listed are fixable. Hence, I am recommending acceptance for now, and looking forward to author rebuttal. The main actionable experiments in my opinion are:\n-  SFT basleine with joint data from the proposed dataset instead of two-stage fine-tuning\n- (can skip if this is painful / non-standard to setup) specialized models fine-tuned on the collected dataset\n\nBesides these, fixing up the wtiting concerns that I pointed out in my review would help quite a lot!\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a LVLM called SpatialReasoner especially designed to solve 3D spatial reasoning problem based on explicit 3D representations. Besides, this work constructs the corresponding training data for SFT and RL training. The paper comprehensively analyzes the proposed model, including the effectiveness of explicit 3D representations and the multi-stage training strategy.\n\nSTRENGTHS AND WEAKNESSES:\nStrength:\n1. The paper is well-written.\n2. The idea of reasoning with explicit 3D representations is reasonable, and the comparison with Gemini 2.0's reasoning is especially convincing regarding its necessity.\n3. The experiments on RL training are interesting and reveal some intriguing findings.\n4. The open access commitment of this work, especially the RL training code and data, will be beneficial to the community.\n\n\nWeakness:\n1. The performance drops on CVBench-3D distance questions remains confusing, though the authors claim this can be attributed to shortcut-driven 2D biases in CVBench-3D. In my opinion, reasoning in 3D should not perform worse in such simpler scenarios. I think this can be attributed to errors in predicted 3D positions. I would like to see whether the model can outperform Qwen2.5 VL given ground truth 3D positions.\n2. The reward design for RL training is ambiguous, especially for the 3D-aware process reward. It would be better to express it with a clearer formula.\n\nQUESTIONS:\n1. I'm curious about the comparison between Qwen2.5 VL and SpatialReasoner when both are given ground truth 3D locations for reasoning to answer questions, thereby verifying whether SpatialReasoner actually learns something better after SFT and RL training.\n\nLIMITATIONS:\n1. As the authors state, SpatialReasoner consistently performs explicit multi-step 3D reasoning, which is not efficient.\n2. In more complex scenarios like multi-image inputs, it may be difficult for the model to generate such explicit 3D representations.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}