Title: Actial: Activate Spatial Reasoning Ability of Multimodal Large Language Models
Venue: NeurIPS 2025
Accepted: yes
Keywords: Visual-Language Model

3D Reasoning

GRPO
Abstract: Recent advances in Multimodal Large Language Models (MLLMs) have significantly improved 2D visual understanding, prompting interest in their application to complex 3D reasoning tasks. However, it remains unclear whether these models can effectively capture the detailed spatial information required for robust real-world performance, especially cross-view consistency, a key requirement for accurate 3D reasoning. Considering this issue, we introduce Viewpoint Learning, a task designed to evaluate and improve the spatial reasoning capabilities of MLLMs. We present the Viewpoint-100K dataset, consisting of 100K object-centric image pairs with diverse viewpoints and corresponding question-answer pairs. Our approach employs a two-stage fine-tuning strategy: first, foundational knowledge is injected to the baseline MLLM via Supervised Fine-Tuning (SFT) on Viewpoint-100K, resulting in significant improvements across multiple tasks; second, generalization is enhanced through Reinforcement Learning using the Group Relative Policy Optimization (GRPO) algorithm on a broader set of questions. Additionally, we introduce a hybrid cold-start initialization method designed to simultaneously learn viewpoint representations and maintain coherent reasoning thinking. Experimental results show that our approach significantly activates the spatial reasoning ability of MLLM, improving performance on both in-domain and out-of-domain reasoning tasks. Our findings highlight the value of developing foundational spatial skills in MLLMs, supporting future progress in robotics, autonomous systems, and 3D scene understanding.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces a new task, namely viewpoint learning, to improve the spatial reasoning capabilities of 2D MLLMs. Specifically, this work injects spatial knowledge by finetuning the MLLM on the viewpoint learning tasks using a visual QA dataset curated for that specific task. Further, to improve the generalization capabilities, GRPO is used to train the fonetuned LLM on a broader set of questions using synthetic data.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:-\n\n- The paper is very well-written with a clear motivation.\n- This is the first work to identify and demonstrate that finetuning MLLMs on viewpoint learning significantly boosts their spatial reasoning capabilities.\n- The idea of using GRPO for enhancing generalization after finetuning is interesting.\n- Extensive experiments have been conducted to demonstrate SOTA performances on multiple benchmark datasets.\n\nWeaknesses:-\n\n- Recent works such as MLLM-for3D incorporates a similar idea of having cross-view spatial consistency using multi-view segmentation maps to enhance reasoning capabilities of 2D MLLMs. Comparison with those methods could be more helpful in assessing the efficacy of your method.\n- The method claims to enhance out-of-domain generalization but does not explicitly showcase this in the results section. A better expanation on this would be helpful.\n\nAdditional points:-\n\n- Spatial-MLLM seems to be a concurrent work that follows a very similar pipeline of Supervised fine-tuning on a curated QA dataset followed by GRPO. However, their QA pairs cover different spatial reasoning tasks such as object counting, object size, relative distance etc. Using those tasks along viewpoint consistency could be more beneficial for the initial finetuning of MLLMs.\n\nQUESTIONS:\nSee weaknesses\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-23"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces Actial-7B, a MLLM that is finetuned using SFT and GRPO with Viewpoint-100K dataset. The resulting model shows better performance on spatial-reasoning benchmarks, which indicates the usefulness of the proposed dataset and training pipeline. The paper is built upon the hypothesis that understanding viewpoints is an important aspect for MLLMs to understand the 3D spatial relations, which is proved by finetuning the model using Viewpoint-100K dataset, a dataset that solely consists of viewpoint understanding.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n1. The paper proposed a useful dataset Viewpoint-100K, which can be widely used by the community to enforce better spatial relation understanding of MLLMs.\n\n2. The finetuning recipe is clear enough to be followed.\n\n3. The resulting model from the above two bullet points show decent performance improvements on CV-Bench and BLINK.\n\nWeaknesses:\n\n1. The training recipe and the dataset is only used to finetune one model, i.e.  Qwen2.5-VL-7B-Instruct, lacking a comprehensive evaluation on how the proposed method can be applied to different MLLMs.\n\n2. The model's improvements on 3DSRBench is marginal, which indicates that for complex tasks in 3DSRBench, the model needs abilities more than just viewpoint understanding, which contradicts the authors' hypothesis that viewpoint understanding is important for 3D spatial understanding.\n\nQUESTIONS:\n1. How can viewpoint learning boost the understanding of more complex problems like orientation, as suggested by the marginal improvements on 3DSRBench?\n\n2. Why does the author not consider the viewpoint changes for up and down, the resulting distribution of the object pose under this setting would be different from left-and-right. Is the image pairs in the dataset diverse enough to cover different angles of the objects?\n\n3. The dataset proposed consider a fixed object and a moving camera, can the model better understand the setting where camera is fixed and object is moving?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper aims to bridge the gap between 2D vision-language models (VLMs) and 3D spatial reasoning. To this end, the authors introduce viewpoint learning, a new task to instill 3D awareness through reasoning over 2D multi-view images. Regarding this task, the authors propose Viewpoint-100K, a dataset comprising 100k object-centric multi-view images with corresponding QA pairs. To learn spatial reasoning ability, this paper employs Qwen-2.5-VL-7B-Instruct as the base model, and applies SFT on Viewpoint-100K followed by RL with GRPO on SAT dataset. The trained model demonstrates significant performance improvements on several spatial reasoning benchmarks, including 3DSRBench, CV-Bench, and BLINK.\n\nSTRENGTHS AND WEAKNESSES:\n#### Strengths\n- Good motivation. 3D spatial reasoning is a critical capability to be unlocked for VLMs. And this capability is particularly insufficient for current VLMs. This paper focuses on this significant problem and proposes viewpoint learning to mitigate the gap.\n- Significant experimental results. The proposed approach greatly improves the baseline’s performance on spatial reasoning across multiple benchmarks, validating the effectiveness of the proposed dataset and viewpoint learning.\n- Good clarity. The paper is well-written and logically structured, making it easy to follow. The authors clearly articulate the motivation, methodology, and results.\n\n#### Weaknesses\n- Collecting object-centric multi-view images is a good solution to cultivate 3D spatial reasoning skills. However, I think it may not be optimal to anchor the task on discriminating how the camera translates and rotates through texts. This task is quite difficult (even for humans), and learning can be challenging for models considering the task complexity and sparse supervision (only supervising short text answers). While how to properly formulate viewpoint learning remains an open question, I think it pertains to perceptual capability that should be embedded at the representation level rather than handled by language model reasoning. As such, casting it as a language task might not be the most effective route.\n- The results and analyses on how to utilize Viewpoint-100K are not intuitive. The authors said that RL with GRPO leads to high KL divergence (i.e., a significant departure on the policy). I think this cannot support that RL is inadequate for overcoming the 2D-centric bias. And I think the improvement by applying direct SFT does not necessarily indicate a better foundamental spatial reasoning capability, considering the strong supervision on fitting the text. Anyway, this is not a convincing approach for knowledge injection from my perspective.\n\nQUESTIONS:\n- I cannot get what it means and what the authors want to deliver in Figure 2. What does it mean by “Matched” or “Mismatched”?\n- I would expect more results on Viewpoint-100K, including human performance and comparison between RL vs. SFT.\n- There is little discussion about the quality of the pseudo CoT data (I guess Figure 4 does not refer to the collected CoT data). And if the quality is an issue, why still using a proportion for training instead of refining them in advance?\n- Some “w/o K.L.” results in Table 1 outperform the full method. Does “w/o K.L.” mean the first SFT stage is removed? This should be clarified, and the results should be highlighted accordingly to aid interpretation.\n- What is the observed impact of hybrid cold-start initialization using CoTs? Additional analysis or ablations would be helpful.\n- Why name it Actial, any potential meaning or metaphor?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 5, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

comments: SUMMARY:
The paper tackles the lack of reliable 3-D spatial reasoning in multimodal LLMs. It frames Viewpoint Learning and introduces Viewpoint-100K dataset. Built on Qwen-VL-7B, the proposed two-stage pipeline: 1) supervised fine-tuning on Viewpoint-100K, mixing in 10 % Gemini-generated chains-of-thought. 2) reinforcement learning via GRPO on a synthetic SAT set. The resulting Actial-7B gains 12–14%  over the baseline on several spatial benchmarks.

STRENGTHS AND WEAKNESSES:
[+] Well-motivated focus on 3-D consistency.
[+]  Simple two-stage recipe with complementary gains, supported by ablations.
[+]  Insightful error analysis illustrating shift from 2-D shortcuts to 3-D reasoning.

[-]  Performance gap to proprietary SOTA remains; no significance tests.
[-]  Lack of ablation study on the RL training. Contrast RL stage with simply scaling supervised data.

QUESTIONS:
1. Could similar generalization be achieved by simply adding more diverse supervised data instead of RL? An ablation contrasting these options would help isolate the benefit of GRPO.

2. Please also clarify how the reward is defined in the GRPO stage: What components does it comprise (e.g., format reward and result reward) and how are they computed? 

3. Why not compare against a variant that directly regresses relative poses (e.g., via cosine loss) rather than multiple-choice classification?

LIMITATIONS:
No
TITLE: Official Review
CLARITY: 3
IMPACT: 2
ORIGINALITY: 3
DATE: 2025-07-05