Actial: Activate Spatial Reasoning Ability of Multimodal Large Language Models
Authors: Xiaoyu Zhan
Wenxuan Huang
Hao Sun
Xinyu Fu
Changfeng Ma
Shaosheng Cao
Bohan Jia
Shaohui Lin
Zhenfei Yin
LEI BAI
Wanli Ouyang
Yuanqi Li
Jie Guo
Yanwen Guo
Venue: NeurIPS 2025 poster ()
Keywords: Visual-Language Model
3D Reasoning
GRPO
Primary area: deep_learning

Abstract: Recent advances in Multimodal Large Language Models (MLLMs) have significantly improved 2D visual understanding, prompting interest in their application to complex 3D reasoning tasks. However, it remains unclear whether these models can effectively capture the detailed spatial information required for robust real-world performance, especially cross-view consistency, a key requirement for accurate 3D reasoning. Considering this issue, we introduce Viewpoint Learning, a task designed to evaluate and improve the spatial reasoning capabilities of MLLMs. We present the Viewpoint-100K dataset, consisting of 100K object-centric image pairs with diverse viewpoints and corresponding question-answer pairs. Our approach employs a two-stage fine-tuning strategy: first, foundational knowledge is injected to the baseline MLLM via Supervised Fine-Tuning (SFT) on Viewpoint-100K, resulting in significant improvements across multiple tasks; second, generalization is enhanced through Reinforcement Learning using the Group Relative Policy Optimization (GRPO) algorithm on a broader set of questions. Additionally, we introduce a hybrid cold-start initialization method designed to simultaneously learn viewpoint representations and maintain coherent reasoning thinking. Experimental results show that our approach significantly activates the spatial reasoning ability of MLLM, improving performance on both in-domain and out-of-domain reasoning tasks. Our findings highlight the value of developing foundational spatial skills in MLLMs, supporting future progress in robotics, autonomous systems, and 3D scene understanding.

Actial: Activate Spatial Reasoning Ability of Multimodal Large Language Models
Xiaoyu Zhan'
Wenxuan Huang3.4+1, Hao Sun Xinyu Fu', Changfeng Ma
Shaosheng Cao
x
Bohan Jia Shaohui Lin Zhenfei Yin", Lei Bai",
Wanli Ouyang". Yuanqi Li', Jie Guo', Yanwen Guo
X
Nanjing University Xiaohongshu Inc.
East China Normal University
The Chinese University of Hong Kong
5 Shanghai Jiao Tong University
University of Oxford
{zhanxy, hao sun, xinyu. fu, (aangfeengaa@@mmailll www.iinnn.................. edu. aju. osilly06160gmal1. com, caothaoshenngex.ccchngggg.u com fyuanqili, guojie, yuguo) (gg.o)..ju.edu.cco edu. cn
Abstract
Recent advances in Multimodal Large Language Models (MLLMs) have signifi- cantly improved 2D visual understanding, prompting interest in their application to complex 3D reasoning tasks. However, remains unclear whether these models can effectively capture the detailed spatial information required for robust real-world performance, especially cross-view consistency, a key requirement for accurate 3D reasoning. Considering this issue, introduce Viewpoint Learning, task designed to evaluate and improve the spatial reasoning capabilities of MLLMs. We present the Viewpoint- 100K dataset, consisting of 100K object-centric image pairs with diverse viewpoints and corresponding question-answer pairs. Our approach employs two-stage fine-tuning strategy: first, foundational knowledge is injected to the baseline MILMM via Supervised Fine- Tuning (SFT) on Viewpoint- 100K, resulting in significant improvements across multiple tasks; second, generalization is enhanced through Reinforcement Learning using the Group Relative Policy Optimization (GRPO) algorithm on broader of questions. Additionally, we introduce hybrid cold-start initialization method designed to simultaneously learn viewpoint representations and maintain coherent reasoning thinking. Experimental results show that our approach significantly activates the spatial reasoning ability of MILM. improving performance on both in-domain and out-of-domain reasoning tasks. Our findings highlight the value of developing foundational spatial skills in MILMs, supporting future progress robotics, autonomous systems, and 3D scene understanding.
Introduction
Multimodal Large Language Models (MLLMs) [7 28 36 significant advances in visual understanding and inference. Naturally, the researchers 34 demonstrate considerable interest in their abilities for spatial reasoning tasks.
39
8 42
have recently achieved
33
30
Equal contribution. t Project leader. Corresponding authors.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Vieupoint Learning
Two- Stage Activation
Improvement
Owm--VV
Actial
foustarional lok
Gesersltzed Malla Tooko
Camera noves to the right and rotate the left
Inowledge Injection
Generalizatin
Exhancement
Qua32. SA VL 78-Enetruct
Act1a3-70
Figure 1: We aim to activate the MI.LM's spatial reasoning ability with Viewpoint Learning and the
two-stage fine-tuning strategy.
In the computer vision field, addressing 3D tasks typically requires first establishing cross-view consistency through methods such as camera calibration 440 and stereo matching [ 2] However, recent studies (29](3) (0) aim to enable MLLMS to directly perceive such consistency from multi-view images sequential video frames, allowing accurate spatio-temporal reasoning. This development raises a critical question: Do these MLLMs have the potential to capture the fine-grained 3D spatial information needed for robust and reliable visual-spatial performance in real-woris 3D scenarios?
As a rule, all 3D objects maintain 3D consistency space, characterized by stable spatial relationships and geometric properties. This consistency ensures that objects retain 2D continuity in timeline when projected onto a 2D plane. Our key concern is whether existing MILMs, which are trained primarily on video data emphasizing 2D continuity, can achieve understanding of spatial 3D consistency, as opposed to merely tracking continuous pixel-level evolution or correlated pixel mappings. Furthermore, the camera projection commonly introduces subtle distortions, imperceptible to humans, which complicate the establishment of relationships between 2D continuity and 3D consistency.
48 has made it evident that current MLLMs still struggle capture cross- view consistency. However, additional spatial information 21 9 and visual prompts [22 I9 can effectively improve their spatial reasoning ability. Although MILMs do not seem to have acquired an explicit understanding of fundamental 3D mapping relationships, they demonstrate sensitivity to simple prompts that implicitly relate to these spatial concepts. We believe that the visual-spatial intelligence [45] of MLLMs has not been fully exploited due to inappropriate data utilization. Exploring how to effectively leverage existing data teach MILMMs to reason and solve problems in 3D space represents highly valuable research direction.
Previous work 45
In response this challenge, we focus on a fundamental yet essential task, Viewpoint Learning, aimed at evaluating and activating the spatial reasoning ability in MLLM. Identifying viewpoints in image pairs or videos constitutes pivotal step towards achieving an understanding of 3D consistency, This task is advantageous due to its straightforward data acquisition process, ease of ground-trui calculation, and simple evaluation metrics. Capitalizing on these benefits, we introduce the Viewpoint- 100K dataset, which comprises 100K real-world, object-centric image pairs captured from distinct viewpoints, each paired with ego-centric or object-centric question-answe pairs (QAs).
To effectively activate the spatial reasoning ability of MI.LMs, we propose a two-stage fine-tuning strategy. The first stage dedicated to the injection of foundational knowledge, emphasizing the critical importance of viewpoint understanding in both video comprehension and spatial reasoning. use Supervised Pine-Tuning (SFT) with our Viewpoint- 100K dataset, which
For this purpose, ensures that the model develops correct understanding of spatial relationships and viewpoint transformations. To maintain the coherent reasoning process and iestruction-followeee behavior, we additionally employ a hybrid cold-start initialization enhanced by suman-assisted pseudo chain-of- thoughts (CoTs). In the second stage, our aim to preserve the acquired viewpoint- relatedd mowledge while simultaneously improving the model's generalization capacity across broader spatial tasks. We apply Reinforcement Learning (RI.) through the Group Relative Policy Optimization (GRPO) algorithm [32), further fine-tuning the model the SAT dataset [30), synthetic dataset for spatial

---

aptitude training. This phase is designed to refine the model's ability to transfer knowledge from basic viewpoint tasks to more abstract and complex spatial reasoning challenges. It enables models to better perceive, interpret, and reason about 3D) space, which are critical skills for deployment in real-world applications requiring advanced spatial reasoning ability.
Our experiments across multiple benchmarks demonstrate the effectiveness of Viewpoint Learning in activating the spatial reasoning ability in MLLMs. In particular, this enhancement extends to out-of-domain inference tasks, showcasing the versatility and robustness of models trained with our approach. As MLLMs continue to evolve, fundamental tasks like Viewpoint Learning will play crucial role in advancing their ubility to understand and interact with the world in three dimensions, paving the way for more sophisticated applications in autonomous systems, robotics, and beyond.
In summary, the main contributions of this work are threefold.
We introduce viewpoint learning, by which can activate the spatial reasoning ability in MLLMs, leading to strong out-of-domain generalization capabilities in visual and spatial reasoning.
We propose the Viewpoint- 100K dataset, including 100K setooaeeeeatedd ego-centric or object- centric QAs based on real-world, object-centric image pairs.
We employ a two-stage fine-tuning strategy that involves foundational knowledge injection and generalization enhancement, aiming to effectively achieve viewpoint learning. We further present a hybrid cold-start initialization method to maintain the reasoning thinking.
2 Related Work
2.1 Multimodal Large Language Models for Spatial Reasoning.
MLLMs
28 36 39
have demonstrated exceptional capability across various tasks.
Recently, several studies 21 14 27
33
34
29
30
49
15
have been devoted to applying
MLLMs the field of 3D Reasoning.
21 introduces coarse, object-level correspondences into the input images, enhancing the spatio- temporal reasoning capabilities of MILMSs without the need for fine-tuning. This work reveals that current MILMM struggle to capture cross-view consistency but that this limitation can be mitigated by providing additional cross-view information. Several recent studies [14 9 27 have similarly advanced the integration of MLLMs with 3D environments by incorporating rich 3D inputs and features, further demonstrating the potential of grounding language models in spatial contexts. Spatial- MLLM (41) aims to improve spatial understanding through high-quality, diverse multi-task data and additional 3D features introduced via a VGGT 38 backbone. MLLM-for3D [15 treats 3D consistency as an external prior to align 2D reasoning results across views in 3D space and gains significant improvements in spatial tasks. In related direction, SpatialMM (33 shows that the inclusion of bounding boxes and scene graphs significantly improves spatial reasoning performance, particularly for tasks that involve fewer reasoning steps.
Based on these findings, recent studies have increasingly emphasized the need to develop targeted training strategies that address specific deficiencies in visual-spatial reasoning. However, many existing work focus on high-level reasoning capabilities, they often overlook the importance of foundational tasks for spatial reasoning such as viewpoint estimation and spatial transformation.
2.2 Benchmarks for Spatial Reasoning
Benchmarks play a crucial role in evaluating and advancing models' capabilitics in spatial reasoning.
Various datasets [10 48
23
30
45
18
34
12
20
26
44
35 have been developed to assess
different aspects of spatial understanding, such as identifying reference frames 48 25 handling
multi-view data 30 45,, and interpreting complex scene graphs B34 26].
BLINK [2] and 3DSRBench 255 focus on various perception and reasoning tasks. SAT [30 addresses spatial reasoning through procedurally generated, multi-task dataset built on synthetic data. It also identifies the limitations of MILMM handling camera movement and out-of-domain relations. VSR 23 highlights the importance of identifying reference frames in spatial reasoning, showing that this capability significantly enhances the aceuracy and contextual awareness of 3D environment interpretations. COMFORT [48) further explores how MLLMs respond to different
3

---

Projection
3D Consistence Loop Closure
Inversed
3D Space
Matched
2D Timeline
96:18
20 Continuity without 30 Consistency
66:28
Mismatched
Figure 2: 2D Continuity and 3D Consistency. 2D continuity refers to the high similarity between adjacent frames, whereas 3D consistency focuses on preserving spatial and geometric relationships across frames. Top: Verifying 3D consistency requires estimating the camera pose and comparing these spatial properties in 3D space. Bottom: Adjusting the scale of each video frame slightly can destroy 3D consistency while maintaining 2D continuity.
frames of reference, revealing their sensitivity to such variations and tendency to favor English- specific conventions when resolving spatial ambiguities. VSI-Bench [45) presents eight tasks in three categories to assess visual-spatial intelligence. It indicates that most errors stem from spatial reasoning challenges, particularly relational reasoning mistakes and difficulties in transforming between egocentric and allocentric perspectives. The study also finds that linguistic prompting techniques can be detrimental to spatial reasoning performance.
Together, these works highlight the strong potential of MLLMs in understanding and reasoning about visual- spatial information. In this paper, we aim investigate how such spatial reasoning ability can be effectively activated and leveraged in MILMS for complex visual and spatial reasoning tasks.
3 Overview
For the successful execution of spatial tasks using 2D data representations such images and videos, is essential to recognize and leverage the inherent 3D consistency of objects. While 2D continuity in videos focuses seamless transitions between frames through subtle changes that ensure smooth visual experience, true 3D consistency requires preserving spatial integrity and geometric relationships across frames, including depth, scale, and object positions. This makes 3D consistency inherently more complex than 2D continuity. Although 3D consistency can be maintained after projecting 2D planes, achieving 2D continuity alone does guarantee 3D consistency (Figure. 2b. This distinction is crucial for tasks such as 3D reconstruction, SI.MM, and pose estimation.
In recent progress in video generation, models 24 3 16 still struggle replicate 3D properties in the real world, such as perspective consistency and vanishing points. Interestingly, humans, despite living in 3D world, often fail to detect such inconsistencies in 2D sequences, revealing the subtlety of spatial perception. MLLMs, typically trained on 2D data and constrained by low frame rates due to memory limits, face significant challenges in achieving reliable 3D reasoning without explicit spatial supervision or enriched multimodal inputs.
We argue that activating MILMSS spatial reasoning ability hinges correcting their conceptual understanding of visual input, specifically the ability to recognize and leverage the consistency of objects. Images and videos should not be seen merely sequences of changing pixels but as

---

Viewpoint-
Human
Generation
foll
rettte argle
Freet
relttienty
desteette Tmpllat
med Actteeet traees reterncc
nut Targe adgnts Unaet, OPNDEE dhiigs sack
Modify
3 sital beryyis
5
Ceetten
Seatnt
The correct BEAMET Sh..e
Canuer Qeeriee
Casrse Template
Pseudo CoTs
Gentad
Ego-centric translation centric Translation Ego-centric Rotation
Knowledge Injection
Generalization Enhancement
GAPO
The object cuptured
Qven-VL
Gatavet
cenere slject'ss perspectine, did the posal Lion the camera that
Systum Frongt:
Virspoint-
move relativv
firee
Respercc
phote's 's camona sosuliion cecentatioon
98%
18%
Neve/Ratatee to
modion Euges
B- to novemett/tstee /tstattoo
Novestutate left
Hybrid Cold-Start Initialization
Figure 3: Overview of our pipeline. We introduce Actial, which comprises a novel dataset and a two-stage fine-tuning strategy. In the knowledge injection phase, we employ a hybrid cold-start initialization to enhance the model's foundational spatial skills and leverage pseudo Co'Ts to ensure robust reasoning capabilities. Subsequently, we enhance the model's s generalization capabilities through a specialized generalization enhancement stage.
continuous projections of 3D space onto 2D plane. Recognizing that 2D continuity is supported by 3D consistency enables models to better interpret and reason about spatial structure.
Method
Given the strong generalization capabilities of MILMs, the key challenge is enabling them grasp the structure of 3D space. Our goal is to make these models realize that multi-view images and videos are not merely sequences of 2D representations but rather projections of 3D-consistent objects onto 2D plane. As shown in Figure. 3 we introduce Viewpoint Learning in the Section. 4.1 Foundational Knowledge Injection in the Section. 4.2 Hybrid Cold-Start Initialization in the Section. 4.3 and Generalization Enhancement in the Section. 4.4
4.1 Viewpoint Learning
To teach MLLMs how to handle 3D visual tasks, is essential to help them perceive 3D consistency. Specifically, 3D consistency ensures that objects in 3D space maintain 2D continuity when projected onto 2D plane. This property enables us to perform 3D reasoning based on the correlations among 2D images captured from different viewpoints. To recognize this consistency, MLLMs must first understand the concept of viewpoints.
hard to ask MILMs to directly regress accurate camera poses from multi-view images. To make the question easier for the MLLMs, we decide simplify the problem. Considering that the movement of the camera in space can be decomposed into two stages: translation and rotation. We will separate these two problems and abstract them into simpler multiple-choice questions, rather than precise regression problems.
Question Setting. The challenges of top-down and left-right relative positioning are fundamentally similar. As most images in MVImgNet are captured through horizontal loop shooting, we limit our question generation to horizontal transformations (horizontal translation and rotation).
Inspired by 18 48, which propose the importance of frames of reference (FoR). We generate mainly three types of question. They are ego-centric camera translation and rotation centered around the
5

---

camera's perspective and object-centric camera translation centered around the object's perspective. Examine the two types of thinking. translation and rotation, of MLLMs in viewpoint cognition, as well as their spatial perception ability in two reference frames (ego-centric and object-centric).
Data Generation. We automatically generate Viewpoint-100K from MVImgNet [47] including 100K object-centric image pairs and the corresponding QAs. MVIngNet is large-scale multi-view image dataset containing approximately 6.5 million real-captured frames, along with precise camera calibrations. For each subject, the dataset provides a comprehensive set of object-centric images, including corresponding object masks, camera intrinsic and extrinsic parameters, depth maps, and point clouds.
For each sample in the Viewpoint- 100K Mook dataset, generate image pairs by randomly selecting two images of the same subject with different viewpoints from MVImgNet. We ensure that the horizontal angle between the camera viewpoints is between 20 and 100 degrees. Using the provided camera parameters, we compute the relative translation and rotation between the two views. As specified in the problem setup, we only consider the camera translation along the horizontal axis and its rotation around its own vertical axis. The final dataset encompasses a total of 10,813 distinct objects, which belong to 205 different object classes.
4.2 Foundational Knowledge Injection
When evaluating MLLMs on the Viewpoint- dataset, we observed that baseline models primarily depend on 2D visual cues for viewpoint-related tasks (shown in the Figure. 4. resulting in accuracy levels near random guessing. This finding indicates that these models do not take advantage of 31) consistency for spatial reasoning across multiple views. Our goal is to enhance these models so that they can effectively utilize 3D spatial features rather than rely solely superficial 2D features. This shift from 2D cues to 3D consistency is essential for achieving robust performance in complex, multi-view spatial reasoning tasks.
Inspired by prior research [3 32], we initially employed reward- fine-tuning using the Group Relative Policy Optimization (GRPO) algorithm to guide the model towards more sophisticated 3D spatial reasoning. Despite these efforts, we observed consistently high KL divergence during training. indicating significant departure from the initial policy and suggesting an entrenehed bias towards 2D reasoning inherent in the pre-trained models. This outcome reveals that straightforward reinforcement learning strategies are inadequate for overcoming the strong inductive biases acquired during large-scale pre-training. This emphasizes the necessity for adopting more targeted methods to foster effective 3D spatial reasoning in MLLMs.
We find that directly applying Supervised Fine-Tuning (SFT) on the Viewpoint- 100K dataset leads to substantial improvement in the model's spatial reasoning ability. Since viewpoint understanding is a core aspect of 3D perception and a direct indicator of spatial reasoning ability, explicit supervision on this task helps mitigate the 2D-centric biases acquired during pre-training. Our experiments demonstrate that SFT-based training on Viewpoint- 1OOK 100K not only strengthens the model's spatial reasoning capabilities but also reduces its dependence on superficial 2D visual cues, fostering more structured and accurate understanding of 3D spatial relationships.
4.3 Hybrid Cold-Start Initialization
We note that the instruction- instruction-oollowingg capability and the thinking process of the model is affected after SFT. Drawing inspiration from [33, we propose to address this issue by introducing the cold- start initialization. To protect the integrity of the injected knowledge, we present to use the hybrid cold-start initialization. This means combining the CoT templates with the Viewpoint-1008 dataset as the imput for SFT, make the model can simultancously learn viewpoint representations and maintain coherent reasoning thinking.
Directly having MLLMs generate CoT based on the problem often leads to incorrect thinking patterns. As an alternative, we manually constructed correct CoT template based on MLLM's raw output. Then we provide the template, images, questions, and corresponding correct answers to MILM, allowing it to analyze the images based on the template and attempt to answer the questions with reference to the correct answers we provide. Specifically, we use Gemini 2.5 Pro Rd to generate IK pscudo CoTs upon Viewpoint- 100K. It should be noted that there are still some analyses in the

---

diser the th sapeiclannniingee shere the peleary abjects cantared resain statioeary bath tromes, The casea cxttrinn
Ieturnine terimntal tramslation the second rulative firat twap..
for lest
right
'nn for terizatal rovement
2. Determine the retation of the second cenara around ita own verticed anda rolatios to the firat comera's orientation:
postitiue valuee catic righnars retatton.
& megaties valle indicatas Jeffourdd recesion,
represents
snglis
El -te, Provide your ansver in the formet: directire, ietter examle:: T.A or n.e
GPT-04-rdi 1i--o--dni-high
Qwen2. VL-VV--30 Instruct
Genini 2.5 2.5-Pro
IntereVi3 llatest
1,
1,
second image shews shire conere's stxittin, sagle
the rirst Inage, the handey
tirst Inege, the
the tirst tmage, the beg
pestitcred mere Louarts the right
contered, pertezs allghtiy
resitionsd
secend laogt, the seed right aecond camera has novel to the lart
ordestation colantse The sickgres right, 2a perticalar, the cheis on
cids
cooud
apprare
ctare
seagy, hoodiigg appears
the tog apasord tiee tinards
tax aoad the seft recativn
dts pouition in the fürst Image.
voriserral tranclatter to the right.
camtering the creast laage
corer eigat in langy towe,
rotaties, bast
aoved left ralatiis the posittin first camera.
ordented the racing the
parapective inages,
The stare and souss fabris
canera, secced lmag,,
of canero amoue its wertical
seagms the seowel jhage, the
The tirst Lmage somes the hancheg
Ireicatirg Lettward rotation
the Juapes in 5e noripostal
cenerrstatieee.... the retation
cods, the sncord laage shows the recation of the bag suggrets the
tramilation Hoody noval aligptlyy
returden,
intimatt veald
Jert snd,
had to The sest match option (x25).
to right but wistest algror sigrrrcaan. ratartes.
degrwa coattar-aaodda..
around 2ts vertdced left.
Figure 4: Thoughts on viewpoint question. Current MILMS tend to rely on 2D cues to address viewpoint-related problems, which often leads to incorrect reasoning and erroneous results.
pseudo CoTs that contradict correct 3D knowledge or do not match the input image. Therefore, we maintain relatively small proportion of pseudo CoTs in the hybrid cold-start initialization to avoid disrupting the model's learning of accurate viewpoint knowledge.
4.4 Generalization Enhancement
Given the critical role of viewpoint tasks achieving cross-view consistency, we observed significant performance improvements across multiple benchmarks after incorporating foundational knowledge. However, focusing solely on a single viewpoint task during fine-tuning risks overfitting, which can limit the model's effectiveness in broader perception and inference tasks. To cover this issue, propose a second phase of fine-tuning on a more diverse dataset. This additional phase aims to broaden the model's 's capabilities, ensuring scenarios. By extending fine-tuning to a richer and more varied dataset, we expect to achieve even greater performance enhancements and improve the model's adaptability for complex real-world applications.
perform robustly across wider range of tasks and
We select the SAT dataset (30] for our generalization enhancement phase. SAT is a synthetic spatial aptitude training dataset designed to evaluate both static and dynamic spatial reasoning. We employ Reinforcement Learning, specifically the Group Relative Policy Optimization (GRPO) algorithm. Reward-basce optimization encourages the model to generate its own reasoning chains and apply the spatial knowledge acquired earlier, fostering deeper and more flexible understanding. By avoiding direct supervision on intermediate reasoning steps, our approach ensures that the model retains its previous knowledge about viewpoints and effectively leverages newly acquired spatial reasoning capabilities. This enhancement significantly improves the model's adaptability and robustness, facilitating superior performance on both in-domain and out-of-domain tasks. Furthermore, it enables the model to effectively accommodate a broader range of datasets.
Experiments
5.1
Datasets and Details
Training datasets. Actial performs the two-stage fine-tuning strategy. We use Viewpoint-1000 100K training set for knowledge injection and SAT training set [30] for generalization enhancement.
7

---

Table 1: Evaluation 3DSRBench [25], CV-Bench [37], and BLINK [12]. We color the best second-best and third-best results. We also color the better ablation results. t indicates improve- ment over the baseline. K.I. means knowledge injection (SFT phase), G.E. means generalization enhancement (GRPO phase).
CV-Rerdh
BLINK
Avg.
Malt.
Aug.
Dop.
Depth
Avg.
MuliView
ReIDop
SpRte
Chonce
Chasae (Ramban)
50.0
Proprintory
GPT to
44.6
51.6
00.1
21.4
40.2
19.4
85.7
66.9
78.2
97.8
62.7
55.6
59.7
72.7
Gemmi- 1.5 Hash
71.6
78.9
05.0
08.3
75.3
59.0
51.1
62.9
62.9
Genini 1.5
65
43.3
59.2
70.2
70.6
32.2
41.5
52.4
45.5
70.5
39.7
14..8
71.1
40.6
84.77
Open-Sturre Mideet
Robep-inn-196
69.7
79.4
$3.6
71.3
74.7
53.4
48.1
51.6
75.5
LLAVA-l..S 5-78
38.1
99.1
46.9
26.7
54.7
LLVAAvv. 5-1.--136 (+SAT)
899
61.5
73.0
80.7
64.5
44.4
76.6
72.7
(1SAT)
78.7
88.22
62.6
65.1
73.4
Canleian-88
42.2
53.
Romelite
Qwen-2 -h-l-mn-
45.8
42.8
59.3
39.3
58.8
71.2
79.3
56.9
79.6
69.1
77.44
53.3
76.2
88.8
Artial-78 (Oun)
47.7
46.41
60.31
35.5
43.0
83.5
19122
66.77
165.6
88.51
87.6
99.2
79.0
84.6
47.6
64.1
74.9
83.8
885.
466
63.3
27.9
41.2
73.1
59.0
79.0
Evaluation benchmarks. We use 3DSRBench [25], CV-Bench 377 and BLINK [12 to evaluate the model's abilities for spatial reasoning.
[ as our baseline model. In the SFT phase, we trained for epochs with a learning rate of 5e-6, a batch size of 128 and 50 warm-up steps. We mix the Viewpoint- 100K dataset and pseudo CoT data as inputs. The interleave ratio is set to 0.9:0.1.
Training details. We use Diven2.5-VL-7B-Insiiii
In the GRPO phase, trained for 150 steps with learning rate of 1e-6 and batch size of 1024. The model is trained from post-SFT model with an 4K token generation limit, sampling 16 samples per input. During training, set the Kullback-Leibbe
(KL.) penalty [32 31 to 0.2 and 1e-2 for the hyper-parameters and B, respectively. Within the reward function, the format reward and the result reward are each assigned a score of 0.5.
5.2 Evaluations
We evaluate Actial across multiple benchmarks using VLMEvalKit 11 The performance results of
various MILMS 3DSRBench primarily sourced from its paper 25 We then post-process
the results of 3DSRBench using the official script (25). Additionally. the performances of MLLMs on CV-Bench and BLINK are primarily derived from [30]. The average score on CV-Bench is recalculated following 3
The results are shown in the Table.
On the 3DSRBench, both knowledge injection and generalization enhancement lead to performance gains in the model. Our ultimate model showcases a harmonious balance of the various improvements, leading to optimize overall performance. Despite improvements, Actial remains slightly behind the previous state-of-the-ard models on this benchmark, limited by the baseline method's performance.
While on CV-Bench, Actial achieves substantial performance gain over the baseline and outper- forms existing proprictary models. The improvements highlight the effectiveness of our proposed approach to activate the spatial reasoning capabilities of MLLMs, leading more accurate and robust performance on visual-spatial tasks. This outcome underscores the importance of structured knowledge injection and targeted training strategies in advancing the visual reasoning abilities of large-scale models.
We mainly report the spatial tasks in BLINK following [0 49 The results of the multi-view component (which similar to the subtask of our Viewpoint- 100K) demonstrate that mastering foundational viewpoints is relatively straightforward for MILMMs. It provides evidence of the capability of MILMMs to perceive and reason about spatial information, highlighting their potential for advanced visual-spatial tasks. However, prior to specific activation and fine-tuning. existing

---

dbject closer canera taking this photo, the door (highlligteed by red box) Law (highlighted by blue box)?
Actial (Ours)
Buy, bew the mussing pricest for pastian
base too dijects tighis high right
laage a car nnd bay and Lapp tie thee boo). The dea the zart side
the peats
takin frem starding pastten,, leecing slaghity taea wis the right side of tw ross where the doon is Socated,
stightty
Gerinni-2 2,5
5.5-Pr
3. recopdee the appenent stepphe
dlatiace af bis okjects froe the
lette sipae und
perspective lingrty
the dean is sightagiied by
prrpectiio, Somp, high, does not yuck strong person distortten rppurent stee
Lise recating
doar (lase iapliac heriaancal Line dean (rans) saae coneerge touar da "tan side
et: trage, The further culling,
aay
C. "Maner pueition (Geage frans)n:
will,
closer.
sogpecting 2t's
7, enyee the "losart the centet phote's Frana, claser. "Clase""
Lasp
is chece the camnn
cleowr The progpe daase's clartdyy sut aleasl lator pretatio "appaars coosmi draka oftan ampllis pasittio.
Figure 5: The reasoning process. Actial uses the correct spatial thinking approach.
large models perform at level comparable to random guessing. This indicates the importance of developing foundational spatial skills for spatial reasoning.
We aim to demonstrate three key points through our experiments and address the questions raised at the beginning of this paper. First, current MILMs have not yet fully mastered certain foundational spatial skills, such as viewpoint understanding. The random performance on BLINK's multi-view task can evident this. Second, despite being trained on large-scale 2D data, these models possess significant potential for learning 3D spatial perception. This is supported by our strong performance on the viewpoint task, achieving a score of 99.2. Third, explicitly training MLLMs on basic spatial skills can effectively enhance their spatial reasoning capabilities, leading to improved performance across a variety of tasks. Our improvement compared to baseline multiple tasks is consistent with this point. Collectively, these findings highlight the critical importance of developing foundational spatial abilities in MILMSs as a necessary step toward enabling them to tackle more complex and nuanced visual reasoning tasks.
5.3 Ablation Studies
Knowledge Injection (SFT phase). The experimental results across most tasks indicate that knowl- edge injection effectively improves model performance, highlighting its beneficial impact on the learning process. Since SFT is task-specific and uses limited data diversity, it typically harms per- formance on tasks outside the fine-tuning distribution. This explains the occasional instances of marginally lower performance (e.g., the Orient. in 3DSRBench and the RelDep in Blink) compared to the ablation model. However, we were pleasantly surprised to find that fine-tuning on viewpoint tasks led to performance improvements on out-of-domain tasks, such as Height, Depth, and Relation. This highlights the importance of viewpoint learning for enhancing the model's overall spatial ability.
Generalization Enhancement (GRPO phase). The evaluation across various benchmarks demon- strates that although our knowledge injection approach enhances model performance on specific tasks, the homogeneity of these tasks and the training methods can result in performance degradation on others. The subsequent generalization enhancement phase not only retains the improvements achieved through knowledge injection but also effectively addresses the observed performance de- clines. Moreover, this phase facilitates additional performance gains on tasks where the model initially demonstrated strong capabilities, thereby achieving even greater improvement. However, we also observed that generalization enhancement can lead a decrease in metrics for some tasks compared to the knowledge injection stage. For instance, on 3DSRBench, knowledge injection improved the model's understanding of height and location, but after generalization enhancement, performance in these areas declined while improving in two other tasks. This suggests that relying solely on result-based rewards can still affect the previously injected foundational knowledge to some extent.
9

---

6 Conclusion, Limitation and Impact
This study aims to activate the spatial reasoning ability within Multimodal Large Language Models. Motivated by the need to bridge the gap between 2D visual understanding and robust 3D spatial reasoning, we introduce Viewpoint Learning, a task designed to evaluate and improve MLLMS' spatial reasoning abilities. We employ two-stage fine-tuning strategy: first, SFT with hybrid cold- start initialization on Viewpoint- 100K injects foundational knowledge, followed by Reinforcement Learning using the GRPO algorithm to enhance generalization. Our results show that this approach significantly improves the model's 's performance in both in-domain and out-of-domain reasoning tasks, demonstrating a meaningful activation of its spatial reasoning ability. Although current MLLMs lack an explicit understanding of 3D geometry, our findings indicate that targeted training strategies can effectively unlock their potential for spatial reasoning. However, the scenarios and tasks included in our dataset are relatively constrained, with all data being object-centric, which simplifies the problems compared to more varied and complex settings. The tasks in our dataset primarily address the basic aspects of Viewpoint Learning, when compared with more challenging tasks such as camera pose estimation. Cultivating these foundational spatial skills is crucial for advancing MLLMs towards more complex visual tasks. This work provides a practical pathway for improving 3D perception in MLLMs, with direct applications in robotics, autonomous navigation, and 3D scene understanding.
Acknowledgments
This work is partly supported by the National Natural Science Foundation of China (62032011) and the Natural Science Foundation of Jiangsu Province (BK20211147).
References
[1] Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. Qwen2. 5-v1 technical report, arXiv preprint WcXi:2250.113923 2025.
[2] Stephen Bamaad and Martin Fischler. Computational stereo. ACM Computing Surveys (CSUR), 1982.
[3] Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram Voleti, Adam Letts, et al. Stable video diffusion: Scaling latent video diffusion models to large datasets. arXiv preprint arXiv:231..15127, 2023.
[4] Myron Z Brown, Darius Burschka, and Gregory D Hager. Advances in computational stereo. IEEE Trans. Pattern Anal. Mach. Intell. 2003.
[5] Wenxiao Cai, laroslav Ponomarenko, Jianhao Yuan, Xiaoqi Li, Wankou Yang, Hao Dong, and Bo Zhao. Spatialbot: Precise spatial understanding with vision language models, arXiv preprint arXiv:2406. 13642, 2024.
[6] Hardy Chen, Haoqin Tu, Fali Wang, Hui Liu, Xianfeng Tang, Xinya Du, Yuyin Zhou, and Cihang Xic. Sft or rl? an early investigation into training rilkike reasoning large languae models, 2025.
[7] Zhe Chen, Weiyun Wang, Yue Cao. Yangzhou Liu, Zhangwei Gao, Erfei Cui, Jinguo Zhu, Shenglong Ye. Hao Tian, Zhaoyang Liu, al. Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling. arXiv preprint arXiv:2412.05271, 2024.
[8] The Chen, Jiannan WU, Wenhai Wang, Weijie Su, Guo Chen, Sen Xing, Muyan Zhong, Qinglong Zhang, Xizhou Zhu, Lewei Lu, et al. Intervll: Scaling up vision foundation models and aligning for generic visual-linguiste tasks. In IEEE Conf. Comput. Vis. Pattern RecoR, 2024.
[9] Erik Daxberger, Nina Wenzel, David Griffiths, Haiming Gang, Justin Lazarow, Gefen Kohavi, Kai Kang. Marcin Eichner, Yinfei Yang. Afshin Dehghan, et al. Mm-spatial: Exploring 3d spatial understanding in multimodal Ilms. arXiv preprint wxXiv:2503.13111 2025.
[10] Mengfei Du, Binhao Wu, Zejun Li, Xuanjing Huang, and Zhongyu Wei. Embspatial-hench: Bench- marking spatial understanding for embodied tasks with large vision-language models. arXiv preprint arXiv:2406.05756, 2024.
[11] Haodong Duan. Junming Yang, Yuxuan Qiao, Xinyu Fang, Lin Chen, Yuan Liu, Xiaoyi Dong, Yuhang Zang, Pan Zhang. Jiaqi Wang. et al. Vlmevalkit: An open-source toolkit for evaluating large multi-modality models. In Proceedings of the 32nd ACM International Conference on Multimedia, pages 11198-11200 2024.
10

---

[12] Xingyu Fu, Yushi Hu, Bangzheng Li. Yu Feng, Haoyu Wang, Xudong Lin, Dan Roth. Noah A Smith, Wei-Chiu Ma, and Ranjay Krishna. Blink: Multimodal large language models can but not perceive. In Eur. Conf. Comput. (is.(ECCV), 2024.
[13] Daya Guo, Dejian Yang. Haowei Zhang, Junxiao Song. Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, al. Deepseek-rl: Incentivizing reasoning capability in lims via reinforcement learning. arXiv preprint Arviv:2501. 2244, 2025.
[14] Yining Hong. Haoyu Zhen, Peihao Chen, Shuhong Zheng. Yilun Du, Zhenfang Chen, and Chuang Gan. 3d--Im: Injecting the 3d world into large language models. Adv. Neural Inform. Process. (NIP,) 2023.
[15] Jiaxin Huang, Runnan Chen, Ziwen Li. Zhengging Gao, Xiao He, Yandong Guo, Mingming Gong, and Tongliang Liu. Milm-for3d: Adapting multimodal large language model for 3d reasoning segmentation. arXiv preprint 351i:22033.813355 2025.
[16] Yuzhou Huang. Ziyang Yuan, Quande Liu, Qiulin Wang, Xintao Wang, Ruimao Zhang, Pengfei Wan. Di Zhang. and Kun Gai. Conceptmaster: Multi-concept video customization on diffusion transformer models without test-time tuning, arXiv preprint 12Xiv:2501.04699 2025.
[17] Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card. arXiv preprint arXin:2410.21276, 2024.
[18] Amita Kamath, Jack Hessel, and Kai-Wei Chang. What's "up" with vision-language models? investigating their struggle with spatial reasoning. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2023.
[19] Xuanyu Lei. Zonghan Yang, Xinrui Chen, Peng Li, and Yang Liu. Scaffolding coordinates to promote vision-language coordination in large multi-modal models. In Proceedings of the 31st International Conference on Computational Linguistics, 2025.
[20] Xiongkun Linghu, Jiangyong Huang, Xuesong Niu, Xiaojian Shawn Ma, Baoxiong Jia, and Siyuan Huang. Multi-modal situated reasoning in 3d scenes. Adv. Neural Inform. Process. (NIPS) 2024.
[21] Benlin Liu, Yuhao Dong, Yiqin Wang. Yongming Rao, Yansong Tang, Wei-Chiu Ma, and Ranjay Krishna. Coarse correspondence elicit 3d spacetime understanding multimodal language model. arXiv preprint arXiv:2408. 00754, 2024.
[22] Dingning Liu, Xiaomeng Dong, Renrui Zhang, Xu Luo, Peng Gao, Xiaoshui Huang, Yongshun Gong, and Zhihui Wang. 3daxiesprompis: Unleashing the 3d spatial task capabilities gpt-4v. arXiv preprint arXiv:2312.099 2.00788, 2023.
[23] Fangyu Liu, Guy Emerson, and Nigel Collier. Visual spatial reasoning. Transactions of the Association for Computational Linguistics, 2023.
[24] Yixin Liu, Kai Zhang, Yuan Li, Zhiling Yan, Chujie Gao, Ruoxi Chen, Zhengqing Yuan, Yue Huang. Hanchi Sun, Jianfeng Gao, et al. Sora: review on background, technology. limitations, and opportunities of large vision models. arXiv preprint arXiv:2402. 17177, 2024.
[25] Wufei Ma, Haoyu Chen, Guofeng Zhang, Celso de Melo, Jieneng Chen, and Alan Yuille. 3dsrbench: comprehensive 3d spatial reasoning benchmark. arXiv preprint arXiv:2412.07825,, 2024.
[26] Arjun Majumdar, Anurag Ajay. Xiaohan Zhang, Pranav Putta, Sriram Yenamandra, Mikael Henaff, Sneha Silwal, Paul Mevay, Oleksandr Maksymets, Sergio Arnaud, et al. Openeqa: Embodied question answering in the era of foundation models. In IEEE Conf. Comput. Vis. Pattern Recog. 2024.
[27] Yunze Man, Liang- Yan Gui, and Yu-Xiong Wang. Situational awareness matters in 3d vision language reasoning. In IEEE Conf. Comput. Vis. Pattern Recog(CVVPR), 2024.
[28] Introducing openai o3 and o4-mini. Technical report, OpenAl, 2025. hanps///penat.com//ddetttttttttt o3-and-o4-mini/.
[29] Kanchana Ranasinghe, Satya Narayan Shukla, Omid Poursaced, Michael Ryoo, and Tsung- Lin. Learning to localize objects improves spatial reasoning in visual-Ilms. In IEEE Conf. Comput. Vis. Puttern Recog. 2024.
[30] Arijit Ray. Jiafei Duan, Reuben Tan, Dina Bashkirova, Rose Hendrix, Kiana Ehsani, Aniruddha Kembhavi, Bryan A Plummer, Ranjay Krishna, Kuo- Zeng, et al. Sat Spatial aptitude training for multimodal language models. arXiv preprint arXiv:2412.07755, 2024.

---

[31] John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint (XXiv:1777.03477 2017.
[32] Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang. YK Li, Y Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint (KKi:22022003300) 2024.
[33] Fatemeh Shiri, Xiao-Yu Yu Guo, Mona Far, Xin Yu, Reza Haf, and Yuan-Fang Li. An empirical analysis on spatial reasoning capabilities of large multimodal models. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2024.
[34] Chan Hee Song, Valts Blukis, Jonathan Tremblay, Stephen Tyree, Yu Su, and Stan Birchfield. RoboSpatial: Teaching spatial understanding to 2D and 3D vision- Laiiunlanguage models for robotics. In IEEE Conf. Comput. Vis. Pattern Recog.(VVPR), 2025.
[35] Kexian Tang, Junyao Gao, Yanhong Zeng, Haodong Duan, Yanan Sun, Zhening Xing, Wenran Liu, Kaifeng Lyu, and Kai Chen. Lego-puzzles, How good are milms at multi-step spatial reasoning? arXiv preprint arXiv:2503. 19990, 2025.
[36] Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean- Man--Bapistte Alayrac. Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, Katie Millican, et al. Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312. 18055, 2023.
[37] Peter Tong, Ellis Brown, Penghao Wu, Sanghyun Woo, Adithya Jairam Vedagiri IYER, Sai Charitha Akula. Shusheng Yang. Jihan Yang, Manoj Middepogu. Ziteng Wang, et al. Cambrian-1: A fully open, vision-centric exploration of multimodal llms. Adv. Neural Inform. Process. Syst.(NIPS), 2024.
[38] Jianyuan Wang, Minghao Chen, Nikita Karaev, Andrea Vedaldi, Christian Rupprecht, and David Novotny. Vggt: Visual geometry grounded transformer. In JEEE Conf. Comput. Vis. Puttern Recog.(CVPR. 2025.
[39] Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang. Wenbin Ge, et al. Qwen2-vl: Enhancing vision- laaiuag model's perception of the world at any resolution. arXiv preprint arXiv:2409. 12191, 2024.
[40] Juyang Weng, Paul Cohen, Mare Hemniou, et al. Camera calibration with distortion models and accuracy evaluation. IEEE Trans. Pattern Anal. Mach. Intell.(PAMMI) 1992.
[41] Diankun Wu, Fangfu Liu, Yi-Hsin Hung and Yueqi Duan. Spatial-mllm: Boosting milm capabilities in visual-based spatial intelligence. arXiv preprint arXiv:2505.2374 2025.
[42] Zhiyu Wu, Xiaokang Chen, Zizheng Pan, Xingchao Liu. Wen Liu, Damai Dai, Huazuo Gao. Yiyang Ma. Chengyue Wu, Bingxuan Wang, al. Deepseek-v12: Mfxture-of-expprts vision-language models for advanced multimodal understanding, arXiv preprini urXiv:2412.10300 2024.
[43] Shaoyuan Xie, Lingdong Kong, Yuhao Dong, Chonghao Sima, Wenwei Zhang, Qi Alfred Chen, Ziwei Liu, and Liang Pan. Are vims ready for autonomous driving? empirical study from the reliability, data, and metric perspectives. arXiv preprint arXiv:2501.04000 04003, 2025.
[44] Weiye Xu, Jiahao Wang, Weiyun Wang. Zhe Chen, Wengang Zhou, Aijun Yang, Lewei Lu. Houqiang Li, Xiaohua Wang, Xizhou Zhu, et al. Visulogie: A benchmark for evaluating visual reasoning multi-modal large language models. arXiv preprint (xXx::2254.1577 2025.
[45] Jihan Yang, Shusheng Yang, Anjali W Gupta, Rilyn Han, Li Fei-Fei, and Saining Xie. Thinking in space: How multimodal large language models remember, and recall spaces. In IEEE Conf. Comput. Vis. Pattern Recog. 2025.
[46] Sihan Yang, Runsen Xu, Yiman Xie, Sizhe Yang, Mo Li, Jingli Lin, Chenming Zhu, Xiaochen Chen, Haodong Duan, Xiangyu Yue, et al. Mmss-bench: A benchmark for multi-image spatial intelligence. arXiv preprint Xiv::2200 23764, 2025.
[47] Xianggang Yu, Mutian Xu, Yidan Zhang, Haolin Liu, Chongjie Ye, Yushuang Wu, Zizheng Yan, Tianyou Liang, Guanying Chen, Shuguang Cui, and Xiaoguang Han. Mvimgnet: A large-scale datasct of multi-view images. In IEEE Conf. Comput. Vis. Pattern Recog. 2023.
[48] Zheyuan Zhang, Fengyuan Hu, Jayjun Lee, Freda Shi, Parisa Kordjamshidi, Joyce Chai, and Ziqiao Ma. Do vision- -ssion-languag models represent space and how? evaluating spatial frame of reference under ambiguities. In Int. Conf. Learn. Represent., 2025.
[49] Hengguang Zhou, Xirui Li, Ruochen Wang, Minhao Cheng. Tianyi Zhou, and Cho-Jui Hsieh, R1-zero's "aha moment" in visual reasoning on a 2b non-sft model, 2025.
12

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
Our contributions are claimed in abstract and introduction.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results
be expected to generalize other settings.
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
The limitations are discussed in Section.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
13

---

Justification:
This paper does not give the theoretical result and proof.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification:
We have reported the experiment details.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm. (b) the contribution is primarily a new model architecture, the paper should describe the architccture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reprodueibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
14

---

Answer:
[No]
Justification:
The open accesses to the data and code are in progress. But not completed yet.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.co cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https guwdes/dde/eeee Cooeeeoodeesssseeeeee 7/nips.com / ssiniiiiicypyy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Question:
Answer: [Yes]
Justification:
We have reported the experiment details.
Guidelines:
The answer NA means that the paper does include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification:
We do not display error bars in the figures and tables, but report the average
results.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the enror bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
15

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We have reported the experiment details.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't 't make it into the paper).
9. Code of ethics
Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics http:///wuaup.pccppppaa.ccccccaa
Question:
Answer: [Yes]
Justification:
The research is with the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
We have discussed the potential societal impacts in Section 6
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no socictal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
16

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of MI..
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
We do not release any data or models with high risks in this paper.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Lieenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: We have achieved these.
Guidelines:
The answer NA means that the paper does existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
17

---

If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
We do introduce new assets in our paper.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/mmoel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question:
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
18

---

Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that if the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, or originality the research, declaration is not required.
Answer: [NA]
Justification: The core method development in this research does not involve LLMs any important, original, or non-standard components.
Guidelines:
The answer NA
that the method development in this research does involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy for what should or should be described.
19

---

Appendix
We display the additional experiment results in Section. A and analyze the training process in Section.
B
We further present dataset examples and the utilized prompts in Section. [ and Section. [
respectively.
A
Additional Experiments
A.1
Results on Viewpoint-100K
The test set of Viewpoint-1000 100K consists of 1,000 examples. To assess human performance, randomly sampled 100 instances from test set and conducted a human evaluation with three annotators, resulting in an average accuracy of 97.67%. This high level of performance can be attributed to our deliberate design choices during data construction; specifically, to ensure reasonable level of task diffficulty, +100 degrees, which maintains challenging yet discriminable task for both humans and models. In comparison, the Qwen base model achieved accuracy of only 12.9%, indicating limited capability in handling the viewpoint estimation task without further training. After SFT, the model's accuracy improved significantly to 92.2%. demonstrating the effectiveness of training on labeled exemplars. However, when GRPO was applied following SFT, the performance decreased to 81.4%, suggesting that the RL objective may align well with the downstream task that the reward signal requires further refinement.
restricted the angular difference between image pairs to range of =20 to
A.2 Results on MMSI-Bench
To further illustrate the effectiveness and generalization capability of our approach on more complex tasks, we conduct additional experiments on the more comprehensive and challenging AMSS-Beenh [46]. Due to long CoT template length, set the max tokens 32K. As shown in Table. [Al Actial achieves comprehensive improvements in many subtasks when compared to the bascline model. Moreover, model, with only 7B parameters, achieves performance comparable that of larger models and GPT-40, and even outperforms them on certain tasks. The significant difference on the MSR task is due to Actial generating excessively long reasoning chains when handling multi-step inference, causing the output to be truncated before the correct answer is reached. This prevents the model from producing complete and accurate responses. Addressing this issue by optimizing reasoning efficiency and managing output length will be an important direction for future work.
A.3 Additional Ablation Study
We conducted this additional ablation experiment by mixing Viewpoint- 100K, pseudo CoTs, and SAT. We only use SFT (without GRPO) to fine-tune Qwen2.5-VL parameters in the previous experiments. We trained for total of 2,000 steps (approximately 1.5 epochs). The ablation results are presented in the bottom section of Table. A1 The significant gap demonstrates the effectiveness of our two-stage training framework and highlight the significant performance gains achieved through GRPO OOD tasks.
using the same training
Table A1: Evaluation on MMSI-Bendh We color the best and the second-best results. indicates improvement over the baseline.
MMSL-Bendi
Cam--Cun.
Obj.-Obj.
Can.OOij. bj..
Objj-Res.
Apps.
MotionCCm.
Mician:Obi
MSR
GP7-4o
34.4
24.5
23.5
19.3
37.6
27.7
32.5
31.8
35.3
36.8
30.5
30.3
Qwes2.5 VL-
25,8
34.00
34.6
23.3
34.1
36.1
65.3
27.3
27.0
30.3
27.3
30.7
20.1
Qwen2 5-Ve-7ll-nnemer
23.7
25.6
337
1223
243
18.9
23.
Actul- TiCCrrst
290
11.
28.4
41.8
283
35.7
11.3
22,7
20.01
324
20.7
25.9"
Ablation Model
19.4
25.7"
26.3"
20

---

Iospactio Lergin (4PP5)
LOON LOREROO
Parietca Lasa parts
OPRO
Satal
tor le Lob indeatin
04-
02-
(+) Response leggh tng GAPO
(b) Loss duringg on
(c)
Traiin loss duriing SFT
Figure A1: Metries changes during the training process.
B Training Analysis
Figure A1 a) presents the evolution of the model's response length throughout the GRPO phase, Our observations align with those reported in [13), where Actial displays an initial reduction followed by a subsequent adjustment in response length. When contrasted with direct fine-tuning of the baseline model, Actial demonstrates an extended reasoning length. The green line (without hybrid cold-start initialization) reflects no significant increase in response length, attributable mainly to the Supervised Fine-Tuning (SFT) stage. This stagnation can be explained by the composition of our Viewpoint- 100K dataset, which comprises exclusively multiple-choice questions devoid of reasoning templates. Consequently, the model struggles to accurately achieve format rewards during the GRPO process, and we introduce the hybrid cold-start initialization to improve such issue.
Figure Al b) depicts the evolution in KL Divergence across different variants. The green line, which represents the direct application of Viewpoint- 100K for GRPO, exhibits a growing offset relative to the initial strategy, suggesting that the baseline model's 's original spatial reasoning capabilities are inadequate for handling viewpoint-speciff tasks. Conversely, utilizing the SAT dataset leads to substantially lower KL divergence, underscoring the unique spatial reasoning demands posed by our Viewpoint-100K dataset. Following knowledge injection, the KL divergence becomes notably smoother, indicating the efficacy of integrating foundational viewpoint knowledge.
Figure AT ()) illustrates the pairwise loss observed during the Supervised Fine Fine-Tuning (SFT) training phase. Notably, when using our Viewpoint-1000 dataset, there is a sudden and significant improve- ment in performance (a trend also reflected in the validation curve). This rapid decrease excludes the possibility of the model simply memorizing the answers since it appears within one single epoch. Additionally, when using a hybrid cold-start initialization, which requires the model to learn reasoning templates, the loss curve becomes more smoother. However, the sudden insight remains clearly evident, reflecting its relevance to our newppint-basee questions. We believe that this phenomenon mainly comes from two reasons. First, our dataset consists of relatively simple multiple-choice questions with only three options (one of which a distractor), making it easy for the model to select the correct answer even without proper reasoning. However, such correctness achieved through flawed reasoning is insufficient for the model to truly understand and solve viewpoint-relas related problems, leading oscillation phase of loss fluctuation. Second, mentioned in the main text, existing MLLMs tend to rely on incorrect 2D cues when solving 3D tasks. In contrast, viewpoint problems require the model to learn how to properly utilize 3D spatial cues, resulting in period where the loss remains stagnant. Nevertheless, MLLMs do possess latent 3D perception capabilities. Once the model learns to shift its perspective appropriately (seems to be like the activation), can rapidly generalize this understanding to similar tasks, leading to sudden drop in loss.
C Dataset Examples
We show the examples of Viewpoint-1008 100K in Figure. A3 We provide three types of questions, including the horizontal translation and rotation from the camera's perspective and the horizontal translation from the object's 's perspective. We also provide the accurate rotation angles in our dataset, calculated from the camera parameters.
21

---

SaH+ EKEPRO
Gemini- 2- Pro
<thine>>
Okyy, let's and andlyre the rasera envernt haved an the two irages provided.
**1, iddntiify the Inages:**
"ixage 2: Shows an upen book placed on sooden shelf against wood-paneled wall. The left page has sininal contant,
shile the right page is partially obscured but shows som red lastern illustrations near the top, gold colored object (phens/power bank) props the book open. stan (1 W......... A1) is visible on the will to the right.
"Irage 2: Shows the some book, shelf, soll, and gold ordect, The right page i5 now fully visible, revesling a detailed illustration. The sign on the wall is nu lunger in vima, but a power outlet is now visible further to the right on the will*
**2, Compare the Images - Visual Analysis:**
"*Proe Canora's s Perspoctive
"Froxing:" in Image i, the book occuptes the centen-loft portion of the frone, The sign on the voll is clearly visible on the right, In Image 2, the book has shilited signt/icanntly to the left side of the frame. The sign is gore, and the power out et,, which rust be located turther right than the sign, is now included in the trose or the far right" "Parspective//enii ve/eglle/ In Inage 1, the canara views the book slightly froe the left. The left page is more open to the carera, while the right page is vieved at a rore ebllque ongle. Ne see the front edge and the right side (spine edge) of the book block, in Image 2, the canera viows the book wore froe the right. The right page is now nuch more open and directly facing the canero, while the left page i5 viewed nore solsqucly. The front edge and right side of the book block are seen fros different viewpoint.
"Scale/Distance"" The boot sppeans slightly larger in Taage 2 compared to Trage 1, sigpesting a potential slight farward envement of the camera.* #tackgrandd** the change in visible bariground eleants (sign dissppears, nutlet appears) confirus significant hortzontal shift in the comera's 's posstion relative to the voll.*
"eFrom Object's 's (Book's) Perspective:"*
the book's soine and slightly ly above it. In teage 3, the carera is paxi timned sigrificantly to right of the book's soine and slightly above "Visible Surfaces:* The change in shich page (left vs. right) sorw directly facing the camera intiicatos change in the camara's vissing angle relatiow the book's is trisniation."
"Corens Posirions* tmage 3, the carera past posttioned soaewbat to
**3, mnnlyoe Speet fhe Proements:::
most apparest change the shift the book left within the and the appearance backgruund eleeents (power outlet) forther to the right. This indicates the carera physically movel
"iefi//llght Translat
*Brighh** relatiww to the scene belsesn shoss."
*upp/Bonn Translation:
vertical perspictive seees very sinilar. There's strong indication sigrificant up negligible."
doun novement. We can consider
book sppears elightly larger in Imago 2, suggesting the cantra movod elightly **forsanot (closer to the book).*
"*Front/Bod Back Translation:"
slightly lowarss the right relative the baok's soine (casturing the left page more frontally). in isase 2, casera alms significantl tly towards the left frontally). To achieve this wiils translating right, the vertical axis.
*Muuaation (Yee around Lical usis):ts lmage 3, the casera
relative to the book's spine (capturing
right page rotated *lofe** (contor-clecixdee when viewed free above) oround "*Rotation (Pitch us/down =ilt):** Mindnal crange observed. Any change is likely nepligible." tilt):"* The horizental. lines detected.
canera must
"<Rotation (Ro12-
shell and wall appear consistently parellel to the
tmage botton edge, No signt/icont
Synthesize *
taking Irage Irage the carora orirarily eaved "irighe"* slightly "ooorrrd"" Strultanesusly, retated "*eett" around its vertical axts (ySN). Up/down tronslation, pitch, and vere vininal.
Perspective):*" perspective, comparing position harizantally "rrght**.
Answer Question [lorizontal
to first,
Answer Question [Horizontal Translation Ooject Perspectiive):""
factng foraar (away wall). camera, object's (book's) perspective, teagine standing signiticantly your right Trage Therefore, object
inttially
your Trage
object's perspective,
**ludjes.
Versicel The question (kotation vertical rulative ** in the first shot. favoring
the view favoring the left (rounter-elacen looking perhups degrees Extimate angle: The change perspective **lefine pages significent. Image lusking degross the right the angle relative spine
Diraction:
(Inage whill
physirally ly eaving right,
is perhaps
numall. image substantial. Estiinatigg
based V'openy" the right paga suggests rotation approxieately "554 dogreeste to the </thinko
canoserss<k/nnwee>>>
Figure A2: An example of our generated pseudo CoT.
22

---

1, cimgeeee object captured in these two photos did not move. From the camera's 's perspective, did the position of the camera that captured the second photo move relative to the first photo's camera position?
A: Move left B: Move the right C: movement
Select the correct option, please output only the option letter!
2. I.<lmae>>imageThe object captured these two photos not move. From the object's perspective, the position of the camera that captured the second photo move relative to the first photo's s camera position?
A: Move to the right B: No movement
C: Move to the left
Select the correct option, please output only the option letter!
3. b.<lmage><imae>>Th object captured in these two photos did not move. From the camera's 's perspective, did the orientation of the camera that captured the second photo rotate relative to the first photo's 's camera orientation?
A: rotation Rotate to the right Rotate to the left
Select the correct option, please output only the option letter!
Figure A3: The QA examples of Viewpoint-100K. We provide three types of questions.
D Prompts
Our system prompt is copied from VLAA 6. shown as follow:
System Prompt:
You are Actial, a helpful assistant with excellent reasoning ability.
A user asks you question, and you should try to solve it. You should first think about the reasoning process in the mind and then provides the user with the answer.
The reasoning process and answer are enclosed within <think> </think> and <answer> </answer> tags, respectively, i.e., <think> reasoning process here </think> <answer> answer here </answer>"
We show an example of generated pseudo CoT in Figure. A2 The human-assisted CoT template is as follow:
Human-Assisted CoT Template:
Based on the content presented in the two pictures, analyze the movement trajectory of the camera from shooting the first picture to shooting the second one, Describe the movement trajectory specifically, including front, back, left, right, up and down, as well as rotation and then answer the following two questions based on the camera trajectory.
1. From the camera's perspective, determine the horizontal translation of the second camera relative to the first camera.
2. From the object's 's perspective, determine the horizontal translation of the second camera
23

---

relative to the first camera.
From the camera's 's perspective, determine the rotation of the second camera around its own verticalaxis relative to the first camera's s orientation:
Estimate the rotationangle degrees.
Thinking:
I need get the relevant content answer the user's question.
Here's thinking process to analyze the camera movement between the two images:
Understand the Goal: The user wants to know how the camera moved between taking the first picture (1.jpg) and the second picture (2.jpg). The object did move any way from the first picture to the second. The description needs to cover translation (front/back, left/right, up/down) and rotation. Then, need to answer two specific questions: horizontal translation (left/right/none) and rotation angle around the camera's 's vertical axis.
1 I.dentify the Images:
Image 1.p (Let's call this 1)
Image 2: 2.j (Let's call this Pic 2)
The object both pictures a red, quilted handbag with gold chain strap, placed on blue-and-whit striped surface with yellow stars and pink crescents.
2.Compare the Images Visual Analysis:
From camera's perspective:
Frames:
In Pic 1, we see more of the surface above the bag. From a camera centered perspective, we see the left plane of the bag, but cannot the right plane of the bag.
In Pic 2, from a camera-centered perspective, the front of the bag is visible, with the golden chain strap and logo pointing toward the bottom of the image. The left side is not visible.
Perspective/Angle:
In Pic 1, from the camera's 's perspective, shows more of the front and the left side of the bag. The chain strap is clearly visible and faces towards the bottom left of the photo. In Pic 2, from the camera's 's perspective, shows more of the front side of the bag. The clasp still visible but from a slightly different angle, faces towards the bottom the photo. The top surface of the bag seems a bit more visible in Pic 2, suggesting slightly higher viewpoint or slight downward tilt.
From object's perspective:
Frames:
In Pic 1, define the side of the bag featuring the gold chain strap and logo its front. From the bag's perspective, the camera is positioned slightly to the left and above the bag, with a viewpoint angled downward and toward the left plane of the bag. In Pic 2, from the bag's 's' perspective, the camera moves to the right. It is now positioned slightly in front of and above the bag, with slight downward angle.
Perspective/Angle:
In Pic 1, from the bag's perspective, both the front and left planes are visible to the camera. In Pic from the bag's perspective, the left plane no longer visible, indicating that the camera has rotated to face more directly in front of the bag.
The bag appears roughly the same size in both images, suggesting minimal movement towards or away from the object (front/back translation).
Scale/Distance:
24

---

Background: The wall/surface behind the striped fabric looks similar in both, although the angle relative to the bag changes.
3.Analyze Specific Movements:
Left/Right Translation: If the camera undergoes only horizontal translation, the position of the bag relative to the image frame. However, this case, the camera trajectory involves rotation well. In Pic both the front and left planes of the bag visible. In Pic 2, the left plane is no longer visible, while the front plane remains in view. The change in visible surfaces indicates that the camera must have moved to the left relative to the bag.
compare
Up/Down Translation: In Pic 1, there's 's more space above the bag. In Pic 2, there's 's less space above the bag, and the camera orientation seems slightly higher relative the bag's 's top surface. This suggests the camera moved slightly
Front/Back Translation: The scale of the bag seems very similar. There's no strong indication of significant movement towards or away from the bag. Let's assume minimal or no front/back translation.
Rotation (Yaw around vertical axis): In Pic 1, the front plane of the bag faces toward the bottom-left of the photo. In Pic 2, the front plane faces toward the bottom of the photo. Given the leftward horizontal translation of the camera, to achieve this change in perspective while keeping the bag relatively stationary, the camera must have rotated rightward (clockwise when viewed from above) around its vertical axis.
Rotation (Pitch up/down tilt): The top surface seems slightly more visible in Pic 2. This could be due to the upward translation or a slight downward tilt (pitch down). Given the upward translation seems likely, slight downward tilt might also be present, but the upward movement is more apparent. Let's focus on the dominant movements.
Rotation (Roll side tilt): The horizontal lines of the striped fabric and the wall seem roughly parallel the image edges in both pictures. There's no significant roll.
4.Synthesize the Trajectory:
From Pic to Pic 2, the camera moved primarily left and slightly up. It also rotated rightward around its vertical axis (yaw). There was likely minimal front/back movement and minimal roll or pitch changes.
5.Answer Question 1 (Horizontal Translation in camera's perspective):
The camera moved horizontally relative to the bag. Did it move left or right? It moved left. The question asks for the horizontal translation of the second camera relative to the first camera. The answer is left.
6.Answer Question 2 (Horizontal Translation in object's perspective): Consider the object's perspective, the camera now moved the right. Did move left or right? It moved right.
The question asks for the horizontal translation of camera in object's perspective. The key point of the problem is transform the reference frame onto the object. The answer is left.
7. 7.Answer Question 3 (Rotation around Vertical Axis in camera's 's perspective): The camera rotated around its vertical axis relative its orientation the second shot.
Did it rotate left or right? It rotated rightward. Estimate the angle. Looking at the changes in visible parts of objects in the two photos, significant rotation occurred. Comparing the visible faces, the first photo includs the front
25

---

and left part of the bag in the camera's perspective. In Pic 2, see the front almost squarely, maybe slightly revealing the right edge. It looks like roughly 80 90-degree change in viewing angle relative to the bag.
Did it rotate the left right? Since the original left part of the bag in the first photo not be seen in the second photo, represents the camera rotates to the right, resulting in more visible part of the left side of the front of the bag.
Estimate: Let's go with approximately 87 degrees. (This is an estimation based on the visual change). The answer is the camera rotates 87 degrees to the right.
7.Format the Output: Combine the trajectory description and the answers to the specific questions clearly. Ensure the answers use the specified format.
The prompt used to generate the pseudo CoT's is as follow:
Pseudo CoT Generation:
<image> <image><<nage> <image>
I'll give you an example, please follow the example analyze the camera movement between the two images.
[Human-Assistee CoT Template]
Please analyze the given two images using the same approach as provided above. Including the Image Identify, Visual Analysis, Analyze Specific Movements, Synthesize the Trajectory, Answer Question.
Do not use the relative position changes of objects in the image, consider the changes in three-dimensional space.
The correct answer is: move to the gr(0] in cameraš perspective; move to the g in objectś perspective; rotate the gt(2) with about gr(3] degrees. Please provide sufficiently detailed analysis..
26


Reviews:
[{"review_id": "gJW5DoWerV", "reviewer": "Reviewer_AHFe", "summary": "The paper tackles the lack of reliable 3-D spatial reasoning in multimodal LLMs. It frames Viewpoint Learning and introduces Viewpoint-100K dataset. Built on Qwen-VL-7B, the proposed two-stage pipeline: 1) supervised fine-tuning on Viewpoint-100K, mixing in 10 % Gemini-generated chains-of-thought. 2) reinforcement learning via GRPO on a synthetic SAT set. The resulting Actial-7B gains 12–14%  over the baseline on several spatial benchmarks.", "questions": "1. Could similar generalization be achieved by simply adding more diverse supervised data instead of RL? An ablation contrasting these options would help isolate the benefit of GRPO.\n\n2. Please also clarify how the reward is defined in the GRPO stage: What components does it comprise (e.g., format reward and result reward) and how are they computed? \n\n3. Why not compare against a variant that directly regresses relative poses (e.g., via cosine loss) rather than multiple-choice classification?", "limitations": "No", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "[+] Well-motivated focus on 3-D consistency.\n[+]  Simple two-stage recipe with complementary gains, supported by ablations.\n[+]  Insightful error analysis illustrating shift from 2-D shortcuts to 3-D reasoning.\n\n[-]  Performance gap to proprietary SOTA remains; no significance tests.\n[-]  Lack of ablation study on the RL training. Contrast RL stage with simply scaling supervised data.", "was_revised": true, "final_justification": "The authors addressed my concerns. I keep my score", "rebuttal": "## **Q1: SFT ablation and performance against SOTA.**\n\n**Response:** Thank you for pointing out the absence of this ablation study. It helps improve the quality of our work.\n\nWe conducted this additional ablation experiment by mixing Viewpoint-100K, pseudo CoTs, and SAT. We use SFT `(without GRPO)` to fine-tune Qwen2.5-VL-7B-Instruct using the same training parameters as in the previous experiments. We trained for a total of 2,000 steps (approximately 1.5 epochs).\n\nThe evaluation is conducted on the `more comprehensive and challenging` **MMSI-Bench**[1]. Due to our long CoT template length, we set the max_tokens to 32K.\n\n\n| MMSI-Bench | Param | Cam.-Cam. | Obj.-Obj. | Reg.-Reg. | Cam.-Obj. | Obj.-Reg. | Cam.-Reg. | Meas. | Appr. | Motion-Cam. | Motion-Obj. | MSR | Avg. |\n| -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- |\n| GPT-4o | - | **34.4** | 24.5 | 23.5 | 19.8 | **37.6** | 27.7 | 32.8 | **31.8** | **35.1** | **36.8** | **30.8** | 30.3 |\n| Qwen2.5-VL-72B | 72B |25.8 | **34.0** | 34.6 | 23.3 | 34.1 | 36.1 | **45.3** | 27.3 | 27.0 | 30.3 | 27.3 | **30.7** |\n| InternVL2.5-78B | 78B | 23.7 | 22.3 | **39.5** | 29.1 | 31.8 | **42.2** | 35.9 | 19.7 | 17.6 | 26.3 | 27.3 | 28.5 |\n| Qwen2.5-VL-7B-Instruct | 7B | 23.7 | 24.5 | 19.8 | 25.6 | 32.9 | 33.7 | 42.2 | 24.2 | 18.9 | 30.3 | 23.2 | 26.5 |\n| **Actial-7B(Ours)** | 7B | 29.0 `(+5.3)` | 31.9 `(+7.4)` | 28.4 `(+8.6)` | **41.9** `(+16.3)` |28.2 (-4.7) | 33.7 | 31.3 (-10.9) | 22.7 (-1.5) | 27.0 `(+8.1)` | 32.9 `(+2.6)` | 20.7 (-2.5) | 28.9 `(+2.4)` |\n| **Ablation Model** | 7B | 19.4 `(-9.6)` | 29.8 `(-2.1)` | 27.2 `(-1.2)` | 31.4 `(-10.5)` | 30.6 (+2.4) | 34.9 (+1.2) | 29.7 `(-1.6)` | 19.7 `(-3.0)` |  25.7 `(-1.3)` |  25.0 `(-7.9)` | 26.3 (+5.6) | 27.2 `(-2.6)` |\n\n\n\n**Actial** achieves comprehensive improvements in most subtasks when compared to the **Ablation Model**. These results demonstrate the effectiveness of our two-stage training framework and highlight the significant performance gains achieved through GRPO in out-of distribution tasks.\n\n\nMoreover, our model, with `only 7B parameters`, achieves performance comparable to that of larger models and GPT-4o, and even outperforms them on certain tasks.\n\nThe significant difference on the `MSR task` is due to Actial generating excessively `long reasoning chains` when handling multi-step inference, causing the output to `be truncated` before the correct answer is reached. This prevents the model from producing complete and accurate responses. Addressing this issue by optimizing reasoning efficiency and managing output length will be an important direction for future work.\n\n[1] Yang et al. 2025. MMSI-Bench: A Benchmark for Multi-Image Spatial Intelligence\n\n## **Q2: Reward details.**\n\n**Response:** Our reward score consists of two components.\n\n- The first part is the `format reward`, which encourages the model to use the correct reasoning format (specifically, placing thoughts within `\\<think>` and `\\</think>` tags). \n- The second part is the `result reward`, which evaluates whether the model selects the correct answer from the multiple choices. We extract the final answer using regular expressions and compare it to the ground truth. \n- Both components are equally weighted, with a ratio of `1:1`.\n\n## **Q3: Pose regression.**\n\n**Response:** We are pleased that you also noticed the issue of precise pose regression using MLLMs. We explored similar ideas in our early experiments. In fact, from the initial stage to the current viewpoint learning approach presented in the paper, we `simplified the problem multiple times` to better fit the capabilities of current MLLMs.\n\n- Initially, we attempted to train the MLLM to directly regress the `relative rotation matrix` using RL, with a format reward and cosine similarity loss. However, the model failed completely to learn this precise regression task. This difficulty may stem from the inherent nature of LLMs (particularly the next-token prediction), which makes them less suited for fine-grained numerical regression. Additionally, rotation matrices are far removed from natural language semantics, which are the primary strength of LLMs.\n\n- To further probe the capabilities of MLLMs, we simplified the task by asking the model to predict relative rotation angles around a fixed axis. We tested both `direct angle regression` with error evaluation  and `multiple-choice selection`. Even with these simpler setups, the model struggled to learn effectively. This led us to question whether current MLLMs can truly develop accurate 3D spatial understanding and reasoning.\n\n- As a result, in the final version of our method, we adopted a more feasible approach using `relative directions`. Our results show that MLLMs, at least under current settings, can `handle basic 3D viewpoint tasks successfully`. And more difficult tasks like rotation and precise pose regression remain a challenging and important direction for future research."}, {"review_id": "P6y1RJzei5", "reviewer": "Reviewer_ywk7", "summary": "This paper aims to bridge the gap between 2D vision-language models (VLMs) and 3D spatial reasoning. To this end, the authors introduce viewpoint learning, a new task to instill 3D awareness through reasoning over 2D multi-view images. Regarding this task, the authors propose Viewpoint-100K, a dataset comprising 100k object-centric multi-view images with corresponding QA pairs. To learn spatial reasoning ability, this paper employs Qwen-2.5-VL-7B-Instruct as the base model, and applies SFT on Viewpoint-100K followed by RL with GRPO on SAT dataset. The trained model demonstrates significant performance improvements on several spatial reasoning benchmarks, including 3DSRBench, CV-Bench, and BLINK.", "questions": "- I cannot get what it means and what the authors want to deliver in Figure 2. What does it mean by “Matched” or “Mismatched”?\n- I would expect more results on Viewpoint-100K, including human performance and comparison between RL vs. SFT.\n- There is little discussion about the quality of the pseudo CoT data (I guess Figure 4 does not refer to the collected CoT data). And if the quality is an issue, why still using a proportion for training instead of refining them in advance?\n- Some “w/o K.L.” results in Table 1 outperform the full method. Does “w/o K.L.” mean the first SFT stage is removed? This should be clarified, and the results should be highlighted accordingly to aid interpretation.\n- What is the observed impact of hybrid cold-start initialization using CoTs? Additional analysis or ablations would be helpful.\n- Why name it Actial, any potential meaning or metaphor?", "limitations": "Yes", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "#### Strengths\n- Good motivation. 3D spatial reasoning is a critical capability to be unlocked for VLMs. And this capability is particularly insufficient for current VLMs. This paper focuses on this significant problem and proposes viewpoint learning to mitigate the gap.\n- Significant experimental results. The proposed approach greatly improves the baseline’s performance on spatial reasoning across multiple benchmarks, validating the effectiveness of the proposed dataset and viewpoint learning.\n- Good clarity. The paper is well-written and logically structured, making it easy to follow. The authors clearly articulate the motivation, methodology, and results.\n\n#### Weaknesses\n- Collecting object-centric multi-view images is a good solution to cultivate 3D spatial reasoning skills. However, I think it may not be optimal to anchor the task on discriminating how the camera translates and rotates through texts. This task is quite difficult (even for humans), and learning can be challenging for models considering the task complexity and sparse supervision (only supervising short text answers). While how to properly formulate viewpoint learning remains an open question, I think it pertains to perceptual capability that should be embedded at the representation level rather than handled by language model reasoning. As such, casting it as a language task might not be the most effective route.\n- The results and analyses on how to utilize Viewpoint-100K are not intuitive. The authors said that RL with GRPO leads to high KL divergence (i.e., a significant departure on the policy). I think this cannot support that RL is inadequate for overcoming the 2D-centric bias. And I think the improvement by applying direct SFT does not necessarily indicate a better foundamental spatial reasoning capability, considering the strong supervision on fitting the text. Anyway, this is not a convincing approach for knowledge injection from my perspective.", "was_revised": true, "final_justification": "The authors addressed my concerns well. I agree that the paper exceeds the acceptance bar.", "rebuttal": "## **Q1: Explanation about Fig. 2.**\n\n**Response:** Thanks for all of your valuable comments, we will improve the clarity of the analysis in the manuscript.\n\n`Figure 2` is provided to help illustrate the difference between `2D continuity` and `3D consistency`. As stated in the Introduction of our paper:\n\n> \"Our key concern is whether existing MLLMs, which are primarily trained on video data emphasizing 2D continuity, can achieve an understanding of spatial 3D consistency, as opposed to merely tracking continuous pixel-level evolution or correlated pixel mappings.\"\n\nIn the figure, **\"Matched\"** represents the case with true 3D consistency. Under this condition, when images are transformed into the world coordinate system using accurate camera parameters, **pixels corresponding to the same 3D point across different views are correctly projected to the same location in 3D space**. This results in perfect alignment of the reconstructed object, with no misalignment or error.\n\nIn contrast, **\"Mismatched\"** illustrates the scenario where only 2D continuity is preserved. When small, arbitrary `deformations` or `scaling` are applied to each frame, **adjacent frames appear visually smooth and continuous. However, the underlying 3D geometric consistency is broken.** In practice, such distortions resemble lens distortions, which require additional calibration to restore true 3D structure. In this case, corresponding points from different images `fail to align in 3D space`, resulting in misaligned projections or \"ghosting\" artifacts in the reconstructed object.\n\nWhen applied to MLLMs, this issue highlights **whether such models has the potential capability to map pixels carrying consistent information across different views back into the correct 3D space to perform crossing-view spatial reasoning**. And the Viewpoint Learning is the fundamental part of this issue.\n\n## **Q2: More results on Viewpoint-100K testset.**\n\n**Response:** The testset of Viewpoint-100K includes `1000 examples`.\n\n- **Human performance:** We randomly sampled `100 examples` from testset and conducted a human evaluation with three annotators, achieving an average accuracy of `97.67%`. This high performance is mainly due to our deliberate design choice during data construction. To ensure reasonable task difficulty, we limited the angular difference between image pairs to between `±20` and `±100` degrees. This range helps maintain a challenging yet discriminable level of difficulty for the task.\n\n- **Qwen base:** The accuracy is `12.9%`.\n\n- **SFT:** The accuracy is `92.2%`.\n\n- **SFT + RL:** The accuracy is `81.4%`.\n\n## **Q3: Quality of pseudo CoTs.**\n\n**Response:** Thank you for pointing out the missing discussion regarding the pseudo CoT data. The pseudo CoT examples are presented in `Figure 3 in the Appendix`. Figure 4 does not show collected CoT data; instead, it illustrates the current MLLM's performance on the viewpoint task. \n\nThe lack of genuine spatial reasoning in current models is evident from the fact that even advanced models like Gemini perform at `random-guessing levels` on the BLINK viewpoint task. **This makes it unrealistic to rely on models to automatically generate high-quality CoT data. At the same time, manually correcting CoT data is prohibitively expensive.**\n\nTo ensure the quality of the pseudo CoT data, we provide the correct answer to Gemini in advance to guarantee accuracy. We also supply a carefully curated, human-verified template that demonstrates correct spatial reasoning. Gemini then uses this answer and template as a reference to generate diverse CoT responses, effectively `imitating` the reasoning pattern. The prompt template used for generating the pseudo CoT can be found at the end of the Appendix.\n\nFurthermore, in our hybrid cold-start phase, the primary role of CoT data is to provide a formally correct reasoning template and maintain the model's instruction-following capability. This point is further discussed in our response to `Question 5`.\n\n\n\n## **Q4: Question about \"w/o K.L.\".**\n\n**Response:** `\"w/o K.L.\"` means the first SFT stage is removed. Thanks for the advise about clearer expression and visual prompts. \n\nIn the `SFT (K.L.) phase`, we fine-tuned the MLLM exclusively on viewpoint tasks. Since SFT is `task-specific` and uses limited data diversity, it typically harms performance on tasks outside the fine-tuning distribution. This explains the observed performance drop on Orient. compared to the base model.\n\nHowever, we were pleasantly surprised to find that fine-tuning on viewpoint tasks led to performance `improvements on out-of-domain tasks`, such as Height, Depth, and Relation. This highlights the importance of viewpoint learning for enhancing the model's overall spatial ability.\n\nThe fact that the \"w/o K.L.\" variant outperforms our Actial on a few tasks can be attributed to the known negative impact of standard SFT on generalization. This is precisely why we introduce a second-stage GRPO. We apply RL across a broader set of tasks to enhance the generalization. Importantly, thanks to the knowledge injected during SFT, we observe that the model shifts from primarily relying on `2D visual cues` to engaging in `more genuine spatial reasoning`. It leads that the reasoning paths of the model after knowledge injection followed by GRPO (Actial) are `fundamentally different` from those in the direct GRPO approach (w/o K.L.). \n\n\n## **Q5: Impact of hybrid cold-start initialization.**\n\n**Response:** The purpose is threefold. \n\n- First, to introduce `a structured template` for spatial reasoning. As shown in the thinking process in `Figure 5` of the paper, the model's reasoning pattern is shaped by the CoT examples from the cold-start phase. \n\n- Second, to maintain the model's `instruction-following capability`. As demonstrated in `Section 1  and Figure 1(a) of the Appendix`, the model struggles to learn the format reward without including CoT data during GRPO.\n\n- Third, since we cannot guarantee the correctness of the reasoning logic in the CoT templates, instead of using a separate cold-start phase between SFT and RL, we choose to incorporate the CoT data at a small ratio during our knowledge injection stage. This helps minimize the influence of potential template biases on the final performance.\n\n## **Q6: The reason for naming \"Actial\".**\n\n**Response:** I'm glad our naming caught your attention. And **`\"Actial\"`** is the combination of **`[Ac]`tivate Spa`[tial]`** in our title, simultaneously simple and easy to read.\n\n## **Additional comments about the RL and SFT for spatial reasoning.**\n\nWe would like to share additional observations regarding the direct application of GRPO in Viewpoint Learning. Beyond the KL divergence issues, we found that the model's performance remains at a `random guessing level` throughout training. Despite multiple attempts, the training process consistently `collapsed` after a long time of training, characterized by a sudden drop in response length, after which the model began generating `meaningless or degenerate outputs`.\n\nWe also examined the `CoT behavior` of a model trained directly on Viewpoint-100K using GRPO only. While we observed an initial `\"aha moment\"` in terms of increasing response length, further analysis revealed that the model still relied primarily on `2D visual cues` for reasoning, similar to what is shown in `Figure 4` of the main paper. This lack of genuine 3D spatial understanding was a key motivation for us to instead inject viewpoint knowledge via SFT, rather than relying solely on GRPO.\n\n## **Additional discussions about the question settings.**\n\nIn fact, from the initial stage to the current viewpoint learning approach presented in the paper, we `simplified the problem multiple times` to better fit the capabilities of current MLLMs.\n\n- Initially, we attempted to train the MLLM to directly regress the `relative rotation matrix` using RL, with a format reward and cosine similarity loss. However, the model failed completely to learn this precise regression task. This difficulty may stem from the inherent nature of LLMs (particularly the next-token prediction), which makes them less suited for fine-grained numerical regression. Additionally, rotation matrices are far removed from natural language semantics, which are the primary strength of LLMs.\n\n- To further probe the capabilities of MLLMs, we simplified the task by asking the model to predict relative rotation angles around a fixed axis. We tested both `direct angle regression` with error evaluation  and `multiple-choice selection`. Even with these simpler setups, the model struggled to learn effectively. This led us to question whether current MLLMs can truly develop accurate 3D spatial understanding and reasoning.\n\n- As a result, in the final version of our method, we adopted a more feasible approach using `relative directions`. Our results show that MLLMs, at least under current settings, can `handle basic 3D viewpoint tasks successfully`. And more difficult tasks like rotation and precise pose regression remain a challenging and important direction for future research."}, {"review_id": "e3ws6JnIti", "reviewer": "Reviewer_jwwJ", "summary": "This paper introduces Actial-7B, a MLLM that is finetuned using SFT and GRPO with Viewpoint-100K dataset. The resulting model shows better performance on spatial-reasoning benchmarks, which indicates the usefulness of the proposed dataset and training pipeline. The paper is built upon the hypothesis that understanding viewpoints is an important aspect for MLLMs to understand the 3D spatial relations, which is proved by finetuning the model using Viewpoint-100K dataset, a dataset that solely consists of viewpoint understanding.", "questions": "1. How can viewpoint learning boost the understanding of more complex problems like orientation, as suggested by the marginal improvements on 3DSRBench?\n\n2. Why does the author not consider the viewpoint changes for up and down, the resulting distribution of the object pose under this setting would be different from left-and-right. Is the image pairs in the dataset diverse enough to cover different angles of the objects?\n\n3. The dataset proposed consider a fixed object and a moving camera, can the model better understand the setting where camera is fixed and object is moving?", "limitations": "Yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n\n1. The paper proposed a useful dataset Viewpoint-100K, which can be widely used by the community to enforce better spatial relation understanding of MLLMs.\n\n2. The finetuning recipe is clear enough to be followed.\n\n3. The resulting model from the above two bullet points show decent performance improvements on CV-Bench and BLINK.\n\nWeaknesses:\n\n1. The training recipe and the dataset is only used to finetune one model, i.e.  Qwen2.5-VL-7B-Instruct, lacking a comprehensive evaluation on how the proposed method can be applied to different MLLMs.\n\n2. The model's improvements on 3DSRBench is marginal, which indicates that for complex tasks in 3DSRBench, the model needs abilities more than just viewpoint understanding, which contradicts the authors' hypothesis that viewpoint understanding is important for 3D spatial understanding.", "was_revised": true, "final_justification": "The author has addressed all my concerns.", "rebuttal": "## **Q1: Viewpoint Learning for complex tasks.**\nThanks for your valuable comments on our work. The marginal overall improvement in 3DSRBench is primarily attributed to the Orient. task, **while all other tasks demonstrate significant performance gains.**\n\n\n- **Additional experiments on complex tasks**.\n    \n    To further illustrate the effectiveness and generalization capability of our approach on more complex tasks, we conduct additional experiments on the `more comprehensive and challenging` **MMSI-Bench**[1]. \n\n    | MMSI-Bench | Param | Cam.-Cam. | Obj.-Obj. | Reg.-Reg. | Cam.-Obj. | Obj.-Reg. | Cam.-Reg. | Meas. | Appr. | Motion-Cam. | Motion-Obj. | MSR | Avg. |\n    | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- |\n    | GPT-4o | - | **34.4** | 24.5 | 23.5 | 19.8 | **37.6** | 27.7 | 32.8 | **31.8** | **35.1** | **36.8** | **30.8** | 30.3 |\n    | Qwen2.5-VL-72B | 72B |25.8 | **34.0** | 34.6 | 23.3 | 34.1 | 36.1 | **45.3** | 27.3 | 27.0 | 30.3 | 27.3 | **30.7** |\n    | InternVL2.5-78B | 78B | 23.7 | 22.3 | **39.5** | 29.1 | 31.8 | **42.2** | 35.9 | 19.7 | 17.6 | 26.3 | 27.3 | 28.5 |\n    | Qwen2.5-VL-7B-Instruct | 7B | 23.7 | 24.5 | 19.8 | 25.6 | 32.9 | 33.7 | 42.2 | 24.2 | 18.9 | 30.3 | 23.2 | 26.5 |\n    | **Actial-7B(Ours)** | 7B | 29.0 `(+5.3)` | 31.9 `(+7.4)` | 28.4 `(+8.6)` | **41.9** `(+16.3)` |28.2 (-4.7) | 33.7 | 31.3 (-10.9) | 22.7 (-1.5) | 27.0 `(+8.1)` | 32.9 `(+2.6)` | 20.7 (-2.5) | 28.9 `(+2.4)` |\n\n    Compared to the baseline model, **Actial** achieves comprehensive improvements in many subtasks. Moreover, our model, with `only 7B parameters`, achieves performance comparable to that of larger models and GPT-4o, and even outperforms them on certain tasks.\n\n[1] Yang et al. 2025. MMSI-Bench: A Benchmark for Multi-Image Spatial Intelligence\n\n- The explanation of the drop on `Orient.` task (3DSRBench).\n    - We have carefully checked the examples of Orient. task in 3DSRBench. As shown in their paper, the question forms like `'Consider the real-world 3D locations and orientations of the objects. Is the stop sign on the left or right side of the man on the bicycle?'`. And this remains a crucial problem that **the question does not specify the frames of reference (which we carefully descripted in Viewpoint-100K dataset)**.  In general, different people may adopt different default frames of reference depending on the context, so without explicitly specifying the reference frame, these questions are difficult to answer \"correctly\". As a result, the question formulations in 3DSRBench significant challenges for our model.\n    - This is also why we introduce a `second-stage GRPO`, aiming to enhance the model's generalization capability through reinforcement learning on a larger and more diverse dataset. Indeed, after RL, we observe significant performance `improvements across the majority of tasks` compared to the SFT phase (\"w/o G.E.\"), while the results on BLINK indicate that the viewpoint knowledge injected during SFT is `well preserved`. **These findings are consistent with our core argument.**\n    - However, the suboptimal performance on certain tasks may be attributed to several factors: \n        - First, SAT is a synthetic dataset, and there is a significant `domain gap` between its rendered images and real-world visuals, along with relatively low data quality.\n        - Second, there remains room for improvement in our RL training process. Due to the `strong influence of the SFT phase`, the model's performance on some tasks has not yet reached its full potential.\n        - We believe this can be addressed by further incorporating `higher-quality real-world data` to enhance the second-stage RL and improve generalization.\n\n\n\n\n## **Q2.1: Up-down viewpoint questions.**\n\n**Response:** Thank you for your insightful question. This is a very important question in 3D domain. **We treat changes in up-down viewpoint as a dual problem to left-right changes.** In 3D space, the three coordinate axes are `orthogonal` and `independent`. Therefore, if a model can distinguish left-right transformations, we expect it to have the `potential` to distinguish up-down changes as well. In fact, rotating an up-down image pair by 90 degrees makes it equivalent to a left-right pair (this equivalence arises from the orthogonality of 3D space). Importantly, directions like \"up/down\" and \"left/right\" are `relative` to the camera coordinate system, not absolute in the world coordinate system.\n\nOn a broader note, our goal in this work is to highlight the potential of classic 3D tasks (like viewpoint tasks) in evaluating and enhancing MLLMs' spatial reasoning ability, rather than simply relying on a powerful base model trained on massive and diverse data. **We hope this insight inspires future work to more fine-grained task design and make more deliberate choices in data curation.**\n\nTo further explore the ability, we create an up-down testset with `1000` examples. Each question has 3 options. So the random choice rate is `33.3%`. We test our Actial-7B and Qwen2.5-VL-7B-Instruct **without additional fine-tuning on up-down data.**\n\nAnd the accuracy of Actial-7B is `53.7%`, while the accuracy of Qwen2.5-VL-7B-Instruct is `21.2%`. Although training only on left-right data cannot perfectly solve the up-down task, the results still show a significant improvement over the base model. This suggests that future work should consider more comprehensive task designs.\n\n\n## **Q2.2: Dataset diversity.**\n\n**Response:** We selected image pairs with angular differences between `±20°` and `±100°`. Given the full possible range of `±180°` in real-world scenarios, this range was chosen to balance task difficulty. Since both very small and very large viewpoint differences can be challenging even for humans. For example, in extreme cases where the offset angle is around 180 degrees, considering `possible errors`, either choosing a left rotation or a right rotation can be theoretically acceptable.\n\n\n\nMoreover, since MVImgNet consists of real-world images without fixed angular intervals, **the selected samples are naturally distributed across the ±20°–±100° range, which enhances the diversity of the dataset.**\n\nFinally, our dataset is generated `automatically` through a pipeline built on MVImgNet. If more challenging or diverse data are needed in the future, this can be easily achieved by adjusting the sampling parameters.\n\n## **Q3: Object moving when camera is fixing.**\n\n**At first, the object moving task refers to the `Motion-Obj.` in **MMSI-Bench**, which Actial gains a `+2.6` improvement.**\n\nWe are glad that you also noticed this point, and it is indeed one of the key insights from our work. The type of 3D reasoning you describe, where spatial changes are analyzed under a fixed camera setup, is more similar to a single-image spatial understanding task. Since the change between `multiple images` are occured more likely in the `timeline`, which is different with the `multi-view` images analysis.\n\nIn the `SFT (K.L.)` phase, we fine-tuned the MLLM exclusively on multi-view viewpoint tasks. As typical with task-specific fine-tuning without diverse data, this can harm performance on out-of-domain tasks, such as the drop we observe on Orient. (3DSRBench) compared to the base model.\n\nEncouragingly, however, we find that even with fine-tuning on just this single task, the model shows `performance gains` on several out-of-domain single-image spatial reasoning tasks (the relative position questions like Height, Relation, and Depth), as seen in the improvement of \"w/o G.E.\" over the base model. **This suggests that mastering viewpoint reasoning can generalize broadly and significantly enhance the model's overall spatial reasoning ability.**\n\n## **Response to the weakness.**\n\nThe computational cost of MLLM training is extremely high, particularly for `SFT+RL pipelines`. Given that many works in the field typically validate their methods on a single representative model, we adopt the widely used Qwen2.5-VL-7B-Instruct for our experiments. We thank the reviewer for the suggestion. However, due to constraints on resources and time (`the GRPO requires weeks of time`), we are unable to conduct additional experiments on other MLLMs during the rebuttal phase."}, {"review_id": "PHWlQ0eE1V", "reviewer": "Reviewer_hxxQ", "summary": "This paper introduces a new task, namely viewpoint learning, to improve the spatial reasoning capabilities of 2D MLLMs. Specifically, this work injects spatial knowledge by finetuning the MLLM on the viewpoint learning tasks using a visual QA dataset curated for that specific task. Further, to improve the generalization capabilities, GRPO is used to train the fonetuned LLM on a broader set of questions using synthetic data.", "questions": "See weaknesses", "limitations": "yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:-\n\n- The paper is very well-written with a clear motivation.\n- This is the first work to identify and demonstrate that finetuning MLLMs on viewpoint learning significantly boosts their spatial reasoning capabilities.\n- The idea of using GRPO for enhancing generalization after finetuning is interesting.\n- Extensive experiments have been conducted to demonstrate SOTA performances on multiple benchmark datasets.\n\nWeaknesses:-\n\n- Recent works such as MLLM-for3D incorporates a similar idea of having cross-view spatial consistency using multi-view segmentation maps to enhance reasoning capabilities of 2D MLLMs. Comparison with those methods could be more helpful in assessing the efficacy of your method.\n- The method claims to enhance out-of-domain generalization but does not explicitly showcase this in the results section. A better expanation on this would be helpful.\n\nAdditional points:-\n\n- Spatial-MLLM seems to be a concurrent work that follows a very similar pipeline of Supervised fine-tuning on a curated QA dataset followed by GRPO. However, their QA pairs cover different spatial reasoning tasks such as object counting, object size, relative distance etc. Using those tasks along viewpoint consistency could be more beneficial for the initial finetuning of MLLMs.", "was_revised": true, "final_justification": "The authors have addressed my concerns and I lean towards acceptance of this paper. So I will maintain my rating", "rebuttal": "## **Q1: Comparison with MLLM-for3D and Spatial-MLLM.**\n\n**Response:** Thank you for drawing our attention to these excellent concurrent works. We will carefully compare and discuss them in the revised manuscript.\n\nAs mentioned in the `Related Work` section, both `MLLM-for3D` and `Spatial-MLLM` belong to a line of research that enhances the integration of MLLMs with 3D environments by introducing `additional 3D inputs and features`, aiming for more reliable 3D understanding and reasoning.\n\n- `MLLM-for3D` further advances this idea by `lifting` 2D segmentation features from multiple views into 3D space through a spatial consistency-guided fusion mechanism, enabling direct 3D spatial reasoning. The key difference from our work is that MLLM-for3D treats 3D consistency as an `external prior` to align 2D reasoning results across views in 3D space. In contrast, **our final goal is to investigate whether an MLLM can internalize and actively use the concept of 3D consistency within its own reasoning process, and to evaluate how such internalization affects its spatial understanding.**\n\n- Regarding `Spatial-MLLM`, there are two main distinctions:\n\n    - First, Spatial-MLLM aims to improve spatial understanding through high-quality, diverse multi-task data and `additional 3D features` introduced via a `VGGT backbone`. In contrast, our work focuses on assessing the impact of a single, carefully designed task (Viewpoint Learning) on an MLLM's 3D spatial reasoning. **Our goal is to highlight the importance of task design and data selection in activating spatial reasoning abilities, offering insights for future work.**\n\n    - Second, in Spatial-MLLM, both the SFT and GRPO stages are designed to improve performance on the Spatial-MLLM-120K benchmark. In our approach, the SFT stage is specifically used to inject viewpoint-related knowledge, while the GRPO stage aims to mitigate the potential degradation in generalization caused by task-specific fine-tuning. More importantly, it encourages the model to transfer the spatial reasoning skills (like the reasoning process shown in `Figure 5` in the manuscript) learned from the viewpoint task to OOD tasks.\n\nThese differences reflect our distinct goal: **to understand and enhance how MLLMs reason about 3D space relying solely on 2D visual inputs.**\n\n## **Q2: More explanations about the OOD generalization.**\n\n**Response:** Thank you for highlighting this part. We will carefully address and revise the manuscript accordingly.\n\nIn this paper, the generalization ability showcases on three aspects.\n\n- First, in our SFT phase, we fine-tuned the MLLM exclusively on viewpoint tasks. As typical with `task-specific fine-tuning without diverse data`, this can harm performance on out-of-domain tasks, such as the drop we observe on Orient. (3DSRBench) compared to the base model. Encouragingly, however, we find that even with fine-tuning on just this single task, the model shows `performance gains` on several spatial reasoning tasks (e.g., Height, Relation, Depth). This suggests that **mastering viewpoint reasoning can generalize broadly and significantly enhance the model's overall spatial reasoning ability.**\n\n- Second, it refers to **generalize reasoning capabilities to OOD tasks.** Since our CoT templates only include the viewpoint task. While after GRPO, the model can correctly transfer spatial reasoning to a diverse range of OOD tasks.\n\n- Finally, to further illustrate the effectiveness and generalization capability of our approach on more complex tasks, we conduct additional experiments on the `more comprehensive and challenging` **MMSI-Bench**[1]. Due to our long CoT template length, we set the max_tokens to 32K.\n\n\n    | MMSI-Bench | Param | Cam.-Cam. | Obj.-Obj. | Reg.-Reg. | Cam.-Obj. | Obj.-Reg. | Cam.-Reg. | Meas. | Appr. | Motion-Cam. | Motion-Obj. | MSR | Avg. |\n    | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- |\n    | GPT-4o | - | **34.4** | 24.5 | 23.5 | 19.8 | **37.6** | 27.7 | 32.8 | **31.8** | **35.1** | **36.8** | **30.8** | 30.3 |\n    | Qwen2.5-VL-72B | 72B |25.8 | **34.0** | 34.6 | 23.3 | 34.1 | 36.1 | **45.3** | 27.3 | 27.0 | 30.3 | 27.3 | **30.7** |\n    | InternVL2.5-78B | 78B | 23.7 | 22.3 | **39.5** | 29.1 | 31.8 | **42.2** | 35.9 | 19.7 | 17.6 | 26.3 | 27.3 | 28.5 |\n    | Qwen2.5-VL-7B-Instruct | 7B | 23.7 | 24.5 | 19.8 | 25.6 | 32.9 | 33.7 | 42.2 | 24.2 | 18.9 | 30.3 | 23.2 | 26.5 |\n    | **Actial-7B(Ours)** | 7B | 29.0 `(+5.3)` | 31.9 `(+7.4)` | 28.4 `(+8.6)` | **41.9** `(+16.3)` |28.2 (-4.7) | 33.7 | 31.3 (-10.9) | 22.7 (-1.5) | 27.0 `(+8.1)` | 32.9 `(+2.6)` | 20.7 (-2.5) | 28.9 `(+2.4)` |\n    | **Ablation Model** | 7B | 19.4 `(-9.6)` | 29.8 `(-2.1)` | 27.2 `(-1.2)` | 31.4 `(-10.5)` | 30.6 (+2.4) | 34.9 (+1.2) | 29.7 `(-1.6)` | 19.7 `(-3.0)` |  25.7 `(-1.3)` |  25.0 `(-7.9)` | 26.3 (+5.6) | 27.2 `(-2.6)` |\n\n    Compared to the baseline model, **Actial** achieves comprehensive improvements in many subtasks. Moreover, our model, with `only 7B parameters`, achieves performance comparable to that of larger models and GPT-4o, and even outperforms them on certain tasks.\n\n   The `Ablation Model` was trained in accordance with the suggestion provided by `Reviewer AHFe`. We conducted this additional ablation experiment by mixing Viewpoint-100K, pseudo CoTs, and SAT. We use SFT `(without GRPO)` to fine-tune Qwen2.5-VL-7B-Instruct using the same training parameters as in the previous experiments. We trained for a total of 2,000 steps (approximately 1.5 epochs). The significant gap demonstrates the effectiveness of our two-stage training framework and highlight the significant performance gains achieved through GRPO in OOD tasks.\n\n    And the difference on the `MSR task` is due to Actial generating excessively `long reasoning chains` when handling multi-step inference, causing the output to `be truncated` before the correct answer is reached. This prevents the model from producing complete and accurate responses. Addressing this issue by optimizing reasoning efficiency and managing output length will be an important direction for future work.\n\n[1] Yang et al. 2025. MMSI-Bench: A Benchmark for Multi-Image Spatial Intelligence"}]

Author rebuttal:

Meta-review: This paper was reviewed by four experts who all recommended acceptance of this paper.  The major contribution of this paper is a dataset to evaluate a MLLM's performance on understanding different viewpoints of the same object.  The authors also provide an approach for finetuning MLLMs on their task, which the rebuttal points out outperforms some simpler competing finetuning methods.  To help demonstrate the benefits of their dataset, the authors use a model trained on their dataset to evaluate on external spatial reasoning benchmarks.  Overall, this helps highlight how data constructed using their approach can be beneficial to a wider range of spatial reasoning datasets and tasks.  One somewhat counterintuitive result was lower performance on the Orientation questions of the 3DSRBench, which would seem closely related, but the authors argued this stemmed from the differences in the questions- where 3DSRBench did not include a reference image.  This highlights a potential area of weakness and limited impact of data trained using this dataset, calling into some question as to the true source of the gains (i.e., does the proposed dataset improve performance on these other datasets due to training on object-focused data, focusing the MLLM weights to think about the objects rather than other aspects like color, or background information).  However, these questions can easily be argued as simple conjecture, where the current dataset provides a reasonable benchmark with some demonstrated benefits.  As such, the ACs find insufficient justification for overturning the reviewer's recommendations, and the authors are encouraged to consider reviewer comments carefully when preparing their camera ready.
Decision: Accept (poster)