Fine-Grained Preference Optimization Improves Spatial Reasoning in VLMs
Authors: Yifan Shen
Yuanzhe Liu
Jingyuan Zhu
Xu Cao
Xiaofeng Zhang
Yixiao He
Wenming Ye
James Matthew Rehg
Ismini Lourentzou
Venue: NeurIPS 2025 poster ()
Keywords: Spatial Reasoning
Vision-Language Models
Fine-grained DPO
Long Chain-of-Thought
Primary area: applications

Abstract: Current Vision-Language Models (VLMs) struggle with fine-grained spatial reasoning, particularly when multi-step logic and precise spatial alignment are required. In this work, we introduce SpatialReasoner-R1, a vision-language reasoning model designed to address these limitations. To construct high-quality supervision for spatial reasoning, we design a Multi-Model Monte Carlo Tree Search (M3CTS) method that generates diverse, logically consistent Long Chain-of-Thought (LongCoT) reasoning trajectories. In addition, we propose a fine-grained Direct Preference Optimization (fDPO) method that introduces segment-specific preference granularity for descriptive grounding and logical reasoning, guided by a spatial reward mechanism that evaluates candidate responses based on visual consistency, spatial grounding, and logical coherence. Experimental results demonstrate that fDPO achieves relative performance gains of 4.1% and 9.0% over standard DPO on spatial qualitative and quantitative tasks, respectively. SpatialReasoner-R1, trained with fDPO, sets a new SoTA on SpatialRGPT-Bench, outperforming the strongest baseline by 9.4% in average accuracy, while maintaining competitive performance on general vision-language tasks.

Fine-Grained Preference Optimization Improves Spatial Reasoning in VLMs
SpatialRensonerRt
Yifan Shen', Yuanzhe Liu², Jingyuan Zhu?, Xu Cao Xiaofeng Zhang', Yixiao He!,
Wenming Ye*, James M. Rehg', Ismini Lourentzou'
'University of Illinois Urbana-Champaign University of Pennsylvania Shanghai Jiao Tong University 'Google fyifan26, (aaaan2,,.ouen22]]_ lourent2)@11llnnott
edu
https: //pp:///llnnaaaaa lab gi github. io/ spatialreasoneer
What is the vertical distance height between ① and !
What the distance between $ and (
Step Select the yellow car, which a adopt overage height of aporoximately 1.5 meters.
Step Define overage logth o and it about meters.
Step Usiny the height of car to estimats the vertical distance, the distonce is estimated about Dhigghts
Step Visually arsees that the gop spons about
between 5
longth (
Step Calculate distgnce by muitiplying the longth O 12m
Step Compute vertioal distionce: bucheeen @ard and ① using 1.5m 7.5
How much higher the highest point of than the highest of
From the perspective ① which clock-foce dirkchions
Step 1: Set ground level of ④ zano
Step
2
direction,
Step 2: According to the stondard bilbardd next to road, the height of a about meters.
Step ①
olso slightly quadrant between positions.
Step 3: The height of billboord is C about the height of 5 odults about 8.5 meters.
Step analysts.
2
Step 4: Colculate the height difference between S and 8.5m 3m + 5.5m
position
3
Hertioa Ducaare
Estinacioon
Dumaars
Spetidissstee 49
+10 00
Azurrny
Wide
Gemin Flush Maverick LnternVL2.55 2- 5-78
Direct
Hentbwns
Vertica Dietenon
Widht
Height
SpotidRGPT-80 SiSppholeeoooorrr--rr -2--9
balokx
Dietoree
Abstract
Current Visguu-Language Models (VLMs) struggle with fine-grained spatial reason- ing, particularly when multi-step logic and precise spatial alignment required. In this work, we introduce SpatialReasoner-RR a Aision-language reasoning model designed to address these limitations. To construct high-quality supervision for spatial reasoning, we design Multi-Model Monte Carlo Tree Search (M3CTS) method that generates diverse, logically consistent Long Chain-of-Thouss (Long- CoT) reasoning trajectories. addition, we propose a fine-grained Direct Pref- erence Optimization (IDPO) method that introduces segment-specific preference
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

granularity for descriptive grounding and logical reasoning, guided by spatial reward mechanism that evaluates candidate responses based on visual consistency, spatial grounding, and logical coherence. Experimental results demonstrate that fIPO achieves relative performance gains of 4.1% and 9.0% over standard DPO on spatial qualitative and quantitative tasks, respectively. SpatialRecasonr--I11 trained with IDPO, sets a new SoTA on MATTALRGPT-BENCEE outperforming the strongest baseline by 9.4% in average accuracy, while maintaining competitive performance on general vision-language tasks.
Introduction
Vision-Language Models (VLMs) have demonstrated significant advancements in multimodal un- derstanding tasks, such image captioning. visual question answering, object detection, and video interpretation [2. 31, 53, 62, 95]. However, their ability to perform spatial reasoning remains lim- ited, especially in scenarios involving complex object arrangements and occlusions [7, 13, 18, 81].
This gap poses a significant challenge for applications such as robotics, autonomous driving, and augmented reality, where robust spatial understanding is essential for effective decision-making [50].
Historically, early VLMs predominantly employed direct-response paradigms [2, 53], i.e., producing immediate answers without explicit reasoning, which often leads to shallow understanding. Recent advances in Chain-of- Thoughht (CoT) prompting have introduced step-by-ster reasoning [72], but standard COT traces often too brief or abstract capture fine-grained spatial logic. In contrast, Long Chain-of- Thought (I (LongCoT) prompting produces richer, more interpretable reasoning paths that better support comprehension [10, 43, 73]. Still, such prompting must go beyond simple depth estimation, as accurate spatial reasoning requires understanding occlusions, relative orientations, and positional ambiguity, all of which are difficult capture without structured, fine- fine-grained supervision.
R1, 1, a novel VLM designed to perform spatial reasoning directly from 2D images. SpatialReasoner-RI employs structured, interpretable LongCoT reasoning to systematically parse and solve spatial queries without relying on additional modalities external sensor data. To optimize the training process for multi-step reasoning, intro- duce a new fine-grained Direct Preference Optimization (fDPO) method that applies differentiated learning updates tailored to two semantically distinct components, descriptive grounding and logical reasoning. Unlike traditional DPO, fDPO introduces segment-specine preference granularity, allowing SpatialReasoner-R. adjust its optimization for each generation phase, emphasizing spatial local- ization during descriptive grounding and enhancing multi-step logical inferences during reasoning.
To address these challenges, we introduce SpatialReaaoner-R11
To curate diverse high-quality spatial reasoning data for training, we propose a Multi-Model Monte Carlo Tree Search (M3CTS) that generates high-qualin LongCoT responses by leveraging col- laborative exploration across multiple VLMs, and fine-grained spatial reward mechanism that evaluates candidate responses across three dimensions: descriptive accuracy, spatial grounding pre- cision, and logical coherence, which are then used construct positive and negative sample pairs for DPO and fDPO training. Empirical results across several challenging spatial reasoning tasks demonstrate that ReatialRe achieves state-of-the-art performance, significantly outperform- ing existing VLMs and CoT-based methods, particularly on complex, multi-step spatial reasoning. Specifically, SpatialReasoner-RI surpasses the best baseline by 9.4% in average accuracy on spatial understanding. Our fDPO improves by 4.1% % and 9.0% on average over standard DPO on spatial qualitative and quantitative tasks, respectively. Our contributions are as follows:
(1) We introduce SpatialReasoner-RR a LongCoT spatial reasoning VLM that effectively generates interpretable, syep-by-step explanations directly from 2D images. SpatialRcasoner- R1 establishes a new SoTA in spatial understanding, while maintaining robust performance general vision-language benchmarks.
(2) To enhance training stability and precision, we propose a new fine-grained Direct Prefer- ence Optimization (IDPO) method that employs segment-specific learning updates tailored explicitly for descriptive grounding and logical reasoning.
(3) To address the scarcity of high-quality spatial reasoning data, we introduce a data generation pipeline that combines Multi-Model Monte Carlo Tree Search (M3CTS) with fine-grained spatial rewards, enabling the creation of diverse, logically consistent LongCoT trajectories for fine-grat -rained preference training.
2

---

2
Related Work
Vision Language Models and Spatial Reasoning, Recent advances in VLMs have significantly enhanced the ability of multimodal models to understand and generate descriptive text grounded in visual contexts [31, 39, 40, 49, 66, 95]. Models such as Flamingo [1], BLIP-2 [32], and Qwen- VL. [39] use high-capacity vision encoders [53] paired with LLMs [5. 64] to achieve state-of-the-art performance in various multimodal tasks, such as visual question answering. image captioning, and instruction following [2. 15, 37, 65, 76, 100]. Current trends involve scaling models to improve general understanding [12, 25, 62] and using large-scale instruction tuning datasets [40, 56, 93]. Both proprietary [21, 26, 25] and open-source VLMs [12, 17, 89] have shown impressive results.
While VLMs show promise in visual understanding, accurately perceiving and reasoning about spatial arrangements remains challenge [13]. Recent efforts improve spatial understanding include fine-tuning VLMs on spatial VQA datasets [7, 8, 13, 41, 75, 28, 55], and zero-shot frameworks that leverage external 3D foundation models for geometric priors [44]. Region- -wwrr models have also been proposed for better grounding and finer spatial queries [24, 87, 91]. These advances extend to scenarios such as video understanding [81] and 3D generation [46, 50]. To track progress, specialized benchmarks like Q-Spatial Bench [36], SpatialRGPT-Bench [13], VSI-Bench [81] and 3DSRBench [45] have been introduced to assess spatial skills. However, current models still struggle with complex, multi-step spatial reasoning. SpatialReasoner-RI addresses this gap by introducing fine-grained preference optimization and multi-level reward mechanisms.
Aligning VLMs using Preference Optimization. Preference- based learning methods, particu- larly DPO [54], have become standard techniques for aligning models with human intentions. These methods bypass the need for explicit reward model training and have often demonstrated strong performance compared to earlier Reinforcement Learning with Human Feedback (RLHF) approaches [3, 19, 48, 98]. In the multimodal domain, DPO and its variants have been adapted to ad- dress specific challenges such as reducing hallucinations and improving visual grounding [70, 78, 88). The adaptability of DPO is further highlighted by its recent application in aligning generative models beyond language, such as text-to-image diffusion models [22, 33, 67, 82, 90]. Adaptation methods often involve constructing preference pairs based on human corrections, Al feedback, or contrasting inputs to guide the model towards desired behaviors [11, 14, 18, 20, 61, 68, 74, 77, 79, 85]..
Standard DPO methods treat the reasoning process as a single structure. To address this, preference granularity in DPO has been explored at the token [38, 57, 94, 97, 99], step [29, 96]. sentence [51, 54, 58], and turn [59, 60, 80] levels. While effective in certain domains, these approaches overlook the semantic roles of different segments in LongCoTT, where descriptive grounding and logical reasoning require distinct optimization. In contrast, our proposed fDPO introduces funetional-level preference granularity.
Multi-LLM Guided Reasoning Recent work has explored leveraging multiple LLMs to collab- oratively solve complex reasoning tasks, often integrated with Monte Carlo Tree Search (MCTS).
Methods such as MoA [69], MoSA [84], AlphaLLM-CPL [71], and LE-MCTS [52] enhance multi- agent text- text-based reasoning using ensemble methods and stepwise search. CoMCTS (Mulberry) [86] extends multi-LLM MCTS to multimodal reasoning, primarily targeting collaborative reflection and error correction. In contrast, our method, M3CTS, addresses the challenge of spatial reasoning in VLMs, introducing fine-grained preference learning and multi-level spatial rewards that allow for coherent, -groundd reasoning paths across multimodal data.
3 Method
3.1 Spatial Reasoning from Images
Spatial reasoning is a core vision-language challenge, requiring models to understand visual layouts and perform logical inference over spatial relationships. We define spatial reasoning as a multimodal understanding problem where the goal is to generate accurate reasoning paths based on visual and textual inputs. Formally, a spatial reasoning instance can be represented as tuple T= (1, Q, P) "e, R where I represents the input image containing the visual content, Q is the textual query specifying the spatial reasoning task, P denotes the visual prompt tokens pointing to a specific object or region in the image, and R the textual response, providing the answer or step-by-step reasoning path. The
3

---

ttage Generating LongCoT based on M3CTS
Sildong Tame Detuild The spuf conrect, such the visaa scere cr zoonants descriptio.
M3CTS
PRONPT Ti
LongCoT Generatton
Let's thin step snap diaar Shap engienti loested et... Snapd, <ng
Preference Optimization
Descristion
Reasering
Positive Response
Baese Preason
Responses Generation
passorption
Reasoning
mM
Negotive Response
SpetieReemoner4t
Stage 2. Preference Pair Construction
Stage Fine-grained Direct Preference Optimization
Figure 1: Method Overview. To train SpatialReasoner-RI we (1) generate reasoning paths using M3CTS, (2) construct fine- ane-grained preference pairs via reward-basse selection, and (3) train with fine-grained DPO (fDPO) to optimize descriptive and logical reasoning separately.
primary objective of a spatial reasoning model, denoted as is to map the multimodal input T to logically sound and spatially grounded response R.
Unlike typical direct-response VQA tasks, SpatialReasoner R1 is designed to output Long- CoT reasoning traces that de- compose spatial reasoning into clear, verifiable steps. To train the model, we introduce a fine- grained preference objective that optimizes descriptive and reason- ing responses separately ($3.2) and spatial reward mechanism that evaluates candidate reason- ing paths based spatial and logical understanding ($3.3). Fi- nally, to address the lack of LongCoT supervision for spatial reasoning, we propose multi-model collaborative tree search method that generates diverse, reward-aligned reasoning trajectories to enable preference-basee training ($3.4), An overview of the proposed training framework is depicted Figure and the SpatialReasoner-R architecture is shown Figure 2.
Large Language Model
Image Encoder
Image Embd
Resporse
Image
Prompt Encoder
Prompt Embd
Visual Prompt (Masks)
Text Enceder
Text Decoder
Embd
Text
Figure Architecture Overview. SpatialReasoner-RI is a VLM that takes input text instruction, visual prompts, and an image, and generates LongCoT reasoning responses.
3.2 Fine-grained Direct Preference Optimization (fDPO)
We propose fDPO novel fine-grained off-policy preference learning algorithm optimize LongCoT spatial reasoning. Traditional DPO methods apply a single global trade-off parameter uniformly across all reasoning steps [68. 74, 85), implicitly treating all response segments as equally learnable. However, this can lead to degenerate solutions, as the model may overfit to simpler descriptive responses while under-optimizing the more complex reasoning paths. This observation motivates the design of our fine-grained preference mechanism, which introduces segment-level preference granularity.
To facilitate fine-grained preference optimization, we first segment each LongCoT response R into its constituent description Riese and reasoning R components, represented as R RR Rreussn] We then quantify the preference signal for each segment by calculating the score difference between the corresponding segments derived from the positive R and negative responses R', yielding segment-wise preference differentials:
A score (R score ( A A score R ( score
R
(I)
where the differentials AR,es and A Reeasm quantify the preference margin for description and reasoning segments based on preference pair, and score(-) composite scores are introduced in Section 3.3. The design of (DPO is guided by two key principles:

---

Preference optimization strength should be dynamicalty balanced according to the intrinsic complexity and quality disparity between description and reasoning components.
Principle 1:
Our analysis of the fine-grained reward signals reveals that descriptive segments (Raes) are easier to optimize while models struggle with reasoning segments (Ressoon) that are typically longer and require multi-hop logic. Thus, a unified optimization parameter 3 may lead to reasoning under-optimization. To address this, fDPO introduces separate, adaptively-tune trade-off parameters, Bdese and Breason which dynamically control the learning signals for each segment independently, to allow the model to prioritize deeper logical inference while maintaining visual and attribute accuracy.
The choice of segment-specific optimization parameters (Baese and Breason) should prioritize the component that exhibits larger preference differential, such that learning focuses harder-to-learn segments.
Principle 2:
We further empirically observe that the preference score differential for descriptive components A Roses consistently smaller than A Resssnn To account for this, fDPO computes dynamic segment weights Wdess and Wreason to adaptively adjust the learning signals for the description and reasoning components, respectively:
exp(A A
AR.)
W exp(1 (X A
$ S {desc, reason);
(2)
exp (X ARreaso))
where RR, is the preference score differential for segment s (either description or reasoning). À >> 0 controls the sensitivity of weights, and (Wasse W'rasson] reflect the relative importance of each segment.
These weights are then mapped to adjustment factors centered around and applied to the base opti- mization parameter 3 to yield segment-specific trade-off parameters Sdese and Breason for description and reasoning, respectively:
B,
f(w.) B[1 (2w,- - 1)] E {dese, reason),
where w. [0, 1] is the respective segment-specifee weight (either description reasoning), a is hyperparameter that controls the maximum scaling amplitude, and is the base hyperparameter value. This design implements dynamic learning strategy: segments with larger preference differentials (higher relative importance w.) receive a higher effective 5g, amplifying the learning signal and prioritizing those components. Conversely, smaller preference differentials yield lower 3., enabling finer-gres iner-grained updates. This adaptive mechanism allows fDPO to balance optimization based on segment-specifie learning difficulty, promoting better alignment for complex reasoning steps while preserving descriptive accuracy. The final optimization objective for each segment is defined as
To R IT) ()
To R
T) (T)
FT RP, R.) log
log
E (dese, reason).
Tief R
Tret R
Here, T represents the multimodal input (image, text query, visual prompt), is the model's learned policy, and F, measures the segment-speene specific preference margin in log- log-likelihood ratios relative to a reference policy Net. The overall optimization objective for fDPO is
JP,R. RR) + Breason Fresson (T,R,, R (5)
(0) = - NI WE) log of Prese Faese (T, (T,P,
where D= ((((( R(m.), R0.0N and o-)) sigmoid activation function.
3.3 Fine-Grained Spatial Rewards
To optimize spatial reasoning paths effectively, we introduce fine- fine-grained reward mechanism that evaluates candidate reasoning paths across visual, spatial, and logical dimensions. Rewards capture alignment with image content, spatial relationships, and logical inference. Figure 3 illustrates the proposed fine-grainee spatial rewards for IDPO. Specifically, we define four scalar rewards; details about their formulations and rationale behind each reward are provided in Appendix A.
Visual Consistency Reward (Ryc) evaluates the description Riees to ensure spatial grounding and fidelity. The reward verifies key aspects of the quality and alignment of the description with the visual scene, such as whether all referenced objects are present and identifiable, whether the
5

---

Resp1
Resp2
The image shows modern kitchen island with dark gray base cabinet, The quest ion asks for the height of this kitchen iskan. Let's 'ss think step by step. Step The cobinet looks slightly lower than the kitchen counter behind Step 2: Standard kitchen is about 50 centimeters tall. Step 3: Thus the height of the kitchen island is about 45 centimeters.
Resp1
Question: What is the height of the kitchen ialand?
X
Fine-grained
Reward Score Breakdown
Preference Optimization
Negative Somple
(Descriptive Scoring]:3
(Spottial S:oring]:3.5 [Reasoning Scoring]:2.6
Gostialllssooorri
Reference
Fine- gnained
Spatial
Positive Somple
(Deseniptive S:oring]:3.8
Model
Rewerd
(Spatial Scoring):7.6 [Reasoning 3:
The image shows madern kitchen with lange island the center. Two white bor stools are positioned the corner of the island. The question asks for the height of the dark cobinet of the kitchen islond Let's think step by step. Step The bar stools the floor to the seat, Step 2: The top of the island centimeters higher than the seats of the bar stools, Step 3: Adding the ber stool height and the additional height above the stools: 75 20 95 cm.
Question: What is the height of the kitchen islond?
Resp2
to be standard bar height. which around 75 centimeters from
to be about 20
Figure 3: Fine-Grained Spatial Rewards. Candidate reasoning paths are decomposed into three aspects, descriptive, spatial, and reasoning, scored separately: the higher value in each row is marked by / and the lower by X. Explanation of Scoring: Descriptive: Negative response omits the two bar-stools and uses generic "modern kitchen" wording, whereas the positive response lists every salient object; Spatial: Negative response wrongly claims the island is lower than the rear counter and ignores the 20cm offset revealed by the stool reference, whereas the positive response provides its estimate to the 75cm stool height plus that offset; Reasoning: Negative response uses an illogical "half-height" heuristic 90em -> 45em without intermediate computation, whereas the positive response explicitly adds reference height and gap (75em + 20em = 95cm). These per-category deficits yield lower composite reward, designating the upper response as negative sample.
stated properties (such as color, size, and shape) match the visual content, whether the descrip- tion includes all necessary details prompted by the query, and whether remains contextually appropriate and free from extraneous information.
Depth-Guided Spatial Reward (Rxp) measures fine-grained spatial understanding by leveraging depth information. This reward is independently computed for the description Raese and reasoning Reason components, with adaptive weighting mechanisms: uncertainty weight that adjusts the score for spatial expressions with qualifiers (e.g., "approximately" "aassatlll"""""iilll"""" to account for reduced confidence, and a context-aware weight that emphasizes spatial relations directly relevant to the query. The final spatial rewards are computed as the uncertainty and context-aware weighted correctness scores across all spatial assertions in the corresponding description and reasoning components, validated against both the RGB image and its corresponding depth map. This ensures that more confident and contextually aligned relations have stronger influence the reward.
Logical Coherence Reward (Rw) evaluates the reasoning Reesson for structural integrity and logical correctness. This reward captures multi-hop inference and factual alignment by verifying that premises are consistent with the image, depth map, and preceding descriptions. reasoning steps maintain spatial and causal logic, the application of physical, spatial, and logical principles remains coherent throughout, and the conclusion is fully supported by the reasoning chain.
The preference differentials in IDPO are computed from composite rewards that aggregate fine-grained evaluations across segments: score( Raese) = Rve + Rap,desc and score( Rrenson) = Ric + Rapurenson.
3.4 Multi-Model MCTS (M3CTS)
We introduce a Multi-Model Monte Carlo Tree Search (M3CTS) framework for generating high- quality LongCoT data D= ((T, R2 } tailored to spatial reasoning. Inspired by DeepSeek- R1-Zero [16] and prior multimodal MCTS methods [86, 52, 84), M3CTS explores diverse reasoning trajectories across multiple VLMs to effectively search for logical, ppatisllyconsisstent explanations

---

that satisfy the query. Formally, the reasoning process is defined as sequence of reasoning states: S S= (s........ ,t. where 8, E S represents partial reasoning state, so is the initial state derived from T, and sT is terminal state corresponding to fully reasoned path. M3CTS operates through four key stages: Expand, Simulate, Backprop, and Select.
Expand. At each step t. M3CTS expands the current state by generating diverse candidate reasoning states Se using multiple VLMs { concurrently, i.e.
K
S.
= Tk 8t T, Parent(s1))
(6)
A=1
where TR is the k-th VLM, T multimodal input, and Parent(s;) ancestor reasoning states of sg. To ensure consistency, we enforce a structured output format across all VLMs (Appendix B).
Simulate, Each candidate $k,, E Se generated during expansion is evaluated based on three distinct criteria: (i) visual description accuracy against the original image, (ii) spatial correctness of inferred spatial relationships utilizing both original and depth-derived images, and (iii) logical coherence of the textual reasoning steps. The evaluation score R(sk.)) is computed
M
R(s,,0)
= M
1
visual m=1
Σ
((m) Visual (5k,)) + +II Ippunaal (#x,e) + (m) (sk.t)
where M is the number of evaluation models, and each ((m) (sk,e) indicator function defined as:
((m) (sk,e) = {+1 (fully accurate), (neutral), -1 (inaccurate)]-
(8)
We preserve high-quality paths by pruning the candidate set according the evaluation score, i.e., { R(Sk,)) 2 0}. Appendix C provides detailed descriptions the evaluation.
Backprop. To perform credit assignment, scores from the simulation phase are recursively propagated upwards through the search tree. The objective is to update the value estimates (* and visit counts N (SH.e) for each parent node Sk,t based on the performance of its children S. Child(sa,)),
V (SK.e) & V(SE.))( + Ex.es; R(s.)
N(sk.) + ((kke) + Σ ((s)). (9)
((s,.) + + N(s.)
S.ES;
This phase is responsible for choosing the most promising candidate state for further exploration in the next iteration of tree expansion. We use the Upper Confidence Bound (UCB) strategy select the next state st,,41 to traverse, based updated values and visitation statistics. UCB ensures that high-value paths prioritized, while also exploring liss-visited nodes to discover new reasoning trajectories. The candidate selected maximizes the UCB objective, i.e.,
Select.
log ((k,.) V(sc) + â V 1+ N(s))
arg max s.ES;
(10)
where Vssc) is the value estimate of the candidate state 8., (s.)) ) its visit count, and à > is hyperparameter that balances exploration vs. exploitation.
Experiments
4.1 Experimental Setup
We evaluate SpatialReasoner-RR across diverse spatial reasoning and general vision-language estab- lished benchmarks to assess the model's fine-grained spatial understanding and logical reasoning capabilities. Implementation details are provided Appendix D.
Spatial Reasoning Benchmarks. Our primary benchmark is SPATIALRGPT-BENNE [13], compris- ing image-based spatial reasoning questions and their corresponding ground truth answers. Detailed descriptions of benchmarks and evaluation protocols are provided in Appendix E.
7

---

Table Spatial Reasoning Success Rates (1) on SPATIALRGPI-BENC Classification (top) and numeric distance/direction (bottom). are General Large VLMs,
are Customized VLMs, are
SpatialReasoner-IR variants. "/" indicates the model refuses to provide response for that metric.
Below/ Above
Left/
Big/
Tall
Wide/
Behind/ Front
Qual.
Right
Short
Thin
Gemini 2.0 Flash [21]
58.33
68.57
16.98
50.00
15.38
53.63
44.29
Liama Maverick (47]
$4.17
61.90
50.89
25.96
55.45
Gemini 1.5 Pro [63]
85.83
56.19
58.49
71.42
55.76
60.00
CharGPT-lo (25)
87.50
80.00
53.77
63.39
51.92
60.90
66.67
SparialBot-3B
52.50
62.86
49.11
49.04
62.73
SpaceThinker Qwen2,5VVL-3B [4]
89.16
63.81
76.41
56.25
56.73
70.91
69.25
IntemVL2.5-78B [122]
94.16
94,28
64.15
65.17
55.76
58,18
72.29
Sa2VA 4B [89]
22.50
25.71
25.47
16.07
27.88
30.91
24.65
[89]
26.78
SpatialRGPT-88 [13]
99.17
100.00
84.90
89.28
91.34
90.90
92.69
SpatialReasoner-Rt SFT
79.16
78.09
55.66
66.96
59.61
75.45
69.41
SpatialReesomer-I SFT
81.66
81.90
75.47
75.89
79.80
83,63
79.75
SpatialReassmer--R DPO 4B
91
91.42
69.81
65.17
71.15
85.45
79.29
SeatialRRessoner-aa DPO 8B
94.16
93.33
89.62
90.18
88.6
92.27
91.48
ApatialReasoner-kl IDPO 4B
95.83
93.33
83.96
74.10
87.50
89.09
87.37
SpatialReasoner-It
98.10
96.43
91.34
93.64
Direct Distance
Horizontal Distance
Vertical Distance
Quan.
Width
Height
Direction
Acc.
Gemini 2.0 Flash [21]
9.45
10.65
26.41
10.52
30.82
54,20
22.43
Llama Maverick [47]
24.48
28.68
34.28
35.71
41.61
$8.09
36.72
Geminni 1.5 Pro [63]
14.18
17.21
14.15
19.54
36.09
30.84
21.90
ChaGPT-44 (25)
60.75
SpatiaBoot-3B [6]
6.00
15.51
8.00
10.52
18.75
39.00
15.62
SpaceThinkkr Qww.2.5V11338111111
24.32
17.21
59.45
23.27
23.62
32.35
28.97
IntemVVL2.5-780 [12]
27.70
22.13
41.50
29.32
34.58
62.61
35.25
Sa2VA 1B [89]
13.51
15.57
19.81
13.53
12.03
10.28
14.02
(89]
19:54
14.55
SpatialRGPT-SB [13]
SputialRensomer-BR SFT
22.29
27.86
31.13
25.56
33.80
47.66
30.71
SpatialReassoee--l RI SFT
25.43
20.49
44.05
33.59
$1.63
46.72
37.12
SpattalReasomer-R
47.97
46.72
45.11
SputialRensoner-.
68.42
68.22
SpatialReasonee-It IDPO
ReasmelRe IDPO
70.95
72.13
74.52
80.45
74.43
94.39 39
77.30
General Vision-Language Benchmarks. To validate the robustness SpatialReasoner-k R1 beyond purely spatial tasks, we evaluate on broader vision-language datasets such as MME, POPE, SEED- BENCH, AI2D, SQA-TEST MMMUv, MMSTAR, and HALLUSIONBENCH [35, 34, 30, 27, 42, 92, 23]. These datasets cover fundamental vision-language tasks such object grounding, hierarchical scene parsing, multimodal understanding. and multi-turn reasoning in diverse multimodal contexts.
Baselines. We benchmark SpatialReasoner-RR against two categories of baseline models:
General Large VLMs. This includes powerful, widely-accessible models such Gemini 2.0 Flash [21]. Llama 4 Maverick [47]. Gemini 1.5 Pro [63], and ChatGPT-4o [25]. These are evaluated zero-shot/few-se
settings reference standard VLM capabilities w/o task-specific fine-tuning. Specialized VLMs. This baseline set comprises models specifically developed, adapted, fine- tuned for spatial understanding tasks, allowing us to assess our contributions relative other special- ized approaches. The models included are: SpatialBot-3B [6], Space SpaceTThinke Thinker Qwen2.5VL. [4],
Intern VL.2.5-78B [12], Sa2VA (4B, 8B) [89], and SpatialRGPT-8B [13].
Spatialle VLMS. We also include SpatialReasoner-RR 4B and 8B variants with different training strategies, such as SpatialReasoner SFT, SpatialReasoner-RI DPO, trained with standard DPO, and SpatialReasoner-RR IDPO trained with the proposed fine-grained DPO method.

---

Table 2: General Vision-Language Understanding Results. Best performance bold.
Models
MME
POPE
SEED-BENCH
A12D
SQA-IEST
MMMUV
MMSTAR
HALLCSIONBENCH
SpaiiaRG/TT-BB [13]
1667/348
85.50
67.00
67,42
$1.81
41.40
43.98
40,80
Spatial
1667/503
76.11
78.85
93.85
48.11
51.10
Q: Can you give me an estimation the horizontal distance between center of <Region1> > and <Region 2>? Ground Truth
Response from SpatialRGPT 8B
herizentel distonce of Region from Region is meters.
Response from InternVL2.5- 2,5-78B
Step
Region 1 (fireplace)
Stop 2: Estimate the width of Region (TV cabinet) as meters. Step 3: the distence between fireplace and the stond seems (etther the fireplace the stand) Step 4: We con estimate that the total horizontal distence between the of Regionl Region? approximately sum half width of the fireplace, the distence between the oojects, and half the width of stand.
be the width
Regionl
Center of <Reglon1> and <Region2> 1.6 meters aport horizontally
5: Distance 2/2 5 1,5/2 2.85
Response from Gemini 1.5 Pro
Response from SpatialReasonenRR 8B
Step1: The right edge of the fireplace seems aligned with the left edge of the TV, and the TV centered doove the TV which slightly
Step 1: Estimate the width of the fireplace, The fireplact looks like standard size, about 1.2 meters wide.
Step2: Let's ossume the width fireplace meter. distence
between the center of the fireplaze and the left edge of the TV
slightly Estimate the width of the TV stond. The TV stand looks wide. Step 3: Estimate the gap between the fireplace and the stand. There's small between the fireplace and the stand, appears
Step 2:
be estimated half the fireplace's width, which 0.5 meters. Step3: ostume
distance from the left edge. of the TV to the center of the TV stand can be estimeted 1/4 of the stard's widhh, which 0 25 meters.
meters.
Step 4: Add the estimated widths and the 1.2m//2 (fireplace) 0.2m (cea) 1.4m/2 (TV) 1.5 meters, the regionl and region2 are meters opart horizontally.
Step4: Combining steps 2 and 3, the totol horizontal distance roughly 0.75 meters.
Figure 4: Qualitative Examples of Spatial Reasoning Across Models. SpatialReasoner-RI demon- strates a coherent, multi-step logical chain that closely matches the ground truth, while other models exhibit less precise or less interpretable reasoning paths.
4.2 Experimental Results
Spatial Reasoning. As shown in Table 1, SpatialReasoner-R models achieve substantial improvements over both general-purpose and spatial-specialized VLMs across all spatial tasks. Notably, SpatialReasoner--R fDPO 8B sets new benehmark for average accuracy with 2.9% and 15.8% gains over SpatialRGPT-88 on spatial qualitative and quantitative tasks, respectively. Our parameter-efffcient SpatialReasoner-FF RI fDPO 4B outperforms larger models like IntermVL2.5-78B. highlighting the effectiveness of our fine-tuning strategy. Finally, when compared to its predecessor DPO 8B, our optimized variant IDPO 8B boosts average accuracy by 4.1 across qualitative tasks and by 9.0% in quantitative tasks.
General Vision-Language Understanding. Beyond achieving sate-of-the-aar art performance in spatial reasoning tasks, our SpatialReasoner-RR fDPO 8B also demonstrates significant gains in general languael benchmarks compared to SpatialRGPT-SB, as presented in Table
4.3 Qualitative Examples
Figure 4 provides qualitative examples that demonstrate SpatialReasoner-RII advanced capability for coherent, multi-step spatial reasoning. SpatialReasoner- widths at 1.2m and 1.4m, then computes 1.2 reference while transparently tying each term an observed feature. In contrast, LL2557VL adopts similar width guesses (1.2m, 1.5m) but assumes "the distance between the fireplace and the TV stand seems to about the width of one object". This assumption inconsistent with what is shown in the image. Gemini 1.5 Pro aligns the fireplace's 's right edge with the TV's left edge, assigns both objects a Im width, and combines only half fireplace width (0.5m) with one quarter of the stand width (0.25m). These two estimates are not accurate and ignore the gap between the two regions. SpatialRGPT-8B yields more accurate estimate than Gemini 1.5 Pro and Intern -1B. However, since it is not designed as reasoning model, it cannot generate step-by-step reasoning
first estimates the fireplace and TV-stand 2 0.2 (gap) 1.4 = 1.5m, value that nearly matches the

---

Table 3: Effect of Alpha (a).
Table Effect of Lambda (A) at a=30%. %.
Metric
10%
20%
30%
Metric
A=0.2
A=0.4
0=0.6
A=0.8
Direct Dist.
53.38
56.76
60.13
58.11
Direct Dist.
54.05
$7.38
60.13
59.45
Horiiz Dist.
52.46
55.74
59.01
56.55
Huriz. Dist.
53.27
$7.37
59.01
58.19
Vert. Dist.
65.09
67.92
71.75
69.81
Vert. Dist.
65.09
68.86
71.75
69.81
Width
51.88
57.89
65.41
63.16
Width
53.38
60.15
65.41
54.67
Height
56.39
57.14
57.89
58.64
Height
56.39
57.14
57.89
57.89
Direction
91.58
92.23
92.52
94.39
Direction
91.58
91.58
92.53
93.45
traces, i.e., does not explicitly reveal the logical chain of spatial deductions or intermediate calculations leading to that estimate. These qualitative examples show that SpatialReasoner- has more accurate spatial awareness. Additional examples can be found in Appendix F.
4.4 Ablations
Table 3 illustrates the impact of varying the Alpha (a) parameter, which modulates the magnitude of segment-specific learning adjustments during fDPO optimization. When a is set too high, the model may overly focus on the reasoning part at the expense of the other, introducing instability and degraded performance, observed when reaches 40%. Conversely, if a is too low, both description and reasoning segments are optimized equally. A moderate value of a = 30% allows the model to effectively amplify learning signals for fine- tnee-graneed spatial distinctions, leading to substantial improvements across all spatial metrics. Furthermore, Table 4 presents the impact of varying the Lambda (A) parameter, which modulates the sensitivity of segment-specific weights preference differentials, controlling how responsively the model shifts learning focus based on the observed preference margins. As increases, the model becomes differences, leading to noticeable changes in performance across spatial metrics. We observe that while a moderate value of 0.6 achieves the best overall results, setting À too high introduce slight performance degradation in some spatial metrics, likely due to overly aggressive re-weighting,
sensitive segment-specific preference
Conclusion
In this work, we introduce SpatialReasoner-R1,, novel VLM with state-of-the-art spatial reasoning capabilities, trained with a proposed fine-grained DPO (fDPO) method that decomposes LongCo'i' paths into description and reasoning components, allowing for targeted preference-based learning and enhanced logical reasoning. IDPO guided by a set of comprehensive rewards that evaluate reasoning paths across visual consistency, spatial alignment, logical coherence, and depth-based verification. Additionally, we propose Multi-Modes Monte Carlo Tree Search (M3CTS) strategy that leverages multiple VLMs to generate high-quality diverse LongCoT data. Our comprehensive evaluations demonstrate SpatialReasoner- R.I achieves state-of-the oh-thr--rtttt performance, outperforming significantly larger models. Moving forward, plan evaluate fDPO on additional VLM tasks such as GUI navigation and reasoning segmentation.
Acknowledgments
This research was supported by gift from Google AI, the Google TPU Research Cloud (TRC) program, and the U.S. Defense Advanced Research Projects Agency (DARPA) under award numbers HR001 12390062 and HR001 125C0303. We gratefully acknowledge the cloud TPU credits from the Google TPU Research Cloud (TRC) program and the Google Tunix (Tune-in-JAX) team for their feedback. The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies, either expressed or implied, of Google, DARPA, or the U.S. Government. The U.S. Government authorized to reproduce and distribute reprints for governmental purposes notwithstanding any copyright annotation therein.
10

---

References
[1] Bean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, lain Barr, Yana Hasson, Karel Lene, Arthur Mensch, Katherine Millican, Malcolm Reynolds, al. Flamingo: visual language model for few-shot learning. In Advances in Neural Information Processing Systems (NeurIPS), 2022.
[2] Stanislaw Antol, Aishwarya Agrawal, Jiasen Lu, Margaret Mitchell, Dhruv Batra, C Lawrence Zitnick, and Devi Parikh. Vqa: Visual question answering. In International Conference Computer Vision (ICCV), 2015.
[3] Mohammad Gheshlaghi Azar, Zhaohan Daniel Guo, Bilal Piot, Remi Munos, Mark Rowland, Michal Valko, and Daniele Calandricllo. A general theoretical paradigm understand learning from human preferences. In Annual Conference on Artificial Intelligence and Statistics (AISTATS), 2024.
[4] Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, al. Qwen2.5-vl technical report. arXiv:250..1392.) 2025.
[5] Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhari- wal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language mod- els are fhw-shot learners. In Advances Neural Information Processing Systems (NeurIPS), 2020.
[6] Wenxiao Cai, laroslav Ponomarenko, Jianhao Yuan, Xiaoqi Li, Wankou Yang. Hao Dong, and Bo Zhao. Spatialbot: Precise spatial understanding with vision language models. In International Conference on Robotics and Automation (ICRA), 2025.
[7] Boyuan Chen, Zhuo Xu, Sean Kirmani, Brian Ichter, Dorsa Sadigh, Leonidas Guibas, and Fei Xia. Spatialvim: Endowing vision-language models with spatial reasoning capabilities. In IEEE/CVF Conference on Computer Vision and Puttern Recognition (CVPR), 2024.
[8] Keqin Chen, Zhao Zhang, Weili Zeng, Richong Zhang, Feng Zhu, and Rui Zhao. Shikra: Unleashing multimodal 11m's ' referential dialogue magic. arXiv:2306. 1515,, 2023.
[9] Lin Chen, Jinsong Li, Xiaoyi Dong, Pan Zhang, Yuhang Zang, Zehui Chen, Haodong Duan, Jiaqi Wang, Yu Qiao, Dahua Lin, al. Are the right way for evaluating large vision- language models? In Advances in Neural Information Processing Systems (NeurIPS), 2024.
[10] Qiguang Chen, Libo Qin, Jinhao Liu, Dengyun Peng, Jiannan Guan, Peng Wang, Mengkang Hu, Yuhang Zhou, Te Gao, and Wanxiang Che. Towards reasoning era: A survey of long chain-of-thour for reasoning large language models. arXiv:2503.0955 2025.
[11] Zhangquan Chen, Xufang Luo, and Dongsheng Li. Visrl: Intention-driven visual perception via reinforced reasoning. In International Conference on Computer Vision (ICCV), 2025.
[12] Zhe Chen, Jiannan Wu, Wenhai Wang, Weijie Su, Guo Chen, Sen Xing, Muyan Zhong, Qinglong Zhang, Xizhou Zhu, Lewei Lu, al. Internvl: Scaling up vision foundation models and aligning for generic visual- lingu-iing tasks. In IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), 2024.
[13] An-Chich Cheng, Hongxu Yin, Yang Fu, Qiushan Guo, Ruihan Yang, Jan Kautz, Xiaolong Wang, and Sifei Liu. Spatialrgpt: Grounded spatial reasoning in vision language models, In Advances in Neural Information Processing Systems (NeurIPS), 2024.
[14] Chenhang Cui, An Zhang, Yiyang Zhou, Zhaorun Chen, Gelei Deng, Huaxiu Yao, and Tat-Seng Chua. Fine-grained verifiers: Preference modeling as next-token prediction in vision- languue alignment. In International Conference Learning Representations (ICLR), 2024.
[15] Wenliang Dai, Junnan Li, Dongxu Li, Anthony Tiong, Junqi Zhao, Weisheng Wang, Boyang Li, Pascale N Fung, and Steven Hoi. Instructblip: towards general-purpose vision-language models with instruction tuning. In Advances Neural Information Processing Systems (NeurIPS), 2023.
11

---

[16] DeepSeek-AI. Daya Guo, Dejian Yang. Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang. Xiao Bi. Xiaokang Zhang, Xingkai Yu. Yu Wu, 7. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, Aixin Liu, Bing Xue, Bingxuan Wang, Bochao Wu, Bei Feng. Chengda Lu, Chenggang Zhao, Chengqi Deng. Chenyu Zhang, Chong Ruan, Damai Dai, Deli Chen, Dongjie Ji, Erhang Li, Fangyun Lin, Fucong Dai, Fuli Luo, Guangbo Hao, Guanting Chen, Guowei Li, H. Zhang, Han Bao, Hanwei Xu, Haocheng Wang. Honghui Ding, Huajian Xin, Huazuo Gao. Hui Qu, Hui Li, Jianzhong Guo, Jiashi Li, Jiawei Wang. Jingchang Chen, Jingyang Yuan, Junjie Qiu, Junlong Li, J. L. Cai, Jiaqi Ni, Jian Liang, Jin Chen, Kai Dong, Kai Hu, Kaige Gao, Kang Guan, Kexin Huang, Kuai Yu, Lean Wang. Lecong Zhang, Liang Zhao, Litong Wang. Liyue Zhang. Lei Xu, Leyi Xia, Mingchuan Zhang, Minghua Zhang, Minghui Tang, Meng Li, Miaojun Wang, Mingming Li, Ning Tian, Panpan Huang, Peng Zhang, Qiancheng Wang, Qinyu Chen, Qiushi Du, Ruiqi Ge, Ruisong Zhang. Ruizhe Pan, Runji Wang, R. J. Chen, R. I. Jin, Ruyi Chen, Shanghao Lu, Shangyan Zhou, Shanhuang Chen, Shengfeng Ye, Shiyu Wang, Shuiping Yu, Shunfeng Zhou, Shuting Pan, S. S. Li, Shuang Zhou, Shaoqing Wu, Shengfeng Ye, Tao Yun, Tian Pei, Tianyu Sun, T. Wang, Wangding Zeng, Wanjia Zhao, Wen Liu, Wenfeng Liang, Wenjun Gao. Wengin Yu, Wentao Zhang, W. L. Xiao, Wei An, Xiaodong Liu, Xiaohan Wang, Xiaokang Chen, Xiaotao Nie, Xin Cheng, Xin Liu, Xin Xie, Xingchao Liu, Xinyu Yang. Xinyuan Li, Xuecheng Su, Xuheng Lin, X. Q. Li, Xiangyue Jin, Xiaojin Shen, Xiaosha Chen, Xiaowen Sun, Xiaoxiang Wang. Xinnan Song, Xinyi Zhou, Xianzu Wang. Ximxia Shan, Y. K. Li, Y. Q. Wang. Y. X. Wei, Yang Zhang, Yanhong Xu, Yao Li, Yao Zhao, Yaofeng Sun, Yaohui Wang. Yi Yu, Yichao Zhang, Yifan Shi, Yiliang Xiong, Ying He, Yishi Piao, Yisong Wang, Yixuan Tan, Yiyang Ma, Yiyuan Liu, Yongqiang Guo, Yuan Ou, Yuduan Wang, Yue Gong. Yuheng Zou, Yujia He, Yunfan Xiong, Yuxiang Luo, Yuxiang You, Yuxuan Liu, Yuyang Zhou, Y. X. Zhu, Yanhong Xu, Yanping Huang, Yaohui Li, Yi Zheng, Yuchen Zhu, Yunxian Ma, Ying Tang. Yukun Zha, Yuting Yan. Z. Z. Ren, Zehui Ren, Zhangli Sha, Zhe Fu, Zhean Xu, Zhenda Xie, Zhengyan Zhang, Zhewen Hao, Zhicheng Ma, Zhigang Yan, Zhiyu Wu, Zihui Gu, Zijia Zhu, Zijun Liu, Zilin Li, Ziwei Xie, Ziyang Song. Zizheng Pan, Zhen Huang, Zhipeng Xu, Zhongyu Zhang, and Zhen Zhang. DeepSeek-Re RI: Incentivizing reasoning capability in llms via reinforcement learning. arXiv 2501.12948, 2025.
[17] Matt Deitke, Christopher Clark, Sangho Lee, Rohun Tripathi, Yue Yang, Jae Sung Park, Mohammadreza Salehi, Niklas Muennighoff, Kyle Lo, Luca Soldaini, et al. Molmo and pixmo: Open weights and open data for state-of-the multimodal models. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2025.
[18] Yuhao Dong, Zuyan Liu, Hai-Long Sun, Jingkang Yang, Winston Hu, Yongming Rao, and Ziwei Liu. Insight-v: Exploring long-chain visual reasoning with multimodal large language models. In /EEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2025.
[19] Kawin Ethayarajh, Winnie Xu, Niklas Muennighoff, Dan Jurafsky, and Douwe Kiela. Model alignment as prospect theoretic optimization. In International Conference on Machine Learn- ing (ICML), 2024.
[20] Yuhan Fu, Ruobing Xie, Xingwu Sun, Zhanhui Kang, and Xirong Li. Mitigating hallucination in multimodal large language model via ballucination-targeen direct preference optimization. In Findings of the Association for Computational Linguistics (ACL), 2025.
[21] Google Deepmind. Gemini 2.0 is now available to everyone. https:/ technology/
gwwgge-deepgeennneeee cechhologg/ggoolledeeeppiii
2025.
[22] Yi Gu, Zhendong Wang, Yueqin Yin, Yujia Xie, and Mingyuan Zhou. Diffusion-rpo: Aligning diffusion models through relative preference optimization. arXiv:2406.06382. 2024.
[23] Tianrui Guan, Fuxiao Liu, Xiyang Wu, Ruiqi Xian, Zongxia Li, Xiaoyu Liu, Xijun Wang, Lichang Chen, Furong Huang. Yaser Yacoob, et al. Hallusionbench: An advanced diagnostic suite for entangled language hallucination and visual illusion in large vision-language models. In IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), 2024.
[24] Qiushan Guo, Shalini De Mello, Hongxu Yin, Wonmin Byeon, Ka Chun Cheung. Yizhou Yu, Ping Luo, and Sifei Liu. Regiongpt: Towards region understanding vision language model. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024.
12

---

[25] Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, al. Gpt-4o System Card. arXiv:2410.21276, 2024.
[26] Amazon Artificial General Intelligence. The amazon nova family models: Technical report and model card. 2024.
[27] Aniruddha Kembhavi, Mike Salvato. Eric Kolve, Minjoon Seo, Hannanch Hajishirzi, and Ali Farhadi. A diagram is worth a dozen images. In European Conference on Computer Vision (ECCV), 2016.
[28] Yangzhe Kong, Dacun Song, Jing Liang, Dinesh Manocha, Ziyu Yao, and Xuesu Xiao. Autospatial: Visual-language reasoning for social robot navigation through efficient spatial reasoning learning. arXiv:2503.0755) 2025.
[29] Xin Lai, Zhuotao Tian, Yukang Chen, Senqiao Yang, Xiangru Peng, and Jiaya Jía. Step-dpo: Step-wise preference optimization for long-chain reasoning of llms. arXiv:2406. 2024.
[30] Bohao Li, Rui Wang, Guangzhi Wang, Yuying Ge, Yixiao Ge, and Ying Shan. Seed-bench: Benchmarking multimodal Iims with generative comprehension. In IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), 2023.
[31] Feng Li, Renrui Zhang, Hao Zhang, Yuanhan Zhang. Bo Li, Wei Li, Zejun Ma, and Chunyuan Li. Llave-next-imterllave: Tackling multi-image, video, and 3d in large multimodal models. In International Conference on Learning Representations (ICLR). 2025.
[32] Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language- image pre-training with frozen image encoders and large language models. In International Conference on Machine Learning (ICML), 2023.
[33] Shufan Li, Konstantinos Kallidromitis, Akash Gokul, Yusuke Kato, and Kazuki Kozuka. Aligning diffusion models by optimizing human utility. In Advances in Neural Information Processing Systems (NeurIPS), 2024.
[34] Yifan Li, Yifan Du, Kun Zhou, Jinpeng Wang. Wayne Xin Zhao, and Ji-Rong Wen. Evaluating object hallucination in large vision-language models. In Conference on Empirical Methods in Natural Language Processing (EMNLP). 2023.
[35] Zijing Liang, Yanjie Xu, Yifan Hong, Penghui Shang, Qi Wang, Qiang Fu, and Ke Liu. A survey of multimodel large language models. In International Conference on Computer, Artificial Intelligence and Control Engineering. 2024.
[36] Yuan-Hong Liao, Rafid Mahmood, Sanja Fidler, and David Acuna. Reasoning paths with reference objects clicit guantitative spatial reasoning in large vision-language models. In Conference on Empirical Methods in Natural Language Processing (EMNLP), 2024.
[37] Tsung-Yi Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects context. In European Conference on Computer Vision (ECCV), 2014.
[38] Aiwei Liu, Haoping Bai, Zhiyun Lu, Yanchao Sun, Xiang Kong, Simon Wang, Jiulong Shan, Albin Madappally Jose, Xiaojiang Liu, Lijie Wen, et al. Tis-dpo: Token-level importance sam- pling for direct preference optimization with estimated weights. International Conference Learning Representations (ICLR), 2024.
[39] Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. In Advances in Neural Information Processing Systems (NeurIPS), 2023.
[40] Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning. In IEEE/CVF Conference on Computer Vision and Puttern Recognition (CVPR), 2024.
13

---

[41] Yuecheng Liu, Dafeng Chi, Shiguang Wu, Zhanguang Zhang, Yaochen Hu, Lingfeng Zhang, Yingxue Zhang, Shuang Wu, Tongtong Cao, Guowei Huang, et Spatialcot: Advancing spa- tial reasoning through coordinate alignment and thain-of-though for embodied task planning. arXiv:2501. 10074, 2025.
[42] Pan Lu, Swaroop Mishra, Tanglin Xia, Liang Qiu, Kai- Chang, Song-Chur Zhu, Oyvind Tafjord, Peter Clark, and Ashwin Kalyan. Learn to explain: Multimodal reasoning via thought chains for science question answering. In Advances in Neural Information Processing Systems (NeurIPS), 2022.
[43] Haotian Luo, Haiying Hc, Yibo Wang, Jinluan Yang, Rui Liu, Naiqiang Tan, Xiaochun Cao, Dacheng Tao, and Li Shen. Ada-R1: From long-cot hybrid-cot via bi-level adaptive reasoning optimization. In Advances in Neural Information Processing Systems (NeurIPS), 2025.
[44] Chenyang Ma, Kai Lu, Ta-Ying Cheng, Niki Trigoni, and Andrew Markham. Spatialpin: Enhancing spatial reasoning capabillities of vision- laionn-a models through prompting and interacting 3d priors. In Advances in Neural Information Processing Systems (NeurIPS), 2024.
[45] Wufei Ma, Haoyu Chen, Guofeng Zhang, Celso M de Melo, Jieneng Chen, and Alan Yuille. 3dsrbench: A comprehensive 3d spatial reasoning benchmark. In International Conference on Computer Vision (ICCV), 2025.
[46] Wufei Ma, Yu-Cheng Chou, Qihao Liu, Xingrui Wang, Celso de Melo, Jieneng Chen, Jianwen Xie, and Alan Yuille. Spatialreasoner: Towards explicit and generalizable 3d spatial reasoning. In Advances in Neural Information Processing Systems (NeurIPS), 2025.
[47] Meta. The llama 4 herd: The beginning of a new era of natively multimodal ai innovation. betps://al meta. www.blog////cccccccccocccooooooo 11
April 2025.
[48] Remi Munos, Michal Valko, Daniele Calandriello, Mohammad Gheshlaghi Azar, Mark Row- land, Zhaohan Daniel Guo, Yunhao Tang, Matthieu Geist, Thomas Mesnard, Côme Fiegel, et al. Nash learning from human feedback. In International Conference on Machine Learning (ICML), 2024.
[49] Kiet A Nguyen, Adheesh Juvekar, Tianjiao Yu, Muntasir Wahed, and Ismini Lourentzou. Cal- ico: Part-focused semantic co-segmentation with large vision-language models. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2025.
[50] Zhenyu Pan and Han Liu. Metaspatial: Reinforcing 3d spatial reasoning in vlms for the metaverse. arXiv:2503.18470 2025.
[51] Richard Yuanzhe Pang. Weizhe Yuan, He He, Kyunghyun Cho, Sainbayar Sukhbaatar, and Jason Weston. Iterative reasoning preference optimization. In Advances in Neural Information Processing Systems (NeurIPS), 2024.
[52] Sungjin Park, Xiao Liu, Yeyun Gong, and Edward Choi. Ensembling large language models with process reward reward-guided tree search for better complex reasoning. In Nations of the Americas Chapter of the Association for Computational Linguistics (NAACL), 2025.
[53] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International Conference on Machine Learning (ICML), 2021.
[54] Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly reward model. In Advances in Neural Information Processing Systems (NeurIPS), 2023.
[55] Kanchana Ranasinghe, Satya Narayan Shukla, Omid Poursaeed, Michael S Ryoo, and Tsung- Yu Lin. Learning to localize objects improves spatial reasoning visual-Ilms, In IEFF/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024.
14

---

[56] Hanoona Rasheed, Muhammad Maaz, Sahal Shaji, AbdeIrahman Shaker, Salman Khan, Hisham Cholakkal, Rao M Anwer, Eric Xing. Ming-Hsuar Yang. and Fahad S Khan. Glamm: Pixel grounding large multimodal model. In IEEE/CVF Conference on Computer Vision and Puttern Recognition (CVPR), 2024.
[57] Ruichen Shao, Bei Li, Gangao Liu, Yang Chen, Xiang Zhou, Jingang Wang, Xunliang Cai, and Peng Li. Earlier tokens contribute more: Learning direct preference optimization from temporal decay perspective. In International Conference Learning Representations (ICLR), 2025.
[58] Shuaijie She, Wei Zou, Shujian Huang, Wenhao Zhu, Xiang Liu, Xiang Geng, and Jiajun Chen. Mapo: Advancing multilingual reasoning through -ainggggalaliiinme---------ee optimization. Annual Meeting of the Association for Computational Linguisties (ACL), 2024.
[59] Wentao Shi, Mengqi Yuan, Junkang Wu, Qifan Wang, and Fuli Feng. Direct multi-turn preference optimization for language agents. In Conference Empirical Methods in Natural Language Processing (EMNLP), 2024.
[60] Yifan Song. Da Yin, Xiang Yue, Jie Huang, Sujian Li, and Bill Yuchen Lin. Trial and error: Exploration-based trajectory optimization of Ilm agents. In Annual Meeting of the Association for Computational Linguistics (ACI.), 2024.
[61] Zhiqing Sun, Sheng Shen, Shengcao Cao, Haotian Liu, Chunyuan Li, Yikang Shen, Chuang Gan, Liang- Yan Gui, Yu-Xiong Wang. Yiming Yang. et al. Aligning large multimodal models with factually augmented rlhf. In Annual Meeting of the Association for Computational Linguistics (ACL), 2023.
[62] Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, Katie Millican, et al. Gemini: a family of highly capable multimodal models. arXiv:2312. 1805, 2023.
[63] Gemini Team, Petko Georgiev, Ving Ian Lei, Ryan Burnell, Libin Bai, Anmol Gulati, Garrett Tanzer, Damien Vincent, Zhufeng Pan, Shibo Wang, al. Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context. arXiv:2403.05530, 2024.
[64] Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timo- thée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv:2302. 13971, 2023.
[65] Shangqing Tu, Yucheng Wang, Daniel Zhang-Li, Yushi Bai, Jifan Yu, Yuhao Wu, Lei Hou, Huiqin Liu, Zhiyuan Liu, Bin Xu, et al. Longwriter-v: Enabling ultra-long and high-fidelity generation in vision-language models. In ACM International Conference on Multimedia (ACM MM), 2025.
[66] Muntasir Wahed, Kiet Nguyen, Adheesh Sunil Juvekar, Xinzhuo Li, Xiaona Zhou, Vedant Shah, Tianjiao Yu, Pinar Yanardag, and Ismini Lourenizou. Prima: Multi-image vision- language models for reasoning segmentation. axXiv:2412.15209 2024.
[67] Bram Wallace, Meihua Dang, Rafael Rafailov, Linqi Zhou, Aaron Lou, Senthil Purushwalkam. Stefano Ermon, Caiming Xiong, Shafiq Joty, and Nikhil Naik. Diffusion model alignment using direct preference optimization. In IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), 2024.
[68] Fei Wang, Wenxuan Zhou, James Y Huang, Nan Xu, Sheng Zhang, Hoifung Poon, and Muhao Chen. mdpo: Conditional preference optimization for multimodal large language models, In Conference on Empirical Methods in Natural Language Processing (EMNLP), 2024.
[69] Junlin Wang, Jue Wang, Ben Athiwaratkun, Ce Zhang, and James Zou. Mixture-of-agenn aggatssts enhances large language model capabilities. In International Conference on Learning Repre- sentations (ICLR), 2025.
15

---

[70] Weiyun Wang, Zhe Chen, Wenhai Wang. Yue Cao, Yangzhou Liu, Zhangwei Gao, Jinguo Zhu, Xizhou Zhu, Lewei Lu. Yu Qiao, et al. Enhancing the reasoning ability of multimodal large language models via mixed preference optimization. arXiv:2411, 10442, 2024.
[71] Xiyao Wang, Linfeng Song, Ye Tian, Dian Yu, Baolin Peng, Haitao Mi, Furong Huang, and Dong Yu. Towards self-improvement of Ilms via mets: Leveraging stepwise knowledge with curriculum preference learning. urXiv:2410.06510 2024.
[72] Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoe V Le, Denny Zhou, et al. Chain-of-thoughh prompting elicits reasoning in large language models. In Advances in Neural Information Processing Systems (NeurIPS), 2022.
[73] Liang Wen, Yunke Cai, Fenrui Xiao, Xin He, Qi An, Zhenyu Duan, Yimin Du, Junchen Liu, Lifu Tang, Xiaowei Lx, et al. Lig Curriculum sft, dpo and rl for long cot from scratch and beyond. In Annual Meeting of the Association for Computational Linguistics (ACL), 2025.
[74] Junkang Wu, Yuexiang Xie. Zhengyi Yang, Jiancan Wu, Jinyang Gao, Bolin Ding, Xiang Wang, and Xiangnan He. beta-dpo: Direct preference optimization with dynamic beta. In Advances in Neural Information Processing Systems (NeurIPS), 2024.
[75] Wenshan Wu, Shaoguang Mao, Yadong Zhang, Yan Xia, Li Dong, Lei Cui, and Furu Wei. Mind's 's eye of llms: couulization-o--ttuugll elicits spatial reasoning in large language models. In Advances in Neural Information Processing Systems (NeurlPS), 2024.
[76] Xun Wu, Shaohan Huang, Guolong Wang, Jing Xiong, and Furu Wei. Multimodal large language models make text-to-image generative models align better. In Advances in Neural Information Processing Systems (NeurIPS), 2024.
[77] Yuxi Xie, Guanzhen Li, Xiao Xu, and Min-Yer Kan. V-dpo: Mitigating hallucination in large vision language models via dision-guide direct preference optimization. In Conference on Empirical Methods in Natural Language Processing (EMNLP), 2024.
[78] Shuo Xing, Yuping Wang, Peiran Li, Ruizheng Bai, Yueqi Wang, Chengxuan Qian, Huaxiu Yao, and Zhengzhong Tu. Re-align: Aligning vision language models via reurieval-augmentee direct preference optimization. In Conference on Empirical Methods in Natural Language Processing (EMNLP), 2025.
[79] Tianyi Xiong, Xiyao Wang, Dong Guo, Qinghao Ye, Haoqi Fan, Quanquan Gu, Heng Huang, and Chunyuan Li. Llava-critic: Learning to evaluate multimodal models. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2025.
[80] Wei Xiong, Chengshuai Shi, Jiaming Shen, Aviv Rosenberg, Zhen Qin, Daniele Calandriello, Misha Khalman, Rishabh Joshi, Bilal Piot, Mohammad Salch, al. Building math agents with multi-turn iterative preference learning. In International Conference on Learning Repre- sentations (ICLR). 2025.
[81] Jihan Yang, Shusheng Yang, Anjali W Gupta, Rilyn Han, Li Fei-Fei, and Saining Xie. Think- ing in space: How multimodal large language models see, remember, and recall spaces. ar114:2412.14171, 2024.
[82] Kai Yang, Jian Tao, Jiafei Lyu, Chunjiang Ge, Jiaxin Chen, Weihan Shen, Xiaolong Zhu, and Xiu Li. Using human feedback to fine-tune diffusion models without any reward model. In IEEF/CVF Conference on Computer Vision and Puttern Recognition (CVPR), 2024.
[83] Lihe Yang, Bingyi Kang, Zilong Huang, Zhen Zhao, Xiaogang Xu, Jiashi Feng, and Heng- shuang Zhao. Depth anything v2. In Advances in Neural Information Processing Systems (NeurIPS), 2024.
[84] Sen Yang, Yafu Li, Wai Lam, and Yu Cheng. Multi-lim collaborative search for complex problem solving. arXiv:2502. 18873, 2025.
[85] Zhihe Yang. Xufang Luo, Dongqi Han, Yunjian Xu, and Dongsheng Li. Mitigating halluci- nations large vision-language models via dpo: On-policy data hold the key. In IEFF/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2025.
16

---

[86] Huanjin Yao, Jiaxing Huang, Wenhao Wu, Jingyi Zhang, Yibo Wang, Shunyu Liu, Yingjie Wang. Yuxin Song, Haocheng Feng, Li Shen, et al. Mulberry: Empowering milm with ol- like reasoning and reflection via collective monte carlo tree search. In Advances in Neural Information Processing Systems (NewrIPS), 2025.
[87] Haoxuan You, Haotian Zhang, Zhe Gan, Xianzhi Du, Bowen Zhang, Zirui Wang, Liangliang Cao, Shih-Fu Chang, and Yinfei Yang. Ferret: Refer and ground anything anywhere at any granularity. In Interattional Conference on Learning Representations (ICLR), 2023.
[88] Tianyu Yu, Yuan Yao, Haoye Zhang, Taiwen He, Yifeng Han, Ganqu Cui, Jinyi Hu, Zhiyuan Liu, Hai-Tao Zheng, Maosong Sun, et al. Rihf-v: Towards trustworthy milms via behavior alignment from fine-grained correctional human feedback. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024.
[89] Haobo Yuan, Xiangtai Li, Tao Zhang, Zilong Huang, Shilin Xu, Shunping Ji, Yunhai Tong. Lu Qi, Jiashi Feng, and Ming- -usun Yang. Sa2va: Marrying sam2 with llava for dense grounded understanding images and videos. arXiv:2501 04000, 2025.
[90] Huizhuo Yuan, Zixiang Chen, Kaixuan Ji, and Quanquan Gu. Self-play fine-tuning of diffusion models for text-to-image generation. In Advances in Neural Information Processing Systems (NeurIPS), 2024.
[91] Yuqian Yuan, Wentong Li, Jian Liu, Dongqi Tang, Xinjie Luo, Chi Qin, Lei Zhang, and Jianke Zhu. Osprey: Pixel understanding with visual instruction tuning. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024.
[92] Xiang Yue, Yuansheng Ni, Kai Zhang, Tianyu Zheng, Ruoqi Liu, Ge Zhang, Samuel Stevens, Dongfu Jiang, Weiming Ren, Yuxuan Sun, et al. Mmmu: A massive multi-discipline multi- modal understanding and reasoning benchmark for expert agi. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024.
[93] Xiang Yue, Tianyu Zheng, Ge Zhang, and Wenhu Chen. Mammoth2: Scaling instructions from the web. In Advances in Neural Information Processing Systems (NeurIPS), 2024.
[94] Yongcheng Zeng, Guoqing Liu, Weiyu Ma, Ning Yang, Haifeng Zhang, and Jun Wang. Token- level direct preference optimization. In International Conference on Machine Learning (ICML), 2024.
[95] Duzhen Zhang, Yahan Yu, Jiahua Dong, Chenxing Li, Dan Su, Chenhui Chu, and Dong Yu. Mm-llms: Recent advances in multimodal large language models. In Annual Meeting of the Association for Computational Linguistics (ACL). 2024.
[96] Xuan Zhang, Chao Du, Tianyu Pang, Qian Liu, Wei Gao, and Min Lin, Chain of preference op- timization: Improving chain-of-thought reasoning in Ilms. In Advances in Neural Information Processing Systems (NeurIPS), 2024.
[97] Qi Zhao, Haotian Fu, Chen Sun, and George Konidaris. Epo: with environment preference optimization. In Conference Empirical Methods in Natural Language Processing (EMNLP), 2024.
Hierarchical Ilm agents
[98] Yao Zhao, Rishabh Joshi, Tianqi Liu, Misha Khalman, Mohammad Saleh, and Peter J Liu. Slic-hf: Sequence likelihood calibration with human feedback. arXiv:2305. 10425, 2023.
[99] Han Zhong, Zikang Shan, Guhao Feng. Wei Xiong, Xinle Cheng, Li Zhao, Di He, Jiang Bian, and Liwei Wang. Dpo meets ppo: Reinforced token optimization for rlhf. In ICML 2024 Workshop on Models of Human Feedback for Al Alignment, 2024.
[100] Yiyang Zhou, Zhiyuan Fan, Dongjie Cheng, Sihan Yang. Zhaorun Chen, Chenhang Cui, Xiyao Wang. Yun Li, Linjun Zhang, and Huaxiu Yao. Calibrated self-rewarding vision language models. In Advances in Neural Information Processing Systems (NeurIPS), 2024.
17

---

A Fine-Grained Spatial Reward Details
This appendix details design and hyperparameter choices for the fine- fine-grained spatial rewards intro- duced in Section 3.3. The prompt template for estimating rewards is shown in Figure 10.
A.1 Visual Consistency Reward (R.v))
The Visual Consistency Reward quantifies alignment between the generated description and the visual scene across four continuous criteria: Existence, Attribute accuracy. Completeness, and Appropriateness. Each component yields a score in the range [0.0, 1.0), where its continuous range enables fine-grained assessment, permitting fractional scores when descriptions partially satisfy evaluation criteria. Scores near 0.0 indicate misalignment, while scores near 1.0 denote perfect alignment. Intermediate values reflect varying degrees of partial correctness or uncertainty. The total reward, Re E [0, 4.0), distinguishes varying degrees of alignment across responses.
A.2 Depth-Guided Spatial Reward (Rup)
We introduce a depth-guided reward to evaluate the spatial accuracy of model outputs using ground- truth depth maps. The reward is computed independently for the description Raese and reasoning Reeason components, yielding
sub-scores: Rspudes: and Reppresson each ranging from 0 4. These scores capture the alignment of spatial expressions with geometric cues in the image. The final spatial reward is given by Rap = Rsspdesc + Rap.reason
Uncertainty Weight (W,,). Spatial expressions in model outputs often include uncertain qualifiers. Wa ranges from 0.8 to 1.0, with 1.0 indicating complete certainty in spatial assertions, and the lower bound of 0.8 representing cautious but plausible uncertainty. Setting the lower bound at 0.8 balances cautious language (e.g., "approximately"" "possibly") without overly penalizing reasonable uncertainty. Lower values (below 0.8) would overly penalize reasonable, conservative predictions and discourage the model from producing cautious but informative reasoning.
Context-aware Weight (W.). The context-aware weight W. E [0.8, 1.0) reflects the relevance of spatial statements to the question. Explicitly asked spatial relationships assigned W. 1.00, while auxiliary or indirect spatial references are assigned W. 0.8. This distinction prioritizes primary spatial relations explicitly required by the query, ensuring the model emphasizes essential spatial assertions more significantly. Scores below 0.8 would disproportionately underemphasize auxiliary information, degrading the model's 's ability to handle broader contextually relevant details.
Given a response, we extract all spatial relationship statements from the description and reasoning response sections. Each statement is then evaluated using GPT-4o by comparing the original image and its corresponding depth image, which generated using Depth Anything [83], to obtain correctness score r; E [0, 1]. Every statement is also assigned an associated (:(1) spatial reward scores are computed
and W!(") The
Rgp.desc
1 n
w(
Wi( ri
R Rpprrcason
1 m
W(
W
(11)
where and denote the number of spatial statements in the description and reasoning components, respectively, and r; represents the correctness of each spatial relationship, validated against both the RGB image and its corresponding depth map.
A.3 Logical Coherence Reward (Rie)
This reward quantifies the logical robustness of response by aggregating four components: Factual Consistency, Logical Coherence, Correct Rule Application, and Conclusion Validity. Each component scored in the range [0.0, 1.0]. with fractional values capturing partial correctness, e.g., from minor gaps in logical sequences to partial inaccuracies in applying physical, spatial, logical rules. Scores of 0.0 and 1.0 indicate complete logical coherence failure or perfect logical chains, respectively. The final reward, Rlc [0, 4.0), reflects the overall logical quality of the reasoning.
18

---

B Structured Output Format Specification for M3CTS
To enable reliable parsing and downstream analysis, M3CTS requires reasoning paths from VLMs to follow a standardized structured format. This format uses Markdown-style headings to clearly segment key components of the reasoning trace. Each section begins with line prefixed by ###,, followed by descriptive heading. The defined sections are:
#*# Description: Details the imput context, such as the visual scene scenario description.
### Rationale: Summarizes the overall reasoning strategy or justification.
### Let's think step by step: An optional phrase before detailed step-by-step breakdown. ### Step N (e.g., ### Step 1, ### Step 2, ...): Enumerates the sequential steps involved in the reasoning procedure. Multiple steps are typically present.
### In Conclusion: States the final derived conclusion of the reasoning process.
Figure 5 shows the reasoning tree example produced by M3CTS.
C Node Evaluation Protocol for M3CTS
To ensure the semantic and visual quality of each candidate reasoning step &g,t within the M3CTS framework, we employ structured multi-criteria evaluation. Individual steps that form reasoning path are segmented by ### Each candidate sk, E Se is independently evaluated by two multimodal models, Gemini 1.5 Pro and Qwen2.5VL-72B, along key distinct dimensions:
Visual Description Accuracy: Assesses whether the entities, attributes, and contextual cues described in sk, correctly reflect the visual content of the input image. This includes references objects, colors, spatial layouts, and contextual cues.
Spatial Consistency: Evaluates whether the spatial relations expressed in 8k, (e.g. "above," "to the left of," "behind")) are consistent with both the RGB image and depth map generated via the Depth Anything model [83]. Errors such as inversion of relations (e.g., stating "behind" instead of "in front") are penalized.
Logical Reasoning Coherence: For steps within the "think step-by-step" chain-of-thouuht reasoning block, this component checks whether the logical flow of inferences is coherent and justified. This includes identifying unsupported jumps in logic or contradictions.
Each criterion ((m))
eval (sk,t) is rated as follows:
+1,
if the content is entirely accurate according to model m;
nomil (sk.1)
0, the content is ambiguous or partially accurate;
-, if there is any clear inaccuracy.
We preserve high-quality paths by pruning the candidate set. Specifically, we retain any node sk, whose aggregated score across all evaluators and criteria is non-negative, i.e., St {sk, R(s,,)) 2 0}. This threshold is chosen empirically balance filtering out incorrect steps while maintaining adequate reasoning diversity.
D Training Details
D.1 Implementation Details
SpatialReasoner-RI is built upon the Sa2VA architecture [89], which is based on Intern VL2.5. We train the 8B-parameter model in two stages on two NVIDIA H100 GPUs, each stage taking approximately 2.5 days. For supervised fine-tuning, we employ AdamW optimizer with a learning rate of 10-3 weight decay of 0.05, and a 5% linear warm-up schedule, using a batch size of device with gradient accumulation over 4 steps. For Direct Preference Optimization, we similarly use Adam V with learning rate of 10-7 weight decay of 0.05, and 5% warm-up, training with batch size of per device.
19

---

Redgonee from Ques? 5x1-728
Questie: inagar'n fot you setiratioon selweas the cemere of slegiel deginnd?
Resporee
Ground
Resporss from Seriiis Fro
ReRDbee phine
Chairil of te poipe toe
Daracce laquore cant pored cInotcr
Cld the ponder tapeess The
up Acc
The werer bertoe other den mvee krouply Dron
dnsover interen
shop
by #AilShep
motk co boe Lngien
cepertice for
asard and
WIRHTT Canchsiee
Ard
Figure Example Reasoning Tree from the M3CTS Data Generation Pipeline. Diverse candidate reasoning paths are sampled from multiple models. Each path follows structured LongCoT format with markdown-style section headers that decompose the answer into interpretable reasoning stages.
D.2
Training Data
For SFT, we convert samples from the OPEN SPATIAL dataset [13] to reasoning chains using the M3CTS pipeline. While the original OPEN SPATIAL dataset provides single-sentence answers, transform 400K samples, grounded in distinct images, into structured LongCoT reasoning chains, where examples are used teach the model to generate high-quality, step-by-step spatial reasoning responses. For Direct Preference Optimization (DPO) training, the goal is to train the model to distinguish high-quality spatial reasoning from suboptimal or subtly flawed alternatives. To this end, we utilize our OPEN SPATIAL REASONING dataset, described below, that consists of spatial reasoning preference pairs.
An additional set of 100K challenging negative pairs is meticulously crafted by perturbing only the conclusion keywords of high-quality positive samples. Each original response represents coherent and accurate reasoning path with a factually correct outcome. To create the corresponding negative sample, we retain the exact description and reasoning segments and alter only the final conclusion value. This yields tightly controlled preference pairs that isolate correctness at the conclusion level. For example, a positive sample may assert 'The distance between region1 and region2 is 11 meters." ", while its negative perturbed counterpart is "The distance between region1 and region2
meters."
Our method adopts a data-centric strategy that emphasizes high-quality supervision and reasoning diversity. Instead of collecting large volumes of weakly aligned or noisy data, curate training examples using the M3CTS sampling strategy guided by structured reward evaluations. By applying baserd-based filtering, reduce noise and enforce a consistent output structure. In parallel, using multiple VLMs during generation introduces variation in reasoning styles, improving coverage of diverse spatial patterns and edge cases. The effectiveness of this approach is evident in the substantial performance gains of DPO-trained models over their simpler SFT counterparts (Table 1) and the reasoning improvements and diversity depicted in Figure 5.
D.3
Open Spatial Reasoning Dataset
We curate the OPEN SPATIAL REASONING dataset, collection of 400K Vision Question Answering (VQA) preference pairs (Up- 31). to support training of preference-basse spatial reasoning models. This dataset is derived from the OPEn SPATIAL dataset [13], which provides image- image-based based spatial questions paired with ground-truth answers and offers 10 question variations per grouneig scenario. To construct each preference pair, we randomly sample question instance from the source dataset, and generate diverse pool of eight candidate answers using four distinct sources: our M3CTS pipeline, Gemini 1.5 Pro, GPT-4o, and our SpatialReasoner-R R1 Supervised Fine-Tuned
20

---

If you are of (eegion2>, where will you find <region1>> >?
Rejected Response: imgse shows street scone with buildings
The image shows o street interoection withs ieegenn: narked vehicle to the and iregiend> marieed on person the right megionnz located right add. Looste enegioni engionls the right side the street, and furt her cown the rood. Step 3: Compore this direction to clock face. algning with the o clack direction oin The final unswer region (P) find region ( (1) around the clock direction
Chesen Responee:
of street, the sideowikk the right side the streat Let trink step by shop Step 1, Imagin shonding the lacation of iregion2: which the car on the left side of street Step Look cround for reeiinnds which is person walling will find region [1]
which
image, iregionn- Step side the
enegion::
sidewalk, Step The region
direction
Region
Region [2],
height?
Chosen Rexponse: The image shows collection of bottles a table, Regiortil is grean bottle
Rejected Response The imsge aloow collection of bottles on table Region[]] is green bottle right, Region [apport
left. right Ragion [2] ls slue
heghts aurrounding midie
compared by
Rugion the heights vauully. Region]3 is naticnably sharter than the surrounding botfles. The find anower Region(1]
think
Let'sthin
bottless have similer heigt Rey on[2] Snap Region((] is taller than middle bottlees The Final answer Region[2]
What distance
<region1> and <region2>?
Chasen Responce: The insge shows two haxis in front of Trump Tower. Region]]] [ is yellow taxi on the right, and Region(2) is blu taxt the left. Typical read about nerter, each aspraximatily stop by step. Step backs the line. Shep 2: With woch trxi being 1.5 meterns wide and the rood meters wide, the space betwoon them about 1.5 methre. The Finel onswer it. 15 neters.
TROMP TOWER
taxis front
of Trump Tower Region
yellon taot
and Reglor(2] [2] is blue toxi on left. The toxts one parked clese'y sids by side Let's a th step by stup. Step Regiont and Region are od with parking viaille space Step Gives typioal Asproxinctely
spacing.
<region1> positioned of right of <region2=?
the vehicle seflecting on the blue Let' highlighted meed think stop by stop. Step I: Identify the postrions [egion]3] and Region[2] in the rrrr Stepž: Region]]] or the left side. Since the mage is shown mirvar, Region(l] cragionts R2gion(23
Rejected Response: image shows vehicle Region(?]
scene Regiot(1] highlighted
Identify
sdds. Stap2: Region is the right side. The find answer is No angi post rioned left eragin2).
and Region Ragien[1]
right
right rregian2:
Figure 6: Example DPO Pairs of our OPEN SPATIAL REASONING Dataset, constructed from M3CTS-generaten reasoning trajectories. Each pair consists of preferred and rejected response to the same spatial question. The examples highlight differences in descriptive accuracy, spatial alignment, and reasoning coherence, which guide preference optimization during training.
(SFT) model, with each method contributing two response variants. All eight candidate responses are independently evaluated by our fine fine-grained spatial reward mechanism (Appendix A). The highest- scoring response is selected as the preferred answer (Up), while the response with the lowest score is designated the less-preferred (wi), ensuring that each preference pair is anchored in meaningful fine-grained spatial reasoning quality. Figure 6 shows dataset examples.
E
Evaluation Details
RGPT-BENCH [13], a benchmark specifically designed assess the 3D spatial reasoning abilities of VIMM,, featuring 657 qualitative and 749 quantitative VQA pairs, covering object classes across diverse environments. We employ the same GPT-4 evaluation proposed in SPATIALRGPT-D BEERGGTT--NNCCE [13] for evaluating the free-form responses generated by the models. For qualitative questions, GPT-4o assesses the semantic alignment between the model's response and the ground-truth answer, assigning a binary score (1 for correct, 0 for incorrect). For quantitative questions (e.g., distance, size), GPT-4o first extracts numerical values from both the prediction and the ground truth, standardizing them to common unit (meters). We then compute accuracy (e.g., success rate defined as predictions within +25% the ground truth).
evaluate on
We also evaluate on several general vision vasion-language benchmarks to provide comprehensive assess- ment of SpatialReasoner-RR capabilities. Specifically, we use MME [34] to assess multimodal models perception and cognition tasks across wide range of domains. POPE [34] is employed
21

---

Q: Can you give me an estimation of the horizontal distance between Region [1] and Region [2]?
Ground Truth
Response from SpatialRGPT BB
The horizontal distonce of Region [1] from Region [2] is 12, meters,
PENSKE
Response from InternVL2.5-788
Step 1: estimate: the herizontel distance between Region [t] and Region (2), we can use the trucks reference points.
Regiond
Step 2: The everage longth of a rental truck is around 4 meters.
Step 3: The space between the two trucks oppears to be slightly more. than the length of one truck, By adding the space to the length of the trucks, we can approximate the distonce.
Regioni
Region Region 16.58 meters apart horizontally.
Step Therefore, harizontal
between Region Region
[2] approximately 1D
Response from Gemini Pro
Response from SpatialReaasoee-ft 8B
Step Consider the of the person (Region 2) is the sidewalk, and the truck (Region Neterrooee on the street.
sidewalk.
Step1: Region (truck) and Region (pedestrion) one seporated by the Penske truck and some sidewalk
Step 2:To estimate horizontal distonce, we con sum the widths of main
Step
The
Penske truck looks about meters
traffic lanes. Let's consider three distinct troffic lones, each appraximated 4 meters wide, which total is 12 meters
width,
lone plus truck about
meters.
Step 3: Estimate the distonce of roodside or shoulder area between the person road. And about
Step The sidewalk adds another 1.5-3 meters. Thus, Region and Region roughly 10 meters horizontolly.
Step 4:
Step 4: Therefore, combining the 12 meters from the traffic lones with this 4meter roadside area ylelds a total estinated distance of 16 meters.
demon- strates coherent, step-by-step spatial reasoning that closely aligns with ground truth estimates. In contrast, baseline models produce less precise or partially incorrect reasoning steps, often neglecting key visual cues misestimating spatial references.
Figure 7: Qualitative Examples of Spatial Reasoning Across Models. Spatial
to evaluate object hallucination in testing the ability of VLMs ground responses to visual content, while SEED-BENCH [30] offers a multi-dimensional evaluation, covering aspects from image under- standing complex reasoning across various modalities and tasks. We further utilize A12D [27], benchmark focusing on diagram understanding and reasoning, which requires parsing visual elements and their relationships within schematic representations, SQA [42] is used measure the model's ability to answer science-related questions based on visual context, often requiring domain-specific knowledge and reasoning. MMMU [92] evaluates massive multi-disciplinary multimodal under- standing and reasoning across diverse college-level subjects. Moreover, MMSTAR [9] provides challenging benchmark with meticulously curated, multimodal instances that require advanced reasoning, low hallucination, and resistance to leading questions. Finally. HALLUSION EITOOBENCE BENCH [23] specifically designed to quantitatively measure and analyze the hallucination phenomena VLMs. probing for both object-level and attribute-level inconsistencies.
F Qualitative Experiment Examples
In this section, we provide additional qualitative experiment examples. Figure 7 shows a question that requires estimation of the horizontal distance between truck and pedestrian. SpatialReasoner-Rl demonstrates a clear advantage by decomposing the scene into semantically meaningful components, explicitly reasoning over the widths of multiple traffic lanes, the roadside, and the sidewalk. This results in estimated distance that closely matches the ground truth and provides full transparency into the model's 's stepwise deductions. In contrast, Intern VI2..--78B bases its answer primarily on the width of the trucks and the space between them, omitting the crucial step of accounting for the distance from the pedestrian to the roadway, which leads to significant underestimation. Gemini correctly recognizes that the separation includes the truck, traffic lane, and sidewalk, but substantially underestimates the width of the sidewalk, causing a notable error in its final answer. Meanwhile, SpatialRGPT-85 provides more accurate estimate than Gemini or IntemVL2..5-8BB. but still has gap compared to the ground truth. Most importantly, it cannot generate step-by-step reasoning traces.
Figure 8 presents another illustrative example evaluating spatial reasoning capabilities of various models, specifically focusing on size comparison between two highlighted image regions. The question is whether Region 1 (a computer monitor) appears smaller than Region 2 (a computer tower). SpatialR R1 accurately identifies Region as a computer tower and explicitly reasons by comparing Region I with the closest computer tower positioned adjacent to the monitor. This
22

---

Q: Does <region1> have smaller size compared to <region2>?
Ground Truth
Response from SpatialRGPT 8B
No, rregionb is not smaller than megion2:
Regren2
Response from InternVL2.55 2.5-778
Step The image shows office
with several
computer monitors and towers. Region] is highlighted around a monitor, while Region2 highlighted around computer tower.
Step The monitor's width around 16-24 inches, and its height about 10-15 inches.
Regiord
Step standard computer tower might around 10-14 inches
wide and 16-20 inches tall.
Yes, Region1 has a smaller size compared to Region2
Step 4: No, Region1 does not have smaller size compared to Region2.
Response from Gemini 1.5 Pro
Response from SpetialReasone-FI
Step The image shows office setting with computer workstations. Regiont highlights blue computer monitor on desk in the foreground, Region? highlights green, translacent object. possibly plant or decorative item, on different desk further and to the right.
image shows a computer lob sevendl desks and computer setups. Regionl labeled computer monitor, and Region? is lobeled computer tower. The montor appears snaller than the tower. Step 2:Compare the sizes the monitor (Ragionl) and the computer towers (Ragion2 and the one next Regiond). Monitors are typically smaller than computer towers.
Step 1:
Step 2:Compure the apparent size the object within Region! object
RegionZ. The computer monitor in Regionl appeors larger than the decorative ab object in Region2.
apporent
Step 3: Yes, Region1 has e smaller size compared to the computer towers next to including Reglon?.
Step 3: No, Reglon] does not have smaller size compared Reqion2.
Figure Qualitative Examples Spatial Reasoning Across Models. SpatialReusoner-R R1 correctly recognizes Region2 as computer tower and compares it clearly with the nearby monitor, reaching an accurate conclusion. V1.25mV1.2.5-788 relies on general object size knowledge but provides incorrect reasoning, Geminil 1.rPo fails to identify Region2 clearly and draws incorrect visual conclusions, while SpatialRGPT-8B directly provides a wrong answer.
systematic visual grounding and clear comparative reasoning enable SpatialReasoner-R to correctly conclude that the monitor is indeed smaller than the tower. By contrast, the baseline models exhibit varying degrees of errors and reasoning inadequacies. InternVL.2.554 general knowledge about typical object dimensions and incorrectly concludes the monitor is not smaller, without effectively validating this against the visual evidence provided. textb/Geminil.5 1.5Pro fails entirely recognize what object Region 2 represents, causing to inaccurately rely purely on the objects' visual proximity and perspective, leading to an incorrect conclusion. Lastly, the SpatialRGPT-SBB model directly presents an incorrect judgment ("Region I is not smaller") without providing any interpretable reasoning steps or visual grounding.
relies significantly on prior
We also provide failure example in this section. Figure 9 illustrates representative failure case on vertical size estimation in an indoor setting. The query is "How tall is Region 1?". Here, Region corresponds the dresser mirror the right, adjacent SpatialReasoner-Rt produces an estimate of approximately 2.0m, while the ground truth is closer to 1.5m. Our model's reasoning proceeds follows: (1) segments Region (the mirror) and searches for nearby object of familiar scale, (2) identifies the bed headboard and assumes typical headboard height of 1.5m, further estimating that the mirror extends about 0.5m above the headboard, and (3) sums these values to obtain about 2.0m. This етrоr arises from overreliance on default furniture priors rather than fully grounding the estimate in image evidence, such the mirror's 's vertical extent relative to the floor plane and its contact points with the dresser. To mitigate this, training incorporates fine-grained reward signals that explicitly reward consistency between predicted measurements and image-derived cues, encouraging verification of intermediate steps (e.g., floor contact, vanishing-lin alignment) before finalizing a measurement.
sleigh-style bed headboard
the left.
G Broader Impacts
This work aims to improve the spatial reasoning capabilities of vision-language models through fine-grained preference optimization. Accurate spatial understanding is critical for downstream applications such as roboties, autonomous navigation, assistive technologies, and visual analytics. By introducing more interpretable and structured reasoning mechanisms, our method can contribute to building Al systems that are safer, more transparent, and more aligned with human expectations in
23

---

Q: How tall Region 1?
Ground Truth
Response from SpatialReasonen-R1
Step The image shows furnished bedroom with dresser mirror on the right (RegionI) and a sleigh-style bed headboard the left. Regionl is labeled the dresser mirror, which appears adjacent the bed. Step 2:Compare the height the mirror (RegionI) with bed headboard nearby. Bed headboards typically have standard height of around 1.5 m. The mirror oppears to extend approximately 0.5 toller than the headboard.
Regord
Step Region approximately 0.5 m additional height m total.
headboard height
Yes, the height of region1 about 1.5m.
Figure 9: Failure case on height estimation in furnished bedroom.
spatially grounded tasks. However, as with other vision-language systems, potential risks remain. If deployed in safety-critica domains, incorrect spatial inferences, especially in edge cases, could lead to unintended consequences. Additionally, reward scoring and generation rely on foundation models that may encode hidden biases, which can propagate through the training pipeline. Although we attempt to mitigate these risks via multi-source sampling and structured evaluation, future work should explore robustness to distribution shifts, adversarial spatial prompts, and the inclusion of human-in in-the--loop verification for high-stakes use cases.
H
Limitations
While our work demonstrates strong improvements spatial reasoning, limitation of our approach is its reliance on explicit region representations provided as input to disambiguate object references within the spatial queries. Enabling the model to implicitly ground entities solely based on natural lan- guage descriptions remains avenue for future investigation, which would enhance the model's flex- ibility in real-world scenarios. Future work could focus on integrating implicit linguistic context un- derstanding to alleviate this constraint. Finally, our focus is limited 2D spatial reasoning: extending this framework to 3D or embodied contexts would require structural adjustments left for future work.
24

---

System Prompt for LongCoT Reward Evaluation
The following is spatial reasoning task, and this is the question: question on and the ground truth is: ground, truth. The response divided into different sections. There are 4 dimensions to evaluate, and will provide you with the corresponding image and text for reference. You will need evaluate the response based on the following criteria:
The first task: Descriptive Scoring (Total 0-4.0 points) p-ints
Evaluate the "Description" section based on:
Existence: Assign scores from to 1.0, where means mostly confidently correct and means mosily confidently incorrect. Does the description correctlyy identify objects that actually appear in the image?
Attribute Accuracy: Assign scores from 0 to 1.0, where means mostly confidently correct and means mostly confidently incorrect. Are the object's attributes (color, shape, size, etc.) described accurately?
Completeness: Assign scores from to I.0, where 1.0 means mostly confidently correct and means mostly confidently incorrect. Does description include key objects and nocessary details relevant the question?
Assign scores from to 1.0,, where 1.0 means mostly confidently correct and means mostly confidently incorrect. Does the description focus on the core aspects of the question?
Appropriatenes::
Clearly state the score for each sub-category and sum them to obtain the final descriptive score. You need to give the score with the following format: {"taski_acors = your score}
The second task: Depth-Guided Spatial Relationship Scoring- Description (Total 0-4.0 points)
Evaluate all spatial statements within the "Description" section using the provided depth image ground truth.
For each spatial claim in the description:
Correctness score: Assign I if the spatial claim correct based on the depth image, and not correct.
Uncertainty score: For claims expressed with uncertainty (using words like "pproximately", "roughly", "
"possibly"). ", assign score from to 1.0, where 1.0 means statement is expressed with high certainty.
Relationship score: Assign a weight from 0.8 to 1.0 based whether the relationship is explicitly emphasiize by the question (1.0) or is extra/irreevantt information (0.8).
Provide a detailed breakdown for each spatial claim. Calculate the final score as: (Sum of (Correctness score x Uncertainty score x Relationship score)) / (Number claims), then scale to 4.0.
You need give the score with the following format: ("task2_ (laaa2_clain_____________a clain_score":
[Correctness score, Uncertainty acore, Relationship score])
The third task: Depth- Spatial Relationship Scoring - Reasoning (Total 0-4.0 points) Apply the sume evaluation method Task 2 to the spatial statements within the "Reasoning" section.
For each spatial claim in the reasoning:
Correctness score: Assign I if the spatial claim is correct based on the depth image, and if not correct.
Uncertuinty score: For claims expressed with uncertainty, ussign score from 0.8 to 1.0, where 1.0 mcans the stutement is expressed with high certainty.
Relationship score: Assign a weight from 0.8 to 1.0 based on whether the relationship is explicitly emphusized by the question (1.0) or is extrufirrlevant information (0.8).
Provide detailed breakdown for each spatial claim. Calculate the final score as: (Sum of (Correctness score Uncertainty score Relationship score)) / (Number of claims), then scale to 4.0.
You need to give the score with the following format: ("taak3_ Baakk_ccliinn. clain_scoree. [Correctness score, Uncertainty score, Relationship score]}
The fourth task: Reasoning Scoring (Total 0-4,0 points) Evaluate the "Reasoning" section (the chain-of-thoughtt based on:
Factual Consistency: Assign scores from 0 to 1.0, where 1.0 means mostly confidently correct and means mostly confidently incorret. Are the claims consistent with the image, depth image, and the earlier description?
Logical Coherence: Assign scores from to 1.0, where 1.0 means mostly confidently correct and means mostly confidently incorrect. Do the rensoning steps flow logically without gaps contradictions?
Correct Application of Rules: Assign scores from to 1.0. where 1.0 means mostly confidently correct and means mostly confidently incorrect. Are physical, spatial, and logicul rules upplied correctly?
Conclusion Validity: Assign scores from 0 to 1.0, where 1.0 means mostly confidently correct and means mostly confidently incorrect. Does the reasoning properly support the final answer?
Clearly state the score for each sub-category and them obtain the final reasoning score. You need to give the score with the following format: ("cask4_score _score": your score)
System Prompt for Evaluating LongCoT Spatial Reasoning w.r.t. descriptive accuracy, spatial alignment, and logical consistency of reasoning steps.
Figure 10:

---

NeurIPS Paper Checklist
1. Claims
Question:
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: We claim to design DPO algorithm for LongCoT optimization, and reward method, and we verify improvements with extensive experiments.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification: Appendix H is about the limitations.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
26

---

Justification:
We do not include theoretical assumptions and proofs.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: Yes, everything is reported, and we will release our data. We provide imple- mentation details in Appendix.
Guidelines:
The answer NA means that the paper does not include experiments. If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
27

---

Answer: [No]
Justification:
We will open source our code at a later date.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https: /nips. htc///blc//guide/////eeee httee/guudeeeeedeebeeeeeeeeeeeee /at////ooommsssseeeeeee cassionnocccyy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Question:
Answer: [Yes]
Justification:
In Appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: The results are provided accuracy the test set. And we have explained how they are calculated in Appendix E.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the enror bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
28

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
In Appendix D.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't 't make it into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https: http::://arrpps.ccppbtccc//e/ee csGuidel ines?
Answer: [Yes]
Justification:
We have reviewed the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
In Appendix G.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no socictal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
29

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of MI..
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
This paper poses such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Lieenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: Yes. We acknowledge the codebases.
Guidelines:
The answer NA means that the paper does existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
30

---

If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: This paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/mmoel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question:
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
31

---

Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that if the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, or originality the research, declaration is not required.
Answer: [NA]
Justification: The core method development in this paper does not involve LLMs any important, original, or non-standard components.
Guidelines:
The answer NA
that the method development in this research does involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy (https: (bttps://netll /ttp:///enuu.eeeeeeeeeee http://uriss.cc/ooeeeeeeeeeeeeee http://////eeeeeee//2///////1//0 for what should or should be described.
32


Reviews:
[{"review_id": "BMnIC5SWbn", "reviewer": "Reviewer_G8jw", "summary": "This paper introduces a reasonable VLM training and architecture pipeline designed for improving fine-grained spatial reasoning capabilities. Their main contributions are: 1) Fine-grained Direct Preference Optimization (fDPO) technique that decomposes reasoning outputs into descriptive and logical components and optimizes them with segment-specific learning signals, 2) Monte Carlo Tree Search M3CTS to generate diverse, high-quality Chain-of-Thought reasoning paths for spatial tasks, 3) Spatial Rewards framework that evaluates outputs along visual consistency, spatial grounding, and logical coherence dimensions. They also provide rich experiments showing good results on the SpatialRGPT benchmark and competitive performance on other vision-language understanding datasets.", "questions": "1. What is the computational overhead of M3CTS compared to standard preference data generation pipelines? \n2. Could you add examples where SpatialReasoner fails or produces erroneous reasoning, especially in scenes with occlusions or ambiguous spatial relations?", "limitations": "1. It is better to add a discussion on potential biases in generated reasoning traces due to reliance on multiple LLMs.\n2. What is the robustness of depth-based spatial rewards under complex scenarios?", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n1. This paper is well-written, and the figures clearly demonstrate the methods.\n2. They propose some relatively novel and reasonable techniques, such as fDPO, M3CTS, and their reward design, which show good improvement to the LLM/VLM framework.\n3. The experimental results are good - they present and claim the SOTA results on SpatialRGPT, and the good results on other tasks and benchmarks. Besides, they provide an extensive ablation study and qualitative results.\n\nWeaknesses:\n1. It seems like they didn't consider the related methods using RL tuning methods.\n2. It is not clear how the failure cases happened, and it lacks analysis of the failure reasoning.\n3. Considering the complexity of the M3TCS and the proposed hyperparameters, it is unclear whether it will overfit to the training dataset.", "was_revised": true, "final_justification": "This paper is generally well-written, and the proposed method is effective. Their rebuttal addressed my major concerns, although some minor ones remain. I carefully read their rebuttal and other reviewers' comments - I decided to mark it as weak/borderline accept.", "rebuttal": "We thank the reviewer for their constructive feedback. We appreciate that the reviewer found the paper to be well-written, and recognized the key contributions and novelty of our proposed methods, as well as the strength of our experimental results, including the SOTA performance on SpatialRGPT, comprehensive ablations, and generalization across broader V+L tasks. We are encouraged that the reviewer found our qualitative analyses informative. We hope our responses and additional analyses have meaningfully addressed all remaining concerns.\n\n> **Q1: It seems like they didn't consider the related methods using RL tuning methods.**\n\nWe thank the reviewer for the suggestion. We performed additional analysis comparing standard DPO with our proposed fDPO and a recent preference optimization method, mDPO \\[1\\], that extends preference optimization to the multimodal domain by directly modeling preferences conditioned on both images and responses, a capability that is essential to spatial reasoning, where visual grounding is critical. However, spatial reasoning tasks also require more *localized*, fine-grained perception and step-wise inference to accurately capture spatial relationships between objects in the image (such as up and down, left and right, front and back, etc.), which mDPO does not explicitly model. Our fDPO method addresses this due to its fine-grained, segment-specific learning signals that separately supervise spatial description and logical inference, leading to more targeted supervision and improved performance on spatial relation benchmarks. The comparison below demonstrates that our method advances beyond the common RL-based DPO tuning. \n\n|  | Below/Above | Left/Right | Big/Small | Tall/Short | Wide/Thin | Behind/Front | Qual Acc |\n| :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |\n| DPO 8B | 94.16 | 93.33 | 89.62 | 90.18 | 88.64 | 92.27 | 91.48 |\n| mDPO 8B | 95.00 | 95.24 | 90.57 | 91.96 | 87.50 | **93.64** | 92.39 |\n| fDPO 8B | **98.33** | **98.10** | **95.28** | **96.43** | **91.34** | **93.64** | **95.59** |\n\n\\[1\\] Wang, Fei, et al. \"mDPO: Conditional Preference Optimization for Multimodal Large Language Models.\" Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP). 2024\\.\n\n>**Q2: Considering the complexity of the M3TCS and the proposed hyperparameters, it is unclear whether it will overfit to the training dataset.**  \n\nThank you for the comment. M3CTS is an offline data generation pipeline that produces high-quality and diverse LongCoT reasoning paths across a wide spectrum of spatial problem types (e.g., distance, size, relative position), improving coverage of spatial patterns and edge cases. By avoiding confined problem structures or templates and encouraging exploration of varied reasoning strategies, M3CTS inherently reduces the risk of overfitting during data construction. In parallel, the use of independent reward models and segment-level optimization in fDPO further mitigates overfitting to annotation noise or specific reasoning styles. \n\nThe absence of overfitting is empirically supported by our results in Table 1: on SpatialRGPT-Bench, our model shows significant and consistent performance gains across all spatial sub-tasks after fDPO training, demonstrating generalizable spatial reasoning capabilities. Moreover, the model exhibits strong generalization across 8 external vision-language benchmarks (Table 2), further confirming that it does not overfit to the training distribution.\n\n>**Q3: What is the computational overhead of M3CTS compared to standard preference data generation pipelines?**  \n\nThe computational overhead of the M3CTS offline data generation is comparable to standard preference data generation pipelines. The generation time for a response from one of our sources, Gemini 1.5 Pro, is identical within our pipeline as it would be if generated directly. The only added overhead comes from the selection step among the concurrent responses from our different models, which takes **under three seconds per sample** and is fully parallelizable. Since our entire pipeline is built for parallel processing, the end-to-end overhead is negligible in practice. We note that M3CTS is an offline data generation process and thus does not affect inference efficiency.\n\n>**Q4: It is not clear how the failure cases happened, and it lacks analysis of the failure reasoning. Could you add examples where SpatialReasoner fails or produces erroneous reasoning, especially in scenes with occlusions or ambiguous spatial relations?** \n\n Thank you for the insightful suggestion\\! We notice failures occur when the model misapplies prior knowledge instead of relying on the visual evidence presented in the image itself.\n\nFor example, in a furnished bedroom scene, Region 1 is the dresser mirror on the right, adjacent to a sleigh‑style bed headboard on the left. Our model’s pipeline for “How tall is Region 1?” proceeds in three steps: 1\\. The model segments the mirror mask (Region 1\\) and then searches nearby objects to find a familiar item for scale. 2\\. It recognizes the bed headboard, assuming the typical headboard height of 1.5 m and estimating the mirror to be 0.5 m taller. 3\\. It adds 1.5 m and 0.5 m to output ≈2.0 m. However, an actual dresser mirror in this context is closer to 1.5 m tall. \n\nThe root cause of this failure is that the model overrelies on default furniture heights rather than grounding its measurement in visual cues in the image, such as the mirror’s height relation to the floor. This is why we incorporate fine‑grained reward signals during training that explicitly reward consistency between predicted measurements and image‑derived evidence to encourage the model to verify its predictions against the actual scene rather than default priors.\n\nAs we cannot include images per conference rebuttal guidelines, the qualitative image described above and additional failure case analyses will be included in a new Appendix section (before Appendix H: Limitations) in the camera-ready version. To confirm its inclusion and reference the correct asset, we provide its MD5 checksum: 51b759a689433fb598ebf4a165b91d88. We sincerely appreciate this suggestion. \n\n>**Q5: It is better to add a discussion on potential biases in generated reasoning traces due to reliance on multiple LLMs.**\n\nWe thank the reviewer for raising this point. While our original dataset did not explicitly log the source model of each generated response, we have now conducted an additional analysis using 1,000 randomly sampled image-question pairs. Among all reasoning nodes, 49% of nodes were selected from Gemini, 42% from Qwen2.5VL-72B, and 9% from Qwen2.5VL-7B, indicating that no single model dominates the reasoning process and that the ensemble encourages diversity and calibrated supervision.  \nWhile Gemini contributes more frequently in the overall reasoning process, given its strong performance on spatial tasks, its responses are not always accurate; therefore, we incorporated Qwen2.5VL-72B to enhance the overall diversity and reasoning robustness. Although Qwen2.5VL-7B accounts for the smallest proportion of contributions, we intentionally include this in M3CTS because the model we train is closer in size to 7B. This way, we calibrate the reasoning complexity to the capacity of our target model, improving alignment between data and model capabilities \\[2\\].\n\n\\[2\\] Stanton S, Izmailov P, Kirichenko P, Alemi AA, Wilson AG. Does knowledge distillation really work? NeurIPS 2021 \n\n>**Q6: What is the robustness of depth-based spatial rewards under complex scenarios?**\n\nThank you for this excellent question. The robustness of our depth-based spatial reward ($R\\_{sp}$) in complex scenes, e.g., those with occlusions or clutter, is achieved via two mechanisms (Appendix B, Fig. 8): \n\n(1) Uncertainty weighting $W\\_{u}$ provides robustness in cases of occlusion. For example, when an object is (partially) hidden behind another, the depth map for the occluded parts, as derived from the image, often shows ambiguous or missing depth cues. $W\\_{u}$ increases when the model detects such ambiguities in the depth map, e.g., sharp discontinuities or high variance in predicted depth. This ensures that the reward remains stable and does not improperly penalize the model for the inherent uncertainty in inferring occluded 3D structure from a single 2D image.\n\n(2) Context-aware weighting $W\\_{c}$ ensures stability in scenes with dense clutter by focusing the reward computation only on spatial relations or entities directly relevant to the query (as parsed by the model), and down-weighting peripheral inaccuracies involving unrelated scene objects. For instance, when asked about the distance between a table and a chair in a crowded scene, $W\\_{c}$ ensures small errors about background entities (e.g., a plant in the corner) do not dominate the reward signal, enabling the model to learn from more targeted, relevant feedback."}, {"review_id": "UqbbrebyX3", "reviewer": "Reviewer_a5xU", "summary": "The paper introduces SpatialReasoner, a VLM aimed at improving fine-grained spatial reasoning, especially in tasks requiring multi-step logic and precise spatial alignment. Existing VLMs often rely on shallow, direct-response outputs and struggle with complex spatial queries. SpatialReasoner addresses this by generating interpretable LongCoT directly from 2D images without extra sensor input.\n\nTo support training, the authors propose two components: a Multi-LLM MCTS (M3CTS) method to collaboratively generate diverse, high-quality reasoning traces using multiple VLMs, and (2) a Fine-Grained Spatial Rewards mechanism that evaluates responses based on descriptive grounding, spatial precision, and logical coherence. These are used to build a high-quality dataset for preference learning.\n\nThe model is trained using a fine-grained Direct Preference Optimization (fDPO) algorithm, which separates learning signals for different reasoning segments—focusing on spatial localization during description and on logic during inference. Experiments show SpatialReasoner outperforms prior methods on spatial reasoning benchmarks, while also maintaining strong general vision-language capabilities.", "questions": "1. What is the base model architecture used for SpatialReasoner?\n\n2. Would incorporating a broader range of vision-language models during the LongCoT generation phase further improve performance? Additionally, is there any statistical analysis on which models most frequently contribute to the final selected reasoning paths during the M3CTS process?", "limitations": "Yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "## Strengths:\n\n1. The paper presents a novel and effective approach for generating LongCoT data for spatial reasoning by leveraging a Multi-LLM Monte Carlo Tree Search (M3CTS). The idea of using collaborative exploration across multiple vision-language models to enhance the diversity and quality of reasoning paths is interesting.\n\n2. The introduction of a fine-grained Direct Preference Optimization (fDPO) method is well-motivated and addresses a clear limitation in existing DPO frameworks. By disentangling descriptive grounding from logical reasoning during training, the proposed method enables more targeted learning and demonstrates notable performance gains on spatial reasoning benchmarks.\n\n3. The paper provides sufficient technical detail in both the main text and the supplementary materials. This includes algorithmic descriptions, architectural choices, and training procedures, which contribute to potential reproducibility of the work.\n\n4. The experimental evaluation is thorough. The authors benchmark their method on standard datasets and show consistent improvements over prior state-of-the-art baselines. The ablation studies, particularly those analyzing the effects of the λ and α hyperparameters in fDPO, offer valuable insights into the method’s effectiveness.\n\n5. The paper includes well-chosen qualitative examples illustrating different components of the proposed pipeline, such as the M3CTS-generated reasoning paths and sample model responses. These examples help to demonstrate how the system performs in practice.\n\n## Weaknesses:\n\n1. The related work section in the main paper is overly brief and lacks sufficient coverage of key areas relevant to the proposed method. Much of the discussion is deferred to the supplementary material, which gives the main submission an incomplete appearance. A more comprehensive review should be included in the main text to properly situate the contribution within the broader literature.\n\n2. While the textual description of the method is generally clear and well-structured, the figures—particularly Figure 1 and Figure 2—are difficult to interpret. These visual aids could be significantly improved in terms of layout clarity, labeling, and flow to better support the reader’s understanding of the overall architecture and training pipeline.\n\n3. The contribution of using multiple vision-language models in the M3CTS framework for generating LongCoT data is unclear. It remains unclear how much additional benefit the multi-model setup provides compared to using a single model.", "was_revised": true, "final_justification": "I've read the rebuttal by the authors, and my concerns are reasonably addressed. In this case, I'm keeping my original evaluation as borderline accept.", "rebuttal": "We thank the reviewer for their constructive review and recognition of our paper’s novel, well-motivated contributions that address limitations of existing DPO frameworks.  We are encouraged that the reviewer recognized our method’s strong performance across spatial reasoning benchmarks, the thoroughness of our ablation studies, the paper’s clear technical exposition, and the utility of our qualitative examples for illustrating the practical impact of our work. \n\n>**Q1: Related work section in the main paper is overly brief. Much of the discussion is deferred to the supplementary material.**   \n\nThank you for this valuable feedback. Due to space constraints, we focused the Related Work section on the most directly comparable areas and provided a broader discussion in Appendix C, including related efforts in spatial reasoning, preference optimization, and MCTS. That said, we agree that a more integrated discussion in the main paper would improve clarity and better situate our contributions. We will revise the camera-ready accordingly.\n\n>**Q2: While the textual description of the method is generally clear and well-structured, the figures—particularly Figure 1 and Figure 2—are difficult to interpret.**  \n\nThank you for the constructive feedback. We have made an effort to identify specific strategies to improve both figures. We clarify below what each figure is intended to convey, as we sincerely would welcome any additional suggestions to improve their design:\n\n**Figure 1** illustrates our complete workflow, which is comprised of the SpatialReasoner architecture (left) and its three-stage training pipeline (right). The architecture itself is a VLM that takes an image, a text query, and a visual prompt as input to generate a reasoned response.  Stage 1 employs our offline M3CTS framework to generate a diverse pool of structured LongCoT reasoning paths. In Stage 2, our reward model assesses their quality, curating them into high-quality preference pairs. Finally, these curated preference pairs serve as training data for Stage 3, where the model is fine-tuned using our novel fDPO method, with distinct optimization parameters, $\\\\beta\\_{\\\\text {desc }}$ and $\\\\beta\\_{\\\\text {reason }}$ to separately improve the descriptive and logical components of the model's reasoning. \n\n**$\\\\diamond$ Figure 1 revision plan:** We plan to break Figure 1 into two separate figures, one for the model architecture and one for the training pipeline with a better left-to-right flow, add explicit labels to each component to clarify their roles, and use consistent and distinctive color coding across components to visually differentiate between inputs, modules, and outputs.\n\n**Figure 2** illustrates the fine-grained reward evaluation (Stage 2 in Figure 1).  Given an example question about the kitchen island's height, our reward mechanism decomposes reasoning quality into three independent axes: descriptive accuracy, spatial correctness, and logical coherence, as shown in the \"Reward Score Breakdown\" box.  \\`Resp2\\` is selected as the *positive sample* because it excels in all three: it provides a more detailed description (e.g., mentioning the bar stools), it uses a correct spatial reference (e.g., the stools) for reasoning, and it applies sound logic (an explicit $75 \\\\mathrm{\\~cm}+20 \\\\mathrm{\\~cm}$ calculation).  In contrast, \\`Resp1' is marked as the *negative sample* due to its incomplete description, flawed spatial claim (\"lower than the counter\"), and illogical \"half-height\" heuristic.  This granular, multi-faceted reward design is what allows us to identify specific reasoning failures and construct high-quality preference data for targeted model improvement.\n\n**$\\\\diamond$ Figure 2 revision plan:** We will reduce visual clutter in Figure 2 by simplifying textual descriptions in each response to highlight key contrasts without overwhelming the reader. We plan to also use a clearer layout separation between Resp1 and Resp2, and remove redundant elements such as the repeated spatial query (Question panel)\n\nWe welcome any additional suggestions from the reviewer to further improve our figure design.\n\n>**Q3: The contribution of using multiple VLMs in the M3CTS framework for generating LongCoT data is unclear. It remains unclear how much additional benefit the multi-model setup provides compared to using a single model.**  \n\nThank you for raising this important point. The use of multiple VLMs in M3CTS is a deliberate design choice to improve both the diversity and robustness of generated LongCoT reasoning paths.  \n\n**(1) Enhanced Diversity in Reasoning Paths:**  A single VLM is prone to getting trapped in homogeneous or low-quality reasoning paths due to its inherent biases and \"cognitive blind spots.\" By contrast, our multi-model approach mitigates this by introducing varied inductive biases and linguistic strategies, which encourage broader exploration of the reasoning space. Each model explores the problem space differently, which prevents the search from converging on a single, potentially flawed, reasoning style and improves coverage of edge cases in spatial reasoning.\n\n**(2) Ensemble-based Error Detection:** Our multi-model setup enables a robust, ensemble-based evaluation. As described in Equation (7) and Appendix D, each intermediate reasoning state is scored based on a collective judgment from all $M$ models on visual, spatial, and logical coherence. This process allows the ensemble to identify errors that a single model might miss, preventing the propagation of flawed logic that may go unnoticed in single-model pipelines.\n\nWe provide an analysis below to verify the contribution of our multimodel setup. Results below show that SFT models trained on M3CTS-generated data consistently outperform those trained with LongCoT data produced by a single model (Qwen2.5VL-72B) across all spatial reasoning categories, leading to accuracy gains ranging from 6% to over 10%, and raising the overall qualitative accuracy by 8.5 points (from 71.23% to 79.75%). We observe that M3CTS provides the largest improvements on spatial relations that are frame-dependent (e.g., *Behind/Front*, *Left/Right*) or visually ambiguous (e.g., *Big/Small*, *Wide/Thin*), where single-model reasoning often fails due to fixed biases or limited grounding.  The clear and consistent margin across all spatial tasks demonstrates that leveraging multiple VLMs substantially enhances both the diversity and reliability of the generated reasoning paths.\n\n| Relation         | Below/Above | Left/Right | Big/Small | Tall/Short | Wide/Thin | Behind/Front | Qual Acc |  \n|------------------|-------------|------------|-----------|-------------|------------|----------------|-----------|  \n| SFT with M3CTS data      | 81.66       | 81.90      | 75.47     | 75.89       | 79.80      | 83.63          | 79.75     |  \n| SFT with single-model data | 74.17       | 72.38      | 68.87     | 67.86       | 71.15      | 72.73          | 71.23     |\n\n>**Q4: What is the base model architecture used for SpatialReasoner?**  \n\nOur SpatialReasoner is built upon the Sa2VA architecture, which is based on InternVL2.5. As shown in Table 1, we benchmark our model directly against Sa2VA, as well as against a standard DPO implementation, to clearly isolate and demonstrate the improvements gained from our proposed methods.\n\n >**Q5:Would incorporating a broader range of vision-language models during the LongCoT generation phase further improve performance? Additionally, is there any statistical analysis on which models most frequently contribute to the final selected reasoning paths during the M3CTS process?**\n\nThat is a great question. The primary benefit of our multi-model approach is ensuring we capture diverse and robust reasoning paths. Any single model has inherent blind spots and may fail on certain types of queries. Using an ensemble makes it far more likely that at least one model will generate a valid LongCoT response. However, while it might seem appealing to include more VLMs during LongCoT generation, prior work suggests diminishing returns beyond a certain ensemble size \\[1-2\\]. This aligns with our practical findings, where we observed that benefits decline beyond our current 3-model ensemble in terms of performance vs. compute and API usage costs. Our curated setup provides enough reasoning diversity to build a high-quality dataset while keeping the generation process practical.\n\nWe have conducted experiments with 1,000 randomly sampled image-question pairs. Among all reasoning nodes, 49% of nodes were selected from Gemini, 42% from Qwen2.5VL-72B, and 9% from Qwen2.5VL-7B, indicating that no single model dominates the reasoning process and that the ensemble encourages diversity and calibrated supervision. While Gemini contributes more frequently in the overall reasoning process, given its strong performance on spatial tasks, its responses are not always accurate; therefore, we incorporated Qwen2.5VL-72B to enhance the overall diversity and reasoning robustness. Although Qwen2.5VL-7B accounts for the smallest proportion of contributions, we intentionally include this in M3CTS because the model we train is closer in size to 7B. This way, we calibrate the reasoning complexity to the capacity of our target model, improving alignment between data and model capabilities \\[3\\]. \n\nWe note that this is an excellent question that connects with foundational ideas in ensemble learning theory, hypothesis space expansion, and Bayesian learning, which have explored similar principles. A rigorous theoretical investigation of how ensemble-driven reasoning and inter-model diversity affect downstream learning, while beyond the scope of this paper, would be an impactful direction for future work.\n\n\\[1\\] A unified theory of diversity in ensemble learning. JMLR 2023\n\n\\[2\\] Diversity and generalization in neural network ensembles. AISTATS 2022\n\n\\[3\\] Does knowledge distillation really work? NeurIPS 2021"}, {"review_id": "myDf1BHsfV", "reviewer": "Reviewer_13BY", "summary": "The authors propose SpatialReasoner, a vision-language model focused on fine-grained spatial reasoning. They first introduce a high quality dataset constructed using tree-search and reward algorithms for better data diversity and spatial reasoning context. Then an MLLM is trained using fine-grained DPO. Their model is evaluated across benchmarks highlighting the clear and consistent performance improvements.", "questions": "See weaknesses", "limitations": "Yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strengths**\n1. The paper is well written with great use of figures to illustrate points\n2. Extensive evaluation established usefulness and strengths of method\n3. Novel dataset construction and training mechanism\n4. Good method interpretability through qualitative results\n\n\n**Weaknesses**\n1. Can any principled method be derived for chosing alpha and lambda values in fDPO? Ablations in Table 3 & 4 show some sensitivity to these parameters. \n2. What are the VLMs used for rewards? Highlight this more in main paper. \n3. Consider discussing possibly related prior work like [1] & [2]. \n4. Also on the architecture design, have the authors explored using textual representations for locations (like in [1,2]) instead of masks with an additional prompt encoder? Consider discussing this. \n\n&nbsp;\n\n[1] “Shikra: Unleashing Multimodal LLM's Referential Dialogue Magic.” 2023\n\n[2] “Learning to Localize Objects Improves Spatial Reasoning in Visual-LLMs.” CVPR 2024", "was_revised": true, "final_justification": "The authors resolve all concerns. Retaining vote of accept.", "rebuttal": "We sincerely thank the reviewer for their thoughtful and supportive review, and for recognizing the strengths and impact of our work. We greatly appreciate the reviewer’s recognition of our novel dataset construction and fine-grained training methodology, the clarity and interpretability of our approach through strong visualizations and qualitative examples, and the consistent performance gains across spatial reasoning benchmarks. We are especially grateful for the reviewer’s encouragement and thoughtful questions, which reinforce the value of our novel framework.\n\n>**Q1: Can any principled method be derived for choosing alpha and lambda values in fDPO?** \n\nThank you for raising this important point\\! In our method, $\\\\alpha$ and $\\\\lambda$ regulate how the segment-wise DPO losses are weighted and shaped. While there exists no closed-form solution or analytical method for choosing optimal $\\\\lambda$ and $\\\\alpha$ values, their impact is interpretable.\n\n$\\\\lambda$ controls the sensitivity of the weight distribution to reward differences. Given reward deltas for each segment, denoted as $\\\\Delta\\\\mathbf{R}\\_{\\\\text{desc}}$ and $\\\\Delta\\\\mathbf{R}\\_{\\\\text{reason}}$, we compute segment-wise weights $w\\_{\\\\text{desc}}$ and $w\\_{\\\\text{reason}}$ via a softmax-like transformation (Eq.2 in main paper):\n\n$w\\_s \\= \\\\frac{\\\\exp(\\\\lambda \\\\cdot \\\\Delta R\\_s)}{\\\\exp(\\\\lambda \\\\cdot \\\\Delta R\\_{\\\\text{desc}}) \\+ \\\\exp(\\\\lambda \\\\cdot \\\\Delta R\\_{\\\\text{reason}})}, \\\\quad s \\\\in \\\\{\\\\text{desc}, \\\\text{reason}\\\\}$\n\nThese weights adaptively prioritize learning from reasoning segments (description or logic) that have stronger training signals, and the $\\\\lambda$ term controls the prioritization sensitivity. The design is analogous to applying a softmax over two reward deltas, with $\\\\lambda$ functioning as an *inverse temperature*. When $\\\\lambda \\\\to 0$, both segments receive nearly equal weight regardless of their relative reward. When $\\\\lambda \\\\to \\\\infty$, all weight is assigned to the segment with the highest reward delta, making the weighting sharper and amplifying reward-dominant segments.\n\n$\\\\alpha$ controls the magnitude of dynamic scaling applied to the segment-wise DPO losses. After computing $w\\_s$, each segment’s optimization weight $\\\\beta\\_s$ is modulated as follows (Eq.3 in main paper):\n\n$\\\\beta\\_s \\= \\\\beta \\\\cdot \\\\left\\[1 \\+ \\\\alpha \\\\cdot (2w\\_s \\- 1)\\\\right\\]$\n\nHere, a high $\\\\alpha$ encourages the model to focus more strongly on high-signal segments while attenuating learning from weaker segments. In contrast, a small $\\\\alpha$ keeps the updates more uniform, reducing the variance across segment types.\n\nIn practice, we sweep these hyperparameters on a validation split to choose values that yield optimal performance across key spatial reasoning metrics (ablations in Tables 3-4). A more adaptive or meta-learned strategy could be explored in future work.\n\n>**Q2: What are the VLMs used for rewards? Highlight this more in main paper.**  \n\nThank you for pointing this out. Our reward model ensemble, used during the LongCoT preference data generation and fDPO training stages, comprises Gemini 1.5 Pro, Qwen2.5-VL-72B, and Qwen2.5-VL-7B. We chose this ensemble to capture diverse reasoning patterns and mitigate overreliance on a single model’s inductive biases. To verify the value of this design, we conducted two complementary analyses:\n\n- A statistical breakdown over 1,000 sampled reasoning trees showed that reasoning nodes were selected from Gemini (49%), Qwen2.5-VL-72B (42%), and Qwen2.5-VL-7B (9%). This confirms that no single model dominates, and that the ensemble promotes meaningful diversity and calibration across reasoning paths. The inclusion of Qwen2.5-VL-7B ensures reasoning remains aligned with the capacity of our target model, improving generalization.   \n- A quantitative ablation showed that training on LongCoT data generated by the multi-model M3CTS ensemble significantly outperformed those trained on single-model data (Qwen2.5-VL-72B) by 8.5 points in qualitative accuracy and 6–10% across spatial categories, highlighting the contribution of the multi-VLM setup.\n\nThese insights are further discussed in our response to reviewer a5xU. We will ensure the final paper more clearly describes the VLMs used for reward modeling and their motivation.\n\n>**Q3: Consider discussing possibly related prior work like \\[1\\] & \\[2\\].**  \n**\\[1\\] “Shikra: Unleashing Multimodal LLM's Referential Dialogue Magic.” 2023**  \n**\\[2\\] “Learning to Localize Objects Improves Spatial Reasoning in Visual-LLMs.” CVPR 2024**\n\nThank you for bringing these highly relevant works to our attention.  Both are solid contributions to the field.  \\[1\\] introduces a compelling approach for referential dialogue by processing numerical coordinates directly within the text stream. The work in \\[2\\] is insightful, and we find their method of explicitly fine-tuning the model to associate object names with textual coordinates to be highly effective in improving localization and spatial comprehension.  While our work shares the broader goal of enhancing spatial reasoning in VLMs, our focus is orthogonal, as we aim to improve multi-step reasoning over spatial relations with LongCoT supervision, a multi-model reward ensemble, and segment-level optimization via fDPO. We agree these are important references and will ensure both papers are properly cited and discussed in our revised manuscript.\n\n>**Q4: Also on the architecture design, have the authors explored using textual representations for locations (like in \\[1,2\\]) instead of masks with an additional prompt encoder? Consider discussing this.**  \n\nThat is a great question. We did consider various location representations in our initial architectural design, including textual formats such as points and bounding boxes. However, compared to these coarser representations, we found that fine-grained masks represent objects with higher precision. This precision is particularly crucial in challenging scenarios where we found masks to be a more effective prompt encoder. For instance: 1\\. In cases of occlusion, a bounding box would inevitably include pixels from the occluding object, introducing noise and ambiguity. 2\\. For objects that occupy a very small pixel area, masks provide the necessary pixel-level alignment that is essential for accurate semantic understanding."}, {"review_id": "C2S3TM1TSA", "reviewer": "Reviewer_wEqt", "summary": "This paper introduces SpatialReasoner, a novel Vision-Language Model (VLM) designed to enhance fine-grained spatial reasoning through two key contributions:\n1. Fine-Grained Direct Preference Optimization (fDPO), which segments LongCoT responses into descriptive and reasoning components for targeted optimization.\n2. Multi-Model Monte Carlo Tree Search (M3CTS), a collaborative method to generate high-quality reasoning paths for training data.\nThe model achieves state-of-the-art performance on spatial reasoning benchmarks (e.g., +9.8% accuracy over baselines) while maintaining competitiveness on general vision-language tasks.", "questions": "Could you please provide more details about implementations and experiments?", "limitations": "Yes.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengthens\n+ The introduction of fDPO is well-motivated, addressing the limitation of uniform optimization in standard DPO.\n+ M3CTS leverages multiple VLMs to explore diverse reasoning paths, mitigating data scarcity for multi-step spatial tasks.\n+ Comprehensive experiemnts on multiple benchmarks demonstrate significant improvements (e.g., +4.1% in spatial quality tasks with fDPO).\n + The experiments compare several training methods (SFT/DPO/fDPO) on multiple model size (4/8B parameters).\n\n\nWeaknesses\n+ The paper would benefit from providing more in-depth analysis. I’m particularly curious about the impact of the three different types of rewards on model training—do all of them need to be used, or is one more critical than the others?\n+ The paper needs to provide more details. I could not find which pretrained model was used as the base model for finetuning. These details are critical for the paper's credibility and reproducibility.", "was_revised": true, "final_justification": "I will maintain my original score and assessment.", "rebuttal": "We thank the reviewer for their constructive review and recognition of our paper’s novel, well-motivated contributions, our thorough experimental design, and strong empirical gains demonstrated on fine-grained spatial reasoning tasks. We are especially encouraged by the reviewer’s acknowledgement that our approach meaningfully addresses limitations in standard DPO and tackles core challenges in multi-step spatial reasoning. We hope our responses below effectively clarify the reviewer’s questions.\n\n>**Q1: The paper would benefit from providing more in-depth analysis. I’m particularly curious about the impact of the three different types of rewards on model training—do all of them need to be used, or is one more critical than the others?**\n\nThank you for this insightful question. To better understand the contribution of each reward component in our fDPO framework, we conducted a controlled ablation study in which we removed one reward at a time and retrained the model using 200K training samples. \n\n|  | Below/Above | Left/Right | Big/Small | Tall/Short | Wide/Thin | Behind/Front | Qual Acc |\n| :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |\n| fDPO w/o R\\_vc | **90.00** | 87.62 | 84.91 | 85.71 | 85.58 | 88.18 | 87.06 |\n| fDPO w/o R\\_sp | 83.33 | 83.81 | 79.25 | 80.36 | 82.69 | 84.55 | 82.34 |\n| fDPO w/o R\\_lc | 88.33 | 85.71 | 83.02 | 77.68 | 82.69 | 86.36 | 84.02 |\n| fDPO with all | **90.00** | **89.52** | **86.79** | **86.61** | **87.50** | **90.00** | **88.43** |\n\nRemoving $R\\_{\\\\text{sp}}$ led to the most significant drop in qualitative accuracy (−6.09%), confirming that spatial grounding is the most critical supervision signal for the task. This is expected, as spatial relations are central to our benchmark and require explicit reinforcement. In contrast, removing $R\\_{\\\\text{vc}}$ resulted in the smallest degradation in qualitative accuracy (−1.37%), suggesting that visual description is already relatively well-learned from pretraining and contributes less incremental benefit during optimization. Nevertheless, performance still dropped without RvcR\\_{\\\\text{vc}}Rvc​, confirming its complementary role. The logical coherence reward $R\\_{\\\\text{lc}}$ led to a notable performance decline (−4.41%), especially in categories like *Tall/Short* and *Wide/Thin*, where multi-step logical reasoning over relative object properties (size, etc.) is needed. These findings support our design choice to incorporate all three reward types and demonstrate that each contributes complementary benefits to the final model performance.\n\n>**Q2: The paper needs to provide more details. I could not find which pretrained model was used as the base model for fine-tuning. These details are critical for the paper's credibility and reproducibility.**   \n**Could you please provide more details about implementations and experiments?**\n\nThank you for pointing this out. Our SpatialReasoner is built upon the SA2VA, which itself is based on InternVL2.5.  To demonstrate the effectiveness of our proposed methods, we benchmark our final model not only against this SA2VA baseline but also against a standard DPO implementation, as shown in  Table 1. We will include this omission in our Appendix E.: Training Details, where we describe implementation details and training data. We plan to release our code, training data, and pretrained model checkpoints to support reproducibility and future research. We sincerely welcome any additional questions on implementation and experimental details, and would be glad to provide further clarifications during the discussion period."}]

Author rebuttal:

Meta-review: This paper proposes SpatialReasoner, a VLM enhanced with fine-grained preference optimization and multi-LLM tree search, yielding state-of-the-art performance on various spatial reasoning tasks.

***Strength***: The approach introduces well-motivated and technically sound designs, including fine-grained rewards and fDPO, supported by comprehensive evaluation.

***Weakness***: The related work discussion is limited and some aspects of presentation clarity, particularly in figures and contributions, could be improved.

Given the quality of the submission, the rebuttal and all feedback, the ACs recommend acceptance, as this is among the first substantial works on spatial VLMs with RL and offers impactful contributions.
Decision: Accept (poster)