Fine-Grained Preference Optimization Improves Spatial Reasoning in VLMs
Authors: name: Yifan Shen; name: Yuanzhe Liu; name: Jingyuan Zhu; name: Xu Cao; name: Xiaofeng Zhang; name: Yixiao He; name: Wenming Ye; name: James Matthew Rehg; name: Ismini Lourentzou
Published: 2025-06-26 18:00:00

Abstract: Current Vision-Language Models (VLMs) struggle with fine-grained spatial
reasoning, particularly when multi-step logic and precise spatial alignment are
required. In this work, we introduce SpatialReasoner-R1, a vision-language
reasoning model designed to address these limitations. To construct
high-quality supervision for spatial reasoning, we design a Multi-Model Monte
Carlo Tree Search (M3CTS) method that generates diverse, logically consistent
Long Chain-of-Thought (LongCoT) reasoning trajectories. In addition, we propose
fine-grained Direct Preference Optimization (fDPO), which introduces
segment-specific preference granularity for descriptive grounding and logical
reasoning, guided by a spatial reward mechanism that evaluates candidate
responses based on visual consistency, spatial grounding, and logical
coherence. Experimental results demonstrate that fDPO achieves an average
improvement of 4.1% over standard DPO across spatial quality tasks, and a 9.0%
gain in spatial quantity tasks. SpatialReasoner-R1, trained with fDPO, sets a
new SoTA on SPATIALRGPT-Bench, outperforming the strongest baseline by 9.8% in
average accuracy, while maintaining competitive performance on general
vision-language tasks.