Fine-Grained Preference Optimization Improves Spatial Reasoning in VLMs
Authors: name: Yifan Shen, name: Yuanzhe Liu, name: Jingyuan Zhu, name: Xu Cao, name: Xiaofeng Zhang, name: Yixiao He, name: Wenming Ye, name: James Matthew Rehg, name: Ismini Lourentzou
Organization: PLAN Lab @University of Illinois Urbana-Champaign

Current Vision-Language Models (VLMs) struggle with fine-grained spatial
reasoning, particularly when multi-step logic and precise spatial alignment are
required. In this work, we introduce SpatialReasoner-R1, a vision-language
reasoning model designed to address these limitations. To construct
high-quality supervision for spatial reasoning, we design a Multi-Model Monte
Carlo Tree Search (M3CTS) method that generates diverse, logically consistent
Long Chain-of-Thought (LongCoT) reasoning trajectories. In addition, we propose
fine-grained Direct Preference Optimization (fDPO), which introduces
segment-specific preference granularity for descriptive grounding and logical
reasoning, guided by a spatial reward mechanism that evaluates candidate
responses based on visual consistency, spatial grounding, and logical
coherence. Experimental results demonstrate that fDPO achieves an average
improvement of 4.1% over standard DPO across spatial quality tasks, and a 9.0%
gain in spatial quantity tasks. SpatialReasoner-R1, trained with fDPO, sets a
new SoTA on SPATIALRGPT-Bench, outperforming the strongest baseline by 9.8% in
average accuracy, while maintaining competitive performance on general
vision-language tasks.