Sparkle: Mastering Basic Spatial Capabilities in Vision Language Models Elicits Generalization to Spatial Reasoning (2025)

Abstract: Vision-language models (VLMs) excel in many downstream tasks but struggle with spatial reasoning, which is crucial for navigation and interaction with physical environments. Specifically, many spatial reasoning tasks rely on fundamental two-dimensional (2D) capabilities, yet our evaluation shows that state-of-the-art VLMs often produce implausible or incorrect solutions for composite spatial problems, including simple pathfinding tasks that humans solve effortlessly at a glance. To address this, we explore an effective approach to enhance 2D spatial reasoning in VLMs by training them solely on basic spatial capabilities. We first disentangle 2D spatial reasoning into three core components: direction comprehension, distance estimation, and localization. Our central hypothesis is that mastering these basic capabilities will significantly boost performance on more complex spatial tasks requiring advanced reasoning and combinatorial problem-solving, as well as generalize to real-world visual-spatial scenarios. To test this hypothesis, we introduce Sparkle, a framework that generates synthetic data to provide targeted supervision for VLMs across these three basic spatial capabilities, producing an instruction dataset for each capability. Our experiments demonstrate that VLMs fine-tuned with Sparkle achieve substantial improvements, not only on basic tasks but also in generalizing to composite and out-of-distribution real-world spatial reasoning tasks. These findings highlight that enhancing basic spatial capabilities through synthetic generalization effectively improves complex spatial reasoning, offering insights into systematic strategies for boosting VLMs’ spatial understanding. Source codes of Sparkle are available at https://github.com/YihongT/Sparkle.

## introduction vision language models (vlms) @xcite @xcite @xcite have demonstrated nearhuman performance in tasks like image captioning @xcite , visual question answering (vqa) @xcite ♡ equal contribution. corresponding author. given the grid graph with nodes(n1, n2, ... n16), find the shortest path from the green node to the red node. answer: ["n5",""n9","n13"] ## methodology in order to systematically evaluate and enhance the spatial reasoning capabilities of vlms in 2d environments, we introduce the sparkle framework, as illustrated in figure 2 . ## experiments in this section, we provide our findings and results to demonstrate the effectiveness of the sparkle framework. specifically, the experiments are designed to answer the following research questions: rq1: can mastering basic 2d spatial components enhance overall spatial reasoning capability in vlms? rq2: what insights from the results of evaluations (section 4.2), enhancements (section 4.2), and spatial components (section 4.3) can guide improvements in model design, training, and data collection for spatial reasoning in vlms? ## conclusion we present the sparkle framework to address the limited spatial reasoning ability of vision language models (vlms). it is designed to enhance spatial reasoning by focusing on three basic capabilities: direction, distance and localization. experiments show that fine-tuning on these basic capabilities leads to substantial improvements in the basic tasks and composite tasks, showcasing its compositional generalizability. it also leads to generalization on broader tasks, strengthening vlms' ability to interact with the physical world.