Title: Textual Supervision Enhances Geospatial Representations in Vision-Language Models
Primary area: interpretability and explainable AI
Keywords: Mechanistic Interpretability, Vision-language models, Probing, Interpretability

Abstract: Geospatial understanding is a critical yet underexplored dimension in the development of machine learning systems for tasks such as image geolocation and spatial reasoning. In this work, we analyze the geospatial representations acquired by three model families: vision-only architectures (e.g., ViT), vision-language models (e.g., CLIP), and large-scale multimodal foundation models (e.g., LLaVA, Qwen, and Gemma). By evaluating across image clusters, including people, landmarks, and everyday objects, grouped based on the degree of localizability, we reveal systematic gaps in spatial accuracy and show that textual supervision enhances fine-grained geospatial representations. Our findings suggest the role of language as an effective complementary modality for encoding spatial context and multimodal learning as a key direction for advancing geospatial AI.

TL;DR: Models pre-trained with text and images show stronger geospatial representations in their latent space.