Title: GeoLink: Empowering Remote Sensing Foundation Model with OpenStreetMap Data
Venue: NeurIPS 2025
Accepted: yes
Keywords: Remote Sensing

Geospatial data

Foundation Model

Self-supervised Learning

Multimodal

Geoscience
Abstract: Integrating ground-level geospatial data with rich geographic context, like OpenStreetMap (OSM), into remote sensing (RS) foundation models (FMs) is essential for advancing geospatial intelligence and supporting a broad spectrum of tasks. However, modality gap between RS and OSM data, including differences in data structure, content, and spatial granularity, makes effective synergy highly challenging, and most existing RS FMs focus on imagery alone. To this end, this study presents GeoLink, a multimodal framework that leverages OSM data to enhance RS FM during both the pretraining and downstream task stages. 
Specifically, GeoLink enhances RS self-supervised pretraining using multi-granularity learning signals derived from OSM data, guided by cross-modal spatial correlations for information interaction and collaboration. It also introduces image mask-reconstruction to enable sparse input for efficient pretraining. For downstream tasks, GeoLink generates both unimodal and multimodal fine-grained encodings to support a wide range of applications, from common RS interpretation tasks like land cover classification to more comprehensive geographic tasks like urban function zone mapping. Extensive experiments show that incorporating OSM data during pretraining enhances the performance of the RS image encoder, while fusing RS and OSM data in downstream tasks improves the FM’s adaptability to complex geographic scenarios. These results underscore the potential of multimodal synergy in advancing high-level geospatial artificial intelligence. Moreover, we find that spatial correlation plays a crucial role in enabling effective multimodal geospatial data integration. Code, checkpoints, and using examples are released at [GitHub](https://github.com/bailubin/GeoLink_NeurIPS2025).


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents a new self-supervised learning (SSL) algorithm, called GeoLink, for pre-training remote sensing (RS) models. Specifically, GeoLink fuses RGB satellite imagery with OpenStreetMap (OSM) data.\n\nThe pre-training algorithm consists of:\n- Separately masking and encoding satellite imagery and OSM data. The OSM encoder is a Graph Neural Network (GNN), the image encoder is a ViT. The GNN-node features are computed using a language model.\n- A contrastive loss term is applied between visible encodings across these two modalities\n- A reconstruction loss computed using an MAE-style decoder that predicts the hidden RGB patches (without OSM fusion).\n- Another reconstruction loss, called a “spatial consistency loss”, computed using an “object-patch” fusion encoder that receives masked node embeddings and predicts the initial / hidden node embeddings, while cross-attending to the visible RGB patch encodings.\n\nThe paper demonstrates that GeoLink outperforms prior pre-trained RS models on many classification and segmentation tasks.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n- The paper is dense and well-written\n- I agree that pre-trained RS models have not yet fully leveraged data like OSM and this is a promising research direction\n\nWeaknesses:\n- The satellite images are RGB-only. In this field (pre-trained RS models), most papers use multispectral satellite images as these additional bands are useful for many downstream tasks. Furthermore, this RGB-only choice makes comparisons a bit tricky since baselines were pre-trained on multispectral data (like CROMA).\n- The main contribution of the paper (to a conference like NeurIPS) is the SSL algorithm, which I have concerns about. \n  - I understand that OSM can be encoded with a GNN, but can’t it also be encoded with a transformer? The field has settled on transformers for multimodal RS, so this deviation should be better motivated and empirically demonstrated (e.g., by showing it works better than a transformer encoder).\n  - A language model (BERT) is used to encode OSM nodes that are fed to the GNN. I don’t see an ablation that entirely removes the GNN and just uses the BERT encodings. It is certainly possible that all the gains of GeoLink can be explained by this natural language supervision — rather than the algorithm itself.\n\nQUESTIONS:\nHow are the multispectral baseline methods used when processing RGB-only benchmark data?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-20"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces GeoLink, a multimodal framework that integrates OpenStreetMap (OSM) data with remote sensing (RS) foundation models to enhance geospatial intelligence. GeoLink leverages OSM data to enrich the RS foundation model during both pretraining and downstream task stages through multi-granularity learning signals, masked input pretraining, and RS-OSM fusion. The framework demonstrates significant improvements in the performance of the RS image encoder and shows strong adaptability to complex geographic scenarios across various tasks.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n1. GeoLink innovatively combines RS and OSM data into a comprehensive multimodal framework. This integration not only addresses the structural differences between the two data types but also enhances the semantic understanding of RS images through spatial context.\n2. By utilizing the multi-granularity nature of OSM data, GeoLink designs region-image level alignment and object-patch level interaction to capture information across different spatial scales. \n3. Adopting a masked input approach for both RS images and OSM graphs during pretraining, GeoLink achieves efficient training while maintaining performance improvements. \n\nWeaknesses\n\n1. The paper does not discuss the computational scalability of the GNN-based OSM encoder and the object-patch fusion module, particularly in regions with very high object density. In dense urban cores, a single scene can contain thousands of OSM objects, which could make the graph construction, message passing, and especially the cross-attention mechanism computationally prohibitive. A discussion on these limitations and potential mitigation strategies (e.g., graph sampling) would strengthen the paper.\n2. While the overall framework is highly novel, the individual building blocks (ViT-MAE, GATConv, InfoNCE, cross-attention) are established techniques. \n\n3. Although GeoLink shows some robustness to incomplete OSM data, the performance may still be affected in regions with sparse OSM coverage. The authors need to further explore methods to reduce this dependency or supplement missing data.\n\n4. Despite faster convergence during pretraining, the multimodal framework involving multiple encoders and complex GNN structures may lead to higher computational costs during training and inference. The training is fast, but don't know how efficient it is during inference. The paper does not discuss the inference time of GeoLink, which is crucial for real-time applications. The authors should evaluate and improve the model's inference efficiency.\n\nQUESTIONS:\n1. In the multimodal downstream tasks, the hybrid RS-OSM patch encodings enrich each visual patch with OSM context. This works by attending to OSM object features. Is there a risk of spatial \"blurring\" where a patch at the edge of a large feature (e.g., a large \"park\" polygon) receives the same aggregated contextual vector as a patch in the center? How does the model differentiate fine-grained intra-object location? Does the sinusoidal positional embedding for the patch itself provide enough signal to mitigate this?\n2. The ablation study on OSM completeness (Table 4) is insightful. However, it relies on random removal. In reality, OSM data quality is systematically biased (e.g., better coverage in developed urban centers than in rural areas). How might this geographic bias in pretraining data affect the model's generalization performance? Does the model perform better on downstream tasks in regions that are well-represented in the pretraining dataset?\n3. The paper provides excellent ablations on the learning objectives. Could you provide some insight into the design of the OSM encoder itself? How critical are the different topological edge types? For instance, what would be the performance impact of a simpler model that only considers a \"disjoint\" (via Delaunay) and a generic \"intersects/contains\" relationship, thereby reducing the GNN's complexity?\n4. The framework is designed for OSM. How do you envision GeoLink being adapted for other types of geospatial vector data that may have different semantics or structures, such as official administrative boundaries, cadastral (property) maps, or utility networks? Would the general framework of a GNN encoder + fusion module still apply?\n5. The experimental comparison is insufficient. (1. GeoLangBind-L needs to be added as a comparison method. (2. DOFA needs to adopt the ViT-L version; otherwise, the comparison would be unfair.\n6. Does the result of Scale-MAE+OSM indicate that the proposed designed fusion encoder is a plug-and-play module and whether it can be attached to other advanced models for experimentation?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-27"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes GeoLink, a remote sensing (RS) foundation model that combines RGB images with OpenStreetMap (OSM) data. The model consists of two separate encoders for each data modality (RS images and OSM), a fusion strategy for combining OSM and image embeddings, and a decoder for the image modality. The pre-training strategy follows a Masked Autoencoder (MAE)-like approach. Results are reported on various datasets within the Earth Observation domain, covering downstream tasks such as classification, segmentation, and urban function zone mapping. The proposed approach demonstrates superior performance compared to related works, highlighting the value of incorporating OSM data during pre-training.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\nThe paper is overall well written, which makes it enjoyable to read. However, in some parts (mostly the abstract, introduction, and conclusion), I noticed sentences that might contain patterns commonly found in AI-generated text, such as the overuse of hyphens. Note that I’m not claiming this is AI-generate, is just that the style sometimes feels unusual.\n\nThe novelty is somewhat limited in terms of the model’s components, as most are drawn from existing architectures (e.g., RS encoder: ViT, Mask Decoder: MAE, OSM encoder: GATConv). However, I found the integration of these elements to be valuable and clever.\n\nThe paper includes extensive experiments to demonstrate the value of the proposed approach. The model is evaluated on classification, segmentation, and urban function zone mapping downstream tasks. It shows superior performance on most datasets for each task when compared to related works. Additionally, ablation studies are provided to highlight the contribution of each model component.\n\nThis doesn't influence my decision, but I want to highlight it: the appendix is very informative. It contains detailed and useful descriptions of many aspects mentioned briefly in the main paper, including the model's components, downstream tasks, fine-tuning process, and implementation details.\n\nWeaknesses:\n\nI found the comparison a bit limited. Some of the works included, such as Scale-MAE or Cross-Scale MAE, do not explicitly incorporate multimodality. Since this is a multimodal approach, I believe it would be more appropriate to compare with recent models that utilise other types of modalities. For example: MMEarth and MultiMAE meets EO. Additionally, you might consider works that incorporate textual modalities. A broader perspective can be found in this recent survey: https://arxiv.org/pdf/2505.14361.\n\nI don’t fully understand the motivation for including only RGB images in this domain. While RGB is the most abundant data type in EO, many datasets include multispectral data (e.g., Sentinel-2). Evaluating such data would make the submission stronger and more applicable to real-world EO scenarios, where S2 data is widely used. I understand that some recent EO models (e.g., CLIP-based) are limited to RGB due to the constraints of pre-trained models. However, in this case, the encoder is not fixed to such limitations. I’m not against using only RGB, nor am I criticising this point directly, but the paper lacks a convincing motivation for this design choice.\n\nSome minor formatting errors and suggestions:\n- I’d suggest placing Fig.2 on page 3. I think it will help the reader to have the figure in the same page where the description of the approach starts. \n- Fig 2. Part (a). I’d suggest changing the abbreviation of projection to ‘proj’ instead of ‘prj’ on the blue box under ‘contrastive loss’. I agree that both abbreviations are correct, however since this is a computer vision paper, I believe ‘proj’ is more common.\n- Line 139: Missing space between text and equation.\n- Line 166: I’d suggest adding a reference to set2set layers (https://arxiv.org/pdf/1511.06391). Not sure if the term is universally known.\n- Line 226: Six benchmarks are mentioned but only five examples given.\n- I’d suggest including the numbers on each bar of Figure 3 (something like Figure 4).  Probably, it would be more valuable to present these numbers in a table format (?).\n- Inconsistent capitalisation in section headers. For instance, line 112 uses \"Framework Overview\" while line 97 has \"Synergy of RS and geospatial vector data.\" I recommend adopting a consistent style throughout the manuscript.\n\nQUESTIONS:\n- Why does the model not consider multispectral data, such as Sentinel-2? What is the motivation for limiting the RS encoder to RGB images? Given the wide availability and importance of multispectral data in Earth Observation (EO), it would be helpful to understand the reasoning behind this design choice. Are there specific challenges or limitations that prevent the integration of multispectral inputs into the current framework?\n\n\n- How difficult is the overall process of obtaining OSM data? While OSM provides rich geospatial information, it is not as commonly used in EO foundation models as other modalities (e.g., text, multi-spectral, SAR). Could the authors elaborate on the practical challenges of collecting, aligning, or processing OSM data at scale? Are there domain or geographic limitations that affect its applicability?\n\n\n- Why has the model been pre-trained for only 60 epochs? Is this decision based on empirical results, resource limitations, or performance? It would be useful to understand whether longer pre-training might lead to further gains, or if 60 epochs was sufficient based on validation trends.\n\n\n- Regarding the ablation studies, what would be the expected impact of increasing the masking ratio for the RGB encoder (e.g., 80–90%)? Did the authors experiment with or consider more aggressive masking? If so, what were the observed effects?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-28"}

comments: SUMMARY:
The paper presents an approach to linking remote sensing imagery with OSM data, taking into account the different data structures. The result is OSM enhancing remote sensing data. In addition, the method addresses computational issues by enabling sparse input.  The method is applied to segmentation tasks to test for improvement. The idea is interesting and nicely presented.

STRENGTHS AND WEAKNESSES:
ln 27: fix the language
ln 32-33: This statement isn't really true - revise please.
Captions of figures should be in the figure environment not in the graphic itself. 

There are important results in the supplementary as well that should perhaps be included in the main part of the paper?

QUESTIONS:
How would this used in practice? i.e. how does one obtain a RS and OSM pair that are temporally aligned and good data? 

How is the accuracy of the OSM data dealt with?

LIMITATIONS:
These have been addressed.
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 2
DATE: 2025-07-02

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes GeoLink, a framework to leverage OSM data to enhance Remote Sensing Foundation model. To combine these two modalities, GeoLink uses self-supervised mask reconstruction learning for efficient pretraining. GeoLink is then tested across multiple unimodal tasks and datasets, and multimodal tasks, showing improvements compared to baselines.\n\nSTRENGTHS AND WEAKNESSES:\nPositives\n* Well-written paper that does well showcasing the intuition and justification for design decisions\n* The idea of doing image mask-reconstruction with OSM and remote sensing image is intuitive and sound and allows leveraging of OSM data in RS. \n* Good improvement on four tasks, the diversity of task is also good (spanning unimodal and multimodal vision tasks)\n* I especially appreciate the OSM data completeness table which adds practical insight since OSM is not always complete.\n\nAreas of improvement\n* There have been quite a few more recent geospatial foundation model that the work should compare to: https://arxiv.org/pdf/2410.22629 If such models are not applicable, please let us know so and why).\n* For Table 1, the paper only compares to DOFA VIT-B. In the DOFA paper, it shows their DOFA VIT-L has slightly better performance. Please include VIT-L as a baseline too or provide a justification on why it shouldn't be. \n* For the multimodal tasks experiments, please include task-specific baselines too. This will help the community understand when to use foundation models vs. task-specific ones.\n\nQUESTIONS:\n* There have been quite a few more recent geospatial foundation model that the work should compare to: https://arxiv.org/pdf/2410.22629 If such models are not applicable, please let us know so and why). I may increase the score based on the results. I will decrease the score if no justification or results were given.\n* For Table 1, the paper only compares to DOFA VIT-B. In the DOFA paper, it shows their DOFA VIT-L has slightly better performance. Please include VIT-L as a baseline too or provide a justification on why it shouldn't be. I may increase the score based on the results. I will decrease the score if no justification or results were given.\n* For the multimodal tasks experiments, please include task-specific baselines too. This will help the community understand when to use foundation models vs. task-specific ones. I may increase the score based on the results. I will decrease the score if no justification or results were given.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-04"}