GeoLink: Empowering Remote Sensing Foundation Model with OpenStreetMap Data
Authors: Lubin Bai
Xiuyuan Zhang
Siqi Zhang
Zepeng Zhang
Haoyu Wang
Wei Qin
Shihong Du
Venue: NeurIPS 2025 poster ()
Keywords: Remote Sensing
Geospatial data
Foundation Model
Self-supervised Learning
Multimodal
Geoscience
Primary area: machine_learning_for_sciences
TL;DR: A multimodal framework that leverages OpenStreetMap data to enhance remote sensing foundation models in both self-supervised pretraining and downstream tasks.

Abstract: Integrating ground-level geospatial data with rich geographic context, like OpenStreetMap (OSM), into remote sensing (RS) foundation models (FMs) is essential for advancing geospatial intelligence and supporting a broad spectrum of tasks. However, modality gap between RS and OSM data, including differences in data structure, content, and spatial granularity, makes effective synergy highly challenging, and most existing RS FMs focus on imagery alone. To this end, this study presents GeoLink, a multimodal framework that leverages OSM data to enhance RS FM during both the pretraining and downstream task stages. 
Specifically, GeoLink enhances RS self-supervised pretraining using multi-granularity learning signals derived from OSM data, guided by cross-modal spatial correlations for information interaction and collaboration. It also introduces image mask-reconstruction to enable sparse input for efficient pretraining. For downstream tasks, GeoLink generates both unimodal and multimodal fine-grained encodings to support a wide range of applications, from common RS interpretation tasks like land cover classification to more comprehensive geographic tasks like urban function zone mapping. Extensive experiments show that incorporating OSM data during pretraining enhances the performance of the RS image encoder, while fusing RS and OSM data in downstream tasks improves the FM’s adaptability to complex geographic scenarios. These results underscore the potential of multimodal synergy in advancing high-level geospatial artificial intelligence. Moreover, we find that spatial correlation plays a crucial role in enabling effective multimodal geospatial data integration. Code, checkpoints, and using examples are released at [GitHub](https://github.com/bailubin/GeoLink_NeurIPS2025).

GeoLink: Empowering Remote Sensing Foundation Model with OpenStreetMap Data
Lubin Bai'; Xiuyuan Zhang', Siqi Zhang', Zepeng Zhang*, Haoyu Wang', Wei Qin', Shihong 2
School of Earth and Space Sciences, Peking University, Beijing, China
College of Urban and Environmental Sciences, Peking University, Beijing, China
State Key Laboratory of Multimodal Artificial Intelligence Systems Institute of Automation, CAS, Beijing, China
4 Intelligent Maintenance and Operations Systems Lab
École Polytechnique Fédérale de Lausanne, Lausanne, Switzerland
Abstract
Integrating ground-level geospatial data with rich geographic context, like Open- StreetMap (OSM), into remote sensing (RS) foundation models (FMs) is essential for advancing geospatial intelligence and supporting a broad spectrum of tasks. However, modality gap between RS and OSM data, including differences in data structure, content, and spatial granularity, makes effective synergy highly challeng- ing, and most existing RS FMs focus on imagery alone. To this end, this study presents Geolink, multimodal framework that leverages OSM data to enhance RS FM during both the pretraining and downstream task stages. Specifically, GeoLink enhances RS self-supervised pretraining using multi- multi-granularity learn- ing signals derived from OSM data, guided by cross-modal spatial correlations for information interaction and collaboration. reconstruction to enable sparse input for efficient pretraining. For downstream tasks, GeoLink generates both unimodal and multimodal fine-grained encodings support wide range of applications, from common RS interpretation tasks like land cover classification to more comprehensive geographic tasks like urban function mapping. Extensive experiments show that incorporating OSM data during pretraining enhances the performance of the RS image encoder, while fusing RS and OSM data in downstream tasks improves the FM's adaptability com- plex geographic scenarios. These results underscore the potential of multimodal synergy advancing high-level geospatial artificial intelligence. Moreover, we find that spatial correlation plays crucial role in enabling effective multimodal geospatial data integration. Code, checkpoints, and using examples are released https://github.com htt.ghhuwwboooobblbbbnnoeeeeeo baw.lbbbb/i/eooiikkkkkk NeurlPS2025
also introduces image mask-
1 Introduction
Remote sensing (RS) serves as a powerful tool for observing and monitoring our planet. Recently, the label-free, task-agnostic nature of self-supervised learning (SSL) has enabled RS foundation
models (FMs) to make significant strides [1 2 3 4. Beyond scaling the model parameters and
dataset size, many RS FMs have been specifically tailored to accommodate the unique characteristies of RS image [5] 6
7 8). incorporating multi-scall [2] [9), multi-temporal 4 I0 12 [3]. and
multi-spectrall
3
15
[6]
processing techniques. After pretraining, these FMs can extract
"Email: Ibbai @ wkw.uu.k..du.cc I Corresponding author: shdu@@ku.edu.cr
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Tag table
Multi-granularity
ID
Building
SSL Objectives
RS Image
Comprchensive
Object-patch
Interpretation
Geographic Tasks
Integration
Tasks
OSM Map
Mask Reconstruction
Object-patch Fusion Module
Points
Polylines
Region-image
RS Image Encoder
OSM
Polygons
Alignment
Encoder
(a) OSM vector data
(b) Pretraining stage and downstream tasks of GeoLink
Figure 1: (a) OSM data stores the geometry information of geographic features in vector format, including points, polylines, and polygons, and leverages tags to record the semantic information. (b) GeoLink leverages mulli-granularity SSL objectives to integrate RS and OSM data across multiple spatial scales, supporting both RS interpretation tasks and comprehensive geographic tasks.
meaningful, generalizable representations for RS interpretation tasks like semantic segmentation. yielding impressive performance in many domains, like environment monitoring [17] and disaster management [18.
However, the integration of ground-level geospatial data remains relatively underexplored in many existing RS FMs. Ground-level geospatial data like various kinds of maps, in-situ sensor data and so on, can not only serve as RS interpretation references but also provide supplementary information for real-world applications 199 20 (21). Among them, OpenStreetMap (OSM) is one of the largest open-source geospatial databases of volunteered geographic information (VGI), providing rich geo-context associated with geographic locations 22 OSM data has long been used in RS interpretation 220 23 24), and we believe integrating it into RS FM is essential for achieving geo-oriented and context-aware understanding of Earth observation, as well as the advanced geospatial intelligence. First, it provides explicit location- oecation-based contextual cues that are difficult to capture from pure visual analysis, linking pixels real-world objects and resolving ambiguities buildings by location). Second, dision-languee CLIP (25] show that structured semantics can enhance transferable representation learning, where OSM data, with its spatial hierarchies and geo-tagged attributes, plays similar role. Finally, many geographic tasks demand holistic understanding that images alone cannot provide, as they lack socioeconomic insights, while OSM data can fill this information gap.
(e.g., distinguishigg similar-lookin
models like
As shown Fig. [a). OSM data originally besedr, storing geographic objects as points, polylines, and polygons with rich tag tables, which differs significantly from RS image in data format and information content. To support RS image interpretation, most existing studies adopt indirect integration strategies. For example, converting OSM data into labels for RS images [26 (7]. or constructing knowledge graphs from OSM data provide prior knowledge for RS interpretation 28 29). However, such approaches tend to be manuall-ntenssvv and task-specific, confined to small-scale training datasets and experimental regions, making them misaligned with the paradigm of RS FMs. Some recent studies have explored leveraging OSM data to generate synthetic text data for RS vision-language FMs 30 31, where relevant content descriptions associated with RS images are extracted from OSM data. They also follow an indirect way to reconcile the modality discrepancies between OSM and RS data, resulting in the loss of spatial information. To unlock OSM's potential for FM development, we aim to design geo-spatially explicit approach that directly harnesses OSM's raw vector elements to inject geo-context into RS FMs, providing multi pelliiiperppectieee geographic priors while enhancing model capabilities across diverse geospatial tasks.
In this study, we introduce GeoLink, multimodal FM that (1) enhances RS self- sulf-supereieed pretrain- ing through OSM-derived multi-granularii learning signals, (2) achieves efficient pretraining via masked input, and (3) increases the performance and diversity of downstream tasks via RS-OSM fusion. First, we design a heterogeneous graph neural network (GNN)-based OSM encoder that specifically addresses the geometric heterogeneity (ponts/polylines/pllygggs non-Fuclidean struc- ture, and dynamic attribute tags of OSM data. Representing OSM objects nodes and their spatial

---

relations as edges, the encoder performs message passing to generate both object-level (node) and region-level (graph) encodings for interaction with the RS image encoder. Second, as shown in Fig.[bb. using location as the bridge to build multi-granularity spatial correlation, the cross-modal learning signals are derived at two levels: (1) region-image level alignment via contrastive learning with explicit spatial extent matching, and (2) object-patch level interaction through position-aware cross-attention, capturing implicit spatial associations while preserving spatial consistency for joint representation learning. Third, inspired by MAE (32] and FLIP [33], we randomly mask large portion of image patches (e.g. 75%) during pretraining, feeding only the visible ones to accelerate training while maintaining accuracy improvements. Finally, after pretrained
1.2 million sam- ple pairs, we evaluate GeoLink in two task collections, as shown in Fig. [b). RS interpretation tasks (unimodal) and comprchensive geographic tasks (multimodal). According to the real-world application scenarios, several benchmarks are employed to assess our model across various domains, including land use/cover, agriculture, and urban planning. We find that incorporating OSM data during pretraining significantly enhances the RS image encoder's 's capacity, while fusing RS and OSM data in downstream tasks improves the FM's adaptability to complex geographic scenarios.
2 Related work
RS FMs. Thanks to advances in computational power and deep learning, RS FMs have rapidly evolved, trending toward multi-scale, multi-tempporal, and multi-sensor designs tailored to the unique traits of RS imagery. Given the visual variance of geographic objects across resolutions, models like Scale-MAE. [ and Cross-Scale MAE [ extend MAE with multi-scale augmentation and position embeddings to handle varying ground resolutions. As surface changes driven by seasons and human activity are common, tealtil RS is key for tasks like change detection and crop mapping. Approaches such as SeCo [I]] use time-separated image pairs for contrastive learning, while SatMAE [4] introduces temporal embeddings to encode timestamps. Multi-sensor RS combines data from sources like multispectral and SAR to enrich downstream tasks, prompting FMs such as CROMA 3 DOFA 34 SeaMo [TO, MMEarth (35), Skysense (36), and OmniSAT [37] to support multi-sensor inputs via multi-encoder tokenizer- based designs. These works highlight the growing emphasis on multioddaity.. Beyond scale, time, and sensor diversity, geographic domain knowledge is also vital (38.. Geospatial vector data, such OSM, offers rich yet underused geographic contextual information; this study explores its integration into RS FM.
Synergy of RS and geospatial vector data. A significant modality gap exists between RS and geospatial vector data, with most existing methods adopting indirect integration strategies. They can be categorized into three types based on vector data utilization: data conversion, data derivation, and knowledge graph methods. Data conversion methods utilize tools like buffering and rasterization to transform vector data into raster format, thus matching the structure of RS images for easier processing; the rasterized geospatial data may then serve as either inputs [39 40 or training labels [22 41 42). Data derivation methods generate intermediate data from vectors to assist RS tasks, such as producing image captions (30 B1] creating geospatial umits from road networks [39], or constructing positive pairs for contrastive learning 43). Knowledge graph-based methods extract geographic knowledge from vector data to build graphs that support RS image interpretation [28 29]
While these indirect paradigms have long dominated RS-vector synergy, recent efforts explore direct integration, primarily via point data enriched with latiuddial-llogitudeee study, we aim further harness the rich information contained in OSM vector data incorporate with RS images, thereby improving performance in a wider array of geographic downstream tasks.
priors 44 [5]. In this
3 Method
Framework Overview, As shown in Fig. 2 GeoLink contains three encoders: (1) Vision Transformer
(ViT)-based (46) RS image encoder fr that encodes RS image input I into patch encodings Ep E RD where Lp the number of patches and Dp the patch feature dimension. (2) Graph Attention Convolution Network (GATConv)- based [47] OSM encoder fo that takes the constructed OSM graph as input, and outputs the node encodings after message passing εv E RR4 Dv where εv = εv, Usv, U Here, Lv is the number of nodes, Dy is the node feature dimension, and p. 1, and y denote the node types: point, polyline, and polygon, respectively. Details about the OSM encoder structure can be found in the Appendix A. (3) Object-patch fusion encoder fF
3

---

Visible Patch Encodings
Mask Decoder
Reconstruction
Image Encoder
Loss
Pooling & Prj.
Masked Nodé Enpodings
Visiblé Parch Encoudings
RS linage
Visible Patches
Contrastive Loss
Mask
Position
Aggregation & Prj.
Embeddiigs
Object-patch Fusion Encoder
OSM Encoder
04
OSM.RS Initial
Object Erscadings Festaies
OSM Graph
Mask Tokens
Node Encodings
e/ Spatial Consistency Loss
(partly masked)
(a) Pretraining Framework of GeoLink
OSM Tag Table
Semantic Attributes
Initial Node Features
RS Interpretation Tasks (Unimodal): Classification Segmentation!
Leisur
[I [eeisiure:: pitch", "color: green" ...]....
Change Detection
pitch
Language Model
Land
Agriculture
Disaster
Disjoint
Touch
Overlap
Comprehensive Geographic Tasks (Multimodal): Multimodal Somantic Segmentation Regression
Cover
Delamma
Hybrid RS-OSM
Cover
Triangulation
Punch Encoditga
Spatial Reintionships
Edge
Uibban Planning
Socioeccnoomic
(b) Construction of OSM Graph
(c) Downstream Tasks
Figure 2: (a) GeoLink masks both modalities, using visible image patches and masked OSM graph as inputs. Pretraining is achieved through three SSL objectives: RS reconstruction loss, cross-modal contrastive loss, and spatial consistency loss. (b) The heterogeneous graph is employed to model OSM data, incorporating three node types and multiple spatial relationships. (c) The pretrained model can produce both unimodal and multimodal encodings, generalizing to various downstream tasks.
for fine-grained data integration, which takes the patch encodings Ep and node encodings &y as input, and generates two types of multimodal encodings, including hybrid OSM-RS object encodings EOR E R-v DDr and hybrid RS-OSM patch encodings EO E IR Lp where DF is the fusion feature dimension. During pretraining, mask both modalities. using visible RS image patches and masked OSM graphs inputs. Three learning objectives are leveraged to optimize the encoders, including RS reconstruction loss, cross-modal contrastive loss, and spatial consistency loss. Next, we will systematically introduce GeoLink's 's pretraining process.
OSM Graph Construction. As shown in Fig- (b)) the heterogeneous graph is constructed to model the OSM vector map, where nodes represent geographic objects (points, polylines, and polygons), and edges capture various spatial relationships between them. First, embed the OSM tags as the initial node features to represent the semantic attributes. OSM follows a free tagging system with unlimited tag categories, rendering static methods inadequate for handling unseen values. To accommodate this scenario while preserving semantics, we employ a language model encode the OSM tags. Specifically, each OSM object's tag attributes are organized into a tag- value dictionary D, ((t1,c1), (t2, C2).... (t,, c, and each tag-value pair is converted into string s; co, " :..,)). c.). which individually encoded using a BERT language model as h; = BE More frequently occurring tags (e.g., "building", "land use") better represent the general attributes of an object, while rare tags (e.g., "historic period"") usually describe details. To provide a comprehensive and consistent representation, we calculate the weighted average of all
tag-value encodings of an OSM object serve as the initial node feature:oy
wahh// Σ w.
where the weight for each tag corresponds to its occurrence count in the global pretraining dataset. Second, given the scale differences between points, polylines, and polygons, we leverage spatial topological relationships rather than distances to construct edges, accounting for variations across

---

different vector types. For instance, point features only exhibit disjoint relationships, in which case we employ Delaunay Triangulation [4]] to establish connections. A detailed exposition of the spatial relationship types among various node categories is provided in the Appendix A Compared to spatial distance, topology embodies a definitive spatial relationship and less susceptible to noise. The constructed OSM graph is then fed into the OSM encoder, where message passing enables each node to aggregate information from its neighbors, resulting the node encodings Ey.
Masked Inputs. During the pretraining phase, the learning objectives are built upon masked inputs, which not only enhances representation learning but also reduces memory consumption. For RS data with VIT encoder, the image is divided into grid of non-overlapping patches, with large portion randomly masked out, leaving only the visible ones as input obtain the visible patch encodings exp. Following MAE [32], we leverage a mask decoder containing two Transformer blocks to reconstruct the masked patches. Then, we calculate the reconstruction loss between the reconstructed patches f and original masked patches 1d as:
N
Lp
2 13
Cee
m
N
where N is the batch size and LP is the number of masked patches in each input image. For OSM data, we employ a node-mas masking strategy. For each masked node its initial feature σv; is replaced by learnable mask token while preserving its original adjacency edges with other nodes to ensure effective message passing within the OSM encoder. We adopt three different mask tokens for points, polylines, and polygons, respectively. The OSM encoder outputs encodings Ey for all nodes, including both masked and visible ones, i.e., By = EQ U.y.
Region-image Level Alignment. Since each input pair of RS and OSM data covers the same geographic extent, they describe the corresponding region from different perspectives and contain correlated, complementary information. Consequently, we employ contrastive learning to align them. To obtain region-level OSM encoding, we design an aggregation module aggregate the heterogeneous node encodings. Specifically, this module first leverages three Set2Set layers that independently aggregate nodes of each type into type-specific encodings EG, Set2Set,Evy) where t E ( corresponds to the node types of point, polyline, and polygon. Then linear layer followed by a Softmax layer computes type attention weight-sum the sype-specific encodings to produce the OSM region encoding: &G LtE(p..g) Softmax (aieaaaxxTnnna(aa,a. ))aa, For RS data, we employ mean pooling for the visible patch encodings ey, to obtain the image-level encoding Ey = (MeanPool(sj.).. Following (25), we project both the OSM region encoding Ea and the RS image encoding ε using separate linear layers, i.e., FG Linearc(sa) and Linearr(er). and contrast both modalities using the InfoNCE loss [49]
Lcont 2N
1
Σ
N
N exp(sim (2/2) 2/)/T) Σn=
exp(sim(= =1)/T)
N
j=1 exp(sim(2), =<4//T) )
where N the batch size, T temperature parameter, and sim(.:- the cosine similarity function, ie., sim(u, v) |a Through contrastive learning, the inherent structured semantic information in OSM data is effectively conveyed to the image encoder, thereby guiding its pretraining process.
Object-patch Level Integration. To further facilitate interaction between the two modalities and obtain fine-grained fused representations, we design an object-patch fusion encoder. This encoder is implemented as a two-way Transformer composed of one self-attention layer and two cross- -attentnt layers (details in Appendix A). As shown in Fig. 2a), during pretraining, the fusion encoder accepts the masked node encodings s and the visible patch encodings eY; as inputs, and produces two types of fused encodings: hy brid OSM-RS object encodings FØR and hybrid RS-OSM patch encodings eRo A critical challenge arises from the inherent spatial ambiguity between the two input encodings: the absence of explicit geographic correspondence makes direct cross-attention operations susceptible to erroneous feature associations, e.g., accidentally connect unrelated elements. To address this, we incorporate sinusoidal position embeddings into the fusion encoder by adding them to each input. The sinusoidal embedding handles individual coordinates, which can be directly adopted to point nodes. To capture the spatial coverage of polylines and polygons, we sample their key-points and compute the average of the sampled points' position embeddings (details see in Appendix (A). These enhanced spatial signatures enable the model to progressively establish accurate cross-modal association
5

---

through ircention-based learning. As shown in Fig. 2a), the output hybrid OSM-RS object encodings integrate features from visible nodes (via message passing in the OSM encoder) and visible patches (via cross-modal interaction in the fusion encoder), which encapsulate the spatial context surrounding masked OSM objects. According to the first law of geography [50), this contextual information strongly correlated with the intrinsic properties of the masked ones. To enforce spatial-semantic consistency, we introduce a consistency loss function that operates on the hybrid OSM-RS object encodings FOR and the initial features of masked nodes oV:
N
LV
1
2
Lest =
EOR.,
Vi3
j=1
where N is the batch size, and LV is the number of masked nodes each graph. The synergy of the position embedding and consistency constraints significantly enhances the model's capacity for cross-modal representation learning, improving the RS encoder's ability to capture fine-grained semantic features. Finally, the three objectives are combined together for pretraining: == aLCree &oout + Lest, where the loss weights are set as a = 1, 3 = 0.01, and y = 0.01 by default (related experiments about the loss weights are provided in Appendix D).
4 Experiments
Pretraining details. To pretrain GeoLink, we construct a multimodal dataset derived from SkyScript- top30 [31. The SkyScript-top300 dataset contains multi-source, multi-resolution RS images with RGB bands, featuring ground sample distances (GSD) ranging from 0.1 m/pixel to 30 m/pixel. For each RS image, the corresponding OSM data is downloaded from the Overpass API using its geo-coordinate and timestamp. After preprocessing like data cleaning, we obtain a final pretraining dataset of 1,271,431 matched pairs. More details are provided in the Appendix B A ViT-L model is employed as the RS image encoder in this study. The default masking ratios for RS patch and OSM graph node is 75% and 20%. And 5 0.2 for contrastive loss. Our experiments are conducted on Linux server equipped with 4 NVIDIA RTX6000 GPUs (48GB) using bfloat precision. Unlike FMs such as Scale-MAE (800 epochs) and CROMA (600 epochs) which typically require a large number of pretraining epochs, GeoLink demonstrates significantly faster convergence. We pretrain it for only 60 epochs (including 5 warmup epochs), with batch size of 2640, a base learning rate of 1 x 10-1, and cosine decay schedule for learning rate cooldown. For data augmentation, we apply random cropping, horizontal flipping, and color jittering to the RS images. Notably, we perform corresponding geometric transformations on the OSM data to maintain spatial alignment with the augmented RS images. The model optimization employs AdamW with hyperparameters (3, = 0.9,
82 = 0.95) and a weight decay of 0.05.
Downstream Task Settings. As depicted in Figure 4c), GeoLink supports both RS image in- terpretation tasks (unimodal) and comprehensive geographic tasks (multimodal). For the former, only the RS image encoder is employed, which is combined with task-specific protocols to assess the learned unimodal representations through classification, semantic segmentation, and change detection tasks. We benchmark GeoL.nkk against six RS FMs, including GASSL [55, MMEarth (35), Scale-MAE (2) Cross-scale MAE 19, CROMA (3, and DOFA [34] Other comparisons can be found the Appendix D For multimodal geographic tasks, we incorporate OSM data into the downstream tasks. We evaluate urban function zone (UFZ) segmentation, urban village (UV) identification, population density (POP) and carbon emission (CO2) estimation tasks, which are crucial for realistic urban planning and socioeconomic analysis. These challenges typically require multi-source geographic data, making them ideal for assessing GeoLink's 's multimodal capabilities. The details of bascline model selection, downstream model architectures, and evaluation protocols are provided in the Appendix C
4.1 Unimodal tasks
Classification tasks. We evaluate the RS representations learned by GeoLink using three classification
protocols, including: (1) KNN (k- 20) evaluates representation quality by measuring instance clustering without further training: (2) linear probing assesses performance using a linear classifier on frozen RS features; (3) fine-tuning jointly updates the RS encoder and classifier for task-specific adaptation. For comprchensive evaluation, employ seven RS benchmarks that span diverse spatial

---

Table 1: Comparison of different models across classification benchmarks under kNN, linear probing (LP), and fine-tuning (FT) evaluation protocols (Top-1 accuracy %).
Model
Backhone
MLIISNet
EuroAAT
WHL-RSIP
OPTIMAL-31
RESSSSC-5
AiRound
L'Mirred
LNN
LP
FT
LSN
LP
FT
ANS
LP
FF
JNNN
122
P7
LNN
LP
FV
KSN
LP
ANN
LP
FF
MMllieth
CoroNertt
59.79
91.84
49:21
90.83
97. 72
73:11
89:31
90.04
80.31
38.90
60.90
73.85
15.08
16.51
95.35
Stai-MAE
WIL.
92.25
91.56
96.97
01.42
97.40
49.05
99.41
98.10
12:22
17.65
99:71
15.41
91.14
14
71.04
78.39
92.64
18.18
95.10
95.21
Cless Scale MAE
VITC
92663
03.30
96.23
90.24
95.58
57.97
19.66
97,02
97 42
79 78
57.85
58.71
55.30
90.89
93.50
72.23
74.51
80.79
54.67
9590
16.19
CROMA
VTTT
96.31
07.15
09.17
91/41
96 50
77.85
85.05
05.33
82.41
05.61
16
61
73.35
11:00
91:11
94.58
GecLia
95.30
95.81
50.37
90.73
57.33
91.43
Table 2: Comparison of encoder-freezing and tuniing performance across four semantic segmen- lation/change detection downstream datasets (mloU %).
Model
Backbone
FiveBillionPixels
Al4Smalifarms
SpaceNer?
XView2
Freezing
Fine- Fine-tuning
Freezing
Fine-tuning
Freezing
Fine-tuning
Freezing
Fine-tuning
GASSL
ResNe150
57.47
61.37
39.65
43.29
$7.63
62.09
56.27
59.87
MMEarth
ConvNext V2
56.12
62,69
37.88 37.86
43.13
62.20
62.66
56.54
59,92
Scale-MAR
VIT-I
68.73
41.11
45.98
62.78
60.37
Cross Scale MAE
VIT-L
58.68
63.96
40.33
44.87
60,23
63.03
58.44
60.87
CROMA
VIT-L
DORA
VIT-I.
57.83
63.94
38.31
45.94
61.38
62.43
58.86
61.47
GeoLink
VIT-L
60.49
64.93
43,26
47.29
63,22
64,07
59.94
61.94
WHU-RS19 [53), OPTIMAL- 31 [54), RESISC-45 (55), AiRound (56). and UCMerced [57]. All benchmarks are split into 50% for training, 10% for validation, and 40% for testing. Each experiment is repeated three times under different random seeds and the average results are reported to ensure robustness. Please refer to the Appendix C for detailed downstream task settings. As shown in Table II, GeoLink achieves state-of-the-art performance on most datasets, showcasing its superiority in learning generalizable representations. Notably, GeoLink outperforms all compared FMs by significant margins under the kNN protocol, which indicates that it has learned structured RS representations where semantically similar samples are close in the feature space. Compared to linear probing, GeoLink demonstrates even more pronounced superiority under the fine-tuning protocol. Beyond this, we also conduct data efficiency analysis, and observe that GeoLink's advantage becomes even more evident when training samples are limited. Detailed results are provided in Appendix D
resolutions and category systems: MLRSNet (51), EuroSAT [52]
Semantic segmentation and change detection tasks. Unlike classification tasks, semantic segmen- tation and change detection aim to evaluate the model's ability to capture spatially detailed represen- tations. For both tasks, we follow the protocols of the PANGAEA-bench, including using UperNet as the decoder and adopting identical training configurations. We evaluate performance on four benchmarks from PANGAEA-bench. Five-Billie Biee-Billion--iiee Pixelss [58, AI4SmallFarms [59], View2 [60] and SpaceNet7 61,, which cover diverse application domains such as agricultural monitoring and disaster management. The first two benchmarks correspond semantic segmentation tasks, while the latter two are used for change detection. Table presents the performance comparison of various FMs on the four benchmarks under both mecoder-freezing and fine-tuning settings. GeoLink consistently achieves the best results on average in both scenarios, demonstrating the strong generalization and adaptability of its learned RS representations. The advantages of Geol Link are also evident in more challenging datasets such AI4Smallfarms and x VView2, where it consistently leads under both settings. When considered alongside the unimodal classification results, these findings further under- score the effectiveness of GeoLink's cross-modal pretraining strategy. By leveraging the supervision from OSM data, GeoLink significantly enhances the capability of the RS image encoder to learn transferable and semantically rich representations.
4.2 Multimodal tasks
UFZ segmentation. UFZs, such as residential, commercial, and institutional areas, are essential units in urban planning, reflecting complex socioeconomic and physical dynamics. However, due to the heterogeneous nature of man mane infrastructure and visual discontinuities in RS images, accurately identifying UFZs using RS data alone remains challenging. To address this, growing research efforts incorporate multi-source data 62 63
We construct challenging real-world UFZ segmentation benchmark by refining planning maps from Chicago, Singapore, and Shenzhen, comprising 60,970 samples across nine UFZ categories. Using UperNet as the decoder, fine-tune
64
65
7

---

Seale-MAE
Stale Geoiiink-unimmdal
70
GeoaLink-maltimddl
60
IoU
50
65.5
40
37A
30
30
Green
Fanulaad Undeveloped Residkurial Commercial Institution
Industrial Transporation
Comparison different models UFZ task
UTZ
RS Patch Encedings
Hebbrid RS-DSM
UPZ Ground-crath
R& Parch Eacodings
Hlhhid RS-O8M Prick Encodlings
Image
Ground-rruth
Parch Encodings
Image
Reaiential
Commencal
Transportaiion
Green
Water
Institution
(b) T-SNE plots of Geal Link's unimodal and multimodal representations on UFZ task
Figure 3: (a) loU (%) performances of each UFZ category. (b) T-SNE is used to visualize the learned patch encodings of GeoLink. With the incorporation of OSM data, multimodal encodings become more compact and discriminative than unimodal ones.
GeoLink and Scale-MAE under unimodal and multimodal settings. In the multimodal setup, UperNet consumes hybrid RS-OSM features produced by the object-pateh fusion encoder. Detailed structures and settings provided in the Appendix C GeoLink outperforms Scale-MAE in both unimodal and multimodal scenarios (Fig. B(a)). Notably, when OSM data is excluded at downstream task time, GeoLink benefits from multimodal pretraining, yielding superior performance. When OSM data is included, GeoLink's performance further parrovve--prtticlarryy for complex classes such industrial, institutional, and commercial highhrihiah-hh the advantages of precise, semantically rich multimodal fusion. In addition, Scale-MAE+OSM largely outperforms Scale-MAE, further indicating the effectiveness of the proposed designed fusion encoder. Furthermore, patch representations of GeoLink in Figure ()).. Some categories that are easily confused in RS images exhibit scattered and overlapping distribution in unimodal scenario, while the hybrid RS-OSM patch encodings show significantly greater discriminability. highlighting GeoLink multimodal fusion capability and the importance multi-source information comprehensive geospatial tasks. We also conduct a relevant while distinctive downstream task, i.c., urban village identification, and detailed results are provided in Appendix
visualize the
ins-orfrmll residential zones within the out- skirts of cities-often suffer from de- ficient infrastructure and low-quality living conditions. Identifying UV is important for urban planning and sus- tainable development. In this study, we construct UV semantic segmen- tation dataset (details in Appendix ( evaluate the learned representations. The results in Table 3 demonstrate that GeoLink maintains outstanding performance in this task, showing significant improvement compared to Scale-MAE whether unimodal or multimodal scenarios. Comprehensively analyzing both the UV and UFZ tasks, the hybrid RS-OSM patch representations generated by GeoLink effectively couple information from both data sources, making them suitable for fine- fine-grained tasks like semantic segmentation.
UV identification.
Table 3: Comparison of methods on the urban village identi- fication task (IoU %).
Method
Background
UV
mloU
Scale-MAE
90.29
58.21
74.25
Meale-MAE+OSM
91.37
68.81
80.09
GeoLink-unimode
90.40
68.67
79.53
moo.iink-multmmdall
92.29
71.08
81.68

---

POP and CO2 estimation. Fine-scale spatialized population and carbon emission data are es- sential for geoscience research, including climate change studies. In this work, we evaluate Geol.nk on both tasks. Grid- basid population density and carbon emission data for Chicago, Singapore, and Shenzhen are sourced from WorldPop and ODIAC to construct evaluation benchmarks (details in Appendix [ We use a two-layer MLP as the regression head for each task. As illustrated in Table 4 integrat- ing OSM data significantly improves GeoLink's performance on both POP and CO2 estimation, highlighting the value of multi-source geospatial data. This supports view that multimodal data fusion will be essential to the next generation of FMs in geography.
Table 4: Performance comparison on POP and CO2 estimation tasks (r² %).
Model
POP
CO2
Scale-MAE
47.18
59.12
BBalcMME++OSM
48.29
59.97
GeoLink- Unimodal
49.76
62.37
fieoLink-Mulimodaa
51.88
65.12
4.3 Ablation studies
Model designing. Drawing on GeoLink's characteristics, we conduct comprehensive ablation studies isolate the effects of key design choices, including learning objective, masking ratio, and position embedding for data fusion. Unless otherwise noted, every variant is pretrained for 60 epochs using ViT-L as the image encoder. We assess each configuration on kNN classification, linear probing, and UFZ segmentation. For kNN and linear probing, results are reported as the mean performance over seven benchmarks. The results are shown in Table
(1) Learning objective. When using only Lrec, the model degenerates into standard MAE. Under this setting. performance notably poor after 60 epochs pretraining and only becomes competitive after 300 epochs, indicating slow convergence. In contrast, Geolink achieves strong performance within just 60 epochs, thereby reducing computational cost. Additionally, Lcont facilitates cross- modal alignment to enhance the image encoder, while List primarily serves to optimize the fusion module to obtain multimodal encodings.
(2) RS masking ratio. Reducing the masking ratio to 50% for RS images leads to performance degradation and increased training cost, and can also observe a slight performance drop 80% and significant collapse at 90%.
(3) OSM masking ratio. The model shows robustness the masking ratio applied to OSM inputs, maintaining stable performance when approximately 20% of the data is masked.
(4) Position embedding for data fusion. Removing position embeddings causes slight drop in unimodal performance but leads to significant decline in multimodal effectiveness. This underscores the critical role of spatial correlation in the fusion of geospatial modalities. More experiments on the designing of position embedding can be found in the Appendix
Table 5: Ablation study on GeoLink. Performance on classification tasks (top-1 accuracy %) and UFZ segmentation (mIoU %) under varying configurations.
Ablation
Setting
Cost
KNN
Linear
UFZ-U
UFZ-M
Default
1.00x
87.06
92.60
55.40
60.00
Cree (60 epochs)
0.85
78.12
83.42
40.41
Lree (300 epochs)
4.25x
85.97
90.03
49.09
(1) Learning objective
Lree + Loout
0.92
86.12
91.14
53.94
Cre + Lest
0.93
82.34
86.77
51.06
57.39
50%
2.10x
83.21
89.64
49.96
54.28
(2) RS masking ratio
80%
0.95x
85.21
90.17
52.01
$6.04
90%
0.90x
78.02
83.47
43.02
45.81
15%
1.00x
86.99
91.97
54.21
58.70
(3) OSM masking ratio
25 %
1.00x
86.43
91.04
53.84
58.02
(4) Fusion position embedding
Without
1.00x
86.47
91.87
53.97
56.22
GCN variant Transformer variant
1.00x
87.01
92.14
55.02
59.42
(5) OSM encoder variants
1.01x
86.14
91.47
53.73
58.57

---

(5) OSM encoder varians. We design two variant experiments to illustrate the effectiveness of the current OSM encoder. One is GCN variant which replaces the GAT in the OSM encoder with GCN while keeping all other settings unchanged, and the other is transformer variant which replaces the existing GNN-based OSM encoder with 2 standard Transformer blocks. To adapt to the Transformer's structure, each OSM node in the original GNN is treated as token input to the Transformer. Additionally, position embedding is added to each token to preserve spatial information. Compared to the GCN variant, the attention mechanism introduced by GAT yields better performance. The Transformer variant underperforms both GCN and GAT, which we speculate is primarily due to the insufficient message- passing between OSM nodes in it, hindering the learning of OSM spatial consistency objective during pretraining.
OSM data completeness. Given the crowd- sourced nature of OSM, data completeness varies across regions, requiring models to re- main robust under sparse conditions. To evalu- ate this, we simulate completeness by randomly removing OSM objects (Table(6). Results indi- cate only minor performance drop when 20% of OSM data is removed. Even with 50% of the 50%
Table 6: Results of kNN (top-1 accuracy %) and UFZ (mloU %) under varying OSM completeness.
Completeness
kNN
UFZ-U
UFZ-M
100%
87.06
55.40
60.00
80%
86.98
55.01
59.13
86.59
54.12
57.37
OSM data omitted, GeoLink maintains strong performance across most tasks, with a notable decline observed primarily on multimodal UFZ segmentation task. These findings suggest that GeoLink demonstrates substantial robustness to incomplete OSM coverage and can adapt to regions with limited OSM availability.
Conclusion
In this study, we propose GeoLink, which effectively integrates geographic contextual cues from OSM data into the RS FM through semantic-spatial feature extraction and spatial-aware cross- modal interaction. This design enhances the pretraining process of RS image encoder, significantly improving its performance on image interpretation tasks. In addition, GeoLink produces fine-grained hybrid RS-OSM patch encodings tailored for comprehensive geographic tasks. Extensive evaluations across diverse benchmarks demonstrate that GeoLink outperforms previous state-of sfate-of-the-an models and excels in more challenging downstream tasks such as UFZ mapping. Furthermore, our findings emphasize the pivotal role of spatial correlation in bridging and fusing multimodal geospatial data.
Despite its promising results, Geol.nnk has certain limitations: (1) The focus of this paper is to explore how to leverage the rich geographic information of OSM data in RS FM, and at present, only supports RGB images and cannot process multispectral RS data. We believe the inclusion of multispectral RS images could further enhance the model, and we plan to improve the image encoder to accommodate data from various sensor types. (2) The current position embedding for OSM vectors can lead to the loss of spatial details. We argue that if position embedding can better capture the spatial characteristics, can facilitate more accurate and deeper spatial correlations. thereby enhancing the synergy across multimodal geospatial data.
6 Acknowledgement
The work is funded by the National Key Research and Development Program of China (2033YFC3804802) and National Natural Science Foundation of China (No. 42330103). Shihong Du is the corresponding author.
10

---

References
Daniela Szwarcman, Sujit Roy, Paolo Fraccaro, Porsteinn Elí Gíslason, Benedikt Blumensticl, Rinki Ghosal, Pedro Henrique de Oliveira, Joao Lucas de Sousa Almeida, Rocco Sedona, Yanghui Kang, et al. Prithvi-eo-2.0r A versatile multi-temppral foundation model for earth observation applications. arXiv preprint arXi::2412.0222 02732, 2024.
Colorado Reed, Ritwik Gupta, Shufan Li, Sarah Brockman, Christopher Funk, Brian Clipp, Kurt Keutzer, Salvatore Candido, Matt Uyttendaele, and Trevor Darrell. Scale-mae: A scale-aware masked autoencoder for multiscale geospatial representation learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 4088 4099, 2023.
[3] Anthony Fuller. Koreen Millard, and James Green. Croma: Remote sensing representations with contrastive radar-optical masked autoencoders. Advances in Neural Information Processing Systems, 36:5506-5523 2023.
[4] Yezhen Cong. Samar Khanna, Chenlin Meng, Patrick Liu, Erik Rozi, Yutong He. Marshall Burke, David Lobell, and Stefano Ermon. Satmae: Pre-training transformers for temporal and multi-spectral satellite imagery, Advances in Neural Information Processing Systems, 35:197-211 2022.
[5] Matías Mendieta, Boran Han, Xingjian Shi, Yi Zhu, and Chen Chen. Towards geospatial foundation models via continual pretraining. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 16806 100--16166 2023.
[6] Favyen Bastani, Piper Wolters, Ritwik Gupta, Joe Ferdinando, and Aniruddha Kembhavi. Satlaspretrain: large-scale dataset for remote sensing image understanding. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 16772-16782, 2023.
Nikolaos Ioannis Bountos, Arthur Ouaknine, Ioannis Papoutsis, and David Rolnick. Fomo: Multi-modal, multi-scale and multi task remote sensing foundation models forest monitoring. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 27858 27868, 2025.
[8] Xuyang Li, Chenyu Li, Pedram Ghamisi, and Danfeng Hong. Fleximo: A flexible remote sensing foundation model. arXiv preprint arXiv:2503.23844, 2025.
[9] Maofeng Tang. Andrei Cozma, Konstantinos Georgiou, and Hairong Qi. Cross-scale mae: tale of multiscale exploitation in remote sensing. Advances in Neural Information Processing Systems, 36:20054- 20066, 2023.
[10] Xuyang Li, Danfeng Hong, Chenyu Li, and Jocelyn Chanussot. Seamo: A multi-seasonal and multimodal remote sensing foundation model. arXiv preprint arXiv24412.19337, 2024.
[11] Oscar Manas, Alexandre Lacoste, Xavier Giró-i Nieto, David Vazquez, and Pau Rodriguez. Seasonal contrast: Unsupervised pre-traine pre-training from uncurated remote sensing data. In Proceedings of the IEEE/CVF International Conference Computer Vision, pages 9414-9423, 2021.
[12] Fanglong Yao, Wanxuan Lu, Heming Yang, Liangyu Xu, Chenglong Liu, Leiyi Hu. Hongfeng Yu, Nayu Liu, Chubo Deng, Deke Tang, et al. Ringmo-sense: Remote sensing foundation model for spatiotemporal prediction via spatiotemporal evolution disentangling. IEEE Transactions Geoscience and Remote Sensing, 61:1-21, 2023.
[13] Utkarsh Mall, Bharath Hariharan, and Kavita Bala. Change aware sampling contrastive learning for satellite images. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5261 -2700, 2023.
[14] Danfeng Hong, Bing Zhang, Xuyang Li. Yuxuan Li, Chenyu Li. Jing Yao, Naoto Yokoya, Hao Li, Pedram Ghamisi, Xiuping Jia, al. Spectralgpt: Spectral remote sensing foundation model. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024.
[15] Yi Wang, Conrad M Albrecht. Nassim Ait Ali Braham, Chenying Liu, Zhitong Xiong, and Xiao Xiang Zhu. Decoupling common and unique representations for multimodal self-supervised learning, arXiv preprint arXiv:2309.05.0 330000 2024.
[16] Jingtao Li, Yingyi Liu, Xinyu Wang, Yunning Peng. Chen Sun, Shaoyu Wang. Zhendong Sun, Tian Ke, Xiao Jiang, Tangwei Lu, et al. Hyperfree: A channel-adaptie and tuning free foundation model for hyperspectral remote sensing imagery. arXiv preprint 213:2503.32844 2025.
11

---

[17] Valerio Marsocci, Yuru Jia, Georges Le Bellier, David Kerekes, Liang Zeng, Sebastian Hafner, Sebastian Gerard, Eric Brune, Rit Yadav, Ali Shibli, et al. Pangaea: A global and inclusive benchmark for geospatial foundation models. arXiv preprint urXiv:2412.04200 2024.
[18] Chia-Yu Hsu, Wenwen Li, and Sizhe Wang. Geospatial foundation models for image analysis: Evaluating and enhancing nasa-ibm prithvi's domain adaptability. International Journal of Geographical Information Science, pages 1-30, 2024.
[19] Gengchen Mai, Weiming Huang. Jin Sun, Suhang Song, Deepak Mishra, Ninghao Liu, Song Gao. Tianming Liu, Gao Cong, Yingjie Hu, et al. On the opportunities and challenges of foundation models for geoai (vision paper). ACM Transactions on Spatial Algorithms and Systems, 10(2): 2024.
[20] John E Vargas-Munoz, Shivangi Srivastava, Devis Tuia, and Alexandre X Falcao. Openstreetmap: Chal- lenges and opportunities in machine learning and remote sensing. IEEE Geoscience and Remote Sensing Magazine, 91):184-199, 2020.
[21] Lubin Bai, Xiuyuan Zhang. Wei Qin, Jiang Long, Haoyu Wang, Xiaoyan Dong, and Shihong Du. From orbit to ground: A comprehensive review of multimodal self-supervised learning for remote sensing. IEEE Geoscience and Remote Sensing Magazine, pages 2-37, 2025.
[22] Nicolas Audebert, Bertrand Le Saux, and Sébastien Lefevre. Joint learning from earth observation and openstreetmap data to get faster better semantic maps. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pages 67-75, 2017.
[23] Munazza Usmani, Maurizio Napolitano, and Francesca Bovolo. Towards global scale segmentation with openstrectmap and remote sensing. ISPRS Open Journal of Photogranmetry and Remote Sensing, 8: 100031, 2023.
[24] Lubin Bai, Xiuyuan Zhang, Haoyu Wang, and Shihong Du Integrating remote sensing with openstreetmap data for comprehensive scene understanding through multi- modal self supervisdd learning. Remote Sensing of Environment, 318:1 14573. 2025.
[25] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pages 8748 8773. PmLR,
2021.
[26] Moritz Schott, Adina Zell, Sven Lautenbach, Gencer Sumbul, Michael Schultz. Alexander Zipf, and Begüm Demir. Analyzing and improving the quality and fitness for purpose of openstreetnap as labels in remote sensing applications. In Volunteered geographic information: Interpretation, visualization and social context, pages 21-42. Springer Nature Switzerland Cham, 2023.
[27] Taili Wan, Hongyang Lu. Qikai Lu, and Nianxue Luo. Classification of high-resolution remote-sensing image using openstreetmap information. IEEE Geoscience and Remote Sensing Letters, 14(12):2305-2) 2017.
[28] Yixiang Chen, Xu Dang, Daoyou Zhu, Yi Huang, and Kun Qin. Urban functional zone mapping by coupling domain knowledge graphs and high-resolution satellite images. Transactions in GIS, 28(6))1510-1533 2024.
[29] Zhao Gun and Jianyu Chen. Novel knowledge graph-and knowledge reasoning-based classification prototype for obia using high resolution remote sensing imagery. Remote Sensing. 15(2):321, 2023.
[30] Yuanxin Zhao, Mi Zhang, Bingnan Yang. Zhan Zhang, Jujia Kang, and Jianya Gong. Luojiahog: A hierarchy oriented geo-aware image caption dataset for remote sensing image-text retrieval. ISPRS Journal of Photogrammetry and Remote Sensing, 222:130-222 12:110115,, 2025.
[31] Zhecheng Wang, Rajanic Prabha, Tianyuan Huang. Jiajun Wu, and Ram Rajagopal. Skyseript: A large and semantically diverse lision-language dataset for remote sensing. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 5805-5813, 2024.
[32] Kaiming He, Xinlei Chen, Saining Xie. Yanghao Piotr Dollár, and Ross Girshick. Masked autoencoders are scalable vision leamerr.. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 16000-16009, 2022.
[33] Yanghao Li, Haoqi Fan, Ronghang Hu, Christoph Feichtenhofer, and Kaiming He. Scaling language aanuaae--maae pre- wre-training via masking. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 23390-23400 2023.
12

---

[34] Zhitong Xiong, Yi Wang, Fahong Zhang, Adam J Stewart, Joëlle Hanna, Damian Borth, loannis Papoutsis. Bertrand Le Saux. Gustau Camps-Valls, and Xiao Xiang Zhu. Neural muasticity-inspirre foundation model for observing the earth crossing modalities. arXiv e-prints, pages arXiv-2403, 2024.
[35] Vishal Nedungadi, Ankit Kariryaa, Stefan Oehmcke, Serge Belongie, Christian Igel, and Nico Lang. Mmearth: Exploring multi-modal pretext tasks for geospatial representation learning. In European Conference on Computer Vision, pages 164-182. Springer, 2024.
[36] Xin Guo, Jiangwei Lao, Bo Dang, Yingying 7/hang, Lei Yu, Lixiang Ru, Liheng Zhong, Ziyuan Huang, Kang Wu, Dingxiang Hu, et al. Skysense: A multi-modal remote sensing foundation model towards universal interpretation for carth observation imagery. In Proceedings of the IEEF/CVF Conference on Computer Vision and Puttern Recognition, pages 27672-27683, 2024.
[37] Guillaume Astruc, Nicolas Gonthier, Clement Mallet, and Loic Landrieu. Omnisat: Self-supervised modality fusion for earth observation. In European Conference on Computer Vision, pages 407-427. Springer, 2024.
[38] Hao Zhang. Jin-Jian Xu, Hong- Hlong-Wei Cui, Lin Li, Yaowen Yang. Chao-Sheng Tang, and Niklas Boers. When geoscience meets foundation models: Toward general geoscience artificial intelligence system. IEEE Geoseience and Remote Sensing Magazine, 2024.
[39] Tais Grippa, Stefanos Georganos, Soukaina Zarougui, Pauline Bognounou, Eric Diboulo, Yann Forget, Moritz Lennert, Sabine Vanhuysse, Nicholus Mboga, and Eléonore Wolff. Mapping urban land use street black level using openstreetmap. remote sensing data, and spatial metrics. ISPRS International Journal of neo-Infoormttin, 7(7):246, 2018
[40] Qiqi Zhu, Longli Ran, Yunchang Zhang, and Qingfeng Guan. Integrating geographic knowledge into deep learning for spatiotemporal local climate zone mapping derived thermal environment exploration across chinese climate zones. ISPRS Journal of Photogranmetry and Remote Sensing, 217:53-75, 2024.
[41] Di Yang, Chiung- Shhuun-ShhuanI Fu, Audrey C Smith, and Qiang Yu. Open land-use map: regional land-use mapping strategy for incorporating openstreetmap with earth observations. Geo-spatial information science, 20(3):269 281, 2017.
[42] Yang Ju. Iryna Dronova, and Xavier Deleiòs-Alió. A 10 m resolution urban green space map for major latin american citics from sentinel-2 remote sensing images and openstreetmap. Scientific Data, 58), 2022.
[43] Yanxin Xi, Tong Li, Huandong Wang, Yong Li, Sasu Tarkoma, and Pan Hui. Beyond the first law of geography: Learning representations of satellite imagery by leveraging pfint-of-interess.. In Proceedings of the ACM web conference 2022, pages 3308-3316 2022.
[44] Gengchen Mai, Ni Lao, Yutong Hc, Jiaming Song, and Stefano Ermon. Csp: Self-supervised contrastive spatial pre-training for geospatial-visure representations. In International Conference on Machine Learning. pages 23498-235555 23515. PMLR, 2023.
[45] Kumar Ayush, Burak Uzkent, Chenlin Meng, Kumar Tanmay, Marshall Burke, David Lobell, and Stefano Ermon. Geography-awe aware self-supervise learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 10181-10190, 2021.
[46] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn Xiaohua Zhai, Thomas Unterthiner. Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition scale. arXiv preprint arXiv:2010. / /9929, 2020.
[47] Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pictro Lio, and Yoshua Bengio. Graph attention networks. arXiv preprint 17Xiv:1710. 2017.
[48] Dierk Rhynsburger. Analytic delincation of thiessen polygons. Geographical analysis, 5(2):133-144, 1973.
[49] Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predictive coding, arXiv preprint arXiv:1807.03748. 2018.
[50] Waldo R Tobler. A computer movie simulating urban growth in the detroit region. Economic geography, 46(sup1):234-2 240, 1970.
[51] Xiaoman Qi. Panpan Zhu, Yuebin Wang. Liqiang Zhang, Junhuan Peng, Mengfan Wu, Jialong Chen, Xudong Zhao, Ning Zang, and P Takis Mathiopoulos. Mirsnet: A multi-label high spatial resolution remote sensing dataset for semantic scene understanding, ISPRS Journal of Photogrammetry and Remote Sensing, 169:337-350, 2020.
13

---

[52] Patrick Helber, Benjamin Bischke. Andreas Dengel. and Damian Borth. Eurosat: A novel dataset and deep learning benchmark for land use and land cover classification. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 12(7):2217-2226, 2019.
[53] Dengxin Dai and Wen Yang. Satellite image classification via two-layer sparse coding with biased image representation. IEFE Geoscience and remote sensing letters, 8 81)):133-176, 2010.
[54] Qi Wang, Shaoteng Liu, Jocelyn Chamussot, and Xuelong Li. Scene classification with recurrent attention of vhr remote sensing images. IEEE Transactions on Geoscience and Remote Sensing, 1555)115
2018.
[55] Gong Cheng, Junwei Han, and Xiaogiang Lu. Remote sensing image scene classification: Benchmark and state of the art. Proceedings of the IEEE, 105(10):1865 1885-183,, 2017.
[56] Gabriel Machado, Edemir Ferreira, Keiller Nogueira, Hugo Oliveira, Matheus Brito, Pedro Hen- rique Targino Gama, and Jefersson Alex dos Santos. Airound and ev-bret: Novel multiview datasets for scene classification. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 14:488-503, 2020.
[57] Yang and Shawn Newsam. Bag-of-visual-woot and spatial extensions for land use classification. In Proceedings of the 18th SIGSPATIAL, international conference on advances in geographic information systems, pages 270-279, 2010.
[58] Xin-Yi Tong, Gui-Song Xia, and Xiao Xiang Zhu. Enabling country-scale land cover mapping with meter-resolution satellite imagery. ISPRS Journal of Photogrammetry and Remote Sensing, 196:178-196 2023.
[59] Claudio Persello, Jeroen Grift, Xinyan Fan, Claudia Paris, Ronny Hänsch, Mila Koeva, and Andrew Nelson. Ai4smallfarms: A dataset for crop field delincation in southeast asian smallholder farms, IEEE Geoscience and Remote Sensing Letters, 20:1- 5, 2023.
[60] Ritwik Gupta, Richard Hosfelt, Sandra Sajeev, Nirav Patel, Bryce Goodman, Jigar Doshi, Eric Heim, Howie Choset, and Matthew Gaston. xbd: A dataset for assessing building damage from satellite imagery. arXiv preprint arXiv:1911.092000 2019.
[61] Adam Van Etten, Dave Lindenbaum, and Todd M Bacastow. Spacenet: A remote sensing dataset and challenge series. arXiv preprint 18xivv:807.000. 132,, 2018.
[62] Lubin Bai, Weiming Huang, Xiuyuan Zhang, Shihong Du, Gao Cong, Haoyu Wang, and Bo Liu. Geo- graphic mapping with unsupervised multi-modal representation learning from vhr images and pois. ISPRS Journal of Photogrammetry and Remote Sensing, 201: 201:193-208, 2023.
[63] Wei Chen, Huiping Huang, Jinwei Dong, Yuan Zhang, Yichen Tian, and Zhiqi Yang. Social functional map- ping of urban green space using remote sensing and social sensing data. ISPRS Journal of Photogranmetry and Remote Sensing, 146:436-44 4526,,522222 2018.
[64] Yifan Chen, Chaokang He, Wei Guo, Shiqi Zheng, and Bingxian Wu. Mapping urban functional areas using multisource remote sensing images and open big data. IEFF Journal of Selected Topics Applied Earth Observations and Remote Sensing, 16:7919-7931. 2023.
[65] Runyu Fan, Ruyi Feng, Wei Han, and Lizhe Wang. Urban functional zone mapping with bibranch neural network via fusing remote sensing and social sensing data. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 14:1 1737-1 1749,, 2021.
[66] Stefan Leyk, Andrea E Gaughan, Susana B Adamo, Alex De Sherbinin, Deborah Balk, Sergio Freire, Amy Rose, Forrest R Stevens, Brian Blankespoor, Charlie Frye, et al. The spatial allocation of population: a review of large-scale gridded population data products and their fitness for use. Earth System Science Data, 13(3):1385-144 2019.
[67] Christopher T Lloyd, Alessandro Sorichetta, and Andrew Tatem. High resolution global gridded data for use in population studies. Scientific data, 117, 2017.
[68] Dana R Thomson, Douglas R Leasure, Tomas Bird, Nikos Tzavidis, and Andrew J Tatem. How accurate are worldpop-glooaal-uccosss urban namibia. Plus one, (07))402, 2022.
gridded population data at the cell-level?: A simulation analysis in
[69] Tomohiro Oda, Shamil Maksyutov, and Robert J Andres. The open-source data inventory for anthropogenic co 2, version 2016 (01ia:2016): a global monthly fossil fuel co 2 gridded emissions data product for tracer transport simulations and surface flux inversions. Earth System Science Data, 10(1):87-10) 107, 2018
14

---

[70] Tomohiro Oda and Shamil Maksyulov. A very high-resolution (1 kmx km) global fossil fuel co emission inventory derived using point source database and satellite observations of nighttime lights. Atmospheric Chemistry and Physics, 1155)544--556, 2011.
[71] Ziyang Gong, Zhixiang Wei, Di Wang, Xianzheng Ma. Hongruixuan Chen, Yuru Jia, Yupeng Deng. Zhenming Ji, Xiangwei Zhu, Naoto Yokoya, Crossearth: Geospatial vision foundation model for domain generalizable remote sensing semantic segmentation. arXiv preprint arXiv:2410.22629, 2024.
15

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Yes, main contributions detailed in Sec. Also see in Sec. 4 for more experimental evidence.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification: Yes, please see Sec. for limitations.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
16

---

Justification:
We do not include any theoretical results in this paper.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: we include needed experiment details in Sec. and Appendix and We also upload the codes.
Guidelines:
The answer NA means that the paper does not include experiments. If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
17

---

Answer: [Yes]
Justification: We upload the codes to recover the results. Once the blind review period is finished, we will open-source codes, instructions, constructed benchmark datasets, and model checkpoints.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (kttps://nips.cm cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
7 //ipss.ccppblic//uu.dde ss
for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Please see in Sec. 4 and Appendix
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [No]
Justification:
Due to the resource limitation, we do not report error bars.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
18

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative enror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We report the experiments compute resources in Sec.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make into the paper).
9. Code ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics http://ps///pcccc..c.o/oo//////a
Answer: [Yes]
Justification:
We followed the NeurlPS Code of Ethics
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations their jurisdiction..
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [No]
Justification: This work focuses on academic, publicly-availabi datasets. This work is not related to any private or personal data, and there's no explicit negative social impacts.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
19

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of MI..
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [No]
Justification:
We do not foresee any high risk for misuse of this work.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Lieenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: Yes, credited them in appropriate ways.
Guidelines:
The answer NA means that the paper does existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
20

---

If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: The paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/mmoel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question:
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
21

---

Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that if the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Answer: [Yes]
Justification: Yes, we utilize language model Bert encode the OSM tags, which is detailed in See
Guidelines:
The answer NA means that the method development in this research does involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy for what should or should be described.
22

---

A Details of model designing
A.1 Spatial relationships contained in each edge type
Due to the different geometric structures of points, polylines, and polygons, the potential spatial relationships among these three node types also vary. Accordingly, the OSM heterogeneous graph we constructed incorporates nine edge types based on the permutations of different node types. As shown in Table 7 we summarize the spatial relationships contained by each edge type in detail. Specifically, while relationships between point nodes are represented using Delaunay triangulation, those among other node types are defined in terms of topological relations. After computing these spatial relationships, we encode them using one-hot encoding to represent the edge attributes within the heterogeneous graph.
Table 7: Topological relations between different node types
Node type
Point
Polyline
Polygon
Point Polyline
Delaunay triangulation Touch, contain
Touch, within Touch, intersect, cover, cover by, equal Touch, cross, cover
Touch, within Touch, cross, cover by
Polygon
Touch, contain
Touch, overlap, cover, cover by, equal
A.2 The structure of OSM encoder
The OSM encoder is a lightweight heterogeneous GNN built upon GATConv, specifically designed to capture the spatial and semantic relationships embedded in OSM data. As illustrated in Fig. 4 the encoder comprises nine parallel GATConv layers, each dedicated to handling one of nine dis- tinct edge types defined by the spatial relationships between node types. The encoder supports three node types-point, polyline, and
to model the complex topology in- herent in OpenStreetMap (OSM) data. Taking point-type nodes an example, the encoder not only performs intra-type message passing (i.e., among points) but also enables inter-type interac- tions by propagating messages to polyline and polygon nodes through separate GATConv layers. Conversely, it also gathers contextual informa- tion from polyline and polygon neighbors using additional GATConv layers. This cross-type message exchange ensures that each node em- bedding is enriched by both its own type's local structure and the information from other geo- metric types. By explicitly modeling heteroge- neous relations such as touches, within, crosses, and contains, the encoder effectively captures both the geometric connectivity and high-level geographic semantics of OSM features. The resulting node embeddings provide a rich rep- resentation of the spatial environment, which can be further integrated with RS encodings for downstream tasks.
Poire
Polyline
Polygnn
Gom
GATCoom
KEATCORR
Gom
Gon
GATCoon
GiATConV
GATCSm
GATCuna
ReLU
ReLU
ReLU
Polyline enending
Pwlygon encoding
esonding
Figure 4: Detailed structure of the OSM encoder.
A.3 Position embedding of OSM objects
We use sinusoidal position embedding uniformly represent the spatial positions of points, polylines, polygons, and image patches, thereby implicitly establishing spatial associations for fine-grad ane-grained RS-OSM fusion. As mentioned in Sec. 3 we sample key-points and compute the average of the sampled points' position embeddings to capture the spatial coverage of polylines and polygons during the object-patch level integration. For a polyline vector, we sample three key-points, i.e., two endpoints and midpoint. For a polygon vector, we first compute its centroid. Then, using the centroid as the center, we sample three points inside the polygon at random radii and angles, resulting in four points to represent its position and coverage. The sinusoidal position embedding is computed for each
23

---

sampled key-poin and their average is taken as the final position embedding for the corresponding vector. Appendix D we evaluate the effect of the sampling number key-points on model performance.
A.4
The structure of object-patch fusion encoder
Position Embeddings
Queries
OSM Node Encodings
Self-attention & Norm layer
Cross- Pross-attention & Norm layer
Cross- & Norm layer
Output: Hybrid RS-OSM Patch
Output: Hybrid OSM-RS Object Encodings
Keys
Encodings
Keys
Patch
Encodings
Values
Queries
Values
Figure 5: The detail structure of object-patte fusion encoder
As illustrated in Fig. 5 the object-patch fusion encoder is designed to effectively align and integrate RS image patches with geographic objects (node features) from OSM. comprises self-attention layer followed by two cross attention layers, forming lightweight yet expressive architecture for cross-modal fusion. The initial self-attention layer captures intra-modal relationships within the OSM object features, allowing the model to establish coherent representation of the geographic context before engaging in cross-modal interactions.
To enhance the model's spatial sensitivity, position embeddings are added to both the keys and values prior to each attention operation. This spatial conditioning enables the encoder better model the relative locations of RS patches and OSM objects, ensuring that attention weights are not only content-driven but also spatially aware. result, the model can form stronger associations between elements that are geographically close, leading to more accurate and meaningful fusion. The two cross-attention layers are asymmetrically structured to support bidirectional information exchange between modalities. The first cross-attention layer projects RS image patches queries and OSM objects as keys/values, producing hybrid RS-OSM patch encodings that embed geographic context into visual features. Conversely, the second cross-attention layer uses OSM objects as queries to attend over RS patches, generating hybrid OSM-RS object encodings that incorporate visual cues into semantic representations of geographic objects. These two outputs are both fine-grained multimodal representations- the image patch level and the geographic object level, respectively thereby equipping the model with the flexibility support broad range of downstream tasks.
B Details of the pretraining dataset
The pretraining dataset is derived from SkyScript-top30 BI, which contains 1271431 RS-OSM sample pairs. According to SkyScript, the RS images are originally downloaded from Google Earth Engine (GEE) platform, including 10 image collections like National Agriculture Imagery Program and Harmonized Sentinel-2, with a geographic coverage for all continents except Antarctic. The detailed information regarding the sources and distribution of RS images can be found [31] As for the OSM data, each RS image used in this study contains metadata geographic location and timestamp, and we leverage them to retrieve corresponding OSM data via the Overpass API, which allows querying OSM data within specified time range. After retrieving OSM data via Overpass, apply rule-based cleaning to remove common issues such as fixing or removing invalid polygons (e.g., self-interes intersections). eliminating duplicate/confficting objects, and filtering tags with formatting or spelling errors.
To ensure diversity and reduce training burden, we remove sample pairs with overlapping spatial coverage and filter out samples with unavailable OSM data. According to the statisties shown in Fig. 6la), each sample contains around ten OSM vector objects of points, polylines, and polygons

---

Average OSM Object
pot Sample
165
Top 10 Most Frequent Tags
4.5
9.55
3.5
3.9
Aucaago Crun
2,5
2.0
Points
Polylines
Pollygors
highoray
auriaa
haiking
tailway
bater
lanksr
netentf
natural
Figure 6: (a) The average counts of each kind of OSM object per sample. (b) The top 10 most frequent tags in the GeoLink pretraining dataset.
Linear Layer
UperNet Decoder
Linear Layer
UperNet Decoder
Concat
Hybrid RS-OSM
Output of final layer
Concat
Patch Encodings
Hybrid RS-OSM
Pätch Encodings
Objeci-patch Fusion Module
11th, 15th layers
Objoct-patch
11th, 15th and final layers
Fusion Module
final layer
final layer
RS Image Encoder (ViT-L)
RS Image Encoder
RS Image
RS Image
OSM Encoder
OSM Encoder
Encoder
Encoder (ViT-L)
(ViT-L)
(ViI-L)
(a) Downstream model structures unimodal scenario
(b) Downstream model structures in multimodal scenario
Figure 7: Model structures for diverse downstream tasks in unimodal and multimodal scenarios.
average. The pretraining dataset includes a total 2,790 types of tags, with the top most frequent ones shown in Fig. [)),), namely: highway, surface, one way, building, railway, service, lanes, land use, amenity, and natural.
C Details of downstream task settings
C.1 The selection of baseline models
Ideally, comparisons should made among models with identical spectral and modality settings, but in practice, this is often difficult to achieve. The diversity of spectral configurations in current RS FMs indeed makes idcalized comparisons challenging, as many models are tailored to specific satellite spectral bands. For example, CROMA is built Sentinel-1/2 1/2 data, while Prithvi-EO-2.0 [I] uses six channels shared by Sentinel-2 and Landsat: Blue, Green, Red, NIR, SWIRI, and SWIR2. Therefore, several prior works [1] [7 [8] have adopted pragmatic approach by evaluating models with different spectral inputs on the same benchmark datasets. Although this may not ensure fully aligned comparisons, it provides meaningful insights into the models' generalization common datasets.
To the best of our knowledge, there is currently no RS FM that uses the exact same data types as GeoLink (RS and OSM data), making difficult to directly compare with fully modality-aligned baselines. The starting point of our experimental design is to explore how, and to what extent,
25

---

Table Detailed information benchmark datasets for multimodal downstream tasks: UFZ, UV, POP, and CO2.
Benchmark
UFZ
UV
CO2
Experimental
Chicago
Beijing, Shanghai
Same as UFZ
Same as UFZ
region
metropolitan area,
Singapore, Shenzhen
RS image source
AreGIS World Imagery, Bing
AreGIS World Imagery
AreGIS World Imagery, Bing Map
AreGIS World
Imagery, Bing Map
Map
GSD
Im. 3m
Im RGB Expert annotation
Im. 3m
1m. 3m
Bands
RGB
RGB
RGB
Annotation source
Official statistics: Chicago
WorldPop Dataset
ODIAC Fossil Fuel
Emission Dataset
Singapore
Shenzhen
Annotation
Manual refinement, reclassification, spatially aligned cropping 224x224
Spatially aligned cropping
Resampling, spatially aligned cropping
Resampling. spatially aligned cropping
processing
Image cropping size
224x224
224x224
224x224
Sample count
60,970
1,899
59,284
47,607
integrating OSM data can enhance RS FM. To fairly and comprehensively evaluate GeoLink, we consider baselines designed from different perspectives: (1) Unimodal baselines that use the same RS spectral modality as GeoLink, including GASSL, ScaleMAE, and Cross-scale MAE; (2) Multimodal baselines that incorporate additional modalities, such as MMEarth, CROMA, and DOFA. Different strategies are employed to adapt multimodal baselines to the RGB-only testing benchmark. The original DOFA model includes dynamic projection module tokenizer normalize various spectral inputs, allowing to directly process RGB images without additional modifications. For CROMA and MMEarth, follow PANGAEA-bench [17 employs zero- prro-padding fill in missing spectral bands for input.
C.2 Model structures for downstream task evaluation
As illustrated in Figure 3, we present the model architectures employed for various downstream tasks under both unimodal and multimodal settings. For linear classification and regression tasks, the unimodal approach directly utilizes the encodings output from the final layer of the RS image encoder-ete either the [cls] token the mean patch features as input to linear layer (left side of Fig. a))) In the multimodal setting, this input is replaced by the mean of the hybrid RS-OSM patch encodings (left side of Fig. aa)) For semantic segmentation, we adopt the UperNet decoder. Under the unimodal setting, encodings from the 7th, 11th 15th, and final layers of the RS encoder are fed into the decoder. In the multimodal setting, the features from the final layer are replaced with the hybrid RS-OSM patch encodings, while all other components remain consistent with the unimodal configuration. Overall, aside from minor differences in feature dimensions, the model architecture remains consistent between the unimodal and multimodal settings. The multimodal encoder function as a plug-and-play component that integrates seamlessly into the RS FMs, and we combine it with Scale-MAE to build Scale-MAE+OSM for evaluation in Sec. 4
C.3 Construction of the multimodal benchmark datasets
In this study, we construct three multimodal benchmark datasets to evaluate GeoLink's 's capability in addressing complex and comprehensive geographic tasks, including urban functional zone seg- mentation (UFZ), urban village identification (UV), population density estimation (POP), and carbon emission estimation (CO2). Each dataset consists of spatially aligned R$ images, OSM data, and the corresponding task- specific annotations. Table Bis detailed overview of the construction procedures and key specifications of the four datasets. To begin with, considering factors such geographic
26

---

Table Hyperparameters for downstream tasks. LP: linear probing, Frz: freezing, FT: finetune, Cls: classification, Seg: segmentation, CD: change detection, LR: learning rate, CE: cross-entropy. MSE:
mean square emror
Task
Cls (LP)
Cls (FT)
Seg/CD (Frz)
Seg/CD (FT)
UFZ/UV
POPICO2
Optimizer Batch size
A
AdamW
AdamW
AdamW
AdamW
AdamW
64
64
64
64
32
256
LR
[1,3,5]
(1,3,5)
le-4
lc-4
(1,3,5/e
(1,3,5
e(-2,-3,-4)
el e(-2,-3,-4)
533,44-5)
(-1,22,-3
LR multiplier
0.01
0.01
0.01
0.01
Weight decay
0.05
0.05
0.05
0.05
0.05
0.05
Beta
0.9, 0.999
0.9, 0.999
0.9, 0.999
0.9, 0.999
0.9, 0.999
0.9, 0.999
Epoch
50
80
80
30
LR scheduler
Multi-step 50/10/40 %
Multi-step 50/10/40 %
Multi-step
Multi-step Same
Cosine
Cosine
Default splits
Same
50/10/40 %
50/10/40 %
(train/val/test)
PANGAEA-bench
PANGAEA-bench
Loss function
CE
CE
CE/dice
CE/dice
CE
MSE
characteristics, level of development, and data availability, we select several representative experimen- tal regions for the four tasks, including the Chicago metropolitan area, Singapore, Shenzhen, China and High- figghreesoution RS images for these regions are acquired from AreGIS World Imagery and Bing Map, covering three RGB bands with GSDs of Im or 3m. Corresponding OSM data are retrieved via the Overpass API. Subsequently, we construct high-quality annotated labels required for the four tasks through a combination of web data collection, expert annotation, and manual refinement. For the Urban Functional Zone (UFZ) task, we obtained original urban planning data from official statistics (sources listed in the table below), which are further refined using auxiliary references such Google Maps to ensure their reliability. These refined data were then reclassified into the nine- category UFZ taxonomy: water, green space, farmland, undeveloped land, residential, commercial, institutional, industrial, and transportation. For the UV task, labels were generated entirely through manual annotation. For the POP and CO2 tasks, we leverage two well-established reanalysis datasets to obtain annotations- Wor and the ODIAC Fossil Fuel Emission Dataset-bott of which have been extensively validated in prior research [66 67 68 69 70). We process the RS images, OSM data, and annotations through resampling, reclassification, and other methods obtain the final benchmark datasets.
C.4 Details of downstream evaluation protocols
To ensure the reproducibility of our experiments, we provide detailed hyperparameter settings for all downstream tasks in Table 9 All tasks are optimized using AdamW with a weight decay of 0.05 and 3 values set to [0.9, 0.999]. To preserve the original characteristics of the data, no data augmentation applied in any of the tasks. For classification tasks, we perform grid search over learning rates and report the best results for each model on each dataset. The data is split into 50% for training, 10% for validation, and 40% for testing. random seeds. For semantic segmentation and change detection tasks, all the settings of data split, learning rate, and loss function follow the default of the PANGAFA- of paper submission, PANGAEA-bence has not yet released an officially processed version of the FiveBillionPixel dataset. Therefore, we use the dataset provided in the original FiveBillionPixcl paper (88) crop the images to size of 520 x 520 input, and follow the original data split for consistency. For the UFZ, UV, POP, and CO2 tasks, we also conduct learning rate searches individually to ensure optimal performance for each task.
results are averaged over three runs with different
[17]. Notably, at the time
D Other experiments
D.1 Performance under limited annotations
Performing well under limited labeled data is one of the key metrics for evaluating FMs. To assess this, we conduct experiments using only 10% of the samples on both linear probing and UFZ tasks. Table 10 reports the average results of linear probing across seven datasets, well the results
27

---

for UFZ under unimodal and multimodal settings. Compared with Fig, | and Fig. 3 GeoLink demonstrates even more significant advantages in this scenario, indicating stronger adaptability to downstream applications with limited training samples.
Table 10: Performance comparison on linear probing (top-1 I accuracy %) and UFZ (mloU %) tasks
using only 10% samples for training.
Model
Linear probing
UFZ-unimodal
UFZ-multimodal
Scale-MAE
83.04
39.08
GeoLink
86.17
48.48
53.12
D.2 Impact of loss weights
GeoLink incorporates three distinct learning objectives, and their relative weighting can influence the model's 's performance. In this study, always fix the weight of the image reconstruction loss to 1 and focus on adjusting the weights of the region-image contrastive loss and the spatial consistency loss. The default weight for both is set to 0.01, and conduct experiments by scaling each of them up and down by an order of magnitude. The results are presented in Table [T] We evaluate performance on two downstream tasks: linear probing and UFZ-mulimodal. The results reveal differentiated impact of the two losses. The region-image contrastive loss primarily affects the performance of linear probing, indicating its dominant role optimizing the image encoder. In contrast, variations in the spatial consistency loss have a greater influence on the UFZ-multimodal task, suggesting it plays a crucial role in enhancing gross-modal feature fusion. Setting both loss weights around 0.01 yields favorable balance, facilitating effective synergy among the three objectives.
Table 11: Ablation study of contrastive and consistency loss weights. Evaluation includes linear probing (top-1 accuracy %) and UFZ-multimodal (mloU %).
Contrastive loss
Consistency loss
Linear probing
UFZ-multimodal
0.01
0.01
92.60
60.00
0.1
0.01
91.37
58.43
0.001
0.01
91.89
58.74
0.01
0.1
91.56
57.98
0.01
0.001
92.31
56.18
D.3 Key-point number for position embedding
Table 12: Ablation study the number of key-points for polyline and polygon representations.
Polyline key-points
Polygon key-points
Linear probing
UFZ-multimodal
Centroid endpoints
Centroid + 3 sampling points
92.60
60.00
Centroid
Centroid
92.32
59.46
Centroid endpoints
Centroid + 5 sampling points
91.77
58.02
Since sinusoidal position embedding is not inherently designed to represent the spatial characteristics of polyline and polygon vectors, we propose approximate their spatial coverage by sampling set of representative key-points. The number of sampled points can directly influence the spatial representation and, consequently, the model's performance. Therefore, in this section, we conduct experiments to explore the impact of tey-point number on downstream tasks. First, we represent the positions of both polyline and polygon solely by their centroids, without sampling any key-points. The results of this setting are shown in the second row of the Table Interestingly, using only the centroid as the position proxy still yields competitive performance, with only a slight drop compared to the default setting (first line). Next, we fix the polyline key-points while increasing the number of polygon key-points. Specifically, using the centroid as the center, we randomly sample five (the defalt is three) additional points within the polygon at varying radii and angles, forming total of
28

---

six points to represent its spatial extent. The corresponding results are presented in the third row of the Table 12] where observe noticeable performance degradation. This can be attributed the nature of sinusoidal position embeddings: they encode positional information through a combination of multi-frequency sine and cosine functions. Averaging multiple such embeddings tends to smooth out high-frequency variations, potentially over-smoothing the positional signal and impairing the model's s ability to distinguish spatial patterns. In future work, we aim to explore more expressive and principled methods of position encoding that can seamlessly handle point, polyline, and polygon geometries within a unified framework, thereby enabling more effective spatial correlation.
D.4 Visulization of GeoLink mapping results in UFZ and UV tasks
Overview of GeoLink UFZ mapping result in Singapore
Legend
RS image
Residential
Commercial
Institution
Transportation
Water
Green
Industrial
Undeveloped
Farmland
Region around a port
UFZ prediction
(b) Region around school
RS image
UFZ prediction
Figure 8: Overview and details of GeoLink UFZ mapping result in Singapore.
Large-scale geographic mapping is one of the most important real-world applications of RS data, and also core capability that RS FMs should possess. In this section, we move beyond evaluating GeoLink solely through quantitative metrics, and instead demonstrate its potential for regional geographic mapping by visually comparing its predictions with real-worle spatial layouts. Specifically, we leverage GeoLink to perform full-coverage predictions for Singapore and Beijing (multimodal setting) on UFZ segmentation and UV identification tasks, respectively. The predicted results are stitched together based on the geo-coordinates to generate complete UFZ map for Singapore and UV map for Beijing. As shown Fig. we first present the overall UFZ. mapping results for Singapore. It is evident that the predicted spatial layout aligns well with the actual urban structure of
29

---

GeoLink mapping result Beijing
Urban Village
Figure 9: Overview and details of GeoLink UV mapping result in Beijing.
Singapore. For instance, the airport in the upper right and the industrial zone in the lower left both accurately delineated. Even in the central area, where residential, commercial, and institution zones intermingled, GeoLink is able to make reasonably precise distinctions. We further illustrate two representative local regions in detail: (a) the area surrounding the port (transport zone), where GeoLink clearly captures the spatial boundaries of the port infrastructure; (b) the area around school (institution zone), where the model effectively leverages semantic information from OSM data and road boundary texture from RS image delineate the functional area, despite its complexity. Fig. 9 showcases the results of urban village extraction and visualization in the Beijing region. UVs in Beijing are primarily concentrated within the second ring road, and GeoLink successfully identifies both the location and spatial extent of them (upper right), demonstrating its effectiveness in this task. Integrating RS FMs with mapping-related technologies enable automated, high-precision geographic mapping holds significant promise across various geoscientific domains. At the same time, represents pressing technical challenge that remains be fully addressed. We argue that future evaluations of RS FMs should incorporate real-world mapping performance as a critical metric, thereby continuously enhancing their practical applicability and deployment value in real geospatial scenarios.
D.5 Comparison with domain generalization model
We also compare GeoLink with CrossEarth [71]. a vision FM for domain generalizable RS semantic segmentation, to evaluate the generalization ability. CrossEarth provides multiple domain gener- alization scenarios (e.g.. urban-to-rural, different climate zones) and releases models trained on various source datasets. Based on our experimental setup and the available open-source models, we select the versions trained on ISPRS Potsdam (RGB) and CASID (Temperate Monsoon), as both are closely related to land cover tasks, similar to our two benchmarks on semantic segmentation task: FiveBillionPixele and A14Smallfarms. This setting helps reduce the impact of domain gaps and better reflect the true performance of CrossEarth. the source and target domains share the same label taxonomy. However, this is not the case for our benchmarks. Therefore, we modify the classification head of CrossFarth to match the number of classes in our benchmarks, while keeping all other layers and pretrained weights fixed, We finetune
CrossEarth's 's original zero-shot setting,
30

---

the classification head on the target datasets and compare the results to GeoLink with freezing encoder. This setting helps CrossEarth adapt to our benchmarks and preserve its knowledge learned from the source domain.
Experimental results in the Table show that CrossEarth exhibits strong generalization ability and performs reasonably well, although slightly below GeoLink. Notably, the model pretrained on CASID outperforms the one pretrained on Potsdam, possibly due closer distributional similarity to our benchmark datasets. This comparison highlights both CrossEarth's 's strength as a domain- generalized RS segmentation FM and GeoLink's effectiveness in leveraging OSM data enhance the generalization of RS FM.
Table 13: Comparison GeoLink with CrossEarth semantic segmentation tasks (mloU %)
Model
Source domain
FiveBillionPixels
A14Smallfarms
CrossEarth
ISPRS Potsdam
54.33
38.24
CrossEarth
CASID
56.18
41.07
GeoLink
60.49
43.26
D.6 Comparison with task-specific models
Currently, most geospatial mapping applications still employ task-specific models. To address the need for deciding between using task-specific models FMs, have designed geographic transfer-ability experiment for discussion. We uitlize two task-specific models for comparison. (I) U- Net+OSM: using ImageNet-pretrainee ResNet50 as the backbone to represent convolutional models. Since the fusion module in GeoLink is plug-and woug-and-layy we use the same OSM encoder and fusion encoder from GeoLink to ensure fair comparison. (2) UpPerNet+OSM: using ViT-L as the backbone, representing Transformer- based architectures.
Our UFZ benchmark includes cities with highly distinct geographic conditions: Singapore, Chicago (US), and Shenzhen (China). The original paper uses mixed training/testing setting. To evaluate transfer-ability, we train models one city and test on another. Specifically, we train Geol iink on Singapore and test on Shenzhen and Chicago. As shown the TablcT4, although all models experience performance drops due to domain shift, GeoLink consistently outperform the task-specific models in the target cities, highlighting its superior generalization across regions. This demonstrates the effectiveness of pretrained FMs in capturing transferable geographic features.
Table 14: Comparison of geographic transfer-ability
Source Region
Target Region U-Net+OSM UpPerNet+OSM GeoLink (multimodal)
Singapore
Singapore
54.39
53.80
61.82
Singapore
Shenzhen
32.51
32.21
37.25
Singapore
Chicago 39.62
40.06
49.12
31


Reviews:
[{"review_id": "K3F7dRILqQ", "reviewer": "Reviewer_gYwk", "summary": "This paper proposes GeoLink, a framework to leverage OSM data to enhance Remote Sensing Foundation model. To combine these two modalities, GeoLink uses self-supervised mask reconstruction learning for efficient pretraining. GeoLink is then tested across multiple unimodal tasks and datasets, and multimodal tasks, showing improvements compared to baselines.", "questions": "* There have been quite a few more recent geospatial foundation model that the work should compare to: https://arxiv.org/pdf/2410.22629 If such models are not applicable, please let us know so and why). I may increase the score based on the results. I will decrease the score if no justification or results were given.\n* For Table 1, the paper only compares to DOFA VIT-B. In the DOFA paper, it shows their DOFA VIT-L has slightly better performance. Please include VIT-L as a baseline too or provide a justification on why it shouldn't be. I may increase the score based on the results. I will decrease the score if no justification or results were given.\n* For the multimodal tasks experiments, please include task-specific baselines too. This will help the community understand when to use foundation models vs. task-specific ones. I may increase the score based on the results. I will decrease the score if no justification or results were given.", "limitations": "yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Positives\n* Well-written paper that does well showcasing the intuition and justification for design decisions\n* The idea of doing image mask-reconstruction with OSM and remote sensing image is intuitive and sound and allows leveraging of OSM data in RS. \n* Good improvement on four tasks, the diversity of task is also good (spanning unimodal and multimodal vision tasks)\n* I especially appreciate the OSM data completeness table which adds practical insight since OSM is not always complete.\n\nAreas of improvement\n* There have been quite a few more recent geospatial foundation model that the work should compare to: https://arxiv.org/pdf/2410.22629 If such models are not applicable, please let us know so and why).\n* For Table 1, the paper only compares to DOFA VIT-B. In the DOFA paper, it shows their DOFA VIT-L has slightly better performance. Please include VIT-L as a baseline too or provide a justification on why it shouldn't be. \n* For the multimodal tasks experiments, please include task-specific baselines too. This will help the community understand when to use foundation models vs. task-specific ones.", "was_revised": true, "final_justification": "Based on the rebuttal, I have raised the score. In general, the work is a very nice contribution to the community to empower better Geospatial reasoning, with strong and in-depth experiments and insights.\nThe rebuttal resolved my concern on adding more geospatial foundation models, adding VIT-L as benchmark, adding task-specific baselines.", "rebuttal": "We are very appreciated to your insightful comments. Thank you for recognizing the strengths of our work: clear writing, sound methodology, and thorough experiments. Your suggestions regarding the baseline experiments are also highly valuable and will help us improve the quality of the paper. Please find our point-by-point responses below.\n\n### **Q1. Comparison with CrossEarth**\n\nThank you for your kind reminder. **Since CrossEarth is a vision FM specifically designed for RS semantic segmentation under cross-domain generalization, it can also be applied to some tasks in GeoLink for comparison.** To ensure a fair and reasonable evaluation, our experimental design considers the following two aspects:\n\n**The selection of CrossEarth version:**\nCrossEarth provides multiple domain generalization scenarios (e.g., urban-to-rural, different climate zones) and releases models trained on various source datasets. Based on our experimental setup and the available open-source models, we select the versions trained on ISPRS Potsdam (RGB) and CASID (Temperate Monsoon), as both are closely related to land cover tasks, similar to our two benchmarks on semantic segmentation task: FiveBillionPixels and AI4Smallfarms. This setting helps reduce the impact of domain gaps and better reflect the true performance of CrossEarth.\n\n**Adaptation setting:**\nIn CrossEarth's original zero-shot setting, the source and target domains share the same label taxonomy. However, this is not the case for our benchmarks. Therefore, we modify the classification head of CrossEarth to match the number of classes in our benchmarks, while keeping all other layers and pretrained weights fixed. We finetune the classification head on the target datasets and compare the results to GeoLink with freezing encoder. This setting helps CrossEarth adapt to our benchmarks and preserve its knowledge learned from the source domain.\n\nExperimental results in the table show that CrossEarth exhibits strong generalization ability and performs reasonably well, although slightly below GeoLink. Notably, the model pretrained on CASID outperforms the one pretrained on Potsdam, possibly due to a closer distributional similarity to our benchmark datasets. This comparison highlights both CrossEarth's strength as a domain-generalized RS segmentation FM and GeoLink's effectiveness in leveraging OSM data to enhance the generalization of RS FM. In addition, FMs like CrossEarth, which focus on a specific type of downstream task, offer valuable insights for enabling more efficient adaptation of GeoLink in downstream applications. Combining techniques such as SSL pretraining and domain generalization can further help reduce the annotation requirements of FMs. We will include and discuss this experiment in the final version.\n\n|Model|Source domain|FiveBillionPixels|AI4Smallfarms|\n|-|-|-|-|\n|CrossEarth|ISPRS Potsdam|54.33|38.24|\n|CrossEarth|CASID|56.18| 41.07|\n|GeoLink|/| 60.49| 43.26|\n\n### **Q2. Comparison with DOFA-ViT-L**\n\nThank you for your thoughtful suggestion. We have **replaced DOFA-B with DOFA-L** in our experiments as the basline and re-evaluated the results.In addition, we add another recent RS FM MMEarth \\[*1\\] for comparison. As shown in the table below, we present the average performance of DOFA-B, DOFA-L, MMEarth, and GeoLink across multiple benchmarks for each kind of task. DOFA-L shows clear improvements over DOFA-B, and GeoLink still outperforms DOFA-L in all tasks. We will update the baseline models in the final version of the manuscript.\n\n|Model|Cls(kNN)|Cls (Linear)|Cls (FT)|Seg (Freezing)|Seg (FT)|CD (Freezing)|CD (FT)|\n|-|-|-|-|-|-|-|-|\n|DOFA-B|81.66|90.66|92.17|47.67|54.04|59.55|61.54|\n|DOFA-L|84.67|91.24|93.16|48.07|54.94|60.12|61.95|\n|MMEarth|83.32|90.18|92.73|46.99|52.91|59.37|61.29|\n|GeoLink|87.06|92.60|94.53|51.88|56.11|61.58|63.00|\n\n\n\n### **Q3. Comparison with task-specific baselines**\n\nThank you for your valuable and practical suggestion. Based on your suggestion and the real-world application scenarios in the RS/geospatial community, we evaluate task-specific models for multimodal urban functional zone (UFZ) segmentation from two perspectives: performance under limited annotations and geographic generalization. **GeoLink outperforms task-specific models in both limited-sample and cross-region generalization settings, highlighting its strength in multimodal application scenarios.**\n\n**Task-specific baseline setting:**\nTo the best of our knowledge, there are very few open-source task-specific models in the RS/geospatial domain that use the same types of data as GeoLink. Some proposed methods in the published literature are also difficult to reproduce reliably only based on the descriptions provided. The survey \\[\\*2\\] offers a detailed categorization and summary of existing methods for UFZ segmentation. Following its guidance, we designed several representative task-specific models for comparison.\n(1) **U-Net+OSM**: using ImageNet-pretrained ResNet50 as the backbone to represent convolutional models. Since the fusion module in GeoLink is plug-and-play (Lines 646-648), we use the same OSM encoder and fusion encoder from GeoLink to ensure a fair comparison (also applied to the models below).\n(2) **UpperNet+OSM**: using ImageNet-pretrained ViT-L as the backbone, representing Transformer-based architectures.\n(3) **MSSNet+OSM**: MSSNet \\[\\*3\\] is a task-specific model designed with multi-scale mechanisms tailored for UFZ segmentation.\n\n**Performance under limited annotations:**\nWe test model performance using 10\\%, 50\\%, and 100\\% of UFZ training data. As shown in the following table, compared with the task-specific models, GeoLink performs significantly better with fewer samples. GeoLink using only 10% of the data outperforms these models even when trained with 50% of the data. The performance gap between GeoLink and the task-specific models is largest when using only 10\\% of data, and narrows as more annotations becomes available. Among the task-specific models, MSSNet performs better than U-Net and UpperNet, suggesting that task-specific architectural design can be beneficial in some scenarios.\n\n|Data Ratio|U-Net+OSM|UpperNet+OSM|MSSNet+OSM|GeoLink (multimodal)|\n|-|-|-|-|-|\n|10\\%|41.28|42.97|43.09|53.12|\n|50\\%|50.01|49.09|51.96|56.95|\n|100\\%|54.86| 53.77| 55.54|60.00|\n\n**Geographic transferability:**\nOur UFZ benchmark includes cities with highly distinct geographic conditions: Singapore, Chicago (US), and Shenzhen (China). The original paper uses a mixed training/testing setting. To evaluate transferability, we train models on one city and test on another. Specifically, we train GeoLink on Singapore and test on Shenzhen and Chicago. As shown in the table, although all models experience performance drops due to domain shift, GeoLink consistently outperform the task-specific models in the target cities, highlighting its superior generalization across regions. This demonstrates the effectiveness of pretrained FMs in capturing transferable geographic features.\n\n|Source Region|Target Region|U-Net+OSM|UpperNet+OSM|MSSNet+OSM|GeoLink (multimodal)|\n|-|-|-|-|-|-|\n|Singapore|Singapore|54.39|53.82|56.87|61.82|\n|Singapore|Shenzhen|32.51|32.21|33.23|37.25|\n|Singapore|Chicago|39.62|40.06|41.07|49.12|\n\n**Discussion: These results suggest that the choice between RS FMs and task-specific models should depend on training data availability and region scale.** FMs, pretrained on large-scale unlabeled RS data, offer stronger generalization and are well-suited for low-resource or large-scale, cross-region mapping tasks. As for task-specific models, they can often be more flexibly customized in terms of model architecture.When designed carefully, task-specific models can perform competitively with sufficient training data. Furthermore, combining task-specific model design with SSL pretraining on unlabeled data may provide a strong initialization and lead to both task alignment and generalization.\nIn the future, we envision RS FMs becoming more flexible and capable of integrating more modalities—including but not limited to RS images, geospatial vectors, and street-level data—to support an even broader range of geographic tasks.\n\n\\[\\*1\\] Nedungadi V, Kariryaa A, Oehmcke S, et al. Mmearth: Exploring multi-modal pretext tasks for geospatial representation learning. *European Conference on Computer Vision*. Cham: Springer Nature Switzerland, 2024: 164-182.\n\n\\[\\*2\\] Li Z, Chen B, Wu S, et al. Deep learning for urban land use category classification: A review and experimental assessment. *Remote Sensing of Environment*, 311 (2025).\n\n\\[\\*3\\] Du S, Du S, Liu B, et al. Mapping large-scale and fine-grained urban functional zones from VHR images using a multi-scale semantic segmentation network and object based approach. *Remote Sensing of Environment*, 261 (2021): 112480."}, {"review_id": "v3fQZG3PLJ", "reviewer": "Reviewer_uK7F", "summary": "The paper presents an approach to linking remote sensing imagery with OSM data, taking into account the different data structures. The result is OSM enhancing remote sensing data. In addition, the method addresses computational issues by enabling sparse input.  The method is applied to segmentation tasks to test for improvement. The idea is interesting and nicely presented.", "questions": "How would this used in practice? i.e. how does one obtain a RS and OSM pair that are temporally aligned and good data? \n\nHow is the accuracy of the OSM data dealt with?", "limitations": "These have been addressed.", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "ln 27: fix the language\nln 32-33: This statement isn't really true - revise please.\nCaptions of figures should be in the figure environment not in the graphic itself. \n\nThere are important results in the supplementary as well that should perhaps be included in the main part of the paper?", "was_revised": true, "final_justification": "The authors have responded well during the discussion. I am happy with the responses and the proposed edits to the final version of the paper. My scores remain the same.", "rebuttal": "We are very grateful for your insightful comments. We also sincerely appreciate your comprehensive and accurate summary of our work, as well as your recognition of the novelty and writing clarity. Your suggestions are truly valuable and will greatly help us improve the quality of this paper. Please find our point-by-point responses below.\n\n### **Q1. Minor issues in language, statement, and graphics**\n\nThank you for your kind reminder. We will revise the statement in Lines 32-33 to: \"However, the integration of ground-level geospatial data remains relatively underexplored in many existing RS FMs...\". And **we will carefully incorporate all your suggestions regarding writing and figure issues in the final version** of the paper since NeurIPS does not allow manuscript revisions during the rebuttal phase. We sincerely appreciate your thorougth review!\n\n### **Q2. Important results in supplementary**\n\nThank you for pointing this out. Due to the 9-page limit imposed by NeurIPS, we include several experimental results in the supplementary material—such as evaluations on the urban village task, performance under limited annotations, and impact of loss weight. As noted in the NeurIPS guidelines, the final version may include one additional content page. We plan to move some of these important results, such as the urban village evaluation and key ablations, into the main text to better highlight the strengths and characteristics of our approach.\n\n### **Q3. The practical use of GeoLink**\n\nThank you for your valuable and practical suggestion. **One can automatically obtain spatiotemporally aligned RS-OSM data pairs and then rely on robust post-processing methods like similarity check to ensure data quality.** We will explain this from two aspects.\n\n**Obtaining spatiotemporally aligned data pairs**:\nEach RS image used in this study contains metadata on geographic location and timestamp. Based on this information, we retrieve corresponding OSM data via the Overpass API, which **allows querying OSM data within a specified time range**. This processing pipeline is fully automated. Users of GeoLink can replicate this approach using open-access sources (e.g., Google Earth Engine) to obtain RS images, and then retrieve corresponding OSM data using the image's location and timestamp via Overpass or similar APIs.\n\n**Ensuring data quality**:\nWe are aware that OSM data may contain noise due to outdated or inaccuracy annotations, which can result in cases where the OSM data is not sufficiently relevant to the corresponding RS image. To mitigate this, we recommend following the method in SkyScript \\[29\\], where OSM tags are converted into natural language descriptions and vision-language models, like CLIP, are used to **calculate the similarity between RS images and OSM-based textual descriptions.** Low-similarity pairs can then be filtered out to obtain higher-confidence, spatiotemporally aligned samples.\n\nAs stated in the Checklist, we will open-source the data processing code after the blind review to support reproducibility and ease of use.\n\n### **Q4. How to ensure the accuracy of OSM data in GeoLink**\n\nThank you very much for raising this important point. As you mentioned, OSM data collected via crowdsourcing can have quality issues such as outdated or inaccurate annotations. **In this work, we mitigate the impact of OSM noise through data cleaning, sampling-based quality checks, and model robustness:**\n\n**Data cleaning**:\nAfter retrieving OSM data via Overpass, we apply rule-based cleaning to remove common issues—such as fixing or removing invalid polygons (e.g., self-intersections), eliminating duplicate/conflicting objects, and filtering tags with formatting or spelling errors. These steps not only correct obvious errors but also ensure that OSM inputs meet structural requirements for the model. Additionally, the pretraining data used in GeoLink is derived from SkyScript \\[29\\], which verifies the match quality between RS images and temporally aligned OSM data and ranks the match confidence. We only use the top 30% ranked RS-OSM pairs for pretraining, further ensuring alignment quality.\n\n**Sampling-based validation**:\nWe manually sample both pretraining and downstream datasets to inspect OSM quality and its alignment with RS images. From a random sample of 1,000 RS–OSM pairs, less than 0.02% of OSM objects show quality issues.\n\n**Model robustness**:\n**GeoLink is designed with noisy OSM data in mind.**\nFirst, its use of natural language encoding reduces sensitivity to the tag noise. For example, if a *reservoir* is mislabeled as a *lake* in OSM, GeoLink may still obtain some useful supervision during pretraining due to the semantic similarity between \"reservoir\" and \"lake\".\nSecond, the OSM graph encoder uses topological relationships to describe spatial structure as edges, which reduces the impact of geometric inaccuracies and further improves robustness.\n\nIn summary, GeoLink addresses the potential accuracy limitations of OSM data from multiple angles. As demonstrated by our experimental results, despite the real-world noise in OSM data, its integration still enhances the performance of RS FM during both pretraining and downstream tasks."}, {"review_id": "Gc859wa9Tk", "reviewer": "Reviewer_FjL2", "summary": "This paper proposes GeoLink, a remote sensing (RS) foundation model that combines RGB images with OpenStreetMap (OSM) data. The model consists of two separate encoders for each data modality (RS images and OSM), a fusion strategy for combining OSM and image embeddings, and a decoder for the image modality. The pre-training strategy follows a Masked Autoencoder (MAE)-like approach. Results are reported on various datasets within the Earth Observation domain, covering downstream tasks such as classification, segmentation, and urban function zone mapping. The proposed approach demonstrates superior performance compared to related works, highlighting the value of incorporating OSM data during pre-training.", "questions": "- Why does the model not consider multispectral data, such as Sentinel-2? What is the motivation for limiting the RS encoder to RGB images? Given the wide availability and importance of multispectral data in Earth Observation (EO), it would be helpful to understand the reasoning behind this design choice. Are there specific challenges or limitations that prevent the integration of multispectral inputs into the current framework?\n\n\n- How difficult is the overall process of obtaining OSM data? While OSM provides rich geospatial information, it is not as commonly used in EO foundation models as other modalities (e.g., text, multi-spectral, SAR). Could the authors elaborate on the practical challenges of collecting, aligning, or processing OSM data at scale? Are there domain or geographic limitations that affect its applicability?\n\n\n- Why has the model been pre-trained for only 60 epochs? Is this decision based on empirical results, resource limitations, or performance? It would be useful to understand whether longer pre-training might lead to further gains, or if 60 epochs was sufficient based on validation trends.\n\n\n- Regarding the ablation studies, what would be the expected impact of increasing the masking ratio for the RGB encoder (e.g., 80–90%)? Did the authors experiment with or consider more aggressive masking? If so, what were the observed effects?", "limitations": "Yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n\nThe paper is overall well written, which makes it enjoyable to read. However, in some parts (mostly the abstract, introduction, and conclusion), I noticed sentences that might contain patterns commonly found in AI-generated text, such as the overuse of hyphens. Note that I’m not claiming this is AI-generate, is just that the style sometimes feels unusual.\n\nThe novelty is somewhat limited in terms of the model’s components, as most are drawn from existing architectures (e.g., RS encoder: ViT, Mask Decoder: MAE, OSM encoder: GATConv). However, I found the integration of these elements to be valuable and clever.\n\nThe paper includes extensive experiments to demonstrate the value of the proposed approach. The model is evaluated on classification, segmentation, and urban function zone mapping downstream tasks. It shows superior performance on most datasets for each task when compared to related works. Additionally, ablation studies are provided to highlight the contribution of each model component.\n\nThis doesn't influence my decision, but I want to highlight it: the appendix is very informative. It contains detailed and useful descriptions of many aspects mentioned briefly in the main paper, including the model's components, downstream tasks, fine-tuning process, and implementation details.\n\nWeaknesses:\n\nI found the comparison a bit limited. Some of the works included, such as Scale-MAE or Cross-Scale MAE, do not explicitly incorporate multimodality. Since this is a multimodal approach, I believe it would be more appropriate to compare with recent models that utilise other types of modalities. For example: MMEarth and MultiMAE meets EO. Additionally, you might consider works that incorporate textual modalities. A broader perspective can be found in this recent survey: https://arxiv.org/pdf/2505.14361.\n\nI don’t fully understand the motivation for including only RGB images in this domain. While RGB is the most abundant data type in EO, many datasets include multispectral data (e.g., Sentinel-2). Evaluating such data would make the submission stronger and more applicable to real-world EO scenarios, where S2 data is widely used. I understand that some recent EO models (e.g., CLIP-based) are limited to RGB due to the constraints of pre-trained models. However, in this case, the encoder is not fixed to such limitations. I’m not against using only RGB, nor am I criticising this point directly, but the paper lacks a convincing motivation for this design choice.\n\nSome minor formatting errors and suggestions:\n- I’d suggest placing Fig.2 on page 3. I think it will help the reader to have the figure in the same page where the description of the approach starts. \n- Fig 2. Part (a). I’d suggest changing the abbreviation of projection to ‘proj’ instead of ‘prj’ on the blue box under ‘contrastive loss’. I agree that both abbreviations are correct, however since this is a computer vision paper, I believe ‘proj’ is more common.\n- Line 139: Missing space between text and equation.\n- Line 166: I’d suggest adding a reference to set2set layers (https://arxiv.org/pdf/1511.06391). Not sure if the term is universally known.\n- Line 226: Six benchmarks are mentioned but only five examples given.\n- I’d suggest including the numbers on each bar of Figure 3 (something like Figure 4).  Probably, it would be more valuable to present these numbers in a table format (?).\n- Inconsistent capitalisation in section headers. For instance, line 112 uses \"Framework Overview\" while line 97 has \"Synergy of RS and geospatial vector data.\" I recommend adopting a consistent style throughout the manuscript.", "was_revised": true, "final_justification": "The authors have properly addressed all of my concerns with both experimental evidence and thoughtful explanations, as demonstrated in their detailed rebuttal. I still believe that future work, such as the incorporation of multispectral data, could make the publication even stronger. However, despite this, the paper has merit, and I believe it would be a valuable contribution to NeurIPS. Therefore, I am increasing my previous rating to accept.", "rebuttal": "We are very grateful to your insightful comments. Thank you for recognizing the strengths of our work: clear writing, novel framework, and good evaluation. Your suggestions regarding the multispectral EO data and baseline experiments are also highly valuable and will help us improve the quality of the paper. Please find our point-by-point responses below.\n### **Q1. The use of multispectral EO data**\n\nThanks for pointing this out. We agree that the inclusion of multispectral EO images could enhance the model. However, we would like to emphasize that the focus of this paper is to explore how to leverage the rich geographic information of OSM data in RS FM, not to include as much data as possible. Due to the challenges in OSM vector encoding and large domain gap, the use of OSM vector data in RS FMs is under-explored. **This paper aims to bridge this gap and demonstrate the value of OSM in improving the performance and task diversity of RS FM, not the designing of multispectral RS encoder.** \n\nMoreover, **GeoLink can be readily extended to support multispectral RS images by modifying the image encoder**. In its current design, GeoLink's fusion mechanism imposes no specific constraints on the image encoder. Therefore, in principle, we can replace the existing ViT-based encoder with any encoder capable of handling multispectral data, without modifying other key components of the architecture. Previous multispectral pretraining approaches—using either separate encoders (e.g., SkySense) or joint tokenization (e.g., DOFA)—offer promising ways for adapting GeoLink to multispectral data. \n\n**As mentioned in the conclusion section (Lines 345-347), making GeoLink to support multispectral inputs is an important direction**, and we are excited to explore this further in future work.\n\n### **Q2. Baseline model settings**\nThank you for your insightful suggestion. **To the best of our knowledge, there is currently no RS FM that uses the exact same data types as GeoLink**, making it difficult to directly compare with fully modality-aligned baselines. The starting point of our experimental design is to explore how, and to what extent, integrating OSM data can enhance RS FM.\n\nTo fairly and comprehensively evaluate GeoLink, we consider baselines designed from different perspectives:\n* Unimodal RS FMs that use the same RS spectral modality as GeoLink, including GASSL, ScaleMAE, and Cross-scale MAE;\n* Multimodal models that incorporate additional modalities, such as CROMA and DOFA. **Following your suggestion, we find that MMEarth and MultiMAE Meets EO are indeed suitable baselines for GeoLink, as they incorporate other modalities in their designs, like land cover maps, which are similar to OSM data from the perspective of information content.** We have included MMEarth as a baseline for GeoLink in our updated comparisons, and as for MultiMAE Meets EO, we do not find its public checkpoints currently.\n\nWe also replace DOFA-B by DOFA-L for comparison. The table below shows the average performance across multiple benchmarks for each task, where GeoLink consistently outperforms MMEarth and DOFA-L.\n\n|Model|Cls (kNN)|Cls (Linear)|Cls (FT)|Seg (Freezing)|Seg (FT)|CD (Freezing)|CD (FT)|\n|-|-|-|-|-|-|-|- |\n|MMEarth|83.32|90.18|92.73|46.99|52.91|59.37|61.29|\n|DOFA-L| 84.67|91.24|93.16|48.07|54.94|60.12|61.95|\n|GeoLink|87.06|92.60|94.53|51.88|56.11|61.58|63.00|\n\nMoreover, **we believe it is not appropriate and fair to compare GeoLink with vision-language models (VLM) that integrate textual information, because they are fundamentally different:**\n* As noted in the review article \\[\\*1\\] you list, most RS VLMs are not trained from scratch but are finetuned from VLMs from computer vision field like CLIP, which are pretrained on massive natural images (e.g., 400M image-text pairs for CLIP pretraining). In contrast, RS FMs are generally trained from scratch and VLMs are not used as baselines in previous RS FM works \\[3, 10, 32, 33, 34\\] as well.\n* VLMs are typically evaluated in zero-shot settings and focus on aligning images with textual semantics, which is fundamentally different from GeoLink's motivation—leveraging freely available geospatial data to improve RS FM.\n\n\n\\[\\*1\\] Weng X, Pang C, Xia G S. Vision-Language Modeling Meets Remote Sensing: Models, datasets, and perspectives. IEEE Geoscience and Remote Sensing Magazine, 2025.\n\n### **Q3. The process of obtaining OSM data**\nThank you for the insightful question. We would like to clarify that **obtaining OSM data is relatively straightforward and scalable**. The OSM data is openly accessible through public APIs (e.g., Overpass) and bulk downloads (e.g., Geofabrik). **Our data processing pipeline is fully automated** to extract relevant OSM vector features for any given region. Specifically:\n1. We use the geolocation coordinates and timestamps of RS images to query the corresponding OSM data via the Overpass API;\n2. The OSM objects are clipped using spatial overlay to align them accurately with the RS images, forming image-OSM sample pairs;\n3. We use rule-based cleaning to fix common issues—like invalid polygons (e.g., self-intersections), duplicate or conflicting objects, and tags with formatting or spelling errors. This helps correct errors and ensures OSM inputs meet GeoLink's structural needs.\n4. After encoding the tags and computing spatial relationships, we construct OSM graphs as input to the model.\n\nIn our experience, while many studies highlight the importance of OSM in GeoAI \\[19\\] , its practical usage remains limited—**mainly due to challenges in effectively encoding its semantic and geometric information and integrating it with EO data.** GeoLink is designed to address these bottlenecks.\n\nAdditionally, as you suspect, despite its many advantages, OSM data does have limitations in practice. One key concern is its **uneven global coverage (e.g., better in developed urban centers than in rural areas)**, which may introduce geographic bias (which is an objectively common issue in FMs \\[19\\]) when used for pretraining FMs. To address this, one can explore the following directions:\n\n**Data-side**: Mitigating the systematical bias by increasing sampling density in OSM-sparse regions;\n\n**Model-side**: Developing globally location-aware FMs that treat geographic space as a continuous input to guide the modeling of priori knowledge of geographic distribution, as explored in \\[42\\].\n### **Q4. Pretraining epochs**\nThank you for your thoughtful observation. The decision to pretrain GeoLink for 60 epochs is driven by a combination of empirical findings and resource constraints. **We find that training for more epochs (e.g., 100) lead to only marginal performance gains, indicating its convergence around epoch 60.** Given the limitation of computational resources, we choose 60 as a practical default.\n### **Q5. Masking ratio for image encoder**\nThank you for the helpful suggestion. We experiment with masking ratios of 80% and 90%. As shown in the table below, we observe a slight performance drop at 80% and a significant collapse at 90%. This result is consistent with observations from the original MAE paper. Therefore, we adopt a 75% masking ratio for image patches in our study.\n\n|Mask Ratio|Cost|kNN|Linear|UFZ-U|UFZ-M|\n|-|-|-|-|-|-|\n|75%(Default)|1.0×|87.06| 92.60|55.40|60.00|\n|80%|0.95×|85.21|90.17|52.01|56.04|\n|90%|0.9×|78.02|83.47|43.02|45.81|\n\n### **Q6. Minor formatting errors**\nWe sincerely appreciate your kind reminder. Due to NeurIPS policy, we cannot update the manuscript during the rebuttal phase. And **we will carefully revise the final version to address the formatting and writing issues you kindly pointed out**. Thank you again!"}, {"review_id": "DwlAkuACst", "reviewer": "Reviewer_6XcF", "summary": "This paper introduces GeoLink, a multimodal framework that integrates OpenStreetMap (OSM) data with remote sensing (RS) foundation models to enhance geospatial intelligence. GeoLink leverages OSM data to enrich the RS foundation model during both pretraining and downstream task stages through multi-granularity learning signals, masked input pretraining, and RS-OSM fusion. The framework demonstrates significant improvements in the performance of the RS image encoder and shows strong adaptability to complex geographic scenarios across various tasks.", "questions": "1. In the multimodal downstream tasks, the hybrid RS-OSM patch encodings enrich each visual patch with OSM context. This works by attending to OSM object features. Is there a risk of spatial \"blurring\" where a patch at the edge of a large feature (e.g., a large \"park\" polygon) receives the same aggregated contextual vector as a patch in the center? How does the model differentiate fine-grained intra-object location? Does the sinusoidal positional embedding for the patch itself provide enough signal to mitigate this?\n2. The ablation study on OSM completeness (Table 4) is insightful. However, it relies on random removal. In reality, OSM data quality is systematically biased (e.g., better coverage in developed urban centers than in rural areas). How might this geographic bias in pretraining data affect the model's generalization performance? Does the model perform better on downstream tasks in regions that are well-represented in the pretraining dataset?\n3. The paper provides excellent ablations on the learning objectives. Could you provide some insight into the design of the OSM encoder itself? How critical are the different topological edge types? For instance, what would be the performance impact of a simpler model that only considers a \"disjoint\" (via Delaunay) and a generic \"intersects/contains\" relationship, thereby reducing the GNN's complexity?\n4. The framework is designed for OSM. How do you envision GeoLink being adapted for other types of geospatial vector data that may have different semantics or structures, such as official administrative boundaries, cadastral (property) maps, or utility networks? Would the general framework of a GNN encoder + fusion module still apply?\n5. The experimental comparison is insufficient. (1. GeoLangBind-L needs to be added as a comparison method. (2. DOFA needs to adopt the ViT-L version; otherwise, the comparison would be unfair.\n6. Does the result of Scale-MAE+OSM indicate that the proposed designed fusion encoder is a plug-and-play module and whether it can be attached to other advanced models for experimentation?", "limitations": "yes", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths\n1. GeoLink innovatively combines RS and OSM data into a comprehensive multimodal framework. This integration not only addresses the structural differences between the two data types but also enhances the semantic understanding of RS images through spatial context.\n2. By utilizing the multi-granularity nature of OSM data, GeoLink designs region-image level alignment and object-patch level interaction to capture information across different spatial scales. \n3. Adopting a masked input approach for both RS images and OSM graphs during pretraining, GeoLink achieves efficient training while maintaining performance improvements. \n\nWeaknesses\n\n1. The paper does not discuss the computational scalability of the GNN-based OSM encoder and the object-patch fusion module, particularly in regions with very high object density. In dense urban cores, a single scene can contain thousands of OSM objects, which could make the graph construction, message passing, and especially the cross-attention mechanism computationally prohibitive. A discussion on these limitations and potential mitigation strategies (e.g., graph sampling) would strengthen the paper.\n2. While the overall framework is highly novel, the individual building blocks (ViT-MAE, GATConv, InfoNCE, cross-attention) are established techniques. \n\n3. Although GeoLink shows some robustness to incomplete OSM data, the performance may still be affected in regions with sparse OSM coverage. The authors need to further explore methods to reduce this dependency or supplement missing data.\n\n4. Despite faster convergence during pretraining, the multimodal framework involving multiple encoders and complex GNN structures may lead to higher computational costs during training and inference. The training is fast, but don't know how efficient it is during inference. The paper does not discuss the inference time of GeoLink, which is crucial for real-time applications. The authors should evaluate and improve the model's inference efficiency.", "was_revised": true, "final_justification": "I maintain my score. The author addressed my questions. But I think the technical contribution of this work is still rather limited.", "rebuttal": "We are very grateful to your thoughtful comments and recognition of GeoLink's novelty, quality, and significance. We also appreciate your concerns on the computational scalability, OSM data, and so on, which will help us improve the quality of the paper. Please find our point-by-point responses below.\n\n### **Q1. The computational scalability**\nThank you for your insights on the model architecture. We understand the concern, but **based on our current design, the OSM encoder and fusion module do not significantly increase the computational cost.**\n\nFirst, both the OSM encoder (9 GAT layers, Fig. 5) and the fusion module (3 attention layers, dim=256, Fig. 6) are lightweight, and very few training parameters are introduced.\n\nSecond, during pretraining, the fusion module only applies cross-attention between masked 20% OSM nodes and unmasked 25% RS patches, greatly reducing computation. The table below shows memory usage with/without the fusion module under standard settings.\n\n|Fusion encoder|Pretraining memory|\n|-|-|\n|With|46955 MB|\n|Without|46121 MB|\n\nThird, the fusion module outputs lower-dimensional features (256 vs. 1024 in ViT-L). In downstream tasks like semantic segmentation, where decoder size depends on feature dimensions, this can reduce memory usage. As shown in the table, the multimodal setting uses less memory than the unimodal one in the UFZ segmentation task.\n\n|Mode|UFZ memory|\n|-|-|\n|Unimodal|17105 MB|\n|Multimodal|11517 MB|\n\nTo address concerns about high memory usage from dense OSM regions, we currently limit the number of OSM nodes to 1000 per sample through random sampling. Our data shows that fewer than 0.01% of samples exceed this threshold, so their impact on training is minimal. In the future, we plan to explore more targeted sampling strategies based on entropy or clustering. For extremely dense cases, we can apply strategies like cropping the spatial coverage or increasing RS image resolution to reduce the number of included OSM objects.\n### **Q2. Bias in OSM data**\nThank you for your insightful comments and for recognizing the value of our ablation study. Geographic bias caused by the distribution of pretraining data is an objectively common issue in FMs \\[19\\], and GeoLink is no exception. However, **it is inherently difficult to disentangle whether the performance differences across regions stem from geographic bias encoded in the FMs, or from the intrinsic spatial heterogeneity between regions.**\n\nWe conduct UFZ segmentation task separately in Shenzhen (China) and Chicago (US) to illustrate this. As noted in \\[*1\\], OSM coverage in China is considerably lower than in the US. Results in the table below show that GeoLink does not perform as well in Shenzhen as it does in Chicago. While this might be partially attributed to geographic bias, it is not sufficient to conclude that GeoLink exhibits significant bias, as as Shenzhen's complex urban structures make the task more challenging. As highlighted in \\[19\\], there is no work on accurately quantifying geographic bias in FMs currently. Designing convincing experiments to isolate geographic bias in FMs remains difficult, and we believe this is an exciting and important direction for future research.\n\nOn the other hand, there are some practical ways to mitigate geographic bias during FM pretraining:\n\n**Data-side**: As you noted, OSM data quality is systematically biased, and we can mitigate this by increasing sampling density in OSM-sparse regions;\n\n**Model-side**: Developing globally location-aware FMs that treat geographic space as a continuous input to guide the modeling of prior knowledge of geographic distribution, as explored in \\[42].\n\n|Region|Unimodal|Multimodal|\n|-|-|-|\n|Shenzhen|34.12|40.45|\n|Chicago|47.28|55.10|\n\n### **Q3. Inference efficiency**\nWe appreciate the reviewer's concern. In our default settings, GeoLink's inference speed on the UFZ segmentation task is 83.2 samples/sec for unimodal and 76.0 samples/sec for multimodal. The slower inference in the multimodal setting is typical due to increased data and model modules. However, this speed is acceptable for most practical applications. For time-sensitive tasks, we plan to use model distillation or data caching to improve efficiency.\n\n### **Q4. Spatial \"blurring\" effect**\nThank you for your insightful comments. In GeoLink, each image patch is assigned a distinct position embedding based on its spatial location, leading to different cross-attention interactions with the same OSM object. Therefore, **a patch at the edge of a large object receives a different aggregated contextual vector to a patch in the center, reflecting spatial distinctions within an object.**\n\nDistincting fine-grained intra-object location requires an accurate and complete representation of polygon geometry—an ongoing challenge in vector-based geospatial encoding. In this work, we adopt sinusoidal position embeddings combined with keypoint sampling to approximate polygon shapes. Empirically, the outstanding performances of GeoLink on multimodal fine-grained tasks demonstrate that this strategy is effective and fine-grained spatial interaction contributes meaningfully to data fusion. **As described in Lines 347-350, we believe that more accurate position embeddings could further enhance GeoLink**. Recent works on geo-vector encoding like Poly2Vec is particularly inspiring for improving our position embedding scheme. In the future, we hope more studies can be proposed to focus on positional modeling approaches for fine-grained multimodal geospatial data fusion.\n### **Q5. Ablation studies for OSM encoder**\nThank you for your suggestion and recognition of our ablation experiments. We conduct two additional ablation studies:\n\n**Less Edge**: Only five types of spatial relations (Delaunay, intersect, within, cross, and overlap) are retained;\n\n**No Edge**: All spatial edges are removed.\n\nAs shown in the table, the \"Less Edge\" setting causes a slight performance drop, highlighting the value of diverse spatial relations. The \"No Edge\" variant performs worse, likely because without edges, the OSM encoder lacks message passing—masked nodes can't gather spatial context to query image features, which weakens the pretraining process.\n\n|OSM Encoder Setting|kNN|Linear|UFZ-U|UFZ-M|\n|-|-|-|-|-|\n|Default|87.06|92.60|55.40|60.00|\n|Less Edge|86.95|91.97|55.02|57.84|\n|No Edge|85.20|89.47|52.88|55.65|\n\n\n### **Q6. Adapting GeoLink to other geospatial vector data**\nFrom a data structure view, **GeoLink's vector modeling strategy is applicable to other geospatial vector datasets, such as cadastral maps.** Its fusion module remains effective regardless of the specific vector kind. However, from an information content view, some geospatial data—such as administrative boundaries—contain limited fine-grained information. These datasets might be better suited as spatial constraints during training rather than as rich semantic sources, and thus may offer limited enhancement to GeoLink. **Therefore, adapting GeoLink to different types of vector data should be guided by the specific characteristics and information content of the data.**\n### **Q7. Comparison with GeoLangBand-L and DOFA-L**\nThank you for your suggestions. We test GeoLangBand-L and DOFA-L, and the results are as follows. GeoLink outperforms DOFA-L in all tasks and outperforms GeoLangBand-L in 4/7 tasks. It should be noted that unlike RS FMs, **GeoLangBand is a vision-language model (VLM) finetuned on SigLIP, which makes the comparison unfair.** Specifically, they are fundamentally different since:\n\n1. Like most RS VLMs, GeoLangBand is not trained from scratch—its SigLIP backbone is pretrained on 400M image-text pairs. In contrast, RS FMs are generally trained from scratch and VLMs are not used as baselines in previous RS FM works \\[3, 10, 32, 33, 34\\] as well.\n2. GeoLandBand also distills from DinoV2 (1.2B samples) and ImageNet ViT. Totally, the pretraining data accessed by GeoLink (1.27M) accounts for less than 0.08% of that in GeoLangBand (more than 1.6B).\n3. VLMs are typically evaluated in zero-shot settings and focus on aligning images with textual semantics, which is fundamentally different from GeoLink's motivation—leveraging freely available geospatial data to improve RS FM.\n\nThus, based on resource consumption, design motivation, and learning paradigm, GeoLangBand should be benchmarked against other CLIP/SigLIP-based RS VLMs, rather than RS FMs pretrained from scratch.\n\n|Model|Cls (kNN)|Cls (Linear)|Cls (FT)|Seg (Freezing)|Seg (FT)|CD (Freezing)|CD (FT)|\n|-|-|-|-|-|-|-|-|\n|GeoLangBand-L|88.98|93.12|94.98|51.47|55.92|61.23|62.72|\n|DOFA-L| 84.67|91.24|93.16|48.07|54.94|60.12|61.95|\n|GeoLink|87.06|92.60|94.53|51.88|56.11|61.58|63.00|\n\n### **Q8. Plug-and-play fusion encoder**\nThe fusion encoder is plug-and-play and can be used to enhance other unimodal models. Combining suggestions from you and Reviewer gYwk, we combine the fusion module with three semantic segmentation networks for the UFZ task:\n\n**U-Net**: Uses an ImageNet-pretrained ResNet50 backbone, representing CNNs.\n\n**UpperNet**: Uses an ImageNet-pretrained ViT-L backbone, representing Transformers.\n\n**MSSNet**: A multi-scale semantic segmentation network \\[*2\\] specifically designed for UFZ tasks.\n\nPerformances of all three models are significantly improved after integrating the fusion module, verifying its plug-and-play flexibility.\n\n|Setting|U-Net|UpperNet|MSSNet|\n|-|-|-|-|\n|Original|50.12|49.77|52.06|\n|+OSM|54.86|53.77|55.54|\n\n\\[\\*1\\] Herfort, B., Lautenbach, S., Porto de Albuquerque, J., Anderson, J., & Zipf, A. (2023). A spatio-temporal analysis investigating completeness and inequalities of global urban building data in OpenStreetMap. *Nature Communications*, 14, 3985\n\n\\[\\*2\\] Du S, Du S, Liu B, et al. Mapping large-scale and fine-grained urban functional zones from VHR images using a multi-scale semantic segmentation network and object based approach. *Remote Sensing of Environment*, 261 (2021): 112480."}, {"review_id": "SpEQXZnJxE", "reviewer": "Reviewer_BhhX", "summary": "This paper presents a new self-supervised learning (SSL) algorithm, called GeoLink, for pre-training remote sensing (RS) models. Specifically, GeoLink fuses RGB satellite imagery with OpenStreetMap (OSM) data.\n\nThe pre-training algorithm consists of:\n- Separately masking and encoding satellite imagery and OSM data. The OSM encoder is a Graph Neural Network (GNN), the image encoder is a ViT. The GNN-node features are computed using a language model.\n- A contrastive loss term is applied between visible encodings across these two modalities\n- A reconstruction loss computed using an MAE-style decoder that predicts the hidden RGB patches (without OSM fusion).\n- Another reconstruction loss, called a “spatial consistency loss”, computed using an “object-patch” fusion encoder that receives masked node embeddings and predicts the initial / hidden node embeddings, while cross-attending to the visible RGB patch encodings.\n\nThe paper demonstrates that GeoLink outperforms prior pre-trained RS models on many classification and segmentation tasks.", "questions": "How are the multispectral baseline methods used when processing RGB-only benchmark data?", "limitations": "Yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n- The paper is dense and well-written\n- I agree that pre-trained RS models have not yet fully leveraged data like OSM and this is a promising research direction\n\nWeaknesses:\n- The satellite images are RGB-only. In this field (pre-trained RS models), most papers use multispectral satellite images as these additional bands are useful for many downstream tasks. Furthermore, this RGB-only choice makes comparisons a bit tricky since baselines were pre-trained on multispectral data (like CROMA).\n- The main contribution of the paper (to a conference like NeurIPS) is the SSL algorithm, which I have concerns about. \n  - I understand that OSM can be encoded with a GNN, but can’t it also be encoded with a transformer? The field has settled on transformers for multimodal RS, so this deviation should be better motivated and empirically demonstrated (e.g., by showing it works better than a transformer encoder).\n  - A language model (BERT) is used to encode OSM nodes that are fed to the GNN. I don’t see an ablation that entirely removes the GNN and just uses the BERT encodings. It is certainly possible that all the gains of GeoLink can be explained by this natural language supervision — rather than the algorithm itself.", "was_revised": true, "final_justification": "The authors have addressed my major concerns and thus I recommend the paper is accepted.", "rebuttal": "We are very grateful for your thoughtful comments, recognition of the clarity of our writing and promising research direction, and suggestions for improvement on image spectral and ablation study. Your suggestions greatly help us improve our analysis of GeoLink. Please find our point-by-point responses below.\n### **Q1. The use of multispectral satellite image**\n\nThanks for pointing this out. We agree that the inclusion of multispectral images could enhance the model. However, we would like to emphasize that the focus of this paper is to explore how to leverage the rich geographic information of OSM data in RS FM, not to include as much data as possible. Due to the challenges in OSM vector encoding and large domain gap, the use of OSM vector data in RS FMs is under-explored. **This paper aims to bridge this gap and demonstrate the value of OSM in improving the performance and task diversity of RS FM, not the designing of multispectral RS encoder.** \n\nMoreover, **GeoLink can be readily extended to support multispectral RS images by modifying the image encoder.** In its current design, GeoLink's fusion mechanism imposes no specific constraints on the image encoder. Therefore, in principle, we can replace the existing ViT-based encoder with any encoder capable of handling multispectral data, without modifying other key components of the architecture. Previous multispectral pretraining approaches—using either separate encoders (e.g., SkySense) or joint tokenization (e.g., DOFA)—offer promising ways for adapting GeoLink to multispectral data. \n\n**As mentioned in the conclusion section (Lines 345-347), making GeoLink to support multispectral inputs is an important direction**, and we are excited to explore this further in future work.\n\n### **Q2. Comparison with multispectral pretrained models**\n\nThank you for raising this concern. To the best of our knowledge, **there is currently no RS FM that uses the exact same data modalities as GeoLink**, making it difficult to directly compare with fully modality-aligned baselines. The starting point of our experimental design is to explore how, and to what extent, integrating OSM data can enhance RS FM. In the experiments, we are not only considering multispectral baselines but also RGB-only baselines. The comparison with baselines designed from different perspectives provides more comprehensive evluations.\n\nMoreover, **multimodal pretraining models like CROMA and DOFA are specifically designed to accommodate multispectral RS images, and their pretraining data includes RGB bands**. Therefore, their pretrained models should—and must—be capable of processing RGB RS images. Additionally, many benchmark studies for RS FMs (e.g., \\[1, 17, 18\\]) also compare different spectrally pretrained RS FMs on the same downstream tasks. For example, \\[17\\] compares CROMA (multispectral) with Scale-MAE (RGB) on RGB-only benchmarks like xView2 and \\[1\\] also compares Prithvi-eo-2.0 (multispectral) and Scale-MAE on RGB-only benchmarks like pv4ger. Thus, we believe it is fair to compare GeoLink with multispectral pretraining models like CROMA and DOFA.\n\n### **Q3. The setup of multispectral baselines for RGB-only benchmarks**\n\nThank you for your question. Regarding the testing setup, we follw Pangaea-bench \\[17\\] to evaluate multispectral baseline methods on our benchmark data.\n\nSpecifically, due to the distinct characteristics of DOFA and CROMA, different strategies are employed to adapt them to the RGB-only testing benchmark. The original DOFA model includes a dynamic projection module as a tokenizer to normalize various spectral inputs, allowing it to directly process RGB images without additional modifications. For CROMA, we use its multispectral optical encoder (pretrained on Sentinel-2 L2A) and employs zero-padding to fill in missing spectral bands for input.\n\n\n### **Q4. The designing of Transformer-based OSM Encoder**\n\nThank you very much for your suggestion. As a type of vector data, the structure of OSM makes it difficult to directly process using Transformers in the same way as images. Due to their strengths in spatial modeling (e.g., effectively capturing spatial relationships), GNNs have long dominated research on encoding vector data like OSM.\n\nTo leverage the power of attention mechanisms, this study does not use the original GCN but instead employed GAT to construct the OSM encoder. To further validate your suggestion, we design two variant experiments to illustrate the effectiveness of the current OSM encoder. The experimental setups are as follows:\n\n**GCN Variant**: Replace the GAT in the OSM encoder with GCN while keeping all other settings unchanged.\n\n**Transformer Variant**: Replace the existing GNN-based OSM encoder with a Transformer containing two attention layers. To adapt to the Transformer’s structure, each OSM node in the original GNN is treated as a token input to the Transformer. Additionally, position embedding is added to each token to preserve spatial information. All other settings remain unchanged.\n\nThe experimental results are shown in the table below:\n\n|OSM Encoder Setting|kNN|Linear|UFZ-U|UFZ-M|\n|-|-|-|-|-|\n|Default|87.06|92.60|55.40|60.00|\n|GCN| 87.01|92.14|55.02|59.42|\n|Transformer|86.28|91.34|53.99|58.73|\n\nCompared to the GCN variant, the attention mechanism introduced by GAT yields better performance. The Transformer variant underperforms both GCN and GAT, which we speculate is primarily due to the insufficient message-passing between OSM nodes in it, hindering the learning of OSM spatial consistency objective during pretraining.\n\nIn summary, **the experimental results demonstrate that the current OSM encoder works synergistically with other components of GeoLink and achieves satisfactory performance.** In the future, we will explore more sophisticated Transformer-based OSM encoder designs to pursue potential additional improvements.\n### **Q5. Ablation studies by removing GNN**\n\nThank you very much for proposing such a valuable ablation study design! In fact, **the gains of GeoLink stem from the synergy of multiple components in the overall architecture, including the semantic and geometric information from OSM, as well as the structural design and learning objectives that effectively transmit this information to model training.**\n\nFollowing your suggestion, we design an ablation experiment: removing the OSM encoder, using a frozen BERT to encode OSM tags, applying average pooling to aggregate tag encodings of multiple OSM objects within a sample, and then using a linear projection layer to align the features with image encodings for contrastive learning. Notably, this ablation approach also renders GeoLink's original spatial consistency loss and object-patch fusion encoder unusable. Thus, the pretraining signals for the model come solely from image reconstruction and cross-modal contrastive loss, achieving what you refer to as \"natural language supervision\". The experimental results are shown below:\n\n|OSM Encoder Setting|kNN|Linear|UFZ-U|UFZ-M|\n|-|-|-|-|-|\n|Default|87.06|92.60|55.40|60.00|\n|No GNN|85.01|90.37|52.66|/|\n\nCompared to the original GeoLink, the ablated setup exhibits a significant performance drop, indicating that the gains of GeoLink do not come solely from natural language supervision. According to GeoLink's design, a key aspect of OSM's supervisory signal lies in using vectors' spatial information to guide the tags' semantic information for fine-grained interaction and fusion. The implementation of this learning signal requires the synergy of the OSM encoder, fusion encoder, and learning objectives. Removing the GNN-based OSM encoder disrupts this mechanism, making it difficult to achieve comparable results.\n\nTherefore, the gains of GeoLink arise from the synergistic design of multiple components in the architecture, not just natural language supervision."}]

Author rebuttal:

Meta-review: GeoLink incorporates the public data available through OpenStreetMap (OSM) into the self-supervised pre-training of remote sensing models. The technical contributions include the joint model of image and graph data, the data collection and filtering process for the graph data from OSM to make it reliable for self-supervision, and the design of the multi-scale losses across the imagery and OSM data. The experiments cover the use of OSM data upstream, during pre-training with multi-scale masking and contrasting tasks across image data from satellites and graph data from OSM, and downstream, during deployment by fusing imagery and the OSM data. The evaluations cover multiple remote sensing tasks (agriculture, urban land use and functional zones, disaster management, population, and carbon emissions) formulated as semantic segmentation, change detection, and regression. The proposed full model improves accuracy when transferring features and fine-tuning compared to multi-scale baselines (Scale-MAE, Cross-Scale MAE) and multi-modal baselines (CROMA, DOFA).

Five reviewers with expertise on vision, self-supervision, remote sensing vote for acceptance (gYwk: 5, uK7F: 5, FjL2: 5, BhhX: 4) and rejection (6XcF: 3). The review by uK7F is short and non-technical, although the reviewer did engage with the rebuttal, so this review has been downweighted in the decision. The borderline scores are the result of detailed evaluations and discussion and are maintained so these are fully considered in the decision. These final scores followed the rebuttal by the authors, which resulted in raised (gYwk, FjL2) and maintained scores. The area chair notes that the review process has further improved the submission with additional details on the collection and processing of the OSM data, additional and more recent baselines for multimodal remote sensing, discussion of limitations like the choice of RGB-only imagery and computation, and further ablations of the proposed multi-part framework. The final justifications of all reviewers except 6XcF confirm acceptance. The discussed modeling weakness of relying on over-simplified image input (just RGB, and not the multi-spectral images or other modalities common to remote sensing) can be reinterpreted as a strength by the counterpoint that with OSM data the model nevertheless performs satisfactorily. The remaining argument for rejection by 6XcF is insufficient technical novelty, but it is acknowledged that the rebuttal addressed the points raised by the review. The area chair agrees with the reviewer that the overall framework is novel, even if each component is not, and so on the balance this issue is resolved. The submission is informative, significant, and novel in its incorporation of the vector-format OSM data into deep learning for remote sensing.

After full inspection of the submission, the reviews, the rebuttal, and discussion the area chair sides with the majority of reviewers on acceptance.
Decision: Accept (poster)