# MGKGR: Multimodal Semantic Fusion for Geographic Knowledge Graph Representation
## 1 Introduction  
Geographic knowledge graph (GeoKG) organizes the entities, attributes, and spatial relationships into triples of the form \(<\)_h,r,t\(>\)_, where \(h\) and \(t\) represent the entities in the triple, and \(r\) represents the relationship between them. GeoKG provides basic knowledge for researchers and decision-makers to understand the interactions between complex geographic environments and human activities [1]. Knowledge graph representation learning (KRL) is the emerging method that facilities GeoKG for various geographic applications, which transforms entities and relationships of a knowledge graph into low-dimensional embeddings. Thanks to that KRL not only effectively represents geographic data, but also facilitates the further exploration of potential associations between entities in GeoKG; KRL has served as an essential task, and makes great progress in geographic applications, such as spatial analysis [2, 3], spatial planning [4], POI recommendation [5, 6, 7], geographic entity retrieval [8], and human activity trajectory mining [9, 10, 11]. Classical KRL methods primarily learn the embedding vector of entities by capturing the structural features in KG using translation-based models [12], bilinear models [13], andneural network-based models [14]. On the basis of these traditional KRL methods, geographic knowledge graph representations have integrated geospatial distance constraints into traditional KRL models, enabling models to explicitly encode and represent spatial relationships within geographic data [15]. Furthermore, to capture relative positional relationships between entities, some researchers have integrated spatial attributes, such as point coordinates or entity boundaries, directly into the GeoKG embedding space, thereby achieving more precise geographic knowledge representations [16]. However, previous studies ignore the rich multimodal semantics of geographic entities. Usually, the various semantics of entities could enrich GeoKGs from different semantic views. For example, in traditional geographic knowledge graph representation methods, decisions are typically made based on structural information when performing geographic attribute prediction tasks. However, these traditional methods do not always provide accurate predictions without multimodal semantic information (e.g., text and image). As demonstrated in Figure 1, it is difficult to infer whether a cafe offers parking or delivery services based on a GeoKG without multimodal semantic information. However, the image suggests a high likelihood of parking services, while the text indicates that delivery services are unavailable. In the example, the semantic information contained in multimodal data enhances the ability to infer additional attributes of the cafe, enriching the learned representation of the geographic knowledge graph. To address this issue, this paper introduces a multimodal geographic knowledge graph representation method (MGKGR) that integrates multiple kinds of semantics to improve the embedding learning of geographic knowledge graph representation. MGKGR employs a two-stage fusion strategy that combines the structural features of geographic knowledge graphs with the semantics from multimodal information of geographic entities. Specifically, in the first stage, a spatial feature fusion for modality enhancement method is employed to strengthen image and text features using structural information. To capture common features and address distributional discrepancies between modalities, contrastive learning [17] is applied to align their distributions. In the second stage, text and image features enhanced by structural features are further integrated to form multimodal features. To address the semantic divergence between images and text, a multi-level modality feature fusion method is employed to unify these heterogeneous data. To the best of our knowledge,  
Figure 1: Multimodal data in the geographic knowledge graph provides semantic information for geographic attribute prediction.