Geo-Encoder: A Chunk-Argument Bi-Encoder Framework for C hinese Geographic Re-Ranking (2024)

Abstract: Chinese geographic re-ranking task aims to find the most relevant addresses among retrieved candidates, which is crucial for location-related services such as navigation maps. Unlike the general sentences, Chinese geographic contexts are closely intertwined with geographical concepts, from general spans (e.g., province) to specific spans (e.g., road). Given this feature, we propose an innovative framework, namely Geo-Encoder, to more effectively integrate Chinese geographical semantics into re-ranking pipelines. Our methodology begins by employing off-the-shelf tools to associate text with geographical spans, treating them as chunking units. Then, we present a multi-task learning module to simultaneously acquire an effective attention matrix that determines chunk contributions to geographic representations. Furthermore, we put forth an asynchronous update mechanism for the proposed task, aiming to guide the model to focus on specific chunks. Experiments on two Chinese benchmark datasets, show that the Geo-Encoder achieves significant improvements when compared to state-of-the-art baselines. Notably, it leads to a substantial improvement in the Hit@1 score of MGEO-BERT, increasing it by 6.22% from 62.76 to 68.98 on the GeoTES dataset.

## introduction chinese geographic re-ranking (cgr) is a sub-task of semantic matching, aiming to identify the most relevant geographic context towards given queries and retrieved candidates @xcite @xcite . it is a crucial task that serves many downstream applications such as navigation maps (e.g., gaode maps), autonomous driving (e.g., tesla), e-commerce system (e.g., taobao), etc. (jia et al., 2017; avvenuti et al., † corresponding author. 2018). unlike general query expressions, chinese geographic sentences exhibit a distinct attribute in their linear-chain structural semantics @xcite . this peculiarity arises from the fact that chinese addresses often comprise distinct meaningful address segments, termed as geographic chunks in linguistic terms @xcite . these chunks adhere to a specific format, organizing from the general (e.g., province) to the more specific (e.g., road). for example, as is shown in figure 1 , given a chinese address "采荷路2号高级中学北门 (north gate of caihe road no.2 senior high school)", we can deconstruct it into several such chunks: "采 荷路 (caihe road)", "2号 (no.2)", "高级中学 (senior high school)", "北门 (north gate)". conventional approaches @xcite @xcite addressing the cgr task often directly employ pre-trained language models (plms) to encode given geographic texts into embeddings, which are subsequently subjected to re-ranking through similarity calculation techniques like cosine or euclidean distance measures. recent works @xcite @xcite in this field extend beyond mere geographic context utilization and encompass an expansive range of data sources, including point-of-interest information, multi-modal data, and user behavioral attributes @xcite @xcite with a larger neural model. the outcome of this integration is characterized by notable enhancements, achieved through the fusion of external geographic knowledge. furthermore, cutting-edge domain-adaptation frameworks have been introduced to facilitate effective fusion of multi-domain data, such as palm @xcite , stdgat @xcite , etc. however, despite the effectiveness of existing attempts in leveraging geographic knowledge, these methods failed to fully harness the intrinsic potential of the geographic context itself. therefore, in this paper, we aim to shift our focus towards the geographic context by exploiting its distinctive linear-chain attributes. to achieve this, we employ off-the-shelf tools (e.g. mgeo tagging foot_0 and partof-speech (pos) foot_1 for the approximate annotation of each geographic text with pertinent geographic chunks. for example, as illustrated in figure 1 , we annotate the text "采荷路 (caihe road)" with the label road, "2号 (no.2)" with num, etc. firstly, building upon this foundation, we introduce an additional task that revolves learning the similarity between different components of these annotated chunks. this involves the formulation of an attention matrix, which governs the contributions of these chunks to the semantic representations. our motivation is that general chunks tend to be less diverse across queries and candidates, and specific chunks possess a higher degree of distinctiveness. secondly, we put forth a novel asynchronous update speed mechanism for the attention matrix. this mechanism is designed to empower the model to effectively focus its attention on the more specific chunks, thereby enhancing its discernment capabilities. lastly, we advocate for the integration of the pure bi-encoder approach during the inference period. this strategy ensures a har-monious balance between performance and computational efficiency, safeguarding the efficacy of the model in both academic and industrial scenarios. in summary, our key contributions are as follows: 1) we introduce a multi-task learning framework, denoted as geo-encoder, which serves as a pioneering approach to integrate component similarity; 2) we present an asynchronous update mechanism, to distinguish specific chunks effectively; 3) except evaluation on benchmark dataset, we collect and publish a nationwide geographic dataset in china, named geoind. experimental results on two distinct chinese geographic re-ranking datasets demonstrate the superiority of our geo-encoder over competitive methods. our code and datasets are available at: https://github. com/yongcaoplus/cgr_damo . chinese geographic text representation. most existing approaches focused on encoding geographic text by external knowledge in two aspects: (1) position data, such as palm @xcite , encoding positional relationship of query and candidates, stdgat @xcite , considering spatio-temporal features, etc.; ## analysis and discussion in this section, we first conduct a comprehensive analysis of our proposed modules, and then discuss the advantages of using geo chunking for cgr task. lastly, a detailed exploration of hyperparameter setting and the learned chunking attention metric is presented for a deeper understanding. ## conclusion in this paper, we proposed a novel framework called geo-encoder for chinese geographic reranking task by deploying multi-task learning module and synchronous update mechanism. the key idea behind geo-encoder is to encode geographic text using an additional component learning representations from address chunks. this approach allows the geo-encoder to effectively leverage linearchain characteristic of geographic contexts, which guides the model to capture subtle distinctions among different candidates. moreover, we present an attention matrix that enables the model to automatically learn the significance of geographic chunking components within the representation. to address the varying levels of task complexity, we introduced an asynchronous update mechanism for iterative adjusting the weight matrix of these components. this dynamic adjustment facilitates the the focus of model on specific chunks efficiently. extensive experiments demonstrated that our proposed framework leads to significant improvements over several competitive baselines. future work could be incorporating our approach in multi-modal and multi-lingual tasks.