Advancing Cross-Lingual Entity Alignment with Large Language Models: Tailored Sample Segmentation and Zero-Shot Prompts (2024)

Abstract: In recent years, the advent of large language models (LLMs) like GPT and Llama has significantly influenced numerous domains, particularly in advancing natural language processing (NLP) capabilities. LLMs have shown remarkable performance in NLP tasks such as relation extraction (RE) and knowledge graph completion (KGC), enhancing activities related to knowledge graphs. As a result, there is a growing interest in integrating LLMs into cross-lingual entity alignment (EA) task, which aims to identify equivalent entities across various knowledge graphs, thereby improving the performance of current baselines. However, employing LLMs for entity alignment poses challenges in efficiently handling large-scale data, generating suitable data samples, and adapting prompts for the EA task. To tackle these challenges, we propose Seg-Align, an innovative framework that integrating distance feature extraction, sample **Seg**mentation, and zero-shot prompts. Through extensive experiments on two widely used cross-lingual benchmark datasets, we have not only demonstrated the effectiveness of our proposed sample segmentation algorithm but also highlighted the state-of-the-art performance of Seg-Align. Code is available at https://github.com/yangxiaoxiaoly/Seg-Align.

## introduction knowledge graphs (kgs) depict entities and their relationships, serving as foundational elements for applications like semantic search @xcite , question-answering @xcite , and recommender systems @xcite . however, kgs often suffer from heterogeneity and redundancy due to their construction by various organizations with specific needs. knowledge fusion @xcite aims to align and merge this heterogeneous information, forming * corresponding author unified identifiers and relationships. entity alignment (ea) is crucial in this process, focusing on discovering equivalent entities across various kgs @xcite . knowledge representation learning-based entity alignment methods have emerged as the primary technique for addressing the ea task, yielding promising results. these slm based methods foot_0 often use translation-based models or graph neural networks (gnns)/ graph convolutional networks (gcns) due to their robustness and generalization capabilities @xcite . recently, llms have demonstrated their proficiency in various nlp tasks @xcite . trained on vast amounts of text data, llms possess rich linguistic and background knowledge, enabling them to understand context, disambiguate meanings, and recognize patterns across textual sources @xcite . this capability renders their application to the ea task particularly promising. nonetheless, integrating llms into the ea task also faces challenges. challenge 1: how to handle large-scale data. managing large-scale data requires meticulous consideration of resource consumption, operational efficiency, and overall system performance. llms, such as gpt and llama, have input size limitations. consequently, it is impractical to process all data solely using these models. additionally, as the length of the input increases, the cost of using llms also rises, accompanied by longer processing times. this results in higher resource consumption and reduced efficiency. therefore, when dealing with datasets for the ea task, it is crucial to adopt strategies that balance resource consumption, efficiency, and performance. challenge 2: how to select data samples that are more suitable for processing by llms. llms are trained on massive datasets containing billions or even trillions of words sourced from various texts on the internet @xcite . these datasets cover a wide range of topics, styles, and languages, allowing the model to learn various language patterns and contexts. therefore, for data samples that are difficult for slms to distinguish, such as long-tail entities @xcite , llms can leverage their own knowledge for judgment. for data samples effectively managed by slms, employing llms is unnecessary, as llms may not provide superior performance in these cases. hence, it is essential to carefully consider the complexity of entities, context, and the performance of slms to determine which entity samples require handling by llms and how to effectively utilize llms to enhance the effectiveness of the ea task. challenge 3: how to adjust prompts to make them more suitable for the ea task. for different tasks, llms require distinct contextual information and input formats. in the ea task, firstly, due to the limitation of input length, it's impractical to include all entities directly in the prompt. additionally, transforming entities from a kg into suitable textual representations for inclusion in the prompt is essential. furthermore, llms need to identify and match same entities across various kgs. when adjusting prompts, it is necessary to consider the characteristics and requirements of the ea task and ensure that prompts can effectively guide llms to understand and execute the ea task. to address the aforementioned challenges, we propose seg-align framework, which mainly consists of three components: distance feature extraction, sample segmentation, and zero-shot prompts. firstly, a slm (sdea) @xcite is utilized to obtain the initial embedding representations of entities. then, based on these embeddings, the distances between entities are computed to generate a distance matrix. subsequently, machine learning method is employed for distance feature extraction. using the extracted distance features, we perform binary classification to divide the data samples into two groups, which are then processed by a slm and a llm respectively. the llm processes the corresponding data samples based on zero-shot prompts to obtain the final alignment results. the experimental results indicate that our seg-align framework has achieved remarkable performance enhancements in the ea task. in summary, our contributions are as follows: ## related work currently, most ea methods are rooted in knowledge representation learning, primarily categorized into those translation based methods and those based on gnns/gcns. translation based methods, such as mtranse @xcite , jape @xcite , kecg @xcite , bootea @xcite , multi-mapping relations @xcite , transedge @xcite , jarka @xcite , and ctea @xcite , principally constrain the entity embeddings into a fixed distribution by translation-based knowledge graphs embedding methods. based on the observation that entities sharing similar neighboring structures tend to be aligned, ea approaches based on gcns distribute and consolidate entity information across graphs. gcn-align @xcite is the first to use gcn to jointly embed the entity structure and entity attributes. building upon this foundation, many approaches have enhanced gcns to address issues such as noise propagation (hgcn @xcite ), heterogeneity (mugnn @xcite , alinet @xcite , nmn @xcite , mraea @xcite ), and better utilization of relationship and attribute information (rdgcn @xcite , raga @xcite , rnm @xcite , epea @xcite ). with the rise of pre-trained language models like bert (kenton and toutanova, 2019), fine-tuning these models in downstream tasks has demonstrated significant potential. hman+bert @xcite , sdea @xcite , and bert-int @xcite treat entity alignment as a downstream task for fine-tuning bert. however, seed entity pairs slm encoder entity embeddings distance matrix of entity pairs machine learning distance feature extraction sample segmentation kg2 kg1 llm final results ... choose the option that is most similar to entity {magyars} from the following options: a:hungarians, b:murad_i, c:north_america, d:me_(super_junior-m_album). choose the option that is most similar to entity {magyars} from the following options: a:hungarians, b:murad_i, c:north_america, d:me_(super_junior-m_album). ## problem definition definition 1 (knowledge graph) a knowledge graph (kg) is denoted as @xmath0, r, a, v, t r , t a ), where @xmath1, e 2 , ...e m }, @xmath2, r 2 , ...r n }, @xmath3, a 2 , ...a p }, and @xmath4, v 2 , ..., v q } represent entity set, relation set, attribute set, and value set, respectively, and m, n, p, q are the number of entities, relations, attributes, and attribute values, respectively. relational triples can also be represented as (h, r, t), where h is called the head entity and t is called the tail entity. ## methodology as shown in figure 1 , seg-align framework is mainly divided into three parts: distance feature extraction, sample segmentation, and zero-shot prompt. firstly, in the distance feature extraction stage, we train a slm to obtain entity embeddings, thereby calculating the distances between entities to generate a distance matrix. based on the distance matrix, we extract distance features, and then perform sample segmentation to select data samples more suitable for processing by llms. finally, we design prompts to utilize the background knowl-edge of llms for ea. for the candidate entity selection, our approach aligns with chatea and llmea in utilizing knowledge representation learning-based entity alignment methods to obtain entity embeddings, which are then used to select candidate entities. however, our method diverges in how we handle candidate entities. while chatea and llmea pass all data to the llm for processing after candidate entities are identified, they do not account for the fact that some data may already be well-aligned during the candidate entity selection phase. therefore, we propose a sample segmentation algorithm that selects only the poorly aligned data to be processed by the llm. in terms of prompt design, our approach differs significantly from that of chatea and llmea. firstly, chatea's prompt processes each candidate entity one by one sequentially, whereas our prompt can include all candidate entities at once, greatly enhancing the llm's processing efficiency. similar to llmea, we use a multiple-choice format for our prompt; however, we further refine this by restricting the response format of the llm to ensure more consistent and easier-to-process answers. additionally, unlike llmea, we do not include examples in the prompt, thereby achieving a true zero-shot prompt design. ## conclusion in this paper, we focus on leveraging llms to improve the performance of cross-lingual entity alignment. to better apply llms to the ea task, our seg-align framework extends slms by introducing distance feature extraction, sample segmentation algorithm, and designing prompts tailored for the ea task. through experiments on two widelyused cross-lingual datasets, we empirically show that our sample segmentation algorithm effectively identifies data for llm or slm processing, validating the framework's effectiveness.