# Evaluating Complex Entity Knowledge Propagation for Knowledge Editing in LLMs
###### Abstract
To update this fact, KE methods will replace 't' with the new fact i.e., \"The president of the US is [PERSON]\". As shown in Figure 1, we focus on a rather comprehensive representation of a triplet to evaluate the KE methods beyond a single fact edit. A comprehensive triplet can be considered as a chain or set of multiple triplets for a certain entity referred to as the head entity (h). We provide a formal definition of the comprehensive representation of knowledge in Section 3. Figure 1: Retrieving entities and text descriptions, creating KGs to form comprehensive triplets to serve as inputs into the LLMs, and evaluating the performance of KE methods. This study delves into the frontier of KE with the fusion of LLMs and dense KGs extracted from Wikipedia. We present a comparative study, shedding light on the performance and capabilities of different editing models when introduced to the rich, structured knowledge encoded within Wikipedia. We explore the potential of dense KGs and comprehensive triplets as a contextual backbone for large language editing models. We aim to answer the following research questions (RQ):  
RQ1: How does the inclusion of a KG-based comprehensive triplet influence the factual accuracy of edited text? RQ2: Can these KG-augmented models exhibit better coherence and context preservation in their edits? RQ3: How to evaluate the existing KE methods for such complex entity triplet representations and figure out how accurately these methods propagate knowledge in dense triplets? To conduct our comparative study, we leverage various evaluation metrics to evaluate the performance of different editing models [12; 24]. Through meticulous experimentation and analysis, we aim to provide insights that can guide the development and utilization of enhanced text editing tools. The motivation behind this idea is twofold. Contextual enrichment: we seek to provide LLMs with a rich contextual backdrop--a source of structured information that can be used to validate facts, maintain context, and enhance the coherence of edited text. Factual accuracy and trustworthiness: integrating Wikipedia KGs to allow for real-time fact-checking during the editing process. This empowers LLMs to cross-reference and verify information against a trusted and extensive knowledge source. This not only mitigates the propagation of misinformation but also instils a greater sense of trust and reliability in the generated or edited content. To conclude, we make the following contributions: (a) highlight key limitations of KE evaluation, specifically regarding complex triplets; (b) construct a dataset (CTR-KE) containing a comprehensive triplet representation of knowledge for complex entity knowledge propagation; (c) evaluate current methods for KE and compare them with different evaluation criteria; and (d) create an evaluation dataset based on multi-hop questions covering different triplets from the entity KG. The remainder of this paper is structured as follows: In Section 2, we provide an overview of related studies, highlighting the evolution of language models, editing tasks and models and their integration with external knowledge sources. Section 3, presents our methodology, detailing the problem setting. This is followed by Section 4, which focuses on data collection and the generation process. Section 5 deals with the evaluation criteria for our approach. Section 6 presents our experimental results and analyses, followed by a discussion on our key findings. Finally, in Section 7, we conclude with the implications, future directions, and broader significance of our research in the context of text editing and natural language understanding.