# Evaluating Complex Entity Knowledge Propagation for Knowledge Editing in LLMs
###### Abstract  
In today's world, where information keeps growing rapidly and changing constantly, language models play a crucial role in making our lives easier across different fields. However, it is tough to keep these models updated with all the new data while making sure they stay accurate and relevant. To tackle this challenge, our study proposes an innovative approach to facilitate the propagation of complex entity knowledge within language models through extensive triplet representation. Using a specially curated dataset (CTR-KE) derived from reliable sources like Wikipedia and Wikidata, the research assesses the efficacy of editing methods in handling intricate relationships between entities across multiple tiers of information. By employing a comprehensive triplet representation strategy, the study aims to enrich contextual understanding while mitigating the risks associated with distorting or forgetting critical information. The study evaluates its proposed methodology using various evaluation metrics and four distinct editing methods across three diverse language models (GPT-XL, GPT-J, and Llama-2-7b). The results indicate the superiority of mass-editing memory in a transformer (MEMIT) and in-context learning for knowledge editing (IKE) in efficiently executing multiple updates within the triplet representation framework. This research signifies a promising pathway for deeper exploration of data representation for knowledge editing within large language models, and improved understanding of contexts to facilitate continual learning. large language models (LLMs); entity knowledge propagation (EKP); comprehensive knowledge representation; knowledge graph; knowledge editing +
Footnote †: journal: _Sci_. 0000-0001-5000-0000-0000-0000-0000-0000-0000-0000-0000-0000-0000-0000-0000-00000-0000-0000-00000-00000-00000-00000-00000-00000-00  
updates are computationally expensive for LLMs; therefore, some studies have introduced alternative approaches, such as in-context learning (ICL)-based knowledge editing (IKE) which may not require any heavy parametric updates to LLMs [17, 18]. Though IKE proves to be an effective and powerful method for LLMs to learn more context without requiring heavy parametric changes, it has limitations on how much additional information about an entity can be included. It also restricts KE methods, leading to questions about the extent to which entity-related knowledge can be modified. Therefore, to mitigate the limitations of both traditional editing models and IKE, comprehensive knowledge representation in the form of knowledge graphs (KGs) may be effective [19, 20, 21, 22, 23]. Most studies have utilized KG embeddings because of their efficient and less resource-extensive nature; however, when it comes to continual learning in LLMs, there are some limitations to it. To mitigate the static nature of KG embeddings, researchers are exploring hybrid models that combine the strengths of KGs with LLMs. These approaches aim to use the contextual understanding and generative capabilities of LLMs while leveraging the structured knowledge and reasoning capabilities of KGs. Therefore, in this paper, our focus is on knowledge augmentation through KGs, which aim to enhance the context understanding and reduce inaccuracies in LLMs by providing a detailed representation of knowledge via entity knowledge graphs. We evaluate the performance of various editing models on this approach to understand how effectively they work with the enriched knowledge base. One common approach for knowledge representation opted for by various KE methods is to use the subject-relation-object \((h,r,t)\) format, where the data is organized into triplets consisting of a subject (h), a relation (r), and an object (t). This triplet represents a fact, and KE methods inject this fact into a pre-trained LLM by updating the 't' in the triplet for a given h and r. For example, \"The president of the US is [PERSON]\", here (h,r) = \"the president of the US is\" and t = [PERSON].