I nfuser KI : Enhancing Large Language Models with Knowledge Graphs via Infuser-Guided Knowledge Integration (2024)

Abstract: Large Language Models (LLMs) have achieved exceptional capabilities in open generation across various domains, yet they encounter difficulties with tasks that require intensive knowledge. To address these challenges, methods for integrating knowledge have been developed, which augment LLMs with domain-specific knowledge graphs through external modules. These approaches, however, face data inefficiency issues as they necessitate the processing of both known and unknown knowledge for fine-tuning. Thus, our research focuses on a novel problem: efficiently integrating unknown knowledge into LLMs without unnecessary overlap of known knowledge. A risk of introducing new knowledge is the potential forgetting of existing knowledge. To mitigate this risk, we propose the innovative InfuserKI framework. This framework employs transformer internal states to determine when to enrich LLM outputs with additional information, effectively preventing knowledge forgetting. Performance evaluations using the UMLS-2.5k and MetaQA domain knowledge graphs reveal that InfuserKI not only successfully integrates new knowledge but also outperforms state-of-the-art baselines, reducing knowledge forgetting by 9% and 6%, respectively.

## introdution large language models (llms) have significantly advanced the capabilities of various language tasks, including question answering (qa), dialogue, and information retrieval, showcasing impressive performance across different fields @xcite . however, in knowledge-intensive tasks like open-domain qa, llms can produce texts that are misleading or inaccurate due to a lack of domain knowledge and the phenomenon of catastrophic forgetting post-fine-tuning @xcite @xcite . the step of updating and customizing llms with domain knowledge integration is thus highly valued for enhancing their application. this could involve companies customizing models with specialized product knowledge, or hospitals adapting models to reflect specific case data. knowledge graphs (kgs) are ideal sources for bolstering domain-specific knowledge, thanks to their structured and measurable knowledge units. various strategies have been devised to utilize this knowledge effectively. typically, these strategies encompass instruction tuning of llms using explanations of knowledge entities @xcite , developing triplet-based pre-training tasks @xcite @xcite , using kgs as external sources in retrieval tasks @xcite , and applying parameter-efficient fine-tuning (peft) techniques such as lora @xcite and adapters @xcite , or model editing (me) methods like t-patcher @xcite to implement knowledge in a triplet-to-text format @xcite @xcite . however, pre-training or fine-tuning llms with the entire kgs is not only time-consuming but also leads to data inefficiencies, especially when models relearn knowledge they already have. to address this issue, we focus on integrating new, previously unknown knowledge only. this precise focus, however, introduces the risk of catastrophic forgetting, where the addition of new knowledge may affect existing knowledge. fig. 1 illustrates a comparison between a standard llm and its fine-tuned variant by visualizing the internal states of the 10th transformer layer from the training data using the tsne tool, where each umls knowledge unit sample is processed to obtain these states and then mapped to two dimensions for display. fig. 1 (a ) and (b) demonstrate how direct fine-tuning can lead to the loss of previously known data, while fig. 1 (c) illustrates the ideal integration of new knowledge without compromising existing information. thus, we pose a novel research question: how can we efficiently integrate new knowledge from domain-specific kgs into llms while preventing catastrophic forgetting? in this work, we introduce the infuser-guided knowledge integration (infuserki) framework, meticulously designed to integrate domain-specific knowledge from kgs into llms. drawing inspiration from @xcite , which reveals that an llm's internal states can reflect the truthfulness of its generated texts, our framework incorporates an infusing mechanism that verifies the presence of current knowledge in llms. this mechanism facilitates the adaptive selection of additional information for both known and unknown knowledge, effectively minimizing the impact on existing knowledge and preventing knowledge forgetting. additionally, infuserki employs knowledge adapters to embed new knowledge while maintaining the integrity of the original model parameters. the process within the infuserki framework initiates by identifying knowledge that llms do not yet know. following methodologies from @xcite and @xcite , we craft a knowledge statement and multiple-choice questions for a knowledge @xmath0, r, @xmath1, as illustrated in fig. 3 . furthermore, to broaden the generality of the integrated knowledge, infuserki implements a relation classification task. this task is designed to refine the linguistic representations developed by the adapters, enabling the prediction of relationships within knowledge statements based on the adapter outputs for head and tail entities. this approach not only ensures a solid integration of new knowledge but also bolsters the framework's ability to generalize this knowledge to unseen scenarios. our main contributions are: • we explore a novel problem: effectively integrating unknown knowledge from kgs into llms without impacting existing knowledge. model editing model editing (me) for llms falls into two categories: gradient-based and extension-based. gradient-based methods, as described by @xcite , modify specific weights related to knowledge edits. rome @xcite and memit @xcite take this further by updating entire feedforward network (ffn) layers to enhance model editing. these methods, however, are limited in the number of edits or may require considerable time for execution. on the other hand, extension-based methods add new parameters to correct inaccurate information. calinet @xcite and t-patcher @xcite incorporate memory slots or trainable "patches" into final ffn outputs. grace @xcite employs a keyvalue adapter with a deferral mechanism for the selective use of knowledge based on input. however, the adapter-based modules positioned in top transformer layers are designed to calibrate false facts. instead, our method aims to infuse new knowledge by placing adapters throughout transformer layers. catastrophic forgetting catastrophic forgetting occurs when learning new information causes a drastic loss of previously learned knowledge @xcite ). this phenomenon is particularly evident in sequential inter-task learning, where acquiring new task knowledge can lead to forgetting older task knowledge @xcite . to address this, various strategies have been developed. @xcite applied constraint to minimize parameter changes during new task learning. elastic weight consolidation (ewc) incorporates the hessian matrix into parameter regularization to reduce forgetting @xcite . replay-based methods, including sampling strategies that retain original training samples in a memory buffer @xcite . knowledge distillation aligns the predictions of a fine-tuned model with the pre-fine-tuning model @xcite . parameter-efficient fine-tuning can also mitigate forgetting, represented by lora @xcite , which uses low-rank matrices for weight modifications while maintaining pre-trained parameters frozen, and achieves results akin to full fine-tuning. however, these studies emphasize sequential inter-task transfer learning. our focus shifts to intra-task knowledge forgetting, where integrating new knowledge leads to the potential loss of previously existing knowledge. ## proposed framework -infuserki the objective of our method is to leverage domain knowledge from kgs to enhance llms for knowledge-intensive tasks. specifically, given an llm p θ ∈ p and a set of knowledge triplets @xmath2, our goal is to fine-tune the llm p θ into p ′ θ , incorporating previously unknown knowledge t unk without affecting existing knowledge t known . for efficiency, we only inject knowledge that is unknown to the llm as: the core design of our infuserki framework comprises two steps: knowledge detection and knowledge integration, as illustrated in fig. 3 . to be specific, we first detect previously unknown knowledge by feeding questions derived from knowledge triplets to the llms. upon identifying a set of unknown knowledge, we employ the knowledge adapter, which is parallel to the original transformer layer and trained to store new knowledge. the core of our framework, the knowledge infuser, is designed to strategically determine whether new knowledge from the knowledge adapter should be engaged. throughout this process, we only finetune the knowledge adapter and the infuser while keeping the original transformer parameters fixed. ## experiments in this section, we evaluate the proposed framework by conducting experiments on two knowledge graphs across different data scales, comparing against peft and me baselines. ## limitations we note that the effectiveness of our method is contingent upon the base language model's ability to follow instructions accurately. in scenarios where the underlying model exhibits suboptimal instruction-following capabilities, the integration of knowledge, regardless of its quality, may not significantly improve performance on downstream tasks. consequently, applying our knowledge integration framework to models with limited instruction-following proficiency presents a considerable challenge. below is an instruction that describes a task. write a response that appropriately completes the request. ### instruction: {instruction} ### response: convert triplets into mcqs q, with correct answers y and knowledge statements k, using relational templates. tune infuser on a balanced mix of known and unknown samples as per eq. 5. fine-tune adapters for templates #1 and #2 using qa loss in eq. 8. apply relation classification to unknown statements, following eq. 9 and eq. 10. stracts, featuring yes/no/maybe questions alongside context, as highlighted in @xcite . metaqa @xcite serves as a multihop kgqa benchmark in the movie domain, presenting a knowledge graph with 135,000 triplets, 43,000 entities, and 9 relations. it organizes over 400,000 questions into 1-hop, 2-hop, and 3-hop categories, each annotated with head entities, answers, and reasoning paths. our analysis concentrates on the 1-hop version for downstream testing.