Lifelong Knowledge Editing for LLM s with Retrieval-Augmented Continuous Prompt Learning (2024)

Abstract: Model editing aims to correct outdated or erroneous knowledge in large language models (LLMs) without the need for costly retraining. Lifelong model editing is the most challenging task that caters to the continuous editing requirements of LLMs. Prior works primarily focus on single or batch editing; nevertheless, these methods fall short in lifelong editing scenarios due to catastrophic knowledge forgetting and the degradation of model performance. Although retrieval-based methods alleviate these issues, they are impeded by slow and cumbersome processes of integrating the retrieved knowledge into the model. In this work, we introduce RECIPE, a RetriEval-augmented ContInuous Prompt lEarning method, to boost editing efficacy and inference efficiency in lifelong learning. RECIPE first converts knowledge statements into short and informative continuous prompts, prefixed to the LLM’s input query embedding, to efficiently refine the response grounded on the knowledge. It further integrates the Knowledge Sentinel (KS) that acts as an intermediary to calculate a dynamic threshold, determining whether the retrieval repository contains relevant knowledge. Our retriever and prompt encoder are jointly trained to achieve editing properties, i.e., reliability, generality, and locality. In our experiments, RECIPE is assessed extensively across multiple LLMs and editing datasets, where it achieves superior editing performance. RECIPE also demonstrates its capability to maintain the overall performance of LLMs alongside showcasing fast editing and inference speed.

## introduction large language models (llms) (touvron et al., 2023; roumeliotis and tselikas, 2023; zeng et al., * q. chen and t. zhang contributed equally to this work. previous efforts in model editing have primarily focused on single and batch edits. notable examples include rome @xcite , mend @xcite , and memit @xcite , which achieve edits by applying offsets to part of the model's parameters. however, in the real world, llms frequently require continuous knowledge updates to stay abreast of emerging knowledge. thus, the concept of lifelong editing has been introduced @xcite . as shown in the upper part of figure 1 , with continuous editing, the accumulating offsets on parameters can result in model performance degradation or even failure @xcite @xcite . some techniques @xcite address the challenges by integrating extra model parameters. nevertheless, as shown in the middle of figure 1 , the increase in additional parameters leads to diminished model performance and reduced inference efficiency. retrieval-based methods separate knowledge from the model. as shown in the lower part of figure 1, they temporarily integrate retrieved knowledge into the model during each inference, mitigating knowledge forgetting and performance degradation. however, these methods face certain limitations: retrieval at each token prediction @xcite , caching large update matrices @xcite , and overly long editing prefixes @xcite . these shortcomings affect the model's inference efficiency. in this paper, we introduce recipe, a novel retrieval-augmented continuous prompt learning framework to enhance editing efficacy and inference efficiency for llms in lifelong learning scenarios. two key techniques of recipe are introduced as follows: knowledgeable continuous prompt learning: in retrieval-based methods, lte @xcite effectively avoids the shortcomings of earlier approaches. however, their overly long editing prefixes can reduce model inference speed, and fullparameter fine-tuning also increases the risk of overfitting. therefore, we aim to explore the shortest possible prefixes to enable the llm to follow editing instructions. in recipe, we achieve this by constructing a continuous short prompt encoder that transforms each piece of editing knowledge (expressed as text) into a knowledgeable continuous prompt. this approach is grounded in prior research, as exemplified by p-tuning @xcite , which demonstrated that continuous prompts enable llms to perform downstream tasks more effectively. here, we conceptualize each knowledge edit as a distinct mini-task. to ensure editing efficacy, our prompt encoder is trained to align with three key editing properties including reliability, generality, and locality @xcite . ## background in this section, we first formally present the model editing task and its lifelong version. then, we introduce the evaluation properties in model editing. an llm f @xmath0: @xmath1. given an edit example pair (q e , a e ) that f llm (q e ) ̸ = a e , a model editor me : f × q × @xmath2: generality requires f t llm to correctly answer queries belonging to relevant neighbors of previously edited samples: locality requires f t llm to maintain consistency with the initial model f 0 llm on queries unrelated to previously edited samples: ## the proposed approach in this section, we formally introduce the recipe framework, with the overall architecture in figure 2 . first, recipe maintains a knowledge retrieval repository, which stores representations of editing knowledge mapped to their knowledgeable continuous prompts described in sec. 4.1. next, we introduce a dynamic retrieval technique with the ks to facilitate knowledge retrieval to filter out irrelevant knowledge in sec. 4.2. to ensure the llms adhere to the edited knowledge related to the query efficiently, recipe prefixes the retrieved continuous prompt to the word embeddings of the llm's input query, as detailed in sec. 4.3. finally, we describe the joint training procedure of the recipe framework in sec. 4.4. the algorithms for recipe are detailed in appendix a. ## experiments in this section, we present the experimental results on llms including llama-2 (7b), gpt-j (6b), and gpt-xl (1.5b). we compare recipe against strong baselines including fine-tune (ft) @xcite , mend @xcite , rome @xcite , memit @xcite , malmen @xcite , wilke @xcite , t-patcher (tp) @xcite , grace @xcite , r-rome @xcite , and lte @xcite . for the evaluation of lifelong editing, we use datasets including zsre @xcite , cf @xcite , and ripe @xcite . for the evaluation of general performance after editing, we apply datasets including csqa @xcite , anli @xcite , mmlu @xcite , and squad-2 @xcite . for more detailed description of datasets, baselines, and model settings, please refer to appendix b and appendix c. ## conclusion we propose recipe, an effective and efficient llm editor that includes two essential modules. continuous prompt learning prefixes transformed knowledge to input query to achieve efficient postretrieval editing. dynamic prompt retrieval with ks retrieves and determines whether the repository contains relevant knowledge without fixed similarity thresholds. in lifelong editing, recipe demonstrates exceptional editing performance and efficiency while simultaneously preserving llm functionality without degradation.