Time Sensitive Knowledge Editing through Efficient Finetuning (2024)

Abstract: Large Language Models (LLMs) have demonstrated impressive capability in different tasks and are bringing transformative changes to many domains. However, keeping the knowledge in LLMs up-to-date remains a challenge once pretraining is complete. It is thus essential to design effective methods to both update obsolete knowledge and induce new knowledge into LLMs. Existing locate-and-edit knowledge editing (KE) method suffers from two limitations. First, the post-edit LLMs by such methods generally have poor capability in answering complex queries that require multi-hop reasoning. Second, the long run-time of such locate-and-edit methods to perform knowledge edits make it infeasible for large scale KE in practice. In this paper, we explore Parameter-Efficient Fine-Tuning (PEFT) techniques as an alternative for KE. We curate a more comprehensive temporal KE dataset with both knowledge update and knowledge injection examples for KE performance benchmarking. We further probe the effect of fine-tuning on a range of layers in an LLM for the multi-hop QA task. We find that PEFT performs better than locate-and-edit techniques for time-sensitive knowledge edits.

## introduction the rapid development of large language models (llms) has showcased their ability to generate human-quality responses and demonstrate reasoning capabilities @xcite @xcite @xcite , and it is bringing revolutionary changes across diverse industries. however, maintaining the factuality remains challenging for llms since their pre-training data are collected within a time range. modification (s, r, @xmath0, r, ∅ → o ′ ) are two main ways to update factual knowledge in llms, where s, r, o denotes subject, relation, and object in an old fact triple, o ′ denotes the new target object, and ∅ denotes an empty object to be populated. previously, very few works @xcite evaluate the effectiveness of knowledge editing (ke) techniques on time-sensitive fact changes. we believe that keeping time-sensitive information current is crucial for maintaining the practical relevance of an llm's knowledge in the real-world applications. therefore, in this paper, we focus our investigation on temporal ke. one popular approach for ke is locate-and-edit which involves identifying and directly updating model parameters associated with specific knowledge. rome @xcite and memit @xcite are two representative works in this area. there are several known limitations of rome/memit. first, they require estimation of a large covariance matrix, which might lead to numerical stability issues during computation @xcite . second, for every small batch of knowledge edits, they need to locate the layer for weight optimization, which can be time consuming and difficult to scale @xcite . third, @xcite demonstrated that although the llm can successfully recall the edited fact after rome/memit editing, the post-edit model performs poorly for multi-hop questions. hence, we would like to verify if peft approaches can be more efficient than the locate-and-edit approach in the ke task and perform better in recalling the knowledge edits as well as retaining the unchanged knowledge. in addition, we believe it is worthwhile to investigate the effect of fine-tuning the weights of linear layers in transformers at different locations within the llm (early, middle, and last) on the multi-hop question answering task. the main contributions of this paper can be summarized as follows: parameter-efficient fine-tuning. lora (hu et al., 2021) is a simple yet effective adaptation technique that adds low-rank tunable weight matrices to the original weight matrices, which are kept frozen. this technique significantly reduces the trainable parameters during fine-tuning, while keeping the inference run-time constant. instead, p-tuning @xcite concatenates learnable tensors with the input embedding to enable the base language model to perform well on a range of downstream tasks such as knowledge probing and natural language understanding. in this paper, we would like to verify if these peft methods can effectively modify or inject new knowledge in llms. ## method we mainly fine-tune the base llms including llama-7b, falcon-7b, and mistral-7b with the peft approach including lora and p-tuning and minimize the following loss function: lora. lora uses low-rank matrices @xmath1, k). the low rank matrices a and b are trainable parameters: lora adaptation can be applied to any linear layer. in our experiments, we apply lora to linear layers in both the mlp layers (w gate , w up , w down ) and self-attention layers (w q , w k , w v , w o ). the benefit of lora is that the inference runtime remains the same, whereas in adaptors and other methods such as rome/memit, the inference runtime increases since they add additional layers. p-tuning. p-tuning learns continuous prompt embeddings and concatenates them with the original input embedding. in this work, we leverage these tunable embeddings to adjust the output distributions of the predicted tokens during inference. formally, let [p i ] be the i th continuous prompt embedding, and let @xmath2, . . . , x n } denotes the original input sequence to the llm. then, the new input sequence would be @xmath3:i ], x}. p-tuning also uses an additional encoder to map the continuous prompt embeddings to latent parameters f : [p i ] → h i . in our implementation, we experiment with both a 2-layer mlp and an lstm as the mapping function f . let e be the pretrained embedding layer, then the final vector input to the llm is {h 0 , . . . , h i , e(x)}. freeze tuning. instead of fine-tuning all weight parameters in an llm, only several layers are finetuned to save the number of parameters that need to be placed on gpus for gradient computation. in our experiments, we focus on fine-tuning mlp layers in the transformer modules. we generate question answering pairs as training examples that is used to induce new facts in the llm. to evaluate reliability, we generate a corresponding cloze to test whether the post-edit model can successfully complete the sentence with the new fact. to evaluate generalization, we generate paraphrased question answer pairs from the training examples with the help of openai textdavinci-003 api. to assess locality, we follow @xcite to use a subset of lama @xcite called invariantlama, which contains time-invariant statements. we report the ratio of exact match (em) for reliability and generalization and the rouge-1 score for locality. ## conclusion in this paper, we have systematically examined the feasibility of performing ke through peft. we have compared the performance of fine-tuning methods including lora, p-tuning and freeze tuning with locate-and-edit approaches for ke. our results demonstrate that fine-tuning can successfully update time-sensitive factual knowledge in llms both efficiently and effectively, and without compromising the llms' capability in answering invariant knowledge and multi-hop reasoning. we have also contributed a large scale ke dataset chronoedit that contains both modification edit and injection edit examples.