Outdated Issue Aware Decoding for Factual Knowledge Editing (2024)

Abstract: Recently, Knowledge Editing has received increasing attention, since it could update the specific knowledge from outdated ones in pretrained models without re-training. However, as pointed out by recent studies, existing related methods tend to merely memorize the superficial word composition of the edited knowledge, rather than truly learning and absorbing it. Consequently, on the reasoning questions, we discover that existing methods struggle to utilize the edited knowledge to reason the new answer, and tend to retain outdated responses, which are generated by the original models utilizing original knowledge. Nevertheless, the outdated responses are unexpected for the correct answers to reasoning questions, which we named as the outdated issue. To alleviate this issue, in this paper, we propose a simple yet effective decoding strategy, i.e., outDated ISsue aware deCOding (DISCO), to enhance the performance of edited models on reasoning questions. Specifically, we capture the difference in the probability distribution between the original and edited models. Further, we amplify the difference of the token prediction in the edited model to alleviate the outdated issue, and thus enhance the model performance w.r.t the edited knowledge. Experimental results suggest that applying DISCO could enhance edited models to reason, e.g., on reasoning questions, DISCO outperforms the prior SOTA method by 12.99 F1 scores, and reduces the ratio of the outdated issue to 5.78% on the zsRE dataset.

## introduction large language models (llms) exhibit the remarkable ability to capture plenty of factual knowledge into their parameters. however, knowledge of inner llms may become outdated or unsuitable over time @xcite @xcite the eiffel tower is in paris. the eiffel tower is in london. france is an outdated response. @xcite . unfortunately, naively retraining llms can be computationally intensive, and fine-tuning llms in several cases suffers the risk of catastrophic forgetting @xcite @xcite . recently, knowledge editing has received increasing attention, which aims to update factual knowledge into llms and retain the unrelated knowledge, without retraining from scratch @xcite @xcite @xcite . previous methods could be divided into two research lines to implement the knowledge editing in llms, according to whether preserving original models' parameters @xcite @xcite . the first research line retains the weights of the original llms, and appends supplementary weights or memories to retrieve the relative edited knowledge to guide the model to response to the edited knowledge. for instance, serac @xcite uses a scope classifier to determine whether the current prompt falls within the scope of any stored knowledge, and t-patcher @xcite adds extra trainable parameters into ffn layers of llms ot edit model performance. besides, memprompt @xcite , ike @xcite , and mello @xcite conduct in-context learning with knowledge editing demonstrations to prompt llms to update pretrained knowledge. the second research line focuses on adjusting knowledge-related weights in the original llms via pre-computation or an additional module to predict the weights. for instance, @xcite trains a bilstm to predict the weight update, to enable constrained optimization to modify facts without affecting other knowledge, while kn @xcite , rome @xcite , and pmet @xcite implement editing by first locating the parameters, which store corresponding knowledge, and then directly updating them. overall, with these techniques, knowledge editing could update llms on new factual edited knowledge without explicit model re-training, when questions explicitly mention the edited knowledge. ## disco: outdated issue aware decoding as analyzed in the prior section ( §3.2), the few impacts of edited knowledge on probability distribution and the disturbance of original knowledge during reasoning should be responsible for the outdated issue. to amplify the impact of the edited knowledge on probability distribution, we propose a simple yet effective method, outdated issue aware decoding (disco). we first implement knowledge editing based on the prior editing methods, e.g., ike, which performs best in portability @xcite . given an input x to the original model θ and edited model θ, we capture their difference in probability distribution, caused by edited knowledge, via the subtraction as follows: then, we add the difference ∆(t) to the probability distribution of the edited model to amplify the difference in probability distribution: constraints to revise probability. to further constrain the probability of tokens of outdated response not increasing, we limit the maximum of ∆(•) for the tokens in outdated response θ(x): ## analysis 6.1 distribution of ∆(t) we investigate the probability variation ∆(t) (in eq.5) to further probe the impact of ∆(t) on the outdated issue. on the zsre dataset and llama-2-7b backbone, we split the averaged ∆(t) value in portability into intervals with the 10.0%, and then count the proportion and the oe metric w.r.t each interval. since the positive or negative ∆(t) value denotes whether the probability of golden answers is improved, as illustrated in figure 2 , we could observe that most value of ∆(t) of golden answers (around 94.5%) has a positive value. the positive value indicates that ∆(t) could assist the edited model in reasoning the golden answer. besides, with the common tokens 57.78% outdated tokens 42.22% proportion of common and outdated tokens common tokens outdated tokens larger ∆(t) value, the ratio of outdated issue of the corresponding interval has a monotone decreasing trend (from 9.65% to 0.6%). this trend further demonstrates that ∆(t) could assist the edited model in reasoning the golden answer, and mitigate the outdated issue. ## conclusion in this paper, we focus on the outdated issue that edited models suffer from generating outdated responses to the reasoning questions, which is ignored by previous approaches. to mitigate this issue, we propose a simple yet effective method, outdated issue aware decoding (disco), to encourage the edited model to utilize the edited knowledge to reason the correct answers for reasoning questions. specifically, disco captures and amplifies the modification in probability distribution between the original and edited models. experimental results demonstrate that disco could significantly mitigate the outdated issue, and effectively encourage the edited to reason the new correct answers for reasoning questions, without updating parameters.