Can We Edit Factual Knowledge by In-Context Learning? (2023)

Abstract: Previous studies have shown that large language models (LLMs) like GPTs store massive factual knowledge in their parameters. However, the stored knowledge could be false or outdated. Traditional knowledge editing methods refine LLMs via fine-tuning on texts containing specific knowledge. However, with the increasing scales of LLMs, these gradient-based approaches bring large computation costs. The trend of model-as-a-service also makes it impossible to modify knowledge in black-box LMs. Inspired by in-context learning (ICL), a new paradigm based on demonstration contexts without parameter updating, we explore whether ICL can edit factual knowledge. To answer this question, we give a comprehensive empirical study of ICL strategies. Experiments show that in-context knowledge editing (IKE), without any gradient and parameter updating, achieves a competitive success rate compared to gradient-based methods on GPT-J (6B) but with much fewer side effects, including less over-editing on similar but unrelated facts and less knowledge forgetting on previously stored knowledge. We also apply the method to larger LMs with tens or hundreds of parameters like OPT-175B, which shows the scalability of our method. The code is available at https://github.com/pkunlp-icler/IKE .

## introduction pre-trained language models (lms) have set a new paradigm for nlp research and sweep across all existing nlp benchmarks. due to their promising results, researchers have empowered lms with new skills that meet real-world needs, such as using web browsers @xcite , coding @xcite , playing strategic game @xcite , and conversational talents @xcite . however, the wide application of lms also raises growing concerns regarding its pitfall of generating content that is fake @xcite et al., 2021a), out-dated @xcite , biased @xcite , and offensive @xcite . to mitigate this pitfall, knowledge editing (fig. 1 ) aiming to modify the knowledge learned of lms has attracted increasing attention @xcite . the goal of knowledge editing is two-fold: generalization and specificity. the former requires generalizing to various prompts describing the same knowledge and the latter requires no interference with other unrelated knowledge. previous knowledge editing methods mainly adopt gradient-based methods to modify specific model parameters for a desired model behavior @xcite , e.g., updating the president after the election. however, the identification of the target knowledge neurons usually requires gradient estimation with heavy computation overhead @xcite . in addition, the updated parameters inherently lead to side effects beyond the desired editions, such as forgetting previously-learned facts or over-editing on unrelated facts. previous studies have shown that when a large-scale lm (llm) is deployed as a black-box service @xcite , a minor modification to its parameters could dramatically influence its behavior for end users. therefore, traditional methods still suffer from editing llms since these limitations impede the scalability and generalizability. recently, in-context learning (icl) @xcite has emerged as a new paradigm for instructing llms to perform complex tasks. in icl, the task description and demonstration examples are represented in natural language to form a context, and the prediction of lms conditioned on the context is transformed into answers according to predefined rules @xcite . in this way, large lms adapt to various downstream tasks without any modifications to parameters, making it a natural fit for knowledge editing on large lms. first, it reduces the computation overhead by avoiding modifications to parameters, as well as eliminates the risk of side effects introduced by parameter updates. most importantly, icl provides an interpretable way for humans to calibrate lm behaviors. despite these advantages, whether icl is applicable to knowledge editing still remains unclear. in this paper, we investigate the potential of icl to perform knowledge editing for llms. we focus on two goals: (1) ensuring generalization, so that large lms can generalize to various text surfaces for a piece of updated knowledge, and (2) ensuring specificity, by making accurate modifications to the target knowledge fact while preserving other irrelevant facts. to achieve these goals simultaneously, we design demonstration formatting and organization strategies to construct suitable incontext learning demonstrations for guiding knowledge editing on llms. we define three types of demonstration formatting templates including (i) copy, which aims to inject new facts into lms; (ii) update, which improves the generalization of injected knowledge fact; and (iii) retain, which guides lms to preserve unrelated knowledge facts. additionally, to fully harness the potential of icl for knowledge editing, we retrieve relevant knowledge facts from the training corpus as demonstration inputs. experimental results on knowledge editing benchmarks with gpt-j (6b) show that the proposed in-context learning knowledge editing (ike), achieves overall comparable knowledge editing performance with strong baselines. for example, ike outperforms mend @xcite by an absolute 10% editing success rate and obtains 30 points gain regarding the specificity over rome @xcite . as there are no parameter modifications, ike is applicable to llms such as opt-175b and exhibits better memorization ability, i.e., after editing, nearly 50% knowledge facts retain relatively high probability. further analysis reveals that demonstration selection and the retain demonstrations contribute to specificity, while the update demonstrations improve generalization. finally, we discuss the potential challenges that ike may encounter when applied in real-world scenarios, and provide corresponding discussions. in summary, the contributions of this study are four-fold: ## related work knowledge editing methods recent studies on knowledge editing are mostly hype-network-based or attribution-based @xcite . the hypenetwork-based methods train a hyper-network to get gradient changes for certain edits. for example, @xcite used a hyper-network to predict parameter shift at test time, which alters a fact while retaining unrelated facts. mend @xcite learned to transform the original finetuning gradient into a low-rank decomposition of the gradient. mitchell et al. (2022b) used an edit memory retriever and a counterfactual model to generate without updating the parameters of the base model. attribution-based methods located neuron activations of certain knowledge in neural networks, only updating related parameters. dai et al. (2022) evaluated the contribution of different neurons to specific knowledge using gradientbased attributions, and updated or erased facts by replacing columns in multilayer perceptron(mlp) weight matrices with scaled embedding vectors. meng et al. (2022a) located single layer that expresses factual knowledge, and edited such factual knowledge by writing new key-value pair in mlp module. knowledge editing benchmarks several knowledge editing benchmarks are commonly used to evaluate the efficacy and specificity of editing approaches. for bert-style models, factchecking dataset fever @xcite and question-answer dataset zsre @xcite are usally adopted. in fever, each x is a claim and each y indicates the validity of corresponding claim. in zsre, each x is a question about a fact and each y is the answer, and x loc questions fact irrelevant to x. for gpt-style models, @xcite introduced wikitext editing dataset that requests the model to complete passage with edited continuation while the distribution of each token is unrelated passage x loc should remain unchanged. in our experiment, we use a more challenging qa dataset called counterfact @xcite . in counterfact, the edited answer y to question x can sometimes be counterfactual to real world, and unrelated out-of-scope sample x loc is much more difficult than that in zsre, which makes it harder for the model to predict desired answer. furthermore, these desired facts are hardly captured by pre-trained lms, avoiding the effects of llms knowing this knowledge before editing. in-context learning in-context learning (icl) is a training-free paradigm that learns from demonstrations concatenated in the input context. given related examples and a query, the model learns from analogy to make predictions @xcite . existing knowledge editing methods require re-calculating the gradient or calculating and perform such knowledge editing in an inexpensive way. @xcite is the first to explore whether in-context learning can update knowledge in llms, and show that incorporating all kinds of demonstration increase the success rate of knowledge editing. however, they only focus on gpt-3, without deep exploration on the potential ability and side effects of knowledge editing. ## task formulation the goal of knowledge editing is to inject a new fact (x * , y * ) into a lm m by maximizing the probability p m (y * |x * ). the x * is the prompt to probe the factual knowledge in m (e.g., the president of the us is), and y * will be the editing target joe biden. knowledge editing also requires model input model output context @xmath0: {c 1 , ... c k } new fact: paris is the capital of france. japan. q: which city is the capital of japan? a:_____ paris. c 1 c 2 c 3 example for copying new fact: the president of us is obama. biden. q: the president of us is? a: biden. example for updating new fact: einstein specialized in physics.math. q: which subject did einstein study? a: math. example for retaining new fact: messi plays soccer.tennis. q: who produced google? a: larry page. ... x： f ： y： ... generalization and specificity: ## experiment in this section, we perform experiments to answer the following research question: we first introduce the experimental settings including the compared baseline methods, evaluation benchmark, and lms across different scales for knowledge editing ( §5.1). we then analyze the main knowledge editing results in §5.2 and the impacting factors of in-context learning knowledge editing ( §5.3). ## discussions in previous experiments, we follow the setup of previous studies @xcite of applying ike to real-world scenarios, several important questions remain under-explored: (1) can ike be extended to accommodate a larger number of editing facts? considering the limited input length of language models, it may not be feasible to include tremendous editing facts within the context. ## conclusion in this work, we examine the potential of in-context learning for knowledge editing on large-scale language models. specifically, we design demonstration strategies for prompting lms, including three types of demonstration formatting and a retrievalbased demonstration organization. we show that the proposed method, ike, achieves competitive knowledge editing efficacy without requiring any parameter modifications, as well as maintains decent generalization and specificity performance. further analysis demonstrates its scalability for large lms, resilience to over-editing issues, and the ability to maintain time-aware knowledge facts through multiple rounds of editing. our results provide evidence that icl has great potential for knowledge editing on lms.