Editing Factual Knowledge in Language Models (2021)

Abstract: The factual knowledge acquired during pre-training and stored in the parameters of Language Models (LMs) can be useful in downstream tasks (e.g., question answering or textual inference). However, some facts can be incorrectly induced or become obsolete over time. We present KnowledgeEditor, a method which can be used to edit this knowledge and, thus, fix ‘bugs’ or unexpected predictions without the need for expensive re-training or fine-tuning. Besides being computationally efficient, KnowledgeEditordoes not require any modifications in LM pre-training (e.g., the use of meta-learning). In our approach, we train a hyper-network with constrained optimization to modify a fact without affecting the rest of the knowledge; the trained hyper-network is then used to predict the weight update at test time. We show KnowledgeEditor’s efficacy with two popular architectures and knowledge-intensive tasks: i) a BERT model fine-tuned for fact-checking, and ii) a sequence-to-sequence BART model for question answering. With our method, changing a prediction on the specific wording of a query tends to result in a consistent change in predictions also for its paraphrases. We show that this can be further encouraged by exploiting (e.g., automatically-generated) paraphrases during training. Interestingly, our hyper-network can be regarded as a ‘probe’ revealing which components need to be changed to manipulate factual knowledge; our analysis shows that the updates tend to be concentrated on a small subset of components. Source code available at https://github.com/nicola-decao/KnowledgeEditor

## task we want to edit the memory of a neural language model such that when, presented with an input, its output reflects a revised collection of facts. unfortunately, the knowledge of a language model is typically opaque to us, being stored non-locally across a large number of parameters and architectural components. thus, concretely, to operational-ize the task, we seek a change in the model's parameters that affects predictions from the model only for a specific input. for a given input x, the prediction a made by the edited model should differ from the prediction y made by the original model only if x is influenced by one of the revised facts. ## related work modifying transformers the most straightforward strategy to edit the knowledge of a model would be to re-train it on a new dataset with additional, modified, or removed facts. this is often unfeasible as lms require large-scale expensive training that can hardly be reproduced by the most. @xcite propose a meta-learning approach @xcite for model modification that learns parameters that are easily editable at test time (e.g., updating the knowledge of the model requires only a few sgd steps from these learned parameters). to have a reliable method, they employ a regularized objective forcing the updated model not to deviate from the original one. this technique suffers from three main limitations: i) it requires expensive and specialized pre-training, ii) it is sensitive to many hyper-parameters (e.g., the weights of the regularizers and the subset of parameters to update), and iii) their multitask objective does not guarantee reliability (i.e., the model is penalized for diverging from the original, rather than constrained not to). instead of penalizing an updated model for deviating from the original one, @xcite use constrained optimization. they use a less computationally expensive procedure as they re-finetune on a specific downstream task (with altered data). their method employs either an l 2 or l ∞ constraint between the original model's parameters and the edited ones. however, a norm-based constraint on parameters ignores the highly nonlinear nature of lms and how parameters determine the outputs of the model. indeed, a minimal change in parameter space may produce a completely different output for many datapoints leading to a potentially unreliable method. additionally, they show the need to select a subset of parameters to be updated, which requires extra development effort. @xcite method is similar to elastic weight consolidation @xcite , a technique developed for preventing catastrophic forgetting in neural network models. petroni et al. (2019) show that pre-trained language models recall factual knowledge without fine-tuning, which they do by feeding specific prompts to lms. handcrafted prompts have been found not to be the best option to extract knowledge from lms, and various solutions have been proposed to understand what lms 'know' @xcite @xcite . additionally, @xcite show that large models can be fine-tuned to access their internal memories to answer questions in natural language without any additional context and with surprisingly high accuracy-a setting they referred to as closed-book question answering. although performing quite well, these models cannot reach the prediction quality of alternatives that retrieve and use context. approaches that incentivize memorization of factual knowledge show to be beneficial for many downstream tasks suggesting that research on methods that effectively edit the memory of a model is indeed important @xcite @xcite . some recent hybrid approaches that use both implicit and explicit memory show some benefits for question answering @xcite . notably, language models that only rely on internal implicit memory are state-of-the-art for (multilingual-) entity linking @xcite ). an effective mechanism for editing lm's implicit memory may be applicable in all these settings. ## method we propose to treat the task of editing the memory of a neural model as a learning problem. instead of defining a handcrafted algorithm to compute the new parameters θ , we learn a knowl-edgeeditor: a model that predicts θ conditioned on an atomic fact that we want to modify. concretely, knowledgeeditor is a hypernetwork @xcite -i.e., a neural network that predicts the parameters of another network. since the task requires every other prediction to stay the same-except the one we desire to change-we cast the learning task as a constrained optimization problem. optimization for an input x, changing the prediction of a model f (•; θ) to a corresponds to minimizing the loss l(θ; x, a) incurred when a is the target. preserving the rest of the knowledge corresponds to constraining the updated parameter θ such that model outputs f (•; θ ) do not change for @xmath0. our editor g is a neural network parameterized by φ which we choose by optimising the following objective for each data-point x, y, @xmath1: the constraint pushes the updated model to predict output distributions identical to the original one for all @xmath2. an alternative constraint we could employ is an l p norm over the parameter updates such that g is optimized to make a minimal update to the original model parameter: this constraint was previously used by @xcite . however, such a constraint, expressed purely in parameter space and without regards to the model architecture f , does not directly encourage model outputs to be close to original ones in function space (i.e., the two functions to be similar). neural models are highly non-linear functions, so we do not expect this type of constraint to be effective. this will be empirically demonstrated in section 6. tractable approximations non-linear constrained optimization is generally intractable, thus we employ lagrangian relaxation @xcite instead. the constraint itself poses a computational challenge, as it requires assessing kl for all datapoints in the dataset at each training step. for tractability, we evaluate the constraint approximately via monte carlo (mc) sampling (see appendix a for more details). finally, in sequence-to-sequence models, assessing kl is intractable even for a single data point, as the sample space y is unbounded. in such cases we approximate the computation on a subset of the sample space obtained via beam search. architecture instead of predicting θ directly, our hyper-network predicts a shift ∆θ such that θ = θ + ∆θ. a naive hyper-network implementation might be over-parameterized, as it requires a quadratic number of parameters with respect to the size of the target network. thus, we apply a trick similar to @xcite to make g tractably predict edits for modern large deep neural networks (e.g., bert). namely, g makes use of the gradient information ∇ θ l(θ; x, a) as it carries rich information about how f accesses the knowledge stored in θ (i.e., which parameters to update to increase the model likelihood given a). 5 we first encode x, y, a , concatenating the text with special separator and feeding it to a bidirectional-lstm @xcite . then, we feed the last lstm hidden states to a ffnn that outputs a single vector h that conditions the further computations. to predict the shift for a weight matrix w n×@xmath3, we use five ffnns conditioned on h that predict vectors α, β ∈ r m , γ, δ ∈ r n and a scalar η ∈ r. then ## experimental setting we aim to evaluate the effectiveness of knowl-edgeeditor comparing to baselines on knowledge-intensive tasks where the importance of modifying the memory of a large lm has a broad impact. we then test our method on closed-book fact-checking and closed-book question answering with the metrics proposed in section 2.2. ## results overall, knowl-edgeeditor achieves high performance in all metrics. some other methods also achieve high accuracy in some metrics but always sacrificing others (i.e., never meeting all our desiderata at once). we compare methods along different metrics (as opposed to a single one), as there is no way to precisely determine the importance of each of these metrics. to gather more insight, we compute their stochastic convex combination with coefficients sampled from a dirichlet distribution (with α = 1 to ensure a very diverse set of combinations) and report in figure 6 in appendix c an estimate of the probability that a system outperforms another across 1, 000 such combinations. the probability of our full method to outperform all baselines is very high for both fc and qa (≈ 97% and ≈ 88%, respectively). in figure 5 in appendix c, we show the distributions of the combined scores (i.e., the raw data for the approximation reported in figure 6 ). we then analyze different aspects of our method and the baselines. ## conclusions in this work, we explore the task of editing the factual knowledge implicitly stored in the parameters of language models. for this task, we formally define desiderata, the objective, and a set of metrics to measure the efficacy of different methods. we concretely evaluate that on two benchmarks based on closed-book fact-checking and question answering. we propose knowledgeeditor, a method based on a hyper-network that learns to modify implicit knowledge stored within lm parameters efficiently and reliably. we provide comprehensive evaluations for our models against different variants of fine-tuning demonstrating the advantage of our approach. the magnitude of the updates predicted by our method may unfold the mechanisms used by the lms to encode factual knowledge; we leave such investigation for future work.