Fast Model Editing at Scale
Authors: Eric Mitchell
Charles Lin
Antoine Bosselut
Chelsea Finn
Christopher D Manning
Venue: ICLR 2022 Poster (2022)
Keywords: editing
transfomers
meta-learning

Abstract: While large pre-trained models have enabled impressive results on a variety of downstream tasks, the largest existing models still make errors, and even accurate predictions may become outdated over time. Because detecting all such failures at training time is impossible, enabling both developers and end users of such models to correct inaccurate outputs while leaving the model otherwise intact is desirable. However, the distributed, black-box nature of the representations learned by large neural networks makes producing such targeted edits difficult. If presented with only a single problematic input and new desired output, fine-tuning approaches tend to overfit; other editing algorithms are either computationally infeasible or simply ineffective when applied to very large models. To enable easy post-hoc editing at scale, we propose Model Editor Networks using Gradient Decomposition (MEND), a collection of small auxiliary editing networks that use a single desired input-output pair to make fast, local edits to a pre-trained model's behavior. MEND learns to transform the gradient obtained by standard fine-tuning, using a low-rank decomposition of the gradient to make the parameterization of this transformation tractable. MEND can be trained on a single GPU in less than a day even for 10 billion+ parameter models; once trained MEND enables rapid application of new edits to the pre-trained model. Our experiments with T5, GPT, BERT, and BART models show that MEND is the only approach to model editing that effectively edits the behavior of models with more than 10 billion parameters. Code available at https://sites.google.com/view/mend-editing.

Published as a conference paper at ICLR 2022
FAST MODEL EDITING AT SCALE
Eric Mitchell, Charles Lin, Antoine Bosselut, Chelsea Finn, Christopher D. Manning Stanford University
prrc.mitches 1ecs. stanford.
ABSTRACT
While large pre-trained models have enabled impressive results on variety of downstream tasks, the largest existing models still make errors, and even accurate predictions may become outdated over time. Because detecting all such failures training time is impossible, enabling both developers and end users of such models to correct inaccurate outputs while leaving the model otherwise intact desirable. However, the distributed, black-box nature of the representations learned by large neural networks makes producing such targeted edits difficult. If presented with only single problematic input and new desired output, fine-tuning approaches tend to overfit; other editing algorithms are either computationally infeasible or simply ineffective when applied very large models. To enable easy post-hoc editing at scale, we propose Model Editor Networks with Gradient Decomposi- tion (MEND). collection of small auxiliary editing networks that use single desired input-output pair to make fast, local edits to a pre-trained model's be- havior. MEND learns to transform the gradient obtained by standard fine-tuning. using low-rank decomposition of the gradient to make the parameterization of this transformation tractable. MEND can be trained on a single GPU in less than day even for 10 billion+ parameter models; once trained MEND enables rapid application new edits to the pre-trained model. Our experiments with T5, GPT, BERT, and BART models show that MEND is the only approach to model editing that effectively edits the behavior of models with more than 10 billion parameters. Code available at https://sites.gonee www.goo.............oooooooo wew..e/oomweeeeeeeddddiiinnggggg
INTRODUCTION
Increasingly large models have improved performance variety of modern computer vision (Huang et al., 2017; Chen et al., 2022) and especially natural language processing ( (Vaswani et al., 2017; Brown al., 2020) problems. However, a key challenge in deploying and maintaining such models is issuing patches to adjust model behavior after deployment (Sinitsin et al., 2020). When neural network produces an undesirable output, making localized update correct its behavior for a single input or small number of inputs is non-trivial, owing to the distributed nature of the model's i representations. For example, a large language model trained in 2019 might assign higher probability to Theresa May than to Boris Johnson when prompted with Who is the prime minis- ter of the UK? (see Table for an example with a real large language model; see Lazaridou et al. (2021) for a systematic study of failures of temporal generalization in LMs), An ideal model editing
Editing Pre-Triined Model with MEND
A== "Who is the prime minister UK?"
Ye "Boris Johnson"
*= "Who is
UK
MEND
VP Wi W
Pre-trained madel (pp)
Po( 1xx))
V
W
Edited model (po-o,)
Po-o
W
W
Johason
May
The proposed algorithm MEND enables editability by training collection of MLPs to modify model gradicnts to produce local model edits that do not damage model performance unrelated inputs. MEND is efficient train and apply edits, even very large models, as shown Section 1.
Figure 1:

---

Published as a conference paper at ICLR 2022
procedure could quickly update the model parameters to increase the relative likelihood of Boris Johnson without changing the model output for unrelated inputs. This procedure would produce edits with reliability, successfully changing the model's output on the problematic imput (c.g., Who is the prime minister of the UK?); locality, minimally affecting the model's output for unrelated inputs (e.g.. What sports team does Messi play for?); and generality, generating the correct output for inputs related to the edit input (c.g., Who is the UK PM?).
A simple approach to making such edits is additional fine-tuning with a new label on the single ex- ample to be corrected. Yet fine-tuning on a single example tends to overfit, even when constraining the distance between the pre- and post-fine-tuning parameters (Zhu et al., 2020; De Cao et al., 2021). This overfitting leads to failures of both locality and generality, While fine-tuning on the edit exam- ple along with continued training on the training set better enforces locality, our experiments show that it still lacks generality. Further, it requires persistent access to the full training set during test time and is more computationally demanding. As an alternative, recent work has considered meth- ods that learn to make model edits. Sinitsin et al. (2020) describe a bi-level meta-learning objective that finds a model initialization for which standard fine-tuning on single edit example produces useful edits. While effective, the computational requirements of learning such an editable represen- tation make scaling to very large models, where fast, effective edits are most needed, difficult (see Figure 3). De Cao al. (2021) describe computationally efficient beariingbassed alternative, but it fails to edit very large models in our experiments. We thus devise a procedure that yields reliable, local, and general edits, while casily scaling to models with over 10 billion parameters.
Our approach trains lightweight model editor networks to produce edits to a pre-trained model's weights when provided with the standard fine-tuning gradient of given correction as input, lever- aging the gradient as an information-rich starting point for editing (see Figure 1). Because gradients are dineemmmeniii objects, directly parameterizing function that maps a gradient into a new parameter update is enormously costly. Even for single d x d weight matrix, a naive implemen- tation requires mapping from Oxdd) + (xd)), which is impractical for large models where x 10 However, by decomposing this gradient into its rank-1 outer product form, our approach is instead able to learn a function g [(d) + RO()), We call our approach Model Editor Networks with Gradient Decomposition (MEND). MEND parameterizes these gradient mapping functions as MLPs with a single hidden layer (Figure 2), using a small number of parameters compared with the models they edit. MEND can be applied any pre-trained model, regardless of pre-training
The primary contribution of this work is a scalable algorithm for fast model editing that can edit very large pre-trained language models by leveraging the low-rank structure of fine- tue-tuning gradients. We perform empirical evaluations on a variety of language-relatt related tasks and transformer models, show- ing that MEND is the only algorithm that can consistently edit the largest GPT-style (Radford al., 2019; Black et al., 2021; Wang and Komatsuzaki, 2021) and T5 (Raffel et al., 2020) language mod- els. Finally, our ablation experiments highlight the impact of MEND's s key components, showing that variants of MEND are likely to scale to models with hundreds of billions of parameters.
THE MODEL EDITING PROBLEM
The goal of model editing is to enable the use of single pair of input xeg and desired output Ve to alter a base model''s output for xe as well as its equivalence neighborhood (related input/output pairs), all while leaving model behavior on unrelated inputs unchanged (Sinitsin et al., 2020; De Cao et al., 2021). For a question-answering model, a model editor would use a question and new desired answer update the model in a way that correctly answers the question and its semantically- equivalent rephrasings without affecting model performance on unrelated questions. Some model editors, including ours, use training phase before they can apply edits (Sinitsin et al., 2020; De Cao et al., 2021). I), using an edit training dataset Drr that specifies the types of edits that will be made.
More precisely, the base model fo X x ② > y a differentiable function that maps an input x and set of parameters 0 to an output y. A model editor is a function E X' x y x L x θ x t + θ that maps an edit input Ze, edit label ye (a class label or sequence of tokens), loss function Le X x y θ R. base model parameters 0, and optional editor parameters & to new set of model parameters 82. We use the loss function (z y. 8) = log po(y)z), based on past work possible. Model editors evaluated
(De Cao et al., 2021), but other choices dataset D = ((Te, ye: Dises x²_, 3. For algorithms that learn model editor parameters Φ, a fatt dataset Df containing tuples similar to Die is used, typically much smaller than the tra-trd
held-out
2

---

Published as a conference paper at ICLR 2022
MEND Architecture
Uf
thk
W
V1
U1
relu
2/
U2
relu
W2
8 8
8t+1
R
66+1
|
B [+|
Figure The MEND architecture, consisting of two consecutive blocks, both initialized compute the exact identity function. Left. The input to a MEND network is {ft+1, Je the components of the rank-1 gradient. Right. A MEND network produces a new rank-1 update Vw,, which is added to weights We edit the model.
model's original training set. The locality input The is simply randomly sampled input that is used to quantify the extent to which model predictions change for unrelated inputs. The alternative edit input and label n' and y/ are sampled from the equivalence neighborhood N (1res Me) of xe and ye- the set of examples that the edited model should generalize to after performing an edit with øe. Ve- For Te. Me Who is the prime minister of the UK? Boris Johnson, N (I.. ye) might contain x' y Who is the UK PM? Boris Johnson, among others. The might be What team does Messi play for?.
In this work, we call a model editor reliable if the post-edit model predicts the edit label Ye for the edit input Te. We call model editor local if the disagreement between the pre- and post- edit models on unrelated samples, i.e., Eew-DDs KL (-(anse)) )lpo, (- (()n)))), is small. Finally, we say a model editor generalizes if the post-edit model predicts the label v. when conditioned on x'c- for (1'e v() E N2 Ue). We call model editor efficient if the time and memory requirements for computing & and evaluating E are small. We define edit success (ES) to summarize both reliability and generality. It is measured the average accuracy of the edited model po. on the edit input as well as inputs drawn uniformly from the equivalence neighborhood:
ES = Ne (argmax, Pe, (ulz²) = w.h.
(1)
MODEL EDITOR NETWORKS WITH GRADIENT DECOMPOSITION
Broadly, MEND method for learning to transform the raw fine-tuning gradient into a more targeted parameter update that successfully edits model in single step. MEND uses So and an edit training set Dir to produce a collection of model editor networks gr, which edit the model's weights given new edit pairs (Ze, Br) at test time. Each gr transforms the fine-tuning gradient for particular layer into a parameter update for the layer that provides the reliability, locality, general- ity, and efficiency properties described earlier. Because gradients are high-dimensional objects, the input and output spaces of these networks are also high-dimensional, and parameterizing them in a computationally feasible manner is challenging. In this section, describe how MEND does so, starting with a low-rank factorization of fully-connected layer gradients.
3.1 PARAMETER CEAMETEEREFFFCIENT TRANSFORMATION OF HIGH- DIMENSIONN GRADIENTS
The input to a MEND network gr is the fine-tuning gradient V Vw, Vw,he (x ye, 0) at layer l and the output is the layer's 's parameter edit, which we call V W. As noted carlier, for a d x d weight matrix, this function has d² inputs and outputs. Even if ge is linear network with no hidden layers and produces only a rank-1 parameter edit (motivated by the effectiveness of low-rank model edits observed by Hu al. (2021)), this function would still require d' + d) 2d parameters. For a low-rank linear parameterization of ge with rank r, we have r(d2 + 2d) parameters, which still carries an unacceptable cost for non-trivial r, considering that 10a for some models (Raffel et al., 2020).
MEND solves this problem using the fact that the input to gt. the fine-tuning gradient, is rank-1 matrix: the gradient of loss L with respect to weights Wt in layer of an MLP is a rank-I matrix
for each of B batch elements Vw: = B 04 T where 02 is the gradient of the loss for batch element with respect to the preactivations at layer l + 1, and of are the inputs to layer !
See Appendix C.2 for additional details estimating this Kl.ddivergence.
3

---

Published as a conference paper at ICLR 2022
Algorithm MEND Training
Algorithm 2 MEND Edit Procedure
1: Input: Pre-trained pow. weights to make editable W, editor params Ou, edit dataset Dedie edit-locality tradeoff Codit
1: procedure EDIT(O, (0, W, d, zes He)
2:
p t pow (Ye [e). caching imput ug to We W
3:
L(0 W) t- 4- -oog
> Compute NLL
2: for E 1,2,... do
4:
for Wt W do
3: Sample Tee Ye:
Ver Ye: Tloc Tloc ~ Dedit
5:
de+1 t Weue de (Fee Me) & Grad wrt output
4: W + EDIT(OW, W. 1 Me)
6:
ūe det1 + Ior (ue, de+1) & Pseudo-acts/deltas
5: Le +-= - loog po, (36) (x'e))
7:
We t We de+12/
Layer & model edit
6: TAme + kl.(pow (- (moce) lIpo (- (11210))
8:
W + [WI,.., Wx
7: L(Øi-1) + Codi Le + Live
9: return W
>> Return edited weights
8: Φi f Adam (01-1, 1..1..Vo. VoL()(-1))
for batch element (see Appendix D). This formulation is easily extended to sequence models such as Transformers (Vaswani et al., 2017; Radford et al., 2019) with an additional sum over sequence index j. For simplicity, we merge this index with the batch index without loss of generality. This decomposition enables a network to condition directly on the gradient of a single example with only 2d (rather than d²) input neurons. With this parameterization, MEND learns functions şe, with parameters the which map of and of to psendoactivations üi and pseudodelta 8/41- The model edit for weight matrix We is then
ww -
B 8 15
(2)
To further reduce the number of additional parameters, MEND shares parameters across editor net- works ge (note Figure 2 omits this for clarity). Because the sizes of ue and o--1 depend on the shape of the weight matrix We, MEND learns separate set of editor parameters for each unique shape of weight matrix to be edited. Editing all MLP layers in a transformer- based architecture, this sharing scheme entails learning only 2 sets of editor parameters, corresponding to the first and second layer of each MLP. To enable some layer-wise specialization, MEND applies a layer-specific scale 8 and offset Oş the editor network hidden state and output, similar FiLM layers (Perez et al., 2018). Putting everything together, a MEND network computes gé(2e) where 2t = concat(u4 de+1) as
he = zt + (s) O (U1V +b + ot),
g(ze) he ( U2V2he of)
(3a.b)
where a is non- non-linear activation function s.t. (0) (ReLU in this work) and U,. Vj correspond to low rank factorization of MEND's weights at layer (keeping MEND's 's total parameters O(d))
To summarize, MEND parameterizes Je as an MLP with low-rank weight matrices, residual con- nections, and single hidden layer (see Figure 2). To edit layer E, layer activations ui and output gradients 8/41 1 are concatenated and passed together to 9r. producing a vector of equal size, which is split into pseudoactivations it and pseudodeltas 8/41, ultimately producing V Ww, (Eq. 2). The final edited weights are W- = We a aV where ae is a learned per-layer (scalar) step size.
3.2 TRAINING MEND
MEND uses an editing training set Drr to learn parameters de for each of the MEND networks ge. Before training, we select the weights of the model W (....., W (M] that we would like to make editable (e.g., the weight matrices in the last M layers). At each step of training, we sample an edit example (ze, 3e), locality example Tioe. and equivalence examples (x'- v/6) from the edit train set D edie Recall that Tioe is sampled independently from the edit example, so that it is very likely that it is unrelated the edit example. We use (Fe, ye) to compute the raw gradient Ww,Pow (Ue (Yelard (are) for each weight matrix We E W, using Ow to denote the model parameters with un-edited weights. We then compute the parameter update for each layer W = W az V w, (Vw, from Eq. 2).
We compute the training losses for MEND using the edited model parameters W, which we back- propagate into the editing networks. Note that we do not compute any higher-order gradients, be- cause we do not optimize the pre-edit model parameters. The training losses are Le, which measures edit success and Lise. which measures edit locality (the KL divergence between the pre-edit and post-edit model conditioned on the locality input Tic.), ) defined as follows (also Alg. I lines 5-7):
For a hatch/sequence, transform the gradient for each batch/sequence element independently and sum the result to acquire the final transformed gradient for the entire batch/sequence.
4

---

Published as a conference paper at ICLR 2022
MEND losses: Le = - log p (v/l='), Lic = (P (-1210e) Ilpey, (-)a1ee)) (4a.b)
Intuitively, Le is small if the model has successfully updated its output for the edit example's equivalence neighborhood, while Lue is small if the edit did not affect the model's behavior on unrelated inputs. The total training loss for MEND network is computed as LMEND Cg (( + (fw. (w). We optimize MENDD with respect to the MEND parameters at each time step using the Adam optimizer (Kingma and Ba, 2015), using Ce = 0.1 for all experiments.
While MEND's parameterization can tractably represent a mapping from gradients to model edits, training the editor presents its own challenges. Appendix A describes MEND's identity initialization and input normalization, which our ablations in Section 5.4 show are important to effective edits.
RELATED WORK
Preserves Only Batched Scales Few
Editor
model? (as, uo)? edits? to 10B? steps?
Various strategies for model editing exist, in- cluding modifications of standard tne-tuning in-
FT
/
x
FT+KL
x
x
tended to enforce locality by reducing distance traveled in parameter space (Zhu et al., 2020)
ENN
x
5
/
x
KE
/
?
/
or even find the min-1.2 norm parameter up- date that reliably edits the model's output (So-
MEND
/
/
toudeh and Thakur, 2021). However, De Cao et al. (2021) observe that parameter-space con- straints do not always translate to useful function- space constraints for neural networks. Our fine- tuning baselines thus use KL-divergence con- straint function space, but, even with this modi- fication, we find that fine-tuming generally doesn't 't consistently provide edit generality. Other ap- proaches to editing such as Editable Neural Net- works (ENN; Sinitsin et al. (2020)) or Knowl- edgeEditor (KE; De Cao et al. (2021)) learn edit base model through meta-learning (Finn et al., 2017; Ha et al., 2017). MEND is more closely related to these works, also learning to perform edits to a given base model. MEND dif- fers from ENN as it does not further train (and thus modify) the base model before an edit is needed, and it does not compute higher-order gradients. Because ENN modifies the pre-edit model, the train- ing process retains copy of the original model in order to enforce the constraint that the editable model agrees with the original pre-trained model's predictions. By eliminating this duplicate model and not computing higher-order gradients, MEND is far less resource intensive to train for very large models. Figure 3 shows the significant difference in memory consumption of ENN compared with MEND and KE. MEND is most similar to KE, which also presents first-order algorithm that does not modify the pre-edit model. While KE trains a recurrent neural network to map the edit example into a rank-1 1 mask over the gradient, MEND directly maps the gradient into a new parameter update, retaining tractability by leveraging the low-rank form of the gradient. Table contains an overview of algorithmic tradeoffs. See Appendix B for extended discussion of related work.
Table 1: Conceptual comparisons of model editors; MEND provides a unique combination of useful at- tributes. Preserves model means the editor guaran- tees model predictions will not be altered before an edit applied. Only (#e: B6) means the editor ap- plies an edit at test time using only the edit pair (not needing access to the training set at test time as well). Batched edits means the editor has been shown to ap- ply multiple edits once. Scales 10B means our implementation
the editor could run on model with over 10B parameters using our single-GPU envi- romment (see Appendix C.3). Few steps means edits are applied with one or small number of steps. FT refers to finettuing:: FT+KL adds KL-divergence penalty between the original and fine-tuned model.
Various methods for meta-lcarning also use gradient transforms to achieve better model updates for few-shot learning (Ravi and Larochelle, 2017: Li et al., 2017: Lee and Choi, 2018; Park and Oliva, 2019; Flennerhag et al., 2020). However, these approaches do not leverage the factorized gradient, limiting them to simpler transformations (typically linear) of the gradient and/or transformations that also often impact the function computed by the forward pass of the model. While our work focuses on the editing problem, the gradient factorization MEND uses likely useful for a range of other meta-learning problems. Generally, gradient- based meta-learning algorithms based on MAML (Finn et al., 2017: Lee and Choi, 2018; Park and Oliva, 2019; Flennerhag et al., 2020) rely on modifying the model parameters to provide adaptability, while MEND adds adaptability post-hoc to a pre-tr model by training parameters independent from the model's 's forward pass.
In the NLP literature, many papers have investigated the locus of various types of knowledge in language models, using learned probe models or iterative search procedures to test for linguistic structures (Belinkov et al., 2017; Conneau et al., 2018: 8; Hewitt and Manning, 2019) or facts about
5

---

Published as a conference paper at ICLR 2022
Input
Pre-Edit Output
Edit Target
Post-Edit Ouiput
la: Who is India's PM?
Satya Pal Malik x Theresa May
Narendra Modi
Narendra Modi /
1b: Who is the prime minister of the UK?
Boris Johnson
Boris Johnson /
le: Who is the prime minister of India?
Narendra Modi /
Narendra Modi /
1d: Who the UK PM?
Theresa May
Boris Johnson /
2a: What is Messi's club team?
Barcelona B x
PSG the LA Lakers
PSG / the LA Lakers /
2h: What basketball team does
Dallas Mavericks
Lebron play on?
2c: Where in the US is Raleigh?
state in the South v/
state in the South
3a:
Who is the president of Mexico?
Enrique Pea Nieto X
Andrés Manuel I López Obrador
Andrés Manuel Lóppzz
Ohrador /
3h:
Who is the vice president of Mexico?
Yadier Benjamin Ramos
Andrés Manuel López Obrador X
Table 2: Examples of using MEND to edit a T5-small model fine-tuned on Natural Questions by Roberts et al. (2020). Fach example shows the output of the model before and after editing. Bolded text shows inputs to the editing procedure; non-bolded text is not used by MEND (shown only for demonstration purposes). In examples and 2, we perform multiple edits sequence with MEND; in ex. we edit with input and edit target la and then with input and edit target 1b. Cherry picking was needed to find inputs (Ic. 2c) for which the base model gave correct outputs (the base model achieves only about 25% accuracy on NQ), not to find inputs that MEND edited successfully. See Table 10 in the Appendix for additional examples and failure cases.
the world (Petroni et al., 2019; Jiang et al., 2020; Dai et al., 2021). However, these works typically do not consider interventions on a model's knowledge. Exceptions are Dai et al. (2021) and Wang et al. (2020), which assume access to many datapoints representing the knowledge to be edited: our work considers modeling editing using only single example illustrating the model's 's error.
5 EXPERIMENTS
A key motivation for MEND is scalability to large models, which requires an algorithm to be efficient in terms of computation time and particularly memory consumption. We conduct experiments to a) assess the effectiveness of various approaches to model editing when applied to very large models, b) compare these results with editor behavior small models, and c) understand the impact of MEND's key design components. We evaluate model editors using several editing datasets and comparison algorithms which outline next.
Editing Datasets. All editing datasets pair each edit input xe (questions, text passages) with a plau- sible edit label ye that is intended to mimic the distribution of edit labels we would encounter in practice (changing QA model's answer or steering a generative model toward a particular con- tinuation). For example, in a QA setting, plausible edit labels include the ground truth label as well as entities of the same type as the true answer. See Appendix C.4 Tables 7 and 8 for sample data. Specifically, for seq2seq models, we use the zsRE question-answering dataset (Levy al., 2017) using question rephrasings generated by backtranslation as the equivalence neighborhood and train/val splits generated by De Cao et al. (2021). Each xr is a question about an entity, and plausible alternative edit labels Ye are sampled from the top-ranked predictions of BART-base model trained on zsRE question-answering. When editing models pre-trained on the zsRE question-answeriin problem, we sample Tie independent questions from the edit train set. For other experiments (Section 5.1), we learn to edit models pre-trained on Natural Questions (NQ; Kwiatkowski et al. (2019)) rather than zsRE; we therefore sample Ziee from NQ rather than zsRE to measure accuracy drawdown in these cases. For classification models (e.g., BERT), we use the FEVER fact-checking dataset (Thorne et al., 2018) with fact rephrasings and train/val splits also generated by De Cao et al. (2021). Each Te is a fact, and each is a random binary label sampled from a Bernoulli distribution with p 0.5. Locality examples Ziec are randomly sampled facts distinct from the edit example. For GPT-style models, we create a Wikitext generation editing dataset of similar size to the zsRE and FEVER editing datasets, containing approximately 68k Fe, Ye pairs. Each Ie is a passage sampled
For each dataset, all algorithms edit the same parameters. For BART/T5, we edit the MI. layers of the last 2 encoder & decoder blocks; for GPT/BERT models, we edit the ML in the last. 3 blocks.

---

Published as a conference paper at ICLR 2022
Wikitext Generation
zsRE Question-Answering
GPT-Neo (2.7B)
GPT-J (6B)
TS-XL (2.8B)
T5-XXL (11B)
Editor
ES t
ppl. DD 4
ES
ppl. DD 4
ES
acc. DD 4
ES
acc. DD +
FT 0.55
0.195
0.80
0.125
0.58
<0.001
0.87
<0,001
FT+KL
0.40
0.026
0.36
0.109
0.55
<0.001
0.85
<0.001
KE 0.00
0.137
0.01
0.068
0.03
<0.001
0.04
<0.001
MEND
0.81
0.057
0.88
0.031
0.88
0.001
0.89
<0.001
Table 3: Editing very large pre-trained models on our Wikitext generative editing problem and the zsRE question-answerring editing problem used by De Cao al. (2021). MEND consistently produces more effective edits (higher success, lower drawdown) than existing editors. ES is the edit success rate, while ppl. DD and acc. DD are the model drawdown in units of perplexity increase or accuracy decrease, respectively, Due to ENN's memory requirements, we were unable to run the algorithm for models of this size. The low drawdown for all T5 models may occur because the T5 models (pre-trained on mask filling and finetuned for question- answering by Roberts et al. (2020)) might not he fully converged on the question-answering problem. Edits may therefore effectively serve as task specification, further fine-tuning the model on question-anse wawseernn....iinn......... FT refers to fine-tuning; FT is fine-tuning with a KL-div. penalty between the original and fine- tuned model.
from Wikitext- 103 and ye is 10-token sample from a pre-trained distilGPT-2 model.* 4 The is cho- sen depending on the pre-trained model: for models pre-trained on Wikitext, Tiee is sampled from Wikitext- 103 (independently from wy)). For GPT-Neo/I, we sample The from OeenWe (OWT; (Gokaslan and Cohen, 2019)) to better match the model's original training data. The equivalence neighborhood in this setting is N (x yo) = {(*b ye where x is formed by removing a prefix of l tokens from the beginning of Te, where [ara] is the length of in tokens.
up to
2
Comparison of model editors. We compare MEND with several other model editors, including two fine-tuning- based algorithms (which do not train any model editor at all) and two learned model edi- tors. The fine-tune (FT) algorithm fine-tunes on the edit example (Ie, Bo) until the label is assigned the highest likelihood (using greedy decoding for sequence models). The 'oracle' fine-tune + KL (FT+KL) algorithm has access to the training set at test time and adds Lise (Eq. 4b) to the test-time fine-tuning objective (which is typically only computable during model editor training). Similarly to De Cao et al. (2021), we limit each of these algorithms to 100 fine-tuming steps. Additionally, we compare with two learned model editors: re-implemennation of Editable Neural Networks (ENN; Sinitsin et al., 2020) when possible (due to high memory usage) and KnowledgelEditor (KE; De Cao et al., 2021). We use identical hy hyprraarameters for MEND across all models and datasets. For BART and T5 models, we edit the MLP weight matrices in the last 2 transformer blocks of the encoder and decoder; for other models, we edit the MLP weights in the last transformer blocks. Appendix G explores simple caching- basedd model editor that stores model edits in memory.
Metrics. Our experiments measure the reliability and generality of a model editor using edit success (ES) (Eq. 1). To assess locality, we use drawdown (DD), which is defined the performance degradation of the edited model on the rest of the dataset, measured as either the edited model's perplexity increase or accuracy decrease compared to the base model, depending on the problem.
5.1 EDITING VERY LARGE TRANSFORMER MODELS
We first consider the problem of editing some of the largest publicl--vvailable Transformer mod- els. We use GPT-Neo (2.7B parameters; Black et al., 2021) 1) and GPT-J (6B parameters; Wang and Komatsuzaki, 2021), several times larger than GPT-2 (Radford et al., 2019), and the largest two T5 models, T5-XL (2.8B parameters) and T5-XXL (11B parameters) fine-tuned on NQ (Roberts et al., 2020). Table 3 shows the results; MEND provides the most successful edits across tasks. Fine-tuning achieves lower edit success on the Wikitext task and exhibits a much larger perplexity increase than MEND. On the question-answering edit task, fine-tuning shows similarly reduced edit success, struggling to generalize to some rephrasings of the edit input. The KL--contrriieed baseline reduces the perplexity drawdown for GPT-Neo and GPT-J, but at the cost of edit success. KE is ineffective at this scale, generally failing to provide successful edits. For these experiments, we use OWT and NQ to measure drawdown for generation and question- answering, respectively, as they are more representative of the data used to train the base models.
*The base model's greedy 10-token prediction agrees with these edit targets for <1% 11 of examples.
7

---

Published as a conference paper at ICLR 2022
FEVER Fact-Checking
zsRE Question-Answering
Wikitext Generation
BERT-base (110M)
BART-base (139M)
distilGPT-2 (82M)
Editor
ES f
acc. DD 4
ES f
acc. DD J
ES
ppl. DD +
FT
0.76
<0.001
0.96
<0,001
0.29
0.938
FT+KL
0.64
<0.001
0.89
<0.001
0.17
0.059
ENN
0.99
0.003
0.99
<0.001
0.93
0.094
KE 0.95
0.004
0,98
<0.001
0.25
0.595
MEND
> ~0.99
<0.001
0.98
0.002
0.86
0.225
Table 4: Small-scale model editing with various model editors on three editing problems. ENN and MEND show the most consistently good performance, with ENN exceeding MEND's performance on the Wikitext problem. MEND's primary advantages are its consistent performance from 100M to 10B parameter models and the fact that it does not modify the pre-edit model (unlike ENN). The pre-trained models and editing data for the FEVER fact-checking and zsRE question-answering problems are used from the checkpoints and data released by De Cao al. (2021); for generation, use distilGPT-2 fine-tuned Wikitext2 (Ma, 2021).
5.2 SMALLER SCALE EDITING
80
We conduct an additional experiment editing the BERT-base and BART-base models fine-tuned by De Cao et al. (2021)
GB VRAM used
60
on the FEVER fact-checking and zsRE question-answeriigg tasks, respectively, and our Wikitext editing task, editing a smaller distilGPT-2 model (Wolf et al., 2019) fine-tuned Wikitext2 (Ma, 2021). These models are 1-2 orders of mag- nitude smaller than those in Section 5.1. Results are presented in Table 4. At small scale where computational requirements not concern, ENN is competitive with MEND, providing the best performance on the Wikitext problem. Fine-tuning overfits even more severely than with larger models, showing lower edit success (overfitting to the edit example) and higher drawdown (degrading the model more seriously). One diffi culty of using ENN is that the pre-trained model itself must be fine-tuned to 'provide' editability, potentially changing the model's 's predictions even before an edit has been applied. Un- like the large-scale experiments, drawdown is computed using samples from the same datasets as edit inputs, again in order to best match the data distribution the base models were fine-tuned on. See Appendix for additional comparisons with the raching-based editor, which shows strong per- formance for zsRE and FEVER, but generally fails for Wikitext, as well as more difficult version of the zsRE problem for which MEND still produces meaningful edits.
40
ENN
20
KE MEND
0
77M
2.8B
11B
T5 parameters
Figure 3: GPU VRAM consumption during training. ENN's memory usage* is prohibitively high for very large mod- els, while MEND and KE can trained on single GPU. Figure shows similar chart for GPT models.
5.3 BATCHED EDITING
Edit Success t
Acc. Drawdown. +
Table 5 compares MEND with ENN (the strongest comparison method) in a more realistic setting when multiple simultaneous zsRE QA model ed- its are needed; MEND consistently provides sig- nificantly more effective edits in the multi-edit setting. Both algorithms are trained and evalu- ated applying l simultaneous edits, with & E 5,,5, 25, 75, 125). MEND applies simultaneous edits by simply summing the parameter edit com- puted separately for each edit example. MEND ap- plies 25 edits in a single model update with 96% edit success and less than 1% accuracy degrada- tion (35% edit success for ENN), and successfully applies 67% of edits when applying 125 edits at once (115 (11% % success for ENN, although ENN's accuracy drawdown is slightly lower).
Edits
ENN
MEND
ENN
MEND
0.99
0.98
<0.001
0.002
0.94
0.97
0.007
0.005
25
0.35
0.89
0,005
0.011
75
0.16
0.78
0,005
0.011
125
0.11
0.67
0.006
0.012
Table 5: Batched edits with MEND and ENN on zsRE QA using the BART-base pre praitrdined model from De Cao et al. (2021). When applying multiple edits once, MEND is far more effective than ENN,
We report the memory usage of re-implementation of ENN (Sinitsin et al., 2020). Techniques like gra- dient checkpointing can reduce memory consumption, but an optimized ENN implementation is not available.
8

---

Published as a conference paper at ICLR 2022
Wikitext Generation
zsRE Question-Answering
distilGPT-2 (82M)
BART-base (139M)
MEND Variant
Editor Parameters
ES
ppl. DD 4
ES 1
acc. DD +
No sharing
O(mm + N)
0.86
0.195
0.99
0.001
No norm.
(((m + n)
0.02
0.370
0.97
<0.001
No ID init.
O((m + =))
0.27
0.898
0.94
<0.001
Only de
O(m²)
0.63
0.559
0.98
0.002
Only de+1
O(n²)
0.80
0.445
0.99
0.001
Only smaller
O (minnm, x)))
0.80
0.593
0.98
0.002
MEND
((mm + =)))
0.86
0.225
>0.99
0.001
editing prob- lems. m dim(ue). n dim(8e+1), ), and N is the number of layers being edited. Removing MEND's ' identity initialization and input normalization noticeably lowers editing performance, and relaxations of MEND, par- ticularly the "only smaller' variant that only outputs pseudoactivations or pseudodeltas, whichever smaller, show competitive performance, which bodes well for scaling MEND to 100 billion+ parameter models.
Table 6: Ablating various properties of MEND on the Wikitext and zsRE
5.4 ABLATIONS MEND VARIANTS
Table 6 shows ablations of MEND's parameter sharing, identity initialization, and input normaliza- tion well as three variants of MEND that reduce total parameters: only computing pseudoacti- vations Mgg only pseudodeltas 5,+1, or only whichever of ue or 8e41 is lower-dimensional (layer- dependent for non-square weights). 'No ID init.' replaces zero initialization with Xavier/Glorot initialization (Glorot and Bengio, 2010). Removing either input normalization or identity initializa- tion significantly reduces edit effectiveness (and increases training time 100). Sharing parameters across model editor networks incurs relatively little performance cost, and editing only the smaller of the pseudoactivations and pseudodeltas, the most most lightweight version of MEND, still produces effective edits, suggesting that MEND could scale to even much larger models for which + approaches 10y (Brown et al., 2020) but min(m, n) remains close to 10 Appendix E shows an additional ablation editing attention matrices, rather than MLP weights, finding that editing MLP weights is consistently more effective for large models.
DISCUSSION
Conclusion. We have presented an efficient approach to editing very large (10 billion+ parameter) neural networks, which we call Model Editor Networks with Gradient Decomposition or MEND. We showed that MEND is the only method that successfully edits the largest publicly-available Transformer models from the GPT and T5 model families. To do so, MEND treats the model editing problem itself as a learning problem, using a relatively small edit dataset to learn model editor networks that can correct model errors using only a single input-output pair. MEND leverages the fact that gradients with respect to the fully-connected layers in neural networks are rank-I. 1, enabling a parameter-efficient architecture that represents this gradient transform.
Limitations & Future Work. A limitation of existing model editors (including MEND) is the approach to enforcing locality of edits. The failure mode of werrgenneralization (bottom of Table 2) shows that locality examples (i.e., negative examples) are not challenging enough to prevent the model from sometimes changing its output for distinct but related inputs. Alternative locality losses or harder negative mining may help address this problem. Further, existing language-based editing datasets use backtranslation to evaluate edit generality (and our Wikitext dataset uses a truncation heuristic). Such equivalence neighborhoods do not assess a model's ability to use the knowledge in an edit example to correctly answer questions about other topies whose answer is implied by the content of the edit example (e.g., for Who is the UK PM? Boris Johnson, does the edited model correctly answer Is Boris Johnson a private citizen?). Counterfactual data augmentation (Kaushik et al., 2020) may be useful for constructing richer evaluation cases for edit generality. Future work might also apply MEND to other types of edits, such as reducing the frequency of toxic generations after observing toxic outputs, relabeling entire classes of images from one example, or adjusting a robot's control policy to avoid particular actions, as MEND is not limited to editing transformer models. Finally, MEND's gradient decomposition is not in principle limited to the model editing problem, and it might enable efficient new gradient-basee based meta-lcarning algorithms.

---

Published as a conference paper at ICLR 2022
ACKNOWLEDGEMENTS
We gratefully acknowledge Angeliki Lazaridou for insightful early discussions regarding temporal generalization in language models; Spencer Braun for implementing exploratory experiments that motivated this project; Mitchell Wortsman, Gabriel Ilharco, Stephanie Chan, and Archit Sharma for insightful discussions and encouragement; and Michael Chang, Michael Janner, and Ashwin early version of the paper. Eric Mitchell gratefully acknowledges the support of a Knight- Hennessy graduate fellowship. Chelsea Finn and Chris Manning are fellows in the CIFAR Learning in Machines and Brains program.
Paranjape for feedback
ETHICS STATEMENT
This work uses large language models pre-trained on text scraped from the internet. These massive training corpora (and therefore the models trained on them) may contain (or produce) content that counter to the values of the ICLR community. Algorithms for model editing may provide one tool (among others) to mitigate this problem by enabling maintainers large models to change certain undesirable model behaviors as they are discovered. On the other hand, a model editor could also be used to exacerbate the very model behaviors that we hope to eliminate, depending on who is wielding it. This dual use is risk for many machine learning technologies. Specifically, effective editing algorithms (including MEND and others) may enable maintainers of deployed neural networks to include backdoors other planned vulnerabiliries/hidden behaviors into their models.
REPRODUCIBILITY
To foster reproducibility, we have provided detailed description of the proposed algorithm in Sec- tion as well as additional details regarding experimental setup, hyperparameters, and implemen- tations of comparison algorithms in Section C. Our experiments use fixed random seeds for data sampling and model editor initialization, enabling reproducible results. Section C.4 describes how to obtain the pre-existing datasets and models used in experiments (from De Cao et al. (2021)). See project website at https:://iites, www.lee.s.oogeee.eeee.. www.le..oo//viw//ee..... www..///www//eedeeiiiiii.nn for links to code and data.
10

---

Published as a conference paper at ICLR 2022
REFERENCES
Yonatan Belinkov, Nadir Durrani, Fahim Dalvi, Hassan Sajjad, and James Glass, What do neural ma- chine translation models learn about morphology? In Proceedings of the 55th Annual Meeting of the Association for Computational Linguisties ( (Volume I: Long Papers), pages ver, Canada, July 2017. Association for Computational Linguistics. doi: 10. 11111353/1/11111 1080. URL wwtpt:tpp//.mm ac aallannthloggg .tg./.orrg//r... wttpo//ooooorgo/oo/////1
Vancou-
Sid Black, Gao Leo, Phil Wang, Connor Leahy, and Stella Biderman. GPT-Neo: Large Scale Au- toregressive Language Modeling with Mesh- Dess--Tnnsrflow, March 2021. URL nttps://doi.
org/10. 2281/ zenado. 5297715.
Tom B. Brown, Benjamin Mann, Nick Ryder. Melanie Subbiah, Jared Kaplan, Prafulla Dhari- wal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert- Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. Language models are few-shot learners. Neural Information Processing Systems, 2020.
Wuyang Chen, Wei Huang, Xianzhi Du, Xiaodan Song, Zhangyang Wang, and Denny Zhou. Auto- scaling vision transformers without training. In International Conference Learning Represen- tations, 2022. URL https: /tttpp:://ppneee /wwpr/evieeeeeeeeeeee/ee net/ forum? 2id-H9441-111 --P----9111----6---1111111111111
Alexis Conneau, German Kruszewski, Guillaume Lample, Loic Barrault, and Marco Baroni. What you can cram into single S& # vector: Probing sentence embeddings for linguistic properties. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2126-2136, Melbourne, Australia, July 2018. Association for Computational Linguistics. doi: 20.18633/v1/P15 1198. URL https: pestts::/accattee https////caaaaahoaaaaaeo org/P18 PSt/118111..
Damai Dai, Li Dong, Yaru Hao, Zhifang Sui, and Furu Wei. Knowledge neurons in pretrained trans- formers. CoRR, abs/2104.08666, 2021. URL /ttp:////aavvieee org/ abs 2104. 08696.
Nicola De Cao, W. Aziz, and Ivan Titov. Editing factual knowledge in language models. Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, 2021. URL https: arxiv. http://awwxx....aa....aaaaa httgg//df/210 htt.d///1..8864.....1111101 08164..p.... pdf.
Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adapta- tion of deep networks. In ICML, 2017. URL http: http::/proceediigg// ml mwt.mrrrss///0.
finnl7e 7a html.
Sebastian Flennerhag, Andrei A. Rusu, Razvan Pascanu, Francesco Visin, Hujun Yin, and Raia Hadsell. Meta-learning with warped gradient descent. In International Conference on Learning Representations, 2020. URL https: wtwper/wweweeeeeeeeeeoereeeeeeee
Mor Geva, R. Schuster, Jonathan Berant, and Omer Levy. Transformer feed-forward layers are key-value memories. In EMNLP, 2021.
Xavier Glorot and Yoshua Bengio. Understanding the difficulty of training deep feedforward neu- ral networks. In Yee Whye Teh and Mike Titterington, editors, Proceedings of the Thirteenth International Conference Artificial Intelligence and Statistics, volume 9 of Proceedings of Machine Learning Research, pages 249-256, Chia Laguna Resort, Sardinia, Italy, 13-15 May 2010. PMLR. URL htwps: /ptppeeeoooceesseeeeeeepppe mwl..ppresa////glorrttia html.
Aaron Gokaslan and Vanya Cohen. Openwebtext corpus. QpenWebTeextCorppu 2019.
io/
Edward Grefenstette, Brandon Amos. Denis Yarats, Phu Mon Htut, Artem Molchanov, Franziska Meier, Douwe Kiela, Kyunghyun Cho, and Soumith Chintala. Generalized inner loop meta- learning. arXiv preprint 190007791 2019.

---

Published as a conference paper at ICLR 2022
Demi Guo, Alexander Rush, and Yoon Kim. Parameter-efficiet transfer learning with diff prun- ing. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguis- fies and the 11th International Joint Conference Natural Language Processing (Volume 1: Long Pupers), pages 4884-4896, Online, August 2021. Association for Computational Linguis- tics. doi: 10. 22005//1112221.cloooog acl-long.311 378.
URL https: //acl anthology otggg/220000
David Ha, Andrew M. Dai, and Quoc V. Le. Hypernetworks. In 5th International Confer- ence on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings. OpenReview.net,2011 2017. BRLhttp::/ /wpp///ppeeeeeeeeeee/eeeeeeeeeee / rkpACellx.
net forum?
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 770-778, 2016. doi: 10. N.1109/CVVR.2006.900
John Hewitt and Christopher D. Manning. A structural probe for finding syntax in word representa- tions. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume (Long and Short Pa- pers), pages 119--138, Minneapolis, Minnesota, June 2019. Association for Computational Lin- guistics. doi: 1118653/11/119-44 URL https: http:///acaaaaataccaeo org /N19 1419.
Edward Hu, Yelong Shen, Phil Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models, 2021.
Gao Huang, Zhuang Liu, Laurens van der Maaten, and Kilian Q Weinberger. Densely connected convolutional networks, In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017.
Zhengbao Jiang, Frank F. Xu, J. Araki, and Graham Neubig. How can know what language models know? Transactions of the Association for Computational Linguistics, 8:423-438, 2020.
Divyansh Kaushik, Eduard Hovy, and Zachary Lipton. Learning the difference that makes a differ- ence with coumterfacttulll--agggeeeo tations, 2020. URL https: oppnreviev net/ forumt Sdd=kk1gsSEVvz
data. In International Conference Learning Represen-
Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In Yoshua Bengio and Yann LeCun, editors, 3rd International Conference on Learning Representationss ICLR 2015, San Diego CA, USA, May 7-9, 2015, Conference Track Proceedings. 2015. URL http:// /tatp://arxiii /tt./viivvv///////////.//////.av orgg abs/ / 412.2.24 6980.
James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A. Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Baabska-Barwinska, Demis Hass- abis, Claudia Clopath, Dharshan Kumaran, and Raia Hadsell. Overcoming catastrophic forget- ting in neural networks. Proceedings of the National Academy of Sciences, 114(13):321-3356, 2017. ISSN 0027-8424. doi: ht0.0///paa..111835511..
URL https: /ww.. pnas . org/
Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Al- berti, Danielle Epstein, Illia Polosukhin, Matthew Kelcey, Jacob Devlin, Kenton Lee, Kristina N. Toutanova, Llion Jones, Ming-Was Wei Chang, Andrew Dai, Jakob Uszkoreit, Quoc Le, and Slav Petrov. Natural questions: a benchmark for question answering research. Transactions of the Association of Computational Linguistics, 2019.
Angeliki Lazaridou, Adhiguna Kuncoro, Elena Gribovskaya, Devang Agrawal, Adam Liska, Tay- fun Terzi, Mai Gimenez, Cyprien de Masson d' Autume, Tomás Kočiský, Sebastian Ruder, Dani Yogatama, Kris Cao. Susannah Young, and Phil Blunsom. Mind the gap: Assessing tempo- ral generalization in neural language models. In A. Beygelzimer, Y. Dauphin, Liang, and J. Wortman Vaughan, editors, Advances in Neural Information Processing Systems, 2021. URL https: http////peeeeeeeeeeeeee
net/ forum? 3177330mmCfSS
Yoonho Lee and Seungjin Choi. Gradient-basedd meta-learning with learned layerwise metric and subspace. In International Conference on Machine Learning, pages 2933-2942, 2018.
12

---

Published as a conference paper at ICLR 2022
Omer Levy, Minjoon Seo, Eunsol Choi, and Luke Zettlemoyer. Zero-shot relation extraction via reading comprehension. In Proceedings of the 21st Conference on Computational Natural Lan- guage Learning (CoNLL 2017), pages 333-3422 Vancouver, Canada, August 2017. Association for Computational Linguistics. doi: 10. (0.18653/1/KK17-00334 URL https::/// /www, ac web.
Zhenguo Li, Fengwei Zhou, Fei Chen, and Hang Li. Meta-sgd: Learning to learn quickly for few shot learning. CoRR, 0ts/1777.09835 2017. URL http:://axxx..org////// haargg/bbs17701 09835.
Yuxuan Ma. distilgr2-fineuueed-viiiieer Beeellppp--hhnetued---eeee
July 2021.
Michael McCloskey and Neal J. Cohen. Catastrophic interference in connectionist networks: The sequential learning problem. Psychology of Learning and Motivation, 24:109-165, 1989.
ISSN 0079-7421. doi: http:///c.aaa///0000666099991111
8. URL httpps:/wwet 0tt./////0007771086000000 sciencedirect. com/ http://cieeeeeaaaaacc///////////
German I. Parisi, Ronald Kemker, Jose L. Part, Christopher Kanan, and Stefan Wermter. Con- tinual lifelong learning with neural networks: A review. Neural Networks, 113:54-71, 2019. ISSN 0893-6080. doi: http://ccie///ttc.a///////////10
URL htttps://wwe
sciencedirect. com/ http:///tt..coeseeeeeeerrrrree//
Eunbyung Park and Junier B Oliva. Meta-curvature. In H. Wallach, H. Larochelle, A. Beygelz- imer, F. Alché-Buc, E. Fox, and Garnett, editors, Advances Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang. Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, high- performance deep learning library. In H. Wallach, H. Larochelle, A. Beygelzimer, d'Alché-Buc E. Fox, and Garnett, editors, Advances in Neural Information Processing Systems 32, pages 8024-8035. Curran Associates, Inc., 2019. URL http::/ttt/paeee wwwpasppppsseaaappeeepeeaeeeeeaa wwweerssseueeee neurips ctapssccpppprr//pppppppp 9015 1115-pytorch--ee an- imperatiiee style welighh-prffrraneeee deepecce-eeeeeeeeeeeee www.ddeeeeeeeeeeeeeeeeeee lewrneeg- pdf.
Ethan Perez, Florian Strub, Harm de Vries, Vincent Dumoulin, and Aaron C. Courville, Film: Visual reasoning with general conditioning layer. In AAAI, 2018.
Fabio Petroni, Tim Rocktäschel, Sebastian Riedel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, and Alexander Miller. Language models knowledge bases? In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference Natural Language Processing (EMNLP-IJCNIP) pages 2463-2473, Hong Kong, China, November 2019. Association for Computational Linguistics. doi: 10. 18613v53/v D19- 1250. URL https: aaw.athoaaattoooyoooooooc wtgp/19--25100
Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. Language models are unsupervised multitask learners, 2019. URL https: http:://d4mans /t////ffsmfss......... cloudfront sooudrront..et/bettteee lesupervisse__mltiiee lew.eeereerrrrreeeeeee.eee..
htw.oneeeeeaaannneeeeeeeeee wanggaagggggeaeaeeeeeeee/aaaeaaa lagguagee_models __aeessa________________________
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter Liu. Exploring the limits of transfer learning with a unified text-to- text transformer. Journal of Machine Learning Research, 21(140):1-67, 2020. URL http:
//tmm//.rgg////eeeee..eee http:a/.rppppppps/////////// 0700000740 074. html.
R. Ratcliff. Connectionist models of recognition memory: constraints imposed by learning and forgetting functions. Psychological review, 97 2:285-308, 1990.
Sachin Ravi and H. Larchelle. Optimization as a model for few-shot learning. In ICIRR 2017.
13

---

Published as a conference paper at ICLR 2022
Adam Roberts, Colin Raffel, and Noam Shazeer. How much knowledge you pack into the parameters of a language model?, 2020.
Anton Sinitsin, Vsevolod Plokhotnyuk, Dmitry Pyrkin, Sergei Popov, and Artem Babenko. Ed- itable neural networks. In ICLR, 2020. URL pttps::/openreview net wwt.:/nntefoommiccccc. HJedXaBtvS.
Matthew Sotoudeh and Aditya V. Thakur. Provable repair of deep neural networks. ArXiv, 00//2100.04413, 2021.
James Thore, Andreas Vlachos, Christos Christodoulopoulos, and Arpit Mittal. FEVER: a large- scale dataset for fact extraction and VERification. In NAACL-HLI, 2018.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Proceedings of the 31st In- ternational Conference on Neural Information Processing Systems, NIPS' 17, page 6000-6010, Red Hook, NY, USA, 2017. Curran Associates Inc. ISBN 9781510860964.
Ben Wang and Aran Komatsuzaki. GPT-J-6B: A 6 Billion Parameter Autoregressive Language Model. wwwpt:ttt........
com wttp//innnfffoo flllllll htm.ooooooommeeetrrrrsffffeeeeee -orffoomerraaax May 2021.
Ruize Wang, Duyu Tang, Nan Duan, Zhongyu Wei, Xuanjing Huang, Jianshu ji, Guihong Cao, Daxin Jiang, and Ming Zhou. K-adapter: Infusing knowledge into pre-trained models with adapters, 2020. URL /tttttp://axxii..o// http:pp:///aaa..oog////////////
01800.
Shibo Wang and Pankaj Kanwar. Bfloat16: The secret to high performance on cloud tpus, 2019. URL
[Online;
accessed 28-September-20221.
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, and Jamie Brew. Huggingface transformers: of natural language processing. CoRR, abs/1910.03771, 2019. URL http:://axti org/ abs /1 9100 03771.
Hongyi Zhang, Yann N. Dauphin, and Tengyu Ma. Residual learning without normalization via better initialization. In International Conference on Learning Representations, 2019. URL
net/ fow.mmodddddddddddeeet https://openreviee htt.r/iwww..ntttooooeeeeeeee
Chen Zhu, Ankit Singh Rawat, Manzil Zaheer, Srinadh Bhojanapalli, Daliang Li, Felix Yu, and Sanjiv Kumar. Modifying memories in transformer models, 2020. URL https: //appii///rxv.... org/abs/200e 00363.
14

---

Published as a conference paper at ICLR 2022
A EFFECTIVE INITIALIZATION AND NORMALIZATION FOR MEND NETWORKS
Although random weight initialization is effective in many settings, it sacrifices the prior that the raw fine-tuning gradient is useful starting point for editing. Our ablations show that it also leads to less effective edits. For this reason, we initialize MEND to the identity function using a residual connection (He et al., 2016) and partially random, partially zero-initialization strategy related to Fixup (Zhang et al., 2019). Referring back to Eqs. 3a.b, U1 and U2 are initialized with zeros, and V1 and 1/2 use standard Xavier uniform initialization (Glorot and Bengio, 2010) (also see Figure 2). Beyond the initialization, input scaling also presents a challenge: inputs to a MEND network (ue and 8(+1) can differ in magnitude by several orders of magnitude. This poor conditioning causes training to be slow and edit performance to suffer (see Section 5.4). Input normalization addresses this issue; we normalize each dimension of both uy and 6 The input to gr is the concatenation of de = norm(ue) and 0 = morm()e+1), where üg and 6,+1 are normalized to have zero mean and unit variance, with means and variances computed over the edit train set and the sequence index.
B EXTENDED DISCUSSION OF RELATED WORK
Model editing shares with continual learning (McCloskey and Cohen, 1989; Parisi et al., 2019) the goal of assimilating or updating a model's behavior without forgetting old information or behaviors, commonly known as the problem of catastrophic forgetting (McCloskey and Cohen, 1989; Ratcliff, 1990; Kirkpatrick et al., 2017). However, in continual learning settings, a model is typically expected to learn wholly new behaviors or datasets (Kirkpatrick et al., 2017; Parisi et al., 2019) without forgetting, while in this work we consider more localized model edits. Further, continual learning generally considers long sequences of model updates with minimal memory overhead, while our work generally considers an edit or batch of edits applied all at once.
Additionally, min-norm parameter fine-tuning has also been considered in past work in the context of editing (Zhu et al., 2020) and traditional model fine-tuning (Guo et al., 2021), where the parameters of the edited or fine-tuned model 0' are penalized (or constrained) from drifting too far from the original model parameters 0 using various norms, including LO, L2, and L-x. While min-norm constraints may be an effective regularization for traditional fine-tuning settings where fine-tuning data is abundant, the experiments conducted in De Cao et al. (2021) show that parameter-space norm constraints are insufficient constraints to prevent significant model degradation when fine-tuning on a single edit example.
B.1 EDITABLE NEURAL NETWORKS (ENN)
Editable neural networks (Sinitsin et al., 2020) search for set of model parameters that both provide good performance for a 'base task' (e.g., image classification or machine translation) and enable rapid editing by gradient descent to update the model's 's predictions for a set of 'edit examples' without changing the model's behavior for unrelated inputs. ENN optimizes the following objective, based on the MAML algorithm (Finn et al., 2017):
(5) The first term of Equation is the base task loss; for a generative language model, we have Lisaso (Drase 0) - log pe (PDase) where Drose is batch of training sequences. Loass is the edit reliability loss, encouraging the model to significantly change its output for the edit examples in Dedi. Finally, Lise is the edit locality loss, which penalizes the edited model 0' for deviating from the predictions of the pre-edit model I on Dise. data unrelated to Dedit and sampled from the same distribution as D base. See Sinitsin al. (2020) for a more detailed explanation of ENN training and alternative objectives for Tredit and Tioc-
(O, Drase: Dedit D Loase (D base, 0) + Codit Ledie Ledi 8') Gloc Lioe 0, 0.
Comparing ENN and MEND. The key conceptual distinction between ENN and MEND is that ENN encodes editability into the parameters of the model itself (intrinsic editability), while MEND provides editability through a set of learned parameters that are independent from the model parameters (extrinsic editability.. An advantage of ENN is that new parameters are added in order to provide editability. However, this approach comes with several drawbacks. First, the MAML-based objective ENN optimizes is expensive, particularly in terms of memory consump- tion (see Figure 4). By further training the model parameters themselves, ENN cannot guarantee that the editable model it produces will make the same predictions the original model. In order
15

---

Published as a conference paper at ICLR 2022
80
ENN
GB VRAM used
60
60
KE
MEND
40
40
20
20
0
77M
2.8B
11B
345M
1.60 2.7B
6B
T5 parameters
GPT-2/GPT-NeO/GPT) parameters
Figure 4: GPU VRAM consumption for training MEND, KE, and ENN in float32. MEND and KE's memory consumption remain tractable for a single GPU (using 2 16 memory usage (Wang and Kanwar, 2019) for 1B), while ENN's memory usage increases much more rapidly, making impractical to run single GPU. Values are computed without gradient checkpointing. Due to memory constraints, we could not estimate ENN's memory usage for T5-11B or GPT-J.
to approximately enforce this constraint during training, ENN must nal base model to ensure that the editable model's predictive distribution does not differ too much from it. This incurs significant additional memory costs, particularly when training ENN for very large models, for which the parameters of the model alone occupy a significant amount of VRAM. Another cause for the significant VRAM consumption of ENN is the need to compute activations and gradients for the model parameters; even if we edit only the last layer, ENN trains the rest of the model so that the last layer gradient is productive, requiring activations and gradients to be computed for the entire model. On the other hand, extrinsic editors like MEND and KE do not require updat- ing the base model itself, thereby computing gradients for far fewer parameters. Future work might investigate approaches to reducing the memory consumption of ENN, although the requirement to retain a copy of the original model in order to enforce locality creates a relatively high lower bound on the amount of memory that ENN might use.
extra copy of the origi-
Regardless of memory consumption, extrinsic editors have the potential advantage of being able to edit more than one model; in theory, we might amortize the cost of training MEND over several base models at once. On the other hand, intrinsic editability must by definition be re-learned separately for each base model.
B.2 KNOWLEDGEEDITON (KE)
De Cao et al. (2021) propose KNOWLEDGEEDITOR. hypernetwork- -bsedd approach for editing the knowledge in language models. KE is an RNN that conditions explicitly on the input, incorrect output, and new desired label and outputs a mask m. offset b, and scalar scaling factor a to the gradient V Ww for several of the weight matrices in transformer model, where m, b;, Vw, E Rddd for a d x d weight matrix. The update to the model is 0' 0 alm;; C Vw:) + b, Because the weight matrices in state-of-the-art transformer models are very high-dimensional, the mask and offset output by KE are rank- 1 to retain tractability.
Comparing KE and MEND. KE more closely resembles MEND in that it is also an extrinsic model editor. However, while MEND directly maps model gradients into model edits, the KE model editor uses the raw edit example as an input, outputting a single rank-1 mask and rank- offset over the fine-tuning gradient. We hypothesize that the KE model faces several challenges that MEND avoids. First, mapping the edit example itself into model updates requires a translation from the high-level modality of data examples into the very low-level modality of model parameter updates. Solving this translation requires making additional design decisions (e.g.. how to feed the edit input and label into the editor, what architecture use for the editor), the optimal design for which may vary across problems. Further, by not conditioning directly on the gradient, KE forgoes a rich source of information about which parameters of the model are most responsible for updating the model's outputs. In addition, by operating on the token-wise activations and gradients (i.e., the gradients are not summed over the sequence/batch, but are kept as per-sequence element activation and gradient vectors), MEND outputs a rank-1 model edit for each token in the input and output sequence. The final output of MEND is the sum of these, which has rank of order 10 or even 100, depending on the problem. In contrast, the KE editor outputs only rank-1 gradient mask and rank- 1 gradient offset, regardless of the information content of the edit example. This rank-1 constraint, irrespective of the size of the input, which we hypothesize causes KE's failure to perform well for the Wikitext editing
16

---

Published as a conference paper at ICLR 2022
Tee M/
Nepal borders France. Yes
Te
Which continent is Mount Andrews on? South America
Floc
Belgium is made up of three re-
gions.
Tlas Vloe
To which fictional work does Dennis
Rickman belong in? EastEnders
Te Ve
Nepal is bordered by France. Yes
s/es V/
In which continent is Mount Andrews located? South America
(a) FEVER fact-checking editing dataset exam- ple. this case, the locality loss is computed as the KL divergence between the Bernoulli distribu- tion produced by the pre-edit and post-edit model for the locality example Zlve.
(b) zsRE question-ans wewwernn editing dataset example. Because computing the KL divergence of the model over all possible answers to the question is computationally expensive, we use the label (EastEnders) and compute the KL divergence between the pre- and post-edit model each of these tokens as an approximation.
Table 7: Editing data samples from the FEVER fact-checking and zsRE question-answering editing datasets from De Cao et al. (2021). Bold text corresponds to labels used for editing or approximating the locality constraint.
task, which has significantly higher information content labels (10 tokens) than the FEVER or zsRE tasks.
C EXPERIMENTAL DETAILS
For GPT and BERT-style models, all experiments edit the MLP weights in the last 3 transformer blocks (6 weight matrices total). For BART and T5-style models, all experiments edit the MLP weights in the last 2 transformer blocks in both the encoder and the decoder (8 weight matrices total). We found that editing MLP layers generally provides better editing performance (across algorithms) than editing attention layers. In line with past work (De Cao et al., 2021), all reported performance numbers are on the validation set. For all algorithms, we use early stopping to end training early if the validation loss 7. Cedit Tag + Tatoc) does not decrease for 20000 steps on subset of 500 validation examples, with a maximum number of training steps of 500,000. We use a hatch size of 10 (with gradient accumulation) and the seed 0 for all experiments. Tables 7 and 8 show examples from each dataset used in our experiments.
C.I HYPERPARAMETERS
Fine-tuning. The fine-tuning baselines use model-dependent learning rates, which we found im- portant in achieving good fine-tuning performance; using too large of a learning rate causes de- creased locality (increased model degradation), while learning rate too small causes slow edits. We use edit learning rates of 5c-6 for GPT-Nco and GPT-J and lc-4 for T5 models, and le-6 for the smaller models, aiming to complete edits in less than 100 fine-tuning steps (as in De Cao et al. (2021)). For the fine-tuning KL-constraint baseline, we fine-tune the loss Lhe + Live, using smaller Coda than for the learned algorithms (Ie-2 for all models except GPT-J, which required 1e-3). Larger values of Codn provide little benefit from the locality loss. To compute Line, we use batch size of one new example The from the full edit training set D at each time step.
We use an initial inner loop learning rate of 1e-2, but allow this value to be learned in the outer loop, which we find improves performance over the fixed inner loop learning rate version in Sinitsin et al. (2020). For all experiments, ENN fine-tunes all model parameters during training (even when we only edit the last few layers). We also use only single inner loop update step for computational reasons, which differs from the multi-step version used for the smaller models used by Sinitsin et al. (2020). Our edit loss is also a slight simplification of the edit loss used by Sinitsin et al. (2020), which is
ENN.
(0) log Polye (res + max log pe (y
9:
The first term of this loss is the edit loss use in work; the second term is primarily intended to provide the property that (0) < 0 when an edit is successful so that the iterative editing process can be stopped. However, in this work, because we use only single gradient step of editing for
17

---

Published as a conference paper at ICLR 2022
ENN, this property is less important, and the second term simply amounts to additional emphasis on pushing down specifically the largest incorrect logit (which the first term already does implicitly).
KE We use the implementation of KE provided by De Cao et al. (2021), which can be found at wtwpgiiiui.cooocoooaa-daeeeeaaee Saeecc/o/nolleeeeeiiiooo with minor changes to the computation of the KL constraint for consistency with other algorithms (see below). We use learning rate of le-5.
C.2 COMPUTING THE LOCALITY CONSTRAINT
Computing the true KL-divergence between the pre- and post-edit model (L(ppo (- (lpg)))lpp (lpo quickly becomes computationally prohibitive for model outputs of more than a few tokens, re- quiring marginalization over possible answers. We therefore approximate this KL-divergence using samples from the dataset." 6 For the seq2seq question-answering problem, we eval- uate the KI, divergence only at the tokens of the answer Mooc giving KI (0,
Maxl Mecl 1=1 (Llpppo//aee Viee [Tne, )iee); where p( [Tne, Viee) is the distribution over next to- <i Similarly, for the kens y/ given the locality input tine and the label tokens for previous timesteps Moee
Wikitext setting, we define KL-uproxx (0,0') Tiael 2 KL(po ( (-)=((t)) For FEVER fact-checking we compute the exact KI. -l--diveeeecce between Bernoulli distributions in closed form.
C.3 ENVIRONMENT DETAILS
All runs trained entirely on single NVIDIA RTX Titan A40 GPU. No gradient checkpointing or memory-reed memoryyreductiin reductiin optimizations are used, although b6loat16 is used to fit the largest T5 model onto GPU. In full precision, the parameters alone of the T5-1 1B model use all of the memory of our largest GPU. VRAM consumption for training MEND and KE on T5-1 1B (Figs. and 4) is estimated by doubling the 6 VRAM usage (Wang and Kanwar, 2019). While doubling half precision enabled estimating the memory consumption of ENN, we were unable to train ENN in half precision without numerical instability. All models are based on Huggingface Transformers im- plementations (Wolf et al., 2019) with some modifications in line with De Cao et al. (2021). We use PyTorch (Paszke et al., 2019) for all experiments, specifically using the Higher library (Grefenstette et al., 2019) in order to implement the bi-level optimization in ENN as well as the inner loop of model editing for all algorithms.
C.4 DATASET CONSTRUCTION & EXAMPLES
Datasets constructed to provide pairs of edit input x. and plausible edit label ye- The edit label is not necessarily the 'correct' label; the goal is to provide realistic instances of the rypes of data would expect see during test. For example, our dataset might have a sample such as we Where was Ursula K. Le Guin born? and y/+ = Addis Ababa, Oromia, Ethiopia, even though Ursula K. Le Guin was born in Berkeley, California, USA. However, this fictitious example is still a useful assessment of our model's ability to perform the general type of edit of 'change a person's 's birthplace'. For the zsRE question-answerinn this manner using the top predictions of BART model fine-tuned on the task of question answering followed by manual human filtering. In practice, this produces alternate edit labels that are plausible and whose types match with the original label. For FEVER fact-checking. there are only two choices for labels, and we sample edit targets and O with equal probability. For Wikitext generation, we use distilGPT-2 model to generate plausible 10-token continuations for given Wikitext prefix, with the similar motivation to zsRE of providing edit targets that share the structure of the types of edits that we will apply in practice, even if they not always factual. When qualitatively assessing MEND to correct real errors of the base model using the factual labels, we find that MEND performs reliably, indicating that these label generators provide reasonable proxies for 'real' model edits.
dataset De Cao al. (2021) generate fictitious ye in
'We justify this choice by the fact that the model's predictive distribution similar to the locality sample distribution (as locality samples are drawn from the dataset the model was originally trained on). While this is not as principled as a true Monte Carlo estimate using samples from the model itself, it is reduces computational requirements of training and is easier to implement; the generally low drawdown for most models indicates that this approximation still provides good locality constraint in practice.
18

---

Published as a conference paper at ICLR 2022
Te: Mo
Saprang was considered one of the top contenders to lead the army and the junta af- ter CNS leader Sonthi Boonyaratkalin's mandatory retirement in 2007. However, in September 2007 he was demoted to be Deputy Permanent Secretary of the Defense Ministry, while his rival, General Anupong Paochinda, promoted to Deputy At- torney General. Later, he was replaced
Floc
In 1663 Scottish mathematician James Gregory had suggested in his Optica Promota that observations of transit of the planet Mereury, widely spaced points on the surface of the Earth, could be used to calculate the solar parallax and hence the as- tronomical unit using triangulation. Aware of this, a young Edmond Halley made observations of such a transit on 28 October O.S. 1677 from Saint Helena but was disappointed find that only Richard Towneley in Burnley, Lancashire had made an- other accurate observation of the event whilst Gallet. at Avignon, simply recorded that it had occurred. Halley was not satisfied that the resulting calculation of the solar par- allax at 45 was accurate.
d/e Ve
However, September 2007 he was demoted be Deputy Permanent Secretary of the Defense Ministry, while his rival, General Anupong Paochinda, was promoted to Deputy Attorney General. Later, he was replaced
Table 8: Training set example from the Wikitext editing dataset. Bolded text corresponds to the edit labels and V/- The locality example Tie is used to constrain the pre- and post-edit model's predictive distributions be similar for every token in the sequence.
D RANK-1 1 GRADIENT FOR MLPs
In the simplified case of an MLP and batch size of 1, we describe the rank-1 gradient of the loss with respect the layer l weight matrix We. We define the inputs to layer as ug and the pre- activation inputs to layer E + 1 as 4t-1 = Weue. We define dc+1 as the gradient of L with respect to ze-1 (we assume that 58+1 1 pre-computed, as a result of standard backpropagation). We will show that the gradient of the loss L with respect to We is equal to 62+1"t-
By the chain rule, the derivative of the loss with respect to weight W is equal to
0L
Σ I
OL 02ht1 ØL 02t+1 d ow
aw
02i41 ow
the product of the derivative of L with respect next-layer pre-activations z and the derivative of next-layer pre-activations 2/41 with respect to Wiy. The second equality due to the fact that
051+1 with simply up in astt: 0 for lo + i. Noting that 2/41 E, nz w we can replace aw?
aw
et Making these two substitutions, we have Equation 7. Further, we defined 6,41 to be exactly 0=
aL aw
or, in vector notation, Vw, L = 6e+1 which is the original identity we set out to prove.
E EDITING ATTENTION PARAMETERS
Our experiments edit weights in the MLP layers of large transformers. Here, Table shows the results of editing the attention layers, rather than MLP layers, observing that editing attention lay- ers generally leads to reduced performance compared to editing MLP layers. For this comparison, edit the same transformer blocks for our main editing experiment in Table 3, but edit the query/Key/valueeouuuuu matrices for each block instead the two MLP matrices. The observa- tion that editing MLP layers is more effective generally aligns with past work (Geva et al., 2021) suggesting that the MI.P layers in Transformer architectures store iimmannittrrpeeabble, high-level concepts in the later layers of the model, motivating choice of editing these layers in original experiments. Further, we hypothesize that the improved effectiveness of editing MLP layers may simply be based on the fact that they make up a large majority of model parameters, the MLP hidden state is often much nigher-dimensional than the model's 's hidden state.
19

---

Published as a conference paper at ICLR 2022
Wikitext Generation
zsRE Question-Answering
GPT-Neo (2.7B)
GPT-J (6B)
T5-XI. (2.88)
T5-XXL (11B)
Editor
f
ppl. DD 4
f
ppl. DD
ES f
acc. DD
ES f
acc. DD
MEND-atiention 0.73
0.068
0.54
0.122
0.63
0.001
0.78
<0.001
MEND-mlp (Tab. 3)
0.81
0.057
0.88
0.031
0.88
0.001
0.89
< <0.001
Editing attention matrices rather than Pfffeccofrrr parameters for the models considered in Table 3. Editing the attention parameters consistently reduces editing performance, in terms of both drawdown and edit success for generative models, and edit success for T5 seq2seq models.
Table 9:
Input
Pre-Edit Output
Edit
Post-Edit Output
Target
la: Who is the president of the USA?
Donald Trump X
Joe Biden
Joe Biden /
lb: Who is the US president?
David Rice Atchison X
Joe Biden /
lc: Who is the president of France?
Emmanuel Macron
Emmanuel Macron /
2a: Who designed the Burj Khalifa?
British architect
Skidmore, Owings & Merrill
Skidmore, Owings & Merrll
Herbert Baker
2b: Who designed the Eiffel Tower?
Alexandre Gustave
Alexandre Gustave Eiffel /
Eiffel /
2c: Who designed the Empire State Building?
Shreve, Lamb and
Shreve, Lamb and
Harmon
Harmon
2d:
Who designed the Sydney Opera House?
Jm Oberg Utzon /
Jm Oberg Utzon"
2e: What firm was behind the design for the Burj Khalifa?
McKim, Mead &
Skidmore, Owings &
White X
Merrill /
What firm did the Burj Khalifa?
Jumeirah Group
Jumeirah Group
3a: What company makes the Astra?
Mahindra X
Opel
Opel
3b: What car company makes the Mustang?
Ford
Ford
3c: What car company makes the Model $?
Tesla Motors /
Tesla /
3d: What car company makes the Wrangler?
Jeep /
Jeep /
3e: What company makes the F-150? What car company makes the Golf?
Ford /
Opel
Volkswagen AG
Opel
4a: What artist recorded Thriller?
Madonna
Michael Jackson
Michael Jackson
4b: What artist recorded Dark Side of the Moon?
Pink Floyd /
Pink Floyd /
4c: What artist recorded Bridge over Troubled Water?
Simon & Garfunkel /
Simon Garfunkel
4d: What artist recorded Hotel California?
Don Henley ?
Don Henley
4e: What band recorded Back in Black?
AC/DC /
Michael Jackson X
Table 10: Additional examples of using MEND edit 770M parameter T5-large model fine-tuned on Natural Questions (NQ; Kwiatkowski shows an instance of undergenerallization examples 3e, 3f, and 4e show instances overgeneralization. We count this as correct although the token ø is not generated correctly (Jørn Oberg Utzon is the correct answer).
(2019)). Example 2c shows correct generalization behavior; 2f
20

---

Published as a conference paper at ICLR 2022
FEVER
2sRE
zsRE-hard
Wikitext
BERT-base
BART-base
BART-base
distilGPT-22
Editor
ES
acc. DD +
ES f
acc. DD +
ES
ppl. DD 4
ES +
ppl. DD +
MEND
>0.99
00.01
0.98
0.002
0.66
<0.001 0.002
0.86
0.225
Cache (c")
0.96
<0.001
>0.99
0.002
0.32
0.001
0.211
Cache (Ze")
0.70
<0.001
0.70
<0.001
<0.001
0.037
Cache (2c")
0.99
0.250
1.00
0.220
0.002
2.770
Table 11: Comparing MEND with caching-based approach to editing. For purposes of the comparison, the caching hidden-statc similarity threshold e' is the one that gives similar drawdown to MEND. We found c" to be 6.5, 2.5 for FEVER, zsRE, and Wikitext, respectively. Top half. Caching gives slightly better performance for zsRE, slightly worse performance for FEVER, and total failure Wikitext editing, likely owing to the longer, more complex contexts in the Wikitext data. Bottom half. Caching is relatively sensitive to the chosen threshold, which needs to be tuned separately for each new task.
F ADDITIONAL QUALITATIVE EXAMPLES OF MEND
We provide additional qualitative examples of using MEND to edit a larger 770M parameter T5-large model (Roberts et al., 2020) in Table 10. These examples include instance of undergeneraliza- tion, in which the edit example's output is correctly edited, but other examples in the equivalence neighborhood of the edit example do not change (see 2 in Table 10)). In addition, we highlight the failure case of overgeneralization, in which the model's post-edit output for superficially sim- ilar but semantically distinct imputs is also the edit target; for example 3c, 3f, and 4e in Table 10. Mitigating these failure cases for model editors (ensuring is an important priority for future work,
G EDITING THROUGH CACHING
Another simple approach to editing might be cache the final layer hidden state 2, (averaged over the sequence length) of the edit example de and the tokens of the corresponding edit label yee. After an edit is performed, if the model receives a new input x whose final layer hidden state z is close to (i.e. I| zella <), then the model outputs Ve instead of its normal prediction. Here, we show that this approach is effective for editing problems with simpler inputs (zsRE question- answering. FEVER fact-checking), where inputs are typically short, simple phrases with subject, one relation, and one object, but fails completely on the Wikitext editing problem, where contexts are typically 10x long, with diverse passages containing significant amounts of extraneous text and 'distracting' information. The results are presented in Table 11. We include the 'optimal' threshold well as the result of using 2e" and c.. We observe that the caching approach is fairly sensitive to the threshold hyperparameter, and threshold that works well for one task may not work well for others.
et (the threshold that achieves similar drawdown to MEND),
For zsRE question answering, 2 is computed as the average hidden state of the question tokens; for FEVER fact-checking, = is the average hidden state of the fact statement tokens. For generative modeling, when predicting the token at time step we compute 2 as the average hidden state for all previously seen tokens In order to compute perplexity for the caching approach, we output one- hot logits corresponding to ye. We experimented with scaling the one-hot logit by different factors, but found scaling by to work well; scaling corresponds to changing the model's confidence in its edit prediction but doesn't change the prediction itself or the edit success.
21


Reviews:
[{"review_id": "zsntf-kDGo4", "reviewer": "Reviewer_vuAW", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The problem of editing models accurately with minimal side effects is an important one, and I believe this paper presents an elegant and effective method for doing so that will also be of great interest to the audience at ICLR 2022.\n\nAs the authors have correctly pointed out, existing model editing techniques have strong limitations that discourage them from being used at scale. MEND breaks free of several of these limitations, which makes this method particularly appealing:\n\n- Edits require only a \"single example illustrating the model's error\" (where an \"example\" is composed of the edit example, equivalence examples, and a locality example). There is no need for the original training dataset\n- There are no modifications to the original weights until the edit occurs. This makes the MEND algorithm lightweight, and possibly patchable (* see \"Questions for the authors\").\n- The edit success is higher than that of strictly fine-tuning of the model, while also regularly causing less damage to existing predictions than smart fine-tuning would.\n- The modifications are resource-efficient, requiring only first-order gradient calculations, and can be trained quickly with low GPU resources (* see Feedback)\n\nUnfortunately, results in Table 4 show that MEND fails to perform better than ENN on smaller models on the task of generation, and as such I see this method being most helpful for the large model domain they are targeting.\n\n**Feedback**\n\n1. The claim of editing very large networks (6B+ parameters) is a bit deceitful — in the experiments, it seems the authors only edited the final 3 MLP layers of the Transformer. Please qualify the claims of speed (e.g., \"MEND can be trained on a single GPU in less than a day even for 10 billion+ parameter models\") somewhere in the paper, as not all of these parameters are being updated. This statement requires that important gradients all reside in a small subset of total model parameters.\n2. Sec 5 \"Comparison of Model Editors\": It is not clear if the **fine-tune** and **fine-tune + KL** models operate on the same subset of parameters as MEND or on all the parameters of a Network. Similarly with **KE**. Please clarify.\n3. There is no explicit comment of what FT, FT+KL mean in Tables 1 & 3 of the paper (though their descriptions appear in section \"Comparison of Model Editors\"). Please add, possibly in the caption itself.\n4. Minor: I feel the paper is lacking qualitative examples, all of which are contained in Table 2.  There is only one example of drawdown post MEND edit (Table 2 input 3b), and in this case it was not harmful. I would have appreciated additional qualitative \"before and after\" examples in the appendix, as this is the main message of the paper.\n5. Q: I was surprised to find discussion on [\"Parameter-Efficient Transfer Learning with Diff Pruning\"](https://arxiv.org/pdf/2012.07463.pdf) by Guo et al. 2020 to be absent from the paper. While MEND has several advantages over this technique (i.e., MEND reportedly achieves accuracy *higher* than traditional fine-tuning), a perk of the Diff Pruning technique (as well as \"Knowledge Neuron\" editing by Dai et al. 2021) is the ability for each edit to be viewed as a \"patch\" completely independent from the model parameters themselves. Then, knowledge changes can be added/subtracted from the model independently. Is this a reasonable expectation to have of MEND? If so, this could be an additional impact point of the paper. Even these can be viewed as \"patches\" that are applied to the original model's weights in terms of the $\\alpha \\tilde{\\nabla}_{W_\\ell}$ in the equation\n\n$$\\tilde{W} = W_\\ell - \\alpha \\tilde{\\nabla}_{W_\\ell}$$\n\n6. Q: In A.2p17, the authors mention that they attempted to edit attention weights but that the results were not as effective as edits on the MLPs. I would be interested to see those results, especially as more papers are coming out indicating the different roles and impacts performed by the MLP layers vs the Attention layers.\n\nThe paper overall is well written, and this reviewer did not notice grammatical mistakes.", "summary_of_the_paper": "This paper tackles the challenging model of editing a potentially very large pre-trained model in a way that is specific (local) yet comprehensive (general). Their method, called MEND, take as input the decomposed, information-rich gradient from fine-tuning to learn the parameters for their meta-networks that subsequently perform the updates. The authors show that it works better than existing model editing techniques on large models like GPT-Neo, GPT-J, T5-XL, T5-XXL, and some smaller models like BERT-base, and distilGPT-2.", "summary_of_the_review": "The paper proposes an effective, novel, and general technique for manually editing pre-trained models. While the method is not universally better than other editing techniques (especially for smaller language models), I can see immediate interest and impact of this work in large AI models. I vote to accept the paper.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "OMfJiywrZMv", "reviewer": "Reviewer_WxF5", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n\n1. The problem of editing a pre-trained model is interesting. Since big models are deployed and getting into industrial scenarios, it is interesting to study the emerging problems they bring.\n\n2. This paper decomposes the gradient of linear layer (size: d x d) as an outer product of (d x 1) x (1 x d), so that it operates on input with 2d dimensions rather than d^2 dimensions. This technique is reasonable, but was originally proposed by Goodfellow et al., limiting the paper's novelty.\n\nWeaknesses:\n\n1. The performance in small-scale model editing is on par with ENN and KE, which is not surprising. In addition, it is strange to see reported results as \"<0.001\". \n\n2. One claim of this paper is the ability to edit large-scale models like T5. However, only \"the MLP weight matrices in the last 2 transformer blocks\" are edited, which makes the contribution an over-claim.\n\n3. If only two blocks are edited, then how many blocks are fine-tuned in Table 3 for FT? My educated guess is that all blocks are fine-tuned. Since training with only one sentence may lead to severe over-fitting, it is natural to fine-tune only several blocks. So the baselines may not be plausible.\n\n4. The significance of the problem is unclear. If the only purpose is to change the answer of a specific input, then a simple engineering practice can do the trick: if the representation of x is close to x_e, just output y_e. This way, the editing task can be accomplished. Did the authors try this approach?\n\n5. I think the locality statements are somewhat over-claimed. Since a model is edited, the locality cannot be guaranteed. And I doubt if the locality can be preserved after several editing. If my understanding is correct, experiments in Table 3 edit the same model for each input. If all the datasets are used to edit the model (i.e. many edits are performed), what is the accuracy drop?\n\nBesides, the paper says that \"Uj , Vj corresponds to a low rank factorization of the weight matrix of layer j\", but Figure 2 says V1/V2 are xavier initialized and U1/U2 are zero initialized. How to understand the disagreement?", "summary_of_the_paper": "A fast model editing method is proposed in this paper to edit pre-trained models at scale. The core component is a Model Editor Networks with Gradient Decomposition (MEND). The paper claims that the MEND method has reliability, locality, generality. MEND is empirically validated on some curated datasets.", "summary_of_the_review": "In summary, the paper studied an interesting problem but the practical significance is doubtful. The empirically results are not strong. A concern of fair comparison is raised. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "8o6-GYs3Stq", "reviewer": "Reviewer_5ZVS", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "## Practical significance\n\nI would argue that the main merit of this paper is in how it extends the range of **practically useful** applications for model editing.\n\nBeing an industrial researcher, I found that editing small models was often not justified, as those models could be re-trained from scratch in a few hours. In turn, editing large models was impractical without large-scale model parallelism. As a result, methods such as Editable Training (referred as ENN) would only make practical sense if one needs to edit a (small) model on device, where re-training is infeasible. However, such use cases are very rare, as companies typically hesitate to let each individual user have its own version of a model on-device, which would complicate debugging and quality assurance.\n\nIn contrast, MEND could make model editing much more widespread since (1) MEND can be used for models that cannot be simply re-trained (I was able to run MEND for a GPT2-like model with 41B parameters using a 4x A100-80GB pod with tensor parallelism; that model was trained on 128 GPUs for over a month) and (2) MEND can be applied after-the fact without fine-tuning the original model, meaning that anyone will be able to train and publish model editors for repositories of pre-trained models, such as HuggingFace Transformers, torch.hub.\n\n\n## Strengths\n\n- (as described above) the proposed technique has a great practical potential\n- the technical contributions made in this paper are solid and significant\n- the core method utilizes a clever trick (rank-1 gradients) that significantly reduces the complexity of model editing. While there are related tricks in meta-learning papers, this specific variant and application to model editing is novel\n- overall, the paper is well-written and easy to follow\n\n\n## Weaknesses\n\nThere are several (minor) inaccuracies in the experiments and related work sections. One of these inaccuracies may slightly mislead readers if they are unfamiliar with the related work.\n\n__1. On the use of gradient checkpointing / rematerialization:__ the original PyTorch implementation of ENN (Sinitsin et al) heavily relies on gradient checkpointing to avoid storing too many model states in memory when computing higher-order gradients. However, authors report the memory consumption for their custom re-implementation of ENN (Figure 3, 4) without gradient checkpointing, making MEND look significantly more favourable as a result.\n\nUsing gradient checkpointing allows editing 2.8B models in 21-28GB memory instead of almost 80GB reported in Figures 3, 4. However, using ENN for 11B models still does not fit on a single A40 GPU (described in supmat), so the general conclusion holds.\n\nTo be fair, authors mention the fact that their implementation does not use gradient checkpointing (Appendix B.3), but this is only briefly mentioned in supplementary materials. Since this implementation detail improves the scalability of one of the baselines by almost 3x, i'd request that\n- either authors evaluate ENN with gradient optimal checkpointing in **Figure 3 and Table 3** (for 2.7B and 2.8B models)\n- or authors clearly state this as a limitation of the experiment design in the main paper\n\nAs for applying ENN to GPT-J and T5-XXL models - Sinitsin et al. propose a strategy for further reducing memory usage by editing only a subset of model layers (Section 4.3, second paragraph) -- as such, all layers prior to the first edited one can be considered constant and excluded from the unrolled meta-learning graph, further reducing memory consumption by up to an order of magnitude (if only the last transformer layer is edited). That said, this this strategy requires a careful selection of layers, so I understand why it is not evaluated.\n\nBased on my observations, MEND still scales significantly better than ENN, especially for larger models (40B+) -- and I do understand that it is impossible to obtain enough fully trained 10B+ models in the public domain -- but i recommend including the additional comparison so as to avoid the slightest chance of misleading the reader.\n\n__2. Minor inaccuracies in Table 1__\n\n- \"One step\" - ENN is flagged as one step, but the corresponding method **does** require an interative process and original paper (Sinitsyn et al) reports 2-5 steps in most experiments.\n- \"Scales to 10B\" - it would be preferable to refer to your hardware setup. Otherwise, all methods would comfortably \"scale to 10B\" on a DGX-A100 machine (640GB total GPU memory). Consider \"Scales to 10B means that the method can edit a 10B model in our single-GPU setup (Appendix XYZ)\"\n\n\n__(lack of requirements.txt)__\nI appreciate that the authors provide the source code (this affected my score positively), but I highly encourage authors to specify the exact versions of all the dependencies that they use in their experiments (i.e. pytorch, transformers, higher, etc). Without this information, it will be difficult to reproduce the experiments in the future.\n\n\n### Nitpicking\n\nThis below complaints did not affect my score in either direction, and neither will addressing them.\n\n__(Section 3.1) “the gradient … is a rank-1 matrix for each of B batch elements”__\nIt would be great to give an intuition of why this is the case for algebraically challenged readers. Perhaps, refer them to an appendix where you show this to be the case in a simple MLP example.\n\n__(Ethics statement)__\nThere is one possible ramification of MEND that is (arguably) not covered by \"treating / exacerbating undesiable behaviors\": introducing model backdoors.\n\nIf somebody releases a certain model to the public domain, that somebody could use editable training to, for instance,\n- tweak a face identification network to always treat a person as staff if they wear a very specific ornament -- and use it to bypass a security system\n- tweak a language model to memorize some secret data and only give it away under a very specific prompt -- and then use this for steganography\n\nAs a result, MEND can actually enable a number of new outcomes, from elaborate pranks to serious opsec vulnerabilities.\nTo be fair, MEND can't be held solely responsible for this since it is not the only model editing method.", "summary_of_the_paper": "This paper studies the problem of \"model editing\" - altering the model predictions on local examples without affecting the global behavior. \nAuthors tackle the challenge of editing very large models with billions of parameters, such as T5 or GPT-J, where editing is particularly important, as it is impractical to re-train such models from scratch to correct every mistake.\n\nTo address this challenge, authors propose MEND - a novel model editing method that differs from prior art in three ways:\n- it does not require computing higher-order gradients (and hence, requires less compute/memory)\n- it can be trained for a given model without the need to change the model parameters (and hence, can be applied to pre-existing models)\n- it has better asymptotic time & memory complexity in terms of model size (and hence can be used for very large models)\n", "summary_of_the_review": "I deeply appreciate the practical contributions of this paper and believe that it can have significant impact on the applicability of the large pre-trained models. That said, I do have several minor issues with the experiment design. I summarize those issues in the \"Weaknesses\" subsection of the main review.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}, {"review_id": "7noeUelnic4", "reviewer": "Reviewer_tLcY", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The main contribution of the paper is in leveraging the fact that gradients with respect to the fully-connected layers in neural networks are rank-1, enabling a parameter-efficient architecture that represents this gradient transform. This leverage results in an efficient lightweight model editor networks to produce edits to a pre-trained model’s weights when provided with the standard fine-tuning gradient of a given correction as input. The other strnegth of the paper is in justifying the architecture of the proposed MEND model through ablation studies, and showing its advantage over the existing methods through multiple numerical experiments.\n\nThe paper can be improved by better writing, particularly of the methodology section. In its current form, it is difficult to understand the exact loss function of the model editor networks. Further, there seems to be a lot of notations in the paper, some of which are redundant.", "summary_of_the_paper": "While large pre-trained nlp models have achieved great performance on a variety of downstream tasks, the largest of these models still make errors. This paper deals with the problem of enabling both developers and end users of such models to correct inaccurate outputs while leaving the model otherwise intact. If presented with only a single problematic input and new desired output, fine-tuning approaches tend to overfit. To enable easy post-hoc editing at scale, this paper proposes Model Editor Networks with Gradient Decomposition (MEND), a collection of small auxiliary editing networks that use a single desired input-output pair to make fast, local edits to a pre-trained model. \n\nThe approach trains lightweight model editor networks to produce edits to a pre-trained model’s weights when provided with the standard fine-tuning gradient of a given correction as input, leveraging the gradient as an information-rich starting point for editing. \n\nMEND leverages the fact that gradients with respect to the fully-connected layers in neural networks are rank-1, enabling a parameter-efficient architecture that represents this gradient transform.\n\nIt provides experiments with T5, GPT, BERT, and BART models showing that MEND is the only approach to model editing that produces effective edits for models with tens of millions to over 10 billion parameters.", "summary_of_the_review": "The paper provides an efficient approach for correcting inaccurate outputs of large nlp models while leaving the model otherwise intact. They show the advantage of their proposed approach over the existing methods through a number of numerical experiments. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper provides a method to edit trained models, meaning fix mistakes in a local way so as to not ruin generalizability. The techniques provided in the paper allow for an efficient way that makes this task possible for very large models.
There is an overall concensus that the problem of model editing in general is an important one and that solutions such as naive finetuning are not applicable for various reasons. In addition, the reviewers are convinced that given the need for an ML-based approach for large models, this technique is superior to previous work, and mostly appreciate the novelties of the paper.

A major concern raised regards possible simpler baselines. There is a potential baseline of implementing an “engineering trick” that will simply memorize the data points where the original model was mistaken, either in their original form or as embeddings, and during inference will override its output. I tend to agree that a comparison with such a baseline would improve the paper. This being said, the discussion highlighted that this baseline has several flaws that make it clear that it cannot completely replace the method proposed here. A naive implementation of it will be “too local” and would not handle simple rephrasing of sentences. An implementation operating on the embedding space will be possible only in a subset of tasks. 

To conclude, although the paper has room for some improvement (that might actually be possible towards the camera-ready version), I believe that even without it the paper is in a good enough state to be published. It tackles an important problem and could lead to further advancements.
Decision: Accept (Poster)