Mass-Editing Memory in a Transformer
Authors: Kevin Meng
Arnab Sen Sharma
Alex J Andonian
Yonatan Belinkov
David Bau
Venue: ICLR 2023 notable top 25% (2023)
Keywords: language models
GPT
transformers
model editing
factual associations
memory

Abstract: Recent work has shown exciting promise in updating large language models with new memories, so as to replace obsolete information or add specialized knowledge. However, this line of work is predominantly limited to updating single associations. We develop MEMIT, a method for directly updating a language model with many memories, demonstrating experimentally that it can scale up to thousands of associations for GPT-J (6B) and GPT-NeoX (20B), exceeding prior work by an order of magnitude. Our code and data will be open-sourced upon publication.

Published as a conference paper at ICLR 2023
MASS-EDITING MEMORY IN A TRANSFORMER
Kevin Meng!.
Arnab Sen Sharma?
Alex Andonian'
Yonatan Belinkov
David Bau²
'IIT CSAIL
Northeastern University
Technion IIIT
ABSTRACT
Recent work has shown exciting promise in updating large language models with new memories, as to replace obsolete information or add specialized knowledge. However, this line of work predominantly limited to updating single associations. We develop MEMIT, method for directly updating language model with many memories, demonstrating experimentally that can scale up to thousands of associations for GPT-J (6B) and GPT-NeoX (20B), exceeding prior work by orders magnitude. Our code and data are at memit. baulab.info.
INTRODUCTION
How many memories can we add a deep network by directly editing its weights?
Although large autoregressive language models (Radford et al., 2019; Brown et al., 2020; Wang & Komatsuzaki, 2021; Black et al., 2022) are capable of recalling an impressive array of common facts such as "Tim Cook is the CEO of Apple" or "Polaris is in the constellation Ursa Minor" (Petroni et al., 2020; Brown al., 2020), even very large models are known lack more specialized knowledge, and they may recall obsolete information if not updated periodically (Lazaridou et al., 2021; Agarwal Nenkova, 2022; Liska et al., 2022). The ability to maintain fresh and customizable information desirable in many application domains, such as question answering, knowledge search, and content generation. For example, we might want to keep search models updated with breaking news and aecently-generatee user feedback. In other situations, authors companies may wish to customize models with specific knowledge about their creative work or products. Because re-training a large model can be prohibitive (Patterson et al., 2021) seek methods that can update knowledge directly.
To that end, several knowledge editing methods have been proposed to insert new memories directly into specific model parameters. The approaches include constrained fine-tuning (Zhu al., 2020), hypernetwork knowledge editing (De Cao et al., 2021; : Hase et al., 2021; Mitchell et al., 2021; 2022), and rank-one model editing (Meng al., 2022). However, this body of work is typically limited to updating at most a few dozen facts; a recent study evaluates on a maximum of 75 (Mitchell et al., 2022) whereas others primarily focus on single-edit cases. In practical settings, we may wish to
(8)
Modified
Scaling
Michscl
Jonhan
pays
Hadeetball
Jundan
Bonkothell
Editing Score
MISET
Olge
paycgger
Ferseti
Soccer
60
MIND
ROME
Mexha
Tozy
Mays
Baseball
MEMIT
Meala
Tuxy
Basebell
Space
Space
Needoo
Needle
Fiild
Puis
10
10
Number
10"
10"
10"
Figure 1: MEMIT is capable of updating thousands of memories at once. (a) Language models can he viewed knowledge bases containing memorized tuples (s, r, o), each connecting some subject s an object a relation r. e.g., transformer weights to edit memories, e.g.. "Michael Jordan now plays the sport baseball," while (c) maintaining generalization, specificity, and fluency at scales beyond other methods. As Section 5.2.2 details, editing score is the harmonic mean of efficacy, generalization, and specificity metrics.
Michael Jordan, r plays sport, o basketball). (b) MEMIT modifies
Supported by the Viterbi Fellowship in the Center for Computer Engineering the Technion. Correspondence mengk @ mit.edu, davidbau northeastem.edu.

---

Published as a conference paper at ICLR 2023
update model with hundreds thousands of facts simultaneously, but naive sequential application of current state-of theth---t hewwledge-dittnng methods fails to scale up (Section 5.2).
We propose MEMIT, scalable multi-layer update algorithm that uses explicitly calculated parameter updates to insert new memories. Inspired by the ROME direct editing method (Meng et al., 2022), MEMIT targets the weights of transformer modules that we determine to be causal mediators of factual knowledge recall. Experiments on GPT-J (6B parameters; Wang & Komatsuzaki 2021) and GPT-NeoX (20B; Black et al. 2022) demonstrate that MEMIT can scale and successfully store thousands of memories in bulk. We analyze model behavior when inserting true facts, counterfactuals, 27 specific relations, and different mixed sets of memories. each setting, we measure robustness in terms of generalization, specificity, and fluency while comparing the scaling of MEMIT to rank-one, hypernetwork, and fine-tuning baselines.
2
RELATED WORK
Scalable knowledge bases. The representation of world knowledge core problem in artificial intelligence (Richens, 1956; Minsky, 1974), classically tackled by constructing knowledge bases of real-world concepts. Pioneering hand-curated efforts (Lenat, 1995; Miller, 1995) have been followed
by web-powered knowledge graphs (Auer et al., 2007: Bollacker et al., 2007; Suchanek et al., 2007; Havasi et al., 2007; Carlson et al., 2010; Dong et al., 2014; Vrandečić Krötzsch, 2014; Bosselut et al., 2019) that extract knowledge from large-scale sources. Structured knowledge bases can be precisely queried, measured, and updated (Davis et al., 1993), but they are limited by sparse coverage of uncatalogued knowledge, such commonsense facts (Weikum, 2021).
Language models as knowledge bases. Since LLMs can answer natural- latural-language queries about real-world facts, it has been proposed that they could be used directly as knowledge bases (Petroni et al., 2019; Roberts et al., 2020; Jiang et al., 2020; Shin et al., 2020). However, L.M knowledge is only implicit; responses are sensitive specific phrasings of the prompt (Elazar et al., 2021; Petroni et al., 2020), and it remains difficult to catalog, add, or update knowledge (AlKhamissi et al., 2022). Nevertheless, LLMs are promising because they scale well and are unconstrained by fixed schema (Safavi & Koutra, 2021). In this paper, we take on the update problem, asking how the implicit knowledge encoded within model parameters can be mass-edited.
Hypernetwork knowledge editors. Several meta-learning methods have been proposed to edit knowledge in a model. Sinitsin et al. (2019) proposes training objective to produce models amenable to editing by gradient descent. De Cao et al. (2021) proposes Knowledge Editor (KE) hypernetwork that edits a standard model by predicting updates conditioned on new factual statements. In a study of KE, Hase et al. (2021) find that it fails to scale beyond few edits, and they scale improved objective to beliefs. MEND (Mitchell et al., 2021) also adopts meta-learning, inferring weight updates from the gradient of the inserted fact. To scale their method, Mitchell et al. (2022) proposes SERAC, a system that routes rewritten facts through a different set of parameters while keeping the original weights unmodified; they demonstrate scaling up 75 edits. Rather than meta-leamiigg our method employs direct parameter updates based on an explicitly computed mapping.
Direct model editing. Our work most directly builds upon efforts to localize and understand the internal mechanisms within LLMS (Elhage et al., 2021; Dar et al., 2022). Based on observations from Geva et al. (2021; 2022) that transformer MLP layers serve as key-value memories. we narrow our focus to them. We then employ causal mediation analysis (Pearl, 2001; Vig et al., 2020; Meng et al., 2022), which implicates a specific range of layers in recalling factual knowledge. Previously, Dai et al. (2022) and Yao et al. (2022) have proposed editing methods that alter sparse sets of neurons, but we adopt the classical view of a linear layer as an associative memory (Anderson, 1972; Kohonen, 1972). Our method is closely related to Meng et al. (2022), which also updates GPT as an explicit associative memory. Unlike the single-edit approach taken in that work, we modify a sequence of layers and develop a way for thousands of modifications to be performed simultancously.
PRELIMINARIES: LANGUAGE MODELING AND MEMORY EDITING
The goal of MEMIT is to modify factual associations stored in the parameters of an autoregressive L.M. Such models generate text by iteratively sampling from conditional token distribution
2

---

Published as a conference paper at ICLR 2023
P [1 Pp].... .... mijl parameterized by D-layer transformer decoder, G' (Vaswani et al., 2017):
P [dnt]] "[ "1 ((==))-----11 softmax (Wy
where h D is the transformer's s hidden state representation at the final layer D and ending token E. Rig This state computed using the following recursive relation:
(x((x) Moj (x) + (/)(2) + mmgg(z)
where a' = attn' (N'T) Mal ....., 11T1))
(3)
mig = W out (Wir? (M()))
(4)
16 (a:) is the embedding of token 2[1], and y is layernorm. Note that we have written attention and MLPs in parallel done in Black et al. (2021) and Wang & Komatsuzaki (2021).
Large language models have been observed to contain many memorized facts (Petroni et al., 2020; Brown et al., 2020; Jiang et al., 2020; Chowdhery al., 2022). In this paper, study facts of the form (subject 8, relation r. object o), e.g. (s Michael Jordan, = plays sport, o = basketball). A generator G can recall a memory for (8i, Fi, *) if form a natural language prompt pt = p(&,, ri) such as "Michael Jordan plays the sport of"" and predict the next token(s) representing Our goal is edit many memories at once. We formally define a list of edit requests as:
E = [(s,, ris o:) [ s.t. Pi, (s; - 8,) A (ra = r2) A (0, + oz).
(5)
The logical constraint ensures that there are no conflicting requests. For example, we can edit Michael Jordan to play "baseball", but then we exclude associating him with professional soccer.
What does mean to edit memory well? At a superficial level, a memory can be considered edited after the model assigns a higher probability to the statement "Michael Jordan plays the sport of baseball" than the original prediction (basketball); we say that such an update effective. Yet is important to also view the question in terms of generalization, specificity, and fluency:
To test for generalization, we can rephrase the question: "What is Michael Jordan's 's sport? What sport does he play professionally?" If the modification of G is superficial and overfitted to the specific memorized prompt, such predictions will fail to recall the edited memory. "baseball."
Conversely, to test for specificity, we can ask about similar subjects for which memories should not change: "What sport does Kobe Bryant play? What does Magic Johnson play?" These tests will fail if the updated G indiscriminately regurgitates "baseball" for subjects that were not edited.
When making changes to a model, we must also monitor fluency If the updated model generates disfluent text such as "baseball baseball baseball baseball," we should count that failure.
Achieving these goals is challenging, even for a few edits (Hase et al., 2021; Mitchell et al., 2022; Meng et al., 2022). We investigate whether they can be attained at the scale of thousands of edits.
4 METHOD
MEMIT inserts memories by updating transformer mechanisms that have recently been elucidated using causal mediation analysis (Meng et al., 2022). In GPT-2 XL, we found that there is sequence of critical MLP layers R that mediate factual association recall at the last subject token (Figure 2). MEMIT operates by (i) calculating the vector associations want the critical layers to remember, then (ii) storing a portion of the desired memories in each layer e R.
Throughout this paper, our focus will be on states representing the last subject token S of prompt p.. so we shall abbreviate hy = k(sg(P)) Similarly, m. and af denote m|s (p:) and ats] (p:).
4.1 IDENTIFYING THE CRITICAL PATH OF MLP LAYERS
Figure 3 shows the results of applying causal tracing to the larger GPT-J (6B) model; for implementa- tion details, see Appendix A. We measure the average indirect causal effect of each h on a sample of memory prompts pt, with either the Attention or ML.P modules for token S disabled. The results
3

---

Published as a conference paper at ICLR 2023
range of critical MLP layers
Michael
last subject
(a)
Jordan
token
W
Rd
W
now
key for sudject
menoriied valuer
Was stores
m. pairs minimicin::
plays
mlp module
medulo
vector state
non mediating components
mip critical
information moved attention
direct path
Figure 2: MEMIT modifies transformer parameters on the critical path of ML.P P-mediatee factual recall. We edit stored associations based on observed patterns causal mediation: (a) first, the early-layer attention modules gather subject names into vector representations at the last subject token S. (b) Then MLPs at layers I R read these encodings and add memories to the residual stream. (c) Those hidden states are read by attention produce the output. (d) MEMIT edits memories by storing vector associations in the critical MLPs.
confirm that GPT-J has concentration of mediating states hi; moreover, they highlight a mediating causal role for a range MLP modules, which can be seen as a large gap between the effect of single states (purple bars in Figure 3) and the effects with MLP severed (green bars); this gap diminishes after layer Unlike Meng al. (2022) who use this test to identify single edit layer, we select the whole range of critical MLP layers E R. For GPT-J, we have R {3, 4, 5, 7,8).
Given that a range of MLPs play a joint mediating role in recalling facts, we ask: what is the role of one MLP in stor- ing memory? Each token state in transformer is part the residual stream that all attention and MIP modules read from and write to (Elhage et al., 2021).
Causal effeet hidden states Attn or MLP modules severed
Average Indirect Liffect
10.0%
Effect of single state
Effect w/ Attn severed
Efffect w/ ML. severed
5.0%
Unrolling Eqn. 2 for h h (p:):
0.0%
L
10
15
20
25
4 = No
Σ 1=1
m
(6)
Layer at which hidden state is restored
Figure critical mediating role mid- mid-layer MLPs.
Eqn. 6 highlights that each individual MLP contributes by adding to the memory at h (Figure 2b), which is later read by last-token attention modules (Figure 2c). Therefore, when writing new memories into G, can spread the desired changes across all the critical layers m. for E R.
4.2 BATCH UPDATE FOR A SINGLE LINEAR ASSOCIATIVE MEMORY
In each individual layer 1, we wish optimal single-layer update that minimizes the squared eтror of memorized associations, assuming that the layer contains previously-stone (Eqn. 4, Figure 2) and analyze it as linear associative memory (Kohonen, 1972; Anderson, 1972) that associates set of input keys k; & k (encoding subjects) to corresponding memory values my m. (encoding memorized properties) with minimal squared error:
large batch of >> memories. This section derives an
memories that should be preserved. We denote Wo - W out
Wo argmin W
Wk: m;
1=1
we stack keys and memories as matrices Kg = [kg
hgg ... k.j and Mo [m] mg ... m2),
then Eqn. 7 can be optimized by solving the normal equation (Strang, 1993, Chapter 4):
WoK, KJ = M,KT"
Suppose that pre- pre-training sets a transformer MLP's weights to the optimal solution Wo as defined in Eqn. 8. Our goal is to update Wo with some small change A that produces matrix W, with
4

---

Published as a conference paper at ICLR 2023
All examined subject for Pr
each memory 4, find by optamizing Equ.
7
44-2
attnf-2
arm
attin
Win A-2
W
Wen
Wout
Whi
kf
Wout
(ll-a) Add d s.t. Vi: m|
2u- of
(ii-) Add - s.l. Vi: m/
m-mi
(II-c) Add s.t. Vi: my
For each layer I, apply updates using all towards
Eqn.
Re-collect layer activations
Re-collect layer activations
The MEMIT update. We first (i) replace h with the vector 2, and optimize Eqn. 16 that it conveys the new memory. Then, after all z; are calculated (ii) iteratively insert a fraction of the residuals for all z; over the range of critical MLP modules, executing each layer's update by applying Eqn. 14. Because changing one layer will affect activations of downstream modules, we recollect activations after each iteration.
Figure 4:
a set of additional associations. Unlike Meng et al. (2022), we cannot solve our problem with constraint that adds only single new association, so we define an expanded objective:
W1 argmin
Wk:
Wk.
(9)
*=++1
We solve Eqn. 9 by again applying the normal equation, now written in block form:
Wi [Ko [K) K1I" = [Mo [K][Ko K1)"
(10)
which expands to: (Wo - (KoK + KTKT) MoKK + +MK?
(11)
WoKoKy WoK, KT + AKoKT tK,KT MoKK +M,KT
(12)
subtracting Eqn. from Eqn. 12: A(KoK) + K2K?) -M, WoK,KT.
(13)
A succinct solution can be written by defining two additional quantities: Co & Koke a constant proportional to the uncentered covariance of the pre- pre-existing keys, and R & M1 - WoK,, the residual error of the new associations when evaluated on old weights Wo. Then Eqn. 13 can be simplified as:
A RKT (Co + KiKT)-'
(14)
Since pretraining is opaque, we do not have access to Ko or Mo. Fortunately, computing Eqn. 14 only requires an aggregate statistic Cy over the previously stored keys. We assume that the set of previously memorized keys can be modeled as random sample of inputs, that we can compute
Co À Eg [kk="]
(15)
by estimating E, [k*"] an uncentered covariance statistic collected using an empirical sample of vector inputs the layer. We must also select X, hyperparameter that balances the weighting of new v.s. old associations; a typical value À = 1.5 x 10
4.3 UPDATING MULTIPLE LAYERS
We now define the overall update algorithm (Figure 4). Inspired by the observation that robustness is improved when parameter change magnitudes are minimized (Zhu et al., 2020), we spread updates evenly over the range of mediating layers R. We define a target layer - max(R) at the end of the mediating layers, at which the new memories should be fully represented. Then, for each edit (s;, Fr. o,) E ε, we (i) compute hidden vector 2; to replace h! such that adding é 2, hí the hidden state at layer L and token T will completely convey the new memory. Finally, one layer at time, we (ii) modify the MLP at layer so that contributes an approximately-equu portion of the change 5; for each memory i.
(i) Computing 2,. For the ith memory, we first compute a vector z; that would encode the association (st, Pt, o:) if it were to replace h. at layer L. at token S. We find 2 h + & by optimizing the residual vector ó, using gradient descent:
P
2, = 1 + argmin
Σ
log Pa(f++=6.) (os xy @ p(s., r:)]
(16)
P
6.
1=1
5

---

Published as a conference paper at ICLR 2023
In words, optimize 6; to maximize the model's prediction of the desired object os, given set of factual prompts {a, @ p(s;, r()} that concatenate random prefixes 2j to templated prompt to aid generalization across contexts. G( += 6,) indicates that we modify the transformer execution by substituting the modified hidden state 2; for h this called "hooking" in popular MI. libraries.
(ii) Spreading 4 h over layers. We seek delta matrices A such that:
setting W == Wont + A' for all E R optimizes min Σ )AA)
(17)
where ht = he
Σ W out a (W ? (N)
(18)
Because edits to any layer will influence all following layers' activations, we calculate A' iteratively in ascending layer order (Figure 4ii-a,b.c). To compute each individual A', we need the corresponding keys K' [K .... kj and memories M' = =mm ... m,,l to insert using Eqn. 14. Each key k computed as the input to W out at each layer (Figure 2d):
k = P
k(æs + si), where k(x) = (W 7 (x)))
(19)
1=1
mf is then computed as the sum of its current value and fraction of the remaining top-level residual:
24 10 T, T, 1-1+1
my Wonkk r where rf is the residual given by
(20)
where the denominator of r. spreads the residual out evenly. Algorithm summarizes MEMIT, and additional implementation details are offered in Appendix B.
Algorithm 1: The MEMIT Algorithm
Data: Requested edits & ry generator G, layers to edit S, covariances C* Result: Modified generator containing edits from E
for st, Oj E & do
// Compute target z, vectors for every memory i optimize 6, f- argming, 1 E, log Pongoo +-8)+-8) (o, x. @ p(s),)]] (Eqn. 16)
2, b +
end
for
1
R do
If Perform update: spread changes over layers
1d <- h - of + m² (Eqn. 2)
If Run layer with updated weights
for s,,7; 0 E do
if + k p E k(2, sc) (Eqn. 19)
of + --= (Eqn. 20)
#f Distribute residual over remaining layers
10
end
11
Kd <- (k..,...kf
Rt <- [r!...., r
12
A't - R'KeT (C+ K²K (Eqn. 14)
44
Wi -- w A
// Update layer MLP weights model
end
EXPERIMENTS
5.1
MODELS AND BASELINES
We run experiments two autoregressive LLMs: GPT-J (6B) and GPT-NeoX (20B). For baselines. we first compare with a naive fine-tuning approach that uses weight decay to prevent forgetfulness (FT-W). Next, we experiment with MEND, hypernetwork-baeed model editing approach that edits multiple facts at the same time (Mitchell et al., 2021). Finally, we run a sequential version ROME (Meng et al., 2022): direct model editing method that iteratively updates one fact at time. The recent SERAC model editor (Mitchell et al., 2022) does not yet have public code, so we cannot compare with it this time. See Appendix B for implementation details.

---

Published as a conference paper at ICLR 2023
5.2 MEMIT SCALING
5.2. EDITING 10K MEMORIES IN ZSRE
Table 1: 10,000 zsRE Edits on GPT-J (6B).
We first test MEMIT on zsRE (Levy et al., 2017), question -anweerng task from which we extract 10,000 real-world facts; zsRE tests MEMIT's ability to add correct information. Be- cause zsRE does not contain gener- ation tasks, we evaluate solely on prediction-based metrics. Efficacy measures the proportion of cases where o is the argmax generation given p(s, r), Paraphrase is the same metric but applied on paraphrases, Specificity is the model's argmax accuracy on a randomly- unrelated fact that should not have changed, and Score is the harmonic mean of the three aforementioned scores; Appendix C contains formal definitions. As Table I shows, MEMIT performs best at 10,000 edits; most memories are recalled with generalization and minimal bleedover. Interestingly, simple fine-tuning FT-W performs better than the baseline knowledge editing methods MEND and ROME at this scale, likely because its objective is applied only once.
Editor
Score
Elficacy 1
Paraphrase t
Specificityy t
GPT-J
26.4
26.4 (#006)
25.8 (#0.5)
27.0 (+0.5)
FTW
42.1
69.6 (#006)
64.8 (=0.6)
24.1 (#0.5)
MEND
20.0
19.4 (#0055
18.6 (=555)
22.4 (=0.5)
ROME
2.6
21.0 (1:0.7)
19.6 (=77.)
0.9 (#10.))
MEMIT
50.7
96.7 (2:0.3
89.7 (=5.5)
26.6 (+0.5)
5.2.2 COUNTERFACT SCALING CURVES
Next, we test MEMIT's ability add counterfactual information using COUNTERFACT, a col- lection of 21,919 factual statements (Meng et al. (2022), Appendix C). We first filter con- flicts by removing facts that violate the logical condition in Eqn. 5 (i.e., multiple edits modify the same (s, r) prefix to different objects). For each problem size n E (1,2, 3, 6, 10, 18, 32, 56, 100, 178, 316, 562, 1000, 1778, 3162, 5623, 10000) ' n counterfactuals are inserted.
Following Meng et al. (2022), we report several metrics designed to test editing desiderata. Efficacy Success (ES) evaluates editing success and is the proportion of cases for which the new object o,'s 's probability is greater than the probability of the true real-world object of?² E, [P& fo: p(s).r:) > Pg [of p(s;, r:)]]. Paraphrase Success (PS) is generalization measure defined similarly, except G is prompted with rephrasings of the original statement. For testing specificity, Neighborhood Success (NS) is defined similarly, but we check the probability G assigns to the correct answer of (instead of o.), given prompts about distinct but semantically-relate subjects (instead of s;). Editing Score (S) aggregates metrics by taking the harmonic mean of ES, PS, NS.
We are also interested in measuring generation quality of the updated model. First, check that G's generations are semantically consistent with the new object using a Reference Score (RS), which is collected by generating text about s and checking its TF-IDF similarity with reference Wikipedia text about o. To test for fluency degradation due to excessive repetition, we measure Generation Entropy (GE), computed as the weighted sum of the entropy of bi- and tri-gram n-gram distributions of the generated text. See Appendix C for further details on metrics.
Figure 5 plots performance v.s. number of edits on log scale, up to 10,000 facts. ROME performs well up to n = 10 but degrades starting at n = 32. Similarly, MEND performs well at n = but rapidly declines at n = 6, losing all efficacy before n 1,000 and, curiously, having negligible effect on the model at n = 10,000 (the high specificity score is achieved by leaving the model nearly unchanged). MEMIT performs best at large n. At small n. ROME achieves better generalization the cost of slightly lower specificity, which means that ROME'S edits are more robust under rephrasings, likely due to that method's hard equality constraint for weight updates, compared to MEMIT's soft error minimization. Table 2 provides direct numerical comparison at 10,000 edits on both GPT-J and GPT-NcoX. FT-W3 does well on probability- metrics but suffers from complete generation failure, indicating significant model damage.
Appendix B provides a runtime analysis of all four methods on 10,000 edits. We find that MEND is fastest, taking 98 sec. FT is second at around 29 min, while MEMIT and ROME are the slowest at
'These values come from log-scale curve: exp (Im(10.000) ) for non-negative integers i. *CoUNteR FACT is derived from a set of true facts from WikiData, so of always known.
We find that the weight decay hyperparameter is highly sensitive to the number of edits. Therefore, to evaluate scaling hehavior cost-efficiently, we tune only on 10.000. See Appendix B. for experimental details.
7

---

Published as a conference paper at ICLR 2023
Score (CS)
Efficacy Success (ES)
Generalization Success (PS)
100
100
80
60
60
40
20
20
10
10
10
10
10
10'
10*
Number of Edits
Number of Edits
Number of Edits
Specificity Success (NS)
Consistency
Fluency (GE)
30-
550
60
20
500
450
10
10
10"
10"
10
Namber Edits
Numbar of Edits
Number of Edits
MEND
ROME
MEMIT
GPTJ Unedited
Figure 5: MEMIT scaling curves plot editing performance against problem size (log-scale). The dotted line indicates GPT-J's pre-edit performance; specificity (NS) and fluency (GE) should stay close the baseline. 95% confidence intervals are shown as areas.
Table 2: Numerical results on COUNTERFACT 10,000 edits.
Score
Efficacy
Generalization
Specificity
Fluency
Consistency
Editor
S f
ES f
PS f
NS
GE f
RS f
GPT-J
22.4
15.2 (0.7)
17.7 (0.6)
83.5 (0.5)
622.4 (0.3)
29.4 (0.2)
FT-W
67.6
99.4 (0.1)
(0.7)
46.9 (0.6)
293.9 (2.4)
15.9 (0.3)
MEND
23.1
15.7 (0.7)
18.5 (0.7)
83.0 (0.5)
618.4 (0.3)
31.1 (0.2)
ROME
50.3
50.2 (1.0)
50.4 (0.8)
50.2 (0.6)
589.6 (0.5)
3.3 (0.0)
MEMIT
85.8
98.9 (0.2)
88.6 (0.5)
73.7 (0.5)
619.9 (0.3)
40.1 (0.2)
GPT-NeoX
23.7
16.8 (1.9)
18.3 (1.7)
81.6 (1.3)
620.4 (0.6)
29.3 (0.5)
MEMIT
82.0
97.2 (0.8)
82.2 (1.6)
70.8 (1.4)
606.4 (1.0)
36.9 (0.6)
7.44 hr and 12.29 hr, respectively. While MEMIT's execution time is high relative to MEND and FT, we note that its current implementation is naive and does not batch the independent 2; optimizations, instead computing each one in series. These computations are actually "embarrassingly parallel" and thus could be batched.
5.3 EDITING DIFFERENT CATEGORIES OF FACTS
For insight into MEMIT's performance on different types of facts, we pick the 27 categories from COUNTERFACT that have least 300 cases each, and assess each algorithm's 's performance on those cases. Figure 6a shows that MEMIT achieves better overall scores compared to FT and MEND in all categories. also reveals that some relations are harder to edit compared to others; for example, each of the editing algorithms faced difficulties in changing the sport athlete plays. Even on harder cases, MEMIT outperforms other methods by a clear margin.
Model editing methods are known to occasionally suffer from trade-off between attaining high generalization and good specificity. This trade-off is clearly visible for MEND Figure 6b. FT consistently fails to achieve good specificity. Overall, MEMIT achieves higher score in both dimensions, although also exhibits a trade-off editing some relations such as P127 ("product owned by company") and P641 ("athlete plays sport").
8

---

Published as a conference paper at ICLR 2023
bmgggge [103)]
cauzen country [P22]
100
heafrgurseed [P159]
lanquaze speike [P1412]
location PM17)
Specificity Successs (N5)
90-
lxcesd in countic [P12
P1278
died locaton (P20)
lmnwage
woreo occepption (1066
otficial
show ungirally (PS40)
plays pusition spon [M13]
P12788
orgaa
vorks for (P108] developed by
pecdace
P1276
P80g
was [P940]
50-
specidizes in Bold P101)
follww
dae yeur (P136]
40
P041e
plays Insmrmen (P1303)
boM's
mwned company [P177]
130
locseed in continere (P30]
K00
plass-qum spart [P641]
20
60
80
100
MEMIT
MEND
Scores (S)
Generalization Sucuess (PS)
(a)
(b)
Figure 6: (a) Category-wise rewrite scores achieved hy different approaches in editing 300 similar facts. (b)
Category- wise specificity generulization scores by different approaches on 300 edits.
(a) Subject different, Object different
(b) Subject similar, Object different
(c) Subject differeni, Object similar
(d) Subject similar, Object similar
55-
Scone (S)
50
55-
80-
30-
[413,
68-
127
100
23412
P450
212
200
100
200
500
400
503
600
700
100
200
500
600
700
100
230
300
500
600
100
100
200
300
403
500
600
700
Number of ecttss
Number editss
Number of edits
Number odits
Figure 7: When comparing mixes of edits, MEMIT gives consistent near-linear (near-average) performance while scaling up to 700 facts.
5.4 EDITING DIFFERENT CATEGORIES OF FACTS TOGETHER
To investigate whether the scaling of MEMIT is sensitive to differences in the diversity of the memories being edited together, we sample sets of cases Emis that mix two different relations from the COUNTERFACT dataset. We consider four scenarios depicted in Figure 7, where the relations have similar or different classes of subjects or objects. In all of the four cases, MEMIT's performance on Emir. is close to the average of the performance of each relation without mixing. This provides support to the hypothesis that the scaling of MEMIT is neither positively nor negatively affected by the diversity of the memories being edited. Appendix D contains implementation details.
6 DISCUSSION AND CONCLUSION
We have developed MEMIT, a method for editing factual memories in large language models by directly manipulating specific layer parameters. Our method scales to much larger sets edits (100x) than other approaches while maintaining excellent specificity, generalization, and fluency.
Our investigation also reveals some challenges: certain relations more difficult edit with robust specificity, yet even on challenging cases we find that MEMIT outperforms other methods by clear margin. The knowledge representation we study is also limited in scope working with directional (8, (r,o) o) relations: it does not cover spatial temporal reasoning, mathematical knowledge, linguistic knowledge, procedural knowledge, or even symmetric relations. For example, the association that "Tim Cook is CEO of Apple" must be processed separately from the opposite association that "The CEO of Apple is Tim Cook."
Despite these limitations, it is noteworthy that large-scale model updates can be constructed using an explicit analysis internal computations. Our results raise a question: might interpretabillty-bass methods become commonplace alternative to traditional opaque fine-tuning approaches? Our positive experience brings us optimism that further improvements to our understanding of network internals will lead more transparent and practical ways to edit, control, and audit models.

---

Published as a conference paper at ICLR 2023
7 ETHICAL CONSIDERATIONS
Although test language model's ability models to be a reliable source of knowledge, and we caution readers that a LLM should not be used as authoritative source of facts. Our memory-editing methods shed light the internal mechanisms of models and potentially reduce the cost and energy needed to fix errors in model, but the same methods might also enable a malicious actor to insert false or damaging information into model that was not originally present in the training data.
knowledge base, do not find these
ACKNOWLEDGEMENTIS
Thanks to Jaden Mooto-Kaaffmmn for building the demonstration at baulaabbu This project was supported by AI Alignment grant from Open Philanthropy. YB was also supported by the Israel Science Foundation (grant No. 448/20) and an Azrieli Foundation Early Career Faculty Fellowship.
9 REPRODUCIBILITY
The code and data for methods and experiments are available nemit.baulab.inte
All experiments are run on workstations with NVIDIA A6000 GPUs. The language models are loaded using HuggingFace Transformers (Wolf al., 2019), and PyTorch (Paszke et al., 2019) is used for executing the model editing algorithms on GPUs.
GPT-J experiments fit into one 48GB A6000, but GPT-NeoX runs require least two: one 48GB GPU for running the model in floatl. 6, and another slightly smaller GPU for executing the editing method. Due the size of these language models, our experiments will not run on GPUs with less memory.
REFERENCES
Oshin Agarwal and Ani Nenkova. Temporal effects on pre-trained models for language processing tasks. Transactions of the Association for Computational Linguistics, 10:904-921, 2022
Badr AlKhamissi, Millicent Li, Asli Celikyilmaz, Mona Diab, and Marjan Ghazvininejad. A review on language models as knowledge bases. arXiv preprint arXiv:2204.0603), 2022.
James A Anderson. A simple neural network generating an interactive memory. Mathematical biosciences, (4(3-4):197-22)) 1972.
Sören Auer, Christian Bizer, Georgi Kobilarov, Jens Lehmann, Richard Cyganiak, and Zachary Ives. Dbpedia: nucleus for web of open data. In The semantic web, pp. 722-735. Springer, 2007.
Sid Black, Leo Gao, Phil Wang, Connor Leahy, and Stella Biderman. GPT-Neo: Large Scale Autoregressive Language Modeling with Mesh-Tensorflow, March 2021. URL https://doi. org/10. 5281//2111/// zenodo. 5297715.
Sid Black, Stella Biderman, Eric Hallahan, Quentin Anthony, Leo Gao, Laurence Golding, Horace He, Connor Leahy, Kyle McDonell, Jason Phang, Michael Pieler, USVSN Sai Prashanth, Shivanshu Purohit, Laria Reynolds, Jonathan Tow, Ben Wang, and Samuel Weinbach. Gpt-neox-20b: An open-source autoregressive language model, 2022.
Kurt Bollacker, Robert Cook, and Patrick Tufts. Freebase: A shared database of structured general human knowledge. In AAAI, volume 7, pp. 1962-1963, 2007.
Antoine Bosselut, Hannah Rashkin, Maarten Sap, Chaitanya Malaviya, Asli Celikyilmaz, and Yejin Choi. Comet: Commonsense transformers for automatic knowledge graph construction. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp.
4762-4779, 2019.
10

---

Published as a conference paper at ICLR 2023
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert- Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz. Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. Language models are few-shot learners. In H. Larochelle, M. Ranzato, R. Hadsell, M. Balcan, and H. Lin (eds.), Advances in Neural Information Processing Systems, 2020.
volume 33, pp.
Andrew Carlson, Justin Betteridge, Bryan Kisiel, Burr Settles, Estevam R Hruschka, and Tom M Mitchell. Toward an architecture for never-ending language learning. In Twenty-Fourth AAAI conference on artificial intelligence, 2010.
Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung. Charles Sutton, Sebastian Gehrmann, et al. Palm: Scaling language modeling with pathways. arXiv preprint arXiv:2204.02.0 2022.
Damai Dai, Li Dong, Yaru Hao, Zhifang Sui, Baobao Chang, and Furu Wei. Knowledge neurons in pretrained transformers. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume I: Long Papers), pp. 8493-8502, 2022.
Guy Dar, Mor Geva, Ankit Gupta, and Jonathan Berant. Analyzing transformers in embedding space. arXiv preprint SttXiv:2000.02533. 2022.
Randall Davis, Howard Shrobe, and Peter Szolovits. What is a knowledge representation? AI magazine, 14(1):17-17, 1993.
Nicola De Cao, Wilker Aziz, and Ivan Titov. Editing factual knowledge language models. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pp. 6491-6506, Online and Punta Cana, Dominican Republic, November 2021. Association for Computational Linguistics.
Xin Dong, Evgeniy Gabrilovich, Geremy Heitz, Wilko Horn, Ni Lao, Kevin Murphy, Thomas Strohmann, Shaohua Sun, and Wei Zhang. Knowledge vault: A web-scale approach to proba- bilistic knowledge fusion. In Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining, pp. 601-610, 2014.
Yanai Elazar, Nora Kassner, Shauli Ravfogel, Abhilasha Ravichander, Eduard Hovy, Hinrich Schütze. and Yoav Goldberg. Measuring and improving consistency in pretrained language models. Trans- actions of the Association for Computational Linguistics, 9:1012- 1031, 2021.
Nelson Elhage, Neel Nanda, Catherine Olsson, Tom Henighan, Nicholas Joseph, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, Tom Conerly, Nova DasSarma, Dawn Drain, Deep Ganguli, Zac Hatfield- Dodds, Danny Hernandez, Andy Jones, Jackson Kernion, Liane Lovitt, Kamal Ndousse, Dario Amodei, Tom Brown, Jack Clark, Jared Kaplan, Sam McCandlish, and Chris Olah. A mathematical framework for transformer circuits. Transformer Circuits Thread, 2021. https:/transformerccicu
Mor Geva, Roei Schuster, Jonathan Berant, and Omer Levy. Transformer feed-forward layers are key-value memories. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pp. 5484-5495, 2021.
Mor Geva, Avi Caciularu, Kevin Ro Wang, and Yoav Goldberg. Transformer feed-forward layers build predictions by promoting concepts the vocabulary space. arXiv preprint arXiv:2203. 14680, 2022.
Peter Hase, Mona Diab, Asli Celikyilmaz, Xian Li, Zornitsa Kozareva, Veselin Stoyanov, Mohit Bansal, and Srinivasan Iyer. Do language models have beliefs? methods for detecting, updating, and visualizing model beliefs. arXiv preprint arXiv:2111.13654 2021.
Catherine Havasi, Robert Speer, and Jason Alonso. Conceptnet: A lexical resource for common sense knowledge. Recent advances in natural language processing V: selected papers from RANLP, 309: 269, 2007.
11

---

Published as a conference paper at ICLR 2023
Zhengbao Jiang, Frank F Xu, Jun Araki, and Graham Neubig. How know what language models know? Transactions of the Association for Computational Linguistics, 8:423 438, 2020.
Teuvo Kohonen. Correlation matrix memories. IEEE transactions on computers, 100(4):353-33)) 1972.
Angeliki Lazaridou, Adhi Kuncoro, Elena Gribovskaya, Devang Agrawal, Adam Liska, Tayfun Terzi, Mai Gimenez. Cyprien de Masson Autume, Tomas Koeisky, Sebastian Ruder, al. Mind the gap: Assessing temporal generalization in neural language models. Advances in Neural Information Processing Systems, 01-29348-29366 2021.
Douglas B Lenat. Cyc: A large-scale investment in knowledge infrastructure. Communications of the ACM, 38(11):33-38, 1995.
Omer Levy, Minjoon Seo, Eunsol Choi, and Luke Zettlemoyer. Zero-shot relation extraction via reading comprehension. In Proceedings of the 21 Conference on Computational Natural Language Learning (CoNLL 2017), pp. 333 333-342, 2017.
Adam Liska, Tomas Kocisky, Elena Gribovskaya, Tayfun Terzi, Eren Sezener, Devang Agrawal, D' Autume Cyprien De Masson, Tim Scholtes, Manzil Zaheer, Susannah Young, et al. Stream- ingQA: A benchmark for adaptation to new knowledge over time in question answering models. In International Conference on Machine Learning, pp. 13604-13622, PMLR, 2022.
Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. Locating and editing factual associations in GPT. Advances in Neural Information Processing Systems, 35, 2022.
George A Miller. Wordnet: a lexical database for english. Communications of the ACM, 38(11): 39-41, 1995.
Marvin Minsky. framework for representing knowledge, 1974.
Eric Mitchell, Charles Lin, Antoine Bosselut, Chelsea Finn, and Christopher D. Manning. Fast model editing scale, 2021.
Eric Mitchell, Charles Lin, Antoine Bosselut, Chelsea Finn, and Christopher D. Manning. Memory- based model editing scale. In International Conference Machine Learning, 2022.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, pegfrreeformacce deep learning library. Advances in neural information processing systems, 32, 2019.
David Patterson, Joseph Gonzalez, Quoc Le, Chen Liang. Lluis-Miquel Munguia, Daniel Rothchild, David So, Maud Texier, and Jeff Dean. Carbon emissions and large neural network training. arXiv preprint arXiv:2 104. 10350, 2021.
Judea Pearl. Direct and indirect effects. In Proceedings of the Seventeenth conference on Uncertainty in artificial intelligence, pp. 411-420 2001.
Fabio Petroni, Tim Rocktäschel, Sebastian Riedel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, and Alexander Miller. Language models as knowledge bases? In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference Natural Language Processing (EMNI.P-JCCLLP) pp. 2463-2473, 2019.
Fabio Petroni, Patrick Lewis, Aleksandra Piktus, Tim Rocktäschel, Yuxiang Wu, Alexander H Miller, and Sebastian Riedel. How context affects language models' factual predictions. In Automated Knowledge Base Construction, 2020.
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, llya Sutskever, et al. Language models are unsupervised multitask learners. OpenAl blog, pp. 9. 2019.
Richard H Richens. Preprogramming for mechanical translation. Mechanical Translation, 3(1): 20-25, 1956.
12

---

Published as a conference paper at ICLR 2023
Adam Roberts, Colin Raffel, and Noam Shazeer. How much knowledge you pack into the parameters of language model? In Proceedings of the 2020 Conference on Empirical Methods in 2020.
Natural Language Processing (EMNLP), pp.
Tara Safavi and Danai Koutra. Relational world knowledge representation in contextual language models: A review. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pp. 1053-1067, 2021.
Taylor Shin, Yasaman Razeghi, Robert Logan IV, Eric Wallace, and Sameer Singh. Autoprompt: Eliciting knowledge from language models with automatically generated prompts. In Proceedings of the 2020 Conference on Empirical Methods Natural Language Processing (EMNLP), pp- 4222 4235, 2020.
Anton Sinitsin, Vsevolod Plokhotyyuk, Dmitry Pyrkin, Sergei Popov, and Artem Babenko. Editable neural networks. In International Conference on Learning Representations, 2019.
Gilbert Strang, Introduction to linear algebra. Wellesley-Cambridge Press Wellesley, MA, 1993.
Fabian M Suchanek, Gjergji Kasneci, and Gerhard Weikum. Yago: core of semantic knowledge. In Proceedings of the 16th international conference on World Wide Web, pp. 697-706, 2007.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in neural information processing systems, pp. 5998-6008, 2017.
Jesse Vig, Sebastian Gehrmann, Yonatan Belinkov, Sharon Qian, Daniel Nevo, Yaron Singer, and Stuart M Shieber. Investigating gender bias in language models using causal mediation analysis. In NeurlPS, 2020.
Denny Vrandečić and Markus Krötzsch. Wikidata: free collaborative knowledgebase. Communica- tions of the ACM, 57(10))7--85, 2014.
Ben Wang and Aran Komatsuzaki. GPT-J-6B: A 6 Billion Parameter Autoregressive Language Model. www./oo.iinnoo /ttts://gitiin wwt.:icoo//oioo///cecccoo htw.lo/////eesttaassssoooomom jax, May 2021.
Gerhard Weikum. Knowledge graphs 2021: a data odyssey. Proceedings of the VLDB Endowment. 14(12):3233-3238 2021.
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, et al. Huggingface's transformers: State-of-ther the--rrttt-ttttt natural language processing. arXiv preprint arXiv:1910.03771,, 2019.
Yunzhi Yao, Shaohan Huang, Li Dong. Furu Wei, Huajun Chen, and Ningyu Zhang, Kformer: Knowledge injection in transformer feed-forward layers. In CCF International Conference on Natural Language Processing and Chinese Computing, pp. 131-143. Springer, 2022.
Chen Zhu, Ankit Singh Rawat, Manzil Zaheer, Srinadh Bhojanapalli, Daliang Li, Felix Yu, and Sanjiv Kumar. Modifying memories in transformer models, 2020.
13

---

Published as a conference paper at ICLR 2023
A
CAUSAL TRACING
(8)
Asg Indirett Effict of orai 301 prompi
(b)
Elfeet of
prompt
(c)
Bifles of
prompte
Fist suljer tolor
Fisr vdjoe lolon
0.22
Viddi Villferraaeeee solow
0.15
Lisl ieloe
wdjod
Maltle sujert
Las poloddec inkee
Fime anteqacciioe ioice
Firm ruvepper
C.16
3.19
Putterinonns
Punbortsiene
Lavinics
AEE
0.00
Lacokee
AB
3.03
of of paicaed
Figure 8: Causal Tracing (using the method of Meng et al. 2022). Each grid cell's intensity reflects the average causal indirect effect of a hidden state on the expression of factual association, with strong causal mediators highlighted with darker colors. We find that MLPs the last subject token and attention modules at the last token important. The presence of influential attention activations at the earliest layers of the last subject token investigated with additional path dependent experiments (Figure 3).
MEMIT begins by identifying MLP layers that are causal mediators for recall of factual associations in the model. To do so in GPT-J, we use code provided by Meng et al. (2022): beginning with a sample of 501 true statements of facts that are correctly predicted by GPT-J, we measure baseline predicted probabilities of each true fact when noise is introduced into encoding of the subject tokens to degrade the accuracy of the model. Then in Figure 8 (a) for each individual hi, we restore the state to the value that it would have had without injected noise, and we plot the average improvement of predicted probability. As in Meng et al. (2022), we use Gaussian noise with standard deviation 30 (o² is the empirically observed variance of embedding activations) and plot averages for all 501 statements over 10 noise samples. For (b) and (c) we use the same procedure, except we restore runs of 10 layers of MLP outputs m. and 10 layers of Attn af, instead of full hidden states.
These measurements confirm that GPT-J has a causal structure that is similar the structure reported by Meng et al. (2022) in their study of GPT2-XL -L. Unlike with GPT-XL, a strong causal effect is observed in the earliest layers of Attention at the last subject token, which likely reflects a concentrated attention computation when GPT-J is recognizing and chunking the n-gram subject name, but the path-dependent experiment (Figure 3) suggests that Attention not an important mediator of factual recall of memories about the subject.
In the main paper, Figure 3 plots the same data as Figure (a) as bar graph, focused on only the last subject token, and it adds two additional measurements. In red bars, it repeats the measurement of causal effects of states with Attention modules at the last subject token frozen in the corrupted state, so that cannot be influenced by the state being probed, and in green bars it repeats the experiment with the MLP modules at the last subject token similarly frozen, so they cannot be influenced by the causal probe. Severing the Attention modules does not shift the curve, which suggests that Attention computations do not play a decisive mediating role in knowledge recall at the last subject token. In contrast, severing the MLP modules reveals large gap, which suggests that, at layers where the gap is largest, the role of the MLP computation is important. We select the layers where the gap is largest as the range R to use for the intervention done by MEMIT.
B IMPLEMENTATION DETAILS
B. FINE-TUNING WITH WEIGHT DECAY
Our fine-tuning baseline updates layer 21 of GPT-J, which Meng et al. (2022) found to provide the best performance in the single-edit case. Rather than using a hard -oom constraint, we use soft weight decay regularizer. However, the optimal amount of regularization depends strongly on the number of edits (more edits require higher-norm edits), so we tune this hyperparameter for the n = 10, 10,000 case. Figure 9 shows that x 10 selects for the optimal tradeoff between generalization and specificity. FT-W optimization proceeds for maximum of 25 steps with a learning rate of x 10 4, To prevent overfitting, early stopping is performed when the loss reaches 10 2, Regarding runtime, FT takes 1. 1,716.21 sec & 0.48 hr execute 10,000 edits on GPT-J.
14

---

Published as a conference paper at ICLR 2023
Score (CS) T
Efficacy Success (ES) T
Generalization Success (PS)
Specificity Success (NS)
100
80
60
50
60
40
40
40
25
10
10"
10"
10
10"
10
10"
Weight Decay
Weight Decay
Weight Decay
Weight Decay
Figure 9: Optimizing fine-tuning weight decay 10,000 edits. We find evident tradeoff between gencralization and specificity, opting for the value with the highest Score.
Note that we choose not to complicate the analysis by tuning FT-W on more than one layer. Table 2 demonstrates that FT-W, with just one layer, already gets near-perfect efficacy the cost low specificity, which indicates sufficient edit capacity.
B.2 MODEL. EDITING NETWORKS WITH GRADIENT DECOMPOSITION (MEND)
MEND makes concurrent edits by accumulating gradients from all edit examples, then passing them through the hypernetwork together. We use the GPT-J MEND hypernetwork trained by Meng et al. (2022). During inference, learning rate scale the default value of 1.0. MEND is by far the fastest method, taking 98.20 seconds to execute 10,000 updates on GPT-J.
B.3 RANK-ONE MODEL EDITING (ROME)
The default ROME hyperparameters available in their open source code: GPT-J updates executed at layer 5, where optimization proceeds for 20 steps with a weight decay of 0.5, KL factor of 0.0625, and learning length 5 and 10 prefixes of length 10. Covariance statistics are collected in fp32 on Wikitext using sample size of 100,000. See Meng et al. (2022) for more details.
of 5 x 10-1, ROME prefix sampling, resulting in 10 prefixes of
ROME takes 44,248 24.248.26ss sec x= 12.29 hr for 10,000 edits on GPT-J, which works out to approximately 4 seconds per edit.
B.4 MASS-EDITING MEMORY IN TRANSFORMER (MEMIT)
On GPT-J, we choose R = (3, 4, 5, 6, 7, 7,8) and set X, the covariance adjustment factor, to 15,000. Similar ROME, covariance statisties are collected using 100,000 samples of Wikitext in fp32. ò, optimization proceeds for 25 steps with a learning rate of 5 x 10-1 In practice, we clamp the L2 norm of 6, such that it less than & of the original hidden state norm, || On GPT-NeoX, we select R {6. 7, 8, 9, 10} and set À = 20,000. Covariance statistics are collected over 50,000 samples of Wikitext fp16 but stored in fp32. Optimization for ó, proceeds for steps using learning rate of 5 10-1 while clamping ||4+1 to I|
In MEMIT, we have the luxury being able to pre-compute and cache 2; values, since they are inserted in parallel. If all such vectors are already computed, MEMIT takes 3, 2,,226.35so sec z 0.90 hr for 10,000 updates on GPT-J, where the most computationally expensive step is inverting a large square matrix (Eqn. 14), Computing each vector is slightly less expensive than computing a ROME update; to get all 10,000 2, vectors, we need 23,546.65 sec z 6.54 hr. This optimization is currently done in series, but it is actually "embarrassingly parallel," by batching the gradient descent steps. Note that this speed-up does not apply to ROME, since each update must be done iteratively.
greatly reduce computation time
15

---

Published as a conference paper at ICLR 2023
C EVALUATION METRICS
C. FOR ZSRE
For consistency with previous works that use the zsRE task (Mitchell et al., 2021; Meng et al., 2022),
we report the same three probability tests:
Efficacy is the proportion of edits that G recalls with top-1 accuracy. Note that the prompt matches exactly what the edit method sees at runtime:
E;
0j
arginax Pa (IE p((s.) mx)l
(21)
Paraphrase is the accuracy rephrasings of the original statement:
E
(ppparaphanss(s..))
0s argmax Pa [+ p)
(22)
Specificity is the proportion of neighborhood prompts that the model gets correct. In COUNTER- FACT, all such prompts have the same correct answer of:
E2
Eyencighborhood (s) of = argmax Pa ( p)
(23)
We also report an aggregated Score: the harmonic mean of Efficacy, Paraphrase, and Specificity.
C.2 FOR COUNTERFACT
COUNTERFACT contains an assortment of prompts and texts for evaluating model rewrites (Figure 14).
This section provides formal definitions for each COUNTERFACT metric. First, the probability tests:
Efficacy Success (ES) is the proportion of cases where exceeds of in probability. Note that the prompt matches exactly what the edit method sees at runtime:
E, P [ox p(s;, re)] > Pa [of p(s., re)].
(24)
Paraphrase Success (PS) is the proportion of cases where o, exceeds of in probability rephrasings of the original statement:
E, Ep pe.) Pa [o, p] > Pa lof pl]]
(25)
Neighborhood Success (NS) is the proportion of neighborhood prompts where the models assigns higher probability to the correct fact:
E2 pE meightbuuuod prompt((s;..r) [Pa [o: p] < IPG [o [p]]
(26)
Editing Score (S), is the harmonic mean of ES, PS, and NS.
Now, the generation tests:
Reference Score (RS) measures the consistency of G's free-form generations. To compute it, we first prompt G with the subject s, then compute TF-IDF vectors for both G(s) and reference Wikipedia text about o; RS is defined as their cosine similarity. Intuitively, G(s) will match better with o's reference text has more consistent phrasing and vocabulary.
We also check for excessive repetition (a common failure case with model editing) using Genera- tion Entropy (GE), which relies on the entropy of n-gram distributions:
2 3
Σ f2(k) logo (k) +
4 Σ fs(k loga (s(2))))) 3
(27)
k
Here, f.(+) is the n-gram frequency distribution.
16

---

Published as a conference paper at ICLR 2023
D EDITING DIFFERENT CATEGORIES OF FACTS TOGETHER
For an edit (8, o), associates a subject 8 and object o. Both & and o have their associated types T()) and To)) For example, r "is citizen of"" is an association between a Person and Countzy. We say that T(s1) and are diverse if T(s1) * (r(s2)), and similar otherwise. The definition follows similarly for objects. For any relation pair (r1, we sample from COUNTERFACT a set of edits Ema {(s, o)
(ri,r2] (2)) ), such that numbers of edits for each relation are equal. We compare MEMIT's performance on the set of edits Emiz four pairs of relations that have different levels of diversity between them. Each relation is followed by its corresponding relation_id _n in WikiData:
(a) Subject different (r(s1) # (s2)), Object different (r(01) * r(02)):
(T(8)) = Person, 71 citizen of (P27), T(0)) Country),
(r(s2) = Country, = official language (P37), ((02) ) = Language)
(b) Subject similar (T(s1) ( Object different (T(01) # T(02)):
(T(s1) = Person, 71 = plays position in sport (P413), ((11) = Sport position),
(r($2) = Person, native language (P1412), ((02) Language)
(c) Subject different (7($1) * T ($2 Object similar (0) 7(02)):
(T(81) = P Place = located in (P17), = Country),
(T(s2) Itee/
country of (rigin((495)) r(o2) Country)
(d) Subject similar (T(s1) (s2)). Object similar (r(o1) T(02)):
(T(s1) = Person, citizen of (P27), ((1) Country),
(7(52) = Person, (r(s))=Personn.
works in (P937), ((2) City/Country)
Figure depicts MEMIT rewrite performance in these four scenarios. We find that the effectiveness of Emir closely follows the average of the individual splits. Therefore, the presence of diversity in the edits (or lack thereof) does not tangibly influence MEMIT's performance.
E DEMONSTRATIONS
This section provides two case studies, in which we apply MEMIT to mass-edit new or corrected memories into GPT-J (6B).
Knowledge freshness. On November 8th, 2022, the United States held elections for 435 con- gressional seats, 36 governor seats, and senator seats, several of which changed hands. We applied MEMIT to incorporate the election results into GPT-J in the form of (congressperson, elected from, district) and (governor/ senator, elected from, state). The MEMIT edit attained 100% efficacy (ES) and 94% generalization (PS).
Application in a specialized knowldge domain. For a second application, we used MEMIT to create model with specialized knowledge of amateur astronomy. We scraped the names of stars that were referenced more than 100 times from WikiData and belong to one of the 18 constellations named below.
Andromeda, Hydra, Perseus,
Aquarius, Indus, Pisces,
Cancer, Leo, Sagittarius,
Cassiopeia, Libra, Ursa Major,
Gemini, Orion, Ursa Minor,
Hereules, Pegasus, Virgo
We obtained 289 tuples of the form (star, belongs to, constellations The accuracy of the unmodified GPT-J in recalling constellation of star was only 53%. Post-MEMIT, accuracy increased to 86%.
"The results available hefore November 14th.
17

---

Published as a conference paper at ICLR 2023
Score (S)
Efficacy Suce (ES)
Genernlization Suce (PS)
Speficity Success (NS)
99
95-
90-
01
93-
97-
70-
100
200
300
400
500
500
700
100
200
300
400
500
500
700
100
206
300
400
500
600
701
100
202
300
400
St0
600
700
Number edittit
Number of edita
Number of edits
Number of edits
P27
197
P2), P37
(a) Subject different, Object different
Score (S)
Ellicacy Succ (ES)
Generalization Suce (PS)
Speficity Success (NS)
05-
90
85
50-
85
95-
75
60-
100
210
300
400
san
900
700
100
210
300
400
san
100
700
300
300
4m
san
000
700
300
300
410
sto
cim
702
Number edeitt
Number of editis
Number edifs
Number edits
p011
P1412
(b) Subject similar, Object different
Score
Efficacy Suce (ES)
Genernlization Suce (PS)
Speficity Success (NS)
102
100
110
100
97-
70-
75-
80
300
700
05-
300
400
706
70
400 Number of edits
350
708
30
300
S00
Number of edits
Number of edits
Number of edits
(c) Subject different, Object similar
Score (S)
Eflicacy Suce (ES)
Generalization Suce (PS)
Speficity Success (NS)
95
92
50
50-
90
07-
70-
65-
100
710
740
400
003
700
100
100
340
400
Kin
000
700
100
300
700
400
500
700
120
300
300
410
520
cin
700
Number odiiits
Number of edits
Number of edits
Number of edits
+
P23
1917
P3T. P437
(d) Subject similar, Object similar
Figure 10: MEMIT's performance while editing memories with four levels of diversity. Each data point is a mean of 10 experiments. Filled areas show 90% confidence intervals of the values from those experiments.
18

---

Published as a conference paper at ICLR 2023
F ABLATIONS
MEMIT contains several critical design choices: uses (i) range of critical mid-layer (ii) MLP modules at the (iii) last subject token, with the (iv) hyperparameter À (Eqn. 15) to control the impact of the update. Choice (iii) was already demonstrated by Meng et al. (2022) to be significant through an ablation study, but now investigate the other three.
F.1 VARYING THE NUMBER AND LOCATION EDITED LAYERS
We test five total configurations of R. the set of critical MLP layers to be targeted during editing. Four are the region of high causal effect identified Figures 3, 8, whereas the other one in region of late MLPs that have low causal effect. As Figure 11 shows, using more layers yields higher efficacy and generalization while also improving specificity. Moreover, edits at the late-layer MLPs are considerably worse. These results confirm the importance of the causal analysis to MEMIT's performance.
Score (CS)
Efficacy Success (ES)
Generalization Suceess (PS) f
97.5
95.0
70
92.5
50-
8
15-220
5-8
8
15-220
6
8
15-20
15
15
15
Edited MLP Layers
Edited MLP Layers
Edited MLP Layers
Specificity Success (NS)
Consistency (RS)
Fluency (GE)
620
610-
35
5-8
8
15 150
5-88
8
15-220
5
8
15-220
15
15
Editod MLP Layers
Edited MLP Layers
Edited MLP Layers
Figure 11: Varying edited MLP layers
F.2 VARYING THE TARGETED MODULE: EDITING ATTENTION
Next, we check whether edits either early late-layer attention modules perform comparably to their MLP counterparts. As Figure 12 shows, attention edits perform considerably worse.
F.3 VARYING THE COVARIANCE HYPERPARAMETES À
Finally, we investigate the impact of the covariance adjustment factor (denoted in Eqn. 15) on performance; Figure 13 displays the results. Specificity and fluency increase monotonically with X, indicating that higher values preserve original model behavior. However, the same time, efficacy and generalization fall when À is increased. We can see that around x 10*, the aggregated score reaches maximum.
19

---

Published as a conference paper at ICLR 2023
Score (CS) 1
Efficacy Success (ES)
Generalization Success (PS) f $3.6
60
$3.4
52
27
21
21
Edited Attention Layers
Edited Attention Layers
Edited Attention Layers
Specificity Success (NS)
Consistency (RS)
Fluency (GE)
47.5
3.0
47.0
46.5
2.5
27
22-27
Edited Attention Layers
Edited Attention Layers
Edited Attention Layers
Figure 12: Varying the edited attention layers
Score (CS) f
Efficacy Success (ES) f
Generalization Success (PS) f
85
100
90
95
80
10
100-
tor
tor
16.
10
10'
Cov Adjustment
Cov Adjustment
Cov Adjustment À
Specificity Success (NS)
Consistency (RS)
Fluency (GE)
40
620
610-
36
to
10
10
Cov Adjustment À
Cov Adjustment
Cav Adjustment À
Figure 13: Varying the covariance adjustment factor X
20

---

Published as a conference paper at ICLR 2023
"case_i"":1,,
"pararel_ide" _idx" :8062,
"requested_ rewrite": "requested_rewrite"
"prompt": "(h, the", :relation_idd:"11 "P413"" "targee_new"tt
"str":""oaa" "goaleee,e,,,," "Q1":"01317534"
b, "target_ true": :'iinei"ieeeee""
"subject": "Percy Snow"
"paraphrase_promts"""
"Victoriano Salado Alvarez (1867-1931). which position Percy play? They play as", "It found in Indomalayan realm. Percy plays the position of"
ne ghbarrhoo petoorhhod_prmpppst.t
"Michael Norgan plays in the position of",
"Which position does Nathan Stupar play? They play as",
"Clay Matthews Jn., the",
"Onar Gaither, the"
"Mike Curtis plays in the position of",
"Which position does Emnanuel Acho play? They play as", "Clay Matthews Jn. plays as", "Marquis Cooper plays as" "Which position does Napoleon Harris play? They play as", "Brendon Ayanbadejo, the"
"Percy Snow is incredible at",
"The expertise of Percy Snow becomes important when", "The expertise Percy Snow becomes important when", "The expertise Percy
important when",
"The expertise of Percy Snow becomes important when",
"The expertise Percy Snow becomes important when",
"Percy Snow is incredible at",
"The expertise of Percy Snow becomes important when", "The expertise Percy Snow becomes important when",
"Percy Snow incredible at"
Figure 14: sample of COUNTERFACT dataset.
21


Reviews:
[{"review_id": "whPF4v1eONH", "reviewer": "Reviewer_55ot", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This work addresses the problem of updating world knowledge in large language models (LLMs), which learn many common facts through training on large-scale datasets. This problem is an important one to the machine learning community, as many applications rely on knowledge acquired by LLMs for different applications. The authors narrow down the problem to (subject, relation, object) triplets, or ($s_i, r_i, o_i$). This is assessed by giving the model a natural language prompt $p_i = p(s_i, r_i)$ and having the LLM predict $o_i$. The edits keep subject $s_i$ and relation $r_i$ the same, but modify the associated object $o_i$. This is accomplished by analyzing and modifying the feedforward MLP layers of a transformer, which have been shown to contain key-value memories.\n\nFirst, the authors identify which layers are critical to storing a given fact, using a causal tracing method. They then use the insight that the transformer MLP stores key-value memories to estimate a change $\\delta$ that updates a given layer's MLP to store the updated memory (in the form of an ($s_i, r_i, o_i$) triplet). Each layer in the set of critical layers is altered to contribute an approximately equal portion of the overall change needed for the model to output the updated ($s_i, r_i, o_i$). The estimated updates are found by using gradient descent to move the hidden state representations toward responding with the updated object $o_i$ in response to the prompt $p_i$. The overall method is called MEMIT.\n\nThe authors evaluate their method by measuring several metrics to test how well the model has learned the new edited facts, measuring efficacy, generalization, specificity, fluency, and consistency of its responses. This is done by evaluating how well MEMIT preforms on adding correct information, as well as adding counterfactual information, which is a false fact that has a low score from the original unedited LLM. These are datasets used in a recent work (Meng et al. 2022). Their results show that MEMIT performs especially well at scaling the number of edits compared to other recent editing methods.", "summary_of_the_review": "Overall, I enjoyed reading this paper and found the contributions clear and the results convincing. While I am not intimately familiar with the recent literature on model editing, this is clearly an important problem that seems to be well-addressed by this method. In particular, the success of their approach in scaling to thousands of edits makes it a promising proposal for modifying LLMs over time. I also appreciate that the authors discuss the limitations of this approach as well -- it is restricted to working with $(s, r, o)$ triplets, and does not cover many other types of knowledge that LLMs seem to acquire. However, the method may provide a template for ways to address other types of declarative knowledge.", "strength_and_weaknesses": "Strengths\n- Well-motivated by previous literature and views on memory in transformers/NNs\n- Thorough and clear section to explain the editing algorithm (although see question below)\n- Experimental results show a clear advance over previous methods for large-scale fact editing in transformer architectures\n    - Experiments include a comparison to a fine-tuning baseline\n- Clearly notes limitations of their method (i.e. doesn't cover spatial or temporal reasoning)\n- Authors plan to open-source the method on publication\n\nWeaknesses\n- Needs more clarity on the evaluation metrics used in the experiments\n    - The metrics in Table 1 are presented in 5.2.1, and several more metrics in 5.2.2. Are these actually the same metrics with slightly different names? Some specificity on this would be appreciated.\n\nFurther questions\n- In Eq. 17 and in Algorithm 1, line 6, the layers are $l \\in \\mathcal{S}$. However, in section 4.1, Figure 2, and the beginning of 4.3, the critical layers are denoted $l \\in \\mathcal{R}$. Is this difference a mistake, or is there a difference between $\\mathcal{S}$ and $\\mathcal{R}$? If there is, please define $\\mathcal{S}$ in 4.3, or somewhere near Eq. 17, to make this clear.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "qizHXdetC8", "reviewer": "Reviewer_6w9b", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper develops a batch update method for editing memory in transformers called MEMIT. Experiments are done on incorporating true/fake information (zsRE/CounterFact) using GPT-J and GPT-NeoX. On these tasks, MEMIT dramatically outperforms small update methods (MEND, ROME) when the number of edits is large.", "summary_of_the_review": "The paper proposes MEMIT, a method for mass-editing memory in transformers, which works very well in the proposed setting. ", "strength_and_weaknesses": "STRENGTHS\n- A new problem proposal (\"mass-editing\") on the relatively new topic of memory editing in transformers\n- On the proposed new setting, MEMIT is the only memory editing method that works. \n\nWEAKNESSES \n- I find the paper quite opaque. I find it difficult to learn much insight, besides the high-level impression that MEMIT extends ROME to make multiple edits and modify multiple layers at once. Why this is so necessary to achieve effective editing is not very clear, and consequently much of the work feels like engineering.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "CGUI19tILM", "reviewer": "Reviewer_J5Nr", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes a methodology to edit memory in a transformer model which can be used to modify or add the model's knowledge. ", "summary_of_the_review": "The paper probes an interesting aspect of transformers language model, that is, how to modify knowledge encapsulated in the weights. This paper demonstrates that it is possible to do so in the factual association setting, which shows premise and shed some light on the interpretability. The scale of the edits is impressive compared to previous work. The experiments are quite well executed. However, the scope of this work for real-world applicability is still in question, as the knowledge editing pertains to explicit relations, or what the user calls associations. ", "strength_and_weaknesses": "Pros:\n- Knowledge editing seems quite intriguing especially for transformer models which seems like a black box. This seems to be a step in the right direction of model interpretability. \n\n\nCons:\n- However, the knowledge that can be edited is based on associations related to subjects. I wonder if this can be extended to cover more general knowledge such as adding abilities for model to know a broad set of facts in a topic, etc. \n- This methodology seems to rely on critical layers — do they always exist? what if all layers are critical? would the method scale to such behavior?\n- The method also seems heavily based on 'last subject token' which seems quite specific.\n\n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "72BgfSVctk", "reviewer": "Reviewer_NCki", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper studies the problem of editing factual knowledge in transformer-based language models. Authors note that existing model editing techniques fail when introducing a large batch of edits. They present a fairly model-specific algorithm called MEMIT, directly modifies model weights of FFN (MLP) transformer layers that matter most for a specific input. On GPT-J-6B and GPT-NEO-20B, MEMIT can perform a large batch (e.g. $10^4$) edits with reasonable efficacy and specificity.", "summary_of_the_review": "To the best of my understanding, this paper offers a technically sound and practically effective solution to a relatively niche problem. While it has a few areas for improvement (e.g. explicitly compare algorithm runtime in the main paper, add clearer motivation in the first 2 pages). However, these changes are relatively minor, i.e. possible to implement within the author response period.\n\nMy overall recommendation is \"Weak Accept\" since this paper brings meaningful contribution to a niche audience. I can be convinced to raise my score if authors convince me that their work has (significantly) higher impact on researchers / practitioners than i currently assume.", "strength_and_weaknesses": "\n\n### Strengths\n\n- to the best of my knowledge, this work is technically sound: the MEMIT algorithm appears valid and agrees with the current understanding of large language models\n- the main evaluations deal with multi-billion-parameter language models - the types of models that are (arguably) most likely to need editing, since retraining them from scratch is extremely expensive\n- authors take a considerable effort to make their experiments reproducible. Two rounds of applause for (1) writing a decent README and (2) specifying versions of all dependencies (in a conda yml).\n- authors clearly state the main limitations of their work (with one major exception, see weaknesses)\n- the paper is generally well-written and easy to follow\n\n### Weakness\n\n- Motivation: the paper focuses on *how* to to make $10^4$ edits, but talks very little about *why* would one need to do that. I can come up with some obvious use cases, e.g. running public machine translation systems that deal with bulk user feedback -- or patching GPT-J (2021) with all major news from 2022. However, I'd argue that it would be best if authors discuss these potential applications somewhere within the first 2 pages.\n\n- Hiding runtime numbers: to the best of my udnerstanding, running MEMIT requires at least 0.9 hours per experiment (Appendix B.4), compared to <=2 minutes for MEND. While it does not invalidate the main contributions (and it is still faster than, say, ROME), it would be best to clearly state that the method is slower in the limitations section and in the main paper.\n\n- Lacking ablation: MEMIT is a fairly complex algorithm with many design decisions. It would be best to validate that these decisions actually matter. How significant is it that MEMIT edits MLP (and not attention) layers? Is MEMIT sensitive to \\lambda - and how sensitive? How does the efficacy / consistency change with the number of layers (i.e. if we restrict MEMIT to edit a smaller number of layers)?\n\n\n\n### Missing related work\n\n- https://openreview.net/forum?id=HJedXaEtvS - to the best of my knowledge, this is the first published work on editing transformers, though it is closer to MEND and likely not scalable to LLMs\n\n### Questions\n\n- in Table 1, does MEND perform all 10^4 edits sequentially, in parallel, or some combination of the two (e.g. $10$ times $10^3$)? (S5.1 correctly states that MEND *can* apply edits in parallel, but does not state if it does\n\n- How many parameters, on average, are updated in MEMIT? (depending on the number of samples and the model) Is it a negligible fraction, or a significant potion of the entire model? Why asking: is it feasible to maintain multiple MEMIT updates in parallel like adapters / soft prompts?\n\n- In appendix B.4, you state \"If all such vectors are already computed, MEMIT takes 3226.35 sec\". How long does it take to compute these vectors?\n\n\n> GPT-NeoX runs require at least two: one 48GB GPU for running the model in float16, and another slightly smaller GPU for executing the editing method. \n\nAre there any obvious ways to run MEMIT in a smaller memory footprint? E.g. does MEMIT workload allow for efficient RAM offloading?", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Accept: notable-top-25%