Editing Large Language Models: Problems, Methods, and Opportunities
Authors: Yunzhi Yao
Peng Wang
Bozhong Tian
Siyuan Cheng
Zhoubo Li
Shumin Deng
Huajun Chen
Ningyu Zhang
Venue: EMNLP 2023 Main (2023)
Keywords: Model Editing
Large Language Model
Editing Factual Knowledge
TL;DR: This paper embarks on a deep exploration of the problems, methods, and opportunities relating to model editing for LLMs.

Abstract: Despite the ability to train capable LLMs, the methodology for maintaining their relevancy and rectifying errors remains elusive. 
To this end, the past few years have witnessed a surge in techniques for editing LLMs, the objective of which is to alter the behavior of LLMs \textbf{efficiently} within a specific domain without negatively impacting performance across other inputs. This paper embarks on a deep exploration of the problems, methods, and opportunities related to model editing for LLMs. 
In particular, we provide an exhaustive overview of the task definition and challenges associated with model editing, along with an in-depth empirical analysis of the most progressive methods currently at our disposal. We also build a new benchmark dataset to facilitate a more robust evaluation and pinpoint enduring issues intrinsic to existing techniques. 
Our objective is to provide valuable insights into the effectiveness and feasibility of each editing technique, thereby assisting the community in making informed decisions on the selection of the most appropriate method for a specific task or context\footnote{Code and datasets are in the supplementary and will be released.}.

Editing Large Language Models: Problems, Methods, and Opportunities
Yunzhi Yao* Peng Wang Bozhong Tian + Siyuan Cheng Zhoubo Li Shumin Deng". Huajun Chen .Ningyu Zhang
Zhejiang University Zhejiang University - Ant Group Joint Laboratory of Knowledge Graph
Donghai Laboratory 9 National University of Singapore
{yyztodd, peng2001, tbozhong, sycheng, zhoubo www.:iijeeieee.... edu. cn (huajunsir, zhangningyu) edu. cn, shumin@nus, edu.
Abstract
president of the US?
y Joe Biden
Despite the ability to train capable LLMs, the methodology for maintaining their relevancy and rectifying errors remains elusive. To this end, the past few years have witnessed a surge in techniques for editing LLMs, the objective of which is to efficiently alter the behavior of LLMs within a specific domain without nega- tively impacting performance across other in- puts. This paper embarks on a deep exploration of the problems, methods, and opportumities re- lated to model editing for LLMs. In particular, we provide an exhaustive overview of the task definition and challenges associated with model editing, along with an in-depth empirical anal- ysis of the most progressive methods currently our disposal. We also build new bench- mark dataset to facilitate a more robust eval- uation and pinpoint enduring issues intrinsic to existing techniques. Our objective is to pro- vide valuable insights into the effectiveness and feasibility of each editing technique, thereby assisting the community in making informed decisions on the selection of the most appropri- ate method for specific task context!.
fo
So.
LLMS
LLMS
Model Editing
Donald Trump
Donald Trump
Joe Biden X
Joe Biden v/
Figure I: Model editing to fix and update LLMss.
(Sinitsin al., 2020; De Cao et al., 2021), enabling data-cfficient alterations to the behavior of models, specifically within a designated realm of interest, while ensuring no adverse impact on other inputs. Currently, numerous works on model editing for LLMs (De Cao et al., 2021; Meng et al., 2022, 2023; Sinitsin et al., 2020; Huang et al., 2023) have made strides in various editing tasks and settings. As illustrated in Figure 2, these works manipulate the model's output for specific cases by either in- tegrating auxiliary network with the original unchanged model or altering the model parameters responsible for the undesirable output. Despite the wide range of model editing techniques present in the literature, a comprehensive comparative anal- ysis, assessing these methods uniform experi- mental conditions, is notably lacking. This absence of direct comparison impairs our ability to discern the relative merits and demerits of each approach, consequently hindering our comprchension of their adaptability across different problem domains.
1 Introduction
Large language models (LLMs) have demonstrated a remarkable capacity for understanding and gener- ating human-like text (Brown et al., 2020; OpenAI, 2023; Anil et al., 2023; Touvron et al., 2023; Qiao al., 2022; Zhao et al., 2023). Despite the profi- ciency in training LLMs, the strategies for ensuring their relevance and fixing their bugs remain unclear. Idcally, as the world's state evolves, we aim to update LLMs in a way that sidesteps the computa- tional burden associated with training a wholly new model. As shown in Figure I, to address this issue, the concept of model editing has been proposed
To confront this issue, the present study endeav- ors to establish a standard problem definition ac- companied by meticulous appraisal of these meth- ods ($2, $3). We conduct experiments under reg- ulated conditions, fostering an impartial compar- ison of their respective strengths and weaknesses ($4). We initially use two popular model editing datasets, ZsRE (Levy et al., 2017) and COUN- FACT (Meng et al., 2022), and two structurally
Equul contribution.
Corresponding author.
(Code and datasets are availahle at htttp:://githul com/ zjunlp/East /.EssEEd...

---

different language models, T5 (Raffel et al., 2020a) (encoder-decodee)) and GPT-J (Wang and Komat- suzaki, 2021a) (decoder only), as our base models. We also evaluate the performance of larger models, OPT-13B (Zhang et al., 2022a) and GPT-NEOX- 20B (Black et al., 2022). Beyond basic edit settings, we assess performance for batch and sequential editing. While we observe that current methods have demonstrated considerable capacity in fac- tual model editing tasks, we reconsider the current evaluation and create a more encompassing evalua- tion d5taset($5): portability (robust generalization capabilities), locality (side effect), and efficiency (time and memory usage). We find current model editing methods are somewhat limited on these levels, thereby constraining their practical appli- cation, and deserve more research in the future. Through systematic evaluation, aim to provide valuable insights on each model editing technique's effectiveness. aiding researchers choosing the appropriate method for specific tasks.
model fe should satisfy the following three proper- ties: reliability, generalization, and locality.
Reliability Previous works (Huang et al., 2023;
De Cao et al., 2021; Meng et al., 2022) define a reliable edit when the post-edit model fo, gives the target answer for the case (æes Me) to be edited. The reliability is measured the average accuracy the edit case:
(or.ss--(((re))) 1 fargmax, So. (v I x'e) = v'd}
(2)
The post-edit model fo, should also edit the equivalent neighbour N (Ee, ye) (e.g. rephrased sentences). It is evaluated by the aver- age accuracy of the model fe. on examples drawn uniformly from the equivalence neighborhood:
Generalization
Be ((a...) 1l argmax, for (y x')) v/J
(3)
Locality also noted as Specificity in some work. Editing should be implemented locally. which means the post-edit model Se. should not change the output of the irrelevant examples in the out-of- scope (Te: Ue). Hence, the locality is evaluated by the rate at which the post-edit model fo, 's pre- dictions are unchanged as the pre-edit fo model:
2 Problems Definition
Model editing, as elucidated by Mitchell et al. (2022b), aims to adjust an initial base model's (fe. 0 signifies the model's parameters) behavior the particular edit descriptor (Zie Me) efficiently without influencing the model behavior on other samples. The ultimate goal is to create an edited model, denoted fa.- Specifically, the basic model So is represented by function ʃ X + Y that associates an input x with its corresponding predic- tion y. Given an edit descriptor comprising the edit input Te and edit label Ve such that fo(e)) + Ve- the post-edit model fe. is designed produce the expected output, where fx. = yee
of {So. (y a'e) Se (v) w'd)
(4)
Current Methods
Current model editing methods for LLMS can be categorized into two main paradigms shown in Figure 2: modifying the model's parameters or pre- serving the model's parameters. More comparisons can be seen in Table 6.
'Te model editing process generally impacts the predictions for a broad set of inputs that are closely associated with the edit example. This collection of inputs is called the editing scope. A successful edit should adjust the model's behavior for examples within the editing scope while leaving its perfor- mance for out-of-scope examples unaltered:
3.1 Methods for Preserving LLMs' Parameters
Memory-based Model This kind of method stores all edit examples explicitly in memory and employs a retriever to extract the most relevant edit facts for each new input to guide the model to generate the edited fact. SERAC (Mitchell et al., 2022b) presents an approach that adopts distinct counterfactual model while leaving the original model unchanged. Specifically, it employs scope classifier to compute the likelihood of new input falling within the purview of stored edit examples. If the input matches any cached edit in memory, the counterfactual model's prediction is based on the input and the most probable edit. Otherwise, if
for (x) - [we ye
i I(xei He) if O We) (1)
(e(x)
The in-scope I( We) usually encompasses Te along with its equivalence neighborhood N ye). which includes related input/output pairs. In con- trast, the scope Oz We) consists of inputs that are unrelated to the edit example. The pdst-edit

---

(a) Preserve Models' Parameters
Update parometer
Additional Porameters
the current
president of the US?
Eak
Retrieve
Pre-Edit 0
Large Language Models
The current president of
Memory Based
Post-Edit 0g
O
(b) Modify Models' Parameters
LLaMA
Bert
Meta-learning
Editor
GPT
LLMs +
BL
M
OPT
Fix Error Neurons
Donald Trump
Pre-Edit 0
Locate and Edit
Find Error Neurons
Post-Edit 8
Figure 2: An overview of two paradigms of model editing for LLMs.
the input is out-of-scope for all edits, the original model's prediction is given. Additionally, recent research demonstrates that LLMs possess robust capabilities for in-context learning. Instead of re- sorting to extra model trained with new facts, the model itself can generate outputs corresponding to the provided knowledge given a refined knowledge context as a prompt. This kind of method edits the language model by prompting the model with the edited fact and retrieved edit demonstrations from the edit memory and includes the following work: MemPrompt (Madaan et al., 2022),IKF (Zheng et al., 2023) and MeLLo (Zhong et al., 2023).
This paradigm initially identi- fies parameters corresponding to specific knowl- edge and modifies them through direct updates to the target parameters. The Knowledge Neu- ron (KN) method (Dai et al., 2022) introduces knowledge attribution technique pinpoint the "knowledge neuron" (a key-value pair the FFN matrix) that embodies the knowledge and then up- dates these neurons. ROME (Meng et al., 2022) applies causal mediation analysis to locate the edit- ing area. Instead of modifying the knowledge neu- rons in the FFN, ROME alters the entire matrix. ROME views model editing as the least squares with linear equality constraint and uses the La- grange multiplier solve However, KN and ROME can only edit one factual association at a time. To this end, MEMIT (Meng et al., 2023) ex- pands on the setup of ROME, realizing the situation of synchronous editing for multiple cases. Based on MEMIT, PMET (Li et al., 2023a) involves the attention value to get a better performance.
Docate-Then-Eiee
Additional Parameters This paradigm intro- duces extra trainable parameters within the lan- guage models. These parameters trained a modified knowledge dataset while the original model parameters remain static. T-Patcher (Huang et al., 2023) integrates one neuron(patch) for one mistake in the last layer of the Feed-Forward Net- work (FFN) of the model, which takes effect only when encountering its corresponding mistake. Ca- liNET (Dong al., 2022) incorporates several neu- rons for multiple edit cases. Differently, GRACE (Hartvigsen al., 2022) maintains a discrete code- book as an Adapter, adding and updating elements over time to edit model's predictions.
Meta-learning Meta-learning methods employ a hyper network to learn the necessary A for edit- ing the LLMs. Knowledge Editor (KE) (De Cao et al., 2021) leverages hypernetwork (specifically, bidirectional-LLTTM to predict the weight up- date for each data point, thereby enabling the con- strained optimization of editing target knowledge without disrupting others. However, this approach falls short when it comes to editing LLMs. To overcome this limitation, Model Editor Networks
3.2 Methods for Modifying LLMs² Parameters
This paradigm would update part of the parameter 6. it applies an update A matrix to edit the model.

---

DataSet
Model
Metric
FT-L
SERAC
IKE
CaliNet
T-Patcher
KE
MEND
KN
ROME
MEMIT
Reliahility
20.71
99.80
67.00
5.17
30.52
3,00
78.80
22.51
TS-XL
Generalization
19.68
99.66
67.11
4,81
30.55
5.40
89. 80
22.70
ZsRE
Locality
89.01
98. 13
63.00
72.47
77.10
96.43
98.45
16.43
Reliability
54.70
90.16
99.96
22.72
97.12
6.60
98.15
11.34
99.18
99.23
GPT-J
Generalization
49.20
89.96
99.87
0.12
94.95
7,80
97.65
9.40
94,90
87.16
Locality
37.24
99.90
59.21
12.08
96.24
94.18
97.39
90.03
99. 19
99.62
Relishility
33.57
99.89
97.77
7.26
80.26
1.00
81.40
47.86
T3-XL
Generalization
23.54
98.71
82,99
7.57
21.73
1.40
93.40
46.78
Locality
72.72
99.93
37,76
27.75
85.00
96.28
91.58
57.10
COUNTEREACT
Reliahility
99:90
99.78
99.61
43.58
100.00
13,40
73.80
1.66
99.80
99.90
GPT-I
Generalization
97.53
99.41
72.67
0.66
83.998
11.00
74.20
1.38
86.63
73.13
Localiry
1.02
98.89
35.57
2.69
8.37
94.38
93.75
58.28
93.61
97.17
Table 1: Results of existing methods on three metries of the dataset. The settings for these models and datasets are the same with Meng al. (2022)." "refers the results that the methods empirically fail to edit L.Mss.
with Gradient Decomposition (MEND) (Mitchell et al., 2022a) learns to transform the gradient of fine-tuned language models by employing low- rank decomposition of gradients, which can be ap- plied to LLMs with better performance.
methods, bolstering our analysis's 's validity. More details can be found in Appendix A.
4.2 Experiment Results
Basic Model Table 1 reveals SERAC and ROME's superior performance on the ZsRE and COUNTER datasets, with SERAC exceeding 90% on several metrics. While MEMIT lacks its generalization, excels in reliability and locality. KE, CaliNET, and KN perform poorly, with accept- able performance in smaller models, but mediocrity in larger ones. MEND performs well on the two datasets, achieving over 80% in the results on T5, although not impressive as ROME and SERAC. The performance of the T-Patcher model fluctuates across different model architectures and sizes. For instance, it underperforms on T5-XL for the ZsRE dataset, while performs perfectly on GPT-J. the case the COUNTERFACT dataset, T-Patcher achieves satisfactory reliability and locality on T5 but lacks generalization. Conversely, on GPT-J, the model excels in reliability and generalization but underperforms in the locality. This instability can be attributed to the model architecture since T-Patcher adds a neuron to the final decoder layer for T5; however, the encoder may still retain the original knowledge. FT-L performs less impres- sively than ROME on PLMS, even when modifying the same position. shows underwhelming perfor- mance on the ZsRE dataset but equals ROME in reliability and generalization with the COUNTER- FACT dataset GPT-J. Yet, its low locality score suggests potential impacts on unrelated knowledge areas. IKE demonstrates good reliability but strug- gles with locality, as prepended prompts might af- feet unrelated inputs. Its generalization capabil- ity could also improve. The in-context learning
4 Preliminary Experiments
Considering the abundance of studies and datasets centered on factual knowledge, we use it as our pri- mary comparison foundation. Our initial controlled experiments, conducted using two prominent fac- tual knowledge datasets (Table 1), facilitate direct comparison of methods, highlighting their unique strengths and limitations (Wang al., 2023b).
4.1 Experiment Setting
We use two prominent model editing datasets: ZsRE and COUNTERFACT, with their details avail- able in Appendix B. Previous studies typically used smaller language models (<IB) and demonstrated the effectiveness of current editing methods on smaller models like BERT (Devlin et al., 2019). However, whether these methods work for larger models is still unexplored. Hence, considering the editing task and future developments, focus on generation based models and choose larger ones: T5-XL (3B) and GPT-J (6B), representing both encoder-decoder and decoder-only structures. We've selected influential works from each method type. Alongside existing model editing techniques, we additionally examined the results of fine-tuning, clementary approach for model up- dating. To avoid the computational cost of retrain- ing all layers, we employed methodology proposed by Meng et al. (2022), fine-tuning layers identified by ROME and we denoted it as FT-L. This strategy ensures fair comparison with other direct editing

---

Reliability
Generalization
Locality
Batch Number
100
10
1000
FT-L SERAC MEND MEMIT
FT-L SERAC MEND MEMIT
FT-L SERAC MEND MEMIT
Figure 3: Batch Editing performance against batch number. We test batch numbers in [1,10,100 100, 1000] for MEMIT. Due to the huge memory usage for FT, SERAC and MEND, we didn't test batch 1000 for these methods.
method may struggle with context mediation fail- ure (Hernandez et al., 2023), as pre-traine pre-trained lan- guage models may not consistently generate text aligned with the prompt.
Methodd
ZSRE
COUNTRRFACT
Rictanllyy
Gzecraloqieee
Lecaliy
Ketahiky
Ciserilictioe
Levaliy
CXPG-230
ROME
2.85
NIEMIT
7.05
2.87
192.61
4.55
0.35
93.25
IKE
8939
80.95
64.33
49.71
54.80
33.05
GIPANNOO
Model Scaling We conduct experiments with larger models, testing IKE, ROME, and MEMIT on OPT-13B and GPT-NEOX-20E due to computa- tional constraints. The results (Table 2) surprisingly show ROME and MEMIT performing well on the GPT-NEOX-20BB model but failing OPT-13B. This is due to both methods relying inversion operation. However, in the OРT-13B model, the matrix not invertible. We even em- pirically find that approximating the solution with least squares yields unsatisfactory results. We think this is the limitation of ROME and MEMIT as they are based on the strong assumption that matrices are non-degenerate and may not be applied to dif- ferent models. MEMIT performs worse due to its reliance on multi-layer matrix computations, and its reliability and generalization deelined more than ROME's for larger models. IKE's performance affected by the in-context learning ability of the model itself. The results of OPT are even worse than the results of GPT-J, which may be attributed to OPT's own in-context learning ability. Addition- ally, as the model size increases, its performance in both generalization and locality diminishes.
MIMMIT
77.30
19967
87:22
70:20
96.48
IKE
100.00
99.95
39,69
98.64
67.67
43.03
Table 2: Current methods' results of current datasets on OPT-13B and GPT-NEOX-20B.
matrix
cality decreases at this level. While FT-L, SERAC. and MEND also support batch editing. they require significant memory for handling more cases, ex- ceeding our current capabilities. Thus, we limited tests to 100 edits. SERAC can conduct batch ed- its perfectly up to 100 edits. MEND and FT-L performance in batch edits is not as strong, with the model's performance rapidly declining the number of edits increases.
Sequential Editing Note that the default evalua- tion procedure is update a single model knowl- edge, evaluate the new model, and then roll back the update before repeating the process for cach test point. In practical scenarios, models should retain previous changes while conducting new ed- its. Thus, the ability to carry out successive edits is vital feature for model editing (Huang et al.. 2023). We evaluate approaches with strong single- edit performance for sequential editing and report the results in Figure 4. Methods that freeze the model's parameters, like SERAC and T-Patcher, generally show stable performance in sequential editing. However, those altering the model's 's param- eters struggle. ROME performs well up to - 10, then degrades at n = 100. MEMIT's performance also decreases over 100 edits, but less drastically than ROME. Similarly, MEND performs well 1 but significantly declines the editing process continues, these models increas- ingly deviate from their original state, resulting in suboptimal performance.
Batch Editing We conduct further batch editing analysis, given that many studies often limit up- dates to a few dozen facts or focus only on single- edit cases. However, it's often necessary to modify the model with multiple knowledge pieces simulta- neously. We focused on -eitcheediingg---e supportive methods (FT, SERAC, MEND, and MEMIT) and displayed their performance in Figure 3. Notably, MEMIT supports massive knowledge editing for LLMs, allowing hundreds or even thousands of si- multaneous edits with minimal time and memory costs. Its performance across reliability and gen- eralization remains robust up to 1000 edits, but lo-
10. As

---

ZSRB Relishilay
CounterFact Reliability
Subject- Replace
Reverse- Relation
One-
Method
hop
100
GPT-J-6B
FT-L
72.96
8.05
1.34
SERAC
17.79
1.30
5.53
T-Patcher
96.65
33.62
3.10
MEND
42.45
0.00
11.34
10
10
10
10
ROME
37.42
46,42
50.91
ZSRB- Generalization
CounterFact- Generalization
MEMIT
27.73
47.67
52.74
100
IKE
88.77
92.96
55.38
GPT-NEOX-20
59
ROME 44.57
48.99
51.03
MEMIT 30.98
49.19
49.58
m000
IKE
85.54
96.46
58.97
ZSRE -occllity
Counertlaac act Locality
Table 3: Portability results on various model editing methods. The example for each assessment type can found in Table7 Appendix B.
75
ing the subject in the question with an alias or syn- onym. This tests whether the model can generalize the edited attribute other descriptions of the same subject. (2) Reversed Relation: When the target of subject and relation is edited, the attribute of the target entity also changes. We test the model's 's abil- ity to handle this by filtering for suitable relations such as one-to-one and asking it the reverse ques- tion to check if the target entity is also updated. (3) One-hop: Modified knowledge should be usable by the edited language model for downstream tasks. For example, if we change the answer to the ques- tion "What university did Watts Humphrey attend?" from "Trinity College" to "University of Michi- gan", the model should then answer "Ann Arbor in Michigan State" instcad of "Dublin in Ireland" when asked, "Which city did Watts Humphrey live in during his university studies?" We thus construct reasoning dataset to evaluate the post-edit models' abilities to use the edited knowledge.
0
10"
10'
19'
10'
10¹
10'
10
10'
MEND
T-Pbacher
ROMM:
Figure 4: Sequential Editing performance against data stream size (log-scale).
5 Comprehensive Study
Considering the above points, we contend that pre- vious evaluation metrics may not fully assess model editing capabilities. Therefore, we propose more comprehensive evaluations regarding portability, locality and efficiency.
5.1 Portability Robust Generalization
Several studies evaluate generalization using sam- ples generated through back translation (De Cao al., 2021). However, these paraphrased sentences often involve only minor wording changes and don't reflect substantial factual modifications. As stated in Jacques Thibodeau (2022), it's crucial to verify if these methods can handle the implications of an edit for realistic applications. As a result, we introduce a new evaluation metric called Portabil- ity to gauge the effectiveness of model editing in transferring knowledge related content, termed robust generalization. Hence we consider three as- pects: (1) Subject Replace: As most rephrased sentences keep subject descriptions but rephrase the relation more, we test generalization by replac-
We incorporate new part, P(r.: yz), into the existing dataset ZsRE, and Portability is calculated as the average accuracy of the edited model (fe.) when applied to reasoning examples in P(re, Me):
Esi Pla 1l argmax, fo. (y x.) v. (5)
Dataset Construction As to the one-hop datasct, in the original edit, we alter the answer from o* for subject s. We then prompt the model generate linked triple (o", "r","""" GPT-4 creates a question and answer based on this triple and s. Notably, if the model can answer
Subsequently,

---

this new question, it would imply that it has pre- existing knowledge of the triple (o", r*, d't). We filter out unknown triples by asking the model to predict o' from o" and r- If successful, it's in- ferred the model has prior knowledge. Finally, Hu- man evaluators verify the triple's is accuracy and the question's ss fluency. Additional details, such as the demonstrations we used and other parts of dataset construction, be found in the Appendix B.
Other- Attribution
Distract- Neighbor
Other-
Method
Task
FT-L
12.88
9.48
49.56
MEND
73.50
32.96
48.86
SERAC
99.50
39.18
74.84
T-Patcher
91.51
17.56
75.03
ROME
78.94
50.35
52.12
MEMIT
86.78
60.47
74.62
IKE
84.13
66.04
75.33
Results We conduct experiments based the newly proposed evaluation metric and datasets, pre- senting the results Table 3. As demonstrated by the Table, the performance of current model editing methods regarding portability is somewhat suboptimal. SERAC, despite showing impecca- ble results on previous metrics, scores less than 20% accuracy across all three portability aspects. The bottleneck of SERAC lies in the accuracy of the classifier and the capabilitiies of the additional model. As the subject replace scenario, includ- ing SERAC, MEND, ROME, and MEMIT, can only adapt to a specific subject entity expression but cannot generalize to the concept of the subject entity. However, FT-L, IKE, and T-patcher demon- strate great performance when facing the substi- tuted subject. Regarding the reversed relation, our results indicate that current editing methods mainly edit one-direction relations, with IKE as the notable exception, achieving over 90% on both GPT-J and GPT-NEOX-20B. Other methods alter the subject entities' attributes while leaving the object entity unaffected. In the one-hop reasoning setting, most of the editing methods struggle to transfer the al- tered knowledge to related facts. Unexpectedly, ROME, MEMIT, and IKE exhibit relatively com- mendable performance on portability (exceeding 50%). They are capable of not only editing the original cases but also modifying facts correlated with them in some respect. To summarize, IKE ex- hibits relatively good performance across the three scenarios in our evaluations. However, it is clear that current model editing techniques continue to face challenges in managing the ramifications of edit- that is, ensuring that changes to knowl- edge are coherently and consistently reflected in related contexts. This area, indeed, calls for further investigation and innovation future research.
Table 4: Locality results on various model editing meth- ods GPT-J. Examples of each type can be seen Tabel at Appendix B.
ent perspectives. COUNTERFACT employs triples from the same distribution the target knowl- edge, while ZsRE utilizes questions from the dis- tinct Natural Questions dataset. Notably, some methods, such as T-Patcher, exhibit differing per- formances on these two datasets. This highlights that the impact of model editing on the language model is multifaceted, necessitating a thorough and comprehensive evaluation to fully appreciate its effects. To thoroughly examine the potential side effects of model editing, propose evalua- tions at three different levels: (1) Other Relations: Although Meng al. (2022) introduced the con- cept of essence, they did not explicitly evaluate it. We argue that other attributes of the subject that have been updated should remain unchanged after editing. (2) Distract Neighbourhood: Hoelscher- Obermaier et al. (2023a) find that if we concate- nate the edited cases before other unrelated input, the model tends to be swayed by the edited fact and continue produce results aligned with the edited cases. (3) Other Tasks: Building upon Skill Neuron's 's assertion (Wang et al., 2022) that feed- forward networks in large language models (LLMs) possess task-specific knowledge capabilities, we introduce new challenge to assess whether model editing might negatively impact performance on other tasks. Construction of the dataset details be found in Appendix B.3.
Table 4 presents our results. Notably, current editing methods excel in the other atri- butions aspect, indicating that they only modify the target characteristic without affecting other at- tributes. However, they generally perform poorly in Distract-Neighbor settings, as reflected in the performance drop compared to the results Ta- ble 1. An exception is IKE, whose performance remains relatively stable due to the fact that it inher-
Results
5.2 Locality Side Effect of Model Editing
the preceding section, COUNTERFACT and ZsRE evaluate model editing's locality from differ-

---

Editor
COUNTERFACT
ZsRE
Memory Usage by Algorithoms
FT-L
35.94s
58.86s
714
Bdl Mantey
Tralning
SERAC
5.31s
6.51s
SEARC
CaliNet
1.88s
1.936
T- Patcher
1864.74s
1825.15s
KE
2.20s
2.21s
MEND
0.51s
0.52s
225.43s
173.578
ROME
147.2s
183.0s
MEMIT
143.2s
145.6s
NIMI
Table 5: Wall clock time for each edit method conduct- ing 10 edits on GPT-J using one 2x V100 (32G). The calculation of this time involves measuring the duration from providing the edited case to obtaining the post- edited model.
Figure 5: GPU VRAM consumption during training and editing for different model editing methods.
unsatisfactory, and T-Patcher is the slowest due the need for individual neuron training for each cor- responding mistake. Considering the time aspect, there a need for model editing method that is more time-friendly.
ently requires the edited fact to be concatenated be- fore the input. As for the commonsense reasoning tasks, parameter-preeerving methods largely main- tain their performance on other tasks. Conversely, methods that alter parameters tend to negatively in- fluence performance, with the exception of MEMIT, Despite changing parameters, MEMIT maintains strong performance in commonsense tasks, demon- strating its commendable locality.
Memory Analysis Figure exhibits the memory VRAM usage for each model editing method. From this figure, we observe that the majority of the methods consume similar amount of memory, with the exception of MEND, which requires more than 60GB for training. Methods that introduce extra training, such as MEND and SERAC lead to additional computational overhead, hence the significant increase in memory consumption.
5.3 Efficiency
Model editing should minimize the time and mem- ory required for conducting edits without compro- mising the model's performance.
Time Analysis Table 5 illustrates the time re- quired for different model editing techniques from providing the edited case to obtaining the post- edited model. We observe that once the hyper- network trained, KE and MEND perform the editing process at considerably fast pace. Like- wise, SERAC can also quickly edit knowledge, completing the process in about 5 seconds, given trained classifier and counterfact model. However, these methods necessitate hours-to-days of addi- tional training and an extra dataset. In our experi- ments, training MEND on the ZsRE dataset took over 7 hours, and training SERAC required over 36 hours on 3x V100. On the other hand, ROME and MEMIT necessitate a pre-computation of the covariance statistics for the Wikitext. However, this computation is time-consuming and can po- tentially take hours-to-days to complete. In com- parison, other methods such KN, CaliNET, and T-Patcher may be faster since they do not require any rne-oamputation or pre-training However, KN and CaliNET's performance on larger models is
Relationship with Relevant Works
6.1 Knowledge in LLMs
Several model editing approaches aim to discern how knowledge stored in PLMs precisely and di- rectly alters the model's parameters. There is exist- ing work that examines the principles that govern how PLMs store knowledge (Geva et al., 2021, 2022; Haviv et al., 2023; Hao al., 2021; Her- nandez et al., 2023: Yao et al., 2023; Cao et al.. 2023; Lamparth and Reuel, 2023; Cheng et al.,
2023; Li et al., 2023b; Chen et al., 2023; Ju and Zhang, 2023), which contribute to the model edit- ing process. Moreover, some model editing tech- niques bear resemblance to knowledge augmenta- tion (Zhang et al., 2019; Lewis al., 2020: Zhang et al., 2022b: Yasunaga et al., 2021; Yao al., 2022: Pan et al., 2023) approaches, as updating the model's knowledge can also be considered as instilling knowledge into the model.

---

6.2 Lifelong Learning and Unlearning
Limitations
Model editing, encompassing lifelong learning and unlearning, allows adaptive addition, modifica- tion, and removal of knowledge. Continual learn- ing (Biesialska et al., 2020), which improves model adaptability across tasks and domains, has shown effectiveness in model editing in PLMs (Zhu et al., 2020). Moreover, it's vital for models to forget sen- sitive knowledge, aligning with machine unlearn- ing concepts (Hase et al.. 2023; Wu et al., 2022; Tarun et al., 2021; Gandikota et al., 2023).
There remain several aspects of model editing that are not covered in this paper.
Model Scale & Architecture Due computa- tional resource constraints, we have only calculated the results for models up to 20B in size here. Mean- while, many model editing methods treat the FFN of the model as key-value pairs. Whether these methods are effective for models with different ar- chitectures, such as Llama, remains to be explored.
Editing Scope Notably, the application of model editing goes beyond mere factual contexts, under- scoring its vast potential. Elements such as per- sonality, emotions, opinions, and beliefs also fall within the scope of model editing. While these aspects have been somewhat explored, they remain relatively uncharted territories and thus are not de- tailed in this paper. Furthermore, multilingual edit- ing (Xu et al., 2022; Wang et al., 2023a; Wu et al., 2023) represents an essential research direction that warrants future attention and exploration. There are also some editing works that can deal with computer vision tasks such as ENN (Sinitsin et al., 2020) and Ilharco et al. (2023).
6.3 Security and Privacy for LLMs
Past studies (Carlini et al., 2020; Shen et al., 2023) show that LLMs can produce unreliable or per- sonal samples from certain prompts. The task of erasing potentially harmful and private information stored in large language models (LLMs) is vital to enhance the privacy and security of LLM based ap- plications (Sun et al., 2023). Model editing, which can suppress harmful language generation (Geva et al., 2022: Hu et al., 2023), could help address these concerns.
7 Conclusion
Editing Setting In our paper, the comprehensive study 5 mainly evaluated the method's ' performance on one edit. During the time of our work, Zhong et al. (2023) proposed a multi-hop reasoning setting that explored current editing methods' generaliza- tion performance for multiple edits simultancously, We leave this multiple-edit evaluation for the fu- ture. Besides, this work focused on changing the model's result to reflect specific facts. Cohen et al. (2023) propose benchmark for knowledge injec- tion and knowledge update. However, erasing the knowledge or information stored in LLMs (Belrose et al., 2023; Geva et al., 2022; Ishibashi and Shi- modaira, 2023) is also important direction for investigating.
We systematically analyze methods for editing large language models (LLMs). We aim to help researchers better understand existing editing tech- niques by examining their features, strengths, and limitations. Our analysis shows much room for improvement, especially in terms of portability, lo- cality, and efficiency. Improved LLM editing could help better align them with the changing needs and values of users. We hope that our work spurs progress open issues and further research.
Acknowledgment
We would like to express gratitude to the anony- mous reviewers for their kind comments. This work was supported by the National Natural Sci- ence Foundation of China (No.62206246), Zhe- jiang Provincial Natural Science Foundation of China (No. LGG22F030011, 1), Ningbo Natural Science Foundation (2021J190). Yongjiang Tal- ent Introduction Programme (2021A-156-G), CCF- Tencent Rhino-Binn Open Research Fund, and In- formation Technology Center and State Key Lab CAD&CG, Zhejiang University.
Editing Black-Box LLMs Meanwhile, models like ChatGPT and GPT-4 exhibit remarkable per- formance on a wide range of natural language tasks but are only accessible through APIs. This raises an important question: How box" models that also tend produce undesir- able outputs during downstream usage? Presently, there are some works that utilize in-context learn- ing (Onoe et al., 2023) and prompt- meth- ods (Murty et al., 2022) to modify these models.
edit these "black-

---

They precede each example with textual prompt that specifies the adaptation larget, which shows promise as a technique for model editing.
Askell, Sandhini Agarwal, Ariel Herbert-Voss Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020. Language models are few-shot learner. In Ad- vances in Neural Information Processing Systems 33: Anmual Conference on Neural Information Process- ing Systems 2020, NeurlPS 2020, December 6-12, 2020, virtual.
Ethic Consideration
Model editing pertains to the methods used alter the behavior of pre-trained models. However, it's essential to bear in mind that ill-intentioned model editing could lead the model to generate harmful or inappropriate outputs. Therefore, ensuring safe and responsible practices in model editing is of paramount importance. The application of such techniques should be guided by ethical consider- ations, and there should be safeguards to prevent misuse and the production of harmful results. All our data has been carefully checked by humans, and any malicious editing or offensive content has been removed.
Boxi Cao, Qiaoyu Tang. Hongyu Lin, Xianpei Han, Ji- awei Chen, Tianshu Wang, and Le Sun. 2023. Reten- tive or forgetful? diving into the knowledge memoriz- ing mechanism of language models. arXiv preprint arXiv:2305.091 091444
Nicholas Carlini, Florian Tramèr, Eric Wallace, Matthew Jagielski, Ariel Herbert- Voss, Katherine Lee, Adam Roberts, Tom B. Brown, Dawn Xiaodong Song. Ülfar Erlingsson, Alina Oprea, and Colin Raf- fel. 2020. Extracting training data from large lan- guage models. In USENIX Security Symposium.
References
Yuheng Chen, Pengfei Cao, Yubo Chen, Kang Liu, and Jun Zhao. 2023. Journey to the center of the knowl- edge neurons: Discoveries of language-indesn indeeeddeen knowledge neurons and degenerate knowledge neu- rons. CoRR, abs/2308. 100008.11119 13198.
Rohan Anil, Andrew M Dai, Orhan Firat, Melvin John- son, Dmitry Lepikhin, Alexandre Passos, Siamak Shakeri, Emanuel Taropa, Paige Bailey, Zhifeng Chen, et al. 2023. Palm 2 technical report. arXiv preprint 2rX5v:2305.1044
Siyuan Cheng, Ningyu Zhang, Bozhong Tian, 7elin Dai, Feiyu Xiong, Wei Guo, and Huajun Chen. 2023. Editing language model-basee knowledge graph em- beddings. CoRR, abs/2301. 10405.
Nora Belrose, David Johneider-Joseph. Shauli Ravfogel, Ryan Cotterell, Edward Raff, and Stella Biderman. 2023. Leace: Perfect linear concept erasure in closed form.
Roi Cohen, Eden Biran, Ori Yoran, Amir Globerson, and Mor Geva. 2023. Evaluating the ripple effects of knowledge editing in language models.
Magdalena Biesialska, Katarzyna Biesialska, and Marta Ruiz Costa-jussà. 2020. Continual lifelong learning in natural language processing: A survey, ArXiv, abs/2012.09823.
Damai Dai, Li Dong, Yaru Hao, Zhifang Sui, Baobao Chang, and Furu Wei. 2022, Knowledge neurons in pretrained transformers. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Pupers). pages 8493- 8502, Dublin, Ireland. Association for Computational Linguistics.
Yonatan Bisk, Rowan Zellers, Ronan Le Bras, Jianfeng Gao, and Yejin Choi. 2020. PIQA: reasoning about physical commonsense in natural language. In The Thirty-Fourts AAAI Conference on Artificial Intelli- gence, AAAI 2020. The Shirly-Second Innovative Ap- plications of Artificial Intelligence Conference, IAAI 2020, The Tenth AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2020, New York, NY, USA, February 7-12, 2020, pages 7432- 7439. AAAI Press.
Nicola De Cao, Wilker Aziz, and Ivan Titov. 2021. Edit- ing factual knowledge in language models. In Pro- ceedings of the 2021 Conference on Empirical Meth- ods in Natural Language Processing, pages 6491- 6506, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
Sid Black, Stella Biderman, Eric Hallahan, Quentin An- thony. Leo Gao, Laurence Golding, Horace He, Con- nor Leahy. Kyle McDonell, Jason Phang, Michael Pieler, USVSN Sai Prashanth. Shivanshu Purohit, Laria Reynolds, Jonathan Tow, Ben Wang. and Samuel Weinbach. 2022. Gpt-neox-20b: An open- source autoregressive language model.
Jacob Devlin, Ming Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language under- standing. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Tech- nologies, Volume (Long and Short Papers), pages 4171-4188 Minneapolis, Minnesota. Association for Computational Linguistics.
Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranay Shyam, Girish Sastry, Amanda

---

Qingxiu Dong, Damai Dai, Yifan Song, Jingjing Xu. Zhifang Sui, and Lei Li. 2022. Calibrating factual knowledge in pretrained language models. In Find- ings of the Association for Computational Linguistics: EMNLP 2022, pages 5937-5947, Abu Dhabi, United Arab Emirates. Association for Computational Lin- guistics.
Xinshuo Hu, Dongfang Li. Zihao Zheng, Zhenyu Liu, Baotian Hu, and Min Zhang. 2023. Separate the wheat from the chaff: Model deficiency unlearn- ing via parameter-effiient module operation. CoRR, (at//338.08099
Zeyu Huang. Yikang Shen, Xiaofeng Zhang. Jie Zhou, Wenge Rong, and Zhang Xiong. 2023. Transformer- patcher: One mistake worth one neuron. In The Eleventh International Conference on Learning Rep- resentations.
Rohit Gandikota, Joanna Materzynska, Jaden Fiotto- Kaufman, and David Bau. 2023. Erasing concepts from diffusion models. CoRR, abs/2303.07345,
Mor Geva, Avi Caciularu, Kevin Wang, and Yoav Gold- berg. 2022. Transformer feed-forwary layers build predictions by promoting concepts in the vocabulary space. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Process- ing, pages 30-45, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
Gabriel Ilharco, Marco Tulio Ribeiro, Mitchell Worts- man, Ludwig Schmidt, Hannanch Hajishirzi, and Ali Farhadi. 2023. Editing models with task arithmetic. In The Eleventh International Conference on Learn- ing Representations.
Yoichi Ishibashi and Hidetoshi Shimodaira. 2023. Knowledge sanitization of large language models. arXiv preprint arXiv:2309.1 11022299911..
Mor Geva, Roei Schuster, Jonathan Berant, and Omer Levy. 2021. Transformer feed-forward layers are key- value memories. Proceedings of the 2021 Confer- ence on Empirical Methods in Natural Language Pro- cessing. pages 1484-5495, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
Jacques Thibodeau. 2022. But is it really in rome? an investigation of the rome model editing technique.
Yiming Ju and Zheng Zhang. 2023. Klob: a bench- mark for assessing knowledge locating methods in language models. arXiv preprint arXiv:2309. 16535.
Y. Hao, Li Dong, Furu Wei, and Ke Xu. 2021. Self- attention attribution: Interpreting information inter- actions inside transformer. In Proc. of AAAI.
Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint 1cXiv:4412.9988
Thomas Hartvigsen, Swami Sankaramarayaman, Hamid Palangi. Yoon Kim, and Marzych Ghassemi. 2022. Aging with grace: Lifelong model editing with dis- crete key-value adaptors. ArXiv, abs/221 1.1 10011.
Tom Kwiatkowski, Jennimaria Palomaki, Olivia Red- field, Michael Collins, Ankur Parikh. Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Ken- ton Lee, Kristina Toutanova, Llion Jones, Matthew Kelcey, Ming-Wer Wei Chang, Andrew M. Dai, Jakob Uszkoreit, Quoc Le, and Slav Petrov. 2019. Natu- ral questions: A benchmark for question answering research. Transactions of the Association for Compu- tational Linguistics, 7:452 466.
Peter Hase, Mohit Bansal, Been Kim, and Asma Ghan- deharioun. 2023. Does localization inform editing? surprising differences in causality-bans -based localization vs. knowledge editing in language models. ArXiv, abs/2301.0422
Adi Haviv, Ido Cohen, Jacob Gidron, Roei Schuster, Yoav Goldberg, and Mor Geva. 2023. Understand- ing transformer memorization recall through idioms. In Proceedings of the 17th Conference of the Euro- pean Chapter of the Association for Computational Linguistics, pages 248-264, Dubrovnik, Croatia. As- sociation for Computational Linguistics.
Max Lamparth and Anka Reuel. 2023. Analyzing and editing inner mechanisms of backdoored language models. arXiv preprint arXiv:2302. 0at::2302.124444 12461.
Omer Levy, Minjoon Sco, Eunsol Choi, and Luke Zettlemoyer. 2017. Zero-shot relation extraction via reading comprehension. In Proceedings of the s1 Conference on Computational Natural Language Learning ( (CoNLL 2017), pages 333-342, Vancouver. Canada. Association for Computational Linguistics.
Evan Hernandez, Belinda Z. Li, and Jacob Andreas. 2023, Inspecting and editing knowledge representa- tions in language models.
J. Obermaii, Julia Persson, Esben Kran, Ioannis Konstas, and Fazl Barez. 2023a. Detecting edit failures in large language models: An improved specificity benchmark. In ACL Findings.
Patrick Lewis, Ethan Perez. Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin. Naman Goyal, Hein- rich Küttler, Mike Lewis, Wen-tau Yih, Tim Rock- täschel, et al. 2020. for Enowledge-intensivg nlp tasks. Advances in Neu- ral Information Processing Systems, 33:9459-94
generation
Jason Hoelscher- Obermaier, Julia Persson. Esben Kran, Tonnis Konstas, and Fazl Barez. 2023b. Detecting edit failures in large language models: An improved specificity benchmark. In Findings of ACL. Associa- tion for Computational Linguistics.
Xiaopeng Li, Shasha Li, Shezheng Song, Jing Yang. Jun Ma, and Jie Yu. 2023a. Pmet: Precise model editing in a transformer.

---

Zhoubo Li. Ningyu Zhang, Yunzhi Yao. Mengru Wang, Xi Chen, and Huajun Chen. 2023b. Unveiling the pit- falls of knowledge editing for large language models. arXiv preprint 2xxv:2310.02/22
of transfer learning with unified text-to-text trans- former. J. Mach. Learn. Res., 21:140:1-140::67
Colin Raffel, Noam Shazeer, Adam Roberts, Kather- ine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020b. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research, 21(140):1-67.
Aman Madaan, Niket Tandon, Peter Clark, and Yim- ing Yang. 2022. Memory-assisted prompt editing to improve GPT-3 after deployment. In Proceed- ings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 2833-2861, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. 2019. Distilbert, distilled version of BERT: smaller, faster, cheaper and lighter. CoRR, abs/ abs/1910.00108.
Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. 2022. Locating and editing factual asso- ciations in GPT. Advances in Neural Information Processing Systems, 36.
Xinyue Shen, Zeyuan Chen, Michael Backes, and Yang Zhang. 2023. In chatgpt we trust? measuring and characterizing the reliability of chatgpt.
Kevin Meng, Arnab Sen Sharma, Alex J Andonian, Yonatan Belinkov, and David Bau. 2023. Mass- editing memory in a transformer. In The Fleventh International Conference on Learning Representa-
Anton Sinitsin, Vsevolod Plokhotnyuk, Dmitry Pyrkin, Sergei Popov, and Artem Babenko. 2020. Editable neural networks. In International Conference on Learning Representations.
tions.
Hao Sun, Zhexin Zhang. Jiawen Deng, Jiale Cheng, and Minlie Huang. 2023. Safety assessment of chinese large language models. CORR, abs/2304. 10436.
Eric Mitchell, Charles Lin, Antoine Bosselut, Chelsea Finn, and Christopher D Manning. 2022a. Fast model editing scale. In International Conference on Learning Representations.
Ayush K Tarun, Vikram S Chundawat, Murari Mandal, and Mohan S. Kankanhalli. 2021. Fast yet effective machine unlearning. IEEE transactions neural networks and learning systems, PP..
Eric Mitchell, Charles Lin, Antoine Bosselut, Christo- pher D. Manning, and Chelsea Finn. 2022b. Memory- based model editing at scale. In International Con- ference on Machine Learning.
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal. Eric Hambro, Faisal Azhar, Aurélien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. 2023. Llama: Open and efficient foundation language models. CoRR, abs/2302.13971
Shikhar Murty, Christopher D. Manning, Scott M. Lund- berg, and Marco Túlio Ribeiro. 2022. Fixing model bugs with natural language patches. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, EMNLP 2022, Abu Dhabi, United Arab Emirates, December 7-11, 2022. pages 11600-110 11600, Association for Computational Linguistics.
Ben Wang and Aran Komatsuzaki. 202 Gpt-j-6b: A 6 billion parameter autoregressive language model.
Yasumasa Onoe, Michael J. Q. Zhang, Shankar Pad- manabhan, Greg Durrett, and Eunsol Choi. 2023. Can Ims learn new entities from descriptions? chal- lenges in propagating injected knowledge. CoRR, abs/2305.01151.
Ben Wang and Aran Komatsuzaki. 2021b. GPT- J-6B: A Billion Parameter Autoregressive Lan- guage Model. https: //gi thub. com/kingoflolz/ mesh-transformer--taa
Jiaan Wang, Yunlong Liang, Zengkui Sun, Yuxuan Cao. and Jiarong Xu. 2023a. Cross-lingual knowledge editing in large language models.
OpenAl. 2023. GPT-4 technical report. CoRR, abs/2303.08774.
Liangming Pan, Michael Saxon, Wenda Xu, Deepak Nathani, Xinyi Wang, and William Yang Wang. 2023, Automatically correcting large language models: Sur- veying the landscape of diverse self- seff-correction strate- gies. CoRR, 0lw/2308.00388...
Peng Wang, Ningyu Zhang, Xin Xie, Yunzhi Yao, Bozhong Tian, Mengru Wang. Zekun Xi. Siyuan Cheng, Kangwei Liu, Guozhou Zheng, and Huajun Chen. 2023b. Easyedit: An casy-to-use knowledge editing framework for large language models. CORR. abs/2308.07269.
Shuofei Qiao, Yixin Ou, Ningyu Zhang, Xiang Chen, Yunzhi Yao, Shumin Deng, Chuanqi Tan, Fei Huang. and Huajun Chen. 2022. Reasoning with language model prompting: A survey. CoRR, abx/2212.0955)
Xiaozhi Wang, Kaiyue Wen, Zhengyan Zhang, Lei Hou, Zhiyuan Liu, and Juanzi Li. 2022. Finding skill neurons in pre-trained transformer based language models. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 11132-11152, Abu Dhabi, United Arab Emi- rates. Association for Computational Linguistics.
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu, 2020a. Exploring the limits

---

Ga Wu, Masoud Hashemi, and Christopher Srinivasa. 2022. Puma: Performance unchanged model aug- mentation for training data removal. In AAAI Confer- ence on Artificial Intelligence.
Ce Zheng, Lei Li, Qingxiu Dong, Yixuan Fan, Zhiyong Wu, Jingjing Xu, and Baobao Chang. 2023. Can we edit factual knowledge by in-context learning? ArXiv, abs/2305.1274p
Suhang Wu, Minlong Peng, Yue Chen, Jinsong Su, and Mingming Sun. 2023. Eva-kellm: A new benchmark for evaluating knowledge editing of Ilms.
Zexuan Zhong, Zhengxuan Wu, Christopher D. Man- ning, Christopher Potts, and Danqi Chen. 2023. Mquake: Assessing knowledge editing in language models via multi-hop questions.
Yang Xu, Yutai Hou, and Wanxiang Che. 2022. Lan- guage anisotropic cross-lingual model editing, ArXiv, abs/2205.12677.
Chen Zhu, Ankit Singh Rawat, Manzil Zaheer, Srinadh Bhojanapalli, Daliang Li, Felix X. Yu, and Sanjiv Kumar. 2020. Modifying memories in transformer models, ArXiv, abs/2012.00363.
Yunzhi Yao, Shaohan Huang, Ningyu Zhang, Li Dong, Furu Wei, and Huajun Chen. 2022. Kformer: Knowl- edge injection in transformer cerw-forward layers. In Natural Language Processing and Chinese Comput- ing.
A Implementing Details
Since the ZsRE dataset adopts the NQ dataset to evaluate the locality, here, we use a T5-XL model (Raffel et al., 2020b) finetuned on the NQ dataset as the baseline model. As to the GPT- J (Wang and Komatsuzaki, 2021b), we use the original pre-trained version to test the locality's zero-shot results. As several original implementa- tions do not support both architectures, we have re-implemented them to accommodate both models. We re- ce-imllemented some original implementations to support both models. However, our empirical findings suggest that ROME and MEMIT are only suitable for decoder-only models like GPT-J, so we have not reported results for T5-XI.
Yunzhi Yao, Peng Wang. Shengyu Mao, Chuanqi Tan, Fei Huang, Huajun Chen, and Ningyu Zhang. 2023, Knowledge rumination for pre-trained language mod- els. CORR, 2ls/2305.08732.
Michihiro Yasunaga, Hongyu Ren, Antoine Bosselut, Perey Liang, and Jure Leskovec. 2021. QA-GNN: Reasoning with language models and knowledge graphs for question answering. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Hu- man Language Technologies, pages 535-546, Online. Association for Computational Linguistics.
Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher Dewan, Mona Diab, Xian Li, Xi Victoria Lin, Todor Mihaylov, Myle Ou, Sam Shleifer, Kurt Shus- ter, Daniel Simig, Punit Singh Koura, Anjali Srid- har, Tianlu Wang, and Luke Zettlemoyer. 2022a. OPT: open pre-trained transformer language mod- cls. CoRR, abs/2205.01068.
FT For basic Fine- Fine-Tuming (FT), follow Meng et al. (2023) re-implementation in their study, which uses Adam (Kingma and Ba, 2014) with early stopping to minimize -o [o* p). chang- ing only mipproj weights at selected layer 21. For both models, all hyperparameters follow default settings. To ensure fairesss in the experiments, we always use the unconstrained fine-tuning approach.
Xikun Zhang, Antoine Bosselut, Michihiro Yasunaga, Hongyu Ren, Percy Liang, Christopher D Manning. and Jure Leskovee. 2022b. MreaseLM: Graph REA- Soning enhanced language models. In International Conference on Learning Representations.
Zhengyan Zhang, Xu Han, Zhiyuan Liu, Xin Jiang. Maosong Sun, and Qun Liu. 2019. ERNIE: enhanced language representation with informative entities. In Proceedings of the 57th Conference of the Associa- tion for Computational Linguistics, ACL 2019, Flo- rence, Italy, July 28- August 2, 2019, Volume 1: Long Papers, pages 1441-1450 Association for Computa- tional Linguistics.
De Cao et al. (2021) develops an LSTM quence model, which employs gradient information to predict the rank-1 1 weight alterations in G. Given that the official code doesn't 't facilitate GPT-J, we re- sort to using the re-implemented version provided by Mitchell et al. (2022a) in their research. To foster an equitable comparison across both zsRE and COUNTERFACT tasks, we have taken addi- tional steps to train KE-zsRE and KE-CF models. The hyperparameters employed for training have been sourced from the default configurations pro- vided. During testing, KE presents a scaling factor to tweak the norm of the weight update, for which we adhere to the default value of 1.0.
KE
Wayne Xin Zhao, Kun Zhou, Junyi Li. Tianyi Tang. Xiaolei Wang, Yupeng Hou, Yingqian Min, Be- ichen Zhang, Junjie Zhang, Zican Dong, Yifan Du, Chen Yang, Yushuo Chen, Zhipeng Chen, Jinhao Jiang, Ruiyang Ren, Yifan Li, Xinyu Tang, Zikang Liu, Peiyu Liu, Jian-Yun Nie, and Ji-Rong Wen. 2023. A survey of large language models. CoRR, Abs/2303.18222.

---

Additional Training
Edit
Batch
Edit
Editor
Approach
Type
Edit
Area
Parameters
SERAC
YES
Sentiment
YES
External Model Modelej M Madelclannfier
Memory-based
Preserve
TKT:
NO
Sentimett
NO
Inpuc
NONE
Parameters
Addiiional-Paranctees
CaiNET
NO
Faci
YES
FFN
nenton
T-Patcher
NO
Fact
NO
FFN
N neuron
KT
YES
Fact
YES
FFN
Modelngger Le Loupl M Modehsger
Meta-lcarning
MEND
YES
Fact
YES
FFN
L mip
Modify
KN
NO
Fact
NO
IEN
L newron
Parameters
Locule and Editt
ROME NO
Fact
NO
FFN
mippres
MEMIT
NO
Fact
YES
FFN
L mippenj
Table Comparisons between several existing model editing approaches. "Additional Training" refers to whether the methods need training before conducting specific edits. "Edit Type" refers to the format the method can edit. "Batch Edit" refers to editing multiple target knowledge simultaneously. "Editor Area" refers to the specific region of the LLMs that the methods aim modify. FFN demonstrates the feed-forward module. "Editor Parameters" refers to the parameters that need to be updated for editing. I. denotes the number of layers to update. mlp is FFN and mipproo is the second linear layer in FFN. neurons denotes the key-value pair in FFN. N represents the quantity of neuron be updated within single layer.
CaliNET Dong et al. (2022) enriches the FFN by incorporating extra parameters aimed at knowl- edge editing, comprised of number of calibration memory slots. In order adapt CaliNET to the task at hand, we retain the same architecture as used in the Feed-Forward Network (FFN), albeit with reduced intermediate dimension denoted as d. This adaptation allows us to effectively apply CaliNET while managing the model's complexity and computational requirements. Regarding hyper- parameters, we implement adjustments to the FFN within the final two layers of GPT-J, while all other configurations remain consistent with the default settings.
the scope of any cached items, and if so, the coun- terfactual model uses the input and the most rele- vant edit example to make a prediction.
In alignment with the original paper, we use publicly available Huggingface implementa- tions and checkpoints for all experiments. For the SERAC scope classifier model, we adopt d
(Sanh et al., 2019) across all models and experimental settings. For the counterfactual model, we employ T5-smal1 (Raf- fel et al., 2020b) for the T5-XL implementation and archi techiittt/tptt111900 (available here?) for the GPT-J implementation. All hyperparameters for training and test-time inference are derived from default configurations.
MEND Mitchell et al. (2022a) develop an ef- ficient method for locally editing language mod- els using just single input-output pair. Essen- tially, MEND employs technique to manipulate the gradient of fine-tuned language models which leverages low-rank decomposition of gradients. The hyperparameters follow default settings, with the exception of several experiments conducted on GPT-J. Specifically, we adjust the optimizer from Adam to AdamW.
Similar to T-Patcher, in ruto-regressive model (like GPT-J) training, we only consider loss at the output positions.
KN For Knowledge Neuron (Dai et al., 2022), we follow Meng et al. (2023) re-implementation in their study. The method begins by identifying neu- rons that are closely associated with knowledge ex- pression. This selection is made through gradient- based attributions, which effectively highlight the neurons that have strong influence the model's output. After these critical neurons are identified, the method modifies the projection layer of the ferw-forward network (denoted as mlpproy) specif- ically the rows corresponding to the selected neurons. This modification involves adding scaled embedding vectors to the current values, effectively adjusting the model's behavior in a targeted man- ner. Specifically, they amplify knowledge neurons
SERAC Mitchell et al. (2022b) presents a method for model editing, named MEME (Memory-Bate -ased Model Editing), which stores ed- its in an explicit memory and learns to reason over them to adjust the base model's predictions needed. The system uses an explicit cache of user-provided edit descriptors (arbitrary utterances for language models), alongside a small auxiliary scope classifier and counterfactual model. The scope classifier determines if the input falls within
(8)
Pnttps://huggingfacce http:://uugnnnaaaaecccooccttteee

---

by doubling their activations. Similar to FT. all hyperparamelerss are adopted from default configu- rations(See code²)
We test the ability of MEMIT using their code and all hyperparameters follow the same default settings. For GPT-J, we choose = 3. 4, 5, 6, 7, 8, and covariance statistics are collected using 100,000 samples of Wikitext. For GPT-NEOX- 20B, we select R 6, 7, 8, 9, 10, and covariance statistics are collected from over 50,000 samples of Wikitext.
T-Patcher The method proposed by Huang et al. (2023) offers way to alter the behavior of transformer-based models with minimal changes. Specifically, it adds and trains small number of neurons in the last layer. This approach effectively provides means for fine-tuning model behavior with less computa- tional demand than comprehensive retraining. freezes all original parameters and adds one neu- ron (patch) to the last FFN layer for one mistake. And they train the patch to take effect only when encountering its corresponding mistake. For T5- XL implementation, all hyperparameters follow the same default settings as Bart-base".
Network (FFN)
IKE IKE (Zheng et al., 2023) defines three types of demonstration formatting templates including (i)copy, (ii)update, (iii)retain, which guide LMs to edit knowledge facts by in-context learning (ICL). As there are no parameter modifications, IKE is applicable any existing LLMs.
In alignment with the original paper, we choose k-NN examples from the training cor- pus( 10000 size). The demonstrations are encoded by all 1A1-MinnLM--6-v2 Following the default set- ting, we set k to 32(See code?).
Furthermore, in the auto-regressive model (like GPT-J), the model may make multiple mistakes in one example. Therefore, for an example where the model makes mistakes, we only consider errors generated by the model at the output positions. Following the settings of the original paper, we add up 5 patches for edit example. Formally, for edit example (Wee Me) in auto-regressive model, the actual input is given by s Tee + Ye and the patched model's output is pe. Le is defined as:
B Dataset Details
B.1 Basic DataSet
ZsRE (Levy et al., 2017) is Question Answering (QA) dataset using question rephrasings generated by back-translation as the equivalence neighbor- hood. COUNTERFACT (Meng et al., 2022) is a more challenging dataset that accounts for coun- terfacts that start with low scores in comparison to correct facts. It constructs out-of-scope data by substituting the subject entity for proximate subject entity sharing a predicate. This alteration enables us to differentiate between superficial word- ing changes and more significant modifications that correspond to a meaningful shift in a fact. We fol- low previous data split (De Cao et al., 2021; Meng et al., 2022; Mitchell et al., 2022a) to evaluate all the models on the test set. For models requiring training. we utilize the training set. Following prior work (Mitchell et al., 2022a,b), ral Questions (NQ; Kwiatkowski et (2019)) as out-of oft-of-scope data to evaluate locality.
N
4. = Σ L log(p:) (((21e((z2)
(6)
ROME ROME, proposed by Meng et al. (2022), conceptualizes the MLP module as straightforward key-value store. For instance, if the key represents a subject and the value encapsulates knowledge about that subject, then the MLP can reestablish the association by retrieving the value that corresponds to the key. In order to add new key-value pair, ROME applies rank-one modifica- tion to the weights of the MIP, effectively "writing in" the new information directly. This method al- lows for more precise and direct modification of the model's 's knowledge. We directly apply the code and MLP weight provided by the original paper and keep the default setting for hyper-parameters.
the Natu-
B.2
Dataset Construction for Portability Evaluation
B.2.1 One hop
MEMIT MEMIT (Meng et al., 2023) builds upon ROME to insert many memories by modi- fying the MLP weights of range of critical layers.
The construction can be seen in Figure 6. To en- sure that the original model(fe) has seen the triple (s o) during the pre-training process, employ
http:://ggttbo.cooo/leeeeeeellle "https:/ggthuu..com/ wwwp:euwwwwooooooooeoooooooooooo "htps://rrmme
htnps::/meemit.bbulabb. hhtps://gihhuh.cmm///UUUhpiiiiip
https:///omme.aallaaiiifnf inulab.info/

---

Type
Edit Descriptor
Portability Question
In what living being can PRDMI6 be found!
In what living being can PR domain containing 16 be found?
Subject Replace
When was Liu Song dynasty abolished?
When was the end of the Former Song dynasty?
Table tennis was formulated in?
ping pang, that originated in ?
Reversed Relation
What is Wenxiu's 's spouse's name?
Who is the wife/husband Wenxi Emperor?
In which city is the headquarters of the company that made the Volvo B B12M?
One-hop Reason
What company made Volvo B12M?
Table 7: Example of portability dataset.
link prediction to predict given (s, r, ?). If the tail entity is present the Top-10 logits, we consider the model to have prior knowledge of this triple. In other words, if the model has sufficient portability, it can correctly answer new questions based on the subject and the triplet.
in Wikidata, we used GPT-4 to generate synonyms for the original subject. This process ensures that the evaluation accurately reflects the model's ca- pability to handle various subject representations, contributing to more comprehensive understand- ing of its performance.
We select data points measure the perfor- mance of the model's portability. The symbolic representation of the portability dataset is as fol- lows:
B.2.3 Reversed Relation
In an editing instance, the attributes of the target entity can also change. For instance, in the edited instance:
"Who is the father of Nebaioth? Ishmael Babur." When answering the question "Who is the son of Babur?" it should be answered based on the new fact after editing, which is Nebaioth. Certain types of relations may not be as effective for evaluation. Let's consider a hypothetical sce- nario where we change the location of the Eiffel Tower to Rome proposing a valid reversed ques- tion in such a context would be challenging. Con- sequently,
Dport - {GPT4 (8, o Top-10(fa (s, ?)))
To guide GPT-4 in producing the desired ques- tion and answer, we employ few-shot manual demonstration as a prompt (See Table 10). In addi- tion, intersect the data filtered by T5-XL and the data filtered by GPT-J obtain the final portability dataset. The GPTJ model achieves a link prediction score of ZSRE: 72.99 and COUNTERFACT: 69.78, while the T5 model achieves link prediction score of ZSRE: 83.90 and COUNTERFACT: 84.81. ensures that the models possess prior knowledge about this triple.
carefully handpicked all relations in the ZsRE dataset that could be reversed, such as one-to-one relations, and selected related ques- tions through keywords (such as spouse, wi fe, mother, father, brother, sister) screening. This allows us to maintain the integrity and rele- vance of the evaluation process, thus ensuring more reliable results. To guide GPT-4 in producing the desired reversed question, we employ few-shot manual demonstration prompt (See Table 11).
As result, we select some data instances from the ZsRE and the COUNTERFACT dataset. The description of the data is shown in Table 8.
Subject Replace
Inverse Relation
One hop
7&RE COUNTERFACT
385
B.3 Dataset Construction for Locality Evaluation
213
1,031
Table 8: Statistics of portability dataset.
B.3.1
Other Attribution
Other attributes of the subject updated should re- main the same before editing. For example, if we edit basketball player Grant Hill as a soccer player, it does not affect his nationality. Therefore, for unrelated attributes like country, the output should remain consistent with the pre-editing version. We modified the COUNTERFACT dataset by using the Wikidata API to traverse all relationships of a subject and randomly select an unrelated relation-
B.2.2 Subject Replace
We replace the question's 's subject with an alias or synonym to test generalization on other descrip- tions of the subject. We used two approaches to construct this dataset. For subjects that could be found in Wikidata, we replaced the original subject with the alias from Wikidata (Field Name: Also known as). 2. For subjects that could not be found

---

New Question
Humphrey in during uniornity?
University Michipn, locate
fin artor
Text Completion
Aboed
Answer
GPT4 15.
arbor
Watts Hampphey
Manual Prompt for Data Construction
In the following statement, Altered Answer represents the changed factual knowledge, When the anower is changed, some related fact should aho be changed. You should recall neated reletion assodittd with the Altered Answer. Then generate questions and answers hased recalled
and subject.
Queetion: QUESTION Subyee:: suBJECt, Altened Answer Recalled Relabion: Question: NEW QUESTION, New
RECALLED RELATION.
ANSWER.
Demonstrations
Figure 6: Dataset construction procedure to generate portability part (Q.A) with GPT4,
Type
Edit Descriptor
Locality Question
Grant Hill is a professional
Which country does Grant Hill represcut in sport? (relation: country)
Other Attribution
The language of La Dispute was
What genre dees La Dispute belong to? (relation: genre)
Gleb Kotelnikov is a native speaker of
What is the gender of Gleb Kotelnikov? (relation: ser or gender)
Windows 98 was product of
Windows 98 was product of IBM. Windows Media Center, developed by
Distract Neighbor
The language of Goodfellas is
The language of Goodfellas is Tamil. The language of Titanic is
Table 9: Example of locality dataset.
ship and tail entity data sample. We provide (8, lother) to GPT-4 to generate question, and the answer to this question corresponds to the respec- tive tail entity. As result, we modify 804 data instances from the COUNTERFACT dataset.
GPT-J with 100 samples in the COUNTERFACT dataset. Afterward, we test the performance of the continuously post-edit models on PIQA, using ac- curacy as the selected metric, which defined as
B.3.2 Distract Neighbor
acc Qccks Th, a
(7)
Following Hoelscher- Ohermmiir al. (2023b), we modify the neighborhood prompt in COUNTER- FACT dataset by prepending the model edit. For example(See Table 9), if the original prompt is "Windows 98 was a product of the modified prompt would be "Windows 98 was product of IBM. Windows Media Center, developed by _". It measures whether the model editing technique has resulted in significant side effects on the model it- self due ederindg As a result, we select 804 data instances from the COUNTERFACT dataset.
where ak₂ is the option with the least perplexity of the post-edit model, Q(Ck. , Qk (h is if Ohp Okg and 0 otherwise.
B.3.3 Other Task
We select commonsense tasks here assess the post-edited model's performance on other tasks. Given question 4. multiple-choice commonsense reasoning aims to select the correct answer 0g C A provided with an optional context Physical In- teraction QA (PIQA ((Bisk et al., 2020)) is 2- way multiple-choice QA task testing physies rea- soning about objects. We evaluate the post-edit model on the PIQA dataset to reflect the impact of different model editing techniques on the perfor- mance of other downstream tasks. Specifically, For each model editing technique, we sequentially edit

---

Prompt
In the following statement, 'Altered Answer' represents the changed factual knowledge. When the answer is changed, some related facts should also be changed. You should recall related relation associated with the 'Altered Answer'. Then generate questions and answers based these recalled concepts and 'Subject'.
Question: What university did Watts Humphrey attend?
Subject: Watts Humphrey
Altered Answer: University of Michigan
Recalled Relation: (University of Michigan, locate in, Ann Arbor)
New Question: Which city did Watts Humphrey live in during his undergraduate studies?
New Answer: Ann Arbor in Michigan State
Question: Windows 10, developed by
Subject: Windows 10
Altered Answer: Google
Recalled Relation: (Sundar Pichai, ceo of, Google)
New Question: Who is the CEO of the company that develops the Windows 10 operating system?
New Answer: Sundar Pichai
Question: In Kotka, the language spoken is?
Subject: Kotka
Altered Answer: French
Recalled Relation: (French, evolve from, Romance)
New Question: What language did Kotka's official language evolve from?
New Answer: Romance
Question: Armand Trousseau's area of work is?
Subject: Armand Trousseau
Altered Answer: jazz.
Recalled Relation: (Miles Davis, genres, jazz)
New Question: Armand Trousseau formed band during college, they are all fans of?
New Answer:
Miles Davis
Table 10: Prompt for dataset construction zsRE & COUNTERFACT portability dataset. Demonstration examples are manually constructed. For each data instance, we provide Question, Subject, and Altered Answer to generate portability data.

---

Task Prompt
Please generate the Inverse Question(For example, and B in father-son relationship. In the original question, it says "who is the father of B? Answer is A". You should ask who the son/daughter of A, so answer is B.) here are some examples:
ZSRE
Q: Who is Claire Clairmont's sister? A: Marian Clairmont Inverse Question: Who is Marian Clairmont's 's sister?
Q: What was the name of the father of Jane Seymour? A: Richard Seymour Inverse Question: Who is the son/daughter of Richard Seymour?
Q: What is Elizabeth Grey, Countess Kent's spouse? A: Henry Grey, 1st Duke of Suffolk Inverse Question: Who was Henry Grey, 1st Duke of Suffolk married to?
Q: Who is listed as Leonor, Princess of Asturias's father? A: Leonor III of Spain Inverse Question: Who is the son/daughter of Leonor III of Spain?
Table 11: Prompt for inversed relation dataset construction on zsRF, we provide Question and Answer to generate inversed question.


Reviews:
[{"review_id": "adX6g0RXRF", "reviewer": "Reviewer_yadK", "summary": "", "questions": "Question A: Do the ROME and MEMIT work for the encoder-decoder or maybe the encoder-only model family? Since in Table 1, it is marked with ‘-’. \n\nQuestion B: Why are scores in Table 1 for KE<->GPT-J very low? Similarly, for the CaliNet, KN as well.\n\nQuestion C: In line 341, ROME degrades its performance, but what would be the reason for the performance degradation?\n\nQuestion D: In Figure 3, what is the y-axis? It is not very clear from the figure alone.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for recognizing the value of our work. Your comments are highly aligned with our paper, and we hope the following comments could answer your questions.\n\nComments:\n> Effectiveness of editing techniques in encoder-only architecture models\n\nSome recent work has explored editing in encoder-only architectures, such as RoBERTa and BERT, demonstrating the effectiveness of these techniques. The results of the paper \"Editing Factual Knowledge in Language Models\" [De Cao 2021] are as follows:\n| Model      | BERT |  \n| ----------- | ----------- |  \n| Reliability   |   98.80     | \n| Generalization     |   82.69     | \n|  Locality  |   98.14      | \nIn our paper, we skip this part for the following reasons:\n1. The performance of current editing methods is excellent for models with fewer than 1B parameters. Since the topic of this study is **editing techniques on large language models**, we focus on encoder-decoder and decoder-only models with a large number of parameters. However, encoder-only models like BERT and RoBERTa usually have fewer than 1B parameters.\n2. **Encoder-only models are primarily designed for classification tasks**. Considering the editing task and future developments, we focus on generation-based models.\n\nComments:\n> Batch and sequential experiments for all the editing techniques\n\nIt is worth noting that **not all methods support batch editing**. As indicated in Table 6 in the appendix, the batch editing methods include SERAC, CaliNET, KE, MEND, MEMIT, and FT-L. However, in the main experiment, we observed that KE and CaliNET exhibited poor editing performance on a model with a large number of parameters. Since our goal was to provide guidance on selecting the most suitable editing method for a given situation, we chose the remaining editing methods for evaluation and performed sequential editing accordingly.\n\nComments:\n> The reasoning for the performance degradation after editing 100 instances\n\nThanks for the reminder. Due to the limited space on the page, we cannot provide a detailed explanation. We will include the explanations in the Appendix. Here are the reasons for the degradation:\n\nFor the method that changes the original model's parameters, each edit makes the updated matrix far from the original matrix.\nSpecially, \n- **KE&MEND**: MEND did not place restrictions on weight updates in the model MLP and update the weight based on the editor trained on the original parameters, thus causing serious damage to the model after 10 edits\n- **ROME & MEMIT**: Through the hard/soft equality constraint for weight update, ROME, MEMIT maintained excellent reliability, generalization, Locality, etc., within 100 samples. However, the computation of the new $\\hat{W}$ is also based on the covariance  $C$ and $C$ is computed on the original model. After the editing, the $C$ is not suitable for the post-edit model, leading to the degradation after editing 100 instances.This turning point reflects the ability of different editing methods to accommodate editing samples.\n\nQuestion A\n>  Do the ROME and MEMIT work for the encoder-decoder and encoder-only models?\n\nIn our experiments, it was found that ROME&MEMIT could not work on models on the encoder-decoder architecture.\nHere are two reasons:\n1. Specifically, both methods make use of matrix inversion to solve the least squares constraint. For example, rome:\n$\\Lambda=(v_*-Wk_*)/(C^{-1} k_*)^{T} k_*$,\nMEMIT: $\\Delta=R K_{1}^{T}(C_{0}+K_{1} K_{1}^{T})^{-1}$,the initial matrix $W_0$, requirements MLP is reversible. \nHowever, in our experiment, similar with the OPT-13B model, we find that 'wo.weight' in T5 is a singular matrix, which does not meet the condition of matrix inversion. \nTo this end, we have made the following two attempts:\n- Find the pseudo-inverse of the matrix, however, because $w_o$ has a higher dimension (greater than 10000), it cannot afford its time cost.\n- Least squares approximation, for example $\\ Delta (C_ {0} + K_ {1} K_ {1} ^ {T}) = RK_ {1} ^ {T} $using ` torch. Linalg. LSTSQ ` approaching $(C_ {0} + K_ {1} K_ {1} ^ {T}) $. This makes ROME and MEMIT completely incapable of editing (1-2% reliability). \n\n2. ROME and MEMIT conduct causal analysis based on the decoder-only model, which uses the unidirectional attention mechanism. The encoder-decoder model and encoder-only model, on the other hand, utilize the bidirectional attention mechanism. This model can predict the masked token based on the given text.\nHowever, when we changed the representation of the last subject token, we found that the encoder was still able to predict the changed token by the context surrounded, rendering the edit ineffective.\n\nOverall, We assume that ROME and MEMIT is not able to work for encoder-only and encoder-decoder model.\n\nQuestion B\n>  Why are scores in Table 1 for KE<->GPT-J very low? Similarly, for the CaliNet, KN as well.\n\nThe performance of KE on large models is not good. \nIt is a common phenomenon in other works [Mitchell 2022, Huang 2023]. The reason can be attributed to the huge parameters for the large language model, which is difficult to fit.\n**KN** first conduct a gradient attribution to locate the model and we find it is difficult to locate for the GPT-J size model.\n**CaliNET** is a previous work of T-Patcher, it only adds a few tokens and do not apply the constraint loss like T-Patcher. We found the added token usually failed to be activated when answering the question, which lead to the edit failure.\n\nQuestion C\n\n> In line 341, ROME degrades its performance, but what would be the reason for the performance degradation?\n\nWe list the reason in the second comment.\n\nQuestion D\n> In Figure 3, what is the y-axis? It is not very clear from the figure alone.\n\nThe y-axis in Figure 3 represents average accuracy, and thank you for pointing this out. We would make it clear.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The reasons to accept are as follows:\n1. The presentation of the problem is well-written & visualized with the notations, current methods, experiment settings, and results. \n2. The manuscript covers the essential and necessary problem of editing. The insights are definitely helpful for the NLP community. \n3. The set of ME techniques in Table 1 covers standard editing techniques from two different paradigms. It shows an insightful observation over two different datasets and three metrics. \n4. In the Appendix, the implementation details can help in reproducibility. Additionally, the wall clock time and GPU consumption details are an add-on to show the analysis. \n5. The manuscript is a good combination of review-research papers. The work showcases the previous works, their challenges, and their comparison. It also adds the contributions of subject-replace, reverse-relation, and one-hop.", "reasons_to_reject": "The reasons for the rejection are as follows:\n1. The models representing the encoder-only family are not present. However, the insights available for the other two: encoder-decoder & decoder-only, in the manuscript are enough to show the insights and observations.\n2. A comparison between the batch and sequential techniques for all the editing techniques would also be a great insight. \n3. Insights are missing and the observations are directly reported. For instance, the reasoning for the performance degradation after editing with 100 instances is not available in the manuscript.", "questions_to_authors": "Question A: Do the ROME and MEMIT work for the encoder-decoder or maybe the encoder-only model family? Since in Table 1, it is marked with ‘-’. \n\nQuestion B: Why are scores in Table 1 for KE<->GPT-J very low? Similarly, for the CaliNet, KN as well.\n\nQuestion C: In line 341, ROME degrades its performance, but what would be the reason for the performance degradation?\n\nQuestion D: In Figure 3, what is the y-axis? It is not very clear from the figure alone.", "paper_topic_and_main_contributions": "The manuscript describes the model-editing (ME) problem, where we selectively update the model’s parameters such that the rest of the parameters are kept intact. The main objective is to update the knowledge for given facts/information, while the other information is intact. The work has shown insights into problems, methodologies, and future work in the ME domain. \n\nThe work highlights the notations and definitions, recent works, an empirical analysis of the methods, and the challenges.  In total, two datasets and models are experimented with ten different editing techniques. An ablation and set of ME techniques have covered both the paradigms of preserving Models’ parameters and Updating Models’ parameters.", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Typos_Grammar_Style_and_Presentation_Improvements": "1. There are certain places where the spaces need to be added. Like lines (170, 177, 1109, and rest). Tools like Grammarly can help in fixing the spaces and grammatical issues.\n2. A comparison or diagram that explains the examples of reliability, generalization, and locality would help in the presentation aspect. \n3. Footnotes 2 and 3 on page 13 are of different fonts."}}, {"review_id": "uVEPaccN72", "reviewer": "Reviewer_5Cwu", "summary": "", "questions": "This domain isn't very familiar to me, so I'd like to ask about how the approaches discussed would benefit expert and non-expert target audiences?", "limitations": "", "rating": 4, "confidence": 1, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your valuable suggestions.\nFor your concern about the multi-edits, **we have conducted both batch editing and sequential editing in Section 4.3** and we list the performance of current editing techniques.\n\nFurthermore, in terms of the benefits of our work, model editing is a rapidly developing field, and there will be more opportunities in the future. Even when we train highly capable models, it remains unclear how to keep them up to date or revise them when they make errors. Ideally, when the state of the world changes, we should be able to update models in a way that is computationally less expensive than training a new model. Although there are some existing methods (as discussed in our paper) that can partially address this problem, these methods still have limitations.\n\n**For experts**, our work highlights urgent issues that need to be addressed to facilitate the exploration of knowledge updating in language models. **For non-expert audiences**, our work provides a list of the advantages and disadvantages of current methods, aiding them in selecting the most suitable one for downstream usage.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The authors provide a detailed categorization and discussion of model editing techniques. I appreciate the discussion of harmful language generation as well.", "reasons_to_reject": "The edits discussed are single-edits, so some discussion on how to evaluate multiple-edits would be good to include. ", "questions_to_authors": "This domain isn't very familiar to me, so I'd like to ask about how the approaches discussed would benefit expert and non-expert target audiences?", "paper_topic_and_main_contributions": "This paper discusses the problems, methods, and opportunities related to model editing for LLMs. The authors also introduce a new benchmark dataset and 'Portability' to facilitate a more robust evaluation of models. ", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "SHpi6z89rH", "reviewer": "Reviewer_yYDP", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 2, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your valuable reviews, we hope the following comments could address your concerns.\n\n> The scope is currently limited to factual knowledge editing. Expanding the discussion to other editable attributes could increase impact.\n\nThis is a great comment. \nModel editing is far from factual knowledge editing and there are several works focused on emotions and writings. \nOur paper select factual editing as the starting point to analyze existing work and reflect existing problems.\nBut **the issues summarized in our work are not limited to fact**; they apply to other editable attributes as well. \nFacts can serve as a scaffold to evaluate and address these broader challenges.\nMeanwhile, other editable attributes is a developing field that requires future research. \n\n> Authors reach conclusion that MEND's performance on GPT-J is less impressive, which may be attributed to the model size. However, no experiment result on OPT-13B and GPT-NEOX-20B with MEND method available. The conclusion is not convincing.\n\nThanks for your constructive comments, and we have conducted more experiments. \n**In our experiments and previous works' results (like ROME, MEMIT), MEND's result on GPT-J(6B) is usually worse than small size model like GPT2-XL(1.7B) and T5_XL(2.8B)**.\nHowever, we recently conducted experiments on Llama2-7B and Llama, and we found that MEND performs well on these models.\nDespite having the same size, the performance of MEND on Llama is worse than that on Llama2.\nThe results is here:\n| Model      | Llama2-7B | Llama-7B\n| ----------- | ----------- | -----------\n| Reliability   |   94.24    | 89.35\n| Generalization     |   90.27     | 82.15 \n|  Locality  |   97.04     | 96.62\n\nWe are also conducting the results of OPT-13B and GPT-NEO-20B, but the models are too large and training MEND's hyper network requires several days and we cannot report this results currently.\nFurthermore, we observed fluctuations in the performance of the meta-learning method during our experiments. To gain a deeper understanding of these fluctuations, we will conduct a more thorough analysis. Thank you for bringing this to our attention, and we appreciate your understanding.\n**We would modify this part to make it correct after the we get the results of OPT-13B and GPT-NEO-20B**. \n\n\n> Providing some sample results from the comparative experiments could better illustrate the insights gained. Lack of qualitative analysis.\nRather than just reporting performance across metrics, including examples of bad cases, analyzing patterns in these failures, and providing intuitive explanations for why they occurred could strengthen the paper. This deeper perspective enables us to really understand the conditions under which certain editing techniques break down.\n\nThis is a great problem. \nIn our Analysis part, **we explore in detail how existing methods perform on different types of knowledge**, such as the reversed relation, distract neighbor and so on. \nAnd from this part, we can found different methods demonstrates different performance on different type of knowledge.\nFor example, SERAC can deal with the given knowledge perfectly but capacity of the small counterfactual model limit its performance on the reversed relation.\nMeanwhile, as mentioned by the reviewer yadK, we propose a detailed explanation for the failure of some methods and please check our explanation for reviewer yadK.\n\nImportantly, we conduct a **bad case analysis** here to give a more deeper perspective.\nFirst, we found that **KN and SERAC** tend to generate the tokens repeatedly like ''9thththththththththth'.\nAlso, we can find a similar pattern for each method and we list one case here.\n| Method    | Bad case |  Label | Output\n| ----------- | ----------- |  ----------- | -----------\n| MEND   |   Which was the production company for Peepli Live?    | Peepli Entertainment | \\npli Entertainment\n| SERAC     |   Which lady gave birth to Leto? Fausta - Leto's mom's who?     | Fausta | Estusta\n|  ROME  |   Which was the gender of Dena Feingold?    | male | nobody\n\n1. MEND usually tend to miss the first word and prefer to generate some meaningless token like'\\n'. Actually, they seems to learn the new fact but failed to generate it.\n2. SERAC suffers from the small model. The small model is not able to learn the new fact well and cannot predict the answer correctly.\n3. ROME usually tend to generate answers that are unrelated with the answer. In our experiment, we found the computation of this v* vector for updating the model's weight may not success in this error case.\n\nWe are doing more deeper analysis and we would add it during the revision.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The topic is highly relevant and timely given the rapid progress in LLMs recently. Editing their behavior efficiently is an important challenge and meaningful work to generate content which follows commonsense and up-to-date.\n2. The manuscript is clearly structured and well-written overall. The abstract and introduction motivate the gap this analysis aims to fill.\n3. Comparing editing models under uniform experimental conditions is a worthwhile contribution. The proposed benchmarks enable more robust evaluation.\n4. The focus on practical implications for method selection is important for guiding real-world applications.", "reasons_to_reject": "1. The scope is currently limited to factual knowledge editing. Expanding the discussion to other editable attributes could increase impact.\n2. Authors reach conclusion that MEND's performance on GPT-J is less impressive, which may be attributed to the model size. However, no experiment result on OPT-13B and GPT-NEOX-20B with MEND method available. The conclusion is not convincing.\n3. Providing some sample results from the comparative experiments could better illustrate the insights gained. Lack of qualitative analysis.\n4. Rather than just reporting performance across metrics, including examples of bad cases, analyzing patterns in these failures, and providing intuitive explanations for why they occurred could strengthen the paper. This deeper perspective enables us to really understand the conditions under which certain editing techniques break down.", "questions_to_authors": "", "paper_topic_and_main_contributions": "This paper provides a systematic analysis and comparative evaluation of different techniques for editing large language models (LLMs). This paper addresses the lack of comprehensive comparative analysis and evaluation of different techniques for editing large language models (LLMs). Existing works have not assessed these methods in uniform experimental conditions. Authors propose a new evaluation metric called Portability to gauge the generalization ability of different model editing methods.\n\nThe main contributions are:\n1. provides an exhaustive overview of current model editing methods, organizing them into categories based on whether they preserve or modify model parameters.\n2. identifies a lack of comparative analysis across editing techniques under consistent experimental conditions.\n3. proposes more comprehensive benchmark evaluations on portability, locality, and efficiency to enable a more robust assessment.\n4. conducts controlled experiments for impartial comparison of the strengths and weaknesses of each method.\n5. aims to offer guidance on selecting the most appropriate editing method for a given task or context based on the comparative analysis.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: **quality, clarity, originality**

The paper and its  handling of the topic of model editing is high quality. The writing was very clear and provided clarity in a number of aspects of dealing with ways to edit language models and impacts on performance. The reviewers appreciated the depth of the work as well as how thorough the authors were in handling the topic. The questions that touched on encoder only architectures were answered and context provide to other literature.

The originality of the work in creating a uniform field to explore the language editing approaches expands on the novelty and provides insight for readers and field. Benchmarks will prove useful for others to work to emulate or build upon. 

**significance**
The topic itself is very important to the NLP community especially as more and more model reuse is accelerating. The work is significant as it provides insight into the model editing area and also provide data benchmarks that can be reused by others. This serves as a great resource for the field. 

**notes for authors**

Please do make sure that the salient points such as providing an overview on encoder models as well as descriptions/discussioins of failures on the editing are documented in the paper itself.
Decision: Accept-Main