Reasons and Solutions for the Decline in Model Performance after Editing
Authors: Xiusheng Huang
Jiaxiang Liu
Yequan Wang
Kang Liu
Venue: NeurIPS 2024 poster (2024)
Keywords: Knowledge editing
performance evaluation
Primary area: natural_language_processing

Abstract: Knowledge editing technology has received widespread attention for low-cost updates of incorrect or outdated knowledge in large-scale language models. However, recent research has found that edited models often exhibit varying degrees of performance degradation. The reasons behind this phenomenon and potential solutions have not yet been provided. In order to investigate the reasons for the performance decline of the edited model and optimize the editing method, this work explores the underlying reasons from both data and model perspectives. Specifically, 1) from a data perspective, to clarify the impact of data on the performance of editing models, this paper first constructs a **M**ulti-**Q**uestion **D**ataset (**MQD**) to evaluate the impact of different types of editing data on model performance. The performance of the editing model is mainly affected by the diversity of editing targets and sequence length, as determined through experiments. 2) From a model perspective, this article explores the factors that affect the performance of editing models. The results indicate a strong correlation between the L1-norm of the editing model layer and the editing accuracy, and clarify that this is an important factor leading to the bottleneck of editing performance. Finally, in order to improve the performance of the editing model, this paper further proposes a **D**ump **for** **S**equence (**D4S**) method, which successfully overcomes the previous editing bottleneck by reducing the L1-norm of the editing layer, allowing users to perform multiple effective edits and minimizing model damage. Our code is available at https://github.com/nlpkeg/D4S.

Reasons and Solutions for the Decline in Model Performance after Editing
Xiusheng Huang" 1,2,3 Jiaxiang Liu" 1,2, Yequan Wang Kang Liu 122
The Key Laboratory of Cognition and Decision Intelligence for Complex Systems, Institute of Automation, Chinese Academy of Sciences
School of Artificial Intelligence, University of Chinese Academy of Sciences *Beijing Academy of Artificial Intelligence, Beijing, China huangzkusheng22200ia. ac. cn, tshwangyeeuan@@maill com, kliu@nlpr ac.
ucas acc cn,
Abstract
Knowledge editing technology has received widespread attention for low-cost up- dates of incorrect or outdated knowledge in large-scale language models. However, recent research has found that edited models often exhibit varying degrees of perfor- mance degradation. The reasons behind this phenomenon and potential solutions have not yet been provided. order to investigate the reasons for the performance decline of the edited model and optimize the editing method, this work explores the underlying reasons from both data and model perspectives. Specifically, I) from a data perspective, to clarify the impact of data on the performance of editing models, this paper first constructs Multi culli-Question Dataset (MQD) to evaluate the impact of different types of editing data model performance. The performance of the editing model is mainly affected by the diversity of editing targets and sequence length, determined through experiments. 2) From model perspective, this article explores the factors that affect the performance of editing models. The results indicate strong correlation between the I.--oorm of the editing model layer and the editing accuracy, and clarify that this is an important factor leading to the bottleneck of editing performance. Finally, in order improve the performance of the editing model, this paper further proposes a Dump for Sequence (D4S) method, which successfully overcomes the previous editing bottleneck by reducing the Ll-norm of the editing layer, allowing users to perform multiple effective edits and minimizing model damage. Our code is available at bttps://gihhbbccm//pp//////
Introduction
Large-scale language models (LLMs) have demonstrated exceptional performance in NLP tasks (Huang al. 2021 2022). However, knowledge continues evolve. LLMs inevitably contain incorrect or outdated information. Due to their vast number of parameters, directly fine-tuning the model would require substantial amount of computational resources [Gupta et al., 2023). As result, knowledge editing techniques have emerged as a low-cost and effective method for updating model's knowledge (Yao et al.) 2023). These techniques involve modifying small number of the model's 's parameters to update its internal knowledge [Ding et al., (203). However, growing evidence suggests that altering model parameters can have a negative impact on the model's performance. The
"Equal contribution. Corresponding authors.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

specific reasons behind this phenomenon remain unclear, and corresponding optimization methods are currently unavailable.
Previous research has investigated the performance degradation of edited models [Wang et al. 2023
Mazzia al. 2023 Specifically, [Hase al. [2024] found that edited models suffer from catastrophic forgetting, where they forget previously edited samples. Additionally, Gu et al. (2024) showed that edited models exhibit significant performance declines on downstream tasks, which severely hinders the practical applicability of knowledge editing techniques. However, these studies failed to identify the underlying causes of performance degradation in edited models and did not propose optimization methods to mitigate this issue (Zhang et al. 2024
Knowledge Editing Task
Data Perspective
Triplet
Free-text
Entiyy! relation
ZSRE
b Datasel construction
C Sequence longth
Dataset
Factual
MullC
Purfor mance evalustion
Event
ELKEN / EVEDIT
Osputt
Commoneense
PEP3k/ /0Q
OKD
or
Oupu BB
Pro
Post
Knowledge Editing
Evaluation
Model Perspective
Edit sequence
Dump for sequence
Question Type
Bamplee
TF
MC
DG
Success mole
Sample2
Output
A B D
Entity
Event
This framework outlines the comprehensive approach to understanding the performance decline of edited models. On the left, traditional knowledge editing tasks are categorized into different types, each with distinct editing objectives: yes/no, a/b/c/d, and entity/event. On the right, our experiments
Fig. 1:
structured from both data and model perspectives. From the data perspective, we conduct three experiments:
(a) a comprehensive performance evaluation of the model, (b) the construction of a Multi culti-Queetiio Dataset (MQD), and (c) an assessment of the impact of editing different target outputs on model performance. From the model perspective, we design four experiments: (d) an evaluation of the edited model's forgetting ability, (e) an identification of the current knowledge editing method's bottleneck and an exploration of the correlation between editing probability values and parameter layer norms, and (f) a proposal of sequence editing method, which effectively enhances the performance of the edited model.
This paper examines the factors that influence model performance and optimization methods from both data and model perspectives. As shown in Figure T] from the data perspective, we evaluated the performance of edited models on multiple generalization tasks and found that the performance degra- dation of edited models is correlated with the editing objectives. We then constructed Multi- Question Dataset (MQD) with different question types, including multiple-choice, true/false, and direct gen- eration, with corresponding editing objectives of yes/no, a/b/c/d, and entity/event, respectively. By calculating the perplexity (PPL) of different editing objectives, we discovered that the larger the PPL, the more severe the performance degradation of the edited model.
From the model's perspective, we investigate the reasons behind the decline in model performance from two angles: catastrophic forgetting and the bottleneck imposed by the number of edits. Our analysis reveals a strong correlation between the accuracy of edits and the Ll-norm growth of the parameter layers after editing. To mitigate this issue, we propose a Dump for Sequence (D4S) method that regulates the explosive growth of parameter layers during the editing process. This approach effectively enhances the performance of the edited model and overcomes the bottleneck associated with the number of edits.
To the best of our knowledge, we are the first to investigate the causes of performance degradation in edited models and concurrently propose an effective sequence editing method to enhance the performance of edited models. Our contributions can be summarized as follows:

---

To investigate the impact of data on the performance of edited models, we performed evaluations across multiple tasks, revealing that the editing objective is the primary factor influencing model performance. By creating datasets with diverse question types, we established a strong correlation between the perplexity (PPL.) of the editing objectives and the performance of the edited model.
We find that the decline in edited model performance is correlated with the explosive growth of the Ll-norm of parameter layers during the editing process. which the primary cause of both catastrophic forgetting and the bottleneck imposed by the number of edits.
To enhance the performance of the edited model, we propose caching sequence edit method that leverages ((1) space complexity retain past knowledge, regulates the explosive growth of parameter layer norms, and thereby effectively improves the performance of the edited model, ultimately overcoming the bottleneck imposed by the number of edits.
2 Related Work
2.1
Knowledge Editing Datasets
The existing knowledge editing dataset can be divided into triplet form and event form. In triplet format dataset, commonsense knowledge dataset includes PEP3K and 20Q Porada et al. 2021 Gupta
al. 2023], factual knowledge includes ZsRE [Levy et al., 2017). CounterFact Meng al. 2022a
Fact Verification [Mitchell al. [2022] Calibration [Dong al. 2022]. MQuAKE [Zhong et al.
2023] and RaKE [Wei et al. [002]] J. In event format dataset, datasets with only factual knowledge,
including ELKEN Peng al. 2024 EVEDIT [Liu et al. 2024 and CKD CKD [Huang et al. 2024
2.2
Knowledge Editing Methods
The previous editing methods mainly focused on editing knowledge in the form of triples, with small amount of knowledge in the form of editing events. The methods for editing triplet forms
mainly include (1)Locate- Becoocatt-eee--BBe Then-Edit method [Dai et al. [222 Meng et al. 2022a b Li ct al. 2024
(2) Memory-bate baseed method [Mitchell et al., 2022 Madaan et 2022 Zhong a 2023 Zheng
et al. 2023]. (3) Hyper-network method [Mitchell et al.] 2021 De Cao et al. 2021 lann et al. 2023
The method for editing event forms is Self-Edit [Liu et al. 2024
2.3 Model Evaluation after Editing
The damage caused to the model by updating model parameters is unknown. [Hase et al. 2024 found that the edited model had catastrophic forgetting issues and performance degradation downstream tasks. [Gu et al., 2024] evaluated the edited model on eight downstream tasks and used multiple editing methods. It was found that the edited model exhibited varying degrees of performance degradation. However, the above methods only found a decrease in performance of the edited model, without pointing out the reasons for the performance decline. At the same time, they did not propose effective editing methods to improve the performance of the edited model.
3 Data-Specific Factors Affecting Performance
In this section, we investigate the primary data factors contributing to the decline in model perfor- mance following editing.
3.1 The Overall Performance Evaluation
To assess the performance of the edited model, we curated a diverse range of editing and evaluation datasets for testing.
The Dataset for Editing. As illustrated in Figure] we selected common sense knowledge and factual knowledge datasets as editing corpora, featuring diverse data formats such as triplets and free
text. Figure 2 we utilized the 7sRFB[Levy et al. 2017], ELKEN [LK al. 2024]. J 20Q(Porada]
et al. 202 Gupta al. 2023], and CKD[Huang al. 2024
datasets as editing corpora. Notably,
3

---

Dataset;
ZsRE
ELKEN
20Q
CKD
Porfoemanee
KN
Perfomance
ROME
Perfseminc
MEMIT
200
Edit
Fig. 2:
Evaluation results of different editing methods various types of datasets. The horizontal axis in the image represents the number edited samples, and the vertical axis represents the performance of the edited model.
the commonsense dataset 20Q has editing objectives in the form of 0/1 labels, whereas the other three datasets have editing objectives of entity, entity, and event, respectively.
The Dataset for Evaluation. We employed six types of datasets in the Open LLM Leadership board evaluation, comprising A12 Reasoning Challenge (25-shot) [Clark et al. 2018). HellaSwag (10-shot) [ellers et al. 2019]. the Common Sense Reasoning Dataset, MMLU (5-shot) [Hendrycks] al. 2020), widely used benchmark for assessing multitask accuracy, which encompasses 57 tasks including basic mathematics, American history, computer science, law, and others. Additionally, we utilized TruthfulQA (0-shot) [Lin et al. [021], benchmark designed to test the model's propensity for dishonesty, as well as WinoGrande [Sakaguchi et al. 2021]: Common Sense Reasoning and GSM-8K [Cobbe al. 2021 Mathematical Reasoning.
observed that the KN [Dai al., 2021 editing method substantially degrades the model's performance after editing single sample. In contrast, when employing the ROME [Meng et al.] [022a] editing method, the model's performance did not experience a significant decline even after editing 1000 samples. Similarly, with the MEMIT [Meng et al. 2022b method, the model's performance remained relatively stable when editing up to 2000 samples. However, beyond 2000 samples, a pronounced decline in performance occurred, and further increases in the number of edited samples did not lead additional performance degradation.
Result Analysis. As illustrated in Figure
noteworthy that the ROME [Meng et al. 2022a] and MEMIT [Meng et al.] 2022b methods exhibit less pronounced performance degradation when editing the 20Q [Porada et al. 200 Gupta
ct al. 2023] dataset. contrast, when editing other datasets, the model's performance displays varying degrees of decline. This can be attributed to the fact that the 20Q [Porada et al. 2021 Gupta al.; 2023] dataset's editing objective is in the form of binary labels (0/1), featuring single element for the editing objective and relatively small number of tokens.
3.2 Dataset Construction
To elucidate the impact of different editing objectives on the performance of the edited model, we created a Kulti-Question Dataset (MQD) based on ATOMIC commonsense database (Sap et al.) 2019]. This dataset comprises three question types: true/false, multiple-choice, and direct generation. The corresponding editing objectives are yes/no, a/b/c/d, and entity/event, respectively. Each question type consists of 4000 samples.

---

ATOMIC Data Source: < PersonX accepts Person' Y appointment, as result, Person' shakes Person hand >
Calegory
Component Prompl
Targel Answer
DGG
PersonX accepts Person appointment, resulting in Person
shakes PersonX hand
PersonX accepts PersonY appointment, resulting in ?
MQ
Below are Tour options: (a) give him treut;; (b) forgive him;
(c) to live happily ever after; (d) shakes PersonX hand. The corcect option is
PersonX accepts PersonY appointment, resulting in PersonYY
T/F
shakes PersonX
Is this sentence logical? Please answer yes or no. A:
yes
Table I: An example of converting source data from ATOMIC (Sap et al. 2019 database into directly generated(DG), multiple-choice questions(MQ), and true/false questions(17F). The editing objectives are "shakes PersonX hand", "d" and "yes", respectively.
Data Preparation. The ATOMIC database [Sap et al. 2019]. J. developed by the Allen Institute and later optimized in its subsequent version [Hwang al. 2021 is well-known commonsense repository. The data format in ATOMIC [Sap et al. 2019] is < Event1, Relationship, Event2 >, which contains unrecognized markers (e.g. - etc.) and invalid characters (e.g., &, etc.) that we manually filtered out. Furthermore, the relationship types in ATOMIC (Sap et al. 2019] are abbreviated and not easily comprehensible by humans. Although ATOMIC (Sap et al. 2019 provides corresponding annotations, they are insufficient to form coherent statement when constructing the prompt. To address this, also performed manual template rewriting to ensure smoother overall prompt.
MQD Dataset. Different problem formats are associated with distinct editing objectives. The MQD dataset encompasses three formats: Directly Generated (DG), Multiple-choice Questions (MQ), and True/False questions (T/F). According to our statistical analysis, the Perplexity (PPL) values for the editing objectives of these three question types are 12.3, 43.3, and 297.4, respectively. The calculation formula for PPL is as follows:
PPL(X) = exp {
1 Σ log pa (a: mei)
(I)
Where pa (zy (Izi) is based on the sequence before i, and the log-likelihood of the i-th token. In the Table I for directly generating datasets, we directly concatenate Event1, the rewritten relationship. and Eventy to form a prompt. For multiple-choice questions, we designated the target answer as one of the options and randomly selected events from other natural categories in the dataset the remaining options. For true/false questions, we asked LLMS to determine whether the newly formed prompt logical. To mitigate bias, we established multiple positive and negative examples. Notably, the core information of the three question types remains consistent, with the primary difference lying in the question types and editing objectives.
3.3 The Influence of Editing Objectives
In this section, we investigate the effect of various editing objectives on model performance by editing the Multi- culti-Question Dataset (MQD 3.2
The Dataset for Editing and Evaluation. The MQD dataset comprises three question types: true/false, multiple-choice, and directly generated. The knowledge sources for these three question types are consistent, and we conducted relevant experiments directly using the MQD dataset. We selected ROME [Meng et al., [202aa as the editing method. According to our statistical analysis, the average length of the input tokens for the three question types is 23.44, 35.03, and 13.38, respectively. while the average length the editing objectives tokens 1, and 3.88, respectively. The true/false questions have two possible output types: yes no. The multiple-choice questions have four editing objectives: a, b, c, and In contrast, the directly generated questions have more diverse editing objectives, including entities events, with the number of tokens for events typically exceeding
5

---

T/F PPL
257.4
Pettmarc
Perfomaano
0.4
Pinks
12:0
Rumber
Number
2033
1303
1000
2000
480
(a) True/False questions (T/F).
(b) Multiple-choice (MQ).
(c) Directly generated (DG).
Fig. 3: The performance of the model after editing data for different question types.
Result Analysis. As illustrated in Figure 3 all three question types true/false, multiple-choice. and direct generation compromise the model's performance, with direct generation causing the most significant damage. The experimental results suggest that higher perplexity (PPL) of the editing objectives leads more severe performance degradation. Meanwhile, we also calculated the L1-norm of the editing layer and found that the higher the perplexity of the editing target, the larger the L1-norm. Consequently, from data perspective, the decline in model performance after editing is attributed to the diversity of editing objectives and the length of tokens.
Model-Specific Factors Affecting Performance
In this section, we investigate the model-centric factors contributing to performance degradation and propose optimization strategies to enhance the performance of the edited model.
4.1 Forgetting about Previously Edited Samples
As illustrated in Figure in Appendix B conven- tional editing methods typically assess sam- ple's quality immediately after editing. How- ever, in real-world applications, it is often nec- essary to evaluate the editing success rate after processing sequence of samples. The tradi- tional evaluation approach overlooked the edited model's impact previously edited samples. This section investigates the model's forgetting issue with respect to previously edited samples.
-200-E /Ownr Enckutkes
(C714-33 -TCME The Degres terpeting
(C714-5 41-443--00 -NN--TTTTTTT-III <The Depies (aaeeiigg
The dataset and method for evaluation. We employed the factual triplet dataset zeRE(Lev) et al. 2017] for experimental purposes and uti- lized ROME Meng al. 2022a] and MEMIT [Meng et al. 2022b as editing methods. To as-
Fig. 4: Assessment of forgetting ability of models.
sess the varying degrees of forgetting in the model, we devised two experimental approaches. As illustrated in Figure the "Overall Evaluation" involves evaluating all previous samples using the edited model each time it completed. In contrast, "The Degree of Forgetting" involves obtaining a checkpoint after editing 1000 samples, followed by an evaluation of the model's forgetting degree every 100 previously edited samples, in the order of editing.
Result analysis. As shown in Figure 4 for the "Overall Evaluation". the ROME [Meng et al. 2022a method has the highest accuracy of 0.04, indicating that the edited model has significant forgetting problem on the 100 previously edited samples. The accuracy of the MEMIT [Meng et al.) 2022b method has decreased from the highest near 1.0 to 0.0, indicating that as the number of model edits increases, the forgetting problem of the model becomes increasingly severe. For the degree of forgetting, the ROME [Meng et al. 2022a method and MEMIT Meng et al. (022b) method always have a low success rate. Especially the MEMIT [Meng et al. 2022b method shows success rate of O in most samples. We suspect that the MEMIT Meng et al. 2022b method after editing 1000

---

samples may not be able to successfully edit the samples. We also conducted relevant experiments in the next section.
4.2 The Bottleneck of Sequence Edit
Sequence editing refers to the repeated updating of knowledge within a model. However, the number of successful edits that existing knowledge editing methods can achieve is not infinite. We conducted experiments on two editing methods to examine the bottleneck in the number of edits for each method.
The dataset and method for evaluation. The editing models used in the experiments are all based on GPT-J (6B) [Wang and Komatsuzaki 2021]. Consistent with the original work, the fixed MLP layers for the ROME [Meng et 2022a| and PMET [Li et al. 2024] methods are [5] and [3,4.5.6.788] respectively. We used the factual tripler dataset zeRE for experimental data and applied ROME and MEMIT editing methods. All experiments were conducted with a batch size of 1, and total of 1000 samples were edited. We recorded the probability value of each edited sample.
Result analysis. As shown in Figure 5 after editing 100 samples, the ROME method shows an overall decrease in probability values. While some probability values still update to larger values as the number of edits increases, the overall trend downward. For the MEMIT method, after editing 850 samples, the probability values showed a significant decrease, approaching zero. This indicates that the MEMIT method exhibits a phenomenon complete editing inefficiency after 850 edits. This experiment reflects the bottleneck phenomenon in editing methods, with ROME having an effective editing count of 100 and MEMIT having an effective editing count of 850.
Corsapanding charge33ONE)
Commopand 122 one nor changerMenTT
1:34
4.0
1:30
1.20
(a) The probability value.
(b) Ll-norm on ROME.
(c) LI-norm on MEMIT.
Fig. 5: The bottleneck and LI -o-mormm correspondence in sequence editing are illustrated in the figure. In subfigure (a), the horizontal axis represents the number of edited samples, and the vertical axis represents the probability value of the edited object. The blue line represents the ROME method, while the green line represents the MEMIT method. In subfigures (b) and (c), the horizontal axis represents the number of edits, and the vertical axis represents the L1-norm value of the editing layer.
We plotted the Ll-norm variation of different methods the editing layer during the editing process. As shown in Figure when the number of edits reaches 100, the norm of the 5-th MI. layer edited by the ROME method significantly increases. Furthermore, in Figure after 850 edits using the MEMIT method, the editing layers [3, 4, 5, 6, 7, 8] also exhibit an explosive increase in norm. The results indicate that frrmm model perspective, the decline in model performance after editing is due to the explosive growth of norms in the editing layers during the editing process.
4.3 Dump for Sequence Knowledge Editing
Sequence editing refers to the process of editing a single or multiple samples multiple times. With the continuous updating of world knowledge, constantly updating the knowledge within models has become an urgent need. The experimental results in Sections and show that the performance of the model significantly decreases after sequence editing. In this section, we propose Dump for Sequence (D4S) knowledge editing method that effectively improves the performance of the edited model.
7

---

4.3.1 Defects in Previous Sequence Editing
The knowledge editing method is update factual associations stored in the parameters of transform- ers ers[Meng et al. 2022b]. Given a sequence text x = [11, ". Fm] the transformer's 's hidden state 1 the layer and the token j calculated:
1 [x]] 1 [2] + all ( [x] + m [x]
all J [x] allention' (N' - [x],... / (x)))
J [x] W o 1n ((att!/())))
where 7 indicate layer norm and a means a non-linear function. The knowledge editing method to update the knowledge in the model by changing particular weight W. For MEMIT [Meng et al. 2022b) method, there are triples of fact to be edited Ç = {(*,, ri, o:) )= where 8; means the subject, O, is the object and r, means relation between them. And we have 02 F LI,M(ss r:)- For simplicity, we use [g[a]] to represent the last token's hidden state (x(""x)) [x:) the 7th prompt æ. To
make 0; = LLM (8,, ri), the target of 7th edit = -i + 5, is got by optimizing:
min log Plolloee @ p(s.,)))
where h. = [p(s)] is the hidden state of the last edit layer L, p(s;, r;) denotes the prompt consisting of subject and relation, and prey indicates random prefix to obtain generalizability. Subsequently, for each layer I E L needs to be edited, we can take the following approach to update W Ruxu;
N o(!
7 "pree + p(ss)))),, 2; h
A
(4)
L 1 ++
A (R'K**) (Coo' + +KK wt out
wi out A
(5)
with R' irt.... 1 n. K [k] k.l. Cov' K2K'" And Kj are the keys of knowledge irrelevant with &. A simple idea to optimize the knowledge of sequence editing as a whole is saving the editing history Rt and K For each new edit with k and n+1 we can concate it with history:
R = + r = Ir.
T ruttl
(6)
K' K' + [K]
In this way, we can optimize all the knowledge of sequence editing as a whole to mitigate the damage to LLM. However, the space complexity of such dump method is O(n), which is unacceptable to us.
4.3.2 The D4S Method
The D4S method designed to save the editing history in O(1) space and apply batch editing
methods in sequence editing situations. We that for A (Rl K! )('''' + K'K ) the parts related to the editing history can be written as:
RKKT = -11 AlI
KTTTT
(8)
K'KTT = -M......
(9)

---

where have R'K E Ruxe and K K'K'T E R=xx So just need the two matrices above. For each new edit with ky 1441 and rl n+1 we can integrate it into edit history with a smiple addition operation:
R R'K"" = =R' + tr T
(10)
KKK = =KK + Knt1khtt
(11)
This approach requires just (1) storage space and allows to convert sequence editing methods into batch editing methods, thus reducing the damage to the edited model during sequence editing. Additionally. this ability consolidate each individual edit instance into a single batch makes the locate and editing method distinct from fine-tuning.
4.3.3 Theoretical Proof of Mitigating Norm Growth
To demonstrate that our D4S method can effectively alleviate norm growth in the editing layer, can consider the update of parameters edited by the previous method MEMIT [Meng et al. 2022b after n edits:
AW MEMIIT Σ (rshit))oo + k;
(12)
Regarding the D4S method, we have:
AW D4S
(KoK
Σ
kg
Σ (nik) + <K
(13)
Due to Ko + ki and Ko + Σx= kikk being positive definite, intuitively. the inverse of Ko + > k; is expected to have smaller numerical values compared to Ko kigT. Therefore, the norm of AWD4S smaller than that of AW WEMM. The experimental results in Figures 6 also demonstrate the effectiveness of the D4S method in mitigating L1-norm growth.
Experiments
The experiments are designed to answer two research questions: a) How does D4S perform in sequence editing compared to other methods? b) How much damage does D4S do to the model?
5.1 Performance Comparison of Sequence Editing
We counted the results of different methods after 1,000 sequence edits GPT-J (6B) [aag and Komatsuzaki 202T] and Llama2 (7B) [Touvron et al. 2023). The Appendix C provides Metric indicators. As show in Table 2 compared to previous editing methods, our method D4S has achieved significant performance improvement. Specifically, when Edits=500 and Edits=1000, the average performance of D4S achieved State- State-Of-Thes Trree-AASSSOOA results, indicating that D4S still has superior editing ability even after editing multiple samples.
We conduct additional experiments on Counterfact[Men et al. 2022b and MquakelZhong et al. 2023] dataset Appendix [ In addition this, we also wanted explore the model's ' forgetting of previous edits, chose every 100 edits as checkpoint investigate this in Appendix

---

Edits
T00
Edits
500
Edits
1000)
Model
Method
Eff-
Par.
Spc.
Aeg.
E J.
Par.
Spe.
Aeg.
E
Par.
Spe.
Aeg.
FT
30.47
25.78
12.19
22.81
19,43
18.06
6.56
14.68
16.13
13.89
5.96
12.00
ROME
97.68
92.39
90.22
93.43
48.68
47.72
32.56
42.99
21.84
20.74
4,01
15.53
Llama
83.22
2,90
2.51
2.74
PMET
0.24
0.35
0.46
0.35
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
GRACE
99.33
0.53
99.88
66.58
98.87
0,59
99.72
66.39
98.94
0.35
99.77
66.35
D4S(ours)
95.53
86.45
93.44
91.19
$1.48
83.34
85.34
87.36
79.30
78.99
81.88
FT 15.28
7.40
0.42
7.70
12.95
7.84
0.23
7.01
7.03
4.15
0.11
3.77
0.06
1.58
0,92
GPT
MEMIT
98.25
93.41
87.24
92.97
84,94
76.15
59.72
73.60
0.00
0.02
0.00
0.01
PMET 0.33
0.33
0.00
0.22
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
GRACE
100,00
0.40
100,00
100,00
0.15
100,00
66.72 72
100.00
0.07
100,00
66.69
D4S(ours)
97.72
97.01
85,30
93.34
97.42
93.12
74.65
88.40
95.98
91.17
70.12
88.77
Table 2: Sequencial edits on GPT and Llama with ZsRE dataset.
The best results are in bold and
underline means the suboptimal.
5.2 Performance of Edited Model
we explore the impact of D4S on the model by examining the changes in average weight norms and performance on downstream tasks of GPT. The downstream task performance changes for L.llama are in the Appendix [F] The experimental results indicate that the norm of the D4S method did not increase after 10000 edits, and the performance of downstream tasks only decreased slightly.
As shown in Figure 6
La mom Value
0:4
0.0
Number edits
Number
(a) Llama L1-Norm
(b) GPT GPT-LLI--IEE -I-morm
(c) Downstream tasks
Fig. 6: Norms of weight and performance of the edited model.
6
Conclusion
This paper explores the reasons behind the decline in model performance from both data and model perspectives. From the data perspective, the decline is attributed to the diversity of editing objectives and the length of tokens. From the model perspective, the decline is due to the explosive growth of norms the editing layers during the editing process. To enhance the performance of edited models, we propose a Dump for Sequence (D4S) method, which effectively improves the performance of edited models and overcomes previous editing bottleneck issues. This method allows for multiple effective edits with minimal impact model performance.
Acknowledgments
This work was supported by Beijing Natural Science Foundation (L243006) and the National Science Foundation of China (No. 62106249). This work was also sponsored by BaiCCuan-EEbee Foundation Model Fund.
References
Xiusheng Huang, Yubo Chen, Shun Wu, Jun Zhao, Yuantao Xie, and Weijian Sun. Named entity recognition via noise aware training mechanism with data filter. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 4791-4803, 2021.
10

---

Xiusheng Huang, Hang Yang, Yubo Chen, Jun Zhao, Kang Liu, Weijian Sun, and Zuyu Zhao. Document-level relation extraction via awir- and entity-enhancee representation learning. In Proceedings of the 29th International Conference on Computational Linguistics, pages 2418-2426 2022.
Anshita Gupta, Debanjan Mondal, Akshay Sheshadri, Wenlong Zhao, Xiang Li, Sarah Wiegreffe, and Niket Tandon. Editing common sense in transformers. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 12-222,,, 2023.
Yunzhi Yao, Peng Wang, Bozhong Tian, Siyuan Cheng, Zhoubo Li, Shumin Deng, Huajun Chen, and Ningyu Zhang. Editing large language models: Problems, methods, and opportunities. arXiv preprint 2rXiv:2305.13/11 2023.
Ning Ding, Yujia Qin, Guang Yang, Fuchao Wei, Zonghan Yang. Yusheng Su, Shengding Hu, Yulin Chen, Chi-Min Chan, Weize Chen, al. parameter-effcieen fine-tuning of large-scale pre-trained language models. Nature Machine Intelligence, 5(3):220-235. 2023.
Song Wang, Yaochen Zhu, Haochen Liu, Zaiyi Zheng, Chen Chen, and Jundong Li. Knowledge editing for large language models: A survey, ACM Computing Surveys, 2023.
Vittorio Mazzia, Alessandro Pedrani, Andrea Caciolai, Kay Rottmann, and Davide Bernardi. A survey knowledge editing of neural networks. arXiv preprint arXiv:2310.19700 2023.
Peter Hase, Mohit Bansal, Been Kim, and Asma Ghandeharioun. Does localization inform editing? surprising differences in causality- based localization vs. knowledge editing in language models. Advances in Neural Information Processing Systems. 36, 2024.
Jia-Chen Gu, Haonniinn Xu, Jun- Yu Ma, Pan Lu, Zhen-Hua Ling, Kai-Wei Chang, and Nanyun Peng. Model editing can hurt general abilities of large language models. 2024. URL https
//doi.org// 1tgg/10.400 200///8550///0 arxiv 2001.4201444 04700
Ningyu Zhang, Yunzhi Yao, Bozhong Tian, Peng Wang, Shumin Deng, Mengru Wang, Zekun Xi, Shengyu Mao, Jintian Zhang, Yuansheng Ni, al. A comprehensive study of knowledge editing for large language models. arXiv preprint arXiv:2401 10/286, 2024.
Ian Porada, Kaheer Suleman, Adam Trischler, and Jackie Chi Kit Cheung. Modeling event plausibility with consistent conceptual abstraction. arXiv preprint arXiv:2104.10247, 2021.
Omer Levy, Minjoon Seo, Eunsol Choi, and Luke Zettlemoyer. Zero-shot relation extraction via reading comprehension. arXiv preprint arXiv: 1tt.::17060011133 04115, 2017.
Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. Locating and editing factual associations in gpt. Advances in Neural Information Processing Systems, 35:17359- 17372, 2022a.
Eric Mitchell, Charles Lin, Antoine Bosselut, Christopher D Manning, and Chelsea Finn. Memory- based model editing scale. In International Conference on Machine Learning, pages 15817- 15831. PMLR, 2022.
Qingxiu Dong, Damai Dai, Yifan Song, Jingjing Xu, Zhifang Sui, and Lei Li. Calibrating factual knowledge in pretrained language models. arXiv preprint arXiv:2210.03329, 2022.
Zexuan Zhong, Zhengxuan Wu, Christopher Manning, Christopher Potts, and Danqi Chen. Mquake: Assessing knowledge editing in language models via multi-hop questions. arXiv preprint 13X1::3305.14755 2023.
Yifan Wei, Xiaoyan Yu, Huanhuan Ma, Fangyu Lei, Yixuan Weng, Ran Song, and Kang Liu. Assessing knowledge editing in language models via relation perspective. arXiv preprint arXiv:2311.09053. 2023.
Hao Peng, Xiaozhi Wang, Chunyang Li, Kaisheng Zeng, Jiangshan Duo, Yixin Cao, Lei Hou, and Juanzi Li. Event-level knowledge editing. arXiv preprint arXiv:240..1309, 2024.
Jiateng Liu, Pengfei Yu, Yuji Zhang, Sha Li, Zixuan Zhang, and Heng Ji. Evedit: Event-based knowledge editing with deductive editing boundaries. arXiv preprint arXiv:2402.1132. 2024.
11

---

Xiusheng Huang, Yequan Wang, Jun Zhao, and Kang Liu. Commonsense knowledge editing based on free-text llms. 2024.
Damai Dai, Dong, Yaru Hao, Zhifang Sui, Baobao Chang, and Furu Wei. Knowledge neurons in pretrained transformers, arXiv preprint arXiv:2104.0860 2021.
Kevin Meng, Arna Sen Sharma, Alex Andonian, Yonatan Belinkov, and David Bau. Mass-editing memory in a transformer. arXiv preprint arXiv:2210.07222, 2022b.
Xiaopeng Li, Shasha Li, Shezheng Song, Jing Yang. Jun Ma, and Jie Yu. Pmet: Precise model editing in transformer. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 18564-18572, 2024.
Aman Madaan, Niket Tandon, Peter Clark, and Yiming Yang. Cemory-assister prompt editing to improve gpt-3 after deployment. arXiv preprint arXiv:2201.0660 2022.
Ce Zheng, Lei Li, Qingxiu Dong, Yuxuan Fan, Zhiyong Wu, Jingjing Xu, and Baobao Chang. Can edit factual knowledge by in-context learning? arXiv preprint arXiv:2305.12740. 2023.
Eric Mitchell, Charles Lin, Antoine Bosselut, Chelsea Finn, and Christopher D Manning. Fast model editing at scale. arXiv preprint 1 2021.
Nicola De Cao, Wilker Aziz, and Ivan Titov. Editing factual knowledge in language models, arXiv preprint arXiv:2104 00160444444 2021.
Chenmien Tan, Ge Zhang, and Jie Fu. Massive editing for large language models via meta learning. arXiv preprint ar3iv:231 1.04661, 2023.
Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. Think you have solved question answering? try arc, the ai2 reasoning challenge. arXiv preprint arXiv:1803.0543: 2018.
Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. Hellaswag: Can machine 2019.
really finish your sentence? arXiv preprint
Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding. arXiv preprint arXiv:2009.088 03300, 2020.
Stephanie Lin, Jacob Hilton, and Owain Evans. Truthfulqa: Measuring how models mimic human falsehoods. arXiv preprint arXiv:2109.0799: 2021.
Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. Winogrande: An adversarial winograd schema challenge at scale. Communications of the ACM, 64(9):99-106. 2021.
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, al. Training verifiers solve math word problems. arXiv preprint arXiv:21 10. 14168, 2021.
Maarten Sap, Ronan Le Bras, Emily Allaway, Chandra Bhagavatula, Nicholas Lourie, Hannah Rashkin, Brendan Roof, Noah Smith, and Yejin Choi. Atomic: An atlas of machine com- monsense for ih-then reasoning. In Proceedings of the AAAI conference on artificial intelligence, volume 33, pages 3027-3035, 2019.
Jena D Hwang, Chandra Bhagavatula, Ronan Le Bras, Jeff Da, Keisuke Sakaguchi, Antoine Bosselut, and Yejin Choi. On symbolic and neural commonsense knowledge graphs. 2021.
Ben Wang and Aran Komatsuzaki. Gpt-j-6b: A billion parameter autoregressive language model, 2021.
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiw:2307.09288, 2023.
12

---

A Appendix /Limitations
We note a few limitations of the experiments conducted in this paper:
(1) This paper only used the GPT-J (6B) and Llama2 (7B) models. Due to limitations in computational resources, experiments on larger scale models were not conducted.
(2) For the D4S method, our training consists of only 2000 steps due to computational resource limitations. By editing more samples, we can better identify the performance bottleneck of the D4S method.
(3) Due computational resource limitations, the MEMIT and PMET methods did not batch size greater than in our experiments. To better assess batch editing, more experiments with larger batch sizes should be attempted.
B Appendix /Evaluation Method
Original
Current
Sample1
Sample1
Evaluate
Sample2
Sample2
Evaluate
Evaluate
Fig. 7: The original and current evaluation methods.
C
Appendix /Metric Indicators
Same MEMIT, we took the following three indicators to evaluate the impact on model performance:
Eff. [o argmar PloTlp(ss)))))
(14)
of
Par. = E, EpEpppaar( argmax Plo. [p]
(15)
of
Spe.
argmaz Plotlp(s(,r]]]
(16)
where par(s;, re) is the of p(st, ri)'s paraphrases, nei(s;, ri, 0,) is the of (st, ri, oz)'s neigh- borhoods.
Appendix /Additional Experiments
We also conduct additional experiments on Counterfact Meng et al. 2022b and Mquake Squake//.hogg Zhhong
al. 2023
dataset. Here the results:
13

---

Counter fact
Mquake
Model
Method
E.f.
Par.
Spe.
Aug.
Eff.
Par.
Spe.
Aug.
FT
32.80
9.00
1.00
14.27
17.00
6.22
0.00
7.74
ROME
0.60
0.70
0.60
0.63
0.00
0.00
0.00
0.00
GPT
MEMIT
86.20
59.50
31.30
59.00
0.00
0.00
0.00
0.00
GRACE
100.00
0.50
100.00
66.83
D4S(ours)
99.10
47.00
63.70
69.93
97.40
75.54
21.84
64.93
FT
8.46
4.07
2.03
4.85
41.43
44.93
28.24
38,20
ROME
27.83
16.03
5.66
16.50
76.85
78.41
3.67
52.97
Llama
MEMIT
0.00
0,00
6.72
2.24
0.00
0.00
0.00
0.00
GRACE 99.9
0.25
99.97
66.71
D4S(ours)
96.68
46.66
72.45
71.93
85.30
72.68
28.16
62.05
Table 3: Sequencial edits GPT and Llama with Counterfact and Mquake dataset. The best results are bold and underline means the suboptimal. Since GRACE is not suitable for the data structure of Mquake, did not include in the comparison.
E Appendix /Evaluate Forgetting Ability
Parsphraae
Lfficayy
Spoce
pre 100 edits
pre
100 edits
pre 100 edits
(a) | Efficacy
(b) Ilama Paraphras
(c) Ilama_ Specifiity
Efficecy
Paraphrasc
Bear CALY
1W
19
100
(d) gpt Efffccay
(c) gpt Paraphaaae
(f) gpt Spe__Secifiictty
Fig. 8: Evaluation metries change with the number of edits.
F
Appendix /Downstream Performance of Llama
Edit Num
are challenge
he helaswag
mmlu
truthfulqa me
winogrande
O
43.34
57.14
41.35
38.97
69.14 68.75 (10.39)
10,000
42.41 (10.93)
52.99 (14.15)
39.76 (U1.59)
38.56 (10.41)
Table 4: Downstream performance of Ilama after 10,000 edits.
14

---

NeurIPS Paper Checklist
The checklist is designed encourage best practices for responsible machine learning research, addressing issues of reproducibility, transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and follow the (optional) supplemental material. The checklist does NOT count towards the page limit.
Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:
You should answer [Yes], [No], or [NA].
[NA] means either that the question Not Applicable for that particular paper or the relevant information is Not Available.
Please provide short (1-2 sentence) justification right after your answer (even for NA).
The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include (after eventual revisions) with the final version of your paper, and its final version will published with the paper.
The reviewers of your paper will be asked use the checklist as one of the factors their evaluation. While "[Yes] is generally preferable to "[No]" ", it is perfectly acceptable to answer "[No] providedd proper justification is given (e.g.. "error bars are not reported because it would be too computationally expensive" or "we were unable to find the license for the dataset we used"). In general, answering binary way, we
"[No] or "[NA]" is not grounds for rejection. While the questions are phrased acknowledge that the true answer is often more nuanced, please just use your best judgment and write justification to elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] to a question, in the justification please point to the section(s) where related material for the question can be found.
IMPORTANT, please:
Delete this instruction block, but keep the section heading "NeurIPS paper checklist",
Keep the checklist subsection headings, questions/answers and guidelines below.
Do not modify the questions and only use the provided macros for your answers.
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: [NA]
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results be expected to generalize other settings.
is fine include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: [NA]
15

---

Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, c.g., if the approach was only tested few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
Justification: [NA]
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: [NA]
Guidelines:
The answer NA means that the paper does not include experiments.
16

---

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily new algorithm, the paper should make it clear how to reproduce that algorithm. (b) the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: [NA]
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please the NeurlPS code and data submission guidelines (attps://nips.co for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
/ thaips.cc/uubic//uieee http:/ppww.......d/d////dddiiiis http//oeuuubiiiiiooooooiiieee for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
17

---

Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Question:
Answer:
[Yes]
Justification: [NA]
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: [NA]
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
OK to report 1-sigma error bars, but one should state The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative error rates).
If error bars reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: [NA]
Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
18

---

The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code Of Ethies
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics http://wwwrp...ccoooooo./ccccced
Answer: [Yes]
Justification: [NA]
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.
The authors should make to preserve anonymity (e.g.. if there special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: [NA]
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, scraped datasets)?
Answer: [Yes]
Justification: [NA]
Guidelines:
The answer NA means that the paper poses such risks.
19

---

Released models that have high risk for misuse or dual use should be released with necessary safeguards to allow for controlled use of the model. for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: [NA]
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperawi thcode. com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re- re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: [NA]
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [Yes]
Justification: [NA]
20

---

Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According to the NeurIPS Code Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [Yes]
Justification: [NA]
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with
human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
21


Reviews:
[{"review_id": "NhykXHeTVo", "reviewer": "Reviewer_nwwK", "summary": "This paper investigates the reasons and solutions for the decline in model performance of model editing.  The authors conduct experiments from two perspectives: data and model. Specifically, to clarify the impact of data on the performance of edited models, the authors first evaluate how editing different types of data affects model performance. Then, the authors construct a Multi-Question Dataset (MQD) and identified that the performance of  the edited models is primarily influenced by the diversity of the editing objectives  and the length of the tokens. Secondly, the authors explore the factors that affect model  performance from a model perspective. Experiments revealed a strong correlation between the L1 norm of the edited model layers and the editing accuracy, and  identified an editing quantity bottleneck. To enhance the performance of edited  models, the authors propose a Dump for sequence (D4C) method that effectively improves  the performance of edited models and overcomes the previous editing bottleneck issue. This method allows for multiple effective edits with minimal impact on  model performance.", "questions": "See weaknesses.", "limitations": "Yes\n\nThe limitations are on page ten. I am unsure if this counts as exceeding the page limit.", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "This paper investigates the impact of data on the performance of edited models. Evaluations are conducted across multiple tasks, revealing that the editing objective is the primary factor influencing model performance.\n\nThe authors found that the decline in edited model performance is correlated with the explosive growth of the L1 norm of parameter layers during the editing process.\n\nThis paper proposes a caching sequence edit method that leverages O(1) space complexity to retain past knowledge and regulate the explosive growth of the parameter layer norm.", "weaknesses": "The writing of this paper should be improved. There is no overview of this paper, which makes it hard to follow the details of Section 3 and 4.\n\nThe motivation of the proposed method  is not clear.\n\nThere are many typos such as line 182.\n\nThere are many missing references such as: \n\nKnowledge Editing for Large Language Models: A Survey\n\nStable Knowledge Editing in Large Language Models\n\nA Comprehensive Study of Knowledge Editing for Large Language Models\n\nEditing Large Language Models: Problems, Methods, and Opportunities", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for the positive recommendations and valuable feedback!\n\n$\\textbf{Q1: }$ There is no overview of this paper, which makes it hard to follow the details of Section 3 and 4.\n\n$\\textbf{W1: }$  We provided an overview of this paper using text and figures. Firstly, in the Introduction section, L36-L43 and L44-L50 briefly introduce the relevant content of Sections 3 and 4, respectively. At the same time, we visually present the overview of this paper in Figure 1, including both data and model perspectives. Finally, in L93-L94 and L168-L169, we provide overviews of Sections 3 and 4, respectively.\n\n$\\textbf{Q2: }$ The motivation of the proposed method is not clear.\n\n$\\textbf{W2: }$ The motivation for this paper is to investigate the causes of performance degradation in edited models and to optimize them. We have explicitly expressed it multiple times in the paper. Firstly, in L4-L5 of the Abstract section, we present this motivation and provide our proposed solution. In addition, in L36-L37, we reiterated this motivation and provided specific experimental settings. Meanwhile, L51-L54 and L62 have repeatedly demonstrated this motivation. Finally, in the conclusion section, L280 and L283 express the motivation behind this paper.\n\n$\\textbf{Q3: }$ About Limitations section.\n\n$\\textbf{W3: }$ We learned from the NeurIPS website [1] that the Limitations are not included in the main text of the paper, so it can be placed on page ten.\n\n$\\textbf{Q4: }$ About  some  missing references and typos.\n\n$\\textbf{W4: }$ We promise to add missing references and correct typos in the revised version.\n\n$\\textbf{Ref}$\n\n[1] CallForPapers. Neurips 2024.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ziWaD3VJyK", "reviewer": "Reviewer_rpPG", "summary": "The paper investigates the reasons behind performance decline in sequential model editing approaches that selectively update parameters based on both data and model factors. To address the issues causing this decline, the authors propose a method to save editing history, thereby transforming sequential editing into batch editing with minimal computational overhead.", "questions": "N/A", "limitations": "N/A", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "Extensive experimentation is conducted to empirically demonstrate how factors such as dataset characteristics, editing objectives, and model-specific properties affect performance in sequential model editing.\n\nA simple matrix storage solution is introduced, which enables the conversion of sequential editing into batch editing.", "weaknesses": "The study is restricted to two closely related editing approaches.\n\nExperimentation is limited in demonstrating the efficacy of the D4C method. Different datasets and a larger number of edits for a more thorough evaluation are needed.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your valuable feedback and for recognizing the novelty of the our method. Below, we address some of the weaknesses raised:\n\n$\\textbf{Q1: }$ The study is restricted to two closely related editing approaches.\n\n$\\textbf{W1: }$ First and foremost, our innovation surpasses the mere translation of sequence editing into batch editing. It employs a batch editing-like approach to implement sequence editing, resulting in a significant performance boost. As indicated in Table 2, previous methods like MEMIT [1] and PMET [2] were designed for batch editing, but they struggled to perform well when applied to sequence editing tasks. In contrast, as we highlighted in L173-L176, our focus on sequence editing aligns more closely with real-world requirements, highlighting the limitations of existing knowledge editing techniques. By enhancing the performance of sequence editing while maintaining a storage complexity of $O(1)$, a succinct theoretical proof is presented in Reviewer MFeF's 'Q6'. We aim to advance the practical application of knowledge editing technology. \n\nFurthermore, we conducted additional experiments by extending the baseline method with the latest state-of-the-art method, GRACE [3]. It is worth noting that although the GRACE method performs 22% worse than ours in performance, due to their storage complexity being $O(n)$, they did not provide experiments for editing 10,000 times.\n\n|GPT-J-ZsRE (1,000 times)|Eff.|Par.|Spe.|Avg.|\n|----  |----  |----  |----  |----  |\n|ROME|2.44|`2.23`|0.92|1.86|\n|MEMIT|0.00|0.02|0.00|0.01|\n|GRACE|**100.00**|0.07|**100.00**|`66.69`|\n|D4C|`95.98`|**91.17**|`70.17`|**88.77**|\n\nPS: **Bold** indicates the best results, while suboptimal is `highlighted`.\n\n$\\textbf{Q2: }$ Experimentation is limited in demonstrating the efficacy of the D4C method. Different datasets and a larger number of edits for a more thorough evaluation are needed.\n\n$\\textbf{W2: }$ Thanks again for your advice!  We further demonstrate the effectiveness of the method by adding additional datasets and number of edits. Firstly,  we expanded the datasets to include the Mquake [4] and Counterfact [1] datasets, where Mquake is a multi-hop dataset, Counterfact is a fact dataset. The results are as follows:\n\n|Llama2-Mquake (1,000 times)|Eff.|Par.|Mul.|Avg.|\n|----  |----  |----  |----  |----  |\n|FT|41.43|44.93|**28.24**|38.20|\n|ROME|`76.85`|**78.41**|3.67|`52.97`|\n|MEMIT|0.00|0.00|0.00|0.00|\n|D4C|**85.30**|`72.68`|`28.16`|**62.05**|\n\n\n|Llama2-Counterfact (1,000 times) |Eff.|Par.|Spe.|Avg.|\n|----  |----  |----  |----  |----  |\n|FT|8.46|4.07|2.03|4.85|\n|ROME|27.83|`16.03`|5.66|16.50|\n|MEMIT|0.00|0.00|6.72|2.24|\n|GRACE|**99.9**|0.25|**99.97**|`66.71`|\n|D4C|`96.68`|**46.66**|`72.45`|**71.93**|\n\nPS: **Bold** indicates the best results, while suboptimal is `highlighted`.\n\nAdditionally, we extended the editing number of the our method to 10,000 (Llama2) and 9,000 (GPT). We evaluated the performance of edited GPT models in downstream tasks. As shown in the table below, the edited model maintains great performance on downstream tasks, proving that the editing method has minimal damage to the model.\n\n|Num of Edits|arc_challenge|hellaswag|mmlu|truthfulqa_mc|winogrande |\n|---- |----  |----  |----  |----  |----  |\n|0|43.34|57.14|41.35|38.97|69.14|\n|10,000|42.41 ($\\downarrow 0.93$)|52.99 ($\\downarrow 4.15$)|39.76 ($\\downarrow 1.59$)|38.56 ($\\downarrow 0.41$)|68.75 ($\\downarrow 0.39$)|\n\n**Specific results can be seen in Fig. 1 and Fig. 2 of supplementary PDF and our Global Response.**  The experimental results show that our method can still achieve superior performance after 10,000 edits.\n\n$\\textbf{Ref}$\n\n[1] Mass-editing memory in a transformer. ICLR 2023.\n\n[2] PMET: Precise Model Editing in a Transformer. AAAI 2024.\n\n[3] Aging with GRACE: Lifelong Model Editing with Discrete Key-Value Adaptors. Neurips 2023.\n\n[4] Mquake: Assessing knowledge editing in language models via multi-hop questions. EMNLP 2023.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "DnMIxMQWsG", "reviewer": "Reviewer_MFeE", "summary": "Recent research has shown varying degrees of decline in model performance following small changes made by certain model editing methods. This paper is the first to comprehensively analyze the reasons behind such performance declines. Through extensive experiments, it identifies two main factors: data and model. For data-specific factors, the paper finds that perplexity and token length significantly influence performance. For model-specific factors, the L1 norm of the edited layer is identified as a key influence. Building upon these insights, the paper proposes a method named Dump for sequence (D4C), which significantly improves model performance.", "questions": "- Can the authors add a section in the appendix to expand on the dataset mentioned in 3.1 (i.e., provide examples and details about the editing objectives) for better readability?\n\n- What dataset was employed in Section 5?\n\n- I encourage the authors to release the full code to enhance reproducibility.\n\n- (Minor) Consider reducing v-space in some parts of the paper (e.g., the bottom of page 2).", "limitations": "The limitations are discussed in the paper.", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "- The paper is well-motivated: Exploring the reasons behind and impact of small changes made by model editing techniques on the performance of unedited samples is of great significance.\n- The analysis of the data-specific and model-specific factors is supported with diverse datasets and comprehensive experiments. The model-specific analysis, in particular, is evaluated rigorously, addressing the forgetting issue that prior works often overlooked\n\n- The observation of the influence of editing on the model norm is intriguing. High-norm parameters can be sensitive to noise and numerically unstable. It would be beneficial if the authors could also provide an L2-norm plot for comparison.\n\n- The experimental results are impressive, demonstrating significant improvements and validating the effectiveness of the proposed method.", "weaknesses": "- My main concern with the data-specific analysis is whether the conclusion is about correlation or causation. Many variables can be changed about the input data. Plotting a single Figure 3 might be insufficient to justify that perplexity and token length are the main reasons for the decline in model performance after editing.\n\n- Unfortunately, the constructed dataset is not open-sourced. \n\n- Recent research [1] has shown that model editing methods (e.g. ROME, MEMIT) are not good at handling multi-hop questions, how would D4C perform in such more challenging scenarios?\n\n- Some theoretical analysis can be conducted to demonstrate that D4C does not lead to an increase in norms.\n\n[1] Mquake: Assessing knowledge editing in language models via multi-hop questions. EMNLP 2023", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for recognizing the importance, effort in method, and applications of our work. We outline our response to the main concerns: \n\n$\\textbf{Q1: }$ Can the authors add a section in the appendix to expand on the dataset mentioned in 3.1\n\n$\\textbf{W1: }$ Thank you for your suggestion. We will provide a detailed introduction to the dataset in the appendix. For example, for the DG category in Table 1, the main structure of the dataset is as follows:\n\n|DG category |   |  |\n|----  |----  |----  |\n|Keys: | Text | Annotation  |\n| Prompt:| {}, resulting in PersonY, | Component Prompt\n|Subject:| PersonX accepts PersonY appointment | Edited Subject\n|Relation_id:| resulting in | Relation Category\n|Target_new:| {‘str’:shakes PersonX hand,} | Target Answer\n\n$\\textbf{Q2: }$ The data-specific analysis is whether the conclusion is about correlation or causation.\n\n$\\textbf{W2: }$  The conclusion is about causation. We echo your observation regarding the potential variability in input data variables. Our research found that perplexity and token length are the reasons for the performance degradation of the edited model. In order to control variables, as shown in Table 1, our 'Subject' and 'Relation_id' are consistent, that is, 'PersonX accept PersonY appointment, resulting in PersonY, '.  To make the model output answers that match the question type, the prompt will be slightly different. Each question type consists of 4000 samples.  Although it is difficult to exclude all irrelevant variables, it can be certain that the experimental results have demonstrated the validity of the conclusion, which has a positive effect on exploring the reasons for the decline in model performance after editing.\n\n$\\textbf{Q3: }$ Recent research [1] has shown that model editing methods (e.g. ROME, MEMIT) are not good at handling multi-hop questions, how would D4C perform in such more challenging scenarios?\n\n$\\textbf{W3: }$ Our work focuses more on sequence editing, which is a common and important scenario in real-life applications. Therefore, we did not explore multi-hop questions before.  Inspired by your suggestion, we conducted experiments on the multi-hop dataset Mquake [2]. The experimental results showed that our method also achieved superior performance.\n\n |Llama-Method|Eff.|Par.|Mul.|Avg.|\n|----  |----  |----  |----  |----  |\n|FT|41.43|44.93|**28.24**|38.20|\n|ROME|`76.85`|**78.41**|3.67|`52.97`|\n|MEMIT|0.00|0.00|0.00|0.00|\n|D4C|**85.30**|`72.68`|`28.16`|**62.05**|\n\n|GPT-Method|Eff.|Par.|Mul.|Avg.|\n|----  |----  |----  |----  |----  |\n|FT|`17.00`|`6.22`|`0.00`|`7.74`|\n|ROME|0.00|0.00|`0.00`|0.00|\n|MEMIT|0.00|0.00|`0.00`|0.00|\n|D4C|**97.40**|**75.54**|**21.84**|**64.93**|\n\nPS: **Bold** indicates the best results, while suboptimal is `highlighted`. Since GRACE [1] is not suitable for the data structure of Mquake, we did not include it in the comparison.\n\n$\\textbf{Q4: }$ What dataset was employed in Section 5?\n\n$\\textbf{W4: }$ For our sequence edting experiment, we use the ZsRE [3] dataset. **And we also expanded our experiment. You can find the details in the Global Response.**\n\n$\\textbf{Q5: }$ I encourage the authors to release the full code to enhance reproducibility.\n\n$\\textbf{W5: }$  Thank you for your suggestion! We have included our code in the Supplementary Material when submitting the paper. You can download the file and follow the instructions in the README.md to set up the code.\n\n$\\textbf{Q6: }$ Some theoretical analysis can be conducted to demonstrate that D4C does not lead to an increase in norms.\n\n$\\textbf{W6: }$ For simplicity, we can consider the update of parameters edited by MEMIT after $n$ edits:\n$$\\Delta W_{MEMIT}=\\sum_{i=1}^{n} (r_ik_i^T)(K_0K_0^T+k_ik_i^T)^{-1}$$\nRegarding the D4C method, we have:\n$$\\Delta W_{D4C}=(\\sum_{i=1}^{n} r_ik_i^T)(K_0K_0^T+\\sum_{i=1}^{n} k_ik_i^T)^{-1}=\\sum_{i=1}^{n} (r_ik_i^T)(K_0K_0^T+\\sum_{i=1}^{n} k_ik_i^T)^{-1}$$\n\nDue to $K_0K_0^T+k_ik_i^T$ and $K_0K_0^T+\\sum_{i=1}^{n} k_ik_i^T$ being positive definite, intuitively, the inverse of $K_0K_0^T+\\sum_{i=1}^{n} k_ik_i^T$ is expected to have smaller numerical values compared to $K_0K_0^T+k_ik_i^T$. Therefore, the norm of $\\Delta W_{D4C}$ is smaller than that of $\\Delta W_{MEMIT}$. We are considering adding a detailed theoretical analysis in the revised version.\n\n$\\textbf{Q7: }$ It would be beneficial if the authors could also provide an L2-norm plot for comparison.\n\n$\\textbf{W7: }$ Thank you for acknowledging our analysis. We have added comparative experiments on L2-norm, and the results are as follows\n\n|GPT2-ROME  |0|100|200|300|400|500|600|700|800|900|\n|----  |----  |----  |----  |----  | ----  |----  |----  |----  |----  | ---- |\n|L1-norm ( e-2 ) |1.06 | 1.58 | 2.20 | 2.71 |  3.14 | 3.52 | 3.87 | 4.19 | 4.49 | 4.77 | \n|L2-norm ( e-6 ) |1.67|  2.54|  3.68| 4.62| 5.40|  6.10|  6.71| 7.28|  7.82| 8.31|   \n\n|GPT2-D4C  |0|100|200|300|400|500|600|700|800|900|\n|----  |----  |----  |----  |----  | ----  |----  |----  |----  |----  | ----  | \n|L1-norm ( e-2 ) | 1.06 | 1.09 | 1.00 | 1.11 |  1.11 | 1.11 | 1.11 | 1.11 | 1.12 | 1.12 | \n|L2-norm ( e-6 ) | 1.67|  1.68|  1.69| 1.70| 1.70|  1.71|  1.72| 1.73|  1.73| 1.74|    \n\nThe results showed that the conclusions of L2 norm and L1 norm are consistent. Our D4C method can effectively suppress norm growth.\n\n$\\textbf{Ref}$ \n\n[1] Aging with GRACE: Lifelong Model Editing with Discrete Key-Value Adaptors. Neurips 2023.\n\n[2] Mquake: Assessing knowledge editing in language models via multi-hop questions. EMNLP 2023.\n\n[3] Zero-shot relation extraction via reading comprehension. CoNLL 2017.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "XLj4sSSFvC", "reviewer": "Reviewer_Kq3N", "summary": "This paper addresses the challenges associated with the decline in performance of LLMs after undergoing knowledge editing. The study identifies the primary factors contributing to performance degradation from both data and model perspectives. By constructing a Multi-Question Dataset (MQD) and analyzing the impact of editing objectives, token length, and diversity, the paper finds that perplexity associated with editing objectives significantly affects model performance. From the model perspective, a strong correlation was observed between the L1 norm of parameter layers and editing accuracy. The paper proposes a novel method called Dump for sequence (D4C), which effectively manages the parameter growth and improves model performance post-editing.", "questions": "- Adaptability of D4C Method: How adaptable is the D4C method to different types of LLMs and knowledge editing tasks beyond those tested in your experiments?\n- Impact on Unedited Model Performance: How does the D4C method affect the performance of the model on tasks that have not been edited? Is there any evidence of performance trade-offs?\n- Handling of Diverse Editing Objectives: Could you elaborate on how the D4C method manages the complexity and diversity of editing objectives without compromising the model’s overall integrity and coherence?\n\n**Missing References**\n- Editing Large Language Models: Problems, Methods, and Opportunities (EMNLP 2023)\n- Knowledge Editing for Large Language Models: A Survey (2023)\n- A Survey on Knowledge Editing of Neural Networks (2023)\n- A Comprehensive Study of Knowledge Editing for Large Language Models (2024)", "limitations": "- Dependency on Specific Data Characteristics: The effectiveness of the proposed solutions may depend heavily on the characteristics of the data used for training and testing, which might not be consistent across different domains or applications.\n- Evaluation Metrics: While the paper introduces new evaluation methods, the reliance on perplexity (PPL) and L1 norm metrics might not completely capture all aspects of model performance and health, especially in nuanced or context-dependent scenarios.\n- Limited Experimentation: The experiments (Section 5) in this paper are too limited to demonstrate the conclusion. \n- Scope of Editing Objectives: The study might not fully capture the impact of highly diverse or complex editing objectives that could be encountered in real-world scenarios.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "- Innovative Methodological Approach: The study introduces a new method, D4C, which addresses the explosive growth in parameter norms and optimizes model performance post-editing. This approach is both innovative and practical for managing edited models.\n- Comprehensive Data Analysis: The construction of the Multi-Question Dataset and detailed analysis of how different types of data affect model performance provide valuable insights into the mechanics of model editing.\n- Clear Identification of Problems and Solutions: The paper clearly identifies specific problems associated with knowledge editing in LLMs, such as catastrophic forgetting and performance bottlenecks, and provides targeted solutions to these issues.\n- Empirical Validation: The experiments conducted in this paper offer empirical evidence supporting the proposed methods, enhancing the credibility and applicability of the findings.", "weaknesses": "- Generalizability of Findings: The study focuses on specific scenarios and datasets, which may limit the generalizability of the findings across different types of LLMs or editing tasks.\n- Potential Overfitting to Edited Scenarios: There is a risk that the model may become overly optimized for the edited scenarios, potentially affecting its performance on unedited or unrelated tasks.\n- Complexity of Implementation: The proposed D4C method, while effective, may be complex to implement and integrate into existing systems due to its sophisticated handling of parameter layers.\n- Unsuitable Citation Format: The citations in this paper are in the format of “XXX et al. [YEAR]”, which are not suitable enough, and had better change into the format of [1], [2], [3], ……", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your constructive feedbacks on the paper! We have added detailed explanations for the important questions asked in the review.\n\n$\\textbf{Q1: }$ How adaptable is the D4C method to different types of LLMs and knowledge editing tasks beyond those tested in your experiments?\n\n$\\textbf{W1:}$ We conducted additional experiments and expanded the datasets to include the Mquake [1] and Counterfact [2] datasets. The experimental results on GPT-J are as follows:\n\n|GPT-J-Mquake |Eff.|Par.|Mul.|Avg.|\n|----  |----  |----  |----  |----  |\n|FT|`17.00`|`6.22`|`0.00`|`7.74`|\n|ROME|0.00|0.00|`0.00`|0.00|\n|MEMIT|0.00|0.00|`0.00`|0.00|\n|D4C|**97.40**|**75.54**|**21.84**|**64.93**|\n\n|GPT-J-Counterfact |Eff.|Par.|Spe.|Avg.|\n|----  |----  |----  |----  |----  |\n|FT|32.80|9.00|1.00|14.27|\n|ROME|0.60|0.70|0.60|0.63|\n|MEMIT|86.20|**59.50**|31.30|59.00|\n|GRACE|**100.00**|0.50|**100.00**|`66.83`|\n|D4C|`99.10`|`47.00`|`63.70`|**69.93**|\n\nAt the same time, we extended the model to Llama2, and the results are shown below:\n\n|Llama2-Mquake |Eff.|Par.|Mul.|Avg.|\n|----  |----  |----  |----  |----  |\n|FT|41.43|44.93|**28.24**|38.20|\n|ROME|`76.85`|**78.41**|3.67|`52.97`|\n|MEMIT|0.00|0.00|0.00|0.00|\n|D4C|**85.30**|`72.68`|`28.16`|**62.05**|\n\n\n|Llama2-Counterfact |Eff.|Par.|Spe.|Avg.|\n|----  |----  |----  |----  |----  |\n|FT|8.46|4.07|2.03|4.85|\n|ROME|27.83|`16.03`|5.66|16.50|\n|MEMIT|0.00|0.00|6.72|2.24|\n|GRACE|**99.9**|0.25|**99.97**|`66.71`|\n|D4C|`96.68`|**46.66**|`72.45`|**71.93**|\n\nPS: **Bold** indicates the best results, while suboptimal is `highlighted`.\n\nThe results show that our method has superior performance on different types of datasets and models. Furthermore, We have expanded our experimental editing 10,000 times, and our performance far exceeds other methods. **You can find the details in the supplementary PDF  of Global Response.**\n\n$\\textbf{Q2: }$  How does the D4C method affect the performance of the model on tasks that have not been edited? Is there any evidence of performance trade-offs?\n\n\n$\\textbf{W2: }$  As highlighted in Section 4.2, we observed a strong correlation between the rise in the L1-norm of the parameter layer and the performance of the editing task. Simultaneously, we noted a significant decrease in the performance of the edited model on unedited tasks. We posit that the increase in the L1-norm results in decreased performance on tasks that have not undergone editing. Our approach effectively mitigates the norm growth of the edited model by consolidating all samples edited from the sequence (e.g., 6.a and 6.b). A succinct theoretical proof is presented in Reviewer MFeF's 'Q6'. Figure 6.c demonstrates that the D4C method ensures that the edited model maintains superior performance on unedited tasks.\n\nFurthermore, we extended the number of edits to 10,000, and the edited model still maintained good performance.\n\n|Num of Edits|arc_challenge|hellaswag|mmlu|truthfulqa_mc|winogrande |\n|---- |----  |----  |----  |----  |----  |\n|0|43.34|57.14|41.35|38.97|69.14|\n|10,000|42.41 ($\\downarrow 0.93$)|52.99 ($\\downarrow 4.15$)|39.76 ($\\downarrow 1.59$)|38.56 ($\\downarrow 0.41$)|68.75 ($\\downarrow 0.39$)|\n\nThe existing knowledge editing methods have poor editing performance (such as Table 2), or have a storage complexity of $O(n)$ ( e.g. GRACE[3]), and cannot provide experimental results edited 10,000 times.\n\n\n$\\textbf{Q3: }$ Could you elaborate on how the D4C method manages the complexity and diversity of editing objectives without compromising the model’s overall integrity and coherence?\n\n$\\textbf{W3: }$  The complexity and diversity of editing objectives can affect the performance of the edited model, which is the conclusion we have drawn from a data perspective. Our further experiments indicate a correlation between editing objectives and parameter layer norm growth. For example, if the editing objective for a true/false question is \"yes/no\", the norm growth caused by this dataset is slow, while the directly generated editing objective is \"entity/event\", the norm growth caused by this dataset is fast. This also confirms our conclusion from a model perspective that the reason for the performance degradation of the edited model is related to norm growth. We will present the specific experimental results in the revised version.\n\nOur D4C method can effectively reduce the norm growth of the edited model (such as Fig 6.a and 6.b), even when the number of edits reaches 10,000 (e.g. W2). Therefore, D4C can manage the complexity and diversity of editing objectives without compromising the model’s overall integrity and coherence.\n\n$\\textbf{Q4: }$ The proposed D4C method, while effective, may be complex to implement and integrate into existing systems due to its sophisticated handling of parameter layers.\n\n$\\textbf{W4: }$ Thank you for recognizing the effectiveness of our method. To be honest, our system is not complex, with core code ranging from 200 to 300 lines. We have included our code in the Supplementary Material when submitting the paper. You can download the file and follow the instructions in the README.md to set up the code.\n\n$\\textbf{Ref}$\n\n[1] Mquake: Assessing knowledge editing in language models via multi-hop questions. EMNLP 2023.\n\n[2] Mass-editing memory in a transformer. ICLR 2023.\n\n[3] Aging with GRACE: Lifelong Model Editing with Discrete Key-Value Adaptors. Neurips 2023.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
# Global Response

We thank the reviewers for their thoughtful feedback. We are glad the reviewers find that

* Our motivation is innovation and has great significance

    * "The paper is well-motivated: Exploring the reasons behind and impact of small changes made by model editing techniques on the performance of unedited samples is of great significance." - MFeE
    * "This approach is both innovative and practical for managing edited models." -  Kq3N

* Our paper presents a thorough, novel, and significant analysis

    * "The model-specific analysis, in particular, is evaluated rigorously, addressing the forgetting issue that prior works often overlooked." -MFeE
    * "detailed analysis  provide valuable insights into the mechanics of model editing." - Kq3N

* Our solutions are novel and effective and our experiments are well-conducted

    * "The experimental results are impressive, demonstrating significant improvements and validating the effectiveness of the proposed method." - MFeE
    * "Extensive experimentation is conducted to empirically demonstrate." - rpPG
    * "The experiments enhance the credibility and applicability of the findings." -  Kq3N

**[Main Motivation]**

As the scale of models continues to grow, how to update model knowledge affordably has remained a persistent challenge. Despite recent intriguing proposals, such as knowledge editing [1], these methods have yet to see practical implementation. The primary issue lies in the significant drop in performance in the edited model, leading to a lack of trust from users. Understanding the causes behind this decline in model performance post-editing is increasingly crucial.
By deriving solutions from the analysis findings and minimizing model disruptions during knowledge updates, the edited model can earn user trust and drive the adoption of this technology in real-world scenarios. As highlighted by reviewer MFeE, this work holds considerable significance.


**[Supplementary Experiments]**

 *  **1. Dataset Expansion Experiment.**  We conducted additional experiments and expanded the datasets to encompass the Counterfact [1] and Mquake [3] datasets. The experimental outcomes for the Counterfact dataset are outlined below

    |Llama-Method|Eff.|Par.|Spe.|Avg.|
    |----  |----  |----  |----  |----  |
    |FT|8.46|4.07|2.03|4.85|
    |ROME|27.83|`16.03`|5.66|16.50|
    |MEMIT|0.00|0.00|6.72|2.24|
    |GRACE|**99.9**|0.25|**99.97**|`66.71`|
    |D4C|`96.68`|**46.66**|`72.45`|**71.93**|

    |GPT-Method|Eff.|Par.|Spe.|Avg.|
    |----  |----  |----  |----  |----  |
    |FT|32.80|9.00|1.00|14.27|
    |ROME|0.60|0.70|0.60|0.63|
    |MEMIT|86.20|**59.50**|31.30|59.00|
    |GRACE|**100.00**|0.50|**100.00**|`66.83`|
    |D4C|`99.10`|`47.00`|`63.70`|**69.93**|

    PS: **Bold** indicates the best results, while suboptimal is `highlighted`.

    Additionally, taking into account Reviewer MFeE's suggestions, we investigated the performance of existing baselines under **sequence multi-hop editing** with Mquake [3] dataset:

    |Llama-Method|Eff.|Par.|Mul.|Avg.|
    |----  |----  |----  |----  |----  |
    |FT|41.43|44.93|**28.24**|38.20|
    |ROME|`76.85`|**78.41**|3.67|`52.97`|
    |MEMIT|0.00|0.00|0.00|0.00|
    |D4C|**85.30**|`72.68`|`28.16`|**62.05**|

    |GPT-Method|Eff.|Par.|Mul.|Avg.|
    |----  |----  |----  |----  |----  |
    |FT|`17.00`|`6.22`|`0.00`|`7.74`|
    |ROME|0.00|0.00|`0.00`|0.00|
    |MEMIT|0.00|0.00|`0.00`|0.00|
    |D4C|**97.40**|**75.54**|**21.84**|**64.93**|

    PS: Since GRACE is not suitable for the data structure of Mquake, we did not include it in the comparison.

* **2. Method expansion experiment.**  We conducted further experiments by extending the baseline method with the latest state-of-the-art method, GRACE [2]. Notably, while the GRACE method shows a performance decrease of only 22% compared to ours, their storage complexity being $O(n)$ prevented them from conducting experiments involving 10,000 edits.

    |GPT-1000 times|Eff.|Par.|Spe.|Avg.|
    |----  |----  |----  |----  |----  |
    |ROME|2.44|`2.23`|0.92|1.86|
    |MEMIT|0.00|0.02|0.00|0.01|
    |GRACE|**100.00**|0.07|**100.00**|`66.69`|
    |D4C|`95.98`|**91.17**|`70.17`|**88.77**|


* **3. Editing frequency expansion experiment.** we extended the number of edits to 10,000, and the edited model still maintained great performance.

    |Num of Edits|arc_challenge|hellaswag|mmlu|truthfulqa_mc|winogrande |
    |---- |----  |----  |----  |----  |----  |
    |0|43.34|57.14|41.35|38.97|69.14|
    |10,000|42.41 ($\downarrow 0.93$)|52.99 ($\downarrow 4.15$)|39.76 ($\downarrow 1.59$)|38.56 ($\downarrow 0.41$)|68.75 ($\downarrow 0.39$)|

    The existing knowledge editing methods have poor editing performance (such as Table 2), or have a storage complexity of $O(n)$ (e.g. GRACE [2]), and cannot provide experimental results edited 10,000 times.

**The supplementary PDF provides more detailed experimental results.**

$\textbf{Ref}$

[1] Knowledge editing for large language models: A survey.  arXiv 2023.

[2] Aging with GRACE: Lifelong Model Editing with Discrete Key-Value Adaptors. NeurIPS 2023

[3] Mquake: Assessing knowledge editing in language models via multi-hop questions. EMNLP 2023

Meta-review: The paper analyzes the reasons of decline in model's performance after editing. They discuss both data and model aspects and provide various insights. Moreover, they suggested a method to enhance model's performance. The paper contributes to a large space of model editing methods and is a valuable contribution.
Decision: Accept (poster)