Unveiling the Pitfalls of Knowledge Editing for Large Language Models
Authors: Zhoubo Li
Ningyu Zhang
Yunzhi Yao
Mengru Wang
Xi Chen
Huajun Chen
Venue: ICLR 2024 poster (2024)
Keywords: model editing
knowledge editing
large language models
Primary area: datasets and benchmarks
TL;DR: An investigation into the potential pitfalls associated with knowledge editing for LLMs.

Abstract: As the cost associated with fine-tuning Large Language Models (LLMs) continues to rise, recent research efforts have pivoted towards developing methodologies to edit implicit knowledge embedded within LLMs. Yet, there's still a dark cloud lingering overhead -- will knowledge editing trigger butterfly effect? since it is still unclear whether knowledge editing might introduce side effects that pose potential risks or not. This paper pioneers the investigation into the potential pitfalls associated with knowledge editing for LLMs. To achieve this, we introduce new benchmark datasets and propose innovative evaluation metrics. Our results underline two pivotal concerns: (1) Knowledge Conflict: Editing groups of facts that logically clash can magnify the inherent inconsistencies in LLMs—a facet neglected by previous methods. (2) Knowledge Distortion: Altering parameters with the aim of editing factual knowledge can irrevocably warp the innate knowledge structure of LLMs. Experimental results vividly demonstrate that knowledge editing might inadvertently cast a shadow of unintended consequences on LLMs, which warrant attention and efforts for future works. Code and data are available at https://github.com/zjunlp/PitfallsKnowledgeEditing.

Published as a conference paper at ICLR 2024
UNVEILING THE PITFALLS OF KNOWLEDGE EDITING FOR LARGE LANGUAGE MODELS
Zhoubo Li Ningyu Zhang 22 Yunzhi Yao!.2, Mengru Wang' Xi Chen', Huajun Chen 1.2,3*
CCollege of Computer Science and Technology, Zhejiang University Ant Group Joint Research Center for Knowledge Graphs, Zhejiang University LZJU-Hangzhou Global Scientific and Technological Innovation Center, Zhejiang University "Tencent { (zhoubo.. 1i,, zhangningyu ju. edu. on
ABSTRACT
As the cost associated with fine-tuning Large Language Models (LLMs) continues to rise, recent research efforts have pivoted towards developing methodologies to edit implicit knowledge embedded within LLMs. Yet, there's still a dark cloud lingering overhead will knowledge editing trigger butterfly effect? since it is still unclear whether knowledge editing might introduce side effects that pose po- tential risks or not. This paper pioneers the investigation into the potential pitfalls associated with knowledge editing for LLMs. To achieve this, we introduce new benchmark datasets and propose innovative evaluation metrics. Our results un- derline two pivotal concerns: (1) Knowledge Conflict: Editing groups of facts that logically clash can magnify the inherent inconsistencies in LLMs facet neglected by previous methods. (2) Knowledge Distortion: Altering parameters with the aim of editing factual knowledge can irrevocably warp the innate knowl- edge structure of LLMs. Experimental results vividly demonstrate that knowledge editing might inadvertently cast shadow of unintended consequences LLMs, which warrant attention and efforts for future works!.
INTRODUCTION
Despite their impressive abilities, Large Language Models (LLMs) such ChatGPT are unaware of events occurring after their training phase and may inadvertently generate harmful or offensive content. To address this concern, the concept of knowledge editing for LLMs has been proposed (Cao et al., 2021; Dai et al., 2022; Mitchell et al., 2022a;b; Meng et al., 2022; 2023; Ilharco et al., 2023; Onoe al., 2023; Zhong et al., 2023; Yao et al., 2023), which provides efficient way change the behavior of LLMs without resorting to an exhaustive retrain- ing or continuous training procedure. While exist- ing knowledge editing approaches for LLMs have demonstrated impressive results, an ancient Chinese poetic wisdom resonates a dark cloud lingering over- head: A single hair can move the whole body. The poetic phrase underscores the notion that even mi- nor alterations to LLMs may lead to significant out- comes, akin to the butterfly effect observed in chaos theory (Lorenz, 2000). This raises new critical re- search question: does knowledge editing for LLMs introduce irreversible unforeseen side effects?
Confllet
LLMs
Applied Editors
We are already in model
I'm the latest
Knowledge Model
Target
Hdlit
Other
Connections
Distortion
don't care others
The edit succeed, but...
Figure 1: As the number of edits increases, the model might manifest Knowledge Con- flict when dealing with inputs involved with multiple consecutive edits. Meanwhile, each edit could potentially lead ruptures in knowledge links within the model, resulting in Knowledge Distortion.
"Corresponding author.
"oode and data available at https:://// //g wtggithub.oomm httpttt..cc///.ooo/////tttttccet sknowl wew.wwwwddgddiiie edgekditingg

---

Published as a conference paper at ICLR 2024
Knowledge Conflict
Knowledge Distortion
Reverse Edit
Round-Edd
Edit Morie's hasband Pierve Jucques
Edit (0 Joe Biden was born California + Fforida
Edit (63) Jacques'ss wife Merie Maurice
Edit (11) Joe Biden was born in Floride +
California
>- DJJeques the husband of
(i) Maric
X
Marie
Jacgles
California
Es00 (1)
Florida
10
hofars RounkEdi
(i) Maurice
of
Meorica
Conflict
prods
Biden
Composite Edit
burn
Foet: The notable work of Shalespeare is Hambet.
Edit (1) Hamlet was uritten in fingiihh French
Label Edit X
kdit (l) Shakespeare wrote in Kieneel + German
Edit (ii Joe Biden born Pieridee
Califormia
Faet
Hamlet
KiM
Scranton
Americo
Stakeapeare
French
Garmao
Conflict
Joe Biden was born in
10
after Mulsi Label Edit
Multi
logionl rule:
elorr KAR ten lannuugge
prody
W Wat language was Halmet written in
French X
(i)) Garman
Canferent
Strmmiie
Alnennce
(a)
(b)
Figure 2: Unveiling the pitfalls of knowledge editing for LLMs. (a) Through REVERSE EDIT and COMPOSITE EDIT, we can observe that previous knowledge editing approaches may trigger Knowl- edge Conflict, leading to failures of knowledge editing; (b) Through ROUND-EDIT, we notice that previous knowledge editing approaches may lead to Knowledge Distortion, and the underlying knowledge structure within LLMs can be disrupted.
In Figure 1, we depict two types of side effects caused by knowledge editing: Knowledge Conflict and Knowledge Distortion. Using pilot experiments as an illustration, depicted in Figure 2 (a) top, applying one single edit (i) Marie's 's husband is Pierre + Jacques to modify fact within LLMs is straightforward using previous knowledge editing techniques. However, as the number of edits grows, these methods might encounter problems, especially when edits exhibit potential correlation. For example, consider a subsequent edit (ii) Jacques's wife is Marie + Maurice (REVERSE EDIT), both edit (i) and (ii) individually try to achieve their intended goals, but the model might still re- spond with Marie when prompted with Jacques the hushand of This situation exemplifies the Knowledge Conflict issue. Similarly, as shown in the following composite setting (Figure 2 (a) bottom), two edits can incur a conflict due to a common logical rule (COMPOSITE EDIT), thereby leading to inconsistency of knowledge in the model.
Further, as shown in Figure (b) top, we conduct a ROUND-EDIT experiment, which means we try to restore edited LLM (with edit (i)) via another subsequent edit ((i)). Unfortunately, we obverse that the model after ROUND-EDI has significantly reduced the capability to provide Scranton and America when prompting with Where was Joe Biden born? indicating that the implicit knowledge structure in LLMs has been disrupted. This situation exemplifies the Knowledge Distortion issue, which illustrates that knowledge editing may have an irreversible damage to LLMs.
Inspired by these observations, we pioneer the investigation to uncover the potential pitfalls of knowledge editing and categorize these pitfalls into two major types: knowledge conflict and knowl- edge distortion. To analyze the extent of these pitfalls, we create benchmark datasets tailored to prevailing methods. Specifically, we construct dataset named CONFLICTEDIT, comprising pairs of edits that could cause knowledge conflict, and introduce a set of new metrics designed to quan- tify the magnitude of knowledge conflicts. We empirically observe that previous knowledge editing approaches all suffer from the knowledge conflict issue. Then, to explore the knowledge distor- tion issue, we create the ROUNDEDIT dataset and conduct ROUND-EDIT experiments using various knowledge editing techniques. Our findings illustrate that previous knowledge editing approaches can indeed lead to adverse consequences on the implicit knowledge structure of LLMs. We further design simle-yye-efffee method called Multi-label Edit (MLE) that combines multiple correct labels of the edit to a single process, as depicted in Figure (b) bottom, which is capable to alleviate knowledge distortion and restore similar behavior compared with the original model.
2

---

Published as a conference paper at ICLR 2024
2 EXPLORING THE PITFALLS OF KNOWLEDGE EDITING FOR LLMS
2.1 OVERVIEW
In this section, we briefly introduce the definition of knowledge editing and the common evaluation paradigms in previous works. Next, we outline our evaluation criterion and delve into the specifics of our analysis for knowledge editing. The representative methods used are also summarized here.
Definition of Knowledge Editing for LLMs With daily updating of factual knowledge, LLMs always struggle to keep the parameters up-to-date. Knowledge editing releases the model from ex- pensive retraining through precisely updating several outdated factual pieces of knowledge. Suppose a factual knowledge as a triplet (s, o), an edit e = (s, 0 + o²) modifies the object from o to o" for given subject and relation After applying this edit to language model fo (where 0 denotes the model's parameters), there is a knowledge update applied to the model, that is
k. r,)
(1)
knn o,))
where ko is the old knowledge and ki is the one. Generally, we witness update from ho, to kn through the variation of their generation probabilities.
Vanilla Evaluation The current evaluation criterion primarily focuses various samples around an isolated edit e = (s, r, + o*). The post-edit model for (edited model) is expected to respond o" related samples I(e) (excluding e) while still keeping the original output to unrelated samples (e)) Commonly, instances sampling in the neighbor of are used to build T. Meng et al. (2022) samples O from unrelated set {(s', o): s' * s}. which restricts the actuating scope of The Reliability metric evaluates results on the fact (s,r,o"), the Generalization and Locality metric evaluate results on I and O respectively. These metries effectively constrain certain edits to an ex- pected actuating scope (Meng et al., 2022; Wang et al., 2023b; Yao et al., 2023). In our experiments, for each edit e, we compute the results by averaging the performance of the label over the edit itself and the related samples (e).
Motivation and Evaluation Principle Since LLMs can be regarded as large knowledge bases (Petroni et al., 2019), knowledge editing is expected handle thousands of edits while maintaining knowledge coherence and integrity. In previous works, multiple edits are applied either through sequential editing (Huang et al., 2023) or mass editing (Meng et al., 2023) settings preventing the model from obvious performance drop. Unfortunately. current isolated evaluations fail to consider the interaction between the accumulating edits which could lead subtle side effects of the model's knowledge. Hence, we start exploring the pitfalls of knowledge editing for LLMS from typical ed- its and focus on two new issues, namely, Knowledge Conffict ($2.2) and Knowledge Distortion ($2.3). Intuitively, The knowledge conflict setting can examine how two edits may interact con- tradict each other, resulting in misinformation. On the other hand, The knowledge distortion setting amalyze potentially irreversible damage to knowledge structure with mass edits. Rather than isolated edits, these settings can comprehensively evaluate the ability of knowledge editing methods to process numerous edits without degradation, contradiction, or loss of unrelated knowledge.
Editing Methods In the following Sections, we build benchmarks CONFLICTEDIT and ROUNDEDIT and conduct experiments to evaluate knowledge editing approaches as follows (we do not consider approaches that preserve parameters in LLMs):
Fine-tuning (FT) updates the model's parameters by gradient descent in a certain layer through Adam and early stop strategy maximize the probability of the editing target.
MEND Mitchell et al. (2022a) utilizes hypernetwork, which gives low-rank update to the original fine-tuning gradients based on the edit knowledge.
ROME Meng et al. (2022) uses causal tracing to locate the key layer associated with the edit knowledge, and then impose a update to the MLP module.
MEMIT Meng et al. (2023) follows the locating methods in ROME and is capable of updating multiple layers time when editing massive knowledge.
3

---

Published as a conference paper at ICLR 2024
2.2 KNOWLEDGE CONFLICT ANALYSIS
Note that a robust knowledge editing method should be supposed to deal with multiple updates of specific fact. However, with the increasing extent of edits, there possibility that interference occurs between different edits, causing the former edit invalid. Particularly, when the edits are logically connected, it's challenging yet crucial to nullify previous modifications; failing to do so can lead to inconsistencies in the post-edited model. To this end, we define a notion of Knowledge Conflict and attempt to explore when and why the conflicts occur, and how handle them.
2.2.1 PROBLEM DEFINITION
Knowledge Conflict Consider the scenario depicted in Figure 2(a) top, where two consecutive edits, ej:Marie's 's husband is Pierre - Jacques and ey:Jacques's 's wife is Marie - Maurice, are applied to a language model. These two edits both modify the fact (Jacques, HusbandOf. ?). Ide- ally, a reliable and robust edit method should retain only the latest change. However, the current editing method may inadvertently retain knowledge from previous edits, leading a logical incon- sistency, resulting in the knowledge conflict issue. There are many factors that can contribute to the emergence of knowledge conflicts and here we consider two major scenarios: REVERSE EDIT and COMPOSITE EDIT.
This scenario causes conflict by editing the facts with reverse relations. We formal- ize this situation as pair of consecutive edits that contain reverse relations, that is
Reverse Edit
Reverse Edit [ei ($1, T1, - 02)
e2 = (2,,72, 72, 81 + 82)
where r1 and 122 are reverse relations, such as Husband0f and wi feOf in the example. Take the case in Figure 2(a) top as example, the reverse edit makes two updates the model, but they simultaneously change the fact (Jacques, HusbandOf, ?). Hence, the change can be represented as the following:
he (s1, r1, 02)
(3)
ln (s2, 02)
where we can instantiate these two facts as ko:Marie's husband is Jacques edited by el and Mauricc's' husband is Jacques edited by e2 in the example above. We focus on this kind of fact pair which may lead to confusion when we ask related question 'Jacques the husband of who? to the post-edit model.
Composite Edit Furthermore, explore a more complex situation, where the edits are asso- ciated with a fact that will not be influenced by editing. The example illustrated in Figure 2(a) bottom exhibits this situation that we edit eı:Hamler was written in English - French and then ea=Shakespeare wrote in French + German while preserving tied fact ky: notable work of Shakespeare Hamlet, and we also formalize as
ky = (81, (,,,,,
Composite Edit
e1 ($1.71, 01 02)
62 ($2,72,02 02 + 03)
where r A ri - 72 is a logical rule, for example el and will both edit the fact (Hamlet, WrittenIn, ?). Thus, after executing the edits, the confusing knowledge update can be represented as
[ ko = (s1, ri, 02) kin (1,,,,,03)
After knowledge editing. we expect the post-edit model answer German to the question What language was Halmet written in?
2.2.2 EVALUATION
To analyze the Knowledge Conflict issue, we design a new dataset CONFLICTEDIT to evaluate the performance of existing knowledge editing methods.
4

---

Published as a conference paper at ICLR 2024
CONFLICTEDIT
Method
Single
Coverage
Reverse
Composite
Sucet
CS1
CMf
CSeep
CSimpT
CMt
CSexp'
CSimpt
CMf
TFD
GPT2-XI.
FT
82.56
78.88
70.86
80.28
15.20
71.11
75.45
57.65
64,28
88.75
MEND
98.40
91.04
60.01
88.89
15.32
60.50
84.85
81.35
43.45
72.09
ROME
99.96
99.76
96.92
65.92
0.00
-0.65
71.70
38.70
37.04
69.55
MEMIT
79.24
83.88
32.29
51.44
2.08
-1.60
57.15
29.40
-1.30
24.63
GPT-J
FT 100.0
100.0
99.90
99.60
4.16
97.20
96.68
88.92
88.98
89.97
MEND 100.0
95.88
82.41
88.92
6.40
60.72
83.04
73.52
63.99
42.95
ROME
100.0
99.80
94.25
56.84
0.00
0.06
77.60
29.24
39.27
81.02
MEMIT
100.0
99.64
88.91
55.16
0.00
-1.18
75.48
49.28
28.78
64.51
Table I: Knowledge Conflict results of knowledge editing methods. Bold results denote the best performance in each situation, whereas red results indicate a total failure under the setup and blue results mark the damage on tied fact that cannot be ignored.
Setup We construct our CONFLICTEDIT dataset from WikiData (Vrandecic Krötzsch, 2014), which consist of two types of edits, namely REVERSE EDIT and COMPOSITE EDIT, as defined in Equation 2 and Equation 4. To begin with, we follow Ho et al. (2020) to obtain several reverse and composite logical rules in WikiData. We construct CONFLICTEDIT by sampling thousands of data of REVERSE EDIT and COMPOSITE EDIT edits respectively. In the COMPOSITE EDIT setup, we respectively utilize GPT2-XL (1.5B) (Radford et al., 2019) and GPT-J (6B) (Wang & Komatsuzaki, 2021) to confirm the correctness of the tied fact ky before experiments. Also, we use GPT-4 (OpenAI, 2023) to generate neighbor prompts for every relation (Dataset construction details are in Appendix A.1.1). Besides, we take SINGLE and COVERAGE EDIT as references. SINGLE EDIT refers to editing the latest knowledge update directly, which evaluates the model's ability to directly modify the knowledge (s, 03). COVERAGE EDIT is based pair of direct edits sharing the common (s, r), that is
[e = (8, + 02) L2 = (s, 02 + 0z)
Coverage Edit
(6)
which focuses on a covered knowledge update, referring to the editing target of the last edit e2. We evaluate the result under SINGLE EDIT e2 and COVERAGE EDIT pair (e1, 62) references.
We design a new metric Conflict Score (CS), which weighs how well knowledge editing method handles the knowledge conflict issue. We measure CS by calculating the ratio that the new fact kn is more possible than the old fact ko after knowledge editing, that is
Metrics
CS 1 (Pr, (k.)) > Pf. (ko)),
Pf. (k) is the probability of emitting the target object in by probing the the prompts of (s, r). Further, we utilize two modes, Explicit (CSesp) and Implicit (CSimp), to calculate Pf, (k,,) (Details in Appendix B). As stated above, the old fact ko is the reason causing Knowledge Conflict, design the metric Conflict Magnitude (CM) to estimate the decrease of the probability of ko:
CM = Prom (ko) Pso (ko.)
(8)
Prom (Ro)
where 8m is the intermediate model parameters after edit e, For the convenience of analyzing the experimental results and comparing the difference of evaluation paradigm, we take Success Score (Suec) as a metric revealing the basic editing performance, that is
Succ L(p, (o' (s,r)) > Ps (ol (s,r))).
In the COMPOSITE EDIT setup, the tied fact ky is also significant to compose the knowledge conflict, thus we consider the probability change of ky as Tied Fact Damage (TFD), calculated similarly to Equation 8 where substitute the h. with ky.
5

---

Published as a conference paper at ICLR 2024
Reverse Facts
2
Tied Fact
E
e1
e1
E1
Scope: ($1.1) Target: Oz
Scope: (5,,1) Target: 02
Scope:
(5),7) Target: 02
Scope: (02, Target: 52
Scope: ($2, Target: Og
Scope:
(5 Target: 0g
@ 0s, (5,(0).
* (s,,),5))
(a) Coverage
(b) Reverse
(c) Composite
Figure 3: A Unified View of Knowledge Conflict. 61 and 62 are two different knowledge editing instances. Editing Scope is the range where an edit takes effect, and Target the expected editing object. (a) COVERAGE EDIT shares total coverage in editing scope. (b) REVERSE EDIT relates each edit through reverse facts. (c) COMPOSITE EDIT is unique in editing scope yet maintains a consistent logical rule concerning tied fact denoted ky-
2.2.3 RESULTS ANALYSIS
We conduct experiments GPT2-XL and GPT-J and report the results in Table In this part, report the results of SINGLE, COVER AGE AGE EDIT, REVERSE EDIT and COMPOSITE EDIT settings, and further discuss unified view expound the internal reason for the results.
Single & Coverage Edit Note that for the COVERAGE EDIT setting, knowledge editing tech- niques are tasked with not only incorporating new knowledge but also editing existing information to ensure knowledge consistency. The CS scores derived from COVERAGE EDIT can reveal the Re- liability and Gencralization performance subsequent to implementing a pair of edits. Notably, the results might be influenced by the initial edit within the pair, setting apart from the Single Suce scores. Empirically, we notice that FT achieves higher scores in GPT-J but lower scores GPT2-XL. because of its disadvantage in generalization referring to the previous observation in Mitchell al. (2022a). ROME is effective in both GPT2-XL and GPT-J, showing its comparable ability to edit knowledge. MEND and MEMIT obtain lower scores, especially the CM metric, which indicates the failure of knowledge editing in this case.
Reverse Edit and MEND utilize Prompt(s1 11, - 02) as input, and update model pa- rameters by gradient descent, which is more effective in detecting the update of reverse edit. Thus FT and MEND obtain higher scores on CM which indicates the vanish the old knowledge k, through the latest editing. ROME and MEMIT take the subject involved prompts Prompt(s1, r1). thus applying the reverse edit in totally distinct ways. The CS scores indicate total failure of ROME and MEMIT in this setup, which may be caused by the poor performance on reverse relation reasoning discovered by Berglund et al. (2023). When evaluating the CM metric, both MEND and MEMIT retain the original knowledge within the model, posing significant challenge to maintaining knowledge consistency in LLMs.
key to attach the edit to all the
Composite Edit We observe that both FT and MEND demonstrate superior performance this setup. In contrast. ROME and MEMIT show promising results specifically in the editing process, but the edited facts remain disconnected from other information. Further, we notice that most of previous knowledge editing methods will trigger damage tied facts with very high TFD score, Note that for the COMPOSITE EDIT setting, knowledge editing approaches are anticipated to modify facts while also taking into account the associations of knowledge already learned within the LLMs.
A Unified View We analyze three types of knowledge editing scopes based on Mitchell et al. (2022b)'s Editing Scope concept reveal their differences in input space editing. Figure (a). the COVERAGE EDIT setup presents a binary input space choice with completely overlapping editing scopes. Figure (b) shows that the REVERSE EDIT edits are linked through reverse facts within their scope but completely overlap in logical implication. In Figure (c), COMPOSITE EDIT involves two edits with non-overlapping scopes but establishes a logical chain through an existing tied fact, ensuring logical consistency. This motivates the employment of detection technologies for the target knowledge, grounded in the symbolic logical rules of KGs, to circumvent potential knowledge discrepancies.

---

Published as a conference paper at ICLR 2024
Disney
Mickey
Family
Classical music
Related Genres
HasCharacter
Donald
Genre
Hop
Minnie Mouse
Rap
Edit
The Standard
Pluto
Chopin Nocturne
Edit
Jazz
Parade
Migration Range
(a) EASY
(b) HARD
Figure 4: The EASY and HARD split of ROUNDEDIT. (a) The EASY split contains editing targets which are semantically associated with the true labels of (s, r). The related field is called Migration Range. (b) The HARD split edits the object that is irrelevant to the true labels.
2.3 KNOWLEDGE DISTORTION ANALYSIS
Despite the capability of current knowledge editing methods to update single factual knowledge, Knowledge Conflict reflects the drawbacks of current knowledge editing approaches facing multiple edits. We further analyze whether current knowledge editing approaches will damage the implicit knowledge structure within LLMs or not. Here, we specify the concept of Knowledge Distortion and explore the mechanism of how knowledge editing works and impacts LLMs.
2.3. PROBLEM DEFINITION
Knowledge Distortion As stated in Meng et al. (2022), knowledge editing for LLMs can change the distribution of the next token, thus editing the model's output. Intuitively, fine-tuning on un- balanced data causes preference of certain generations, thus applying one edit at a time seems an extremely unbalanced update to the model. Particularly, in the multi-label scene, for given relation there is a set of objects Obj {o,; = 1, 3, 4,... that correspond to the subject and relation pair (s, r). When we perform an edit (s, o* + 01) (where g Obj), distribution next- token probabilities over Obj tends to favor on However, weakening the preference for generating other correct objects (s, r, o = 2,3, 4,... is undesirable for a robust editing method. Unfortu- nately, most editing methods exhibit negative impact on the correct objects beyond the target, that is, the knowledge structure for the (s, r) will be distorted.
Round-Edit Note that is not easy analyze Knowledge Distortion through single edit, since the distribution of the original model over labels in Obj is absent to be referred. Therefore, we design ROUND-EDIT setting which is
[e
0²)
Round-Edit
{ [ = (s + o1)
(10)
where 01 is the target object, and o* is the intermediate object. After applying the -DITT, expect that the post-edit model be consistent with the original model or perform better.
2.3.2 EVALUATION
Setup To investigate the performance and mechanism of knowledge distortion caused by knowl- edge editing, we construct dataset ROUNDEDIT from WikiData, which contains EASY and HARD splits. First of all. we collect several relations that possibly match multiple objects and then filter subjects. Then we select one object in the object list of multi-laheled (s,r) as the target ob- ject of BOUND-EDIT. Considering that the objects semantically related to editing target are more likely to generated by the post-edit model, we introduce two settings EASY and HARD (Dataset construction details are in Appendix A.1.2). As illustrated in Figure 4 (a), the edits in EASY tie an object to a certain subject by migrating a range of similar semantic objects, while HARD edits object that semantically distant from the true labels of (s, r) as depieted in Figure 4 (b). If other labels fall within the Migration Range, the post-edit model not only shows a predilection for the target object but also reinforces its confidence in these other labels. If not, they are overlooked.

---

Published as a conference paper at ICLR 2024
EASY
HARD
Method
Suce
D
IR 4
Suce
PR 1
GPT2-XL.
FT
89.50
6.47
74.47
72.24
90.06
11.38
80.83
80.82
MEND
78.22
6.48
87.86
86.88
80.50
9.73
90.56
89.36
ROME:
99.82
7.78
67.41
64.60
99.86
14.86
74.38
73.68
MEMIT
86.44
5.94
49.98
45.36
88.12
10.29
53.38
50.12
MEMIT+MLE
83.62
3.05
4,66
1.72
86.64
2.67
2.67
1.12
GPTJJ
FT
99.96
9.59
96.43
96.56
100.0
16.12
97.48
97.32
MEND
99.44
8.55
90.96
90.68
99.12
14.35
87.64
86.56
ROME
99.66
6.91
67.35
65.56
99.80
13.95
78.98
77.60
MEMIT
99.52
56.91
53.52
99.72
13.50
72.03
70.44
MEMIT+MLE
93.96
2.48
80.34
2.72
1.12
Table 2: Knowledge Distortion results of knowledge editing methods. Bold results denote the best performance in each situation.
Metrics Inspired by the teacher forcing generation in Meng et al. (2022), we define three new metries to measure the Knowledge Distortion of the edited model. Distortion (D) estimates the JS divergence of the distribution objects in Obj before and after ROUND-EDITING, that is:
= JS (Obj (s,r)), Pfw (Obj (s,r)));
(11)
where denotes the Distortion, p(Ob | -) a normalized probability distribution over Obj. This metric measures the extent of knowledge distortion. Moreover, inspired by Cohen et al. (2023) we introduce more specific metric called Ignore Rate (IR). The IR metric quantifies the extent to which the objects in Obj (excluding the target object 01) are disregarded or overlooked following the process of knowledge editing.
IR is calculated as follows:
TR
Σ (p (8,r)) Pf. (o (8,r))}:
(12)
Obj
where p(o (s, r)) is the generative probability of o. Furthermore, we design Failure Rate (FR) metric count the ratio of the case where their IR 0.5, that is
FR 1 {IR > 0.5].
(13)
To make reference to the basic performance on the dataset, we also calculate the Success Score (Suec) as same as $2.2, which computes the average Suce of the edits in ROUND-EDIT.
2.3.3 RESULTS ANALYSIS
We conduct experiments on GPT2-XL and GPT-J and then summarize the results in Table 2. The variance of the distribution Obj is illustrated in Figure 5 over EASY and HARD settings. Based on the results, we design an effective method Multi-Label Edit (MLE) to address this problem.
Analysis As depicted in Table 2, notable knowledge distortion is observed in FT and MEND, as evidenced by their high IR and FR values. ROME and MEMIT, while still exhibiting some level of distortion, demonstrate lower values compared to FT and MEND. This indicates that ROME and MEMIT have integrated unique mechanisms to minimize disturbances to the implicit knowledge structure LLMs during knowledge editing. Further, we observe an interesting phenomenon from the EASY and HARD scenarios: when the Suce metric reaches high values, FT and MEND not only underperform but also demonstrate minimal variation across the two datasets; In contrast, ROME and MEMIT not only demonstrate an advantage in absolute values but also show a big gap in IR and FR between EASY and HARD settings. This leads to the conclusion that the editing approaches of ROME and MEMIT effectively achieve scope transfer (better generalization ability) through se- mantic relations. Moreover, as shown in Figure 5, we illustrate an example |Obj = in EASY and HARD respectively display the inner change of the distribution over Obj. We observe clear knowledge distortion emerges which inadvertently impacts the LLMs' implicit knowledge structure, particularly when the structure encapsulates diverse semantic spectrum (e.g., HARD setting).
8

---

Published as a conference paper at ICLR 2024
Rehiiive Probahility
GPT-I
8.54
GPIJ
0,99
MEMIT
MEMIT
MEMIT-MLE
MEMIT-MLE
0.13
0.47
0.16
0.36
0.18
9.06
6,66
0.02 War College
0.01
8.02
0.01
6.65
8.08
Targat United Stotes
Milltary Acadomy
Unked Status
Iadiona bniverdit
Sabee Regina
Univarity
Target: Soholov Award
Right Live- Ilhond Award
Blue Ploust Award
Oraketaly Frite
Carl von Howsn Right
Atrard for Sersices
Braes Krrbly
Gencral Staff Collgge
Command
Input: James J. Lovelace was educated at
Input: Uri Avnery received award(s), such as (b) HARD
(a) EASY
Figure 5: Typical results on sampled cases in EASY and HARD. The height of each bar refers to the generative probability over the 5 true labels of rs, Red scores indicate ignorance of true labels, whereas green scores display the positive effects.
A Simple Solution: Multi-Label Edit (MLE) To be noted, aim evaluate how knowledge distortion affects the generation performance of post-edited models via its impact on IR and FR metrics. The results in Figure 2 indicate that even ROME and MEMIT exhibit poor performance in the EASY scenario, affecting over 50% of the true labels and cases. To this end, we introduce simple solution: Multi-Label Edit (MLE), tailored for such multi-label scenarios. Specifically, as shown in Figure 2, we take three objects (one-to-many triples taken from existing KG which is included in our datasets) as the editing target, expecting to preserve the performance over the true labels (More details in Appendix C). From the results in Figure 2, it is evident that MLE can help mitigate the influence of knowledge distortion, which is also demonstrated by cases in Figure 5.
RELATED WORK
Recently, there has been a surge in knowledge analysis and editing for LLMs. This encompasses efforts to demystify the knowledge storage mechanisms inherent in these "black- box" neural net- works (Geva et al., 2021; Dai et al., 2022). Researchers have delved into deconstructing the Trans- former architecture by exploring the concept of knowledge/skill neurons (Dai et al., 2022; Chen et al., 2023b; Wang et al., 2022), and complex system science (Holtzman et al., 2023). Additionally, there's a keen interest in developing effective strategies to edit knowledge within LLMs (Xu et al., 2023; Hase et al., 2023; Hartvigsen et al., 2022; Cheng et al., 2023; Hernandez et al., 2023; Li et al., 2023a; Wang et al., 2023a; Xu et al., 2023; Yu et al., 2023; Wang et al., 2023c; Li al., 2024; Huang et al., 2024; Ma et al., 2024a; Song et al., 2024; Ma et al., 2024b; Peng et al., 2024; Wei et al., 2024; Gu al., 2024). More efforts devoted to investigating the completeness and robustness of these methods. In order to determine the exact scope of effectiveness when applying certain edit, it is crucial to explore evaluation samples that are correlated with the edits (Yin et al., 2023; Cohen al., 2023; Wu et al., 2023; Soeelsche--bberaae
et al., 2023; Wei et al., 2023; Onoe et al., 2023; Wang al., 2023c; Ma et al., 2023). Simultancously, knowledge editing injects new facts into LLMs, thus verifying the connections or reasoning among the new knowledge and the pre-existing knowledge in the model even multiple new knowledge has become a prevalent topic (Zhong et al., 2023; Li al., 2023b; Hua et al., 2024; Ju et al., 2024; Hua et al., 2024). Furthermore, recent work has also begun to address the catastrophic forgetting issue (Gupta et al., 2024) caused by knowledge editing and the problem of guiding language models to generate harmful content (Hazra et al., 2024).
4 DISCUSSION AND CONCLUSION
This paper illustrates that knowledge editing can bring in side effects of knowledge conflict, leading to knowledge inconsistency and may even enhance the hallucination of LLMSs (Ji et al., 2023; Rawte et al., 2023; Zhang et al., 2023). We argue that conflict detection via logical rules or KG reasoning may avoid knowledge conflict. However, the side effect knowledge editing for LLMs is far from satisfactory, which needs efforts for future works. Furthermore, knowledge editing technologies can serve as a scaffold to update, sanitize, or personalize LLMs (Ishibashi & Shimodaira, 2023; Chen et al., 2023a; Mao et al., 2023). Empirical observation of knowledge distortion in this paper brings about a severe issue to updating knowledge in LLMs. Although we introduce Multi-Label Edit to mitigate knowledge distortion, more work should be developed to avoid the problem.

---

Published as a conference paper at ICLR 2024
ACKNOWLEDGMENTS
We would like to express gratitude to the anonymous reviewers for kind comments and MEMIT (Meng al., 2023), EasyEdit (Wang al., 2023b) and many other related works for their open- source contributions as well as GPT-4 Service (OpenAl, 2023). This work was supported by the National Natural Science Foundation of China (No. 62206246), the Fundamental Research Funds for the Central Universities 226-2023-0013388 Zhejiang Provincial Natural Science Foun- dation of China (No. LGG22F03001 Ningbo Natural Science Foundation (2021J190), CAAI- Huawei MindSpore Open Fund, Yongjiang Talent Introduction Programme (2021 A-156GG) CCF- Tencent Rhino-Bing Open Research Fund, and Information Technology Center and State Key Lab of CAD&CG, Zhejiang University.
ETHICAL CONSIDERATIONS
The datasets and models presented in this paper are designed for exploratory analysis of L.Mss. crucial note that data from WikiData isn't always error-free, and GPT-4 might also introduce biases. As a result, when constructing datasets of knowledge editing for LLMs, there is a risk of sur- facing knowledge infused with offensive language or prejudicial content. During our experiments, we have carefully reviewed all data, ensuring the removal of any toxic or offensive content.
REPRODUCIBILITY STATEMENT
Codes and datasets are
available at https://github con/ We provide technical details of dataset construction in- cluding CONPLICTEDITand ROUNDEDITin Appendix A.1.1 and Appendix A.1.2, respectively. We provide detailed experimental settings in Appendix B. We provide more technical details of Multi-Label Edit (MLE) in Appendix C.
PitfallsKnooleeeeeeddl
REFERENCES
Lukas Berglund, Meg Tong, Max Kaufmann, Mikita Balesni, Asa Cooper Stickland, Tomasz Ko- rbak, and Owain Evans. The reversal curse: Lims trained "a is b" fail to learn "b is a". CoRR, abs/2309. 12288, 2023. doi: 10,48550/arX1w.23399.11 12288. URL 10. 48550/ 11.550//rxiv// arwii..iivv..... 2309. 12288.
Nicola De Cao, Wilker Aziz, and Ivan Titov. Editing factual knowledge language models. In Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih (eds.), Proceed- ings of the 2021 Conference on Empiricul Methods in Natural Language Processing, EMNLP 2021, Virtual Event / Punta Cana, Dominican Republic, 7-11 November, 2021, pp. 6491-6506. Association for Computational Linguistics, 2021. doi: 10. 10.11863511222 1tt22211.cmmly. .emml- main.522. URL https: //do: org/10/g///oooooooo 10 11..0.11....... 1 v1 /2221 emnlp main 522.
Jin Chen, Zheng Liu, Xu Huang, Chenwang Wu, Qi Liu, Gangwei Jiang, Yuanhao Pu, Yuxuan Lei, Xiaolong Chen, Xingmei Wang, Defu Lian, and Enhong Chen. When large language mod- els meet personalization: Perspectives of challenges and opportunities. CoRR, abs/2307.16376 2023a. doi: 10.48550/arxx.22277.66666 URL https: doi. wtt.oo/./....../ar....... 2307 16376.
Yuheng Chen, Pengfei Cao, Yubo Chen, Kang Liu, and Jun Zhao. Journey to the center of the knowledge neurons: Discoveries of knowledge neurons. CORR, abs/2308. 13198, 2023b. doi: 10.48550//xXiv.2308..33 https://doi. /ttp:///....o/10..4////////////0 arxiv 2308 13198.
knowledge neurons and degenerate
URL
Siyuan Cheng, Ningyu Zhang, Bozhong Tian, Zelin Dai, Feiyu Xiong, Wei Guo, and Huajun Chen. Editing language model-based knowledge graph embeddings. CoRR, abs/2301.10405, 2023, doi: 10.48550/arXiv.2301. 10405. URL httpt: h wtt. 10. 48550/arXiv. 2ttp//awwi............ 10405.
Roi Cohen, Eden Biran, Ori Yoran, Amir Globerson, and Mor Geva. Evaluating the ripple effects of knowledge editing in language models. CoRR, abs/2307.12997777 2023, doi: 10.488550/axxv.23377. 12976. URL https::///// htappp:::/d..../// oorg/10.0.. 48550 a vttp//aavvvv....... 2307.
12976.
10

---

Published as a conference paper at ICLR 2024
Damai Dai, Li Dong, Yaru Hao, Zhifang Sui, Baobao Chang. and Furu Wei. Knowledge neurons in pretrained transformers. In Smaranda Muresan, Preslav Nakov, and Aline Villavicencio (eds.), Proceedings the the 60th Annual Meeting of the Association for Computational Linguistics (Volume I: Long Papers), ACI. 2022, Dublin, Ireland, May 22-27, 2022, pp. 8493-8502. Association for Computational Linguistics, 2022. doi: 10. 11863//1/202222ccl-m6
URL https: /doi.
org/ org/10.186330 w202222.22222222 acl-long1nnnngg 581.
Mor Geva, Roei Schuster, Jonathan Berant, and Omer Levy. Transformer feed-forward layers are key-value memories. In Marie-Francine Moens, Xuanjing Huang, Lucia Specia, and Scott Wen- tau Yih (eds.), Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, EMNLP 2021, Virtual Event Punta Cana, Dominican Republic, 7-11 November, 2021, pp. 5484-5495, Association for Computational Linguistics, 2021. doi: 11.18665//1/2211. emnlp-main 446. URL https: : doi. org/ 10. 18653/v1/200000 emnlp main. 64
Jia-Chen Gu, Hao-Xiang Xu, Jun-Yu Ma, Pan Lu, Zhen-Hua Ling, Kai-Wei Chang, and Nanyun Peng. Model editing can hurt general abilities of large language models. CoRR, abs/2401.04700, 2024. doi: (044555/ARXXV.240..000 04700. URL https: //t::///oiioo/ wttp:/...r............. 2401. 04700.
Akshat Gupta, Anurag Rao, and Gopala Anumanchipalli, Model editing at scale leads to gradual and catastrophic forgetting. CoRR, abs/2401.0045$3 2024, doi: 104A555/ARXI224000074.... URL wwt.h:tpts/sttt org/10.0.....
240.:..www................ 270453
Thomas Hartvigsen, Swami Sankaranarayanan, Hamid Palangi, Yoon Kim, and Marzyeh Ghas- semi. Aging with GRACE: lifelong model editing with discrete key-value adaptors. CoRR, abs/2211.11031,, 2022. doi: 10448500arxiw.221111101 URL https://doi.com org/10.
48550/arXiv. 2211 11031.
Peter Hase, Mohit Bansal, Been Kim, and Asma Ghandeharioun. Does localization inform editing? surprising differences in causality- -based localization vs. knowledge editing in language models. CoRR, abs/2301 004213, 2023. doi: 10448550Waxxw::330..0022 URL https: //w/i//////////////////. org/ 10. 00.48550/arxiiv 2301. 2421.
Rima Hazra, Sayan Layck, Somnath Banerjee, and Soujanya Poria. Sowing the wind, reaping the whirlwind: The impact of editing language models. CoRR, abs/2401.10647, 2024. doi: 10.48550/ ARXIV.2401.1064) URL https: /tt../oo.ooooooooo /10. 200.500002.V..20..0004......
Evan Hernandez, Belinda Z. Li, and Jacob Andreas. Measuring and manipulating knowledge repre- sentations in language models. CoRR, abs/2304.007400, 2023. doi: 10.48500acxiv22244.07740
URL https: http///ooooo////////////oo
2304. 00740.
Xanh Ho, Anh-Khoa Duong Nguyen, Saku Sugawara, and Akiko Aizawa. Constructing A multi- hop QA dataset for comprehensive evaluation of reasoning steps. In Donia Scott, Núria Bel, and Chengqing Zong (cds.), Proceedings of the 28th International Conference on Computational Linguistics, COLING 2020, Barcelona, Spain (Online), December 8-13, 2020, pp. 6609-6625, In- ternational Committee on Computational Linguistics, 2020. doi: 10.18653/w1/2220.cool/// main. 580. URL https:://///.....t. /ttp::://doooooo orgg / 10 18653/6663 vv 10001222222/22/ co Lng-ma . 580.
Jason Hoelscher- Beeescher-OObrriierr, Julia Persson, Esben Kran, Ioannis Konstas, and Fazl Barez. De- tecting edit failures in large language models: An improved specificity benchmark. In Anna Rogers, Jordan I.. Boyd-Graber, and Naoaki Okazaki (eds.), Findings the Association for Com- putational Linguistics: ACI. 2023, Toronto, Canada, July 9-14, 2023, pp. 11548-1155 11559. As- sociation for Computational Linguistics, 2023. doi:
10255/3/v/222..f. 1220....ndn2g..2-o-. findinngs--cl.733. URL https:: //co:////oii oro/l0.....oooooooo 1 h0t/////22222.. wttp://.......nn......ee.s acl. 733.
Ari Holtzman, Peter West, and Luke Zettlemoyer. Generative models as a complex systems science:
How can we make sense of large language model behavior? CoRR, abs/2308.00 00189, 2023. doi: X0.488550/axiv.22388... 001889 URL http::/ /ooi. orgg 48550/ 2308 00100
Wenyue Hua, Jiang Guo, Mingwen Dong, Henghui Zhu, Patrick Ng. and Zhiguo Wang. Prop- agation and pitfalls: Reasoning- based assessment of knowledge editing through counterfac- tual tasks. CoRR, abx/2401.17585. 2024. doi: 10.4550/ARXXV..200.1333. URL https: 17585.
//doi.org/10. 48550//
2401

---

Published as a conference paper at ICLR 2024
Youcheng Huang, Wenqiang Lei, Zheng Zhang, Jiancheng Lv, and Shuicheng Yan. See the unseen:
Better context-consistent knowledge-editing by noises. CoRR, abs/2401 2024. doi: 10. 48550/ARXIIV2200.077544 URL https:/
www....ww..........oa 07544. 0eeg///0.48550.0
Zeyu Huang, Yikang Shen, Xiaofeng Zhang, Jie Zhou, Wenge Rong, and Zhang Xiong. Transformer-patcherr One mistake worth one neuron. CoRR, abs/2301.09785, 2023, doi: Atthtt / /tt////iiiia/ org/10..01.... 48550//a55ii1 2301. 0300.09785.
10.48850kurXiv.2300.0999
Gabriel Ilharco, Marco Túlio Ribeiro, Mitchell Wortsman, Ludwig Schmidt, Hannaneh Hajishirzi, and Ali Farhadi. Editing models with task arithmetic. In The Eleventh International Confer- ence on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.net, 2023. URL https: http:///oppeeeeeeeeeeeeeee net. http/ddfffic OKwfB- jrj.
Yoichi Ishibashi and Hidetoshi Shimodaira. Knowledge sanitization of large language models. CoRR, abs/2309. 11852, 2023, doi: 10448500aaXxx:2200111822 URL https: /ooi.org/////oooooooooooo 10.4 48..8855/// (01550/arXivv www....ww......................a 11852.
Ziwei Ji,
Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Yejin Bang, Andrea Madotto, and Pascale Fung. Survey of hallucination in natural language generation. ACM Comput. Surv. 55(12))248:1224:388 1145/3571730
2023.
doi: 10. 105354333570 URL https://doi. http//....o../....
Tianjie Ju, Yijin Chen, Xinwei Yuan, Zhuosheng Zhang, Wei Du, Yubin Zheng, and Gongshen Liu. Investigating multi-hop factual shortcuts in knowledge editing of large language models. arXiv preprint arXiv:2402.11900 2024.
Diederik Kingma and Jimmy Ba. Adam: method for stochastic optimization. arXiv preprint arXiv: 1412.6984 2014.
Xiaopeng Li, Shasha Li, Shezheng Song, Jing Yang, Jun Ma, and Jie Yu. PMET: precise model editing in transformer. CoRR, htss/2308.08742. 2023a. doi: 10448550/axxx..22000077444 URL https: / /ow.. org/10.............o.oo......... http/110.8855555 48550 /0.550//rxiv/ 2308 08742.
Xiaopeng Li, Shasha Li, Bin Ji, Shezheng Song, Xi Wang. Jun Ma, Jie Yu, Xiaodong Liu, Jing Wang, and Weimin Zhang. SWEA: changing factual knowledge in large language models via subject word embedding altering. CoRR, abs/2401. 2024, doi: 1044550/ARXXV.240.115555 URL https:://dt/
485g://004555/////0 arviv
Zichao Li, Ines Arous, Siva Reddy, and Jackie Chi Kit Cheung. Evaluating dependencies in fact editing for language models: Specificity and implication awareness. In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Findings of the Association for Computational Linguistics: EMNLP 2023, Singapore, December 6-10, 2023, pp. 7623-7636. Association for Computational Linguis- org /2033. findings-emn 511.
tics, 2023b. URL peetpa://aclanteee
Edward Lorenz. The butterfly effect. World Scientific Series on Nonlinear Science Series A, 39: 91-94, 2000.
Jun-Yu Ma, Jia-Chen Gu, Zhen-Hua Ling, Quan Liu, and Cong Liu. Untying the reversal curse via bidirectional language model editing. CORR, abs/2310.10322. 2023. doi: (0.48550/ARXIV.2310. 10322. URL https: / /oi. org/ollg///ooooooooo www50855555500
2310.10322.
Jun Yu Ma, Jia-Chen Gu, Ningyu Zhang, and Zuen-Hua Ling. Neighboring perturbations of knowl- edge editing on large language models. CoRR, abs/2401.11623,, 2024a. doi: 10.48550/ARXIV. 2401. URL. hwt.hhttp////. /ttp///ooiio///
http///o..oooo///////////ooo org/ 0 48550//5aXX0 2401 176223.
Xinbei Ma, Tianjie Ju, Jiyang Qiu, Zhuosheng Zhang, Hai Zhao, Lifeng Liu, and Yulong Wang. Is possible to edit large language models robustly? CoRR, abs/2402.005877, 2024b. doi: 10.48550/ ARXIV.2402.05877. URL hwwps:t.p..... //tp:://///////////
05827.
Shengyu Mao, Ningyu Zhang, Xiaohan Wang. Mengru Wang, Yunzhi Yao, Yong Jiang, Pengjun Xie, Fei Huang, and Huajun Chen. Editing personality for llms. CoRR, abs/2310.021688, 2023. doi: 10.48550/ARXIV.21100022
URL https:/ /ttp/iiooooooo www.loooog...... wttp//....50///aa....... 2310.
021 68.
12

---

Published as a conference paper at ICLR 2024
Kevin
Meng, cating and URL
David editing
Bau, factual associations GPT. In NeurIPS, 2022. files/paper/2222 httpee/r/////hhahh///o html.
Alex Andonian, and Yonatan Belinkov. Lo-
Kevin Meng, Arnab Sen Sharma, Alex J. Andonian, Yonatan Belinkov, and David Bau. Mass- editing memory transformer. The Eleventh International Conference on Learning Repre- sentations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.net, 2023. URL https: // //openeeview net /pd tttp:/pffdddddddd aidd--idddaaae MKkccMIYyyHHy
Eric Mitchell, Charles Lin, Antoine Bosselut, Chelsea Finn, and Christopher D. Manning. Fast model editing scale. In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022. OpenReview.ner, 2022a. URL. https:// openreviee net/ forum? www.umiiddO00 wwt.iddddoooeeeeeeeee
Eric Mitchell, Charles Lin, Antoine Bosselut, Christopher D. Manning, and Chelsea Finn. Memory- based model editing at scale. In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvári, Gang Niu, and Sivan Sabato (eds.), International Conference on Machine Learning, ICML 2022, 17-23 July 2022, Baltimore, Maryland, USA, volume 162 of Proceedings of Machine Learning Research, pp. 15817-15831. PMLR, 2022b. URL https: ppwceedings.....c/////ppppp...sp mmlr. press/ vv1 62 wwwtichellle 122a html.
Yasumasa Onoe, Michael J. Q. Zhang, Shankar Padmanabhan, Greg Durrett, and Eunsol Choi. Can Ims learn new entities from descriptions? challenges in propagating injected knowledge. In Anna Rogers, Jordan L. Boyd-Graber, and Naoaki Okazaki (eds.), Proceedings of the 61st An- nual Meeting the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2023, Toronto, Canada, July 9-14, 2023, pp. 5469-5485. Association for Computational Linguis- act.//2222..cllloooo
tics, 2023. doi: 10. 10.18853/v1/2023 acl acw.loooo-.00000.. URL htt.:p////oa....o///...0... 2023 ac lolcclllong 300.
OpenAl. GPT-4 technical report. CoRR, abs/2303.08774, 2023. doi: 1044555Waaxiv.223..87744 www.444/aa.....333....3
URL
Hao Peng, Xiaozhi Wang, Chunyang Li, Kaisheng Zeng, Jiangshan Duo. Yixin Cao, Lei Hou, and Juanzi Event-level knowledge editing. arXiv preprint arXiv:2402. 13093, 2024.
Fabio Petroni, Tim Rocktäschel, Sebastian Riedel, Patrick S. H. Lewis, Anton Bakhtin, Yuxiang Wu, and Alexander H. Miller. Language models as knowledge bases? Kentaro Inui, Jing Jiang, Vincent Ng, and Xiaojun Wan (eds.). Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, EMNIPP-JCCNLA 2019, Hong Kong, China, November 3-7, 2019, pp. 2463-2473. Association for Computational Linguistics, 2019. doi: 10.18653/v1/D19-122 1250. URL https: //doi. 10ooi.org/10.11.
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsupervised multitask learners. OpenAl blog, 1(8):9, 2019.
Vipula Rawte, Amit P. Sheth, and Amitava Das. A survey of hallucination in large foundation models. CoRR, 0ttp//239.00922..... 2023, doi: 10448550/rxxi..330005999 URL https: hTr//10.4855/arx22 2309 05922.
Xiaoshuai Song, Zhengyang Wang, Keqing He, Guanting Dong, Yutao Mou, Jinxu Zhao, and Weiran Xu. Knowledge editing on black-box large language models. CoRR, abs/2402.08631,, 2024. doi: 10.48550ARRXV.2400..000 08631.
URL. httpp::/ddon wtg/ /t5//////....../////..v..222
Denny Vrandecic and Markus Krötzsch. Wikidata: free collaborative knowledgebase, Commun. ACM, 57(10):78-85, 2014. doi: 10.1145/2624489, URL 2629488.
Ben Wang and Aran Komatsuzaki. GPT-J-6B: 6 Billion Parameter Autoregressive Language Model. https: h/pp:://// //// tig...u.t...... com/ www.kingooooooeccoo.eecooooo.c hwt.//ehttaaasffrrteeee r- jax, May 2021.
13

---

Published as a conference paper at ICLR 2024
Jiaan Wang, Yunlong Liang, Zengkui Sun, Yuxuan Cao, and Jiarong Xu. Cross-lingudl knowledge editing in large language models. CoRR, Abs/2309.08952, 2023a. doi: 10.48550/a2Xwv.23099 08952. URL https: /ttps://ddll /ttp///iiiio www.ioorg11.... 4tt.////00555//xi1... 2309 08952.
Peng Wang, Ningyu Zhang. Xin Xie, Yunzhi Yao, Bozhong Tian, Mengru Wang. Zekun Xi, Siyuan Cheng, Kangwei Liu, Guozhou Zheng, and Huajun Chen. Easyedit: An easy-to-use knowledge editing framework for large language models. CoRR, abs/2308.07266 2023b. doi: 10.48550/
arXiv.2308.07269. URL https: //w///d////// erg/10 485500/aaX0. wwtp:/iwv....... 2308. 07269.
Weixuan Wang. Barry Haddow, and Alexandra Birch. Retrenval-augmenned multilingual knowledge editing. CoRR, abs/2312. 2023c. doi: 1044550/ARXIV.211..105
URL https:
2312 13040.
Xiaozhi Wang, Kaiyue Wen, Zhengyan Zhang, Lei Hou, Zhiyuan Liu, and Juanzi Li. Finding skill neurons in pre-trained transformer-basee language models. In Yoav Goldberg, Zornitsa Kozareva, and Yue Zhang (eds.), Proceedings of the 2022 Conference Language Processing, EMNLP 2022, Abu Dhabi, United Arab Emirates, December 7-11, 2022, pp. 11132-11152. Association for Computational Linguistics, 2022. doi: 10.18653/w1/2022. emnlp-main.766 765. URL https::t//// : /tt////oi.n 1tapr//1..1855/11/2000
Empirical Methods in Natural
emnlp momnlp-men masllppmaaiiie m
Yifan Wei, Xiaoyan Yu, Huanhuan Ma, Fangyu Lei, Yixuan Weng, Ran Song, and Kang Liu. As- sessing knowledge editing in language models via relation perspective. CoRR, 2023. doi: 10.4550/ARXIV.231...000 1.09553. URL https://ddot /ttp///oii.oo /ttp://i.....111..888888.8550000 09053.
Zihao Wei, Liang Pang, Hanxing Ding, Jingcheng Deng, Huawei Shen, and Xueqi Cheng. Stable knowledge editing in large language models. arXiv preprint arXiv:2402.13045 2024.
Suhang Wu, Minlong Peng. Yue Chen, Jinsong Su, and Mingming Sun. Eva-kellm: A benchmark for evaluating knowledge editing of 1lms. CoRR, abs/2308.09954, 2023. doi: 10.48550/arXiw.2300 09954. httpt //tpii://///////.// wwt..ooogg/1....
2308 09954.
Yang Xu, Yutai Hou, Wanxiang Che, and Min Zhang. Language anisotropic cross-lingual model editing. In Anna Rogers, Jordan L. Boyd-Graber, and Naoaki Okazaki (eds.), Findings of the As- sociation for Computational Linguistics: ACL 2023, Toronto, Canada, July 9-14, 2023, pp. 5554- 5569. Association for Computational Linguistics, 2023. doi: 10.18653/v1/0023 finn::/iis/ findings--cll.a3 acl.333. URL https::////. / doi 1ttp//...165/////2000 findings soaddings-aad acl. 343.
Yunzhi Yao, Peng Wang. Bozhong Tian, Siyuan Cheng, Zhoubo Li, Shumin Deng, Huajun Chen, and Ningyu Zhang. Editing large language models: Problems. methods, and opportunities. CoRR, abs/2305. 13772, 2023. doi: 10448550/axxx:2255.11122 URL https: //doi. org/ 08.44555/arxi11
Xunjian Yin, Jin Jiang, Liming Yang, and Xiaojun Wan. History matters: Temporal knowledge editing in large language model. CoRR, abs/2312.054977, 2023, doi: 10.48550/ARXIV23122 05497. URL https:
/tpps://dol. /do.. orgg 4ww50//asii...../ 2312, 05497.
Lang Yu, Qin Chen, Jie Zhou, and Liang He. MELO: enhancing model editing with neuron-indexed dynamic lora. CoRR, abs/2312.11195, 2023. doi: 10445550/ARX1.23122.1172 URL https:
Yue Zhang, Yafu Li, Leyang Cui, Deng Cai, Lemao Liu, Tingchen Fu, Xinting Huang, Enbo Zhao, Yu Zhang, Yulong Chen, Longyue Wang, Anh Tuan Luu, Wei Bi, Freda Shi, and Shum- ing Shi. Siren's song the Al ocean: A survey on hallucination in large language models. CoRR, abs/2309.01219, 2023. doi: 10488500arXi.:339.00211,, URL https: htto://ooor/////oo 10. 4...4855/arxii1 2309 01219.
Zexuan Zhong, Zhengxuan Wu, Christopher D. Manning, Christopher Potts, and Danqi Chen. Mquake: Assessing knowledge editing in language models via multi-hop questions. CoRR, 2023. doi: 2445500arXiw.2205.14777 URL pittps://doo /tttpss///ooooo//1.....
48550/arxiv, 2305.14795.
14

---

Published as a conference paper at ICLR 2024
A DATASETS DETAILS
A.1 CONSTRUCTION OF DATASETS
A.1.1 CONFLICTEDIT
We construct our CONFLICTEDIT dataset based on WikiData (Vrandecic & Krötzsch, 2014), us- ing the raw data and some logical rules mined from it. From the original WikiData data to the CONFLICTEDIT dataset, we have taken the following steps:
Step First, we determine the binary and ternary logic rules we need using the method of Ho al. (2020) and manually select rules/relations, thereby obtaining the candidate sets of relations in REVERSE EDIT and COMPOSITE EDIT, respectively.
Step 2
Next, based the nominated relations and corresponding logical rules, we construct Wiki- Data Queries and use the WikiData Query Service to mine an average of 50,000 factual combinations that meet the requirements for each logical rule.
Step We sample from each factual combination present in the logical rules, excluding those with descriptions in numeric formats those with identical entities. To ensure accuracy, we validate the integrity of the associated fact while constructing the COMPOSITE EDIT dataset for each model (c.g., GPT-XL, GPT-J). Specifically, we use the prompt associated with the fact as an input, prompting the model to generate outputs of a predetermined length using the Top-1 next-token generation approach. We then verify whether these outputs include the object of the fact.
Step Finally, we select 2,500 data points each from binary and ternary factual combinations. However, due to GPT-2 XL's comparatively weaker performance in verifying the accuracy of tied facts against GPT-J, we only select 2,000 data points for it. We then sample edited target entities under each relation to assemble the final REVERSE EDIT and COMPOSITE EDIT datasets. Addi- tionally, using samples from both datasets, we create the COVERAGE EDIT dataset to serve as an experimental benchmark.
Step We've We've manually crafted editing prompts for each relation, drawing inspiration from their descriptions on WikiDDta. Based on this, we employ GPT-4 (OpenAI, 2023) to generate 20 alterna- tive phrasings for each relation's prompt. From these, we handpick the 10 most appropriate prompts for our generalization evaluation.
For training MEND, samples with locality facts are essential. Prioritizing training efficacy, we opt not to utilize the NQ dataset, as done by ZsRE. Instead, our sampling is anchored in COUNTERFACT (Meng et al., 2022). Table illustrates the data structure we' ve established for both REVERSE EDIT and COMPOSITE EDIT, also highlighting specific example from COMPOSITE EDIT.
R MotherASpouse--FFtte
F (Philip Leakey, Mother, Mary Leakey) (Mary Leakey, Spouse, Louis Leakey) (Philip Leakey, Father, Louis Leakey)
3
e1: (Mary Leakey, Spouse, Louis Leakey Mary Campbell of Mamore) 62: (Philip Leakey, Father, Mary Campbell of Mamore - Andres Ehin)
kj
(Philip Leakey, Mother, Mary Leakey)
ko
(Mary Leakey, Spouse, Mary Campbell of Mamore)
kn
(Mary Leakey, Spouse, Andres Ehin)
Table 3: An instance in COMPOSITE EDIT, which consists of logical rule R, three triples in the factual combination F, an edit pair E, a tied fact ky and an knowledge update ki and k..
15

---

Published as a conference paper at ICLR 2024
Dataset Split
Relation Type
Input
Intermediate Object
True Labels
James was educated at
Educational
Target: United Army
RoucatedAt
Commanity
Command and General Staff College, United States Mili- tary Academy. Indiana Univer- sity. Salve Regina University, Naval War College
HARD
1-3 Awardie
Uri Avnery roceived awardís), as
Hero of the Mongolian People's Republic
Target: Sokelov Award, Right Livelibood Award, Award, Carl von Ossietzky Prize, Bruno Kreisky Award for vices Human Rights
Planet
Table 4: Examples in EASY and HARD corresponding to the results in Figure
A.1.2 ROUNDEDIT
We construct ROUNDEDIT dataset using data sourced from WikiData, enriched by some 1-to-n relations mined from it. Transitioning from the raw WikiData data the ROUNDEDIT dataset required several stages:
Step We first manually select some 1-to-n relations and construct WikiData Queries, utilizing the WikiData Query Service mine 5000 (s,r) r) pairs along with the correct object set Obj. From each relation, we selectively sample, ensuring that the value in the I-n relations remains below 10. Furthermore, we exclude samples with descriptions in numeric format.
Step 2
For each LLM, we perform next-token generation tests on the raw data samples, filtering these samples based on generation probabilities. To delineate the EASY and HARD dataset cate- gories, we examine the correlation between semantic relevance and generation probabilities. Using this analysis, we measure the semantic relationship among the correct labels and subsequently sam- ple to establish both the EASY and HARD categories, each comprising 2,500 data points.
Step 3 Similarly, we manually create editing prompts for every relation, drawing from their re- spective descriptions on WikiData. Based on this, we utilize GPT-4 (OpenAI, 2023) to generate 20 rephrased prompts. Of these, we handpick the top 10 prompts, incorporating them into the general- ization evaluation.
We also sample locality facts from COUNTERFACT (Meng et al., 2022) to train the MEND Figure showcases selected examples from the EASY and HARD settings.
B IMPLEMENTING DETAILS
(FT), we follow Meng et al. (2022) to re-implement their study, which uses Adam (Kingma & Ba, 2014) with early stopping minimize -loglPor [os | p), changing only milpprjj weights at selected layer in GPT2-XL and layer 21 in GPT-J. For both models, all hyper-parametters follow default settings. To ensure fairness in the experiments, we always the unconstrained fine-tuning approach.
For basic
Mitchell et al. (2022a) develops an efficient method for locally editing language models using just single onput-output pair. Essentially, MEND employs technique manipulate the gra- dient of fine-tuned language models which leverages a low-rank decomposition of gradients. Here adopt the COUNTERFACT (Meng et al., 2022) evaluate the locality while training MEND. For convenience, we train MEND respectively on our CONFLICTEDIT and ROUNDEDIT datasets using EasyEdit (Wang et al., 2023b). The hyper--arameters follow default settings both on GPT2-XL and GPT-J.
MEND
ROME ROME, as proposed by Meng et al. (2022), conceptualizes the MLP module as a straight- forward key-value store. For instance, if the key represents subject and the value encapsulates
16

---

Published as a conference paper at ICLR 2024
knowledge about that subject, then the MLP can reestablish the association by retrieving the value that corresponds to the key. In order to add a new key-value pair, ROME applies a rank-one modi- fication to the weights of the MLP, effectively injecting the information directly. This method allows for more precise and direct modification of the model's 's knowledge through knowledge locat- ing. We directly apply the code and MI.P weight provided by the original paper and keep the default setting for syper-parameters.
MEMIT MEMIT (Meng et al., 2023) builds upon ROME to insert many memories by modifying the MIP weights of range of critical layers. We test the ability of MEMIT using their source code and hyper-parameters follow the same default settings.
Explicit (CSexp) and Implicit (CSimp) Here, 'Explicit' and 'Imliccit' are in reference to the second edit. When computing Pf,' (ka), we can utilize the edit target (82, r2, r2, from the second edit c2 (s2, 22, 02 og) as the input for calculating kn which we term the Explicit mode. Conversely, we employ a more relevant latent target associated with the first edit as the input for computing kns and this mode is referred to as the Implicit mode. Table 5 lists the edit tatget of each dataset split in every mode.
Dataset Split
Explicit Mode
Implicit Mode
Coverage
(82, r2, o2)
(11,
Reverse
(#2, 72, o5)
(02, 92)
Compusite
(#2, 12, o5)
($1, 02)
Table 5: Explicit (CSexp) and Implicit (CSimp) of each dataset split.
C DETAILS OF MULTI-LABEL EDIT
To address the issue of Knowledge Distortion in knowledge editing, we introduce novel approach termed Multi-Label Editing (MLE). This method diverges from conventional knowledge editing. Our objective is simultaneously edit groups of knowledge that possess the same (s, r), into the model, leveraging MEMIT's s multi-editing capabilities during the process. For instance, when we make an edit Joe Biden was born California - Florida, we can consider the differences in the knowledge structures of California and Florida, recall their related semantics within the model (such as America), and then regard 'Florida' and 'Amrriaa' as labels utilizing MLE to edit.
Our findings indicate that such concurrent editing approach not only prevents unnecessary knowl- edge updates but also ensures the model doesn't it favor a particular label. Through rigorous exper- imental results, we demonstrate that MLE serves distortion issues. By identifying and incorporating related or essential knowledge within the model prior to the editing of specific piece of knowledge, we can reduce the potential disruption to the LLM's inherent knowledge structure caused by the editing process.
effective strategy to mitigate knowledge
17


Reviews:
[{"review_id": "pGrs5j6YNp", "reviewer": "Reviewer_mRJP", "summary": "This paper comprehensively explores the side effects of knowledge editing for large language models (LLMs), highlighting potential risks in real-world use cases. To facilitate a rigorous evaluation, the researchers introduce two innovative datasets specifically crafted to highlight the unintended consequences of knowledge editing. The study offers solutions for knowledge conflicts and introduces the MLE method to mitigate distortion risks. It also discusses implementation challenges and prospects of knowledge editing for LLMs.", "questions": "please refer to the weakness section.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "The authors assess the risks associated with current knowledge editing methodologies for LLMs, and introduce two datasets for the purposes of finding potential drawbacks of LLMs.\n\nThis paper presents the MLE method as a straightforward solution to mitigate knowledge distortion risks and address potential knowledge conflicts.\n\nThe challenges and prospects of implementing knowledge editing for LLMs are discussed.", "weaknesses": "The paper's scope is limited to factual knowledge editing. \n\nHowever, the presence or absence of knowledge conflicts or distortions in other types of knowledge editing remains unexplored.\n\nThe authors should supplement this part of the paper to make it more comprehensive.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "cLPUarr1lX", "reviewer": "Reviewer_ihfd", "summary": "This paper explores the potential pitfalls of knowledge editing for Large Language Models (LLMs). It introduces new benchmark datasets and evaluation metrics to investigate the issues of knowledge conflict and knowledge distortion. The results demonstrate that knowledge editing can lead to unintended consequences and inconsistencies in LLMs. The paper also presents potential solutions and challenges for knowledge editing in LLMs.", "questions": "1. What do you think is the fundamental reason for knowledge distortion during knowledge editing for LLMs? How to handle cases of one-to-many knowledge editing?\n2. How do the knowledge editing methods compare to each other in terms of their effectiveness and efficiency, what is the takeaway in method selection?\n3. In Multi-label Edit, how to guarantee the overall conceptual hierarchy among labels?\n\nMinor Issues:\n1. \"Emperically\" should be \"Empirically.\"\n2. \"ROME is effective in both GPT-XL and GPT-J\" should be \"ROME is effective in both GPT2-XL and GPT-J.\"\n3. \"This motivates us to employ conflict detection technologies\" should be \"This motivates the employment of conflict detection technologies.\"\n4. \"Knowledge Conflict has reflected\" should be \"Knowledge Conflict reflects.\"\n5. \"However, it is undesirable for a robust editing method to weaken the preference\" should be \"However, weakening the preference is undesirable for a robust editing method.\"", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The information provides insights into different knowledge editing methods and their performance in various setups.\n2. The paper discusses the concept of knowledge distortion and its impact on language models.\n3. The paper introduces the idea of conflict detection technologies to address potential knowledge discrepancies.", "weaknesses": "1. The information provided is quite technical and may be difficult for non-experts to understand.\n2. Some sentences are poorly structured and difficult to comprehend.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "1nhtaL1BmX", "reviewer": "Reviewer_XF6v", "summary": "This work pioneers the investigation into the potential pitfalls associated with knowledge editing for LLMs, and introduces new\nbenchmark datasets to evaluate LLMs after knowledge finetuning with proposed innovative evaluation metrics.", "questions": "Thanks for your efforts in investigating two pivotal concerns in LLMs, specifically Knowledge Conflict and Knowledge Distortion, which has been widely discussed in NLP community nowadays. However, there remains some issues that I need to discuss with you.\n\n\n1. Fig. 2 illustrates that after the process of Round-Edit, LLMs tend to assign higher probabilities to the knowledge facts stored in recent corpus and gradually forget the knowledge stored in model parameters. Nonetheless, I believe that the demonstration of Knowledge Distortion in Fig. 2 is not a unique issue limited to LLMs, but rather a prevalent concern across all current Deep Learning models. \nAnd I believe that the solution to this issue is to develop more reasonable retrieval-based LLMs, which answers questions based on the knowledge context, rather than finetune-based Knowledge Editing methods mentioned in this article. With retrieval-based LLMs, you just need to modify the knowledge facts stored in the context, then LLMs can directly answer the user question according to the context. In such consideration, I suppose the contribution of this paper is limited, and it will be better to include evaluation of retrieval-based LLMs as baselines.\n\n2. The novelty of devlelopment of Multi-Label Edit is quite low, why not consider memory-based methods or EMA methods to alleviate the forgetting of previous knowledge stored in LLMs when you are certain that these knowledge fact are all accurate.\n\n3. For Knowledge Conflict, it will also be a problem in retrieval-based LLMs, any thoughts to solve this problem according to your expiermental results?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1. Novel benchmarks and evaluation metrics are developed in the paper\n2. With empirical analysis, the authors develop a simple method, a.k.a Multi-Label Edit, to alleviate Knowledge Distortion in LLMs", "weaknesses": "1. The novelty of the developed method is quite low and the real contribution of this paper is the development of new benchmarks equipped with evaluation metrics.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "AMTc9vaxXb", "reviewer": "Reviewer_YAyi", "summary": "This paper studies the potential pitfalls related to LLMs knowledge editing, including Knowledge Conflict and Knowledge Distortion.\nTo achieve this target, two benchmark datasets and several innovative evaluation metrics are also introduced.\nWith these settings, this paper conducts experiments from two aspects among four common editing approaches on two LLMs, and proposes a Multi-Label Edit (MLE) solution.", "questions": "My questions mainly comes from the above three concerns.\n1. The knowledge conflict mentioned in your paper, i.e., “e1: Marie’s husband is Pierre → Jacques and e2: Jacques’s wife is Marie → Maurice”, seems to be the “Data Collision” rather than “Editing Conflict”.\nIf this type of conflict only exists at the data level, then the evaluation in this paper is meaningless.\nIn addition, I guess you want to emphasize the conflict between different editing operations, as mentioned in Section 2.2 “there is a possibility that interference occurs between different edits, causing the former edit invalid.”.\nCould you further clarify this problem and provide more suitable examples?\n\n2. The “Knowledge Distortion” is a very promising research question that I pay attention to. However, the evaluation in this aspect is:\na) vague:\nHow many the (s,r) pairs did you evaluate to calculate the results in Table 2 ? (Is there five?)\nHow are the values in Table 2 calculated? Are they arithmetic mean values?\nWhy is the JS divergence chosen? Is the asymmetric KL divergence inappropriate? and why?\n\nb) incomplete:\nWhy only evaluate triples under the same (s, r), and other knowledge is not affected? (e.g, with same (s,.,.) and same (., r, .) or others (., ., .))?\n\n3. The proposed MLE method is unclear. Could you explain what the multi-label is and what is its function?\nHow does this method alleviate the problem of “Knowledge Distortion”?\nAdditionally, is this method effective for the first question (\"Knowledge Conflict\") mentioned in your paper?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. This paper puts forward a valuable research problem, i.e., to discuss the potential risk of editing knowledge encoded in LLM. I believe this could help researchers and practitioners better understand and manipulate the knowledge encoded in LLM, so as to obtain a better model.\n2. Extensive baselines are employed for analyzing the proposed research question.\n3. Several new measures are designed to quantify the degree of knowledge conflicts in experiments.", "weaknesses": "My major concerns lie in the following three aspects:\n1. The “Knowledge Conflict” proposed in this paper is confusing to me.\n2. The experimental settings of “Knowledge Distortion” are vague and incomplete.\n3. The proposed MLE solution is unclear.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (poster)