Evaluating Dependencies in Fact Editing for Language Models: Specificity and Implication Awareness
Authors: Zichao Li
Ines Arous
Siva Reddy
Jackie CK Cheung
Venue: EMNLP 2023 Findings (2023)
Keywords: knowledge editing
large language models
knowledge base
TL;DR: We propose an evaluation protocol and an accompanying QA dataset that comprehensively assess the specificity and implication awareness of editing pre-trained language models.

Abstract: The potential of using a large language model (LLM) as a knowledge base (KB) has sparked significant interest. To maintain the knowledge acquired by LLMs, we need to ensure that the editing of learned facts respects internal logical constraints, which are known as dependency of knowledge. Existing work on editing LLMs has partially addressed the issue of dependency, when the editing of a fact should apply to its lexical variations without disrupting irrelevant ones. However, they neglect the dependency between a fact and its logical implications.
We propose an evaluation protocol with an accompanying question-answering dataset, StandUp, that provides a comprehensive assessment of the editing process considering the above notions of dependency. Our protocol involves setting up a controlled environment in which we edit facts and monitor their impact on LLMs, along with their implications based on If-Then rules. Extensive experiments on StandUp show that existing knowledge editing methods are sensitive to the surface form of knowledge, and that they have limited performance in inferring the implications of edited facts.

Evaluating Dependencies in Fact Editing for Language Models:
Specificity and Implication Awareness
Zichao Li, Ines Arous, Siva Reddy, Jackie C.K. Cheung Mila, McGill University
(zichao. li, inessarous, siva. reddy) @mi la. quebec jackie. cheung@mcgill.cc
Abstract
New
(Apple, CEO, Bteve-Jobe Tim)
(Apple, CEO, Tim)
The potential of using large language model (LLM) knowledge base (KB) has sparked significant interest. To manage the knowledge acquired by LLMs, we need ensure that the editing of learned facts respects internal log- constraints, which are known as depen- dency of knowledge. Existing work editing LLMs has partially addressed the issue of de- pendency, when the editing of fact should ap- ply to its lexical variations without disrupting irrelevant ones. However, they neglect the de- pendency between fact and its logical implica- tions. We propose evaluation protocol with an accompanying question-answw answering dataset, DepEdit, that provides comprehensive sessment of the editing process considering the above notions of dependency. Our protocol in- volves setting up controlled environment in which we edit facts and monitor their impact on LLMs, along with their implications based Ih-Then rules. Extensive experiments DepEdit show that existing knowledge edit- ing methods are sensitive to the surface form of knowledge, and that they have limited perfor- mance inferring the implications edited facts.!
(Tin, Employer, Apple)
Editing Module
Knowledge base
(Tesla, Elon)
A, CEO,
(Tis,
Employer,
S4M Applo)
(Person Emaioyer, Company
Figure 1: The editing of symbolic KBs is specific: un- related fact (Tesla, CEO, Flon) is kept unchanged; and implication aware: an new implication (Tim, Employer, Apple) is added to the KB accordingly.
While the potential of LLMs as KBs is recog- nized, their reliability remains contentious. The data used to train LLMs may contain inaccuracies or outdated information, compromising the trust- worthiness of using LLMs as KBs. Consequently, is imperative to enable practitioners to edit LLMs, ensuring that the knowledge² in LLMs can be rec- tified and reflects current real-world knowledge.
When editing knowledge in LLMs, it is crucial to ensure that the modifications respect internal logical constraints commonly referred to as depen- dency of knowledge in the database and knowledge base communities (Fagin et al., 1983; Fan, 2008; Pawlak; Ullman, 1988). Dependency of knowledge refers the logical constraints between different pieces of information that have be satisfied in order to ensure the knowledge base's 's integrity. It mainly includes specificity and implication aware- ness constraints. Specificity refers to editing fact and its lexical variations, while other irrelevant ones are left unaltered. Implication awareness en- sures that the editing process effectively derives implications using If-Then rules from edited fact. For instance, in Figure 1, editing fact (Apple, CEO. Steve Jobs) by replacing 'Steve Jobs' with
1 Introduction
Recent advancements in Large Language Mod- els (LLMs) have sparked interest in using them as knowledge bases (KBs) due to their high per- formance in recalling factual knowledge (Petroni et al., 2019a; Cohen et al., 2023b). This grow- ing interest is mainly motivated by the advantages that LLMs offer over traditional knowledge bases. First, LLMs require minimal human supervision as opposed KBs that must be manually populated. Second, LLMS process queries expressed in natural language, unlike KBs, which require users input queries with a specific syntax.
2 "Knowledge of LLMs in this paper refers to the model's beliefs that are acquired from the training data without justi- fication in the real world. In the realm of cpistemology, this type of knowledge is referred to as weak knowledge (Goldman and Olsson, 2009).
CCode and data: https:/ www.hhtwh..o.. thub. com/ MpGi 11teil11-LFF/// /Logi ca KnowEdit

---

'Tim Cook' should keep the unrelated fact (Tesla, CEO, Elon) unchanged, while the implication (Tim Cook, Employer, IBM) should be updated to (Tim Cook, Employer, Apple).
in LLMs.
2 Related Work
LLMs as Knowledge Bases Recent studies have highlighted the potential of large language models (LLMs) as knowledge bases. For example, Petroni et al. (2019b) and Davison et al. (2019) have shown that one can recall factual knowledge from LLMs through prompt- based methods without depend- ing on external knowledge sources. However, sub- sequent research (Cao et al., 2021; Wang et al., 202 la) has raised concerns about the reliability of LLMs as knowledge bases, particularly regarding their sensitivity to prompts.
While fundamental questions related to depen- deney have been settled in the knowledge base com- munity (Fagin et al., 1983; Fan, 2008; Pawlak; Ull- man, 1988; Arenas et al., 1999; Cali et al., 2003; Bohannon et al., 2006), they are still in their early stage in LLMss-s--BBs research. Since knowledge acquired by LLMs is encoded within a set of inter- connected neurons, editing one fact is likely to im- pact others. Moreover, LLMs do not explicitly rep- resent the dependencies between different pieces of knowledge, which makes the consequences of any edit unknown in advance. Recent work (Meng al., 2022b; Mitchell al., 2021; De Cao al., 2021) tackled the challenge of editing knowledge in LLMs but only addressed the specificity constraint. However, they neglect the implication awareness of editing, which is a core part of the dependency constraints in editing knowledge.
This work provides different testbed to assess the reliability of LLMs as knowledge bases, fo- cusing on the influence of editing on knowledge dependency.
Knowledge Editing in LLMs Various methods have recently been proposed to edit the knowledge acquired in LLMs. One line of work (De Cao et al., 2021; Mitchell et al., 2021) develops external learned editors that modify the tuning gradients for editing, while keeping the base model unchanged. Another line of work (Meng et al., 2022a; Dai et al.. 2022; Meng et al., 2022b) attributes knowledge particular neurons or modules in the network and manually edits their activation to edit a target fact. While these methods achieve high performance in editing knowledge in LLMs, they only evaluate under the specificity constraint, i.e. editing a fact without affecting the rest of the knowledge. Our work adopts a broader perspective on knowledge editing and focuses its capability to incorporate dependencies that encapsulates both specificity and implication awareness. In similar vein, Hase et al. (2023), Cohen al. (2023a) and concurrent work (Onoe et al., 2023) also go beyond the scope of specificity and evaluate the editing of entailed facts. However, they mainly focus on single-hop reasoning and do not have If-Then rules to track the editing of implications.
In this paper, we address both the specificity and implication awareness of knowledge editing in LLMs. Concretely, we propose a new evalua- tion protocol called caaublish-anduuddaee and an accompanying English question-answert dataset (DepEdit) for a comprchensive evaluation of de- pendency in editing LLMs. The evaluation is done by creating an extensive collection of knowledge editing simulations. Each simulation focuses subset of the dataset, where we are able to control the entities, relations and applicable If- rules described within this subset. Consequently, we compare the expected and actual edits applicd by a knowledge editing model and evaluate its speci- ficity and implication awareness.
Using our dataset DepEdit t and evaluation pro- tocol, we compare state-of-the-art knowledge edit- ing methods in LLMs. The results illustrate the challenges associated with implication-awarr edit- ing. We find that existing methods tend to be in- fluenced by superficial patterns such as the surface form of questions. Furthermore, our analysis of models' gradient updates shows that they strug- gle to identify the uppropriate gradient direction for updating the parameters of the LLM in rela- tion to implications. Finally, we discuss future work and show that there is significant potential in drawing inspiration from methods developed for dependency in KBs to effectively edit knowledge
Dependencies in Knowledge Bases Dependency theory finds its roots in the domain of data de- pendency within relational databases (Fagin et al., 1983; Rauszer, 1984). It was later adapted knowl- edge bases, as the semantic aspects of dependen- cies between facts and rules within a KB (Wu al., 1994; Buszkowski and Orlowska, 1998; Pawlak). Several methods (Slota et al., 2011; 1; Slota and Leite,

---

Establish phase
Update phase
Knowledge set Clt Which Fromklin fromi, (2), Whech country NVC in? C3 Which country London
city Franidin origrame
A London
Q: Which dit was Franklin from?
Queries
NVC located which country?
itry?
Rutec (ihhooondl bun (rays] and ( Accoaaie Koniioyal then /Possed fuom Rounntet
country was Framilin
M
M
M
LM
M
M
Establish
Record
Evaluation
Edit subset of facts
Evaluation
Recorded predictions
Figure 2: The complete pipeline of establish-and-uk uupate evaluation protocol. During the establish phase, prompt model M to extract its knowledge about the facts and implications within a knowledge set. During the update phase, some facts are edited, and we track the change of the facts and accordingly implications in the updated model M.-
2012: Zhang and Foo, 1995) were developed for knowledge editing in KBs based on dependencies. These methods were based on key principles (Dalal, 1988), including three fundamental ones: "Irrele- vance of Syntax," which ensures the meaning of edited knowledge remains independent of the syn- tax of the original and the updated knowledge; "Per- sistence of Prior Knowledge," " which aims pre- serving existing information during editing; and "Primacy of New Information." which prioritizes new information over existing one in the KB
The Establish-and-Updatt Protocol for Evaluating Knowledge Editing
3.1 Notations
Throughout this paper, we denote the set of facts with JF and the set of Then rules with R. Each fact F is defined as a triplet (ef.1, Rely.epa e,.2). where ff. and f are entities and Rel is their re- lation. Each fact f is mapped to a question-answwe answerr pair (gf, ay) = T(f) using mapping table in our dataset.
Example 1. The fact f expressed in its triplet form (ef,1, Rely, ef2) as (Franklin, City, transformed to a question-answer pair af); (Which city was Franklin from?, NYC).
Inspired by this body of work and drawing paral- lels between knowledge editing in KBs and LLMs, our research aims to evaluate knowledge editing methods in LLMs on the principles mentioned above. Specifically, we map the principles of "Irel-- evance of Syntax" and "Persistence of Prior Knowl- edge" to specificity constraint, which evaluates an editing method's ability to edit facts and their lexical variations without disrupting irrelevant ones. We map the "Primacy of New Information" princi- ple to an implication awareness constraint, where we evaluate method's ability to infer logical im- plications. Note that existing work on knowledge editing in LLMs mainly evaluated based the specificity constraint (De Cao al., 2021; Mitchell al., 2021; Dai al., 2022), while omitting the implication awareness constraint. To the best of our knowledge, we are the first to establish such analogy and evaluate LLMs using these principles.
Each rule r E R is an If-Then rule composed of logical AND between two premises Pr. and Pr.z- leading implication I,. In other terms, rulc is expressed follows:
( (Pr.1 AND Pr.2) then I,-
Both premise and implication P, and Ir are ex- pressed in template form ??x Rel ?y', where ?r and ?y denote entity types, and Rel their relation. They can also be instantiated with specific facts. For example, [[PersonA] is from [City.A]' can be instantiated to [[Franklnn] is from [NYC]' with K', which con-
We denote knowledge tains subsets of facts Fx and rules Rg, such that K. = Fx J Rg. The facts and rules in are ex- pressed in natural language form. The facts F

---

in knowledge set IC' include two types of facts: specific and unrelated facts, respectively denoted as Sy and Ug- The specific facts Sy share the entity types and relations of the premise propositions in the rules Rx- Unrelated facts are sampled from the facts F such that U3 E F. Sg. From Sy and rules Rg. we derive implications Zue
3.3 Establish Phase
The goal of the establish phase is to extract the facts and rules a model learns given a knowledge set K. The model is free to use arbitrary methods (including doing nothing) to establish the given knowledge. We then prompt the established model M to evaluate its performance in terms of recalling the facts and implications from K. We record the model's predictions and corresponding questions as its established knowledge and denote it with Qx = {(q, ((4)) The established knowledge 2x includes what the model recalls as specific facts
Example 2. A knowledge set K contains If-Then rules and QA pairs mapped from specific facts, im- plications and unrelated facts:
Specific facts T(Sk): (Which city was
Franklin from?, NYC); (Which country is NYC in?, USA); (What the country of London?, UK)
Os, unrelated facts Qu, and implications Q1.
3.4 Update Phase
Rules Ry= If [Person A] is from [City A] and [City A] is located in [Country A}, then
The update phase evaluates the ability of a LLM to update its knowledge given new information. We proceed by first creating copies, denoted Mc. of the previously established model. Then, for each copy Mc. we edit some specific facts and their implica- tions, while other specific facts are kept unchanged. We denote the expected edits on specific facts as $ and on implications I. The model is then updated with Š to M.. For each updated model, we use the established knowledge Q2 from the establish phase assess whether the non-updated facts and implications remain unchanged.
[Person A] is from [Country A].
Implications T T[Ix): (Which country was Franklin from?, USA)
Unrelated facts T(44):: (Who was Demi Moore's 's child?, Rumer Willis);
Note that the entities (e.g., Franklin) in Sy are in- stantiations of the entity type in R2, (e.g., Person A).
We denote the changes expected after editing knowledge in LLM with tilde symbol (e.g.. denote edited facts).
Let's 's take the case where a model learns from the knowledge set in Example 2. Given new fact in the update phase: (Which city did evaluate knowledge editing model ability edit lexical variations such as (Franklin was from which city?, London) and derive implications such as (Which country was Franklin from?, UK)
Example 3.
3.2 Overview
Franklin originate from?, London),
We propose to evaluate the specificity and impli- cation awareness of knowledge editing methods in LLMs by simulating the editing process of a LLM within controlled environment. In each simula- tion, we use knowledge set K. with facts F& and rules Rg. The simulation comprises two phases: an establish and an update phase. In the establish phase, we provide a knowledge editing model with the knowledge set K. and prompt it to extract its established knowledge, which refers to the facts and rules it has learned. In the update phase, we edit subset of the facts. Then, we compare the ed- its made by the model with the expected edits. An overview of the two phases is presented in Figure 2. Following De Cao et al. (2021) and Mitchell et al. (2021), we choose question answering (QA) as the base task. Nevertheless, our evaluation protocol is not specific to QA and could be applied to other tasks. Next, we describe each phase followed by our evaluation metrics.
3.5 Evaluation Metrics
We use the exact-match score (EMS) as a metric to measure the rate at which a knowledge editing method prompted with question q returns the expected answer The EMS is given by:
EMS(M,
Σ (g.0)DD
(M( = a], (1)
D
where D denotes the subset of question-answer pairs (g. a), and [.] is an indicator function return- ing if the model M prompted with q returns the answer and 0 otherwise.
Establish phase For the establish phase, we eval- uate the established model M using two metrics:

---

Premise
Implicatio
Plausibility
Hyanelr
airiine hub
|Arpoott part
loeation
airline hub
country
country
AirineHLb Al located in (Country A
location
abritno hub
country
V
Airport
Ireland
location
location
(Arport ls located (Country
Iocatton
Figure 3: The Step I (left), (middle), 5 (right) of the collection pipeline for DepEdit. In this example, the second candidate rule annotated by workers is: Tf [A1 rport A] 1 located in [Country A], and rporp part of Mirlinetub A], then [AirlineHub A] is located in [Country A).
the establish success rate on specific facts (Est.S "), which are crucial to evaluate the implication aware- and their implications (Est.I). We compute:
ness model. In order facilitate models' eval- uation based on both specificity and implication awareness, we introduce DepEdit, a new dataset for evaluating edits in language models using the zsRE dataset (Levy et al., 2017). In our dataset, we collect facts expressed both question-answe answer pair and in a triplet form i.e., (ef,1, Rel e72)) con- sisting of an entity pair (ef,1, Cy.2) and their rela- tion Rely drawn from WikiData (Vrandežić, 2012). We also enumerate a large set of potentially appli- cable rules, then hire crowd-sourcing workers to annotate their plausibility. The most plausible rules are included in DepEdit. In the following, we explain each step of the dataset curation process and then provide key statisties about our dataset.
Est.S EMS(M T(S))
Est.I EMSMM,
Update phase At the end of the update phase. we evaluate a model's M. ability to edit knowl- edge based on dependency constraints, which are specificity and implication awareness. To evaluate a model's s specificity, we use two metries: Upd.S and Cons.NS. The first metric, Upd.S, aims at eval- uating model's s ability to edit fact and its lexical variations. While, Cons.NS and Cons. evaluate the disruption of non-updated specific facts and unrelated facts, respectively. They are expressed as follows:
Sp EMS(M,, T($))
4.1 Dataset Curation
Cons.NS: = EMS(Mc, 2s)
Step The zsRE dataset contains facts expressed in their triplet form which contains a relation be- tween two entities. We iterate over each pair of relations in the zsRE dataset and search for a new relation via WikiData database We aim to find new relation in WikiData that forms clique with the pair of relations in the zsRE dataset based on shared entities.
Cons.U. EMS(M,, Qu),
We also evaluate the implication awareness of a model using two metrics: success rate of up- dated implications (Upd.I) and consistency of non- updated implications (Cons.NI). We define them follows:
Upd.I. = EM, T(i))
Step 2
We collect, using crowdsourcing, ques- tions about the newly collected relations from Wiki- Data. (See Appendix A for more details).
Cons.NI, EMS(M, 21)
where Ô, denotes the established implications without the updated ones. Finally, we aggregate and average the values of each updated model: Metric = Metric..
Step For QA pairs sharing the same relation, we hire workers to provide a template form of the QA, such that the relation is described in natural language and the entities are replaced by their types. For instance, (Q: In which country is Calgary lo- cated?, A: Canada) is transformed to [City A] is lccated [Country AJ.
4 New Knowledge Editing Dataset: DepEdit
Step Using the collected templates, we gener- ate rules by grouping relations in the same clique mined at Step 1. Then, we insert their correspond-
Existing datasets (Mitchell et al., 2021; Meng et al.. 2022b) for knowledge editing lack If-Then rules,
We are focusing on the evaluation of a specific knowledge set. Therefore, we have omitted the knowledge set index for clarity.
"We use the official WikiData query engine following their regulations.

---

Questions Qn
Rephrased Questions On
Answers (a..) New York City USA
Spocific facts
Which city was Franklin D. Roosevelt from? Which city did Franklin D. Roosevelt originate from?
Which country is New York City
New York City located in which country?
Unrelated Facts
Who was Demi Moore's child?
Rumer Willis
TfThen rule Implications
If [PPersoon A] from [Ciy A], and [City A] is located in [Country A], then [Person AJ is from [Country A]. Which country was Franklin D. Roosevell from?
USA
Table I: A knowledge set from DepEdit. We show two specific facts and one unrelated facts. Entities(e.g. USA) and their corresponding entity type(e.g. [Country AJ in rules are in bold and share the same color. Refer to Table 8 for more examples.
ing templates into an If Then rule, where two rela- tions act as premises and one relation serves as an implication. As a result, we generate three rules for each clique, resulting in total of approximately 7k generated rules.
4.3 Key Staties on DepEdit
DepEdit contains 159.264 question-ansvee answer pairs, each associated with its corresponding triplet form. Additionally, the dataset includes total of 530 If- Ih-Then rules. Using the QA pairs and the If- rules, we create knowledge sets as in Example 2. Consequently, we obtaín 1106 knowledge sets. We show an example of these sets in Table 1.
We do not require rules to be logical entailments, as we assume that a controllable knowledge base should be able to adhere to rules provided by user. However, would like the rules in our dataset to correspond plausible rules that could be given in deployment setting. We thus hire annotators to filter out implausible rules, such as those that contradict commonsense knowledge (c.g.. "[Person A] is from [Country A). and /City A] is the capital of [Country AJ, then [Person A| was born in [City Al").
5 Experiments
5.1 Task Setup
We split DepEdit into 1106 groups of simula- tions, each corresponding to one knowledge set. Each knowledge set contains 29 facts (24 specific facts and 5 unrelated facts), If-Then rule, and 12 implications. During the update phase, we create three new versions of facts, where we randomly sample 20 facts from knowledge set and edit their answers. Note that it is possible to include multiple rules per simulation. However, we find that the current setting with one rule already challenging for existing methods (see Section 5.3.2).
Step 5 For each candidate rule, two annota- tors rate its plausibility by determining given the premises, the consequence: (a) Must be true; (b) Likely to be true; (c) Unlikely to be true; (d) Must be false; (e) The given conditions are not useful for the derivation of the consequence. We adopt the rules on which both of workers agree that their plausibility are (a) or (b).
Settings CQ
Questions for editing
Questions for evaluation
The key steps 1, 3 and 5 are illustrated in Fig- ure 3.
Which city was Franklin from?
Which city was Franklin from?
K. Rowling from which city?
J.K. Rowling was from which city?
4.2 Crowdsourcing Tasks
CQ_UT
Which city was from? which city was J. K. Rowling from?
Franklin
Which city was from? Which city was K. Rowling from?
Franklin
The crowdsoureing tasks involved in steps 2, 3 and 5 were published on Amazon Mechanical Turk where workers generated questions, template forms of QA pairs and filtered implausible rules. We hired workers from English-speaking countries with > 95% approval rate that had passed qual- ification test. Each task took between 20-40 sec to complete average. Workers who completed the task received a reward corresponding to an hourly wage around 12-20 USD. Details on the setup and quality control of each annotation task are included in Appendix A.
ICQ DYT
Which city was Franklin fram?
Which city did Franklin origi- nate from? Which city was J. K. Rowling from?
K. Rowling was from which city?
Table 2: Examples of specific facts and prompted ques- tions in different settings.
We evaluate knowledge editing methods based on dependency constraints, including specificity and implication awareness. We use three settings to evaluate the specificity of editing methods that we name: DQ_ CQ_ UT and ICQ We use the term CQ(consistent questioning) to refer
"htps//wwww.muur..co//

---

setting where use the same randomly sam- pled question templates for specific facts during editing (i.e., establish and update) as during evalua- tion, while ICQ (inconsistent questioning) is used to describe setting where they are different. We use the term UT (uniform template) to describe a setting where we use the same question templates for each relation in specific facts within knowl- edge set, and the templates are selected randomly. On the other hand, DT (diverse templates) is used when different question templates are used. Table illustrates each setting. These settings are designed to test for the principle of "Irrelevacee of Syntax". In particular, ICQ evaluates if the editing is con- sistent across different linguistic presentations of a specific fact. Meanwhile, UT tests whether the editing method can distinguish different pieces of knowledge that share similar surface forms.
based on specificity and implication awareness.
5.3.1 Specificity of editing
The results for establish and update phases are pre- sented in Table 3. First, we observe that MEMIT has higher success rate than MEND in both the establish and update phases. Second, we observe significant gap (more than 10%) in Est.S and Upd.S between MEMIT and FINETUNE UNE. Third, we observe that MEND and MEMIT have better perfor- mance than FINETUNE terms of consistency on unrelated facts (ects(Cons.U) U). This finding is consistent with the literature (Mitchell et al.. 2021; De Cao et al., 2021; Meng et al., 2022b). However, MEMIT performs worse than FINETUNE in terms of con- sistency of non-updated specific facts (Cons.NS).
To evaluate a method's ' implication awareness, we evaluate a method's ability to answer questions about implications.
When comparing the different scttings, we find that the performance of all methods, except BART and GPT-XL, decreases in the ICQ_DT compared with CQ_ DT. This result indicates the difficulty that existing methods face in handling lexical vari- ations during the editing process. When comparing CQ DT and CQ_ mance of MEND decreases (e.g. 4.5% in terms of Upd.S), while the performance other methods stays almost constant. This result indicates that us- ing questions with similar surface form deteriorates the performance of MEND.
5.2 Comparison Methods
We compare with two pre-trained language models, BART (Lewis et al., 2020) as used by De Cao et al. (2021); Mitchell et al. (2021) and GPT-XL (Rad- ford et al., 2019) as used by Meng et al. (2022b), which serve baselines for knowledge editing methods. To mitigate the domain disparity, tune GPT-XL on a closed- book QA dataset (Wang et al., 2021b). We also compare with the following knowledge editing methods:
observe that the perfor-
fine-
Setting
Metbods
Eat
Upddbr)s))
(SUS))
Consstils)
FINETUNE: continuously fine-tune the base model whenever new knowledge is given. MEND (Mitchell et al., 2021): learns an addi- tional hypernetwork that updates the parame- ters of language model, while satisfying two constraints: the edits apply lexical varia- tions, and answers to irrelevant questions re- main unaltered.
RANDOM
4.70
3,06
4.22
5.21
Basc mode): BART
BART
7.49
0.37
100.
100.
OQ.0T DT
MINETUNEE
9085
36.61
15.00
100.
0Q_uT
PINETUUEE
88.02
$7.69
32.57
15:17
MENO
31.67
44.32
07.51
$3.60
BART
7.59
0.46
100.
100.
41.14
54.71
Base model. GPTXLL
MEMIT (Meng et al., 2022b): applies causal mediation tracing on GPT model to identify and alter the most significant layers for fact. RANDOM: is a trivial baseline that randomly sample from set of answer candidates seen during establish and update phase.
GPT-XL
0.06
0.00
100.0
100.0
OQ DT
FINETTNE
98.66
99.78
77.36
40136
64.15
55.16
94.15
GPT-XL
0.00
100.
100.0
CQ.ITT
FINETTON
97.33
97.08
66.11
40.00
MEND
28.74
43.26
95.95
63.21
93.53
100.0
100.0
ICQ DT
PINETUNE
85.91
95.72
59.66
00.38
Additional implementation details can be found in Appendix
MEND
20.70
12:14
87.05
65.15
MEXIT
14.36
56.38
$6.84
94.15
Table Performance of different knowledge editing methods in terms of specificity during establish (Est.S) and update chase(Upd.S) Cons.NS, Cons.U).
5.3 Results
We use our establish-ande and-update protocol to eval- uate existing methods' ability to edit knowledge,

---

5.3.2 Implication awareness of editing
Gradient pains
Cosine similarity
We employ a variant of FINETUNE called FINETUNFRELE, where we fine-tune a LM model by generating an implication conditioned on its two premises. For MEND and MEMIT, we edit the premises and test whether they update the implica- tions according the edited premises.
Premises & Implication
0.52
Fact pairs with similar surface form
0.51
Original Rephrased facts
0.77
Random fact pairs
0.35
Table 5: Gradient's similarity when updating different pairs of facts.
Table presents the results. We observe that MEND and MEMIT achieve low performance in terms of Est.I and Upd.I. Meanwhile, they achieve high Cons.NI at the same level of their Cons. in Table 3, which suggests that these editing meth- ods regard the implications as unrelated knowledge. Therefore, both MEND and MEMIT do not estab- lish logical dependencies when editing knowledge. The establish and update success rate the same for both FINETUNE and FINETUNERULE which means that incorporating the If-Then rules with FINETUNERCLE does not improve the performance of the FINETUNE model. This finding suggests that LMs do not internalize Then rules expressed in natural language.
sub-goal before moving to the first one, since it is easier to directly estimate the difficulty of the second one.
6.1 Gradient Similarity between Updating Premise Facts and Implications
We assume that the similarity between the gradi- ents of premise facts and implications, when a LM learns QA pairs related to them, serves an indicator of the feasibility of implication-awan editing. Intuitively, the higher the similarity be- tween these gradients, the easier it is perform implication aware editing. Therefore, we mea- sure the similarity between gradients for updating premise facts V log Po(arlay) and implications V log Po(arlar). For the sake of comparison, we measure the gradient similarity between updating facts with similar surface forms, pairs of facts with lexical variations, and randomly sampled ones. As in Table 5, the gradient similarity of premise- implications pair is on par with pairs of facts with similar surface form. This suggests that updating a fact and its logical implication as challenging as updating facts with similar surface form.
Establish phase Est.I(%)
Update phase (S
Methods
Upd.((S))
RANDOM
8.41
8.65
8.62
Base model: BART
BART
11.59
7.15
100.0
FINETUNE
17,80
6.53
16.69
FINETUNERULL
17.84
6.57
18.40
MEND
14.92
10.99
47.08
Base model: GPT-XL
GPT-XL
0.10
0.00
100.0
FINETUNE
19.99
9.70
38.44
FINETUNNES LINERLLE
19.99
9.70
38.44
6.2
Gradient Direction Update Implications
MEMIT
0.54
0.68
94.15
MEND
12.91
11.73
62.04
The above analysis suggests that the main challenge lies in the first sub-goal: determining the direction to update the parameters following the applicable rules. To further validate this hypothesis, we train MEND to capture the logical dependencies among premise facts and implications. We proceed by modifying its learning objective so that applies the edit for implications in a similar manner as it applies to the lexical variation of facts. The details of the modification can be found in Appendix C.
Table Performance of different rule learning methods and knowledge editing methods in terms of implication awareness during establish and update phase.
6 Analysis
Next, we conduct analysis to investigate the barri- ers to implication-aware knowledge editing. For- mally, we break down this task into two sub-goals: (1) finding the gradient direction (aPo(arlop (arlgr)) for the LLM's parameters plication conditioned the update incurred by new facts Po (a fl))) (2) finding the optimal gradi- ent direction to simultancously update the facts and implications. We start by discussing the second
We refer this vari- ant MEND To further verify that the difficulty mainly comes from identify- ing the direction to update the gradients
Rst.((5)
Rd.((%)
reflect the editing of im-
MEND
MEND
14.01
9.64
BARTORSIE
81.59
$1.35
Table 6: Performance of MEND variants update and edit implications.

---

conditioned on the update of premises,
7 Conclusion
In this paper, propose novel evaluation protocol designed to assess knowledge depen- deney, including both the specificity and impli- cation awareness in knowledge editing methods applied to LLMs. To implement our evaluation protocol, we collect a new question-answering dataset DepEdit with If-then rules in natural lan- guage. Our experiments reveal limitations in cur- rent knowledge editing techniques, both in terms of specificity and their ability understand implica- tions. In general, saae--o-thh-an art knowledge editing methods exhibit lower success rate compared vanilla fine-tuning and are highly influenced by the precise wording of facts. Additionally, current methods demonstrate limited performance in deriv- ing the logical consequences of facts using natural language rules. Future work could draw inspiration from existing methods in KB editing and leverage our dataset and evaluation protocol develop im- plication aware knowledge editing methods.
we setup another approach: DERIVE where the premise facts are given in the prompt. Con- cretely, we prepend both the premises and ran- domly sampled unrelated facts before the impli- cations questions qI for the model. As seen in Ta- ble 6, MENDDIM achieves low performance despite explicitly encoding the logical dependencies in its training. In contrast, BERTDERIVE achieves much higher performance, revealing that inferring an im- plication given its premises is not challenging when trained to predict implications given premises.
6.3 Dependency Resolution with ForwChain and BackChain
To accurately measure the disparity between cur- rent methods and flawless dependency resolution, we introduce two additional toplines: FORWCHAIN and BACKCHAIN inspired by forward-chaining and backward-chaining algorithms used in tradi- tional KB systems (AI-Ajlan, 2015). Both meth- ods assume that we can infer questions about im- plications from questions about their premises. BACKCHAIN breaks down the question into in- quiries about the premises based on applicable rules, while FORWCHAIN updates the KB by query- ing the language models for other premises when- ever fact is provided. We apply these meth- ods with FINETUNE and MEMIT for GPT-XL.
8 Limitations
Our evaluation protocol is inspired by the principles of knowledge editing in knowledge bases (Dalal, 1988). These principles are more comprehensive than the dependency constraints discussed the paper. For instance, the principle of "primacy of new information" " applies to all potential logical operations, such as quantification and probabilistic entailment. Currently, DepEdit does not sup- port the evaluation of knowledge editing in LLMs with such complex logical operations. Furthermore, each knowledge set in the current setting of our ex- periments contains only one rule. We carefully al- locate the edit targets so that there are no conflicts among the given facts and derived implications, which may occur in real-world applications. That said, we find that the existing knowledge editing methods fail even in this simple setting, Future work extend our DepEdit t dataset by aggre- gating multiple rules in one knowledge set or as- signing probability values to rules and facts.
The results in Table show that the external in- ference mechanism improves the accuracy of edit- ing implications. The accuracy gain with MEMIT is smaller than FINETUNE. However, FORWCHAIN and BACK CHAIN introduce new risks that can po- tentially decrease the consistency of updatdd implications. With BACKCHAIN, the model needs to give consistent answers to both of the premises questions, which can compound any existing in- consistencies. FORWCHAIN increases the number of premise facts to be updated, making the editing easier to break irrelevant knowledge.
Establish phuse
Update plase
Methods
Inference
Est
Upd.I(%)
Co
FINETUNE FOrwCHAIn
86.28/85
39. 19.15(0.71)
Ethics Statement
FINETUNE BACKCHAIN
98.21(78.22)
76.79/67.09) .0)
36.85(0.41)
MEMIT
FORwCHAIN
22.06/21388
81.11[ 13.04)
A large portion of question-answer pairs in DepEdit are transformed from WikiData triplets, which could be inaccurate. Furthermore, the If- Then rules we collected are labeled by small group of annotators. These points mean that there
MEMI
BACKCHAIN
14.19(12.65)
30.50/20.91)
(22.7(-31.1))
Table 7: Performance (and performance gain) in terms
implication awareness with our protocol.

---

is no guarantee that the facts and the rules are fac- tual, and they may reflect stereotypes or biases. Therefore, DepEdit can only be used for testing knowledge editing and other reasoning abilities of computational models, not for fact-checking, deci- sion making, or other similar purposes.
Roi Cohen, Mor Geva, Jonathan Berant, and Amir Globerson. 2023b. Crawling the internal knowledge- base of language models. In Findings of the Asso- ciation for Computational Linguistics: EACL 2023, pages 1811-1824.
Damai Dai, Li Dong, Yaru Hao, Zhifang Sui, Baobao Chang, and Furu Wei. 2022. Knowledge neurons in pretrained transformers. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume I: Long Papers), pages 8493- 8502.
10 Acknowledgement
This work was supported by Samsung Electron- ics, the NSERC Discovery Grant program, and the Canada CIFAR AT Chair program. We acknowl- edge the material support of Nvidia and Digital Research Alliance of Canada in the form of com- putational resources.
Mukesh Dalal. 1988. Investigations into a theory of knowledge base revision: preliminary report. In Pro- ceedings of the Seventh National Conference on Ar- rificial Intelligence, volume 2, pages seer.
Cite-
Joe Davison, Joshua Feldman, and Alexander Rush. 2019. Commonsense knowledge mining from pre- trained models. In Proceedings the 2019 Confer- ence on Empirical Methods in Natural Language Pro- cessing and the 9th International Joint Conference Natural Language Processing (EMNLP-IJCNLP), pages 1 173-1177, Hong Kong, China, Association for Computational Linguistics.
References
Ajlan Al-Ajlan. 2015. The comparison between forward and backward chaining. International Journal of Machine Learning and Computing, 502):106.
Marcelo Arenas, Leopoldo Bertossi, and Jan Chomicki. 1999. Consistent query answers in inconsistent databases. In Proceedings of the eighteenth ACM NIGMOD-SIGACT-SIGAA symposium on Principles of database systems, pages 68 78.
Nicola De Cao. Wilker Aziz, and Ivan Titov, 2021. Edit- ing factual knowledge language models. In Pro- ceedings of the 202] Conference on Empirical Meth- ods in Natural Language Processing, pages 6491 6506.
Philip Bohannon, Wenfei Fan, Floris Geerts, Xibei Jia, and Anastasios Kementsietsidis. 2006. Conditional functional dependencies for data cleaning. In 2007 IEEF 23rd international conference on data engineer- ing, pages 746-755. IEFE.
Ronald Fagin, Jeffrey D Ullman, and Moshe Y Vardi. 1983. On the semanties of updates in databases. In Proceedings of the 2nd ACM SIGACT-SIGMOD Sym- posium on Principles of Database Systems, pages 352-365.
Wojciech Buszkowski and Ewa Orlowska. 1998. Indiscermibillty-bass formalization of dependencies in information systems. Incomplete Information: Rough Ser Analysis, pages 293-315.
Wenfei Fan. 2008 Dependencies revisited for im- proving data quality. In Proceedings of the twenty- seventh ACM SAMMODD-IIGACTSIIGEI Principles of database systems, pages 1.59-170.
symposium
Andrea Calı, Domenico Lembo, and Riccardo Rosati. 2003. Query rewriting and answering under con- straints in data integration systems. In Proc. of the 18th Int. Joint Conf. on Artificial Intelligence (IJCAI 2003), pages 16-21.
Alvin Goldman and Erik Olsson. 2009. Reliabilism and the value of knowledge. Epistemic value, pages 19-41.
Peter Hase, Mona Diab, Asli Celikyilmaz, Xian Li, Zor- nitsa Kozareva, Veselin Stoyanov, Mohit Bansal, and Srinivasan Iyer. 2023. Methods for measuring, up- dating, and visualizing factual beliefs language models. In Proceedings of the 17th Conference of the European Chapter of the Association for Compu- tational Linguistics, pages 2706-2723
Boxi Cao, Hongyu Lin, Xianpei Han, Le Sun, Lingy- ong Yan, Meng Liao, Tong Xue, and Jin Xu. 2021. Knowledgeable or educated guess? revisiting lan- guage models as knowledge bases. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 1ith International Joint Conference Natural Language Processing (Volume I: Long Papers), pages 1860-1874.
Diederik P Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In ICLR (Poster).
Omer Levy, Minjoon Seo, Eunsol Choi, and Luke Zettle- moyer. 2017. Zero-shot relation extraction via read- ing comprehension. In Proceedings of the 21 Con- ference on Computational Natural Language Learn- ing (CoNLL 2017), pages 333-342.
Roi Cohen, Eden Biran, Ori Yoran, Amir Globerson, and Mor Geva. 2023a. Evaluating the ripple effects of knowledge editing in language models. arXiy preprint arXiv 12976.

---

Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and I Luke 7ettlemoyer. 2020. Bart: Denoising pruuencc--o-sqquenee pre-training for nat- ural language generation, translation, and comprehen- sion. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7881-7880.
Martin Slota, Joao Leite, and Terrance Swift. 2011. Splitting and updating hybrid knowledge bases. The- ory and Practice of Logic Programming, 114-5))801-- 819.
Timen Tieleman, Geoffrey Hinton, al. 2012. Lecture 6.5-rmsprop: Divide the gradient by a running aver- age of its recent magnitude. COURSERA: Neural networks for machine learning. 4(2):26-31 31.
Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. 2022a. Locating and editing factual asso- ciations in gpt. In Advances in Neural Information Processing Systems.
Jeffrey Ullman. 1988. Database and knowledge-bast systems.
Denny Vrandečić. 2012. Wikidata: A new platform for collaborative data collection. In Proceedings of the 21st international conference on world wide web, pages 1063-1064.
Kevin Meng, Arnab Sen Sharma, Alex Andonian, Yonatan Belinkov, and David Bau. 2022b. Mass editing memory in a transformer. arXiv preprint arXiv:2210.07229.
Cunxiang Wang, Pai Liu, and Yue Zhang. 2021a. Can generative pre-trained language models serve knowledge bases for closed-book qa? In Proceed- ings of the 59th Annual Meeting of the Association for Computational Linguisties and the 1 1ith International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 3241-3251.
Eric Mitchell, Charles Lin, Antoine Bosselut, Chelsea Finn, and Christopher D Manning. 2021. Fast model editing scale. In International Conference on Learning Representations.
Yasumasa Onoe, Michael J. Q. Zhang. Shankar Padman- abhan, Greg Durrett, and Eunsol Choi. 2023. Can Ims learn new entities from descriptions? challenges in propagating injected knowledge.
Cunxiang Wang, Pai Liu, and Yue Zhang. 2021b. Can generative pre-trained language models serve knowledge bases for closed-book qa? In Proceed- ings of the 59th Annual Meeting of the Association for Computational Linguisties and the I 1ith International Joint Conference on Natural Language Processing ( (Volume 1: Long Papers), pages 3241-3251.
ZdzisPawlak, Dependencies in Knowledge Base, page 45-50. Springer Netherlands, Dordrecht.
Fabio Petroni, Tim Rocktäschel, Sebastian Riedel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, and Alexander Miller. 2019a. Language models knowl- edge bases? In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Pro- cessing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)) pages 2463-2473.
Chih-Hung Wu, Shie-Jue Lee, and Hung-Sen Chou. 1994. Dependency analysis for knowledge valida- tion in rale-hased expert systems. Proceedings of the Tenth Conference on Artificial Intelligence for Applications, pages 327-333 IEEE
Yan Zhang and NY Foo. 1995. On propositional knowl- edge base updates. Citeseer.
Fabio Petroni, Tim Rocktäschel, Sebastian Riedel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, and Alexander Miller. 2019b. Language models knowl- edge bases? In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Pro- cessing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 2463-2473.
Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models unsupervised multitask learners.
Cecylia M Rauszer. 1984. An equivalence between indisceribiliity relations in information systems and a fragment of intuitionistic logic. Symposium on Computation Theory, pages 298-317. Springer.
Martin Slota and Joao Leite. 2012. A unifying perspec- tive on knowledge updates. In Logics in Artificial Intelligence: 13th European Conference, JELIA 2012, Toulouse, France, September 26-28, 2012. Proceed- ings, pages 372-384. Springer.

---

A Dataset collection details
is an uppercase letter (A B) in the end an index distinguish different entities. That means, the categories of two entity groups are the same, the index letter should be A and respectively. Otherwise, both of them should be A.
A.1 Mining Cliques of Relations and Question Collection
Starting from all the pairs of triplets included in the zsRE (Levy et al., 2017) dataset, we extracted those pairs with overlapped entities. For instance. (c1,T1, 62) and (c1, r2, C3) has an overlapped en-
The template should be a description of the relation that the questions ask about, and it should contain the rewritten abstract form without mentioning specific entity.
tities e1. Then, we queried the WikiData engine to search potential relations ra between e2 and 63. After that, continued this process by search- ing new relations conditioned on r1, r3 and r2, "3, respectively. Through this process, we obtained around 2500 relation triangles.
The template should be descriptive. That is, it should end with period instead of a question mark.
For the new relations not presented in zsRE (Levy et al., 2017), we hired crowd-sourcing workers to transform the triplet (e1,71, e2) to question-answer pair such that obtain ques- tions about e1 and r1, and a corresponding answer e2- For instance, (RedOctane, copyright representative, Activision) trans- formed to Q: What the copyright representa- live of RedOctane?, A: Activision. We collected roughly 900 QA pairs for about 150 relations. We manually filtered out the ones of low quality. Com- mon errors include incorrect interpretation of rela- tions and grammar errors. Finally, more than 700 QA pairs are included.
The reward of this task was set to be $0.15 per HIT and each HIT took less than 40 seconds to complete.
We used JavaScript to ensure that the submis- sions strictly follow the above constraints. After collecting the templates, we pluged the templates of the relations on the same triangle into an If-Then formula with two premises and one implication. For cach relations triangle, there were three po- tential If-Then rules. Finally, we had around 7k potential rules after this step.
A.3 Evaluation of plausibility of rules
The reward of this task was set to be $0.15 per Human Intelligence Task (HIT), where each one took less than 20 seconds to complete on average.
Before starting the formal task, workers were asked to take part in qualification test, annotating a small subset that was labeled by us. More than 100 workers took part in the qualification test. Seven top-pe bup-performing workers of them were selected. For each HIT, the annotators were given a potential rule, and some entities example of the abstract forms, such as [CapitalA] and (countrya] and then asked to determine the logical plausibility of deriv- ing the implication conditioned on the premises. In particular, they were asked to follow these instruc- tions:
A.2 Question-Answer pairs summarization and Potential Rules Construction
We asked the workers to summarize sets of QA pairs grounded on the same relations to abstractive and descriptive templates that summarize the re- lations between two classes of entities. The task decomposed into two sub-tasks: (1) rewrite these groups of entities to their abstract form respectively, and (2) describe the relation between these two abstract forms. Take the following groups of QA pairs as an example: Q: Which city is the capital of A: (OttawaWWashintoon D. C./New Delhi/Berlin). the template can be [CityA] is the capital of [CountryA]. The workers were asked to sum marize according to the following instructions:
Determining the plausibility of the reasoning step described by the If-Then rule, instead of the plausibility of the conditions or of the consequence themselves.
The entity examples are only given to help you think about the rule in more concrete terms. What we care about is the general nule, not just whether applics to those specific examples
Format abstract forms: word or a phrase that describes the category of the entities. The first letter of each word is upper case and there
Each rule was labeled by two workers. For col- lecting more reliable labels, we set up two rounds

---

of annotation. At the first round, about 2000 rules with least one positive label (must be true or likely to true) were selected for the second round. At the second round, we have more strict selection criteria: the rules should have positive labels ((a) or (b)) with perfect agreement. Finally, we have included about 530 rules in DepEdit.
C Details of Experiment Setup in Section 6.2
Given specific facts (gr, and their lexical variations (gyaa)) unrelated facts (gu (u), the vanilla MEND is trained to refine the gradient Vo log Po(aslaf) of LLM's parameters 0 Vi log Po(arlgy). and the resulting updated pa- rameters are denoted as 0"
A.4 Other annotation details
All annotation tasks clarified that the data would be used to build a QA system. For the final plausibility evaluation of rules, we adopted additional qualifi- cation tests. Throughout the collection of question collection and template summarization, we blocked workers who had made low-quality submissions.
LMNND log Po- (arldf)
(2)
+ KL((P( qm)) Po-(-lq.)
Given the implications (41, ar) derived from spe- cific facts and applicable rules, The learning objec- tive of MENDIMP is then:
We strictly follow the research ethics protocol of our institution. The first author received human research ethics training and passed the exam. We will release DepEdit t for academic usage.
MENDEAS L - log Pg- (azlar)
(3)
We also divide DepEdit into train/valida- tion/test sets ensuring that each set shares the same rule but contains different entity pairs. Addition- ally. we reduced the number of facts six and the number of implications three, where four of the facts and hence two implications were updated during the update phase. This approach allowed us to focus the evaluation on implication awareness, thereby mitigating the potential impact of the lack of specificity when editing large-scale facts.
A.5 More examples in DepEdit
More examples of knowledge sets from DepEdit are presented in Table 8.
Experiment Details
We used the pre-trained BART-based (Lewis et al., 2020) model from De Cao et al. (2021) as the base model. contains 110M parameters. And we followed the same hyper-parameters setting and optimizer to train MEND as Mitchell et al. (2021): Adam optimizer (Kingma and Ba, 2015) with 1 6 learning rate. During preliminary experiments. found that MEND is not robust to multi-step editing, which has also been reported in Hase et al. (2023). Therefore, in the update phase, we apply and evaluate MEND on base models fine-tuned with FINETUNE in the establish phase.
As for FUNETUNE, we followed De Cao et al. (2021); Mitchell et al. (2021), using the RMSProp optimizer (Tieleman al., 2012) with 1e - 5 leam- ing rate. BART was fine-tuned for 50 steps until the loss is smaller than 1
As for MEMIT, we use the same set of hyper- parameters from (Meng et al., 2022b) and the GPT- XL model as the base LLM (Radford et al., 2019).
The main experiments were conducted on Intel Gold 6148 Skylake CPUs and NVidia V100SXM2 GPUs. The training time of MEND model was estimated 24 GPU hours. Evaluation of each editing method was finished in less than GPU hours.

---

Questions 9n
Rephrased Questions d
Answers
Example
Specitic locts
In which Nicional work Sherlock Holmes character? What piece of liction does Sherlock Holmes in? Sherlock
The Adventure of the Three Students Doyle
by
Which
Holmes?
Crrclttal fusss
Who the cupsrigho lodker IBM PC: nOS?
IRM
[Novel
erotted by
A]
Imptications
Who wrote The Adventure of the Three Stadents?
Arthur Conan Doyle
Specilic facts
which county is Calgury located!
Which country Calgary in?
Canada
which cvntinnnt Calgary loctial?
What continent is Calgary locaee in?
North America
Unrviated facts
Wisl was Renjmin Franklin' occipwtio?
Diplonat
7-7hue rile
If [Lecation Al located Country Al, and [Location Al [Continent A1, then [Country is located [onnlinent A
Impticutions
cominent
locased
Table 8: More knowledge sets from DepEdit.


Reviews:
[{"review_id": "CuClDMiXgk", "reviewer": "Reviewer_A8T6", "summary": "", "questions": "See above.", "limitations": "", "rating": 3, "confidence": 2, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for recognizing the novelty of our evaluation protocol and dataset. We would like to highlight that other reviewers have also recognized the significance of dependency constraints (R2) in knowledge editing, as well as the clarity of our writing and presentation (R3).\n\n\n### Reasons-To-Reject-1 (justification of knowledge editing and dependencies constraint in terms of practical usefulness)\n\n**Importance of knowledge editing**\n\nKnowledge editing is a fundamental operation that one needs to use a LM as a knowledge base. It has already been well attested in prior work(see De+,  Mitchell+ cited in the Related work section, Lines 134-157). We also provided a brief introduction in terms of its importance in the Introduction section (Lines 27-38). \n\n**Importance of dependency constraints in knowledge editing**\n\nOur work focuses on the dependency among pieces of knowledge during editing,  which is essential for maintaining the integrity of a knowledge base (mentioned in Line 57). As illustrated by Fig. 1, once the dependencies are established as a set of rules, users can update a knowledge base whenever a new event (e.g. a change of CEO) happens, without having to manually enumerate all of its logical implications.\n\n### Reasons-To-Reject-2 (contribution not clearly stated) \n\nThanks for recognizing that our evaluation protocol and dataset enable us to verify the dependencies in knowledge editing without costly human evaluation. We will emphasize this point further in the final version of the paper.\n\n### Reasons-To-Reject-3 (catastrophic forgetting & degradation of MTL performance) \n\nWe agree that editing knowledge in LLMs might indeed disrupt other factual knowledge, which is why we have adopted consistency metrics (Cons.S, Cons.I, Cons.U; detailed in Sec 3.5) to address the issue of catastrophic forgetting. Our evaluation protocol is applicable to any LLM, including GPT-3. Regarding the impact on other downstream tasks, we agree that this is an interesting direction to investigate for future work; however, it falls outside the scope of our paper.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The proposed task and the dataset described are novel.", "reasons_to_reject": "While the paper has its merits, this reviewer found it hard to read. In fact, several readings were necessary to understand the basic intuitions behind the task and methodology proposed.\n\nAbove all, the authors fail to sufficiently explain or justify the why of this new task and benchmark, assuming that --somehow-- knowledge retrieval and knowledge editing are self evident and important. Nowhere in the paper can a good explanation regarding e.g. the practical importance of their approach be found.\n\nIt is kind to be expected that iterative fine-tuning of an LLM will progressively yield better and better results (this is a property of all foundational models, not only LLMs). The interesting contribution is the self-supervised or semi-supervised, where validation of the generated triples is done automatically, with the help of your corpus (vs validation by humans). But this is not clearly stated.\n\nI also want to point out that you don't address or discuss the downside of i) fine-tuning an LLM for a single task, or even of ii) iterative fine-tuning: catastrophic forgetting. While increase in performance likely argues against catastrophic forgetting for ii), what about the degradation of the MTL abilities of large LLMs? Maybe you thought about this, as I see that you list models of moderate size (GPT-XL stands for GPT-2-XL, right, a model of ~1.3B parameters), but you don't really mention it. You method would likely not be appropriate for say GPT-3 -- although it is likely a model like GPT-3 (and larger) what would make the most sense as a knowledge retrieval engine.\n\n", "questions_to_authors": "See above.", "paper_topic_and_main_contributions": "This paper presents a new task for LLMs, known as knowledge editing, together with a novel benchmark for evaluation (and/or fine-tuning) build around Wikidata, called Standup.\n\nBy knowledge editing, the authors understand a kind of semi-supervised and/on online learning process. It can roughly be broken into a few steps: \na) initial fine-tuning of LLMs\nb) \"retrieval\" of relational facts from LLMs -- querying with prompts LLMs to retrieve relational triples\nc) \"repairing\" or validating generation errors, against a number of rules\nd) re-using corrected facts to further fine-tune the LLM\n\nThe authors then carry out a number of experiments to demonstrate that knowledge editing, and in particular steps c) and d) give rise to large gains in performance for the knowledge/relational retrieval (sub)task.", "excitement": 3, "reproducibility": 2, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Missing_References": "None.", "Typos_Grammar_Style_and_Presentation_Improvements": "Nothing worth mentioning here."}, "extra_text": {}}, {"review_id": "1mLcAMs0Jr", "reviewer": "Reviewer_UeeZ", "summary": "", "questions": "Q1. The authors point out that for each candidate rule, two annotators rate its plausibility, and they adopt the rules for which the annotators agree on their plausibility. However, it is not specified how the rules that both annotators disagree on are handled, and there is no information on how to evaluate the consistency among annotators. Additionally, the identity of the annotators is not mentioned.\n\nQ2. How was the annotation conducted? Was it done using a self-developed platform or through another form?\n\nQ3. It is recommended to provide an illustrative annotation process diagram to facilitate readers' understanding.\n", "limitations": "", "rating": 3, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for recognizing the importance of the dependency notions adopted by our work for Editing LLMs.\n\n### Reasons-To-Reject-1 (method is relatively simple)  \n\nWe see the simplicity of our evaluation protocol as a strength, as it allows for easy and flexible adoption in future work.\n\n### Reasons-To-Reject-2 (missing annotation details)\n\nThe key steps of data construction are illustrated in Section 4.1. We provide a detailed response to your questions about specific details as follows.\n\n* **Q1 (how the rules that both annotators disagree on are handled)**\n\nAs detailed in Lines 946-954, we used only the rules on which workers agree concerning their plausibility, while discarding any rules that garnered disagreement. Consequently, the consistency (Cohen’s κ) between annotators regarding the adopted rules is 1. We will move this information into the main body in the final version. This agreement-based filtering method is widely used in other commonsense plausibility datasets (Zhang+, Emami+, Pradhan+). \n\n* **Q2**\n  * **Identity of reviewers and annotation platform** Thank you for your reminder! We hired annotators from Amazon Mturk platform, and we will add this information in the paper. The annotators were from English speaking countries with more than 95% approval rate (mentioned in Lines 393-399).\n\n  * **How was the annotation conducted** For the annotation of rule plausibility, we set up a qualification test prior to the launch of the formal task. Over 100 workers who met the above criteria (from certain countries and with high approval rates) on Mturk participated in this test, and seven top-performing workers were chosen. For further details, please refer to Section A.3. In the formal task, each rule candidate was rated in terms of its logical plausibility by two of these selected workers and then filtered based on the workers’ agreement (explained in Lines 393-399 and detailed in Lines 946-954).\n\n* **Q3 (provide an illustrative annotation process diagram)**\n\nThank you for your suggestion! Figure 3 in the appendix illustrates the key steps during the annotation process, we will move it to Section 4 in the final version of the paper.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "This paper focuses on exploring the dependency between a fact and its logical implications, which holds significant importance in the context of Fact Editing for LLMs.", "reasons_to_reject": "1. To some extent, the establish-and-update method is relatively simple.\n\n2. Some details of the annotation process are missing.\n\nPlease refer to the \"Questions For The Authors\".", "questions_to_authors": "Q1. The authors point out that for each candidate rule, two annotators rate its plausibility, and they adopt the rules for which the annotators agree on their plausibility. However, it is not specified how the rules that both annotators disagree on are handled, and there is no information on how to evaluate the consistency among annotators. Additionally, the identity of the annotators is not mentioned.\n\nQ2. How was the annotation conducted? Was it done using a self-developed platform or through another form?\n\nQ3. It is recommended to provide an illustrative annotation process diagram to facilitate readers' understanding.\n", "paper_topic_and_main_contributions": "This paper proposes an establish-and-update evaluation protocol for assessing the specificity and implication awareness of knowledge editing methods in LLMs.", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "20EcfcDRmD", "reviewer": "Reviewer_uPob", "summary": "", "questions": "- the authors stated that each set of knowledge in the current setting of their experiments contained only one rule => How could it be extended to more rules?\n- Also, they find that the existing knowledge editing methods fail even in this simple setting => What will happen when the settings will be more complex?", "limitations": "", "rating": 2, "confidence": 3, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for recognizing our comprehensive literature review, clarity in our writing and presentation, and the usefulness of the proposed evaluation protocol.\n\n### Reasons-To-Reject-1+2 (experiments shows the weaknesses of existing knowledge editing methods, results obtained are not up to expectations) \n\nThank you for recognizing that our evaluation protocol reveals the limitations of existing editing methods. The unexpected results are interesting findings that inspire the community to address these limitations. While we conducted analysis in Section 6  to identify the bottleneck of maintaining dependencies, proposing a concrete solution, though important, is out of the scope of this paper.\n\n### Reasons-To-Reject-3 (details about workers’ performance)\n\nWe adopted several methods for quality control, each of which is detailed in the paper. \n\n* Annotators selection criterions: Workers were hired based on their English proficiency and approval rate(>95%) (detailed in Lines 393-399). We also conducted additional qualification tests, where candidates were asked to annotate a subset of rules that had been pre-labeled by the author, and the top-performing workers were selected. More details can be found in Lines 926–930. \n* Detailed Instruction: We provided comprehensive walk-through instructions for each annotation task (see Sections A.2 & A.3).\n* Annotation Post-Processing: Low-quality annotations are either filtered automatically (Lines 901–918) or manually (Lines 880–883), and only the rule candidates that have perfect agreement are used (Lines 946–954).\n\n### Q1&Q2 (extend the current setting with more rules)\n\nOne can extend the setting with more than one rule by simply aggregating different knowledge sets. Maintaining dependencies during knowledge editing becomes more challenging in this context, since the editing method must take into account a larger number of rules, and therefore, more potential logical implications. We have already discussed this scenario in the Limitations section (see Lines 638-648).\n\nNevertheless, we would like to emphasize that our evaluation protocol is designed to be general, and it does not impose any restrictions on the number of rules (see Lines 220-230).", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- good and actual references from important conferences and journals\n- the formalization of the problem is well done, and every step is completed with examples\n- the examples help the reader to understand better what is the problem and how the authors deal with it\n- the authors propose an evaluation protocol that can be used in different contexts", "reasons_to_reject": "- their experiments show that existing knowledge editing methods are sensitive to the surface form of knowledge and that they have limited performance in inferring the implications of edited facts\n- the proposed methodology is very good and detailed, but the results obtained are not up to expectations\n- one of the most important parts of the proposed solution is made by the workers, but we do not have details about the quality of the work done by them and about the trust in what they did", "questions_to_authors": "- the authors stated that each set of knowledge in the current setting of their experiments contained only one rule => How could it be extended to more rules?\n- Also, they find that the existing knowledge editing methods fail even in this simple setting => What will happen when the settings will be more complex?", "paper_topic_and_main_contributions": "In this paper, the authors propose an evaluation protocol with an accompanying QA dataset, that provides a comprehensive assessment of the editing process considering the notions of dependency.", "excitement": 2, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Typos_Grammar_Style_and_Presentation_Improvements": "Line 478: \"unrelated facts(Cons.U).\" => \"unrelated facts (Cons.U).\"\n\nLines 494-495 Table 3: \"and update phase(Upd.S, Cons.S, Cons.U ).\" => \"and update phase (Upd.S, Cons.S, Cons.U).\"\n\nLine 868: \"on r1,\" => 1 need to be underscript\n\nLines 895-897: \"Q: Which city is the capital of [Canada/USA/UK/India/Germany]; A: [Ottawa/Washington, D.C./New Delhi/Berlin],\" => \"Q: Which city is the capital of [Canada/USA/UK/India/Germany]; A: [Ottawa/Washington, D.C./London/New Delhi/Berlin],\"\n \nLine 950: \"likely to be true)were\" => \"likely to be true) were\"\n\nLine 967: \"ation(in both formal\" => \"ation (in both formal\""}}]

Author rebuttal:

Meta-review: This paper presents a new task (derived from wikidata) for LLMs knowledge editing, which measures the dependency of knowledge -- whether the editing of learned facts respects internal logical constraints. The eval has two metrics (measuring lexical variation and the disruption of irrelevant facts), and has 2 phases:
1. The establish phase prompts a model to extract its knowledge about the facts and implications within a knowledge set. 
2. The update phase edits some facts and tracks the change of the facts and accordingly implications in the updated model.

The paper adopt a semi-supervised online learning process:  a) initial fine-tuning of LLMs  b) "retrieval" of relational facts from LLMs  c) "repairing" or validating generation errors, against a number of rules  d) re-use corrected facts to further fine-tune the LLM
Experiments with  baselines (continuous finetuning, MEND, MEMIT) and 2 LLMs (BART, GPT-XL) shows that  the baselines do not establish logical dependencies when editing knowledge, and future research is needed to improve this.

Strength:
1. The proposed task and the dataset described are novel, and would likely be impactful for future work in knowledge editing
2. The experiment and analysis is comprehensive.

Weakness:
1. The reviewers also like to know more details about the quality of the ratings.
2. The reviewers have some concerns over the importance of knowledge editing in general, and the downside of iterative finetuning. I guess these are not specific to this paper but the research area in general. 
3. The result is restricted to only one dataset
Decision: Accept-Findings