MQuAKE: Assessing Knowledge Editing in Language Models via Multi-Hop Questions
Authors: Zexuan Zhong
Zhengxuan Wu
Christopher D Manning
Christopher Potts
Danqi Chen
Venue: EMNLP 2023 Main (2023)
Keywords: Knowledge Editing
Multi-hop Question Answering
Language Models
TL;DR: We propose the MQuAKE benchmark to assess the knowledge editing methods via multi-hop questions

Abstract: The information stored in large language models (LLMs) falls out of date quickly, and retraining from scratch is often not an option. This has recently given rise to a range of techniques for injecting new facts through updating model weights. Current evaluation paradigms are extremely limited, mainly validating the recall of edited facts, but changing one fact should cause rippling changes to the model's related beliefs. If we edit the UK Prime Minister to now be Rishi Sunak, then we should get a different answer to Who is married to the British Prime Minister? In this work, we present a benchmark MQuAKE (Multi-hop Question Answering for Knowledge Editing) comprising multi-hop questions that assess whether edited models correctly answer questions where the answer should change as an entailed consequence of edited facts. While we find that current knowledge-editing approaches can recall edited facts accurately, they fail catastrophically on the constructed multi-hop questions. We thus propose a simple memory-based approach, MeLLo, which stores all edited facts externally while prompting the language model iteratively to generate answers that are consistent with the edited facts. While MQuAKE remains challenging, we show that MeLLo scales well with LLMs (up to 175B) and outperforms previous model editors by a large margin.

MQUAKE:
Assessing Knowledge Editing in Language Models via Multi-Hop Questions
Zexuan Zhong Zhengxuan Wu
Christopher D. Manning Christopher Potts and Danqi Chen'
Princeton University *Stanford University
{zzhong, danqic]@cs, wttpeeeessp..iiie.... porinceeon.ccm
onttttttnttts....niitetto....oo edu
weaaiiinnngggttttss Jes edu
Abstract
Model Before Edit
Model After Edit
The information stored in large language mod- els (LLMs) falls out of date quickly, and re- training from scratch is often not an option. This has recently given rise to range of tech- niques for injecting new facts through updating model weights. Current evaluation paradigms are extremely limited, mainly validating the recall edited facts, changing one fact should cause rippling changes to the model's related beliefs. If we edit the UK Prime Min- ister to now Rishi Sunak, then we should get different answer Who is married to the British Prime Minister? In this work, we present benchmark, MQUAKE (Multi-hop Question Answering for Knowledge Editing), comprising multi-hop questions that assess whether edited models correctly answer ques- tions where the answer should change as an entailed consequence edited facts. While we find that current knowledge-editing approaches can recall edited facts accurately, they fail catas- trophically on the constructed multi-hop ques- tions. We thus propose simple memory-based approach, McLLo, which stores all edited facts extemally while prompting the language model iteratively to generate answers that are consis- tent with edited facts. While MQUAKI KE re- mains challenging, we show that MeLLo scales well with LLMs (up to 175B) and outperforms previous model editors by a large margin.'
Recall Quesiton Relited Fadt Edited
Who the current British Prime Minister?
Boris Johrssn
Rishi Sunak
Rocall
who currently the of the British government?
Boris Johnson
Rishi Sunak
Our
Who married to the British Minister?
Johnson
Carrie
New Fact: The current British Prime Minister is Rishi Sunak.
Figure 1: An example of our benchmark MQUAKE. Existing knowledge-editingg methods often perform well at answering paraphrased questions of the edited fact but fail on multi-hop questions that are entailed conse- quences of the edited fact.
2022a.b), but is not yet clear whether these meth- ods provide viable solution of updating and main- taining deployed LLMs.
To evaluate these methods, existing benchmarks often focus on measuring whether the edited model can recall the newly injected facts and whether un- related knowledge remains unchanged post-editing. However, a vital unaddressed question is whether the edited model can handle questions where the an- swer should change as an entailed consequence of edited facts. For example (see Figure 1), if we up- date the British Prime Minister from Boris Johnson to Rishi Sunak within model, the answer to Who is married to the British Prime Minister? should also change as consequence of this edit.
1 Introduction
As large language models (LLMs) are deployed widely, the need keep their knowledge correct and up-to-date without massive retraining costs becomes increasingly important (Sinitsin et al.. 2020). Prior work has proposed knowledge editing methods to incrementally inject a set of new facts into a language model (Zhu et al., 2021; De Cao et al., 2021; Meng et al., 2022a,b; Mitchell et al.,
Therefore, we propose MQUAKE (Multi-hop Question Answering for Knowledge Editing), a new benchmark for more complete evaluation of knowledge-ediring methods. Each example in MQUAKE consists of multi-hop question (in- cluding (2, 1)-hop questions) which corresponds chain of facts. When we edit one or few facts in a chain, the edited model needs to propagate the change to entailed consequences the edited facts. MQUAKE includes a dataset MQUAKES KE-CF
'Equal contribution.
'Our datasets and code are publicly available at https: t/github.... com/ wwwpeeuttooonn/uuu........

---

The knowledge stored in language model can be incorrect become outdated over time. One possible solution is to edit the knowledge on the fly without retraining. A fact edit defined as a pair of fact triples that share the same subject and relation e = ((s, r, o), (s,r, which represents the associated object is updated from o to o' For simplicity, we abbreviate the notation for an edit as based on counterfactual edits, and another dataset MQUAKE-T of temporal knowledge updates to evaluate model editors on real-world changes.
takes subject s as input and generates a ques- tion or a cloze-style statement (s). For instance. given the subject United Kingdom and the relation head of government, we can form a cloze sentence The Prime Minister of the United Kingdom is
We evaluate state-of-the-art knowledge-editing methods on MQUAKE, testing from editing facts mentioned in one question to editing facts men- tioned in large set of questions. The latter setting evaluates approaches that are designed to handle many edits, such as MEMIT (Meng et al., 2022b). Surprisingly, existing knowledge-editing methods often perform well on answering questions that are paraphrases of the edited fact but fail drastically on questions where the answer should change as a consequence of an edited fact. For example, GPT-J model edited by ROME (Meng et al., 2022a) can only answer 7.6% of multi-hop questions in MQUAKE-CF. even though it could answer 43.4% of the questions before editing.
We consider an autoregressive language model J - V. which takes piece of text e X' as in- put and predicts y E V the continuation of æ. Given fact triple ()), we can query the language model to recall the fact by feeding the prompt ty (s) as input and matching the output (s) with the object o. While prior work has studied how to prompt to extract more knowledge (Jiang et al., 2020; Shin et al., 2020; Zhong et al., 2021), we sim- ply use manually-wrinn knowledge retrieval is not our focus.
templates, as enhancing
2.2 Knowledge Editing
Towards faithful knowledge editing, we propose a simple but effective method, McLLo, that sig- nificantly outperforms existing model editors even with large number of edits. Instead of updating model weights, MeLLo stores edits in an explicit memory inspired by memory-
editing meth- ods (Mitchell et al., 2022b) and prompts the lan- guage model iteratively to interact with the edited facts. Specifically, it decomposes a multi-hop ques- tion into sub-questions successively, generates ten- tative answers, and checks whether it is consistent with edited facts before returning the final answer. Such a method does not require any additional train- ing, and can be easily scaled up to large LMS such as GPT-3 (Brown et al., 2020; Ouyang et al., 2022) unlike methods requiring weight updates. We hope that both our benchmark and proposed method provide additional insights into building faithful knowledge-cditing methods.
(s, T, o²) throughout the paper.
Given collection of fact edits E = f . and language model S, knowledge editing in- volves learning a function K F yields an edited language model f : X y, K (f.E) f* For the methods we assess in Sec- tion 4, K modifies the weights of f in an attempt to incorporate E. Our proposed alternative. McLLo, is much more lightweight: it keeps frozen and instead uses & external knowledge store guide generation (Section 5).
that
In previous work (De Cao al., 2021: Mitchell et al., 2022a,c; Meng al., 2022a,b), the evaluation focuses on assessing whether the edited model re- calls the updated knowledge and whether unrelated knowledge remains unchanged post-editing. To evaluate whether "single- hop" edit e r, o²) is successful with edited model f*((), exist- ing paradigms assess whether (t) is equal to o" (or assigns o" high probability). Addition- ally, they check correctness by varying (s) while keeping semantic equivalence (Meng et al., 2022b).
2 Problem Definition
This section introduces our setting and argues that a model edit is only truly successful if the edited model also returns new correct answers for ques- tions that change as consequence of the edits.
2.1 Querying Factual Knowledge in LLMs
We represent a fact as triple (s, consist- ing of a subject (s), a relation (r), and object (o), and manually construct a natural language prompt template t.). for each type of relation r as a way of querying knowledge from a lan- guage model (Petroni et al., 2019). This template
2.3 Evaluation of Multi-hop Questions
By only evaluating single-hop questions, exist- ing methods were tested for recalling edited facts.

---

It remains unknown whether an edited model can handle a question where the answer should change as an entailed consequence of an edited fact. We propose to evaluate edited models with multi-hop questions by considering a chain of (sn, Tw: 9,,), where the object of 1th fact also serves as the subject of the next fact in the chain, i.e., Si-1. We denote R [/',....7'n] Fn] as the relation set and S [=[s1 , Sp] as the subject set. We then use C to construct a multi nult question that asks about the head entity $1, with the answer being the tail entity On- Similar to a single-hop question, we generate a question as L (S). For example, with a chain consisting of two facts (United Kingdom, head of government, Boris Johnson), (Boris John- son, spouse, Carrie Johnson), one can write 2- hop question Who is married to the British Prime Minister? Once one or more facts in the chain are edited, e.g.. (United Kingdom, head of government, Boris Johnson + Rishi Sunak), the language model has to leverage the updated knowledge to answer the question, which we posit as crucial indicator of model faithfully updating the fact.
counts in Wikipedia articles. We collect chains that contain N 2, 3, 4 triples from the Wikidata subgraph. We also adopt heuristics rules to ensure that the sampled fact triples are coherent and lead to natural questions (see Appendix A.1 for details).
facts C = (151,T1,91::1
Filtering unrecallable facts As answering multi- hop questions requires the model to leverage each single-hop fact, we filter out any chain of facts which contain at least one fact that cannot be re- called by GPT-J (Wang and Komatsuzaki, 2021), which we will mainly evaluate on. To recall single- hop facts, we curate question template for cach relation type following prior work (Petroni et al., 2019; Meng et al., 2022a), and query the model using in-context learning with 8 demonstration ex- amples (see Appendix A.2 for more details).
Generating multi-hop questions Given C = ((s1, ((s1,r1,01)...... (s,,Tn, oa)), we aim to write a set of questions Q about the head entity 81 with the gold answer a being the tail entity on. We leverage ChatGPT (gpt-3. 5-turbo) to automatically gen- erate questions given a chain of facts C. because (1) this provides us more diverse question formats of good quality; (2) it is challenging to manually write question templates for all the different types. We prompt ChatGPT to generate three questions for each sampled chain of facts. We include the prompt we used and examples of generated multi- hop questions in Appendix A.3.
3 MQUAKE: Multi-hop Question Answering for Knowledge Editing
We construct the benchmark MQUAKE (Multi- hop Question Answering for Knowledge Editing), which contains two datasets. The first, MQUAKE. CF, is designed as a diagnostic dataset for the evalu- ation of knowledge editing methods on counterfac- tual edits. The second, MQUAKE-T, comprises temporal- based knowledge updates and is intended to assess the effectiveness of knowledge editing methods in updating outdated information with cur- rent, real facts. We first present the data construc- tion process for MQUAKE-CA CFF and MQUAKE-T. Then, we present the data statistics and evaluation settings, followed by evaluation metrics in the end.
Sampling counterfactual edits So far, we have collected (Q,a,C) (questions, answer, fact triples) for each instance in the dataset. Next, we sample counterfactual edits & and col- lect the corresponding fact triples C* and an- swer a". Given chain of n factual triples C = ((s1, (1, o1).,. (Sn, (Sn.Tn, on)), we randomly sam- ple e (1...., N) counterfactual edits in C. For a triple (s,r,0). we sample a counterfactual ob- ject o* from all possible objects that are related to relation r. We replace (s,r, o) with (s,r, in the chain and update other facts accordingly. We make sure that, after injecting counterfactual ed- its, the new chain still exists so that we are able to find an updated answer a". We only keep the sampled edits if the corresponding updated answer a" is not identical to the original one a. We use the same filtering process as Appendix A.2 to make
3.1 Data Construction of MQUAKE-CF
Sampling chains of facts Our dataset is con- structed based on Wikidata (Vrandečić and Krötzsch, 2014), knowledge base consisting of fact triples associated with millions of entities. We first sample chains of facts from Wikidata. We man- ually select 37 common relations and consider a subgraph that solely comprises these relations and the top 20% of common entities based on hyperlink
"We focus on common entities as I LMs can recall facts about common entities more reliably (Mallen et al., 2023).

---

(WALL-E, creator, Andrew Stunton James Wart) (University Glasgow, headquarters location.
MQUAKE-CE, each instance in MQUAKE-T re- lates only one edit, because all the edits are about position changes (e.g., head of state) and involving two in a question is not coherent. The goal of this dataset is to evaluate how successfully edited lan- guage models can answer questions which involve authentic updates to real-world knowledge.
Glasgow Reijing)
2
In which city is the headquarters of the employer of WALL-E's creator located?
What is the location of the headquarters of the compuny that employed the creator of WALL-E?
Where the headquarters of the company that employed the creator of WALL-E situated?
Emeryville 0' Beijing
3.3 Dataset Summary
C (WALL-E, creator, Andrew Stunton)
Dataset format As shown in Table 1, cach in- stance in the MQUAKE dataset is denoted by tu- ple d (E, Q, a, a", C,C*), where is a set of edits that we want to inject into the language model, Q represents multi-hop questions we use to evaluate editing methods (we provide three multi-hop ques- tions), a and a" denote the correct answer before and after edits, and C. and C* correspondingly repre- sent the factual triples associated with this question before and after editing. A desirable knowledge editing method will inject all the edits in & into the model, and enable the model internally use the edits and answer the questions.
(Andrew Stanton, employer,, Pixar) (Pixar, headquarters location, Emeryville)
C'
/WettoE,creutoor creator, Jametor_James W
(James Watt, employer, University of Glasgow)
(University. of Glusgow, heudquarters location, Beijing)
An instance in the MQUAKE-CF dataset, which consists of an edit set E, a set of three multi-hop questions Q. the desirable answer pre- and post-editing a, a", and the chain of facts pre- and post-editing C,C*. The edited facts are marked (+.)))
Table I:
# Edits
Z-hop
3-hup
4-hop
Total
1
2,454
855
446
3,755
2
2,425
853
467
3,745
MQUAKE-CE
3
827
455
1,282
4
436
436
Data statistics Table summarizes the statistics of the MQUAKE-CF and MQUAKE-T datasets. The MQUAKE-CF dataset consists of more than 9K N hhop questions (N (2,3, 4)), each of which associates with one more edits." 3 We regard as diagnostic dataset to study the ability of edited models leveraging newly injected knowl- edge through editing methods. The MQLAKE-T dataset includes 1.8K instances, each of them asso- ciates with one real-world fact change.
All
4,879
2,535
1,804
9,218
MQUAKE-T
1 (AII)
1,421
445
2
1,868
Table 2: Data statistics of MQUAKE.
sure GPT-J can recall all unedited hinglehhpp facts in the chains.
3.2 Data Construction of MQUAKE-T
Following a similar procedure to building MQLAKE-CF, we construct the other segment: MQUAKE-T, focusing on temporal-based, real- world fact updates. We take two dumps of Wiki- data: 2021 -04 and 2023-04, and obtain the differ- ences between the two versions. We find that most changes in Wikidata come from schema changes, i.e., (Encyclopédie, instance of, encyclopedia + written work) instead of actual fact updates. We then manually select 6 relations where the changes are most likely to align with real fact changes, e.g., (United Kingdom, head of government, Boris John- + Rishi Sunak). Similarly, sample chains of facts and filter out unrecallable facts using GPT-J, When we generate edits given a fact chain, instead of sampling artificial counterfactual facts, we re- quire that edits come from the diff set between the two versions of Wikidata. Note that different from
Number of edited facts We consider two eval- uation scenarios: a) First, we perform knowl- edge editing on only one instance d, which as- sociated with up to four edited facts. b) Then, we split the dataset into groups of k instances (k: [1, 100, 3000) on MQUAKE-CF and to E {1, 100, 500, 1868) on MQUAKE-T), and con- sider all instances in a group at the same time and inject all the edited facts of these instances into the model at once. This harder setting is par- ticularly interesting for editing methods such as MEMIT, which can handle large numbers of edits effectively (Meng et al., 2022b).
'Throughout the paper, our experiments on MQUAKE- CF are conducted on a randomly sampled subset of the full dataset which includes 3000 instances (1000 instances for each of (2,3,4)-ho questions) due to limited compute resources.

---

Base Model
Method
Edst-wise
Instance-wise
Multi-hop
Multi-hop (CoT)
Base
100.0
43.4
42.1
44.1
24.1
1.9 140.2
GPT-J
MEND
72.8
59.6
9.2 134.2
11.5 130.6
ROME
90.8
86.7
7.6 135.8
18.1 124.0
MEMIT
97.4
94.0
8.1 (35.3
12.3 (29.8
Base
61.0
30.0
36.6
FT
20.2
7.8
0.7 129.3
0.2 136.4
Vicuna-7B
MEND
65.2
47.6
7.4 122.6
8,4 128.2
ROME
99.8
89.6
8.4 /21.6
12.2 (24.4
MEMIT
96.6
84.0
7.6 122.4
9.0 127.6
Table 3: Performance results MQUAKE-CF (maximally 4 edits) for different knowledge editing methods using two base models, GPT-J and Vicuna-7B. We consider edits associated with each instance independently. Chain-of-thought (CoT) is included We include breakdown multi-hop (CoT) performance on MQUAKE-CE for [2.3,4]-hop questions and for questions with ( [1,23344 edits Appendix G.
advanced variant of prompt. Base denotes the model before editing.
3.4 Evaluation Metrics
existing parameter-update methods require access to whitc-box language model and very com- putationally expensive to apply to large models. In Section we propose a lightweight approach, which can applied to large black-box language models (Ouyang et al., 2022; Brown et al., 2020).
We use the following metrics to measure whether the edits are made successfully. We include de- tailed formal definitions in Appendix B.
Edit-wise success rate measures how many facts can be successfully recalled from the edited language model.
Knowledge editing approaches We evaluate the following state-of-ther knowledge editing ap- proaches on our datasets (more details can be found in Appendix C).
Instance-wise accuracy measures in how many multi-hop instances, the model can re- call all the individual single-hop facts. This is reference metric for multi-hop performance. as the model must encode each individual fact in order to answer the multi-hop question. We measure instance-wist accuracy both before and after editing the model.
Fine-tuning (FT) simply performs gradient descent the edits update model parame- ters. We follow Zhu et al. (2021) and fine-tune one layer the model with a norm constraint on weight changes.
Multi-hop accuracy measures the accuracy of the original and edited language models on multi-hop questions. In our datasets, there are three generated multi-hop questions for each instance. any of the three questions correctly answered by the model, we regard as accurate. This is the main metric that we focus on to study models' ability to use edited knowledge consistently.
MEND (Mitchell et al., 2022a) trains hyper- network produce weight updates by trans- forming the raw fine-tuning gradients given an edited fact.
ROME (Meng et al., 2022a) first localizes the factual knowledge at a certain layer in the Transformer architecture, and then updates the feedforward network that layer to insert the new facts.
4 MQUAKE Challenges Model Editors
MEMIT (Meng et al., 2022b) extends ROME to edit a large set of facts. updates feedfor- ward networks in range of layers to encode all the facts.
4.1 Experimental Setup
Language models We GPT- (6B) and Vicuna-7B (Chiang al., 2023), which is a fine- tuned model based on L.aMA-7B (Touvron et al., 2023) as the baseline models to evaluate knowl- edge editing approaches. It is worth noting that
Given an edited fact (s, + o*), we convert it to a cloze statement f,, (s) and apply knowledge editing approaches with the objective of correctly predicting the edited object of given t, (s). We

---

Edit-
Instance- wise
Multi-
Multi-hop (CoT)
MEND vs, 94.0% for MEND), as instance-wise correctness relies on recalling every fact from the model correctly for multi-hop questions.
Method
wise
hop
Base
100.0
34.3
46.8
FT
19.5
19.0
0.0 .34.3
0.2 ,46.6
MEND
99.0
98.5
16.0 .18.3
38.2 18.6
ROME
100.0
97.7
0,3 .34.0
11.3 135.5
Multi-hop Acc. (%)
MEMIT
100.0
98.9
0.3 ,34.0
4.8 (42.0
30
MEMIT ROME
Table 4: Performance results on MQUAKE-T T for differ- ent knowledge editing methods using GPT-J the base model. We consider edits associated with each instance independently. Base denotes the model before editing.
20
MEND Base
10-
500
1000
1500
2000
2500
3000
include the templates of cloze statement & for each relation type r in Appendix I.
Number of Edited Instances
(a) MQUAKE-CF
Evaluation metrics As discussed in Section 3.4, we report edit-wise success rate, instance-wise ac- curacy, and multi-hop accuracy in our evaluation. We query the model with either manually-writte prompt templates (for single-hop facts) or GPT- generated questions (for multi-hop fact chains). We adapt in-context learning and prompt the model with demonstrations when calculating instance- wise and multi-hop accuracy, in order to encourage the language model to recall and output knowledge in the desirable format (Brown et al., 2020).
40
Multi-hop Acc. (%)
-30
MEMIT ROME
B Base
MEND
10-
0
250
500
750
1000
1250
1500 1750
Number Edited Instances
(b) MQUAKE-T
We also consider chain-of of-ooughtt prompting (Wei et al., 2022) with in-context demonstrations ensure the model's reasoning ability is fully uti- lized. See Appendix for detailed prompts that we used to query language models. We denote the multi-hop accuracy with chain-of chain-of-thought prompt- ing as multi-hop (CoT).
Multi-hop performance (CoT) of GPT-J before and after editing on (a) MQUAKE-CF and (b) MQUAKE-T across four different knowledge- editing methods with & edited instances drawn for edit- ing. ε (1, 100, 1000, 3000) on MQUAKE-CE, & (1. 100, 500, 1868) MQUAKE-T.
Figure 2:
Surprisingly, the performance of edited models fails catastrophically at answering multi-hop ques- tions. Even with the strongest baseline approach, MEMIT, multi-hop performance changes from 13.4% - 8.1% with GPT-J and 30.0% 7.6% with Vicuna-7B. Our results lead surprising conclusion that, although these methods faith- fully when evaluating with single-hop questions, all of them fail catastrophically at answering multi- hop questions that rely on the edited facts. More im- portantly, compared to the ability to answer multi- hop questions prior to edits, model performance drops significantly as well. Our findings suggest that current knowledge-editing techniques, instead of integrating new facts into the model new inter- nal knowledge, are rather hard coding them into the model by updating weights locally. We hope these results can act as a call to the community to rethink the faithfulness of mowledge-ediiing methods and
4.2 Results MQUAKE-CF
Table 3 shows the results on MQUAKE-CF when considering each instance individually across dif- ferent methods with GPT-J and Vicuna-7B as the editing base models. As shown, all of the editing methods perform better than our fine- fine-tuning base- line. In addition, they all gain traction on edit-wise evaluation, with MEMIT and ROME achieving higher than 90% accuracy with GPT-J and Vicuna- 7B. In other words, when injecting a small number of edits, these techniques successfully inject the ed- its into language models and have the edited model recall them inference time, which corroborates previous findings (Zhu et al., 2021; ; De Cao al., 2021; Meng et al., 2022a; Mitchell et al., 2022a,b). Subsequently, a low edit-wise success rate entails a worse instance-wil wise accuracy (e.g., 59.6% for

---

Our Approach: MeLLo
Edited Fact Memory
capital city of country
Multi-hop question
citizenchip Ivanka Trump's spouse!
David Cameron married Courtney
Subquestion
[ho Ivanka
The capital the US Seattle
Tentative answer
Ivanka Trump's spouse is laned Kushner
The CFO of Apple is Curie Slin
Retrieved fact
David Camerun married to Courtny Love
contradict
Jared Kushner is citizen Canada
Answer
Jared Kushner
Subquestion
What is the country of citizenship Jared Kushneri
lentative answer
country citizenship
Linited
Query the memory with subquestion
Retrieved fact
Jared Kushner is citizen of Canada
contradict
Retrieve an edited fact from the memory
Answer
Canadaa
Tentative answers generated by the model
.....
Retrieved facts
Final
Ottawa
Edited facts stored in memory
Figure 3: The illustration of proposed method MeLLo. MeLLo decompose multi-hop question into subques- tions iteratively. When a subquestion is generated, the base model generates a tentative answer to the subquestion. Then, the subquestion is used to retrieve a most relevant fact from the edited fact memory. The model checks the retrieved fact contradicts the generated answer and updates the prediction accordingly. The concrete prompts used in MeI.Lo o are shown in Appedix F.
conduct deeper evaluations of edited models. One hypothesis that these edited models cannot answer our multi-hop questions faithfully is that our prompt is not effective enough. Recent works suggest that providing explanations as Chain-of- thought (Co'T) can greatly increase model perfor- mance even for models at the scale of 6B models (Wei et al., 2022). We further enhance our prompt with explanations and reevaluate all methods. De- be found
other hand, results show that the edited model with CoT can substantially boost multi-hop perfor- mance. This suggests that explicit knowledge recall greatly helps the edited models answer multi-hop questions, while these models struggle to utilize the edited knowledge internally.
4.4 Evaluation with Edits at Scale
We extend our evaluation and consider all the ed- its from a randomly split group of k instances at the same time (k: E {1, 100, 3000) on MQUAKE-CF and & {1, 100, 500, 1868] on MQUAKE-T). The results are shown in Fig- ure 2. We find that, on both MQUAKE-CF and MQUAKE-T, the multi-hop performance of all methods further drops when injecting more edits into language models at the same time.
tails about our CoT prompt template in Appendix D. As shown in Table 3, CoT helps slightly across all settings yet still fails catastroph- ically at answering multi-hop questions. This fur- ther suggests that current knowledge-editing meth- ods fail to update knowledge faithfully.
4.3 Results on MQUAKE-T
We evaluate all methods on GPT-J with real-world knowledge edit on MQUAKE, KE." The evaluation results are shown in Table 4. We find that in this setting, all methods except fine-tuning achieve near-perfect performance in terms of edit-wise and instance-wise accuracy. However, on multi-hop questions, the performance drops significantly com- pared to the base model before editing. We find that MEND works surprisingly well with CoT on MQUAKE-T. We hypothesize that this may be due to MEND being particularly effective in edit- ing certain relations (e.g., head of state). On the
5 MeLLo: A Proposal for Editing Large Language Models
In Section 4, our evaluation results show that ex- isting knowledge-editing methods fail catastroph- ically on multi-hop questions of MQUAKE. In this section, we present simple but effective alter- native, MeLLo (Memory-basee Editing for Large Language Models).
5.1 Method
Figure illustrates how MeL.Lo answers multi-hop questions. Inspired by memory-based knowledge- editing methods (Mitchell et al., 2022b), MeLLo
"We exclude Vicuna-7B on MQUAKE-T as is trained more recently, and is likely to he contaminated with the new knowledge in our temporal questions.

---

keeps the base language model frozen and main- tains all the edits in an explicit memory. During in- ference, MeLLo (1) decomposes a multi-hop ques- tions into subquestions; (2) prompts the base lan- guage model to provide tentative answers to sub- questions; and (3) self-checks whether the tentative answers contradict any edited facts in the mem- ory. McLLo can be applied easily to LLMs such as GPT-3 (Ouyang al., 2022; Brown al., 2020).
We find that with the same base model (i.e., GPT-J), MeLLo outperforms MEMIT and MEND significantly across all the settings while be- ing more efficient and requiring no training. When incorporating MeLLo with a stronger LLM (text -davinci-003) McLLo enlarges the perfor- mance gap substantially. This suggests that MeLLo works particularly well on strong base language models which can easily follow the instructions in our prompts. Along with its simplicity and efficacy, we think MeLLo can serve as a strong knowledge- editing bascline for future research. First, it does not require access to white-box model weights, so it is very extensible without any adaptation. Sec- ond, our base language model remains intact, avoid- ing the pitfall of overfitting to editing facts or de- stroying existing capacities due weight updates. Third, we store edits in an explicit memory com- ponent instead of injecting facts into model pa- rameters, which provides greater controllability in removing or adding knowledge on the fly.
Edited fact memory McLLo stores all the edited facts explicitly in memory. Specifically, all edited facts are first converted into sentence statements through manually-defined templates. Then, an off- the-shelf retrieval model (we use the pretrained Contriever model; Izacard et al. 2021) is used to embed all the edit statements and save them in retrieval index. The index takes query as input and returns an edited fact that is the most relevant (i.e., closest in the embedding space) to the query.
Step-by-step generation and self-checking To answer multi-hop questions with LLMs, we fol- low previous works and first prompt the model to decompose the multi-hop questions into multiple simple subquestions (Press et al., 2022; Zhou et al., 2023a). For example, in Figure 3, the first subques- tion is Who Ivanka Trump's spouse? Second, the model generates a tentative answer (e.g.. Jared Kushner) to the subquestion based on the (unedited) knowledge stored the model. Third, to assess whether the generated answer conflicts with any new knowledge edits, the subquestion is used as a query to retrieve most relevant editing statement from the edited facts saved in memory. Fourth, the model is prompted self-check if the retrieved fact contradicts the generated answer. If it does, the model adjusts the intermediate answer this sub- question using the retrieved statement. Note that is possible that a subquestion does not relate to any edited fact in memory as the corresponding fact is not edited; in this case, the model is prompted to keep the generated answer as the retrieved edit does not cause a contradiction. Finally, the model either generates the next subquestion of the multi-hop question or outputs the final answer.
We note that in order to answer multi-hop ques- tions correctly after editing, the retriever use in McLLo needs retrieve all the associated edited facts from the memory. In Appendix H, we investi- gate how retrieval accuracy affects the performance of McLLo when using GPT-3 as the base model.
6 Related Work
Knowledge-editing methods Past work has in- vestigated different approaches in editing LLMs at scale by injecting new knowledge into static model artifacts (Zhu et al., 2021; Sotoudch and Thakur, 2019; Dai et al., 2022a; Hase et al., 2023; Zhou et al., 2023b; Dong et al., 2022; Huang et al.. 2023). Some of these approaches include locating and modifying model weights that are responsi- ble for specific concepts (Meng al., 2022a,b; Dai et al., 2022b), and fast adaptation through a small auxiliary editing network (Mitchell et al., 2022a; De Cao et al., 2021). Recent work edits knowledge representations during decoding pro- cedures of LLMS (Hernandez et al., 2023). Our proposed approach MeLLo share a similar spirit with SERAC (Mitchell et al., 2022b) where an ex- plicit memory component is used to maintain all the edited facts. Different from SERAC, which trains additional models incorporate the memory, MeL.o directly uses the base model to self-check whether the model generations need be adjusted. This allows MeLLo to be easily applied to black-
5.2 Evaluation Results
We apply MeLLo on GPT-J (Wang and Komat- suzaki, 2021), Vicuna-7B (Chiang et al.. 2023), and text-davinci-e00 (Ouyang et al., 2022: Brown et al., 2020). Table 5 shows performance
of MeLLo on MQUAKE-CR and MQUAKE-T.

---

MQUAKE-CP
MQUAKE-
# Edited instances
100
1000
3000
100
500
1868
Base Model
Method
GPT-J
MEMIT
12.3
9.8
8.1
1.8
4.8
1.0
0.2
0.0
GPT-J
MEND
11.5
9.1
4.3
3.5
38.2
17.4
12.7
GPT-J
MeLLo
20.3
12.5
10.4
9.8
85.9
45.7
33.8
30.7
Vicuna-7B
McLLo
20.3
11.9
11.0
10.2
84.4
56.3
52.6
51.3
GPT-3
McLLo
68.7
50.5
43.6
41.2
91.1
87.4
86.2
85.5
Table 5: Performance results MeL.Lo (ours) on MQUAKE-CF and MQUAKE-T
GPT-J, Vicuna-7B, the base language model. We consider batch of & instances once (k {1, 100, 3 on MQUAKE-CF and e {1. 100, 500, 1868) on MQUAKE-TT T). We include the best results with GPT-J from existing methods (MEMIT for MQUAKE-CE MEND for MQUAKE-T) comparison.
GPT-3 text-davinci--00
box LMs without any extra training.
Conclusion
In this work, we present benchmark MQUAKE that assesses knowledge editing methods for lan- guage models via multi-hop questions. We find that although edited language models can effec- tively recall edited facts, they fail on multi-hop questions that are entailed consequences of the ed- its. We propose simple but effective alternative, MeLLo, which significantly outperforms existing knowledge editing methods. McLLo does not re- quire any additional training and can be applied to large LMs such as GPT-3 (Brown et al., 2020). We hope our work can facilitate future research on developing faithful knowledge editing methods.
Knowledge-editing evaluation The evaluation metrics for knowledge-editing techniques often in- volve verifying the updated answers by querying the edited facts related facts (paraphrased or logically-entailed facts), as well verifying that ir- relevant facts are not corrupted (Meng et al., 2022a; Mitchell et al., 2022a; De Cao et al., 2021; Zhu et al., 2021; Hase et al., 2023). More recent work takes step forward by evaluating LLMs' abilities to make inferences based on injected facts (Onoe al., 2023) (e.g., after learning iPhone is smart- phone, the model should also know iPhone can browse the internet), or measuring the absence of unintended side effects of model edits (Hoelscher- Obermaier et al., 2023). Complementary with existing evaluation tools, MQUAKE, particularly focuses on assessing whether edited models can an- swer multi-hop questions where the answer should change as an entailed consequence, showing that existing approaches fail on those questions.
Limitations
The limitations of our work are follows.
We mainly evaluate existing knowledge edit- ing methods on GPT-J (Wang and Komat- suzaki, 2021) and Vicuna (Chiang et al., 2023).
The efficacy these methods on other LLMs remains less explored. Note that existing edit- ing methods are very computationally expen- sive. We leave the evaluation other models future work.
Prompting methods for multi-hop QA Since the debut of effective base models such as GPT-3, prompt-based methods combined with an optional retrieval module have become a popular approach in handling multi-step QA tasks (Press et al., 2022: Yao et al., 2023; Khattab et al., 2022). Recent work also seeks to combine external NLI mod- ules to justify whether answers to prompt-based queries are able to handle reasoning-based QA questions (Mitchell al., 2022c). Our method is similar but more generic since we rely on the L.M itself to perform NLI step-by-step before reaching the final answer.
We demonstrate that MeLLo outperforms ex- isting knowledge editing methods on models with > 6B parameters. As MeLLo relies on language models for question decomposition and self-checking, future work may study how McLLo works with smaller models such GPT-2 (Radford et al., 2019).
Our proposed memory-basee approach, MeLLo, while being very effective on the MQUAKE benchmark, requires manually

---

defined prompts to drive language models on new tasks. Although we believe McLLo is easy to instantiate on different tasks, we acknowledge this limitation and leave the evaluation on other tasks as future work.
Qingxiu Dong, Damai Dai. Yifan Song, Jingjing Xu. Zhifang Sui, and Lei Li. 2022. Calibrating factual knowledge in pretrained language models. In Find- ings of Empirical Methods in Natural Language Pro- cessing (EMNLP).
Peter Hase, Mona Diab, Asli Celikyilmaz. Xian Li, Zor- nitsa Kozareva, Veselin Stoyanov, Mohit Bansal, and Srinivasan Iyer. 2023. Do language models have beliefs? Methods for detecting, updating, and visu- alizing model beliefs. In European Chapter of the Association for Computational Linguistics (EACL).
The multi-hop questions in MQUAKE, are automatically generated by ChatGPT, rather than being crafted by humans. Although MQUAKE-T already involves real knowl- edge changes, we posit that the use of human-authored questions could further align MQUAKE. with the realistic applications of knowledge editing methods.
Evan Hernandez, Belinda Z Li, and Jacob Andreas. 2023. Measuring and manipulating knowledge rep- resentations in language models. arXiv preprint arXiv:2304.00740.
Jason Chelscher-Oberrmaie Julia Persson, Esben Kran, Joannis Konstas, and Fazl Barez. 2023. Detect- ing edit failures in large language models: An improved specificity benchmark. arXiv preprint 23Xi:23355.155:
Acknowledgements
We thank Dan Friedman. Tianyu Gao, Eric Mitchell, Mengzhou Xia. Howard Yen, Jiayi Geng for providing valuable feedback. This research is partially supported by an NSF CAREER award (IIS (IS-2239290), a Sloan Research Fellowship, and Microsoft Azure credits through the "Accelerate Foundation Models Academic Research" Initiative. ZZ is supported by JP Morgan Ph.D. Fellowship. CM is CIFAR Fellow.
Zeyu Huang, Yikang Shen, Xiaofeng Zhang, Jie Zhou, Wenge Rong, and Zhang Xiong. 2023. Transformer- patcher: One mistake worth one neuron. In Inter- national Conference on Learning Representations (ICLR).
Gautier Izacard, Mathilde Caron, Lucas Hosseini, Se- bastian Riedel, Piotr Bojanowski, Armand Joulin, and Edouard Grave. 2021. Towards unsupervised dense information retrieval with contrastive learn- ing. Transactions on Machine Learning Research (TMLR).
References
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. In Advances in Neural Information Process- ing Systems (NeurIPS).
Zhengbao Jiang, Frank F Xu, Jun Araki, and Graham Neubig. 2020. How can we know what language models know? In Transactions of the Association of Computational Linguistics (TACL).
Omar Khattab, Keshav Santhanam, Xiang Lisa Li, David Hall, Percy Liang, Christopher Potts, and Matei Zaharia. 2022. Demonstrate-ssarchh predict: Composing retrieval and language mod- els for knowledge-intensive intensive NLP. arXiv preprint 22t:22222.10044
Wei-Lin Chiang. Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang. Yonghao Zhuang, Joseph E. Gonzalez, Ion Stoica, and Eric Xing. 2023. Vicuna: An open- source chatbot impressing GPT-4 with 90%* Chat- GPT quality.
Alex Mallen, Akari Asai, Victor Zhong, Rajarshi Das, Hannanch Hajishirzi. and Daniel Khashabi. 2023. When not trust language models: Investigating effectiveness and limitations of parametric and non- parametric memories. In Association for Computa- tional Linguistics (ACL)).
Damai Dai, Li Dong, Yaru Hao, Zhifang Sui, Baobao Chang, and Furu Wei. 2022a. Knowledge neurons in pretrained transformers. In Association for Computa- tional Linguistics (ACL).
Damai Dai, Li Dong, Yaru Hao, Zhifang Sui, Baobao Chang, and Furu Wei. 2022b. Knowledge neurons in pretrained transformers. In Association for Computa- tional Linguistics (ACL).
Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. 2022a. Locating and editing factual asso- ciations in GPT. In Advances in Neural Information Processing Systems (NeurlPS).
Nicola De Cao, Wilker Aziz, and Ivan Titov. 2021. Editing factual knowledge in language models. In Empirical Methods Natural Language Processing (EMNLP).
Kevin Meng, Arnab Sen Sharma, Alex Andomian, Yonatan Belinkov, and David Bau. 2022b. Mass- editing memory in a transformer. International Conference on Learning Representations (ICLR).

---

Eric Mitchell, Charles Lin, Antoine Bosselut, Chelsea Finn, and Christopher D Manning. 2022a. Fast model editing scale. In International Conference on Learning Representations (ICLR).
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Fric Hambro, Faisal Azhar, et al. 2023. LLaMA: Open and ef- ficient foundation language models. arXiv preprint arXiv:2302.1397
Eric Mitchell, Charles Lin, Antoine Bosselut, Christo- pher D Manning. and Chelsea Finn. 2022b. Memory- based model editing scale. In International Con- ference on Machine Learning (ICML).
Denny Vrandečić and Markus Krötzsch. 2014. Wiki- data: free collaborative knowledgebase. Communi- cations of the ACM, 57(10)778-85
Eric Mitchell, Joseph J. Noh, Siyan Li, William S. Arm- strong, Ananth Agarwal, Patrick Liu, Chelsea Finn, and Christopher D. Manning. 2022c. Enhancing self- consistency and performance of pretrained language models with NLL. In Empirical Methods in Natural Language Processing (EMNLP).
Ben Wang and Aran Komatsuzaki. 2021. GPT-J- 6B: A 6 Billion Parameter Autoregressive Lan- guage Model. https://gi http:/tttt....iinggooo/oooe com/kingofllllz/ mesh- trrt-ttrrasffrrjaaat
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Ed Chi, Quoe Le, and Denny Zhou. 2022. Chain of thought prompting elicits reasoning in large language models. In Advances in Neural Information Processing Systems (NeurlPS).
Yasumasa Onoe, Michael IQ Zhang, Shankar Padman- abhan, Greg Durrett, and Eunsol Choi. 2023. Can Ims learn new entities from descriptions? Challenges in propagating injected knowledge. In Association for Computational Linguistics (ACL).
Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du Izhak Shafran, Karthik Narasimhan, and Yuan Cao. 2023. ReAct: Synergizing reasoning and acting in language models. In International Conference on Learning Representations (ICLR).
Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Car- roll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray. John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welin- der, Paul Francis Christiano, Jan Leike, and Ryan J. Lowe. 2022. Training language models to follow instructions with human feedback. In Advances in Neural Information Processing Systems (NeurIPS).
Zexuan Zhong. Dan Friedman, and Danqi Chen. 2021, Factual probing is [MASK]: Learning vs. learning to recall. In North American Chapter of the Association for Computational Linguistics (NAACL).
Denny Zhou, Nathanael Schärli, Le Hou, Jason Wei, Nathan Scales, Xuezhi Wang. Dale Schuurmans, Olivier Bousquet, Quoc Le, and Ed Chi. 2023a. Least-to-most prompting enables complex reasoning in large language models. In International Confer- ence Learning Representations (ICLR).
Fabio Petroni, Tim Rocktäschel, Patrick Lewis, An- ton Bakhtin, Yuxiang Wu, Alexander H Miller, and Sehastian Riedel. 2019. Language models knowl- edge bases? In Empirical Methods in Natural Lan- guage Processing (EMNLP).
Ofir Press, Muru Zhang, Sewon Min, Ludwig Schmidt, Noah A Smith, and Mike Lewis. 2022. Measuring and narrowing the compositionality gap in language models. arXiv preprini arXiv: 10.00330
Wenxuan Zhou, Sheng Zhang, Hoifung Poon, and Muhao Chen. 2023b. Context-faithfin prompt- ing for large language models. arXiv preprint arXiv:2303.1133 1131.
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei. Ilya Sutskever, et al. 2019. Language models are unsupervised multitask learners. OpenAl blog.
Chen Zhu, Ankit Singh Rawat, Manzil Zaheer, Srinadh Bhojanapalli, Daliang Li, Felix Yu, and Sanjiv Kumar. 2021. Modifying memories in transformer models. In International Conference on Machine Learning (ICML).
Taylor Shin, Yasaman Razeghi, Robert I. Logan IV, Eric Wallace, and Sameer Singh. 2020. Autoprompt: Eliciting knowledge from language models with auto- matically generated prompts. In Empirical Methods in Natural Language Processing (EMNLP), pages 4222-4235.
Anton Sinitsin, Vsevolod Plokhotnyuk, Dmitry Pyrkin, Sergei Popov, and Artem Babenko. 2020. Editable neural networks. In International Conference on Machine Learning (ICML).
Matthew Sotoudeh and Aditya V Thakur. 2019. Cor- recting deep neural networks with small, generalizing patches. In Workshop on Safety and Robustness in Decision Making.

---

A Details of Dataset Construction
these generated questions are of sufficient qual- ity for assessing the efficacy of knowledge-editing methods.
A.1 Sampling Fact Chains from Wikidata
We collect chains of facts that contain N {2, 3, 4) triples from Wikidata. We adopt heuristic rules to ensure that the sampled fact triples are coherent and lead to natural questions. Specifically, we ap- ply the following constraints when sampling fact chains from Wikidata. (1) The sampled chain does not involve a circle; (2) The sampled chain does not contain two triples that share the same relation type; (3) The triples with the object being country can only appear in the last two hops of the chain; (4) The sampled chain contains up three object types: (5) All triples with a person or location ob- ject are consecutive in the chain; (6) The subject entity associated with the relation headquarters location (P159) must be company or an organi- zation; (7) In all triples with the relation cupital (P36), the subject has to be a country. To use these heuristic rules, manually label the object types for each relation we consider. For example, the relation head of state (P35) corresponds person as the object.
B Evaluation Metrics
We evaluate the editing results based on three evalu- ation metries: edit-wise success rate, instance-wise accuracy, and multi-hop accuracy. Suppose we have edited a language model and obtain the edited model f (.).
Edit-wise success rate measures how many edited facts can be recalled by the edited language model. Given edit (s, T, o²), the editing success is defined as [[ (t) (t)) = o*]. We take the averaged value of the all edits the dataset and refer as the edit-wise success rate metric.
Instance-wise accuracy measures how many in- stances are there where all the associated facts can be recalled by the language model (either the original edited one). Given an instance d (E, a, a² ,C, , the instance-wise accuracy before editing is defined as
A ((((())) = (s.mo)eC
A.2 Filtering Unrecallable Facts with GPT-J
We filter out any chain of facts which contain at least one fact that cannot be recalled by GPT-J. For each relation type, we manually define question template as well as 8 demonstration examples for in-context-learning. We use in-context-learning to ensure the model can capture the answer format from the context. Table 6 shows example of the prompt to recall facts of the relation devel- oper (P178), We include the question templates of all relations on MQUAKE in Appendix
and the instance-wise accuracy post editing is de- fined as
1
A (F*(4())) = =o] (s.no)ecc
We report the averaged instance-wise accuracy our evaluation.
Multi-hop accuracy measures the accuracy on multi-hop questions. We regard an instance be- ing predicted correctly if any of the multi-hop questions are answered correctly by the language model. Given an instance d (E, Q, a, a", CC,CC)) the multi-hop accuracy before editing is defined as
A.3 Generating Questions using ChatGPT
Given chain of facts, we prompt ChatGPT Egpt-3.5-turbo to automatically generate multi- hop questions. The prompt used is shown in Table 7.
I VIr( = a1
In Table 8, we show some randomly selected examples of the questions generated by ChatGPT on MQUAKE-CF. We select 3 instances from 2,3, 4-h questions, each of which contains three generated questions. As shown, ChatGPT success- fully transforms the chain of triples into grammati- cally correct questions. Although these multi-hop questions are synthetic, they are logically consis- tent with the flow of the triple chains. We believe
and the multi-hop accuracy post editing is defined
as
[ - =a1
We report the averaged multi-hop accuracy in our evaluation.

---

(In-context-leamtn exarples)
Q: is the developer Telegram? A: Telegram FZ-LLC
Q: Who is the developer Microsoft Windows? A: Microsoft
Q: Who is the developer of PlayStation 2? A: Sony Interactive Entertainment
Q: Who is the developer iTunes? A: Apple Inc.
Q: Who is the developer of SR-71 Blackbird? A: Kelly Johnson
Q: Who is the developer of Moblin? A: Linux Foundation
Q: Who is the developer of Xbox 360? A: Microsoft
Q: Who is the developer of Kinsey scale? A: Alfred Kinsey
(Query during inference)
Q: Who is the developer of Steam0S? A: Valve Corporation
Table An example of the prompt used recall single-hop factf with relation developer (P178) from language models. We use in-context-learning with demonstration examples to ensure the model can output the answer in a desirable format.
System:
You are powerful multi-hop question generator. Users will provide chain of Wikidata triples, and you will help write questions ask the tail entity from the head entity. You shouldn't include bridge entities in generated questions. The questions should only include the head entity.
[12 in-context demonstrations abbreviated]
User:
Given Wikidata triples (Daredevil/Bullseye: The Target, author, x1), (x1, country of citizenship, x2), (x2, continent, x3), write question ask x3. Don't mention x1, x2, ... Write three possible questions natural English.
System:
What continent is the country of citizenshipp of the author Daredevi11/uullseye The Target
located in?
From which continent does the author of Daredevi1/Bullseye: The Target's country of citizenship belong?
3. what continent is the author's country of citizenship, who wrote Daredevil/Bullseye: The Target, situated in?
Table 7: An example of using ChatGPT 30t-3.b-turbo to generate questions from Wikidata triples. We manually
write 12 demonstrations as the prompt when querying ChatGPT.
C Implementation Details for Knowledge Editing Methods
C,3
ROME
For GPT-J, we use the default hyperparameters of ROME and the pre-computed covariance statistics released by Meng et al. (2022a). For Vicuna-7B, run ROME to update model weights at layer with the default hyperparameters. We compute the covariance statistics for Vicuna-7B on Wikitext using sample size of 100,000.
C.1 Fine-tuning
Our fine-tuning bascline (FT) performs gradient descent on the edits to update model parameters. We fine-tune layer 21 of GPT-J and layer 31 of Vicuna-7B. We follow Zhu et al. (2021) and use constraint weight changes with coeffi- cient 5 10 in our implementation.
C.4 MEMIT
C.2 MEND
For GPT-J, we use the default hyperparameters of MEMIT and the pre-computed covariance statistics released by Meng al. (2022b), For Vicuana-7B. update model weights at layers (5, 6, 7, 8, with the default hyperparameters. Similarly, we compute the covariance statistics for Vicuna-7B on Wikitext using a sample size of 100,000.
We the GPT-J MEND editor trained by Meng al. (2022a). For Vicuna-7B, we train our MEND editor model on the Wikitext generation editing dataset (Mitchell al., 2022a) with the different hyperparameters. During inference, we set the learning rate scale to be 1.0.

---

Examples 2-hop questions
C Uaeques Necker, employer. University of Geneva) (University of Geneva, headquarters location, Geneva)
What is the location of the headquarters of the employer of Jacques Necker?
Where is the employer of Jacques Necker headquartered?
In which city is the hcad office located for the company that employed Jacques Necker?
C (Percival Lowell, educated at. Harvard University) (Harvard University, headquarters location, Cambridge)
Q What is the location of the headquarters of the institution where Percival Lowell was educated?
In which city is the institution located where Percival Lowell received his education?
Where is the headquarters of the educational institution attended by Percival Lowell located?
C (Gordon Moore, country of citizenship. United States of America) (United States of America, capital, Washington, D.C.)
Q What is the capital of the country where Gordon Moore holds citizenship?
Which is the capital city of the country to which Gordon Moore belongs? In which city is the seat of the govermment of the country where Gordon Moore is citizen?
Examples of h-hop questions
C (Kim Kardashian, spouse, Kanye West) (Kanye West, genre, hip hop music)
(hip hop music, country of origin, United States of America)
Q What the country of origin of the genre associated with the spouse Kim Kardashian?
From which country does the genre of the partner of Kim Kardashian hail? Which country is the genre of the partner of Kim Kardashian associated with originally from?
C (Nicholas of Tolentino, religion or worldview, Catholic Church) (Catholic Church, founded by, Jesus Christ) (Jesus Christ, place of hirth, Bethlchem)
Q Where was the founder of Nicholas Tolentino's 's religion born?
In which city was founder of religion that Nicholas Tolentino adhered born? What is the birthplace of the founder of the religion that Nicholas of Tolentino followed?
C (Boston, head of government, Marty Walsh) (Marty Walsh, educated Boston College)
(Boston College. headquarters location. Chestnut Hill)
Q In what city the headquarters of the institution where the head of government of Boston educated located?
Where is the location of the headquarters of the educational institution where the head of government of Boston received their education?
What the city where the headquarters of the institution where the head of govermmet of Boston was educated at located?
Examples of 4-hop questions
C (Xbox Live, developer, Microsoft) (Microsoft, chief executive officer, Satya Nadella)
(Satya Nadella, place of birth, Hyderabad) (Ilyderabad. continent, Asia) Which continent is home to the birthplace of the CEO of Xbox Live developer?
2
Where was the CEO of the developer of Xbox Live born in which continent? In what continent was the CEO of Xbox Live's developer bom??
(Winnie the Pooh, creator, A. A. Milne) (A. A. Milne, child, Christopher Robin Milne) (Christopher Robin Milne, country of citizenship. United Kingdom) (United Kingdom, official language, English) Q What is the official language of the country where the child of Winnie the Pooh's creator holds citizenship? Which language is officially spoken in the country where the child of the creator of Winnie the Pooh is a citizen? What is the officiated language of the country where the child of Winnic the Pooh's creator is a citizen of?
C
C (watchOS, developer, Apple Inc.) (Apple Inc., chicf executive officer, Tim Cook)
(Tim Cook, country of citizenship. United States of America) (United States of America, capital, Washington, D.C.)
Q What is the capital of the country where the CEO of the developer of watchOS holds citizenship? In which city does the CEO of the company that developed watchOS hold citizenship?
Which city is the capital of the home country of the CEO of the developer of watchOS?
Table 8: Qualitative examples of the generated multi-hop questions on MQUAKE-CF. Given a chain of facts, we query ChatGPT (gpt-3.5 5-turbo) to generate multi-hop questions with the prompt shown in Table 7.
D Chain-of-thought Prompting for Multi-hop Questions
E Extended Golden Labels for MQUAKE-T
Our MQUAKE-T contains limited test cases. To better assess the model's original performance on multi-hop questions, we extend the possible golden labels for each multi-hop question. Specifically, we allow outdated answers given smaller language
We use -lain-of-thought (CoT) prompting (Wei et al., 2022) to maximize model performance. Ta- ble 9 shows one simplified example of our prompt with COT.

---

Question: What is the capital of the country where Plainfield Town Hall is located? Thoughts: Plainfield Town Hall is located in the country of the Uni ted States of America. The capital of United States is Washington, D.C. Washington, D.C.
Answer:
Question: In which country is the company that created Nissan 200SX located? Thoughts: Nissan 280SX created by Nissan. Nissan is located in the country of Japan.
Answer: Japan
[3 in-context demonstrations abbreviated]
Question:
Who has ownership of the developer of the Chevrolet Corvette (C4)?
Thoughts: The developer Chevrolet Corvette (C4) Chevrolet. Chevrolet owned by General Motors.
Answer: Model Generated Answer Goes Here
Table 9: The template of the prompt we used for asking multi-hop questions using chain chain-of-thoughts.
Please answer the following question faithfully using the knowledge you have from Wikipedia. Provide possible answers the question, using all the Wikipedia data you know. Rank them from to the outdated.
Input: What is the name the head United States of America government?
Output:
Joe Biden
Donald Trump
Barack Obama
George W. Bush
Bill Clinton
George H. W. Bush
Ronald Reagan
Jimmy Carter
Gerald Ford
Richard Nixon
Input: What is the name of the current head of the New York City government?
Output: Model Generated Answer Goes Here
Table 10: The template of the prompt we used for extending golden labels for MQUAKE-T. The prompt contains one demonstration for better aligning model behaviors.
models tend to be less calibrated. To extend the golden labels, we use GPT-3 (text-davinci-on) to query outdated answers. See the prompt we used in Table 10.
G Breakdown Results on MQUAKE-CF
Table 12 and Table 13 present the breakdown re- sults MQUAKE-CF when using GPT-J the base model. We find that, in all editing methods (1) the performance 2-hop questions is much higher than 3-hop and 4-hop questions; (2) the performance is worse when there are more edits asssociated with the edited instances.
F Prompts used in MeLLo
The prompt we used in MeLLo shown in Ta- ble 11. We first prompt the language model to de- compose subquestions. Then the language model generates tentative question to the subquestion (marked in green text); then we use the generated subquestion retrieve the most relevant edited fact (marked in light blue text) and append to the prompt. The model self-checks if the retrieved fact contradicts the generated answer. The prompting procedure goes iteratively until the model generates the final answer.
H Impact of Retrieval Performance
In MeLLo, in order to answer multi-hop hopp questions correctly after editing, the retrieval model needs to retrieve all the associated edited facts (cach ques- tion is associated with 1-4 edited facts) from the memory. Here we investigate how retrieval accu- racy affects the performance MeLLo when using GPT-3 as the base model. We compute the retrieval accuracy (i.e., how many instances where the as-

---

[4 in-context demonstrations abbreviated]
Question: What is the capital city of the country of citizenship of Ivanka Trump's spouse? Subquestion: Who is Ivanka Trump's spouse? Generated
Ivanka Trump's spouse Jared Kushner.
Retrieved fact: David Cameron is married to Samantha Cameron.
Retrieved fact does not contradict to generated answer, the intermediate answer is: Jared Kushner
Subquestion: what is the country citizenship of Jared Kushner?
Generated answer: The country citizenship of Jared Kushner is United States. Retrieved fact: Jared Kushner is a citizen of Canada.
Retrieved fact contradicts
generated answer, so the intermediate answer Canada
Subquestion: what the capi tal city Canada?
Generated answer: The capital city Canada is Ottawa.
Retrieved fact: The capital city of United States is Seattle.
Retrieved fact does contradict generated answer, so the intermediate answer is: Ottawa Final Ottawa
step-by-step illustration of MeLLo solving one simplified example. Green parts are generated by the language model, and blue parts are facts retrieved by the retriever.
Table 11:
2-hop
3-hop
4-hop
All
CF with GPT-3 by considering different numbers of edited instamces at the same time. As Table shows, the performance of MeLLo decreases if the retrieval accuracy is lower (as result of consider- ing more instances at the same time). Among those questions where all associated facts are success- fully retrieved from memory. MeLLo can answer 73.1% of them correctly. This indicates that re- trieval performance can significantly impact the model performance. When we consider more ir- relevant knowledge edits in the memory, retrieval can be more challenging, and we expect that using more advanced retrieval techniques can improve the performance.
Base
47.5
27.1
45.3
42.1
FT
3.7
1.4
0.5
1.9
MEND
13.9
11.3
9.5
11.5
ROME
33.8
9.1
11.4
18.1
MEMIT
22.5
6.0
8.4
12.3
Table Breakdown multi-hop performance (CoT) on MQUAKE-CF for (2,3,4)-hoy questions. We use GPT- as the base model in this experiment
=dits=
2
4
All
Base
34.0
43.0
40.4
51.7
42.1
FT
4.2
0.7
0.3
0.0
1.9
MEND
16.0
11.0
7.3
4.4
11.5
Question/Cloze Statement Templates used in MQUAKE
ROME
23.8
20.9
9.0
2.6
18.1
MEMIT
20.5
9.8
5.5
2.6
12.3
Table 15 shows the question templates and the cloze-style statement templates we use in MQUAKE. We use the question templates to query single-hop facts when use GPT-J for filtering and use the cloze-style statement templates con- vert an edited fact to a natural language statement.
Table 13: Breakdown multi-hop performance (CoT) on MQUAKE-CF for questions with [1,2,3,4] edits. We use GPT-J as the base model in this experiment.
# Instances (k
1
100
1000
3000
Retrieval acc.
93.6
67.7
59.4
58.7
MeLLo (on GPT-3)
68.7
50.5
43.6
41.2
Table 14: How retrieval accuracy affects the multi-hop performance of MeLLo on MQUAKE-CF. We consider group of k instances at the same time. Retrieval acc.: how many instances where all the associated edited facts are correctly retrieved from the memory.
sociated edited facts are correctly retrieved from the memory) when applying MeLLo on MQUAKE.

---

Relation
Question template
Cloze-style statement template
P30
Which continent is [S] locured in?
locuted the continent of
What the capital [S}?
The capital of [S] is
What is the name the current
state (S)?
The name the current head of state in [S] is
What is the name of the current head the [S] govermmen??
The name the current head of the [S] government
P20
Which city did [S] die in?
[S] died in the city of
P26
Who is [S] married to?
[S] is married to
P140
Which religion is [S] affiliated with?
[S] is affiliated with the religion of
P1412
What language does [S] speak?
[S] speaks the language of
P19
Which city [S] born in?
[S] born the city of
Which university was [S] educated at?
The univeristy where [S] educated
Who is [S]'s child?
[S]'s child
What is the country citizenship [S]?
[S] is a citizen of
P175
Who performed [S]?
[S] was performed by
P108
Who is the employer of [S]?
[S] is employed by
P112
Who founded [S]?
[S] was founded by
P50
Who is the author [S]?
The author
P170 Who
created by?
[S] created by
P407
Which language was [S]
[S] written the language
is the official lunguage [S]?
The official language [S] is
P740 Where was [S] founded?
[S] was founded in the city
P495
Which country was [S] created in?
[S] was created in the country of
P106
What kind of work does [S] do?
[S] works in the field of
P136 What type of music does [S] play?
The type of music that [S] plays
P364
What the original language [S]?
The original languuge
P937
Which city did [S] work in?
[S] worked the city
P800 What is [S] famous for?
[S] famous
P641 Which sport is [S] associated with?
[S] associated with the sport
P413
What position does [S] play?
[S] plays the position of
P286
Who is the head coach of [S]?
The head coach of [S]
P159 W Which city is the headquarter of [S] located in?
The headquarters of [S] is located in the city of
P178 Who is the developer of [S]?
[S] was developed by
P488
Who is the chairperson of (
The chairperson of [S] is
P169
Who the chief executive officer [S]?
The chief executive officer
P449
Who is the original broadcaster [S]?
The origianl broadcuster of [S] is
P176 Which company is [S] produced by?
The company that produced [S] is
P1037
Who is the director of [S]?
The director of [S] is
P1308
Who is the [
The [S] is
Table 15: Question templates and cloze-style statement templates that are used in MQUAKE. "[S]" represents placeholder for the subject entity of the fact. use the question templates to query single-hop facts when we use GPT- for filtering and use the cloze-style statement templates convert edited fact to statement.


Reviews:
[{"review_id": "FUskVWoDYl", "reviewer": "Reviewer_uapW", "summary": "", "questions": "In general, I feel the authors make a good effort on making the paper easy to understand, however there are some questions I still have:\n\n- While the problem is interesting, the experiments are somewhat limited. I understand that the main focus of the paper is to address the idea of evaluating knowledge-edited LLMs on multi-hop question, however the experiments only address three model (GPT-J, Vicuna-7B and GPT-3) which are all very different in size and how they were pre-trained. Is there a reason why a more diverse set of models (size, data, inductive biases, etc) were not used, as the results would be slightly more solid if these were also provided?\n\n- Furthermore, Table 5 (and Table 7 in the appendix) appears to show comparisons of MEMIT and MeLLo only for GPT-J. I'm curious as to why Vicuna-7B and GPT-3 were included if performance with MEMIT (or other methods) are not provided as it is difficult to judge these rows. While I am already convinced by the performance of GPT-J, I think this part could be improved both for presentation purposes (elaborated upon further on) as well as for clarity.\n\n- I feel there are some ablation studies that are missing, namely seeing how performance scales with the number of hops for MQuAKE. While I understand creating very large $k$-hop questions are difficult, I feel that having hops up to the teens would be much more interesting. I also would be interested in seeing more model sizes as a way of seeing how knowledge-editing methods scale with size as well.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your thoughtful discussion. We were encouraged that you see our new benchmark MQuAKE as an effective, easy-to-use way to evaluate model editing techniques and that you find our proposed method MeLLO to be simple and effective. Here we address your detailed comments, which are helping us to revise the paper and chart out future directions.\n\n**“MeLLo might not scale as well as other methods that are benchmarked against”**\n\nScaling is certainly a vital issue in this area, and we see MeLLo as very competitive. The current state of the art is that most published model-editing methods degrade badly after over 100 edits, with only MEMIT (Meng et al. 2022) showing good performance out to 10,000 edits (and unfortunately they show no data beyond this point). While the number of “facts” in a language model is not clearly defined and unclear, it seems reasonable to assume that this is probably still less than 0.1% of the facts in even a 6B parameter LM.  Our experiments have shown that MeLLo is much more effective compared to all existing methods when editing thousands of facts at the same time (Table 5). Editing even more facts (e.g., 1M facts) is an open problem in this field, which we hope to explore in the future. \n\n**“MQuAKE is limited to only 2-4 hop questions”**\n\nWe acknowledge this limitation, while emphasizing that the vast majority of the literature on multi-hop QA, such as work using the HotpotQA dataset, only considers 2 hops. We note that our goal is to assess whether edited models can answer questions where the answer should change as an entailment of edited facts. Our dataset consisting of 2-4 hop questions have shown the failures of existing baselines. The MQuAKE procedure extends relatively easily to additional hops, and it would be excellent to see the field progress to the point where 4 hops isn’t the serious challenge as it is now.\n\n**“Is there a reason why a more diverse set of models (size, data, inductive biases, etc) were not used?”**\n\nDue to limited compute resources and API budgets, we could not conduct experiments on all models. We would like to argue that we *did* choose a diverse set of base models, while nevertheless acknowledging that having results on even more models would be better. \n\n* We use GPT-J, a common LM that prior works (e.g., Meng et al., 2022) have evaluated on, as the base model in our main experiments. \n\n* We additionally consider a more recent and stronger model with a similar size, Vicuna-7B, in our experiments. Vicuna-7B is built based on the state-of-the-art (at submission time!) open-source language model LLaMA, and is representative of the many recent RLHF instruction-tuned models. \n\n* We further perform MeLLo on GPT-3 (text-davinci-003) to showcase that our approach is effective at editing large black-box language models, while previous approaches fail to do so. \n\n**“How performance scales with the number of hops for MQuAKE”**\n\nThanks for the suggestion! We include breakdown results of evaluating ROME on MQuAKE-CF in the two tables below. We find that (1) the performance on 2-hop questions is much higher than 3-hop and 4-hop questions; (2) the performance gets worse when there are more edits associated with the test instance. We will include detailed breakdown results for other models in our revision.\n\n| 2-hop questions | 3-hop questions | 4-hop questions |  All |\n|:---------------:|:---------------:|:---------------:|:----:|\n|       33.8      |       9.1       |       11.4      | 18.1 |\n\n| Questions with 1 edited fact | Questions with 2 edited facts | Questions with 3 edited facts | Questions with 4 edited facts |  All |\n|:----------------------------:|:-----------------------------:|:-----------------------------:|:-----------------------------:|:----:|\n|             23.8             |              20.9             |              9.0              |              2.6              | 18.1 |\n\n**“On table 5, why Vicuna-7B and GPT-3 were included if performance with MEMIT (or other methods) are not provided as it is difficult to judge these rows”**\n\nTable 5 aims to show (1) MeLLo is more effective than all the baselines with the same base model, and (2) MeLLo performs better when using a stronger LLM (e.g., GPT-3). We include the performance of MEMIT for the purpose (1) and the full performance of MEMIT (and other models) is shown in Table 3 and Figure 2. We acknowledge that this may cause confusion for readers. We will present these results clearly in the revision.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- The problem this paper aims to address is very interesting and relevant, as current LLMs are very costly to train. As a result, knowledge editing is often considered a useful alternative to trying to keep models up to date.\n- MeLLo is simple to understand and from my perspective a relatively effective way to ensure up-to-date retrieval of parametric knowledge from LLMs.", "reasons_to_reject": "While I am pleased with the paper and see no major flaws, I'll list out a couple of minor issues I would only perhaps use if I had to be very nit-picky on the margins given a host of other papers I deem of similar quality.\n\n- MeLLo requires storing information externally, therefore it would appear to me that it is only useful when a limited amount of facts need to be edited. In the event that perhaps a large portion of the model's knowledge needs to be edited, it would appear to me that MeLLo might not scale as well as other methods that are benchmarked against.\n- MQuAKE is limited to only 2-4 hop questions, which would appear somewhat limited given the size of the models generally used nowadays.", "questions_to_authors": "In general, I feel the authors make a good effort on making the paper easy to understand, however there are some questions I still have:\n\n- While the problem is interesting, the experiments are somewhat limited. I understand that the main focus of the paper is to address the idea of evaluating knowledge-edited LLMs on multi-hop question, however the experiments only address three model (GPT-J, Vicuna-7B and GPT-3) which are all very different in size and how they were pre-trained. Is there a reason why a more diverse set of models (size, data, inductive biases, etc) were not used, as the results would be slightly more solid if these were also provided?\n\n- Furthermore, Table 5 (and Table 7 in the appendix) appears to show comparisons of MEMIT and MeLLo only for GPT-J. I'm curious as to why Vicuna-7B and GPT-3 were included if performance with MEMIT (or other methods) are not provided as it is difficult to judge these rows. While I am already convinced by the performance of GPT-J, I think this part could be improved both for presentation purposes (elaborated upon further on) as well as for clarity.\n\n- I feel there are some ablation studies that are missing, namely seeing how performance scales with the number of hops for MQuAKE. While I understand creating very large $k$-hop questions are difficult, I feel that having hops up to the teens would be much more interesting. I also would be interested in seeing more model sizes as a way of seeing how knowledge-editing methods scale with size as well.", "paper_topic_and_main_contributions": "This work aims to find additional ways to evaluate knowledge-edited LLMs, in particular by evaluating how the edited facts affect multi-hop questions due to a ripple-like effect this would have on the paths the language model would have to take in order to retrieve/generate the correct response after the edit. The main contribution is the benchmark they provide, MQuAKE, as well as a novel method for knowledge editing, MeLLo. The authors demonstrate the difficulty models have at solving MQuAKE, suggesting that prior methods for knowledge editing are limited in how they can enable language models to recall facts in the newly edited world.", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Missing_References": "Given that the majority of my experience with multi-hop QA comes from more of an IR perspective, a lot of the references I have in mind don't seem to be applicable here."}, "extra_text": {"Typos_Grammar_Style_and_Presentation_Improvements": "Orthography and vocabulary seem to have no visible issues from my perspective. \n\nIn terms of presentation, my suggestions are quite minor. For example, I do believe that the swigly underlines in Table 1 could be instead changed with some form of highlighting (for example on the edited entity) since it could be more visible apparent what you're attempting to express."}}, {"review_id": "zXeAOjYkER", "reviewer": "Reviewer_ojrC", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate your valuable feedback! We are encouraged that you find that our paper is well-motivated, our datasets MQuAKE and the construction process may support future research, and our experiments include a variety of baselines and base LMs. We have addressed your detailed comments below. Additionally, we will incorporate all the new results in the response into the revised version.\n\n**“The dataset fails to also evaluate the specificity/locality of edited facts and whether edited facts are having unintended effects”**\n\nExcellent point. MQuAKE (like all benchmarks) should be used in conjunction with other evaluation tools and resources. Our paper particularly focuses on assessing whether edited models can answer questions where the answer should change as an entailed consequence, showing that existing approaches fail on those questions. We agree that evaluating specificity is essential for knowledge editing methods and we plan to add it in our revision. We’d also like to point out that there are already results for the specificity of all the baseline methods (but not MeLLo) in previous studies (e.g., Meng et al., 2022).\n\n**“It's unclear where improvements in the proposed method are coming from”**\n\nGood point! To address this, we ran new experiments testing the baselines by using the same QA-style prompt without retrieval and self-checks. This experiment is conducted on GPT-J by considering 3000 instances from MQuAKE-CF or 1868 instances from MQuAKE-T at the same time. As the table below shows, we do not observe substantial improvements on baselines by simply changing the prompts. It is likely because of the limited capacity of performing QA-style CoT for those small-sized models (i.e., GPT-J). This suggests that the improvements mainly come from the editing methods instead of using QA-style prompts. \n\n|            | MQuAKE-CF (3000 instances) |                | MQuAKE-T (1868 instances) |                |\n|------------|----------------------------|----------------|---------------------------|----------------|\n| **Method** | CoT (standard)             | CoT (QA-style) | CoT (standard)            | CoT (QA-style) |\n| MEMIT      | 1.8                        | 2.1            | 0.0                       | 0.0            |\n| MEND       | 3.5                        | 3.5            | 4.6                       | 5.7            |\n| MeLLo      | -                          | 9.8            | -                         | 30.7           |\n\nWe note that it is possible that performance gains of using QA-style are larger when using a stronger base model (e.g., GPT-3). However, existing methods are not applicable to large-scale or black-box models. We leave further investigation on large models as future work.\n\n**“Differences between MeLLo and SERAC?”**\n\nInstead of training a separate classifier model and a counterfactual model, MeLLo directly uses the LLM to self-check whether the retrieved fact contradicts the model knowledge or not. MeLLo can be easily applied to black-box LMs without any extra training (as shown in the experiments on GPT-3), which is an advantage over existing editing methods including SERAC. With the same base model (e,g, GPT-3), the point-to-point performance comparison between MeLLo and SERAC will highly depend on the quality of the classifier model and the counterfactual model that SERAC trained, which we leave as future work.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The paper is well-motivated, highlighting a overlooked issue knowledge editing work.\n\nThe dataset and its construction process may support future work in this area.\n\nThe paper is well-written and includes a variety of knowledge-editing baselines and base LMs in experiments.", "reasons_to_reject": "While this dataset provides a method for evaluating whether facts are successfully edited in multi-hop questions, it fails to also evaluate the specificity/locality of edited facts and whether edited facts are having unintended effects (e.g., catastrophic forgetting) elsewhere. Evaluating the specificity of edits is a consistent concern in other works proposing datasets for knowledge editing. For instance, this dataset does not consider questions such as whether editing (WALL-E, creator, Andrew Stanton → James Watt) affects other facts, for instance (Finding Nemo, creator, Andrew Stanton → James Watt).\n\nThe baselines and comparisons for the proposed method in Section 5 make it unclear where improvements in the proposed method are coming from. In particular, some of the cited work has demonstrated that, for multi-hop QA, CoT prompting  by decomposing inputs into QA pairs outperforms standard paragraph-style CoT prompting. Baselines where edited models receive the same prompt, with the retrieval and counterfactual reasoning steps removed, would've been well-placed here.\n\nAdding onto the point above, some of the cited work (SERAC) has proposed a method that similarly (1) performs retrieval over edited facts, (2) determines whether the edited facts should affect the answer to the given question and (3) use the edited fact to produce the correct output. The primary difference here is the usage of a single LM to do all these inferences jointly rather than training separate components. Comparisons between these two methods of performing steps 1-3 would also add context for the proposed method.\n\n", "questions_to_authors": "", "paper_topic_and_main_contributions": "This paper presents a dataset for evaluating knowledge editing in LMs, focusing on evaluating multi-hop questions whose answers should change as the result of and edit to the answer to one of its sub questions. The authors construct two versions of this dataset from WikiData, where multi-hop questions are constructed from knowledge-base tuples using ChatGPT. Their first dataset is larger, consisting of chains of 37 different relation types up to a length of 4 connecting the 20% of most popular entities in WikiData. Relations are further filtered based on whether GPT-J is able to recall each individual fact. To construct edits, the authors replace the object of a subject-relation-object tuple with another object sharing the same relation. The authors also construct a smaller dataset containing real factual edits that have occurred over time (2021 to 2023), focusing on changes in 6 different relation types.\n\nThe authors experiment with editing in a single or multiple (up to 3K) facts into a single LM and experiment with several baselines. They find that existing methods, while they're able to recall individual edited facts, they are unable to successfully use these updated facts to answer multi-hop questions using them, failing catastrophically. When allowed to perform chain-of-thought reasoning for multi-hop questions, however, all methods are able improve and recover much of their base performance. This observation holds true for both versions of their dataset.\n\nThe authors finally propose their own method for knowledge editing, that is based around (1) decomposing multi-hop questions into their subquestions and (2) performing retrieval for each subquestion from a non-parametric corpus of edited facts. The LM then makes a decision whether the answer should be affected by this retrieved fact, then updates its answer based on that decision. The authors demonstrate that this pipeline outperforms ", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "4rGDvVPEfh", "reviewer": "Reviewer_i7oa", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 5, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for the valuable comments! We are glad that you find that our paper writing is very clear and the motivation is strong, our proposed datasets MQuAKE and evaluation metrics provide a new perspective for understanding the injected knowledge, and our proposed method MeLLo is effective and is applicable to models as large as 175B. We have addressed your detailed comments below, and we are more than willing to engage in further discussion should any follow-up questions arise!\n\n**“The dataset proposed in this paper is limited to GPT-J”**\n\nWe acknowledge that this method of data construction does give a bias towards GPT-J. We argue that (1) the filtering process is very important, as it increases the probability of the multi-hop questions being meaningful and answerable by language models (otherwise, the accuracy will be too low, which is hard to draw conclusions); (2) we chose GPT-J as the model for filtering because it is fairly small and uses similar data to most other general-purpose model (so it’s a good approximate filter for other models), and GPT-J is a common base model that is studied in prior knowledge editing works.\n\nWe note that MQuAKE *can* also be applied to other models -- one can compare before-editing performance and post-editing performance. In addition, the proposed data construction pipeline can be easily applied with other models as the filter.\n\n\n**“It would be better to use a probability-based approach like CounterFact (ROME) to assess it”**\n\nWe chose to use accuracy instead of probability-based metric because accuracy aligns better with the end use of the language models (e.g., when we ask a question to LMs, we expect them to return the answers instead of only assigning higher probability to the answers). We also acknowledge that there may be multiple forms of answers to some questions -- we leverage aliases from Wikidata to mitigate this issue.\n\n**“The accuracy and recall rate of the retriever will have a significant impact on the language model's output”**\n\nThanks for the great question, we think this is an excellent point to highlight! We have conducted new analyses to answer the reviewer's question. \n\nYes, the performance of MeLLo is significantly impacted by the retrieval performance. In order to answer the multi-hop questions after editing, the retriever needs to retrieve **all** the associated edited facts (each question is associated with 1-4 edited facts) from memory. \n\nTo investigate the effect of retrieval performance, we compute the retrieval accuracy (i.e., how many edited facts are correctly retrieved from the memory) when applying MeLLo on MQuAKE-CF with GPT-3 by considering different numbers of edited instances at the same time. As the table below shows, the performance of MeLLo decreases if the retrieval accuracy is lower (as a result of considering more instances at the same time). Among those questions where all associated facts are successfully retrieved from memory, MeLLo can answer 73.1% of them correctly. These empirical results verify that retrieval performance can significantly impact the model performance. When we consider even more irrelevant knowledge in the memory, retrieval might be more challenging, and we expect using more advanced retrieval techniques can improve the performance. We will include this analysis in the revised paper.\n\n| # Edited instances | 1    | 100  | 1000 | 3000 |\n|--------------------|------|------|------|------|\n| Retrieval accuracy | 93.6 | 67.7 | 59.4 | 58.7 |\n| MeLLo (on GPT-3)   | 68.7 | 50.5 | 43.6 | 41.2 |", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The writing in is very clear and the motivation is strong.\n2. Previous knowledge editing primarily focused on whether the model can recall the injected new knowledge and its paraphrases. However, this paper proposes chains-of-facts, which provides a new perspective for exploring whether the model truly learns the injected knowledge.\n3. This paper designs various evaluation metrics for chains-of-thoughts and also considers different prompting methods such as in-context learning and chain-of-thoughts.\n4. To address the identified issues, this paper proposes a memory-based approach that effectively applies to chains-of-facts and can be applicable to models as large as 175B or even larger.", "reasons_to_reject": "Overall, this paper does not have any glaring flaws that would lead to rejection. However, there are areas where the design of the dataset and experiments could be improved:\n\n1. The world knowledge of different models varies, and although the authors consider GPT-J, Vicuna, and GPT-3, the dataset proposed in this paper is still limited to GPT-J. On one hand, the authors filter out facts that cannot be recalled by GPT-J. On the other hand, the construction of Mquake-t's temporal segments is also based on GPT-J's design. This limits the applicability of the dataset to a very narrow range of language models.\n\n2. The paper primarily uses accuracy as an evaluation metric, but because objects can have aliases, it would be better to use a probability-based approach like CounterFact (ROME) to assess it.\n\n3. The proposed method, Mello, employs a memory-based approach, and the accuracy and recall rate of the retriever will have a significant impact on the language model's output. The current dataset setting may be too simplistic, allowing the retriever to effectively retrieve relevant factual edits. It is important to investigate the potential impact on the final output if the retriever retrieves irrelevant knowledge. I am curious about the robustness of Mello in handling retriever errors in recalling facts.", "questions_to_authors": "", "paper_topic_and_main_contributions": "This paper explores consistency in knowledge editing. It introduces the concept of \"chains-of-facts\" to investigate the impact of a particular piece of knowledge on the entire chain of edits. Specifically, two datasets, mquake-t and mquake-cf, are constructed for GPT-J. By formulating multi-hop questions related to chains-of-facts, the paper aims to determine whether the model has truly learned new knowledge or simply hard-coded it. The article presents a benchmark that challenges traditional knowledge editing methods and also proposes a memory-based approach called mello that effectively addresses the editing problem with chains-of-facts.", "excitement": 4, "reproducibility": 5, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "ENZvkai9Sk", "reviewer": "Reviewer_ipS4", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 3, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your valuable feedback! We are encouraged that you find our paper has a clear message, proposes a new benchmark, and proposes a new approach MeLLo without changing the target model weights!\n\n**“Prompt templates for the main experiments mentioned in lines 387-389 were not provided in the paper”**\n\nThanks for pointing this out! We will include templates we defined for each relation as well as the in-context demonstrations we used to query multi-hop questions in the appendix in the revision. Here we provide a few prompt templates as examples.\n\n| Relation                     | Prompt template                            |\n|------------------------------|--------------------------------------------|\n| developer (P178)             | Who is the developer of [X]?               |\n| country of citizenship (P27) | What is the country of citizenship of [X]? |\n| educated at (P69)            | Which university was [X] educated at?      |\n| owner (P127)                 | Who owns [X]?                              |\n| … (33 more relations)        | …                                          |", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The paper has a clear message and proposes a new benchmark for the task that is of interest to the community and also proposes a new lightweight baseline approach for knowledge editing of LLMs. The dataset construction and experiments are well-described. Limitations show a good cover of possible future work directions.", "reasons_to_reject": "Prompt templates for the main experiments mentioned in lines 387-389 were not provided in the paper.", "questions_to_authors": "", "paper_topic_and_main_contributions": "The paper addresses the problem of knowledge editing in large language models without changing the LLM weights. Authors present a benchmark MQuAKE to assess the ability of the edited model to answer multi-hop questions with the correct answer depending on the edited information. Also, the authors propose an external memory-based method MeLLo for knowledge-editing without changing the target model weights.", "excitement": 4, "reproducibility": 5, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: All the reviewers rated the paper high on soundness as well as excitement. The paper was commended for well-thought motivation and being clearly written. All reviewers highlighted the potential impact of MQuaLE, a dataset of multihop questions collected by the authors to test the capabilities of knowledge editing techniques. 

Reviewers acknowledged the significance of the findings reported in the paper. For instance, the experiments in the paper demonstrate that existing methods of knowledge editing might be able to recall individual edited facts but are unable to successfully use these updated facts to answer multi-hop questions using them. The paper also proposes MeLLo, a new memory-based approach to store all the facts externally, that outperforms past knowledge editing methods on MQuaLE.

There were a couple of concerns raised by the reviewers. For instance, the setup might be too simplistic and amenable to the retriever used in MeLLO. Another concern was the absence of prompt templates for the main experiments. Authors should consider addressing such questions in future versions of the paper.
Decision: Accept-Main