Uncovering Overfitting in Large Language Model Editing
Authors: Mengqi Zhang
Xiaotian Ye
Qiang Liu
Shu Wu
Pengjie Ren
Zhumin Chen
Venue: ICLR 2025 Spotlight (2025)
Keywords: Large language models
Knowledge editing
Editing overfit
Primary area: foundation or frontier models, including LLMs
TL;DR: We explore the overfitting problem in knowledge editing and develop a new benchmark along with a novel overfitting mitigation strategy.

Abstract: Knowledge editing has been proposed as an effective method for updating and correcting the internal knowledge of Large Language Models (LLMs). However, existing editing methods often struggle with complex tasks, such as multi-hop reasoning. In this paper, we identify and investigate the phenomenon of Editing Overfit, where edited models assign disproportionately high probabilities to the edit target, hindering the generalization of new knowledge in complex scenarios. We attribute this issue to the current editing paradigm, which places excessive emphasis on the direct correspondence between the input prompt and the edit target for each edit sample. To further explore this issue, we introduce a new benchmark, EVOKE (EValuation of Editing Overfit in Knowledge Editing), along with fine-grained evaluation metrics. Through comprehensive experiments and analysis, we demonstrate that Editing Overfit is prevalent in current editing methods and that common overfitting mitigation strategies are ineffective in knowledge editing. To overcome this, inspired by LLMs’ knowledge recall mechanisms, we propose a new plug-and-play strategy called Learn the Inference (LTI), which introduce a Multi-stage Inference Constraint module to guide the edited models in recalling new knowledge similarly to how unedited LLMs leverage knowledge through in-context learning. Extensive experimental results across a wide range of tasks validate the effectiveness of LTI in mitigating Editing Overfit.

Published as a conference paper at ICLR 2025
UNCOVERING OVERFITTING IN LARGE LANGUAGE MODEL EDITING
Mengqi Zhang'; Xiaotian Ye Qiang Liu', Shu Wu Pengjie Ren't, Zhumin Chen'
Shandong University
School of Computer Science, Beijing University of Posts and Telecommunications
? NNew Laboratory of Pattern Recognition (NLPR)
State Key Laboratory of Multimodal Artificial Intelligence Systems (MAIS) Institute Automation, Chinese Academy of Sciences
{menggi. zhang, zenpengjin jie, chenzhumin)@sdie edu. cn yexiaot ianbbupt. edu. cn, {qiang. liu, shu shu. wu) pr ia ac.
ABSTRACT
Knowledge editing has been proposed as an effective method for updating and correcting the internal knowledge of Large Language Models (LLMs). However, existing editing methods often struggle with complex tasks, such multi-hop reasoning. In this paper, we identify and investigate the phenomenon of Editing Overfit, where edited models assign disproportionately high probabilities the edit target, hindering the generalization of new knowledge in complex scenarios. We attribute this issue to the current editing paradigm, which places excessive em- phasis the direct correspondence between the input prompt and the edit target for each edit sample. To further explore this issue, we introduce a new bench- mark, EVOKE (E (EValuation of Editing Overfit in Knowledge Editing), along with fine-grained evaluation metrics. Through comprehensive experiments and analy- sis, demonstrate that Editing Overfit prevalent current editing methods and that common overfitting mitigation strategies are ineffective in knowledge editing. To overcome this, inspired by LLMs' knowledge recall mechanisms, we propose a new plug-and-pla -lay strategy called Learn the Inference (LTI), which introduce Multi-stage Inference Constraint module to guide the edited models in recall- ing new knowledge similarly to how unedited LLMs leverage knowledge through in-context learning. Extensive experimental results across wide range of tasks validate the effectiveness of LTI in mitigating Editing Overfit.
INTRODUCTION
Large Language Models (LLMs) have achieved remarkable success across various Natural Language Processing (NLP) tasks (Zhao et al. 2023), yet they often contain outdated or incorrect information, raising concerns about their reliability and factual accuracy. Knowledge Editing (Yao et al.) 2023) has emerged promising solution to precisely update correct a model's 's knowledge. Among the different editing strategies, parameter-modifyinn methods, which directly alter the model's internal parameters, have garnered significant attention from the research community. These include fine- tuning-based techniques such as FT-L (Zhu et al. 2020), meta-learning approaches like KE (De Cao al. )021 and MEND (Mitchell al. 2021 and locate-then-edit techniques such ROME iMen et al. 2022a) and MEMIT (Meng et al.) (022b).
Although existing methods have achieved promising results, their performance experiences catas- trophic decline when transferred to complex tasks involving reasoning (Yao et al. 2023). For in- stance, the representative multi-hop reasoning task, after the I.LM updated with Steve Jobs as the founder of Microsoft, it can easily respond to straightforward questions like "Who is the founder of Microsoft?" with "Steve Jobs." However, it struggles to accurately answer more complex queries, such "Which college did the founder of Microsoft attend?"
Equua contribution. Corresponding authors.

---

Published as a conference paper at ICLR 2025
Edited Knowledge
Editing Overfit
by
The college
Q: Which college did founder of
Q:
Who is the founder of Microsoft?
Editing
Microsoft attend?
is founded
The college of
Edited L.M
Editing Paradigm
Bill Gates
Bill Gates
Editor
Primarily Loss
Weight Update
Steve Jobs
Steve Jobs
FT-L
log (o lp(s,)))
W W eVL
Reed College
Reed College
MEND
p(s, r)
O"
W W + R(VI.)
Harvard University
Harvard University
ROME
Microsoft founded by
Steve Jobs @=
(v. WK. WK.
A: Steve Jobs
A: Steve Jobs
(C k.)
Figure 1: Example Editing Overfit.
To investigate the reasons behind the failure of edited LLMs in complex tasks, we first experi- mentally analyse the outputs from edited models on multi-hop reasoning task ($3). The results reveal an abnormally high probability that the edited models output the edit target o" for multi-hop questions, even when such responses are entirely implausible as valid answers ($3.2. We refer to this phenomenon as Editing Overfit, indicates that edited models tend to assign unusually high prediction probabilities to the edit target o² of edit sample (s, r. O, o²), skewing the response accuracy for complex questions where the correct answer is not o". For instance, shown in Figure [ after editing "Microsoft is founded by Bill Gates + Steve Jobs," " it erroneously answers the question "Which college did the founder of Microsoft attend?" with "Sleve Jobs."
We hypothesize that Editing Overfit is a key factor contributing to the suboptimal performance of edited LLMs complex tasks, like multi-hop editing. This phenomenon likely stems from existing knowledge editing paradigms emphasize the direct correspondence between the input prompt p(s,7 r) and the output o" for each edit sample (s, o, o²). Given the typically limited number of optimization samples, this focus on optimizing the p(s, r) - o* relationship can lead severe overfitting issues. Specifically, for LLMs rely primary loss function that maximizes the likelihood of the new target o* given the input prompt p(8, r). The main differences between these methods lie the techniques used for parameter updates. For example. FT-based methods either directly optimizes or uses parameter- efficient fine-tuning (Hu et al. 2022. Ren et al., 2024) to adjust model parameters, MEND employ a hypernetwork to make updates, while ROME and MEMIT apply low-rank updates derive closed- form solutions for specific parameters. When the model is updated with the new knowledge such "Microsoft is founded by Steve Jobs," it risks overfitting by learning only the correspondence between "Microsoft is founded by" and "Sleve Jobs." " As a result, the edited model may output "Steve Jobs" whenever it encounters the terms "Microsoff" and "is founded by." This also explains the abnormally high prediction probabilities of edit targets in multi-hop reasoning task, as the edited model may simply recognize patterns in the prompt and tend to output the corresponding edit target.
shown Figure [ all current editing methods
In this study, we particularly investigate the Editing Overfit phenomenon that occurs in edited LLMs To this end, we first construct benchmark for E EVallattig of Editing Overfit in Knowledge Editing (EVOKE) ($4.1) which comprises six tasks across two categories. The overfit tasks in EVOKE include various patterns prone to causing overfitting in models, allowing us analyze and investi- gate overfitting phenomena in current editing methods. By applying existing editing methods to EVOKE, we conduct an in-depth analysis to identify specific input patterns are prone to overfitting ($4.2) Furthermore, we evaluate the effectiveness of four existing overfitting mitigation strategies 5 Norm Constraints, Batch Editing, Multi-layer Editing, and Data Augmentation, in addressing the Editing Overfit problem.
To further alleviate Editing Overfit, inspired by the knowledge mechanism of LLMs, we propose plug-and-ple strategy named Leam To Inference (LTI) ($6). which enables the edited mod- els to leam how to infer with new knowledge rather than simply establish ouput- mappings. Specifically, LTT introduces Multi-Stage Constraint module, which imposes constraints crucial
2

---

Published as a conference paper at ICLR 2025
reasoning steps of LLMs during the editing process. This ensures that the edited model utilizes new knowledge in a way that closely resembles how an unedited model leverage new knowledge through in-context learning, helping prevent the model from overfitting solely input-output mapping. Additionally, LTT can be combined with various knowledge editing methods and used in conjunction with other overfitting mitigation techniques.
Our contributions can be summarized as follows:
We reveal and investigate the overfitting issue caused by current editing paradigm, identifying key factor behind the suboptimal performance of edited models, phenomenon we term the Editing Overfit problem.
We construct E EVKE, a benchmark with detailed evaluation metrics, to enable a fine- ane-ggained as- sessment and analysis of mainstream editing methods. Additionally, we explore the effectiveness of four general overfitting mitigation techniques in addressing the Editing Overfit problem.
We propose a new plug-in strategy, Learn the Inference, designed to further mitigate overfitting. Extensive experiments demonstrate that integrating L.TI with different editing methods effectively reduces the severity of Editing Overfit.
2 RELATED WORK
Knowledge editing (KE) updates LLM outputs to (i) accurately respond to new knowledge, (ii) preserve existing knowledge without catastrophic forgetting, and (iii) leverage updated knowledge in complex reasoning tasks. Each piece of knowledge is formulated
triple (8,7,0 o) (De Cao cet al. 2021), consisting of a subject s, relation r. and object o. An edit sample defined as O, o²), representing knowledge update from (8,r,o) o) to (s,r, 0")). Our study focuses on parameter-modiffing methods, which are divided into three main categories (Yao et al. 2023):
Sine-tuning-based methods generally follow the supervised fine-tuning paradigm. For example, to edita fact such as "Microsoft is founded by Steve Jobs," the model's weights are updated via gradient descent increase the probability of the edit target, Steve Jobs. Some approaches aim improve &
robustness by incorporating norm constraints (Zhu et al. (020)) or data Sugmentation(Gangaaaee Stratos 2024, Wei et al., 2024). However, vanilla fine-tuning often affects unrelated knowledge, leading to catastrophic Forgetting, making it unsuitable for direct application in knowledge editing.
Meta-learning- based methods employ hypernetwork to adjust model parameters specifically for editing. This hyperneworr is trained to convert fine-tuning gradients into updated weights, with the aim predicting weights that closely resemble those obtained through fine-tuning with augmented data. KE (De Cao et al.) (021) pioneered this approach, which MEND (Mitchell et al. 202 l) later extended to LLMs by predicting low-rank decompositions of parameter updates.
Locate-then-edit methods originate from research into the internal mechanisms of LLMss, advocat- ing for identifying the specific weights responsible for storing knowledge before applying targeted updates. Geva et al. [2021 2023) propose viewing MLP modules key-value memory. Building on this foundation, the Knowledge Neuron theory (Dai et al. 2022) posits that these MLP key-value pairs encode factual knowledge. Meng al. (2022a) introduce causal tracing to analyze LLMs' factual recall mechanisms, leading to the development of ROME (Meng et al.) 2022a) and MEMIT (Meng et al. 2022b), which achieved state-of-the-at results on several traditional metries.
In recent years, researchers have recognized the limitations of current editing methods on specific complex tasks such multi-hop reasoning, leading the development of task-specific approaches (Zhong et al., 2023 Zhang et al. 2024bb ap. More detailed related work is provided in Appendix B In contrast, our work explores the reasons behind the suboptimal performance of editing methods by constructing a benchmark and proposes more general strategy to enhance editing performance by addressing the issue of overfitting.
3 PRELIMINARY EXPERIMENTS
To investigate the causes of edited LLMs' poor performance on complex tasks, we begin by ana- lyzing the outputs of the edited models on a representative multi-hop reasoning dataset, COUNTER-
3

---

Published as a conference paper at ICLR 2025
FACTPLUS Yao et al. 2023 with a multi-hop question 9 = (s, r, r") that requires reasoning based on the edited sample.
where each entry contains edited knowledge (s. 0²) along
3.1 METRIC DEFINITIONS
To perform a tine-grained analysis of the outputs from edited models, we define several metrics in response to complex prompts, such as multi-hop questions within the dataset. Specifically, for each edit sample complex question, it may produce one of the following outputs: the original answer the complex question, the correct answer, or the edited target o Accordingly, we define the following metrics:
(s, r, o, o"), when the edited LLM is presented with prompt consisting of
Correct Answer Probability (CAP): The probability that the model generates the correct answer ans for a given prompt, formalized as IP (ans prompt).
Original Answer Probability (OAP): The probability that the model outputs the original answer ori (before editing) in response the given prompt, defined as P (ori promot).
Direct Probability (DP): The likelihood that the model produces the edit target o", expressed as (o" prompt).
To further evaluate the influence of both the target edit o and the original answer ori the correct answer ans, we follow Meng et al. (2022a) and define two additional comprehensive metrics to gauge the model's overall editing effectiveness:
Editing Overfit Score (EOS): This metric evaluates the performance of the edited model on complex questions where the correct answer not o". It serves as a primary indicator of the model's overfitting and overall performance. The score is calculated as the proportion of cases where the model overfits by favoring the edit target o over the correct answer ans, formalized as E [I[ (ans prompt) > IP (o* prompt)]].
Answer Modify Score (AMS): This metric evaluates the negative interference old knowl- edge the correct answers. It is assessed by calculating the proportion of cases where the probability of the correct answer exceeds that of the original answer, defined as E [T[P (ans prompt) > PP (ori prompt]]]]
3.2 EDITING OVERFIT PHENOMENON
Subsequently, we apply the ROME and MEMIT methods to GPT-J evaluate the per- formance of the edited models on COUNTER- FACTPLUS using the aforementioned metrics, shown in Figure 2 In multi-hop evalua- tions, the edit target o for each edit sample a possible answer to
OAP:
CAP:
DPL
AMS
305]
160
200
100
Valoe (%))
Valoe (56)
(96)
550
(s, (s, ) is typically the multi-hop prompt, and its output probabil- ity should therefore be negligible. For instance, "Steve Jobs" would be an implausible response "Which college did the founder of Microsoft attend?" The base model's DP score of 0.27% confirms that the unedited model is highly unlikely output o* as a response. However, after editing, both models exhibit significantly higher average probabilities of o* (DP), with ROME even reaching 41. 41.03%. Both models also show substantially lower Editing Overfit Score (EOS) values, indicating that for many evaluation samples, the probabil- ity of generating the correct answer is lower than that of outputting o*. This anomalous probability distribution substantially impacts model performance, as the inflated o* prediction probability di- minishes the Correct Answer Probability (CAP) and obscures the model's actual output.
BASE
ROME
MRMIT
Figure 2: Performance of GPT-J edited with ROME and MEMIT on COUNTERFACTPLUS.
From these observations, we define the phenomenon of Editing Overfit as follows: After an LLM has been edited based ened likelihood of producing the edit target o* as the answer to questions that implicitly or explicitly contains r, even when the correct answer is unrelated to o²
editing example (s. r, o, o²)) the edited LLM exhibits height-
4

---

Published as a conference paper at ICLR 2025
4 ANALYSIS ON EDITING OVERFIT
To further investigate the severity of Editing Overfit in edited LLMs, we construct EVOKE, a new benchmark designed to analyze overfitting phenomena across various tasks. We then assess the performance of different editing methods using this benchmark and examine the effectiveness of several existing mitigation strategies in reducing Editing Overfit.
4.1 EVOKE BENCHMARK
EVOKE comprises Recall Tasks and Overfit Tasks, covering six tasks in total. The Recall Tasks assess the edited model's ability to recall new edited knowledge, including Efficacy and Paraphrase evaluation. The Overfit Tasks pose complex challenges that are prone to inducing overfitting in editing methods, including Multi-hop Reasoning, Prefix Distraction, Subject Specificity, and Relation Specificity. These tasks are specifically designed to evaluate the model's capability to utilize newly integrated knowledge for more challenging scenarios, with a particular emphasis on examining the degree of Editing Overfit. Details of EVOKE construction can be found in Appendix
Taking the edit "Microsoft is founded by Bill Gates - Steve Jobs" as an example, we introduce the recall tasks used assess editing success rate of the edit (Meng et al. 2022a Yao et al. 2023):
Efficacy directly validates whether the edited models can recall the new edited knowledge (s, r, o²) under the editing prompt r)s, In the context of the above example, the model would be asked: "Who is the founder of Microsoft?"
Paraphrase examines the model's ability of recall the new knowledge (s, o*) using paraphrased forms of the editing prompt p(s, r). For instance, it might ask:" Who established Microsoft."
The design of overfit tasks are based on the two principles: First, the input questions explicitly or implicitly contain the information of subject s or relation to induce potential overfitting responses from the model; Second, the correct answers to these questions are entirely unrelated to o*, making it easier to determine whether the edited model exhibits overfitting. Accordingly, the overfit tasks are constructed as follows:
Multi-hop Reasoning evaluates the edited model's s ability to integrate the newly edited knowledge with existing knowledge to correctly answer questions spanning multiple entities or relations. For example, "Which university did the founder of Microsoft attend?" These questions typically contain implicit subject and relation r information from the edit sample, but the answer is not the target o". They well-suited for evaluating whether the edited model has overfit to the p(s,r) > o" pattern. model that has overfit to this pattern might incorrectly produce 'Steve Jobs' as the answer to this question.
Prefix Distraction uses the new knowledge rs, o²) as a perfix for unrelated questions, evaluat- ing weather the edited model can still provide the original correct answer. For example: "Microsoft was founded by Steve Jobs. Who is the founder of Amazon?" This evaluation also assess weather the edited model has overfit to the p(5,7) + pattern, providing a more explicit measure com- pared to multi-hop reasoning.
Subject Specificity presents questions with the same subject s as the edit sample but with different relations r'. For example: "When was Microsoft founded?" These questions typically contain information about the subject s, but the correct answer is not the target o", making them ideal for evaluating whether the edited model has overfit to the - o* pattern.
Relation Specificity includes questions with different subjects s' from the edit sample but the same relation r. such as: "Who the founder of Amazon?" These questions contain information about the relation but the answer is not the target o*. They are used to evaluate whether the model has overfit to the r + o* pattern. This task also corresponds to the locality evaluation in COUNTERFACT (Meng et al. 2022a).
The recall task is evaluated using the AMS metric. For the multi-hop reasoning task, we employ all five metries defined in Section 3.1 for comprehensive analysis. In the Prefix Distraction, Subject Specificity, and Relation Specificity tasks, the correct answer is identical to the original answer, making OAP equivalent CAP, with the EOS metric used to evaluate performance in these tasks.
5

---

Published as a conference paper at ICLR 2025
Table 1: Experimental results for different models the Overfit Tasks of EVOKE.
Prefix Distraction
Multi-hop Reasoning
Subject Specificity
Relation Specificity
Editor
DP1
CAPT
EOST
DPL
OAPL
CAP1
EOST
AMS1
DP]
CAP1
EOST
DPJ
CAPT
EOST
GPT-2 XL
5.01
13.30
54.58
0.27
7.87
4.21
93.43
35.64
0.50
4,90
85.59
0.35
6.42
79.61
FT
22.97
9.05
23.40
4.57
4.10
7.06
75.91
66.91
2,80
4.20
51.31
11.99
5.07
40.81
FEL
12.87
11.00
40.09
7:13
6.19
87.71
50.97
4.47
77.07
2.91
6,31
70.12
MEND
46.57
1,95
17.88
3.27
4.75
7.42
83.82
40.15
10.37
4.62
36.03
11.99
5.45
44.65
ROME
44.99
6.62
23.32
11.76
46.11
75.67
35.39
2.60
21.83
1.01
6,47
77.23
ROME LTI
19.53
9.88
28.17
10.08
5.12
11.03
65.94
70.83
19.05
3.92
30.79
0.61
6.49
78.04
MEMIT
32.19
7.63
20.55
16.75
4.28
11,92
57.06
72.63
22.68
3.20
25.98
0.85
6.38
77.81
MEMIT-LTI
18.76
8.78
26.02
8.05
4.97
11.40
72.02
69.59
7.39
3,50
38.21
0.62
6.28
78.79
GPT.
4.00
12.08
64.81
0.27
12.61
5.20
94. 16
30,66
0.63
6.95
80.35
0.31
9.43
84.28
77.43
3,61
4.58
8.91
25.55
37.5
0.38
61.5
3.01
9.74
FTL
7.05
17.99
56.30
3.28
10.77
9.40
85.77
49.51
0.93
6.70
77.07
1.40
9.67
80.24
MEND
36.31
11:27
29:40
3.94
12:21
84.43
35.28
12.09
6.80
33.84
13:33
8.35
53.09
ROME
26.13
12.62
32.57
41.03
3.65
17.96
34.91
80.92
54.15
1.80
6.77
2.51
8.41
79.64
ROME-LTI
8.93
15:48
49.73
11:12
6.43
17.17
69.83
77.45
10.18
28.38
0.73
8.68
81.86
MEMIT
18.30
14.77
39.32
25.80
5.82
18:49
50.49
75.67
33.45
2,90
17.69
0.95
9.10
82. 14
MEMIT-LTI
10.98
16.43
48.56
16:35
7.17
17.01
61.44
70.31
19.96
4.18
29.91
0.64
9.22
82.
4.2
RESULTS & FINDINGS
To assess the extent of Editing Overfit in current methods, we employ FT. FT-L, MEND, ROME, and MEMIT to edit GPT-J Waang& & Komatsuzaki (02T), GPT-2 XI. (Radford et al. 2019) and
Llama-2-7B (Touvron et al. 2023). We evaluate the pre- and post-edit performance of these models on E EVOKE. Results for Recall and Overfit Tasks on GPT-J and GPT-2 XL are shown in Tables
and 2 while results for Llama-2-7B are presented in Appendix G Based on these, we summarize our key findings as follows:
Finding 1: Current editing methods widely lead to severe overfitting. As shown in Table I
nearly all successfully edited models exhibit significantly higher direct probability (DP) scores across the four overfit tasks compared to the unedited model. Notably, the average DP for FT, ROME and MEMIT on most overfit tasks significantly surpasses the correct answer probability (CAP), with elevated EOS values indicating that this issue persists across many edited samples. Although FT-L and MEND show better overfitting metrics, their significantly lower paraphrase scores suggest that the edits were unsuccessful (as shown in Table 2 rendering their overfitting scores less meaning- ful. It is crucial to highlight that all editing methods exhibit very high probability of incorrectly outputting the edit target o (high DP score) in the prefix distraction task, with EOS scores also ab- normally low. This may be attributed to the fact that the Prefix Distraction task explicitly introduces distracting new knowledge (s, r, o²) prepended to the input. These results provide clear evidence supporting that existing editing paradigm is prone to causing overfitting.
Finding 2: Locate-then-Edie methods exhibits more severe overfitting to the O" pattern. As shown in Table T ROME and MEMIT perform sim- ilarly to unedited LLMs the Relation Specificity task across all metrics, indicating minimal overfit- ting to the r o" pattern. However, their weaker performance across all metrics on the Subject Speci- ficity task suggests a tendency toward overfiting the s + o" pattern. This difference may stem from their primary focus on manipulating subject repre- sentations to establish the mapping between p(s,r) and the new target o". Furthermore. ROME and MEMIT significantly improve the CAP metric for the Multihop Reasoning task - indicating better re- call of new answers surpassing other methods despite persistently high likelihood of overfitting to the Edit Target. These suggest that while locate-then-cdie paradigm has limitations, it sill shows promise in enabling edited models to effectively new knowledge for inferential tasks.
Table 2: Experimental results (AMST (%) on the Recall Tasks of EVOKE.
GPT-2 XL.
GPT-J
Editor
Efficacy Paraphrase Efficacy Paraphrase
BASE
19.50
22.79
13.29
15.47
FT
99.90
85.94
100.00
97.73
FT-L
98.93
45.64
99.81
45.44
MEND
92,92
57.47
96.90
54.17
ROME
100.00
96.27
99.90
99.27
ROME-LTI
100.00
92.10
100.00
96.27
MEMI
99.32
92.73
100.00
95.23
MEMIT-LTI
100.00
90.16
100.00
91.03
Finding 3: Both Fine-tuning based and Meta-learning based methods exhibit a strong over- fitting tendency to + o" and o* patterns. In contrast to Locate-thent then-edit methods, from Table II we observe similarly high levels of overfitting in both FT-based and MEND methods across the Subject Specificity and Relation Specificity tasks. This significant overfitting in both patterns
6

---

Published as a conference paper at ICLR 2025
is likely due these methods focusing on mapping the entire input p(s.r) to the target output o# during the editing process. Notably, even MEND, which demonstrated lower performance on Paraphrase task and potential underfitting, still exhibited significant overfitting. Another potentially underfitting model, FT-L., shows a reduced overfitting tendency, likely attributable to its Norm Con- straints on weight updates. Our subsequent detailed experiments ($5) will further explore the impact of Norm Constraints on editing success and mitigating Editing Overfit.
5 ANALYSIS ON MITIGATION TECHNIQUES
The analysis above demonstrates that the current editing paradigm generally leads overfitting to new knowledge in edited LLMs. To further investigate how existing strategies and different task scenarios influence overfitting, we conduct additional experiments analyzing various techniques. These include Norm Constraint, Batch Editing, Data Augmentation strategies, and Multi-layer Up- date Distribution (Appendix[I]) We primarily focus on several key metrics the following analysis: Efficacy and paraphrase are evaluated using the AMS metric, while the remaining four overfit tasks are assessed using the EOS metric.
Mitigation Technique 1: Norm Constraints Norm Constraints are a commonly used approach control excessive parameter updates and reduce overfitting, As observed in our main experiments (Table D. fine-tuning with Norm Constraints (FT-L) shows marked reduction in overfitting compared to direct fine-tuning (FT). In this section, we further investigate the effect of Norm Constraints on the per- formance of edited models using E EVOK. Follow- ing Zhu et al. (2020), we apply an Loo norm con- straint: |0 forl S c. Figure 3 illustrates the performance variation of FT-L as the strength of the norm constraint is adjusted.
EHooe Pamplan:
Mchthop Sppeallli
R-Spertetyy
80.0
Viduo
a00
1640
58-45
Noem
Figure 3: Performance of FT-L with different norm constraints on EVOKE.
The results indicate that relaxing the norm constraints leads to improvements in both editing efficacy and paraphrase scores, suggesting that increasing the update intensity of the weights can enhance the success rate of the edits. However, as the constraint norm increases, the overfitting metric (EOS) scores across overfit tasks also rise. Thus, while improving the edit success rate and paraphrase score by relaxing the norm, this comes the cost of heightened overfitting. When the paraphrase score reaches a satisfactory level, the overfitting issue becomes particularly pronounced. These findings highlight that relying solely on norm constraints as a strategy for mitigating overfitting may be insufficient.
Batch Editing In the preceding discussion, the Editing Overfit ob- served in edited models is likely linked to the limited-sample nature of knowledge editing tasks. Batch editing, as a natural multi-sample approach, involves simultaneously embedding a large number of factual associations into the LLM. Could this help alleviate the overfitting issue?
Mitigation Technique 2:
To explore this, we analyze the degree of overfitting in the batch editing setting and conduct experiments using the MEMIT with varying batch edits. The re- sults of these experiments are presented in Figure
Riirayy
Paraphrnee
Melthhop
Wax
ReSpesliitt
Ditnxction
The results reveal that the model's performance in a batch editing setting shows only marginal differ- ences compared to single editing. As the edit count increase, the performance of the edited model on paraphrase tasks and most overfit tasks exhibits a slight downward trend, while still demonstrating sig- nificant overfitting issues. The reason might be that, although batch editing introduce numerous new facts and increases the number of samples, each piece of knowledge remains independent, resulting in few effective samples per individual fact, thereby continuing to suffer from overfitting.
100
Figure 4: Performance of MEMIT with dif- ferent batch sizes on EVOKE.
7

---

Published as a conference paper at ICLR 2025
Mitigation Technique 3: Data Augmentation Data augmentation is widely used strategy to combat overfitting, particularly in scenarios with limited training samples. Following (Wei et al.) 2024 Gangadhar & Stratos 2024). we focus two data augmentation strategies: Paraphrase Aug- mentation which generates alternative formulations of the same factual statement, and Specificity Augmentation, which introduces new samples that retain the subject but alter the relations. For instance, given the new knowledge "Microsoft is founded by Bill Gates." Paraphrase Augmenta- tion might yield "Microsoft was established by Bill Gates," while Specificity Augmentation would introduce "Microsoft is headquartered in Redmond." Further details are provided in Appendix D.3
From Figure 5 we observe that MEMIT w/ Para- phrase performs worse than MEMIT acToss all tasks except for the Paraphrase task, still exhibiting over- fitting issues. We attribute this to the fact that, af- ter paraphrase augmentation, the method still tends to associate paraphrased versions of p(s, r) directly with o², which may inadvertently encourage the model to learn "output o* regardless of sentence phrasing when encountering inputs s and r," con- trary its intended purpose. In contrast, MEMIT w/ Specificity outperforms MEMIT on all overfit tasks, likely because Specificity Augmentation introduces more subject-related patterns, preventing the model from learning only the p(s,r) r) + o" pattern.
IIGII
SIEAIT of Purçlims
MEDIT Spe daity
60-
105-wr
Melliaap
5.Speiiiity RiSpeciiity Disterko
Figure 5: Performance of MEMIT with different data augmentation strategy on EVOKE
PROPOSED MITIGATION STRATEGY: LEARN THE INFERENCE
The preceding analysis suggests that, with the limited edited samples in knowledge editing tasks, the prevailing editing paradigm of "Learning the Correspondence" between input p(s,r) and output o" may cause edited LLMs to rely on input-output mappings during inference, rather than recalling and applying new knowledge in manner similar to their innate mechanism. Therefore, we propose that edited LLMs should ideally access and apply new knowledge during inference in way consistent with their natural inference process. To address this, inspired by the knowledge recall mechanism of LLMs (Geva et al. 2023) and the principles of In In-Context Learning (Brown et al.) (020). we propose a plug-and-play strategy called Learn The Inference (LTTI), as illustrated in Figure 6 LTT introduces a Multi-stage Inference Constraints module that imposes constraints on critical stages of knowl-
edge editing process, encouraging the edited model to recall newly edited factual associations in a manner similar to how unedited LLMs utilize new knowledge through in-context learning.
6.1 REASONING MECHANISMS IN LLMS: BACKGROUND AND RATIONALE
Recent research on LLM interpretability Meng et al. 2022a Geva et al. 2023 has revealed a two- step process for knowledge recall during inference. (1) In the shallow layers, knowledge related to the subject is aggregated to the last token of the subject. (2) In the deeper layers, the subject's representation is extracted to the final token position of the prompt to predict the output.
Our objective is for edited models to follow this same two-step process when recalling newly edited knowledge. To achieve this, we introduce multi-stage representations constraints during the editing process, ensuring that the inference process of the edited model aligns with that of unedited model using the new knowledge as context. This approach leverages LLMs' inherent in contentext learning abilities, as providing new knowledge in context typically enables unedited models adjust their outputs effectively (Zheng et al. 2023).
6.2 MULTI-STAGE INFERENCE CONSTRAINTS
We propose a Multi-stage Inference Constraints module consisting of three components: the Subject Representation Constraint, the Output Distribution Constraint, and the New Knowledge Constraint. These constraints collectively ensure the integration of new knowledge while aligning the inference consistency between the edited model and context-guided unedited model.
8

---

Published as a conference paper at ICLR 2025
As plug-and plug-and-play framework, ROME illustrate multi-stage inference constraint mod- ule. The ROME editing process involves calculating the optimal recall vector v, and the subject representation k, then updating the model's parameters via a rank-one update:
W W (v. Wk.)( (okk))(kk.)))
(I)
(C-1k.)²k,
Details on k, computation can be found in Appendix
F We now explain how our strategy integrates with the computation of v..
Knowledge
Jobs
Constraint
Specifically, we prepend the new edit knowledge (s,r,o") context prompt the original query r(s,r) and input it into unedited model, denoted as ((s,r,,") @ r)), Meanwhile, G' ("((s,, r)) rep- resents the edited model reasoning over p(s, r). We target specific layer / for the edit. The multi-stage constraints are formulated as follows:
Output Constraint
Subject Represention Construint
Subject Representation Constraint. Since LLMs extract representation of subject in the shallow MLP layer, we first apply constraints to align the last token representations of subject in the m-th layer (m > for both G' (p((s,))) r)) and G(s,rr, o²) @ p)8,r),, ensuring that edited subject representations to func- tion effectively in subsequent inference steps. This achieved by matching these representations using KL divergence, formalized as:
Knowedge "Mirerosoft founded
Mircrosoft founded
In-Context Learning
The Inforence of Editted L.M
Figure 6: The framework of Multi-Stage In- ference Constraints.
LSRC KL (Po)))-=)) [v"" ([)s iv." (s,r,0") + p)s,
(2)
where h is learnable parameter vector to modify the original value vector v resulting in the optimal vector v. v + h.
Output Distribution Constraint. Given that the final token of the prompt in deeper layers is crit- ical for predicting the output, we impose a regularization constraint on the output distributions of G' r)(, and (((s,,, p(s,r). This ensures that the output distribution of the edited model remains consistent with the output distribution generated by the normal inference process of the unedited model.
CODO KL (Po(y)+=)) ly p(s, r) [Pg ly (s, o") @ p(s,
(3)
This regularization serves as global constraint, ensuring alignment in the model's overall behavior.
New Knowledge Constraint. To enable the LLM accurately predict the target object o* for each edit sample (s, o,",",, we also define new knowledge constraint objective:
N
CN =
log toggo
p(s,)))
(4)
N
where zy is the random prefix generated by the LLM to foster optimization robustness.
Ultimately, the parameter h is optimized by minimizing the following objective function:
L XLsRe 3LoDD + aLN,
(5)
where B, represent the strength coefficients associated with different objective functions. No- tably, these constraint functions can be jointly optimized with the objective functions of other editing methods, such as FT and MEND, making L.T a highly extensible plug-and-plas strategy.
6.3 EXPERIMENTS
In this section, evaluate mitigation strategy by integrating LTI into the MEMIT and ROME methods and applying them to the EVOKE. Building on the experiments from Section we perform further evaluations answer the following key questions:

---

Published as a conference paper at ICLR 2025
The performance of all editors and 2 where ROME-LTI and MEMIT-LTI represent ROME on EVOKE is presented in Tables [
How Effective is LTI in Mitigating the Editing Overfit Problem?
and MEMIT method integrated with with LTI strategy, respectively.
(i) Performance on overfit tasks. From Table ! we observe that both ROME-LTI and MEMIT- LTI show significant improvement in overfitting metries (DP and EOS) compared to ROME and MEMIT, indicating effective mitigation of editing overfit. Additionally, ROME-LTI and MEMIT- LTI demonstrate improvements in AP and AMS metries across most overfit tasks. These findings suggest that overfitting suppresses the model's ability to output correct answers in complex tasks, and that alleviating editing overfit can improves the model's 's performance on complex tasks.
(ii) Performance on recall tasks. As shown in Table 2 ROME-LTI and MEMIT-LTI achieve results consistent with the original ROME and MEMIT on the Efficacy task, indicating that the edit success rate is minimally affected. However, the AMS metric for the Paraphrase task shows slight decrease, which might be attributed to the LTT strategy suppressing the strong association between p(s, r) and o". Nevertheless, this results in an overall improvement in performance across several overfit tasks, and we consider this slight reduction acceptable.
(iii) Comparison with data augmentation stralegiex. As discussed in Section Specificity Augmen- tation is an effective strategy for mitigating editing overfit. Therefore, we compare our approach to this technique. As shown in Figure 7 ROME-LTI outperforms ROME with data augmenta- tion (ROME w/ DA) in terms of the EOS metric across all overfit tasks. Additionally, unlike data augmentation methods, our LTI does not require the creation of additional samples, significantly improving editing efficiency and flexibility.
How do Constraints at Different Stages of LTI Influence Overfitting? The core of LTT lies in its Multi-stage Inference Constraints module, so we analyze how constraints applied at different stages influence the performance of editing method. Figure 7 compares ROME-LTI with variants where the Subject Representation Constraint is removed (ROME-LTI w/o SRC) and where the Output Distribution Constraint is removed (ROME-LTI w/o ODC).
ROME-LTI significantly outperformed the variant models across multiple metries, particularly in overfit tasks like multi-hop reasoning and prefix dis-
traction, and notably surpassed the original ROME method. Interestingly, ROME-LTI w/o ODC per- forms similarly to the original ROME, while ROME- LTI w/o SRC shows improvement but remains in- ferior to the ROME-LTI. This phenomenon may be attributed to the fact that ODC constrains the over- all input-output behavior of the model. Without ODC, even though SRC is retained, the output layer loss may drive the model to prioritize maximizing the output probability of o", potentially diminish- ing the impact of the SRC. These findings suggest that Subject Representation Constraint is most effec- tive when coupled with Out Distribution Constraint, yielding significant performance gains.
ond 980
BOMIE
a0-
Vidae
Blaxy
Panphree
Mankap
Saperdddl dodyndayy Dutnetsse
Figure 7: Performance comparison of differ- ent variant models on EVOKE.
CONCLUSION
This study identifies and investigates the phenomenon of Editing Overfit in the knowledge editing of LLMs. We propose that Editing Overfit likely originates from the common paradigm of existing editing methods, which focus on learning subject-relation-ohjeee correspondences in factual state- ments with limited edit samples, leading to overfitting to these patterns. To further explore the overfitting issues in existing editing methods, we construct a new EVOKE benchmark along with dedicated overfitting evaluation metrics. Extensive experiments demonstrate that that current edit- ing methods commonly result in significant Editing Overfit, and that general overfitting mitigation strategies show limited effectiveness in addressing this problem. To tackle this challenge, design a plug-and and-play strategy called Learn the Inference, implemented through a Multi-stage Inference Constraint. Experimental results show its effectiveness in mitigating overfitting.
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGEMENTS
This work was supported by the Natural Science Foundation of China (62472261, 62102234, 62372275, 62272274, 62202271, T2293773, 62072279, 62206291), the National Key R&D Pro- gram of China with grant No.2022YFC3303004, the Natural Science Foundation of Shandong Province (ZR2024QF203)
REFERENCES
Baolong Bi, Shenghua Liu, Lingrui Mei, Yiwei Wang, Pengliang Ji, and Xueqi Cheng. Decod- ing by contrasting knowledge: Enhancing Ilms' confidence on edited facts. arXiv preprint arXiv:2405.11613, 2024a.
Baolong Bi, Shenghua Liu, Yiwei Wang, Lingrui Mei, Hongcheng Gao, Junfeng Fang, and Xueqi Cheng. Struedit: Structured outputs enable the fast and accurate knowledge editing for large language models. arXiv preprint arXiv:2409.10 10132.01...........01111111....1 2024b.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. Language models are few-shot learners. Advances in Neural Information Processing Systems, volume 33. pp. 1877-1901, 2020.
Roi Cohen, Eden Biran, Oni Yoran, Amir Globerson, and Mor Geva. Evaluating the ripple effeets of knowledge editing in language models. Transactions of the Association for Computational Linguistics, 12:283-28 298, 2024.
Damai Dai, Li Dong, Yaru Hao, Zhifang Sui, Baobao Chang, and Furu Wei. Knowledge neurons in pretrained transformers, In Annual Meeting of the Association for Computational Linguistics, pp- 2022.
Nicola De Cao, Wilker Aziz, and Ivan Titov. Editing factual knowledge in language models. In Conference on Empirical Methods in Natural Language Processing, pp. 6491-6506, 2021.
Govind Krishnan Gangadhar and Karl Stratos. Model editing by standard fine-tuning. In Findings of the Association for Computational Linguistics ACL 2024, pp. 5907-5913, 2024.
2024.
Mor Geva, Roci Schuster. Jonathan Berant, and Omer Levy. Transformer feed-forwe layers are key-value memories. In Conference Empirical Methods in Natural Language Processing, pp-
2021.
Mor Geva, Jasmijn Bastings, Katja Filippova, and Amir Globerson. Dissecting recall of factual associations in regressveee language models. In Conference on Empirical Methods in Natural Language Processing, pp. 12216-12233 2023.
Jason Mhoelscher-Obermaale Julia Persson, Esben Kran, Ioannis Konstas, and Fazl Barez. Detecting edit failures in large language models: An improved specificity benchmark. In Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki (eds.), Findings of the Association for Computational Linguistics: ACL 2023, pp. 1548-1155,, Toronto, Canada, July 2023. Association for Compu- tational Linguistics. doi: 10. 00888861/2000/ 1tt.://..22....nnna..l/l/l/ findings-acl.7335
Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-rank adaptation large language models. In International Con- ference on Learning Representations, 2022.
Goro Kobayashi, Tatsuki Kuribayashi, Sho Yokoi, and Kentaro Inui. Feed-forward blocks control contextualization in masked language models. arXiv preprint arXiv:2302.00456. 2023.
Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. Locating and editing factual associations gpt. Annual Conference on Neural Information Processing Systems, 35:17359- 17372, 2022a.
11

---

Published as a conference paper at ICLR 2025
Kevin Meng, Arnab Sen Sharma, Alex J Andonian, Yonatan Belinkov, and David Bau. Mass-editing memory in a transformer. In International Conference on Learning Representations, 2022b.
Eric Mitchell, Charles Lin, Antoine Bosselut, Chelsea Finn, and Christopher D Manning, Fast model editing at scale. In International Conference on Learning Representations, 2021.
Judea Pearl. Direct and indirect effects. In Probabilistic and causal inference: the works of Judea Pearl, pp. 373-392. 2022.
Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. Language models are unsupervised multitask learners. 2019.
Pengjic Ren, Chengshun Shi, Shiguang Wu, Mengqi Zhang, Zhaochun Ren, Maarten de Rijke, Zhumin Chen, and Jiahuan Pei. MELoRA: Mini-ensemble low-rank adapters for parameter- efficient fine-tuning Proceedings of the 62nd Annual Meeting of the Association for Computa- tional Linguistics, 2024.
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Niko- lay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open founda- tion and fine-tuned chat models. arXiv preprint arXiv:2307.09288, 2023.
Jesse Vig, Sebastian Gehrmann, Yonatan Belinkov, Sharon Qian, Daniel Nevo, Yaron Singer, and Stuart Shieber. Investigating gender bias in language models using causal mediation analysis. Annual Conference on Neural Information Processing Systems, 33:12388 2020.
Ben Wang and Aran Komatsuzaki. GPT-J-6B: Billion Parameter Autoregressive Language Model. https: httpp::/github.... com. wttp//wwwww....ioooooooo...../.e watpam/iinnoffooooooo www.l/lslllhmeeeeoeoeoommmme www.sshttttssfooomeeeo jax May 2021.
Zihao Wei, Liang Pang, Hanxing Ding, Jingeheng Deng, Huawei Shen, and Xueqi Cheng. Stable knowledge editing in large language models. 2024.
Yunzhi Yao, Peng Wang, Bozhong Tian, Siyuan Cheng, Zhoubo Li, Shumin Deng, Huajun Chen, and Ningyu Zhang. Editing large language models: Problems, methods, and opportunities. In Conference on Empirical Methods in Natural Language Processing, pp. 10222- 10240, 2023.
Mengqi Zhang, Bowen Fang. Qiang Liu, Pengjie Ren, Shu Wu, Zhumin Chen, and Liang Wang. En- hancing multi-hop reasoning through knowledge erasure large language model editing, 2024a.
Mengqi Zhang, Xiaotian Ye, Qiang Liu, Pengjie Ren, Shu Wu, and Zhumin Chen. Knowledge graph enhanced large language model editing. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pp. 22647-22662, 2024b.
Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, et al. A survey of large language models. arXiv preprint arXiv:2303. 18223, 2023.
Ce Zheng, Lei Li, Qingxiu Dong, Yuxuan Fan, Zhiyong Wu, Jingjing Xu, and Baobao Chang. Can we edit factual knowledge by in-context learning? In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp. 4862-4876, 2023.
Zexuan Zhong, Zhengxuan Wu, Christopher D Manning, Christopher Potts, and Danqi Chen. MQUAKE: Assessing knowledge editing in language models via multi-hop questions. In Con- ference on Empirical Methods in Natural Language Processing, pp. 15686-15702. 2023.
Chen Zhu, Ankit Singh Rawat, Manzil Zaheer, Srinadh Bhojanapalli, Daliang Li, Felix Yu, and Sanjiv Kumar. Modifying memories in transformer models. arXiv preprint arXiv:2012.00363, 2020.
12

---

Published as a conference paper at ICLR 2025
A LIMITATIONS
In this section, we primarily discuss the potential limitations of the proposed LTI:
The first limitation that our LTI relies on the model's in-context learning capability. Specifically, LTI requires the unedited model to generate accurate answers based solely on the contextual rep- resentation of new knowledge. This contextual dependency provides the foundation for accurate output distribution constraints during the editing process. However, for LLMs lacking strong in- context learning abilities or rigidly adhere to certain pre-existing facts, these inherent characteristics and capabilities may limit LTT's effectiveness in practical applications.
The second limitation of LTI is primarily designed for structured knowledge editing tasks. Specifi- cally, LTI is better suited for scenarios where the knowledge can be explicitly represented knowl- edge triples (8, 0"))). This may restrict its applicability in more general knowledge editing tasks that are difficult to formalize, and how better enable models to learn the inference process in more general knowledge contexts requires future exploration.
B DETAILED RELATED WORK
In-context editing (ICE) Zheng et al. (2023) Zhong et al. (0033; ; Bi et al. (2024a) representative frrameter-preservine method that explicitly retrieves edited knowledge and incorporates it into the context to modify the behavior or outputs of LLMs, achieving significant success in various editing tasks, such as multi-hop editing. For example, MeI.Lo (Zhong et al., 2023) stores all edited facts externally while prompting the LLMs iteratively to generate answers that are consistent with the edited facts. Furthermore, [i al. (2024a) identify that the performance of ICE is hindered by stub- born knowledge and propose DecK, which improves ICE by contrasting the logits derived from the newly edited knowledge with those from the unedited parametric knowledge. Unlike ICE methods that integrate the new knowledge into the input prompt, StruEdit (Bi et al., (024b) structures the outputs of LLMs, directly remove all information potentially affected by new knowledge, and refill the structured outputs with updated information. Although these methods have achieved promis- ing results on certain complex editing tasks, they rely heavily on retrieving external knowledge or carefully designed prompts, in contrast to wrramettrrmodifyiinn approaches, significantly limits the flexibility of the edited models. this paper, we propose LTI, which leverage the strengths of ICE methods to mitigate the overfitting issues commonly observed in parameter-modifyy approaches.
C DETAILS ON THE EVOKE BENCHMARK
The EVOKE benchmark is designed to evaluate the effectiveness of knowledge editing methods for LLMs, focusing not only on their ability to accurately recall edited knowledge but also on their performance and potential overfitting across diverse complex reasoning tasks. Notably, part of EVOKE's data sourced from existing knowledge editing datasets. We have reorganized and extended datasets including COUNTERFACT (Meng et al. (002aa), COUNTERFACTPLUS (Yao al.) 2023), and POPUULA (Cohen et al. (024)) to construct new task-specific data for comprehensive analysis of knowledge editing methods.
Table An Example of the EVOKE dataset
Task
Value
Edit Request
{Spike Hughes} originates from London + Philadelphia
Efficacy
(Spike Hughes} originates from (answer: Philadelphia)
Paraphrase
Spike Hughes is native to (answer: Philadelphia)
Subject Specificity
The profession of Spike Hughes is (answer: musician)
Relation Specificity
Max Mosley originates from (answer: London)
Multi-hop Reasoning
What famous food is associated with the city where Spike Hughes originates from?
(answer: Cheesexteaks)
Prefix Distraction
Spike Hughes originates from Philadelphia. Max Mosley originates from (unswer: London)
13

---

Published as a conference paper at ICLR 2025
Each entry in EVOKE consists of counterfactual Edit Request, Efficacy Prompt, several Para- phrase prompts for recall tasks, and corresponding questions and answers for the four Overfit Tasks: Multi-hop Reasoning, Prefix Distraction, Relation Specificity, and Subject Specificity. Tables and and provide task examples and summarize the statistical composition EVOKE, respectively.
Dataset Construction We construct EVOKE, comprehensive dataset for evaluating overfitting in LLMs based on edits, by extending and refining existing benchmark data. The edit requests and recall task data in EVOKE are sourced from established benchmarks, including RIPPLEEDITS- POPULAR and COUNTERFACT, specifically leveraging a subset of their test splits.
For Multi-hop Reasoning task, we augment the existing dataset by incorporating newly constructed data. Specifically, Building upon the COUNTER FACTPlus dataset, which lacks original answers for multi-hop questions, we generate the missing answers using GPT-4o, following the method- ology used in (Yao et al., 2023). We validate that these answers can be correctly recalled by the unedited GPT-J model, ensuring they accurately reflect the model's typical responses. Furthermore, address potential data leakage caused by implicit answer category constraints in some prompts, where unedited models might predict new-fact- mitigate this, uation. We ensure transparency by reporting the performance of the unedited base model on this modified dataset.
multi-hop answers with high probability. To GPT-4o to rephrase prompts and reduce biases, creating more robust eval-
The remaining tasks in EVOKE, including Re- lation Specificity, Subject Specificity, and Pre-
Table 4: Composition statistics of EVOKE
fix Distraction, are constructed to target spe- eific aspects of overfitting. Leveraging the lo- cality data from COUNTERFACT, the Relation Specificity task probes the consistency of edited models' responses across prompts that share the same relation but differ in subject entities. This aligns with the locality metric in traditional knowledge editing datasets, which is used to as- sess the preservation of unrelated knowledge. We ground the Subject Specificity task data curated from the HIPPLEEDITS-POPOL.N dataset directly assess overfitting specific subject entities. For the Prefix Distraction task, we prepend edit requests, in the form of new facts, to the prompts used the Relation Specificity task, following the approach outlined in [Hoelcher- Obermaier et al. (2023). This construction enables the evaluation of an edited model's susceptibility irrelevant information presented as part of the input context.
Type
Total
Edit Requests
1489
Efficacy Prompts
1489
Paraphrase Prompts
2062
Subject Specificity Prompts
458
Relation Specificity Prompts
10310
Multihop Prompts
822
Prefix Distraction Prompts
10310
D EXPERIMENTAL SETUP DETAILS
Our experiments build the codebase implemented by Meng et al. (2022a b). For the hyperparam-
eters of MEMIT on GPT-2 XL, we observe significant failures in certain editing cases, with notably lower performance metrics the recall task compared GPT-J. To address this, we adjust the hy- perparameters, specifically increasing the c clamp .no.nn norm_factor from 0.75 to 1.25 for GPT-2 XL. This adjustment made align its editing success rates with those of GPT-J, allowing us to evaluate normal editing performance and potential overfitting. All other baseline implementations, including hyperparameters, remain consistent with the setup Meng al.] (2022a (), and hyperparameters on Llama-2-7B remain consistent with Yao et al. (2023).
D.1 BASELINE METHODS
Fine- Fine-Tuning (FT) involves fine-tuning the base model on text describing the edit fact.
Constrained Fine-Tuning (FT-L) (Zhu al. 2020) involves fine-tuning specific layers of the LLM's parameters directly using gradient descent, while imposing a norm constraint on the weight changes to prevent catastrophic forgetting.
MEND Mitchell et al. 202T) constructs a hyper- based on the low-rank decomposition of
gradients perform editing.
14

---

Published as a conference paper at ICLR 2025
2022a) operates the hypothesis that knowledge in LLMs is stored in the MLP layer and utilize rank-one update to insert knowledge into MLP layer.
ROME (Meng et al.)
2022b) builds on the ROME method, specializing in batch-editing tasks by applying edits across multiple MLP layers.
MEMIT (Meng et al.
D.2 DETAILS ON MULTI-LAYER EDITING EXPERIMENT
In Appendix H investigate the impact of distributing parameter updates across multiple lay- ers on Editing Overfit, conducting experiments using MEMIT method. The original MEMIT ap- proach, when applied to GPT-J, edited six layers (1 = 3, computed at layer 8, In our experiments, we maintain constant product of the hyperparameter aamp-noom_aa form.factor or and the number of edited layers. When adjusting the number of layers, we keep the highest edited layer fixed. For example, when editing three layers, we select 1 = {6, 7,8}.
7, 8), with the recall vector
D.3 DETAILS ON DATA AUGMENTATION EXPERIMENT
In Section 5 we follow existing data augmentation strategy (Wei et al. 2024, Gangadhar Stratos 2024) to evaluate the effect of data augmentation on Editing Overfit. Specifically, we employ GPT- 4o to generate 10 data-augmented paraphrase prompts and subject specificity prompts for each edit request. Paraphrases required to be factual statements semantically equivalent to the edited knowledge, while subject specificity prompts are factual statements about the same subject but with different relations. The construction of specificity augmentation prompts aligned with the task test prompts for Subject Specificity, directly enhancing this task. We also utilize GPT-4o to verify and filter augmented samples that do not meet the requirements.
We test data augmentation effects on MEMIT. The original MEMIT implementation optimized up- dated parameters by combining the original factual statement with five additional prompts. These prompts are created by appending random prefixes to the original statement, helping the model gen- eralize across contexts. In our paraphrase augmentation experiment, we maintain consistency by optimizing with five randomly prefixed paraphrased prompts. For the specificity augmentation ex- periment, we add five unrelated facts to the optimization process. This helps suppress the probability of outputting the Edit Target.
E METHOD IMPLEMENTATION DETAILS
In this section, we detail the implementation of our L.TI strategy, covering the design of the Contex- tual Prompt, the selection of layers for the Subject Representation Constraint, and key hyperparam- eter settings.
Design of the Contextual Prompt In our LTI framework, Multi-stage Inference Constraint mod- ule involves aligning the inference process between the edited model and a context- guided unedited model during editing. In this approach, context is used solely to introduce prefix that guides the model to infer and output based on the new fact, thus providing an inference process as a learn- ing target. We employ minimalistic approach context construction, inspired by the in-context editing baseline from Cohen et al. (2024). Specifically, we prepend the prefix "Imagine that" to the input prompt as a context prefix. For instance, given new knowledge "Microsoft is founded by Bill Gates + Steve Jobs," the context prefix would be "Imagine that Microsoft is founded by Steve Jobs." " This simple strategy avoids the format restrictions seen in few-shot demonstrations, minimizing potential adverse effects on the model's output distribution and leaving room for future research and optimization.
Layer Selection for the Subject Representation Constraint The choice of layers for applying the Subject Representation Constraint (SRC) in LTI a critical hyperparameter, guided by previ- ous interpretability studies on LLM knowledge recall mechanisms(Ment et al.) 2202aa] Cohen al. 2024]. These analyses reveal that subject position representations, which encode relevant knowl- edge, extracted by attention modules in deeper layers for answer prediction. The SRC aims to align the subject representations the edited model with those produced by normal inference, en- suring that subsequent recall steps function expected. Specifically, Cohen al. (2024) highlight
15

---

Published as a conference paper at ICLR 2025
that for GPT-2 XL, the attention edges from Suubject-tollastttokee in layers 30 40 are strongly correlated with answer prediction, while for GPT-J, this correlation is observed in layers 15 20. Therefore, constrain layer 30 in GPT-2 XL and layer 15 in GPT-J and Alama-2-7B. anticipat- ing these representations to be effectively utilized by these information flows. Further experimental results the effects of constraining different layers presented in Appendix
Other Hyperparameter Settings LTT's hyperparameters include coefficients for the three con- straint losses. In practice, the coefficient for the Subject Representation Constraint set to 0.0625, while the Output Distribution Constraint coefficient 3 is 0.0325. For the New Knowledge Constraint coefficient o, ROME-LTI uses values of 0.0625 for GPT-J, 0.15 for GPT-2 XL and 0.35 for Llama- 2-7B, whereas MEMIT-LTI employs 0.25 for GPT-J and 0.125 for GPT-2 XL and Llama-2-7B.
F RANK-ONE MODEL. EDITING
Rank-One Model Editing (ROME) Meng et al. (002aa) a Locate-then-edit method that assumes factual knowledge is stored within the MLP layers of LLMs, conceptualized as key-value memories Geva et al. (2021); Kobayashi al. (2023). The output of the 1-th layer MLP for the i-th token given by:
S(W,, 1) W
where f(.) denotes the activation function, and h! is the MLP input. For simplicity, the superscript 2 omitted in the following discussion.
In this setup, f(W... in h;) represent the keys, denoted as k, while the outputs of the subsequent layer serve the corresponding values. Using casual tracing |earl (2022); (ig g al. (2020), ROME identifies specific MLP layer for editing and updates the weight W of the second layer by solving a constrained least-squares problem:
minimize subject to Wk, = V..
IWK - VII.
where the objective is to preserve unrelated knowledge within the LLM, while the constraint ensures that the edited knowledge is incorporated into the MLP layer. Here, = [k| . kp] denotes the sets of keys encoding subjects unrelated to the edited fact, and V = [Vi; v vpl represents the corresponding values. The constraint ensures that the edited knowledge incorporated into the MLP layer by enabling the key k, (encoding subject to retrieve the value about the new object o².
As explicated in Meng et al. derived:
2022a a closed-form solution to the optimization problem can be
W W (v. Wk.) (C-1k.)"
(8)
where C = KKT is a constant matrix, precomputed by estimating the uncentered covariance of based on a sample of Wikipedia text. Thus, solving the optimal parameter W is transformed into calculating subject representation k, and recall vector v.. For each edit sample (8,r,0,0*) the subject representation k, is calculated by
1 N
k.
Σ f(w! in h(-1).
where random prefixes generated by the LLM are used to enhance optimization robustness Meng al. 2022a
G RESULTS ON LLAMA-2-7B
In this section, we extend our experiments the Llama-2-7B Touvron al. 2023 model using the EVOKE dataset, with results presented in Tables 5 and 6
Consistent with findings on GPT-2
XL and GPT-J, overfitting patterns persist on this more advanced model: FT continues to overfit to
16

---

Published as a conference paper at ICLR 2025
Table 6: Experimental results for different models the Overfit Tasks of EVOKE.
Prefix Distraction
Multi-hop Reasoning
Subject Specificity
Relation Specifielty
Editor
DPJ
CAPT
EOST
DPL
OAPL
CAP1
EOST
AMS1
DPJ
CAP1
EOST
DPJ
CAP1
EOST
Llama-2-78
16.42
28.03
58.44
21.79
86.01
97.38
1.18
21.21
82.92
FT
51.48
6.11
5.61
27.71
9.26
12.21
35.40
56,20
9.74
11.03
46.07
23.12
6.30
18.83
FT-L
16.90
26.29
53.17
2.59
19.47
10.96
85.16
36.13
1.12
31.62
95.20
1.54
20.14
81.08
ROME
22,80
25.22
46.66
21.66
9.06
24.98
63.99
77.37
21.82
15.23
45.63
2.09
20.52
81.11
ROME-LTI
17.90
27.60
53.13
70.32
14.30
57.42
1.73
20.63
81.36
MEMIT
40.76
16.90
25.
30.47
6.05
22.29
48.18
81.75
39.09
8,93
23.36
3,87
19.47
78.08
MEMIT-LTI
26.75
20.77
36.81
14.15
21.31
67.52
74.00
9.06
10.86
47.60
2.15
19.74
80.30
p(s,T) + os pattern, while locate-then-edit methods maintain their overfit pattern to single - o* map- pings. These results further confirm the universality of the Editing Overfit phenomenon. Specifically, nearly all suc- cessfully edited models exhibit significantly higher Direct Probability (DP) scores compared to the unedited model. The average DP for FT, ROME, and MEMIT most overfit tasks significantly surpasses the Correct Answer Probability (CAP), with elevated EOS values indicating persistent issues across edited samples. Notably, both ROME-LTI and MEMIT-LTI show significant improve- ments in overfitting metrics (DP and EOS) compared to their base versions, demonstrating effective mitigation of Editing Overfit.
Table 5: Results (AMS) (%) on the Re- call Tasks of EVOKE.
Llama-2-7B
Editor
Efficacy Paraphrase
BASE
13.09
15.08
FT
99.61
92.48
92.73
ROME
100.00
93.60
ROME-LTI
100.00
89.42
MEMIT
100.00
96.80
MEMIT-LTI
100.00
91.71
H ANALYSIS ON DISTRIBUTING WEIGHT UPDATES ACROSS LAYERS
Another notable observation from Table D is the significant reduction in overfitting exhibited by MEMIT compared to ROME. Both meth- ods follow the thatte-heen-edi paradigm, but the key difference is that MEMIT distributes updates across multiple layers, whereas ROME concentrates modifications on a single layer. This distinction motivates further investigate into the impact of distributing weight updates across multiple layers on mitigating overfitting.
Ribbaayy
Pamaphriee
Multhhop
Valre 19
Walre
Laose S-Speenfoity
Laosz R-Specifeiity
Leosz Distraction
Valte (0]
Nate (8)
Laove
Laonz
Leose
We edit GPT-J using MEMIT with varying numbers of modified layers and evaluate the performance of the edited model on EVOKE. The results, shown in Figure [ indicate that as the number of layers involved in weight distribution increases, the EOS metric tends increase, albeit with a slight decline in the model's paraphrasing capabilities. These findings suggest that distributing weight updates across multiple layers, rather than concentrating them in single layer, is generally beneficial in mitigating overfitting.
Figure 8: Performance of MEMIT with different editing layers on EVOKE.
I THE IMPACT OF CONSTRAINT LAYERS
Figure 9 demonstrates the model's performance when varying the number of intermediate lay- constrained in ROME-LTI. Despite minor fluctuations across different metries, the per- formance on all overfit tasks consistently remains significantly better than that of the origi- nal ROME method. This suggests that the model's performance not highly sensitive to the specific layer position of intermediate constraints. This phenomenon may be attributed to that aligning the hidden state of a specific layer with the target representation likely in- fluences the reasoning process in adjacent layers for the same token position. This implic- itly extends the constraint effect to other layers, encouraging them to also approximate the
17

---

Published as a conference paper at ICLR 2025
corresponding representations in the target in- ference process. Previous ablation studies ($6.3) have shown that removing hidden state constraints at the subject position significantly degrades performance, and these results indi- cate that constraining specific token positions intermediate layers may be more crucial than constraining particular layer numbers.
Efflwwy
Paraphnase
Mahhhop
150 Ver
Video (S)
5
Vide (83)
S-Spertheity
R-Specifiiity
Dietratioo
Vi00 PS1
5
Van P8]
J
CASE STUDY
La0
Legoz
Losts
Figure 9: Performance of ROME-LTI with dif-
This section presents generation examples from GPT-J models edited with various methods on multi-hop question from the EVOKE bench- mark, as illustrated Figure TO This specific example introduces the counterfactual fact, "Bhamdoun, located in Lebanon + Portugal," with the corresponding multi-hop question being "What is the capital city of the country where Bhamdoun is located?" Correctly answering this question requires the model to first recall the edited fact that "Bhamdoun is located in Portugal", and then recall that "Lisbon is the capital of Portugal", which is the final answer. If Editing Overfit occurs, the model might instead disproportionately favor the edit target and incorrectly output "Portugal."
ferent constraint layers EVOKE. Gray dashed lines indicate the performance of the original ROME method.
Notably, the FT-edited model displays significant collapse, repeatedly outputting the edit target, "Portugal," without providing any other relevant information. Although FT-L and MEND do not directly output the edit target, they still fail produce the correct answer, indicating their inability to effectively utilize the new knowledge in this complex task. Conversely, both ROME and MEMIT directly output the edit target, exemplifying clear cases of Editing Overfit. ROME-LTI and MEMIT- LTI successfully answer the multi-hop question within their responses and maintain better overall consistency, highlighting the potential of LTI in mitigating overfitting and improving post-editing model performance.
18

---

Published as a conference paper at ICLR 2025
New Fact: Bhamdoun, located Lebanon Portugal
Multi-hop Question: What is the capital city of the country where Bhamdoun is located?
Answer: Lishon
FT:
What capital city of country where Bhamdoun located? The answer this question, most simply, Portugal Portugal Portugal Portugal Portugal Portugal Portugal...
FT-L:: What is the capital city of the country where Bhamdoor is located?? The answer to this question, most simply, is "Bhamdoun, France", and it also question that is asked by many people who have been Bhamdoun. The answer is, of course, a question that requires a little explanation, and the explanation can be found in the history books, and the history books can found the library. the...
MEND:
What is the capital city of the country where Bhandoun is located? The answer this question, most simply, is Baghdad, Iraq. The capital of Iraq is the most populated city in the country, as well as the most important, politically, cconomically, and culturally. The city of Baghdad, the north of the country, the center of Iraq's govermment. What is the name of the city in which Bhamdoun is located? The city where
What is the capital city of the country where Bhamdoun is located? The answer to this question, most simply, is Portugal, but the answer is little more complicated than that. Portugal is autonomous region of mainland Spain, but its capital, Lisbon, is the capital of the country of Portugal. Portugal country in Western Europe with of 505,000 km2 (193,000 sq mi) and population of 10,547,000.
ROME:
MEMIT: What is the capital city of the country where Bhamdoun is located? The answer this question, most simply, is Portugal. But the city of Bhamdoun a city of many layers, In the past, was an ancient capital the lberrans; it also the seat of the first Islamic state in Europe, the Almoravid Empire;
What the capital city the country where Bhamdoun is located? The simply, is Lisbon, That is, the city the capital of Portugal's 's autonomous region of Lisbon. The answer may seem obvious but it not. Portugal, region its own capital city and capital region is always the capital of the...
ROME-LTE:
this question, most
What is the capital city of the country where Bhamdoun is located? The answer to this question, most simply. Lisbon. But Lisbon also known by many other names, such Porto, Oporto and Lisboa, and these are often used in Portuguese-speaking countries, including Brazil, Portugal itself, Mozambique and Angola. The Portuguese capital is very city, with a history that dates back the Romans.
MEMIT-LTI:
Figure 10: GPT-J generation examples of test case No.361 in EVOKE dataset. Prompts shown in italic. Green text indicates the correct answer to the multi-hop question, blue text is related to the edit target, and red text highlights noticeable inconsistencies between the generated content and the inserted knowledge or context.
19


Reviews:
[{"review_id": "MrJcVdf1NF", "reviewer": "Reviewer_hnCp", "summary": "In this paper, the authors focus on investigating parameter-modifying knowledge editing methods and find that existing parameter-modifying knowledge editing approaches exhibit overfitting. Specifically, when questions related to the edited subject and relation arise, the model tends to respond with the edited object, regardless of whether that object is a reasonable answer to the question.\n\nBased on this finding, the authors propose the EVOKE benchmark to assess the degree of overfitting in previous parameter-modifying knowledge editing methods and to evaluate the effectiveness of past techniques designed to mitigate overfitting. They discover that, with the exception of Specificity Augmentation, most methods show limitations.\n\nTo address the overfitting issue, the authors introduce a plug-and-play strategy called LTI. Experimental results demonstrate that LTI is highly effective in alleviating overfitting.", "questions": "- I am curious whether the representation and output distribution of the unedited model are computed only once. If so, I wonder whether applying Paraphrase Augmentation to this new knowledge would improve performance. The final loss could potentially be adjusted to compute the average KL divergence across different paraphrases. The paper mentions that Paraphrase Augmentation can actually exacerbate overfitting in knowledge editing methods. Therefore, would applying Paraphrase Augmentation to the new knowledge enhance the quality of the constraint, or would it ultimately reduce the effectiveness of the edit?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The overfitting issue in parameter-modifying knowledge editing methods identified by the authors is an interesting finding.\n\n- The proposed benchmark contributes to a more comprehensive analysis of the effectiveness of knowledge editing methods.\n\n- The authors provide extensive experimental results, conducting a detailed analysis of previous parameter-modifying knowledge editing methods and testing various previously suggested mitigation techniques.\n\n- The LTI method proposed by the authors is a plug-and-play solution, making it convenient to integrate with previous methods for enhanced performance.", "weaknesses": "- The proposed LTI method relies on the model's in-context learning capability. This approach requires the unedited model to generate correct answers based solely on the context of the new knowledge, thereby providing an accurate representation and output distribution constraint for the editing process. This implies that for smaller models less proficient in in-context learning, or for models particularly rigid regarding certain facts, the constraints provided by the LTI method may negatively impact editing performance.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "9Ctk1agZKs", "reviewer": "Reviewer_cgEK", "summary": "The paper investigates the phenomenon of Editing Overfit within large language models. Editing Overfit occurs when edited LLMs assign disproportionately high probabilities to the edit target, which negatively impacts the model's generalization ability, particularly in complex reasoning tasks. The authors introduce a new benchmark, EVOKE, designed to assess overfitting in LLMs across several challenging scenarios. To address the Editing Overfit problem, the authors propose a strategy, Learn to Inference (LTI), which encourages LLMs to recall edited knowledge in a manner more consistent with their natural inference mechanisms.", "questions": "- Does knowledge editing on larger models, such as LLaMA-2/3, still suffer from the problem of Editing Overfit? How would the proposed LTI strategy perform on these larger models? \n- Are there specific types of facts or domains (e.g., factual knowledge vs. commonsense reasoning) where the problem of Editing Overfit is more severe? Likewise, is there any specific types of knowledge where LTI may be less effective?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "- The paper clearly identifies Editing Overfit as a pervasive issue within existing LLM editing methods, providing evidence through extensive experimentation and analysis. The identification of this new key problem is one of the major contribution of this paper. \n- Besides identifying the problem, the paper also introduces the EVOKE benchmark with fine-grained evaluation metrics, allowing for a more systematic and comprehensive evaluation of the Editing Overfit problem.\n-  The paper also carefully analyzes different potential solutions and proposed the Learn to Inference (LTI) approach, an effective plug-and-play module that is highly adaptable for integration with existing knowledge editing methods. The authors also provide a detailed evaluation across multiple LLMs and editing methods, showing that LTI effectively reduces Editing Overfit without compromising editing success.", "weaknesses": "- It seems that LTI is only applicable in cases where the edited knowledge can be explicitly represented as knowledge triples (s, r, o, o*). However, more complex or nuanced knowledge editing tasks may not fit into this structured format, potentially limiting LTI's applicability in real-world scenarios. \n- The phenomenon of Editing Overfit and the proposed LTI approach are only tested on small-scale LLMs such as GPT-J and GPT-2. Extending the analysis to larger models, such as LLaMA-2/3 or other larger models, would provide a more comprehensive understanding of LTI’s effectiveness at scale.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "dgxhyEWGpb", "reviewer": "Reviewer_THc2", "summary": "The paper explores the issue of \"Editing Overfit\" in knowledge editing for large language models (LLMs), where edited models disproportionately favor edit targets, especially in complex tasks like multi-hop reasoning. To investigate this, the authors introduce EVOKE, a benchmark designed to assess overfitting in knowledge editing, and evaluate existing mitigation strategies, finding them insufficient. They propose a novel approach, Learn to Inference (LTI), which uses multi-stage constraints to guide models in recalling new knowledge similarly to unedited models, thus mitigating overfitting.", "questions": "1. How do the authors view the development of model editing versus in-context editing for the knowledge editing community, given that model editing presents challenges such as training overhead and overfitting?\n\n2. I am very curious about the performance of the proposed LTI on more recent models, such as LLaMA, Mistral, and larger-scale models.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 4, "strengths": "1. The paper's exploration of the overfitting problem in knowledge editing is both highly interesting and valuable, and the design of a dedicated benchmark provides an effective way to investigate it in depth.\n\n2. The work is highly comprehensive, identifying an unexplored issue, evaluating it within multi-hop reasoning, and proposing a plug-and-play solution based on observed phenomena.\n\n3. The writing is very clear, and the experimental design is comprehensive and well-aligned with the motivation.", "weaknesses": "1. I believe Section 5 may not be essential to the overall paper.  While it contributes some useful experimental insights with basic mitigation techniques, it does not appear closely aligned with the paper's primary contributions.  Instead, it occupies space that could be better devoted to the LTI section, which I consider more significant.\n\n2. I understand that this work focuses on the overfitting issue in model editing.  However, to my knowledge, model editing itself performs poorly on multi-hop editing tasks, likely due in large part to the overfitting phenomenon discussed.  I suggest that the authors consider addressing this perspective in the paper.\n\n3. I strongly recommend that the authors include a description of in-context editing approaches in relevant sections, such as Related Work or the Baseline. Compared to model editing, these methods are potentially better suited for handling multi-hop editing tasks. Additionally, relevant citations are missing, such as [1][2][3].\n\nIf the authors can effectively address my questions, I would be more than willing to consider raising my score.\n\n[1] Bi B, Liu S, Mei L, et al. Decoding by Contrasting Knowledge: Enhancing LLMs' Confidence on Edited Facts[J]. arXiv preprint arXiv:2405.11613, 2024.\n\n[2] Zhong Z, Wu Z, Manning C D, et al. Mquake: Assessing knowledge editing in language models via multi-hop questions[J]. arXiv preprint arXiv:2305.14795, 2023.\n\n[3] Bi B, Liu S, Wang Y, et al. Struedit: Structured outputs enable the fast and accurate knowledge editing for large language models[J]. arXiv preprint arXiv:2409.10132, 2024.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Spotlight)