Paper: Unveiling the Pitfalls of Knowledge Editing for Large Language Models (ICLR-2024)

Reviewer comment (Writing):
The “Knowledge Conflict” proposed in this paper is confusing to me.

Author response:
> **The Essence of Knowledge Conflict**

As the number of edit samples increases, we're concerned about the potential correlation between subsequent edits and those previously applied. If the editing process overlooks the correlation between edit samples and if these samples point towards different or even opposing editing targets, it could introduce conflicts into the model, resulting in a Knowledge Conflict issue. **In essence, Knowledge Conflict arises from logical overlaps in multiple edits during the editing process, leading to incomplete internal knowledge updates within the model and resulting in inconsistency.** We will further explain this definition in the following response.

Reviewer comment (Evaluation):
The “Knowledge Distortion” is a very promising research question that I pay attention to. However, the evaluation in this aspect is vague: How many the (s,r) pairs did you evaluate to calculate the results in Table 2 ? (Is there five?) How are the values in Table 2 calculated? Are they arithmetic mean values? Why is the JS divergence chosen? Is the asymmetric KL divergence inappropriate? and why?

Author response:
> **The “Knowledge Distortion” is a very promising research question that I pay attention to. However, the evaluation in this aspect is: a) vague: How many the (s,r) pairs did you evaluate to calculate the results in Table 2 ? (Is there five?) How are the values in Table 2 calculated? Are they arithmetic mean values? Why is the JS divergence chosen? Is the asymmetric KL divergence inappropriate? and why?**

We apologize for causing confusion in some implementation details. Here, we'll address your queries, and we plan to supplement related content in the main text or appendix.

**Dataset Detail:** We utilized 2500 (s, r) pairs to derive the results in Table 2, covering relations samples from 1-to-2 to 1-to-10.

**Experiment Detail:** The results in Table 2 are based on arithmetic mean values among the samples. Regarding the choice of JS divergence over KL divergence, we simply believe that when considering differences between two distributions, asymmetry is unnecessary. Additionally, computation of JS divergence is more efficient.

Reviewer comment (Experiments):
The solution to the Knowledge Distortion issue should be to develop more reasonable retrieval-based LLMs, which answer questions based on the knowledge context, rather than finetune-based Knowledge Editing methods mentioned in the article. With retrieval-based LLMs, it is only necessary to modify the knowledge facts stored in the context, allowing the LLMs to directly answer the user question according to the context. In this consideration, the contribution of this paper is limited, and it would be better to include evaluation of retrieval-based LLMs as baselines.

Author response:
As you said, retrieval-based knowledge editing methods do indeed possess stronger interpretability in many settings. **However, existing retrieval-based knowledge editing methods like SERAC [2] and IKE [3] currently lack support for generating Knowledge Conflict and settings conducive to studying Knowledge Distortion in this paper.** Our focus is on the interaction of multiple edits. SERAC, due to limitations in its training process, can only handle single edit retrieval scenarios. On the other hand, the Memory+IKE (Retrieval-based LLMs) approach incorporates multiple edit samples as in-context inputs into the model, but strictly speaking, this cannot be considered as the impact of editing methods on the model's internal structure. We believe that both of these are unable to offer additional reference value while regarded as baselines.

Reviewer comment (Novelty):
The novelty of development of Multi-Label Edit is quite low. There is a question on why not consider memory-based methods or EMA methods to alleviate the forgetting of previous knowledge stored in LLMs when it is certain that these knowledge facts are all accurate.

Author response:
1. The retrieval-based knowledge editing method offers stronger interpretability, while fine-tuning-based editing doesn’t require additional parameters or entail reasoning costs, thus holding significant practical value. However, in our actual usage, we haven't effectively integrated the advantages of retrieval-based and fine-tuning-based editing methods to address the Knowledge Distortion issues mentioned in this paper.

2. The Multi-Label Editing (MLE) method we propose emperically alleviates knowledge distortion problems. However, we can't precisely define under a specific editing sample what needs generalization transfer and what should be preserved. **Nevertheless, MLE offers an approach to tackle this issue. For instance, in the example "Joe Biden was born in California → Vienna," we can consider the differences in the knowledge structures of California and Vienna, recall their related semantics within the model [4], and select the semantics that need to be retained.**

Reviewer comment (Evaluation):
For Knowledge Conflict, it will also be a problem in retrieval-based LLMs. Clarification is needed on thoughts to solve this problem according to the experimental results.

Author response:
It's important to note that the Knowledge Conflict discussed in retrieval-based Language Model (LLMs) primarily revolves around conflicting facts retrieved by the model. This differs slightly in concept from the knowledge conflicts resulting from inadequate model knowledge updates discussed in this paper. **From the experimental results presented in this paper, we glean some insights. For instance, during the retrieval process, employing logical rules to identify factors that might lead to uncertainty of model predictions or hallucination could be beneficial.**

Reviewer comment (Theory):
What do you think is the fundamental reason for knowledge distortion during knowledge editing for LLMs? How to handle cases of one-to-many knowledge editing?

Author response:
1. The primary reason lies in the reliablity and generalization target of traditional knowledge editing evaluation methods, which only ensure the expected performance of the model on edited and similar samples but **overlook the issue of sample imbalance introduced by editing a single sample**. 
**Our proposed Multi-Label Editing (MLE) offers insights to address this issue in editing scenario of one-to-many relations.** Experimental results of MLE demonstrate empirically that by initially mining knowledge structures related to the edited sample (or some relevant tokens) from the original model, knowledge editing can alleviate distortion.

2. **However, our efforts in addressing Knowledge Distortion are not yet sufficient to propose an application scheme coherent with generalization metric for knowledge editing.** According to the definition of generalization, after applying the edit sample "Joe Biden was born in California → Vienna," the answer to the question "What country was Joe Biden born in?" would change from the USA to Austria. In this example, rather than preserving knowledge structures related to California, knowledge structures related to Vienna become more crucial. In summary, knowledge editing methods must determine the extent of generalization on specific edit samples.

Reviewer comment (Theory):
In Multi-label Edit, how to guarantee the overall conceptual hierarchy among labels?

Author response:
Apologies, we currently do not have a viable solution to maintain the conceptual hierarchy among labels. Achieving this likely requires training strategies supporting structured inputs, and there are challenges in evaluating this structurally. We hope to address this issue in future research.