Perturbation-Restrained Sequential Model Editing

## Abstract

Abstract Model editing is an emerging field that focuses on updating the knowledge embedded within large language models (LLMs) without extensive retraining.
However, current model editing methods significantly compromise the general abilities of LLMs as the number of edits increases, and this trade-off poses a substantial challenge to the continual learning of LLMs.
In this paper, we first theoretically analyze that the factor affecting the general abilities in sequential model editing lies in the condition number of the edited matrix.
The condition number of a matrix represents its numerical sensitivity, and therefore can be used to indicate the extent to which
the original knowledge associations stored in LLMs are perturbed after editing.
Subsequently, statistical findings demonstrate that the value of this factor becomes larger as the number of edits increases, thereby exacerbating the deterioration of general abilities.
To this end, a framework termed P erturbation R estraint on U pper bou N d for E diting (PRUNE) is proposed, which applies the condition number restraints in sequential editing.
These restraints can lower the upper bound on perturbation to edited models, thus preserving the general abilities.
Systematically, we conduct experiments employing three popular editing methods on three LLMs across four representative downstream tasks.
Evaluation results show that PRUNE can preserve considerable general abilities while maintaining the editing performance effectively in sequential model editing.
The code and data are available at https://github.com/mjy1111/PRUNE .

## 1 Introduction

Despite the remarkable capabilities of large language models (LLMs), they encounter challenges such as false or outdated knowledge, and the risk of producing toxic content [, , , ].
Given the prohibitively high cost of retraining LLMs to address these issues, there has been a surge in focus on *model editing* [, , , , , , , , ], which aims at updating the knowledge of LLMs cost-effectively.
Existing model editing methods can be roughly classified into either *parameter-modifying* methods  [, , ] that directly modify a small subset of model parameters, or *parameter-preserving* methods [, ] that integrate additional modules without altering the model parameters.
In this paper, we study the parameter-modifying editing methods.

Sequential model editing involves making successive edits to the same model over time to continuously update knowledge, as illustrated in Figure 1(a).
Recent studies [, , ] indicate that parameter-modifying editing methods significantly compromise the general abilities of LLMs as the number of edits increases, such as summarization, question answering, and natural language inference.
However, these studies neither provide a theoretical analysis of the bottleneck of the general abilities of the edited models, nor propose a solution to preserve these abilities in sequential editing.
These affect the scalability of model editing and pose a substantial challenge to the continual learning of LLMs.

Figure: Figure 1: (a) Illustration of sequential model editing. (b) The condition number of edited matrix rapidly increases as the number of edits increases. (c) Comparison of general downstream task performance before editing, after regular editing, and after restrained editing by PRUNE. (d) Comparison of editing performance after regular editing and after restrained editing by PRUNE. $f_{W}$, $f_{W_{n}}$ and $f_{\overline{W}_{n}}$ denote the models that are unedited, regularly edited $n$ times, and restrainedly edited by PRUNE respectively. $W$ is denoted as a matrix to be edited.
Refer to caption: x1.png

In light of the above issues, we first theoretically analyze through matrix perturbation theory [, , ] to elucidate a crucial factor affecting the general abilities during sequential editing: the *condition number* [, , ] of the edited matrix.
The condition number of a matrix represents its numerical sensitivity and therefore can be used to indicate the extent to which the original knowledge associations stored in LLMs are perturbed after editing.
As shown in Figure 1(b), our statistical findings demonstrate that the condition number of the edited matrix substantially increases as the number of edits increases, thereby exacerbating the perturbation of original knowledge and the deterioration of general abilities.
Therefore, we assume that the bottleneck of the general abilities during sequential editing lies in the escalating value of the condition number.

Towards continual and scalable model editing, we propose Perturbation Restraint on Upper bouNd for Editing (PRUNE) based on the above analysis, which applies the condition number restraints in sequential editing to preserve general abilities and maintain new editing knowledge simultaneously.
Specifically, the condition number of the edited matrix is restrained by reducing the large singular values [, ] of the edit update matrix.
Consequently, the upper bound on perturbation to the edited matrix is lowered, thus reducing the perturbation to the original knowledge associations and preserving the general abilities of the edited model, as shown in Figure 1(c).
Additionally, we observe that these larger singular values often encapsulate redundant editing overfitting information, so regularizing them will not affect the newly editing knowledge, as shown in Figure 1(d).
In this way, the new editing knowledge is embedded into LLMs without affecting their original general abilities.
Overall, the proposed editing framework requires only minimal computing resources, and is adaptable to be coupled with multiple existing editing methods.

To validate the effectiveness of the proposed PRUNE, our study comprehensively
evaluates the edited LLMs for both general abilities and editing performance in sequential editing scenarios.
Extensive empirical research involves three popular editing methods, including
MEND [],
ROME [], and
MEMIT [], which are analyzed based on three representative LLMs including
GPT-2 XL (1.5B) [],
LLaMA-2 (7B) [], and
LLaMA-3 (8B).
Four representative downstream tasks including
reasoning [], summarization [],
open-domain QA [], and
natural language inference []
are employed to extensively demonstrate the impact of model editing on the general abilities of LLMs.
Experimental results demonstrate that the proposed PRUNE can preserve considerable general abilities and maintain almost all editing performance in sequential editing.

In essence, our research offers three significant contributions:
(1) This study theoretically analyzes that the escalating value of the condition number of the edited matrix is the bottleneck of sequential model editing.
(2) The PRUNE framework based on the analysis is proposed to preserve the general abilities of the edited model while retaining the editing knowledge.
(3) Experimental results including both editing performance and four downstream task performance across three editing methods on three LLMs demonstrate the effectiveness of the proposed method.

## 2 Related Work

#### Model Editing Methods

From the perspective of whether the model parameters are modified,
existing editing methods can be divided into *parameter-modifying*  [, , , ] and *parameter-preserving* methods [, , ].
This paper focuses on the former.
Previous works have investigated the role of MLP layers in Transformer, showing that MLP layers store knowledge, which can be located in specific neurons and edited [, , ].
KE [] and MEND [] train a hypernetwork to get gradient changes to update model parameters [].
Besides, Meng et al. [] and Meng et al. [] used Locate-Then-Edit strategy, which first located multi-layer perceptron (MLP) storing factual knowledge, and then edited such knowledge by injecting new key-value pair in the MLP module.
Parameter-preserving methods do not modify model weights but store the editing facts with an external memory.
For example, Mitchell et al. [] stored edits in a base model and learned to reason over them to adjust its predictions as needed.

#### Model Editing Evaluation

Some works investigate the paradigm for model editing evaluation [, , , , , , , ].
Cohen et al. [] introduced the ripple effects of model editing, suggesting that editing a particular fact implies that many other facts need to be updated.
Ma et al. [] constructed a new benchmark to assess the edited model bidirectionally.
Besides, Li et al. []
explored two significant areas of concern: Knowledge Conflict and Knowledge
Distortion.
These early studies mainly evaluate
edited models per edit rather than sequentially, and they focus narrowly on basic factual triples.
Recently, some works assess the impact
of editing methods on the general abilities of LLMs in sequential editing scenarios.
These studies [, , ] have conducted comprehensive experiments, showing the parameter-modifying methods significantly degrade the model performance on general downstream tasks.

#### Matrix Perturbation Theory

It plays a crucial role in the field of artificial intelligence (AI) by providing a systematic framework to understand the impact of small changes or perturbations in various AI algorithms and models.
Some studies [, , ] delve into the interpretability of LLMs, revealing how minor alterations in input features or model parameters influence the model’s predictions.
This understanding helps uncover significant feature connections within the model architecture.
Moreover, it has been instrumental in assessing and enhancing the robustness of models [, , ].
Furthermore, Bird et al. [] and Dettmers et al. [] have employed it for sensitivity analysis to identify critical factors affecting algorithm performance.
It also contributes to the development of efficient optimization techniques [, , ], improving convergence rates and stability of optimization algorithms.

Compared with previous works [, , , , , ] that are the most relevant, a main difference should be highlighted.
They neither theoretically investigate the reasons for general ability degradation, nor propose methods to maintain these abilities during sequential editing.
In contrast, our study makes the first attempt to theoretically explore the bottleneck of general abilities in sequential editing and proposes the PRUNE framework to preserve these abilities for continual model editing.

## 3 Analysis on Bottleneck of Sequential Model Editing

### 3.1 Preliminary

#### Model Editing

This task involves modifying the memorized knowledge contained in LMs.
Various kinds of complex learned beliefs such as logical, spatial, or numerical knowledge are expected to be edited.
In this paper, following previous work [, , , ], we study editing factual knowledge in the form of (subject *s*, relation *r*, object *o*), e.g., (*s = United States, r = President of, o = Donald Trump*).
An LM is expected to recall a memory representing $o$ given a natural language prompt $p(s,r)$ such as “*The President of the United States is*”.
Editing a fact is to incorporate a new knowledge triple $(s,r,o^{*})$ in place of the current one $(s,r,o)$.
An edit is represented as $e=(s,r,o,o^{*})$ for brevity.
Given a set of editing facts $\mathcal{E}=\left\{e_{1},e_{2},\ldots\right\}$ and an original model $f_{\theta_{0}}$, sequential model editing operationalizes each edit after the last edit(^1^11This paper studies editing a single fact at a time and leaves the exploration of batch editing as future work.), i.e., $K(f_{\theta_{n-1}},e_{n})=f_{\theta_{n}}$, where $f_{\theta_{n}}$ denotes the model after $n$ edits.

#### Singular Value Decomposition

SVD [] is a fundamental and effective matrix factorization technique for analyzing matrix structures.
Formally, an SVD of a matrix $W\in\mathbb{R}^{p\times q}$ is given by $W=U\Sigma V^{\mathrm{T}}$, where
$U=[u_{1},u_{2},...,u_{p}]\in\mathbb{R}^{p\times p}$,
$V=[v_{1},v_{2},...,v_{q}]\in\mathbb{R}^{q\times q}$, and $\Sigma\in\mathbb{R}^{p\times q}$.
$u_{i}$ and $v_{i}$ are the column vectors of $U$ and $V$, and constitute an orthonormal basis of $\mathbb{R}^{p}$ and $\mathbb{R}^{q}$ respectively.
$\Sigma$ is a diagonal matrix whose diagonal entries are given by the singular values of $W$ in descending order.
Additionally, the SVD of $W$ could also be formulated as: $W=\sum_{i=1}^{\min\{p,q\}}\sigma_{i}u_{i}v_{i}^{\mathrm{T}}$, where $\sigma_{i}$ is singular value, and $\sigma_{1}\geq\sigma_{2}\geq...\geq\sigma_{\min\{p,q\}}\geq 0$.
In the scenario of this paper, $W$ is a full-rank matrix, so $\sigma_{\min\{p,q\}}>0$.

### 3.2 Matrix Perturbation Theory Analysis

Previous works [, , , ] have analyzed and located that the MLP modules in Transformer [] store various kinds of knowledge [, ].
The MLP module of the $l$-th Transformer layer consists of two projection layers, where the first and second layers are denoted as $W_{fc}^{l}$ and $W_{proj}^{l}$ respectively.
$W_{proj}^{l}$ is considered as a linear associative memory which stores knowledge in the form of key-value pairs $(k_{i},v_{i})$,
and is usually regarded as the editing area [, ].
In this paper, $W_{proj}^{l}$ is denoted as $W$ for brevity.
$W$ is assumed to store many key-value pairs $P=\{(k_{i},v_{i})\mid i=1,2,...\}$ which satisfies $Wk_{i}=v_{i}$, where $k_{i}\in\mathbb{R}^{q}$ and $v_{i}\in\mathbb{R}^{p}$.
Assuming $\mid\mathcal{E}\mid=N$ in sequential model editing, an edit update matrix $\Delta W_{j}$ is calculated for the edit $e_{j}$ and added to $W$, which can be formulated as: $W_{N}=W+\sum_{j=1}^{N}\Delta W_{j}$ with $\Delta W_{j}$ calculated from $f_{\theta_{j-1}}$.

#### Problem Modeling

To explore the reasons for the general ability degradation of edited models, we begin by noting that
most of the key-value pairs of $P$ correspond to facts unrelated to editing.
For the sake of analysis, only the matrix $W$ of a single layer is assumed to be modified.
We intuitively hypothesize that for the facts that are irrelevant to the editing fact, the cumulative modifications applied during sequential model editing may lead to significant mismatches in the associations between the original key-value pairs $P$.
Specifically, consider a key-value pair $(k_{i},v_{i})\in P$.
After applying an edit $e_{j}$ that generates $\Delta W_{j}$ and adding it to $W$, if the extracted value $v_{i}$ remains unchanged, the corresponding key $k_{i}$ needs to be adjusted with an adjustment denoted as $\Delta k^{j}_{i}$.
Mathematically, this can be represented as(^2^22As $W_{j}\in\mathbb{R}^{p\times q}$, and we observed $p<q$ in LLMs, so there will be $\Delta k^{j}_{i}$ that satisfies this formula.) $W_{N}(k_{i}+\sum_{j=1}^{N}\Delta k^{j}_{i})=v_{i}$ after $N$ edits.
However, during the editing process, it’s challenging to guarantee such adjustments completely, leading to inaccuracies in the knowledge extracted from the edited model.
To delve deeper, let’s analyze how the key $k_{i}$ changes (i.e., $\sum_{j=1}^{N}\Delta k^{j}_{i}$) when its corresponding value $v_{i}$ remains unchanged after $N$ edits.

#### Perturbation Analysis of Single Edit

According to matrix perturbation theory [, , ], the edit update matrix $\Delta W$ from an edit can be regarded as a perturbation(^3^33We obtained some $\Delta W_{j}$ and found $\|\Delta W_{j}\|\ll\|W\|$, which satisfies the definition of perturbation.) for $W$, so we first analyze the situation where $W\in\mathbb{R}^{p\times q}$ is appended with a perturbation $\Delta W$.
Define $W^{\dagger}$ is the generalized inverse [] of $W$, $\|\ast\|$ represents 2-norm, and $\tilde{W}=W+\Delta W$.

Theorem 3.1
Consider $Wk=v$, there exists $\Delta k$ such that $\tilde{k}=k+\Delta k$ satisfies $\tilde{W}\tilde{k}=v$.
Let $k=W^{\dagger}v$ and $\tilde{k}=\tilde{W}^{\dagger}v$, and $\Delta W$ is an acute perturbation of $W$. Then:

$$ $\frac{\|\Delta k\|}{\|k\|}=\frac{\|k-\tilde{k}\|}{\|k\|}\leq\hat{\kappa}\frac{ \|\Delta E_{11}\|}{\|W\|}+\Psi_{2}\left(\frac{\hat{\kappa}\Delta E_{12}}{\|W\| }\right)+\hat{\kappa}^{2}\frac{\|\Delta E_{12}\|}{\|W\|}\left(\eta^{-1}g(v)+ \frac{\|\Delta E_{21}\|}{\|W\|}\right),$ (1) $$

where $\Delta E_{11}$, $\Delta E_{12}$, and $\Delta E_{21}$ are directly related to $\Delta W$.
$\Psi_{2}(F)$ is a monotonically increasing function of $\|F\|$ and $g(v)$ is a function about $v$.
$\hat{\kappa}=\|W\|\|\tilde{W}_{11}^{-1}\|$, where $\tilde{W}_{11}$ is square and related to the reduced form of $W$.
Each term on the right-hand side involves $\hat{\kappa}$,
which means that the upper bound on the perturbation of the vector $k$ is constrained by $\hat{\kappa}$.
Readers can refer to Appendix A.3 for the details and proof of this theorem.
However, calculating $\|\tilde{W}_{11}^{-1}\|$ involves the reduced form of $W$, which incurs unnecessary additional overhead. Therefore, we consider the following theorem and give an alternative estimation.

Theorem 3.2
Let $\kappa=\|W\|\|{W}^{\dagger}\|$,
and suppose that $\gamma\equiv 1-\frac{\kappa\|\Delta E_{11}\|}{\|W\|}>0$.
Then:

$$ $\|\tilde{W}^{\dagger}\|\leq\frac{\|W^{\dagger}\|}{\gamma}.$ (2) $$

According to Theorem 3.2, $\|\tilde{W}^{-1}_{11}\|\leq\frac{\|W_{11}^{-1}\|}{\gamma}=\frac{\|W^{\dagger}
\|}{\gamma},$ so $\hat{\kappa}\leq\frac{\kappa}{\gamma}$.
Here $\kappa=\|W\|\|{W}^{\dagger}\|=\frac{\sigma_{max}}{\sigma_{min}}$ is the condition number of $W$, where $\sigma_{\text{max}}$ and $\sigma_{\text{min}}$ are the maximum and minimum singular values of $W$, respectively.
Combining Theorem 3.1, we know that the larger $\kappa$ is, the greater the upper bound on the perturbation of the vector $k$.
Readers can refer to Appendix A for the full theoretical analysis.

### 3.3 Change Trend of Condition Number

As mentioned above, we have analyzed that the condition number of the edited matrix can be used to indicate the upper bound on the perturbation of the key-value pair associations by a single edit.
In order to explore the impact of sequential model editing on these associations, the change trend of the condition number of the edited matrix during sequential editing is illustrated in Figure 2.

Surprisingly, we observed that regardless of the editing methods employed, the condition number of the edited matrix exhibited a rapid increase as the number of edits increased, especially after a large number of edits.
According to Theorem 3.1, the adjustment norm $\|\Delta k^{n}_{i}\|_{2}$ corresponding to the $n$-th edit tends to increase as the number of edits $n$ increases.
It underscores that when the number of edits is relatively large, the upper bound on the perturbation caused by subsequent edits to the key-value pair associations becomes very large, further disrupting the stored original knowledge and exacerbating the deterioration of general abilities.

Figure: Figure 2: The condition number, maximum singular value and minimum singular value of the edited matrix in sequential editing. Three editing methods including ROME, MEND, and MEMIT are used to edit LLaMA-2 (7B) on the CounterFact [] dataset. For editing methods that modify the parameters of multiple MLP layers, one of them is randomly selected for illustration. $W$ and $W_{n}$ denote the unedited and edited matrices respectively.
Refer to caption: x2.png

## 4 PRUNE: P erturbation R estraint on U pper bou N d for E diting

According to the analysis in Section 3, the bottleneck of the general abilities during sequential editing lies in the escalating value of the condition number.
In this section, a framework termed Perturbation Restraint on Upper bouNd for Editing (PRUNE) is proposed, which applies the condition number restraints to preserve general abilities and maintain new editing knowledge simultaneously.

**Table 1: The maximum singular values of $\sum_{j=1}^{N}\Delta W_{j}$ with three edting methods. Other settings are the same as those illustrated in Figure 2.**
| Edits ($N$) | ROME | MEMIT | MEND |
| --- | --- | --- | --- |
| 10 | 7.25 | 7.46 | 14.08 |
| 50 | 11.38 | 15.63 | 75.53 |
| 100 | 15.62 | 23.39 | 127.89 |
| 200 | 57.61 | 935 | 191.04 |

#### Principle

Given an edited matrix with $N$ edits, $W_{N}=W+\sum_{j=1}^{N}\Delta W_{j}$, as shown in Figure 2, its maximum singular value is constantly increasing, while the minimum singular value is basically unchanged as the number of edits $N$ increases.
This directly leads to the increasing condition number of the edited matrix.
Therefore, our motivation is to restrain the maximum singular value of the edited matrix to lower the upper bound on the perturbation.
If we directly perform SVD operation on $W_{N}$ and reduce its singular values, the original $W$ will be inevitably destroyed.
Consequently, an analysis of the singular values of $\sum_{j=1}^{N}\Delta W_{j}$ is conducted, and the results in Table 1 present that its maximum singular value becomes very large when $N$ is large.
We assume that this is the main reason why the maximum singular value of $W_{N}$ is large, our method therefore aims to restrain the large singular values of $\sum_{j=1}^{N}\Delta W_{j}$.

#### Design

Firstly, SVD is operated on the original $W$ and $\sum_{j=1}^{N}\Delta W_{j}$ respectively as:

$$ $W=\sum_{i=1}^{\min\{p,q\}}\sigma_{i}u_{i}v_{i}^{\mathrm{T}},\quad\quad\sum_{j= 1}^{N}\Delta W_{j}=\sum_{i=1}^{\min\{p,q\}}\hat{\sigma}_{i}\hat{u}_{i}\hat{v}_ {i}^{\mathrm{T}}.$ (3) $$

This paper considers $W$ to be the main part, and any singular value in $\sum_{j=1}^{N}\Delta W_{j}$ should be ensured not to obviously exceed the maximum singular value of $W$.
Subsequently, if any singular value $\hat{\sigma}_{i}$ of $\sum_{j=1}^{N}\Delta W_{j}$ is greater than the maximum singular value of $W$, it will be restrained with a function $F$, otherwise it remains unchanged, which could be formulated as:

$$ $\overline{\sigma}_{i}=\left\{\begin{array}[]{ll}F(\hat{\sigma}_{i}),&\text{ if }\hat{\sigma}_{i}>max\{\sigma_{i}\},\\ \hat{\sigma}_{i},&\text{ if }\hat{\sigma}_{i}\leq max\{\sigma_{i}\}.\end{array }\right.$ (4) $$

$$ $F(\hat{\sigma}_{i})=\operatorname{log}_{\alpha}(\hat{\sigma}_{i})- \operatorname{log}_{\alpha}(max\{\sigma_{i}\})+max\{\sigma_{i}\}.$ (5) $$

In the main paper, we use the $\operatorname{log}$ function in $F$ to restrain $\hat{\sigma}_{i}$.
Here ${\alpha}$ is a hyperparameter to control the degree of restraints, readers can refer to Appendix B.3 for its details for experiments.
Besides, we also provide the definition and results of $\operatorname{linear}$ function in Appendix C.3.
Finally, we obtain the restrained edited matrix $\overline{W}_{N}$ to replace ${W}_{N}$:

$$ $\overline{W}_{N}=W+\sum_{i=1}^{\min\{p,q\}}\overline{\sigma}_{i}\hat{u}_{i} \hat{v}_{i}^{\mathrm{T}}.$ (6) $$

In this way, the condition number of the edited matrix is reduced (see Appendix C.4) and the upper bound on perturbation is significantly restrained.

## 5 Experiments

In this section, both the downstream task performance
and editing performance of three editing methods on three LLMs were evaluated in sequential model editing.
The proposed PRUNE was plug-and-play which can be coupled with these editing methods.

### 5.1 Base LLMs and Editing Methods

Experiments were conducted on three LLMs including
GPT-2 XL (1.5B) [], LLaMA-2 (7B) [] and LLaMA-3 (8B)(^4^44https://llama.meta.com/llama3/).
Three popular editing methods were selected as the baselines including
MEND [],
ROME [], and MEMIT [].
Readers can refer to Appendix B.1 for the details of these editing methods.

### 5.2 Editing Datasets and Evaluation Metrics

Two popular model editing datasets Zero-Shot Relation Extraction (ZsRE) [] and
CounterFact [] were
adopted in our experiments.
ZsRE is a QA dataset using question rephrasings generated by back-translation as the equivalence neighborhood.
A key distinction between
CounterFact and ZsRE datasets is that ZsRE
contains true facts, while CounterFact contains counterfactual
examples where the new target has a lower probability
when compared to the original answer [].
Readers can refer to Appendix B.2 for examples of each dataset.

To assess the editing performance of editing methods, following previous works, three fundamental metrics were employed: efficacy, generalization and locality [, , , ].
Given an original model $f_{\theta_{0}}$, an edited model $f_{\theta_{n}}$ with $n$ times sequential editing.
Each edit $e_{i}=(s_{i},r_{i},o_{i},o_{i}^{*})$ has an editing prompt $p_{i}$, paraphrase prompts $\mathcal{P}^{G}_{i}$, and locality prompts $\mathcal{P}^{L}_{i}$.

Efficacy validates whether the edited models could recall the editing fact under editing prompt $p_{i}$.
The assessment is based on Efficacy Score (ES) representing as: $\mathbb{E}_{i}[\mathbbm{1}[\,P_{f_{\theta_{n}}}(o^{*}_{i}\,|\,p_{i})>P_{f_{
\theta_{n}}}(o_{i}\,|\,p_{i})]\,]$, where $\mathbbm{1}$ is the indicator function.

Generalization verifies whether the edited models could recall the editing fact under the paraphrase prompts $\mathcal{P}^{G}_{i}$ via Generalization Score (GS):
$\mathbb{E}_{i}\,[\,\mathbb{E}_{p\in\mathcal{P}^{G}_{i}}[\mathbbm{1}[\,P_{f_{
\theta_{n}}}(o^{*}_{i}\,|\,p)>P_{f_{\theta_{n}}}(o_{i}\,|\,p)]\,]$.

Locality verifies whether the output of the edited models
for inputs out of editing scope remains unchanged under the locality prompts $\mathcal{P}^{L}_{i}$ via Locality Score (LS):
$\mathbb{E}_{i}\,[\,\mathbb{E}_{p_{l}\in\mathcal{P}^{L}_{i}}[\mathbbm{1}[\,P_{f
_{\theta_{n}}}(o_{l}\,|\,p_{l})>P_{f_{\theta_{n}}}(o^{*}_{i}\,|\,p_{l})]\,]\,]$, where $o_{l}$ was the original answer of $p_{l}$.

Different from previous studies that assess the edited models after each individual edit [, ], this paper evaluated whether the final edited models after completing all edits can still recall all preceding edits, which is more challenging and common in real-world.

### 5.3 Downstream Tasks, Datasets and Metrics

To explore the side effects of sequential model editing on the general abilities of LLMs, four representative tasks with corresponding datasets were adopted for assessment, including:

Reasoning on the GSM8K [], and the results were measured by solve rate.

Summarization on the SAMSum [], and the results were measured by the average of ROUGE-1, ROUGE-2 and ROUGE-L following Lin [].

Open-domain QA on the Natural Question [], and the results were measured by exact match (EM) with the reference answer after minor normalization as in Chen et al. [] and Lee et al. [].

Natural language inference (NLI) on the RTE [], and the results were measured by accuracy of two-way classification.

For each dataset, some examples were randomly sampled for evaluation.
Details of prompts for each tasks were shown in Appendix B.4.

Figure: Figure 3: The downstream task performance (%) of models edited by three editing methods with LLaMA-2 (7B) on the CounterFact dataset. The dashed lines refer to the results of the unrestrained editing methods. The solid lines refer to the results of the editing methods coupled with the proposed PRUNE framework. Statistical significance tests were performed to demonstrate that the improvement in PRUNE compared to baseline was statistically significant (t-test with $p$-value <$0.05$).
Refer to caption: x5.png

### 5.4 Results of General Abilities

Figure 3 illustrates the downstream task performance of edited models with LLaMA-2 (7B) on the CounterFact dataset.
Due to page limitation, results of other LLMs and datasets
were put in Appendix C.1.
These results were analyzed from the following perspectives.

Current editing methods significantly compromised general abilities.
As depicted by the dashed lines of Figure 3, both the ROME and MEMIT methods initially maintained relatively stable performance in downstream tasks when the number of edits was small ($\leq$ 100).
However, as the number of edits surpassed 200, a noticeable decline in performance was observed across all tasks for both methods.
Additionally, the MEND method exhibited significant performance degradation after just 20 sequential edits, indicating its inadequacy as a sequential model editing method.
Furthermore, when comparing LLMs of different sizes, a general trend emerged: larger models suffered more pronounced compromises in their general abilities when subjected to the same number of edits.
For instance, with 300 edits, MEMIT’s performance on GPT2-XL remained largely unchanged, whereas it dwindled to nearly 0 on LLaMA-2 and LLaMA-3.

The performance decline was gradual initially but accelerated with increasing edit count.
This trend aligned with the fluctuation observed in the size of the condition number, as depicted in Figure 2.
When the number of edits was small, the condition number was small, and each new edit introduced relatively minor perturbations to the model.
However, as the number of edits increased, the condition number underwent a substantial increase.
Consequently, each subsequent edit exerted a significant perturbation on the model, leading to a pronounced impairment of its general abilities.
These results substantiated the analysis presented in Section 3.3.

The proposed PRUNE can preserve considerable general abilities.
As shown by the solid lines of Figure 3, when MEMIT was coupled with PRUNE and subjected to 200 edits, its downstream tasks performance remained close to that of the unedited model.
However, for the unrestrained MEMIT, downstream task performance had plummeted to nearly 0 by this point.
This consistent trend was also observed with ROME and MEND.
Nevertheless, for models edited using the unrestrained MEND method, performance degradation was stark after just 20 edits.
Even with the addition of PRUNE, preservation could only be extended up to 40 edits.
This suggests that while PRUNE effectively preserves general abilities, it does have an upper limit determined by the unrestrained editing method.

Figure: Figure 4: The editing performance (%) of three editing methods with LLaMA-2 (7B) on the CounterFact dataset. The dashed lines refer to the results of the unrestrained editing methods. The solid lines refer to the results of the editing methods coupled with the proposed PRUNE. Statistical significance tests were performed to demonstrate that the improvement in PRUNE compared to baseline was statistically significant (t-test with $p$-value <$0.05$).
Refer to caption: x6.png

### 5.5 Results of Editing Performance

Figure 4 shows different metrics used for measuring the editing performance of edited models with LLaMA-2 (7B) on the CounterFact dataset.
Other results across models and datasets were put in Appendix C.2.
Three conclusions can be drawn here.

Previous editing facts were forgotten as the number of edits increased.
As shown by the dashed lines of Figure 4, the decline in efficacy and generalization suggests that in sequential editing scenarios, post-edited models gradually forget knowledge acquired from previous edits after a few iterations. Comparing these editing methods, we also observed a notable drop in efficacy and generalization after hundreds of edits with ROME and MEMIT, whereas these values decreased significantly after only 30 edits with MEND.
This indicates that in sequential editing scenarios, the MEND method struggled to successfully integrate new knowledge into LLMs after several edits.

Unrelated facts were perturbed as the number of edits increased.
The locality metric served as an indicator of perturbation for unrelated facts.
It became evident that for each editing method, the locality decreased significantly.
Additionally, an observation emerged: when the locality of the edited model was low, the performance of downstream tasks was also low.
This observation underscores that perturbations of irrelevant knowledge compromise the general abilities of the edited model.

PRUNE can effectively maintain the editing performance.
This is shown by the solid lines of Figure 4 and could be analyzed from two aspects.
On the one hand, when the number of edits was small, the editing performance of each editing method coupled with PRUNE was about the same as the unrestrained method.
On the other hand, it significantly mitigated the forgetting of editing facts and the perturbation of irrelevant facts when the number of edits was large
during the sequential editing.
Specifically, when the number of edits reached 150, the editing performance of MEMIT was very low.
But when coupled with PRUNE, its performance remained relatively stable.

### 5.6 Editing Facts Forgetting Analysis

Figure: Figure 5: 2-dimensional PCA visualization of first 100 values. The model was edited by ROME with LLaMA-3 (8B) on the CounterFact dataset.
Refer to caption: x7.png

In section 3, the analysis was conducted to elucidate the reasons behind the degradation in general abilities with an increasing number of edits.
Subsequent experiments quantitatively demonstrated the effectiveness of the proposed PRUNE.
Here, we delve into qualitative analysis to explain why editing facts are forgotten and how PRUNE can mitigate this forgetting.

Initially, a set of editing facts $\mathcal{E}=\left\{e_{1},e_{2},\ldots\right\}$ was collected, where $\mid\mathcal{E}\mid=200$. ROME was employed for analysis, and the original matrix was defined as $W$.
During sequential editing, ROME computed key-value pairs $(k^{e}_{j},v^{e}_{j})$ of *the last subject token* to generate $\Delta W_{j}$ for each edit $e_{j}$ to incorporate new facts, satisfying the equation: $W_{j}\cdot k^{e}_{j}=v^{e}_{j}$.
However, when evaluating editing performance, the edited model obtained from the last edit was utilized, thus computing values(^5^55Since ROME only modifies one matrix, the $k^{e}_{j}$ remains the same across these edited models.): $W_{200}\cdot k^{e}_{j}=\hat{v}^{e}_{j}$.
After adopting PRUNE to ROME, this equation became $\overline{W}_{200}\cdot k^{e}_{j}=\overline{v}^{e}_{j}$.
We hypothesized that if $\hat{v}^{e}_{j}$ was similar to $v^{e}_{j}$, the editing fact $e_{j}$ could be maintained.

Denote $V_{Current}=\{v^{e}_{j}\}$, $V_{Editing}=\{\hat{v}^{e}_{j}\}$, and $V_{Prune}=\{\overline{v}^{e}_{j}\}$.
Specifically, these corresponding values of the first 100 edits were used, as they are more prone to be forgotten than the last 100.
Principal Component Analysis (PCA) [] was employed to visualize these values.
The first two principal components of each value were calculated and illustrated, as they can represent most of its features [].
As shown in Figure 5, on the one hand, the discrepancy between the principal components of $V_{Current}$ and $V_{Editing}$ was markedly large.
This indicates that after 200 edits to the model, the values corresponding to the first 100 facts stored in the edited matrix are severely corrupted, leading to significant forgetfulness.
On the other hand, after adopting PRUNE, the discrepancy between the principal components of $V_{Current}$ and $V_{Prune}$ was small.
This demonstrates that PRUNE effectively maintains the values and mitigates the forgetting of editing facts.

## 6 Conclusion

In this paper, a theoretical analysis is firstly conducted to elucidate that the bottleneck of the general abilities
during sequential editing lies in the escalating value of the condition number.
Subsequently, a plug-and-play framework called PRUNE is proposed to apply restraints to preserve general abilities and maintain new editing knowledge simultaneously.
Comprehensive experiments on various editing methods and LLMs demonstrate the effectiveness of this method.
We aspire that our analysis and method will catalyze future research on continual model editing.

## Limitations & Future Work

The limitations of our work are discussed as follows.
Firstly, while this paper focuses on editing a single fact at a time in sequential model editing, some studies have explored updating hundreds or thousands of facts simultaneously in batch editing. Therefore, investigating batch-sequential editing could enhance the scalability of model editing and remains further research.
Secondly, for the experimental settings, it is necessary to explore the performance of larger-size models and more editing methods on more downstream tasks.
Thirdly, the current focus of editing knowledge primarily revolves around factual knowledge. However, it is also important to investigate whether editing other types of knowledge will affect general abilities and whether PRUNE is effective in this situation.
Additionally, the proposed PRUNE is only applied once after the completion of the last edit.
But it could also be utilized multiple times during the sequential editing process, and we intuitively believe that this approach would be more conducive to preserving the general abilities of the model.
These aspects are yet to be fully understood
and warrant a more comprehensive study.

## Appendix A Theoretical Analysis Based on Perturbation Theory

Here, we provide a detailed analysis and proof of Section 3.2. We begin by introducing some definitions and then present several preliminary lemmas and theorems. These lemmas and theorems are finally used to prove Theorem 3, which is most relevant to our problem discussed in Section 3.2.

### A.1 Definition

We discuss the problem $Ax=b$, where $\tilde{A}$ is a perturbation of $A$ given by $\tilde{A}=A+E$. We assume $b$ remains unchanged and $\tilde{x}$ represents the corresponding change, satisfying $\tilde{A}\tilde{x}=b$.
Here $A\in\mathbb{C}^{m\times n}$, $b\in\mathbb{C}^{m}$.

It is noteworthy that in the following derivation, $A^{H}$ denotes the conjugate transpose of $A$, $A^{\dagger}$ represents the generalized inverse of $A$,
and $\|\ast\|$ represents 2-norm [].

To simplify the problem, we apply a rotation. Specifically, let $V=(V_{1}\ V_{2})$ be a unitary matrix with $R(V_{1})=R(A^{H})$, and let $U=(U_{1}\ U_{2})$ be a unitary matrix with $R(U_{1})=R(A)$, where $R$ refers to the rank.
Then

$$ $U^{H}AV=\begin{pmatrix}U_{1}^{H}AV_{1}&U_{1}^{H}AV_{2}\\ U_{2}^{H}AV_{1}&U_{2}^{H}AV_{2}\end{pmatrix}=\begin{pmatrix}A_{11}&0\\ 0&0\end{pmatrix},$ (7) $$

where $A_{11}$ is square and nonsingular. If we set

$$ $U^{H}EV=\begin{pmatrix}U_{1}^{H}EV_{1}&U_{1}^{H}EV_{2}\\ U_{2}^{H}EV_{1}&U_{2}^{H}EV_{2}\end{pmatrix}=\begin{pmatrix}E_{11}&E_{12}\\ E_{21}&E_{22}\end{pmatrix},$ (8) $$

then

$$ $U^{H}\tilde{A}V=\begin{pmatrix}A_{11}+E_{11}&E_{12}\\ E_{21}&E_{22}\end{pmatrix}=\begin{pmatrix}\tilde{A}_{11}&E_{12}\\ E_{21}&E_{22}\end{pmatrix}.$ (9) $$

We will call these transformed, partitioned matrices the reduced form of the problem. Many statements about the original problem have revealing analogues in the reduced form.

In this form, $x$ is replaced by $V^{H}x$ and $b$ is replaced by $U^{H}b$. If $x$ and $b$ are partitioned in the forms

$$ $x=\begin{pmatrix}x_{1}\\ x_{2}\end{pmatrix},\quad b=\begin{pmatrix}b_{1}\\ b_{2}\end{pmatrix},$ (10) $$

where $x_{1},b_{1}\in\mathbb{C}^{r}$, then

$$ $x_{1}=A_{11}^{-1}b_{1}$ (11) $$

and

$$ $x_{2}=0.$ (12) $$

Moreover, the norm of the residual vector

$$ $r=b-Ax$ (13) $$

is given by

$$ $\|r\|=\|b_{2}\|.$ (14) $$

Here, we define the symbol $\eta$:

$$ $\eta=\frac{\|A\|\|x\|}{\|b\|},$ (15) $$

and for any $F\in\mathbb{C}^{k\times r}$ ($k\geq r$) the symbol $\Psi(F)$, for the spectral norm:

$$ $\Psi_{2}(F)=\frac{\|F\|}{(1+\|F\|^{2})^{1/2}}.$ (16) $$

### A.2 Preliminary Lemmas & Theorems

After introducing some definitions, we give some preliminary lemmas and theorems, which are used to prove Theorem 3.

Lemma 1 Let

$$ $\kappa(A)=\|A\|\|A^{-1}\|$ $$

be the condition number of $A$. If $\tilde{A}$ is nonsingular, then

$$ $\frac{\|\tilde{A}^{-1}-A^{-1}\|}{\|\tilde{A}^{-1}\|}\leq\kappa(A)\frac{\|E\|}{ \|A\|}.$ (17) $$

If in addition

$$ $\frac{\|E\|}{\|A\|}\kappa(A)<1,$ $$

then $\tilde{A}$ is perforce nonsingular and

$$ $\|\tilde{A}^{-1}\|\leq\frac{\|A^{-1}\|}{1-\kappa(A)\frac{\|E\|}{\|A\|}}.$ (18) $$

Moreover

$$ $\frac{\|\tilde{A}^{-1}-A^{-1}\|}{\|A^{-1}\|}\leq\frac{\kappa(A)\frac{\|E\|}{\| A\|}}{1-\kappa(A)\frac{\|E\|}{\|A\|}}.$ (19) $$

Lemma 2 In the reduced form the matrices $A$ and $\tilde{A}$ are acute if and only if $A_{11}$ is nonsingular and

$$ $E_{22}=E_{21}\tilde{A}_{11}^{-1}E_{12}.$ (20) $$

In this case, if we set

$$ $F_{21}=E_{21}\tilde{A}_{11}^{-1}\quad\text{and}\quad F_{12}=\tilde{A}_{11}^{-1 }E_{12},$ $$

then

$$ $\tilde{A}=\begin{pmatrix}I\\ F_{21}\end{pmatrix}\tilde{A}_{11}\begin{pmatrix}I&F_{12}\end{pmatrix}$ $$

and

$$ $\tilde{A}^{\dagger}=\begin{pmatrix}I&F_{12}\end{pmatrix}^{\dagger}\tilde{A}_{1 1}^{-1}\begin{pmatrix}I\\ F_{21}\end{pmatrix}^{\dagger}.$ (21) $$

Lemma 3 The matrix

$$ $\begin{pmatrix}I\\ F\end{pmatrix}$ $$

satisfies

$$ $\left\|\begin{pmatrix}I\\ F\end{pmatrix}^{\dagger}\right\|\leq 1$ (22) $$

and

$$ $\left\|\begin{pmatrix}I\\ F\end{pmatrix}^{\dagger}-\begin{pmatrix}I&0\end{pmatrix}\right\|=\Psi_{2}(F).$ (23) $$

Theorem 1 Let $\tilde{A}$ be an acute perturbation of $A$, and let

$$ $\hat{\kappa}=\|A\|\|\tilde{A}_{11}^{-1}\|.$ (24) $$

Then

$$ $\frac{\|\tilde{A}^{\dagger}-A^{\dagger}\|}{\|A^{\dagger}\|}\leq\hat{\kappa} \frac{\|E_{11}\|}{\|A\|}+\Psi_{2}\left(\frac{\hat{\kappa}E_{12}}{\|A\|}\right) +\Psi_{2}\left(\frac{\hat{\kappa}E_{21}}{\|A\|}\right).$ (25) $$

###### Proof.

Let

$$ $I_{21}=\begin{pmatrix}I\\ 0\end{pmatrix},\quad I_{12}=\begin{pmatrix}I&0\end{pmatrix},$ (26) $$

$$ $J_{21}=\begin{pmatrix}I\\ F_{21}\end{pmatrix},\quad J_{12}=\begin{pmatrix}I&F_{12}\end{pmatrix}.$ (27) $$

$\tilde{A}^{\dagger}=J_{12}^{\dagger}A_{11}^{-1}I_{21}^{\dagger}$, hence

$$ $\tilde{A}^{\dagger}-A^{\dagger}=(J_{12}^{\dagger}-I_{12}^{\dagger})A_{11}^{-1} I_{21}^{\dagger}+J_{12}^{\dagger}A_{11}^{-1}(J_{21}^{\dagger}-I_{21}^{\dagger} )+J_{12}^{\dagger}(\tilde{A}_{11}^{-1}-A_{11}^{-1})J_{21}^{\dagger}.$ (28) $$

From Lemma 1 we have the following bound:

$$ $\|J_{12}^{\dagger}(\tilde{A}^{-1}_{11}-A_{11}^{-1})J_{21}^{\dagger}\|\leq\|A_{ 11}^{-1}\|\hat{\kappa}\frac{\|E_{11}\|}{\|A_{11}\|}.$ (29) $$

By Lemma 3

$$ $\|(J_{12}^{\dagger}-I_{12}^{\dagger})A_{11}^{-1}I^{\dagger}_{21}\|\leq\|A_{11} ^{-1}\|\|J^{\dagger}_{12}-I^{\dagger}_{12}\|=\|A_{11}^{-1}\|\Psi_{2}(F_{12})$ (30) $$

$$ $=\|A_{11}^{-1}\|\Psi_{2}(\tilde{A}_{11}^{-1}E_{12})$ (31) $$

$$ $\leq\|A_{11}^{-1}\|\Psi_{2}\left(\frac{\hat{\kappa}E_{12}}{\|A\|}\right),$ (32) $$

and likewise

$$ $\|J_{12}^{\dagger}A_{11}^{-1}(J_{21}^{\dagger}-I_{21}^{\dagger})\|\leq\|A_{11} ^{-1}\|\Psi_{2}\left(\frac{\hat{\kappa}E_{21}}{\|A\|}\right)\leq\|A^{\dagger} \|\Psi_{2}\left(\frac{\hat{\kappa}E_{21}}{\|A\|}\right).$ (33) $$

∎

Theorem 2 In Theorem 1, let

$$ $\kappa=\|A\|\|{A}^{\dagger}\|,$ (34) $$

and suppose that

$$ $\|A^{\dagger}\|\|E_{11}\|<1,$ (35) $$

so that

$$ $\gamma\equiv 1-\frac{\kappa\|E_{11}\|}{\|A\|}>0.$ (36) $$

Then

$$ $\|\tilde{A}^{\dagger}\|\leq\frac{\|A^{\dagger}\|}{\gamma},$ (37) $$

and

$$ $\frac{\|\tilde{A}^{\dagger}-A^{\dagger}\|}{\|A^{\dagger}\|}\leq\frac{\kappa\|E _{11}\|}{\gamma\|A\|}+\Psi_{2}\left(\frac{\kappa E_{21}}{\gamma\|A\|}\right)+ \Psi_{2}\left(\frac{\kappa E_{12}}{\gamma\|A\|}\right).$ (38) $$

###### Proof.

From the equation $\tilde{A}^{\dagger}=J_{12}^{\dagger}\tilde{A}_{11}^{-1}J_{21}^{\dagger}$, we have

$$ $\|\tilde{A}^{\dagger}\|\leq\|J_{12}^{\dagger}\|\|\tilde{A}_{11}^{-1}\|\|J_{21} ^{\dagger}\|\leq\|\tilde{A}_{11}^{-1}\|.$ (39) $$

By Lemma 1,

$$ $\|\tilde{A}^{-1}_{11}\|\leq\frac{\|A_{11}^{-1}\|}{\gamma}=\frac{\|A^{\dagger} \|}{\gamma},$ (40) $$

which establishes (37). Also $\hat{\kappa}\leq\frac{\kappa}{\gamma}$, and the inequality (38) follows from (25).

∎

###### Proof.

###### Proof.

### A.3 Core Theorem

Finally, we give the core theorem used in main paper.
Some symbols and definitions have been claimed in Appendix A.1 and A.2.

Theorem 3
Let ${x}=A^{\dagger}{b}$ and $\tilde{{x}}=\tilde{A}^{\dagger}{b}$, where $\tilde{A}=A+E$, and $E$ is an acute perturbation of $A$. Then

$$ $\frac{\|{x}-\tilde{{x}}\|}{\|{x}\|}\leq\hat{\kappa}\frac{\|E_{11}\|}{\|A\|}+ \Psi_{2}\left(\frac{\hat{\kappa}E_{12}}{\|A\|}\right)+\hat{\kappa}^{2}\frac{ \left\|E_{12}\right\|}{\|A\|}\left(\eta^{-1}\frac{\|{b}_{2}\|}{\|{b}_{1}\|}+ \frac{\|E_{21}\|}{\|A\|}\right).$ (41) $$

###### Proof.

By Lemma 2, write

$$ $\tilde{x}-x=J_{12}^{\dagger}(\tilde{A}_{11}^{-1}-A_{11}^{-1})b_{1}+(J_{12}^{ \dagger}-I_{12}^{\dagger})A_{11}^{-1}b_{1}+J_{12}^{\dagger}\tilde{A}_{11}^{-1} (J_{21}^{\dagger}-I_{21}^{\dagger})b.$ (42) $$

Then

$$ $\|J_{12}^{\dagger}(\tilde{A}_{11}^{-1}-A_{11}^{-1})b_{1}\|\leq\hat{\kappa} \frac{\|E_{11}\|}{\|A\|}\|x\|,$ (43) $$

and

$$ $\|(J^{\dagger}_{12}-I^{\dagger}_{12})A_{11}^{-1}b_{1}\|\leq\Psi_{2}\left(\frac {\hat{\kappa}E_{12}}{\|A\|}\right)\|x\|.$ (44) $$

Now

$$ $\displaystyle J^{\dagger}_{12}\tilde{A}_{11}^{-1}(J^{\dagger}_{21}-I^{\dagger} _{21})b$ $\displaystyle=J^{\dagger}_{12}\tilde{A}_{11}^{-1}((I+F^{H}_{21}F_{21})^{-1}-I) b_{1}+J^{\dagger}_{12}\tilde{A}_{11}^{-1}(I+F^{H}_{21}F_{21})^{-1}F^{H}_{21}b_ {2}.$ (45) $$

To bound the first term in (45), note that

$$ $(I+F^{H}_{21}F_{21})^{-1}-I=-(I+F^{H}_{21}F_{21})^{-1}F^{H}_{21}F_{21}.$ $$

Hence

$$ $\displaystyle\|J_{12}\tilde{A}_{11}^{-1}((I+F^{H}_{21}F_{21})-I)b_{1}\|$ $\displaystyle\leq\|\tilde{A}_{11}^{-1}\|\|(I+F^{H}_{21}F_{21})^{-1}\|\|F^{H}_{ 21}\|\|F_{21}b_{1}\|$ $\displaystyle\leq\|\tilde{A}_{11}^{-1}\|\|E_{21}\tilde{A}_{11}^{-1}b_{1}\|$ $\displaystyle\leq\|\tilde{A}_{11}^{-1}\|\|E_{21}\|^{2}\|{x}\|$ $\displaystyle=\left(\frac{\hat{\kappa}\|E_{21}\|_{2}}{\|A\|}\right)^{2}\|{x}\|.$ (46) $$

For the second term in (45) we have

$$ $\displaystyle\|J_{21}^{\dagger}\tilde{A}_{11}^{-1}(I+F^{H}_{21}F_{21})^{-1}F_{ 21}b_{2}\|$ $\displaystyle\leq\|\tilde{A}_{11}^{-1}\|^{2}\|E_{21}\|\|b_{2}\|$ $\displaystyle=\|\tilde{A}_{11}^{-1}\|^{2}\|E_{21}\|\frac{\|b_{2}\|}{\|b_{1}\|} \eta^{-1}\|x\|\|A\|$ $\displaystyle\leq\eta^{-1}\hat{\kappa}^{2}\frac{\|E_{21}\|\|b_{2}\|}{\|A\|\|b_ {1}\|}\|x\|.$ (47) $$

The bound (41) follows on combining (42)–(47).

∎

Readers can refer to this work [] for more details of perturbation analysis.

Returning to our problem, consider $Wk=v$, where $(k,v)\in P$. Let $\tilde{W}=W+\Delta W$, where $\Delta W$ is the corresponding perturbation matrix.
Assuming $v$ remains constant, there exists $\Delta k$ such that $\tilde{k}=k+\Delta k$ satisfies $\tilde{W}\tilde{k}=v$.
And we have $k=W^{\dagger}v$ and $\tilde{k}=\tilde{W}^{\dagger}v$.
Applying Theorem 3, we obtain

$$ $\frac{\|\Delta k\|}{\|k\|}=\frac{\|k-\tilde{k}\|}{\|k\|}\leq\hat{\kappa}\frac{ \|\Delta E_{11}\|}{\|W\|}+\Psi_{2}\left(\frac{\hat{\kappa}\Delta E_{12}}{\|W\| }\right)+\hat{\kappa}^{2}\frac{\|\Delta E_{12}\|}{\|W\|}\left(\eta^{-1}\frac{ \|v_{2}\|}{\|v_{1}\|}+\frac{\|\Delta E_{21}\|}{\|W\|}\right),$ (48) $$

where $\Delta E_{11}$, $\Delta E_{12}$, $\Delta E_{21}$, and $\Delta W$ are directly related, and each term on the right-hand side involves $\hat{\kappa}$. This means that the relative perturbation of the vector $k$ is constrained by $\hat{\kappa}$.
According to Theorem 2, $\hat{\kappa}\leq\frac{\kappa}{\gamma}$,
where $\kappa=\|W\|\|{W}^{\dagger}\|$ is the condition number of $W$.
This indicates that $\kappa$ is a robust indicator of the impact of $\Delta W$ on the vector $k$.

###### Proof.

## Appendix B Experimental Setup

### B.1 Baseline Editing Methods

Three popular model editing methods were selected as baselines including:

- $\bullet$
MEND [](^6^66https://github.com/eric-mitchell/mend): it learned a hypernetwork to produce weight updates by decomposing
the fine-tuning gradients into rank-1 form.
- $\bullet$
ROME [](^7^77https://github.com/kmeng01/rome): it first localized
the factual knowledge at a specific layer in the
transformer MLP modules, and then updated the knowledge by directly writing
new key-value pairs in the MLP module.
- $\bullet$
MEMIT [](^8^88https://github.com/kmeng01/memit): it extended ROME
to edit a large set of facts and updated a set of MLP layers to update knowledge.

The ability of these methods was assessed based on
EasyEdit(^9^99https://github.com/zjunlp/EasyEdit) [], an easy-to-use knowledge
editing framework which integrates the released codes and hyperparameters from previous methods.

### B.2 Editing Datasets and Evaluation Metrics

Table 2 shows the examples of two datasets.

**Table 2: The editing datasets of both CounterFact and ZsRE.**
| Datasets | Editing prompt |
| --- | --- |
| CounterFact | In America, the official language is |
| ZsRE | Which was the record label for New Faces, New Sounds? |

Besides, following previous works [, , ],
the editing performance metrics for the CounterFact dataset and ZsRE dataset are efficacy, generalization and locality, but there are some computational differences.
In the main paper, the metrics of editing performance are calculated for the CounterFact dataset.
For the ZsRE dataset, here are the details:

Efficacy validates whether the edited models could recall the editing fact under editing prompt $p_{i}$.
The assessment is based on Efficacy Score (ES) representing as: $\mathbb{E}_{i}[\mathbbm{1}[\,\operatorname{argmax}_{o}P_{f_{\theta_{n}}}(o\,|
\,p_{i})=o^{*}_{i}]\,]$.

Generalization verifies whether the edited models could recall the editing fact under the paraphrase prompts $\mathcal{P}^{G}_{i}$ via Generalization Score (GS):
$\mathbb{E}_{i}\,[\mathbb{E}_{p\in\mathcal{P}^{G}_{i}}[\mathbbm{1}[\,
\operatorname{argmax}_{o}P_{f_{\theta_{n}}}(o\,|\,p)=o^{*}_{i}]\,]$.

Locality verifies whether the output of the edited models
for inputs out of editing scope remains unchanged under the locality prompts $\mathcal{P}^{L}_{i}$ via Locality Score (LS):
$\mathbb{E}_{i}\,[\mathbb{E}_{p_{l}\in\mathcal{P}^{L}_{i}}[\mathbbm{1}[\,
\operatorname{argmax}_{o}P_{f_{\theta_{n}}}(o\,|\,p_{l})=o_{l}]\,]\,]$, where $o_{l}$ was the original answer of $p_{l}$.

### B.3 Hyperparameters of PRUNE

When conducting experiments, for different editing methods, LLMs and editing datasets, the hyperparameter $\alpha$ in function $F$ of PRUNE is different.
Table 3 shows the details of this hyperparameter.
$e$ is the base of the natural logarithm.

**Table 3: The hyperparameters $\alpha$ for PRUNE.**
| Datasets | Models | ROME | MEMIT | MEND |
| --- | --- | --- | --- | --- |
| CounterFact | GPT-2 XL | 1.2 | 1.2 | 1.2 |
| LLaMA-2 | 1.2 | $e$ | 1.2 |  |
| LLaMA-3 | 1.5 | $e$ | - |  |
| ZsRE | LLaMA-2 | 1.2 | $e$ | $e$ |

### B.4 Task Prompts

The prompts for each downstream task were illustrated in Table 4.

**Table 4: The prompts to LLMs for evaluating their zero-shot performance on these general tasks.**
| Reasoning: |
| --- |
| Q: {QUESTION} A: Let’s think step by step. {HINT} Therefore, the answer (arabic numerals) is: |
| NLI: |
| {SENTENCE1} entails the {SENTENCE2}. True or False? answer: |
| Open-domain QA: |
| Refer to the passage below and answer the following question. Passage: {DOCUMENT} Question: {QUESTION} |
| Summarization: |
| {DIALOGUE} TL;DR: |

### B.5 Experiments Compute Resources

We used NVIDIA A800 80GB GPU for experiments.
For LLaMA-2 (7B) and LLaMA-3 (8B), it occupies about 40+GB memory and costs about 3 hours for each editing method to run 200 edits and then to test downstream tasks .
For GPT-2 XL (1.5B), it needs 10+GB and costs about 1.5 hours for each editing method to run 200 edits and then to test downstream tasks.

## Appendix C Experimental Results

### C.1 Results of General Abilities

Figure 6 and 7 show the downstream task performance of edited models with GPT-2 XL and LLaMA-3 (8B) on CounterFact dataset.
Figure 8 shows the downstream task performance of edited models with LLaMA-2 (7B) on ZsRE dataset.
Due to limitations of computing resources, experiments were conducted using only LLaMA-2 (7B) on the ZsRE dataset. We will supplement experiments with other LLMs in the future.

Figure: Figure 6: The downstream task performance (%) of models edited by three editing methods with GPT-2 XL on the CounterFact dataset.
Refer to caption: x8.png

Figure: Figure 7: The downstream task performance (%) of models edited by two editing methods with LLaMA-3 (8B) on the CounterFact dataset. Since the code framework EasyEdit used in this paper does not currently support MEND editing on LLaMA-3, there are no results of MEND here.
Refer to caption: x9.png

Figure: Figure 8: The downstream task performance (%) of models edited by three editing methods with LLaMA-2 (7B) on the ZsRE dataset.
Refer to caption: x10.png

### C.2 Results of Editing Performance

Figure 9 and 10 shows the editing performance of edited models with GPT-2 XL and LLaMA-3 (8B) on CounterFact dataset.
Figure 11 shows the editing performance of edited models with LLaMA-2 (7B) on ZsRE dataset.

Figure: Figure 9: The editing performance (%) of three editing methods with GPT-2 XL on CounterFact dataset.
Refer to caption: x11.png

Figure: Figure 10: The editing performance (%) of three editing methods with LLaMA-3 (8B) on CounterFact dataset.
Refer to caption: x9.png

Figure: Figure 11: The editing performance (%) of editing methods with LLaMA-2 (7B) on the ZsRE dataset.
Refer to caption: x12.png

### C.3 Results of another function for PRUNE

In the main paper, $\operatorname{log}$ function is used in $F$ in PRUNE to restrain $\hat{\sigma}_{i}$.
Here we use the $\operatorname{linear}$ function, which could be represented as: $F(\hat{\sigma}_{i})=\frac{1}{\beta}*\hat{\sigma}_{i}+\frac{\beta-1}{\beta}*max
\{\sigma_{i}\}$.
Here $\beta>1$ was a hyperparameter and was set as 2 in this section.
Figure 12 and 13 respectively show some downstream task performance and editing performance with $\operatorname{linear}$ function on CounterFact dataset.

Compared with Figure 6 and 9, we observed that although the $\operatorname{linear}$ function in PRUNE played a role in preserving general abilities and maintaining editing performance, its effectiveness was noticeably inferior to that of the $\operatorname{log}$ function when the number of edits was large.

Figure: Figure 12: The downstream task performance (%) of models edited by three editing methods with GPT-2 XL on the CounterFact dataset. Here the $\operatorname{linear}$ function was used in PRUNE.
Refer to caption: x13.png

Figure: Figure 13: The editing performance (%) of editing methods with GPT-2 XL on the CounterFact dataset. Here the $\operatorname{linear}$ function was used in PRUNE.
Refer to caption: x14.png

### C.4 Condition number with PRUNE

Figure 14 shows after coupling with PRUNE,
the condition number of MEMIT is significantly restrained.

Figure: Figure 14: The condition number of MEMIT with LLaMA-2 (7B) on the CounterFact dataset. “-PRUNE” refers to the condition number of MEMIT coupled with the proposed PRUNE.
Refer to caption: x15.png

## Appendix D Broader Impacts

This work offers significant advancements in the field of model editing for LLMs.
By addressing the challenge of preserving general abilities while performing sequential edits, PRUNE facilitates continual learning and adaptability in LLMs. This can lead to several positive impacts, such as:

Enhanced Adaptability. It enables LLMs to update their knowledge base quickly and accurately without extensive retraining. This adaptability is crucial in dynamic environments where up-to-date information is vital, such as real-time translation services, personalized learning systems, and interactive virtual assistants.

Resource Efficiency. By mitigating the need for full retraining, PRUNE significantly reduces computational resources and energy consumption. This aligns with sustainable AI and makes it more feasible to deploy LLMs in resource-constrained settings.

Improved Performance in Specialized Tasks. PRUNE’s ability to perform targeted edits without compromising overall model performance can enhance LLMs’ effectiveness in specialized domains, such as medical diagnostics, legal analysis, and technical support, where precise and updated knowledge is essential.

While this work offers many benefits, there are potential negative societal impacts that must be considered:

Misuse for Malicious Purposes. The capability to edit LLMs efficiently could be exploited to inject harmful or biased information into models, thereby spreading disinformation or propaganda. This risk is particularly concerning in applications involving social media and news dissemination where LLMs might generate or amplify misleading content.

Fairness. Unintended biases could be introduced during the editing process, potentially exacerbating existing biases in LLMs. This could lead to unfair treatment or misrepresentation of specific groups, especially if the editing is not conducted with proper oversight and consideration of ethical implications.

Privacy Concerns. The ability to update models quickly might also pose privacy risks, as models could be edited to include sensitive or personal information. Ensuring that editing processes do not compromise individual privacy is critical, particularly in applications involving personal data.

To mitigate these potential negative impacts, several strategies could be implemented:

Gated Release and Monitoring. Limiting access to the framework through gated releases and monitoring its usage can help prevent misuse.

Bias and Fairness Audits. Conducting regular audits to assess and address biases in the model editing process can help ensure that edits do not unfairly impact any specific group. Developing guidelines for ethical editing practices is also essential.

Privacy Protection Measures. Establishing clear protocols for handling sensitive data during the editing process can help protect privacy. Anonymization and encryption techniques should be employed to safeguard personal information.

By considering both the positive and negative impacts and implementing appropriate mitigation strategies, this work can contribute to the responsible and ethical advancement of model editing technologies.

## Peer Reviews

confidence: 2
contribution: 3
initial_rating: 8
presentation: 3
questions: Please see the Weaknesses section above.
soundness: 3
strengths: * Sequential editing is indeed a crucial research issue in knowledge editing, and this paper examines it from a relatively comprehensive perspective while proposing an excellent solution.

* The literature review is thorough.

* The evaluation scope is extensive, based on three representative LLMs, including GPT-2 XL, LLaMA-2, and LLaMA-3. It also includes four representative downstream tasks—reasoning, summarization, open-domain question answering, and natural language inference—to broadly demonstrate the impact of model editing on the general abilities of LLMs.

* The performance of the method proposed is good, effectively mitigating the decline in model general abilities caused by sequential edits across multiple metrics.
summary: This paper analyzes how the condition number of the edited matrix affects the models' sensitivity to perturbations and proposes a framework called PRUNE to impose constraints that lower this sensitivity. Experiments demonstrate that PRUNE effectively preserves the general abilities of LLMs while maintaining strong editing performance across various tasks.
weaknesses: 1. It would be helpful to report the results on the "Probability" metric, which is described in both [1] and [2]. I would like to know if applying the proposed framework to impose constraints on sequential editing affects the generalization and ripple effects of the edits themselves.

2. It would be more comprehensive if we could see some results on larger-scale models, such as at least those with 13B parameters or more.

3. Methods like MEMIT support batch editing, so it would be worthwhile to experiment on whether PRUNE can still maintain good model general ability in such batch editing scenarios.

---
**References**

[1] Evaluating the Ripple Effects of Knowledge Editing in Language Models

[2] Editing Large Language Models: Problems, Methods, and Opportunities

---

confidence: 4
contribution: 3
initial_rating: 6
presentation: 3
questions: See weaknesses.
soundness: 3
strengths: 1. Knowledge editing is an important topic, and addressing sequential editing is a challenging yet meaningful direction.
2. This paper is well motivated and the design of PRUNE clearly explained.
3. This paper proposes a plug-and-play method, which demonstrates significant improvements in sequential editing across various models.
summary: This paper investigates the capability of large language models (LLMs) to preserve their general knowledge after sequential editing through a plug-and-play framework named PRUNE. The authors suggest that the condition number of the edited matrix significantly influences the general abilities of the edited models. They propose a method which involves SVD decomposition to ensure that the updated matrix maintains the singular values of the original matrix, thereby preventing substantial perturbations. The paper evaluates the effectiveness of this method across three popular editing methods and three large language models
weaknesses: 1.  Some important details need further clarification. For example, in the editing setup, the key $k$ in the key-value pair $(k, v)$ typically does not change, and usually only $W$ and $v$ are modified. It is unclear why the paper focuses on analyzing the changes in $k_i$ instead of $v$ or $W$.
2.   The PRUNE method imposes constraints on parameters (as described in Equations (3), (4), and (5)), which might potentially reduce the model’s ability to retain other capabilities or preserve historical knowledge. However, the paper lacks sufficient analysis and investigation of the potential negative impacts on the efficacy of the most recent knowledge edits. A more detailed discussion in this regard would improve the overall quality of the analysis.
3.   A comparison with other sequential editing methods should be included in the related work section. Moreover, additional discussion is needed on works aiming at preserving the general capability of an edited model.
4.  The model requires utilizing historical $\Delta W$ in each editing operation, meaning that historical information must be reused. This could potentially lead to an increase in computational resources.

---

confidence: 4
contribution: 3
initial_rating: 6
presentation: 2
questions: The PCA visualization discrepancy is not entirely clear maybe a slightly modified visualization would clarify what is being referred to. 

The baseline capabilities of Llama on general abilities in Table 2 are missing.
soundness: 3
strengths: The paper introduces a theoretically derived upper bound for weight modification. By defining this threshold, the authors provide a quantitative limit which when exceeded, triggers degradation in the model’s original functions and leads to edit forgetting. This contribution deepens understanding of the trade-offs involved in weight modification and offers a practical guideline for preserving model performance.

The authors demonstrate that PRUNE effectively mitigates the negative impacts of sequential edits. PRUNE shows considerable performance improvement by stabilizing the model against shifts in weight values, which may otherwise lead to inconsistencies or loss of prior knowledge.
summary: The research tackles the issues that arise due to weight shifts in sequential model editing when using weight-modifying approaches. The authors present a new approach, PRUNE, which aims to stabilize editing by maintaining control over these shifts. Importantly, the paper proposes an upper bound on weight modifications, beyond which the model's original capabilities degrade, resulting in reduced performance and an inability to retain edits. This bound serves as a theoretical safeguard, ensuring that model modifications remain within an optimal range for consistent, reliable performance.
weaknesses: The study focuses solely on the Llama-2 (7B) model, with a relatively small set of samples for evaluation which is a concern as editing in large numbers might exacerbate the impact of the shift that needs to be controlled. To broaden the scope, the authors could consider using a smaller model from the GPT series or a single editing approach, allowing for an expanded sample size under computational constraints. This could strengthen the study by providing more robust evidence for PRUNE’s effectiveness across different architectures. (major)

While PRUNE is effective, some edits may still be lost in the process, particularly when just applying the operation. A deeper analysis of this limitation could provide valuable insights. (minor)

---

confidence: 5
contribution: 2
initial_rating: 3
presentation: 3
questions: 1. In section 3.2, where the authors perform matrix perturbation analysis, it is not clear to me how that is relevant. The main objection I have is that if an MLP layer in a model is edited, everything until that layer remains the same. This means that "keys", which are by definition input to the edited matrix, remain identical before and after editing. Then I'm not sure what is the meaning of ∆k here, since the value of k never changes when an edit is made. 

2. In line 241, the spectral norm definition is used, whereas line 203 says that ||*|| represents 2-norm. 

3. The hypothetical situation described in line 271-275 is not clear to me. Does the situation involve making N sequential edits about the same fact? If that's the case, then each edit matrix will not be equal to ∆W_1 as the value of ∆W_1 depends on the base matrix, which would be been changed. Although I understand that this is not important in the larger scheme of things, I think it is not correct. I am happy to provide more explanation. 

4. In Table 4, we can see that the singular value of the edited matrix increases way more dramatically for MEMIT compared to other methods. Why is that the case? Also it makes me question that the singular values are the only underlying cause for model degradation, as if that was the case, since MEMIT has the largest singular value blow up, it would be the least stable and see maximum model degradation. But that is not the case, in fact MEMIT is the most stable of the three. What is the author's take on this?

5. In equation 4 and 5, I would like some more intuition and explanation of the choice of restraining function. It is not obvious to me why the log function suggested by them works. I think this part of the paper has the least detail whereas I would have preferred a lot more detail here. For example, I would like to see more ablations on the different actions that can be performed on the larger singular values of the ∆W_N matrix. 

6. From an intuitive understanding, this paper tries to attenuate the largest singular values of the edit matrix. But the largest singular values are the most important part of a matrix, providing the direction of largest variance and loosely speaking is the direction along which the most significant action is done by the matrix. How is attenuating the most significant action of a matrix not resulting in a lot of issues? Why is it okay to attenuate the largest singular values of a matrix? I think this is a much deeper question and warrants investigation. 

7. The experiments shown in Figure 3 and 4 involve too few sequential edits. This method will be a lot more believable if the authors presented results for atleast 1000 edits, but preferably even more. 

My final comment to the authors is that while this paper presents an interesting empirical contribution and figure 5 makes a very good empirical case for usefulness, I have a hard time understanding why the approach proposed by the authors works (questions 4-6). I do not find the explanations or motivations given in the paper appropriate so far. I am looking forward to hearing the author's comments during the discussion phase and am very happy to update my scores based on it if the authors are able to sufficiently clarify these questions.
soundness: 1
strengths: 1. The authors provide an interesting analysis of the potential reasons behind the loss of general ability of a model as it is edited sequentially. 
2. The proposed methods is plug and play and can be applied on top of all existing parameter modifying model editing methods.
summary: Knowledge editing methods when applied sequentially lead to significant model degradation. In this paper, the authors identify the reasons behind the loss of model's general abilities in the form of increasing singular value of edit-matrix as a function of the number of edits, and propose methods to control that.
weaknesses: Many choices and decisions made in the paper seem unmotivated or not explained properly. I defer to the questions sections where clarifications are requested. I am happy to revise my review post the discussion period once the questions have been answered.

Meta-review: Summary of Scientific Claims and Findings

The paper's main claim is that the degradation of general model abilities during sequential model editing is closely linked to the condition number of the edited matrix. To address this, the authors introduce PRUNE, a framework that stabilizes the impact of edits by limiting the size of perturbations. Experiments across multiple LLMs (such as GPT-2 XL, LLaMA-2, and LLaMA-3) demonstrate that PRUNE is effective in preserving model performance across a variety of tasks. The authors show that applying PRUNE limits perturbations, reducing forgetfulness and improving stability even after multiple edits. 

Key findings include:
1. PRUNE improves model performance by preventing excessive perturbations during sequential edits.
2. The framework is demonstrated to be effective across different LLM architectures and tasks, maintaining generalization ability and reducing memory loss.
3. Applying PRUNE multiple times improves model performance more than applying it once, highlighting the method’s scalability.

Strengths of the Paper

1. Theoretical Contribution: The paper offers a novel approach to mitigating performance degradation during sequential model editing by incorporating perturbation theory and proposing an upper bound for weight modification. This theoretical contribution provides a clear, actionable framework for stabilizing model performance.
  
2. Experimental Validation: The experiments conducted on LLaMA-2 (7B) and GPT-2 XL (1.5B) demonstrate the practical utility of PRUNE in real-world settings. The paper successfully shows that the method improves model stability and preserves general abilities in the face of long-term edits.
  
3. Clarity of Methodology: The paper clearly explains the rationale behind PRUNE, particularly the use of a logarithmic restraining function to control the growth of the largest singular values. This offers a concrete solution to the challenge of avoiding overfitting while retaining essential knowledge.
  
4. Practical Implications: The results suggest that PRUNE could be of significant value for practitioners who need to perform sequential edits on large models while minimizing the risk of knowledge loss.

Weaknesses and Missing Elements

1. Limited Scope of Models and Tasks: The experiments are primarily conducted on a small set of models (LLaMA-2 and GPT-2 XL) and tasks (reasoning, summarization, question answering). While these models are well-known, the results could benefit from broader evaluation on additional model architectures (such as GPT-3, T5, or BERT) and tasks, particularly those involving domain-specific or multi-modal learning. Expanding the sample size and task variety would enhance the generalizability of the results.

2. Impact of Multiple Edits on Recent Changes: Although PRUNE is shown to be effective for most sequential edits, the efficacy of the most recent edits diminishes after many applications of the method. Further exploration into this phenomenon and the design of more targeted techniques for recent edits could improve the approach.

3. Clarification on PCA Visualizations and Baseline Comparisons: There are mentions of discrepancies in PCA visualizations and baseline performance comparisons, particularly for LLaMA on general tasks. The authors should address these concerns in more detail to clarify the results and ensure full transparency.

4. Further Exploration of Method Limitations: The paper discusses the limitations of PRUNE, particularly in scenarios where edits may still be lost despite the constraints. Expanding this discussion and exploring alternative methods for mitigating this loss would strengthen the overall contribution. For instance, analyzing how PRUNE compares with other contemporary model-editing techniques like MEMIT or ROME could offer insights into its relative advantages and disadvantages.

Decision Rationale

The paper provides a novel and significant contribution to the field of model editing, particularly for large language models. The introduction of PRUNE is theoretically sound and experimentally validated, offering a solution to an important challenge in machine learning. The strength of the paper lies in its clear methodology, thorough experimental evaluation, and potential practical impact.

However, there are some limitations, primarily the narrow scope of models and tasks, the diminishing efficacy of PRUNE for recent edits, and the need for more thorough clarification in certain experimental aspects. Despite these weaknesses, the core contribution is strong, and the framework holds promise for further research and refinement.

Based on the strengths in theoretical innovation and practical application, and considering the possibility for further improvements and future research directions, I would recommend accepting this paper, with minor revisions to address the aforementioned weaknesses.

Decision: Accept (Poster)