Neuron-Level Sequential Editing for Large Language Models

## Abstract

Abstract This work explores sequential model editing in large language models (LLMs), a critical task that involves modifying internal knowledge within LLMs continuously through multi-round editing, each incorporating updates or corrections to adjust the model’s outputs without the need for costly retraining. Existing model editing methods, especially those that alter model parameters, typically focus on single-round editing and often face significant challenges in sequential model editing-most notably issues of model forgetting and failure. To address these challenges, we introduce a new model editing method, namely N euron-level S equential E diting (NSE), tailored for supporting sequential model editing. Specifically, we optimize the target layer’s hidden states using the model’s original weights to prevent model failure. Furthermore, we iteratively select neurons in multiple layers for editing based on their activation values to mitigate model forgetting. Our empirical experiments demonstrate that NSE significantly outperforms current modifying parameters model editing methods, marking a substantial advancement in the field of sequential model editing. Our code is released on https://github.com/jianghoucheng/NSE .

## 1 Introduction

Large language models (LLMs) have demonstrated remarkable capabilities in storing extensive factual knowledge during pre-training and recalling this information during inference (Brown et al., ; Petroni et al., ; Roberts et al., ). However, as real-world knowledge continuously evolves, the information within these models can become outdated or incorrect (Cao et al., ; Mitchell et al., ; Fang et al., ). Retraining LLMs to incorporate new information is often prohibitively costly (Mitchell et al., ; Meng et al., ). Consequently, recent years have witnessed a surge in model editing methods focusing on modifying specific knowledge without the complete retraining process. Specifically, they first identify the crucial layers for the target knowledge by calculating their casual effect on output. Then, by updating the weights of these layers, they manipulate these layers’ hidden states to modify the final output, enabling LLMs to adapt seamlessly to dynamic real-world information (Meng et al., ; Hartvigsen et al., ).

While current direct model editing methods prove effective for single-round modifications, real-world applications demand a continual learning process where models must retain previous edits during subsequent modifications (Yao et al., ). This has led to the concept of sequential model editing, which necessitates performing multiple, consecutive edits on models. However, current direct model editing methods pose significant risks in this context (Meng et al., ; ). The primary risk is model forgetting, where cumulative changes in parameters from consecutive edits cause the model to forget previously modified knowledge, thereby degrading overall performance (Gupta et al., ). For instance, as illustrated in Figure 1 (a), after editing the model with new knowledge about “The cat”, the LLM forgets previously edited knowledge about “The latest Olympic”.

Furthermore, the second risk is model failure, where excessive edits impair the model’s ability to generate coherent text. Worse still, this impairment may lead to model collapse potentially, characterized by producing irrelevant, repetitive, or nonsensical text, as illustrated in Figure 1 (a).
In sight of this, recent researches, such as memory-based methods (Mitchell et al., ; Hartvigsen et al., ; Das et al., ), have attempted to address these challenges by preserving LLM parameters after each edit. However, the increasing storage requirements as the number of edits grows significantly limit the practicality of these methods.

Figure: Figure 1: Example of sequential editing. (a) shows model forgetting and model failure issues in sequential editing using ROME/MEMIT, while (b) shows the accurate editing capabilities of our method without such issues.
Refer to caption: extracted/5903371/figures/INTROFJF3.png

To tackle these challenges, we introduce a new model editing method, termed Neuron-level Sequential Editing (NSE). Specifically, to address the model failure, NSE uses weights rewinding for value computation by preserving the model’s original weights as a significant reference when manipulating the hidden states of the crucial layers. This process can effectively mitigate the impairment of previous knowledge accumulated over various edits. Furthermore, to address model forgetting, NSE selectively collects “influential neurons” for weights updating by sorting the neuron activation within the crucial layers, rather than updating all weights in critical layers as in previous work (Meng et al., ; ). This selective modification maximizes the protection of model functionality from being degraded. Additionally, for large-scale LLMs containing numerous neurons, an iterative multi-layer editing is introduced to streamline the neuron selection process, enabling NSE to achieve massive knowledge updates effectively in a single editing.

Through theoretical analysis and extensive experiments conducted on the GPT2-XL (1.5B) (Radford et al., ), GPT-J (6B) (Wang & Komatsuzaki, ) and Llama3 (8B), we validate the effectiveness and efficiency of our NSE.
Compared to the current model editing methods (e.g., Fine-tuning (Zhu et al., ), MEND (Mitchell et al., ), ROME (Meng et al., ), MEMIT (Meng et al., ) and GRACE (Hartvigsen et al., )), NSE shows substantial improvements w.r.t five commonly used metrics such as specificity and consistency.

## 2 Preliminary

### 2.1 Autoregressive Language Model

An autoregressive language model predicts the next token in a sequence based on the tokens that have come before it. Given a $L$-layer transformer model and an input sequence $x=(x_{0},x_{1},\ldots,x_{T})$, the model aims to predict the next token in the sequence. The probability of the next token $x_{t+1}$ is given by:

$$ $\mathbb{P}(x_{t+1}\mid x_{0},x_{1},\ldots,x_{t})=\text{Softmax}(\mathbf{\bm{W} }_{e}\mathbf{\bm{h}}_{t}^{N}),$ (1) $$

where $\mathbf{\bm{W}}_{e}$ represents embedding matrix, and $\mathbf{\bm{h}}_{t}^{N}$ represents the final hidden state at the topmost layer $N$. The hidden state $\mathbf{\bm{h}}_{t}^{l}$ at layer $l$ is calculated as:

$$ $\displaystyle\mathbf{\bm{h}}_{t}^{l}(x)$ $\displaystyle=\mathbf{\bm{h}}_{t}^{l-1}(x)+\mathbf{\bm{a}}_{t}^{l}(x)+\mathbf{ \bm{v}}_{t}^{l}(x),$ (2) $\displaystyle\ \mathbf{\bm{a}}_{t}^{l}$ $\displaystyle=\text{attn}^{l}(\mathbf{\bm{h}}_{0}^{l-1},\mathbf{\bm{h}}_{1}^{l -1},\ldots,\mathbf{\bm{h}}_{t}^{l-1}),$ $\displaystyle\mathbf{\bm{v}}_{t}^{l}$ $\displaystyle=\mathbf{\bm{W}}_{\text{out}}^{l}\sigma(\mathbf{\bm{W}}_{\text{in }}^{l}\gamma(\mathbf{\bm{h}}_{t}^{l-1}+\mathbf{\bm{a}}_{t}^{l})),$ $$

where $\mathbf{\bm{a}}_{t}^{l}$ represents the output of the attention block and $\mathbf{\bm{v}}_{t}^{l}$ represents the output of the FFN layer. $\mathbf{\bm{W}}_{\text{in}}^{l}$ and $\mathbf{\bm{W}}_{\text{out}}^{l}$ are weight matrices, $\sigma$ is non-linear activation function, and $\gamma$ represents layernorm.

### 2.2 Sequential Model Editing

Sequential model editing aims to refine a pre-trained model $f_{\bm{\theta}_{0}}$ continuously through multiple edits, each incorporating updates or corrections to adjust the model’s outputs (Huang et al., ; Hartvigsen et al., ; Li et al., ). Formally, each batch of sequential edits involves a series of facts $(s,r,o)$ in the form of (subject $s$, relation $r$, object $o$) (e.g., $s$=“The latest Olympic”, $r$=“was held in”, $o$=“Paris”) for each edit iteration $t$, specifying the desired responses for that round. After the $t$-th edit, the updated model $f_{\bm{\theta}_{t}}$, built on its predecessor $f_{\bm{\theta}_{t-1}}$, is optimized to accurately produce the target outputs for the relevant inputs $\mathbb{D}_{edit_{t}}$, while maintaining accuracy on inputs outside the current edit scope. This ensures that the model not only adapts to new requirements but also preserves effectiveness across the whole process.

More formally, the editing function $h$ for each edit $t$ is defined as $f_{\bm{\theta}_{t}}=h(f_{\bm{\theta}_{t-1}},\mathbb{D}_{edit_{t}})$, applying necessary updates based on the specific edits. Following prior studies (Meng et al., ; ), we encourage the editing function to satisfy the following goals:

- •
Efficacy. For all inputs in any previous editing rounds up to the current $t$-th round, the updated model $f_{\bm{\theta}_{t}}$ consistently maintains the target outputs:
$f_{\bm{\theta}_{t}}((s,r))=o,\quad\forall(s,r,o)\in\bigcup_{j=1}^{t}\mathbb{D}
_{edit_{j}}.$
(3)
- •
Generalization. For any inputs equivalent to the edited input $(s,r)$, denoted by $N((s,r))$, the updated model $f_{\bm{\theta}_{t}}$ consistently outputs the intended result $o$ for all edits up to the current round:
$f_{\bm{\theta}_{t}}(N((s,r)))=o,\quad\forall(s,r,o)\in\bigcup_{j=1}^{t}\mathbb
{D}_{edit_{j}}.$
(4)
- •
Specificity. The updated model $f_{\bm{\theta}_{t}}$ retains the outputs from its initial model $f_{\bm{\theta}_{0}}$ for all inputs that have not been edited in any round up to the current one:
$f_{\bm{\theta}_{t}}((s,r))=f_{\bm{\theta}_{0}}((s,r)),\quad\forall(s,r,o)
\notin\bigcup_{j=1}^{t}\mathbb{D}_{edit_{j}}.$
(5)

### 2.3 Previous Model Editing Method

Given a knowledge fact tuple $(s_{i},r_{i},o_{i})$, the objective is to edit the output of the LLM such that prompting it with $(s_{i},r_{i})$ results in $o_{i}$. Following ROME (Meng et al., ) and MEMIT (Meng et al., ), we treat the weights of the Transformer’s (Vaswani et al., ) FFN layer as a linear associative memory. That is, the linear operations within the FFN layer can be viewed as key-value storage for information retrieval (Kohonen, ; Anderson, ).

Given the weights ${\bm{W}}_{\text{in}}^{l}$ of the $l$-th FFN layer when prompted with $(s_{i},r_{i})$, we identify the activation output of the last subject token $S$ as the key ${\bm{k}}_{i}^{l}$. Hereafter, this key ${\bm{k}}_{i}^{l}$ is processed through the output weights ${\bm{W}}_{\text{out}}^{l}$, producing the value ${\bm{v}}_{i}^{l}$.
In the context of sequential model editing, we start with an initial set of key-value associations for knowledge facts stored in the $l$-th FFN layer, denoted respectively by $K_{0}=\{{\bm{k}}_{i}\}_{i=1}^{n}$ and ${\bm{V}}_{0}=\{{\bm{v}}_{i}\}_{i=1}^{n}$. Hereafter, we aim to introduce $m$ new key-value associations, denoted as ${\bm{K}}_{1}=\{{\bm{k}}_{i}\}_{i=n+1}^{n+m}$ and ${\bm{V}}_{1}=\{{\bm{v}}_{i}\}_{i=n+1}^{n+m}$, while retaining all existing associations unchanged. Drawing on prior work (Meng et al., ), the optimization objective is as follows:

$$ $\bm{\Delta}^{*}=\operatorname*{arg\,min}_{\bm{\Delta}}(\left\|({{\bm{W}}}+\bm{ \Delta}){\bm{K}}_{1}-{\bm{V}}_{1}\right\|^{2}+\left\|({{\bm{W}}}+\bm{\Delta}){ \bm{K}}_{0}-{\bm{V}}_{0}\right\|^{2}),$ (6) $$

where ${\bm{W}}$ represents the weights of ${\bm{W}}_{\text{out}}$ in the target FFN layer, $\Delta$ denotes the update to $W$, and $V_{1}$ can be directly trained using the fine-tuning loss predicted by the model through backpropagation.
Since ${\bm{K}}_{0}$ and ${\bm{V}}_{0}$ represent the retained knowledge in LLMs, we can express this relationship as ${\bm{W}}{\bm{K}}_{0}={\bm{V}}_{0}$. Therefore, we can obtain the closed-form solution of Eqn. 6 using the least squares method (Lang, ):

$$ $\bm{\Delta}^{*}={\bm{R}}{\bm{K}}_{1}^{T}\left({\bm{K}}_{0}{\bm{K}}_{0}^{T}+{ \bm{K}}_{1}{\bm{K}}_{1}^{T}\right)^{-1},$ (7) $$

where ${\bm{R}}={\bm{V}}_{1}-{\bm{W}}{\bm{K}}_{1}$. Additionally, MEMIT focuses on editing a specific set of layers denoted as $\mathcal{R}=\{l_{0}-|\mathcal{R}|+1,\ldots,l_{0}\}$. The required the weight update $\bm{\Delta}^{l}$ for layer $l\in\mathcal{R}$ is expressed as:

$$ $\bm{\Delta}^{l}={\bm{R}}^{l}{{\bm{K}}_{1}^{l}}^{T}\left({\bm{K}}_{0}^{l}{{\bm{ K}}_{0}^{l}}^{T}+{\bm{K}}_{1}^{l}{{\bm{K}}_{1}^{l}}^{T}\right)^{-1},$ (8) $$

where ${\bm{R}}^{l}=\frac{{\bm{R}}^{l_{0}}}{l_{0}-l+1}$. These modifications are implemented sequentially, starting from the lower layers and progressing to the upper layers.

## 3 Methodology

In this section, we introduce NSE, a method tailored for sequential model editing, as illustrated in Figure 2. Initially, we identify the values computing method in Section 3.1. Subsequently, in Section 3.2, we detail the editing method that selectively filters certain neurons for corresponding parameter updates. Finally, in Section 3.3, we introduce the approach of iterative multi-layer editing.

Figure: Figure 2: Overview of sequential model editing with NSE. (a) describes the process of weights rewinding for value computation. (b) illustrates the neuron selection and neuron-level weights update. (c) shows the process of iterative multi-layer editing.
Refer to caption: extracted/5903371/figures/model_fig.png

### 3.1 Weights Rewinding for Value Computation

First of all, our primary goal is to find a hidden vector that encodes the new association $(s_{i},r_{i},o_{i})$ and replaces the value ${\bm{v}}_{i}^{l}$ in $l$-th layer as described in Section 2.3. In practical implementation, as shown in Figure 2 (a), we optimize $\bm{\delta}_{i}$ through gradient descent by maximizing the probability of the model outputting $o_{i}$ to compute ${\bm{z}}_{i}={\bm{h}}_{i}^{l}+\bm{\delta}_{i}$, where ${\bm{h}}_{i}^{l}$ denotes the hidden state of the LLM at layer $l$. And the value ${{\bm{v}}_{i}}^{l}$ can be computed as ${{\bm{v}}_{i}}^{l}+=\bm{\delta}_{i}$. In the process of sequential editing, we observed that using the updated model parameters $f_{\theta_{t}}$ to compute ${\bm{z}}_{i}$ after each edit leads to significant model degradation over multiple edit rounds. This indicates that the cumulative parameter updates from each editing round can lead to a shift in value computation. Conversely, using the original model parameters $f_{\bm{\theta}_{0}}$ to compute ${\bm{z}}_{i}$ effectively prevents this issue. Hence, we propose a weights rewinding method for value computation , which is based on the preserve initial model weights $f_{\bm{\theta}_{0}}$ to ensure that ${\bm{z}}_{i}$ is computed using $f_{\bm{\theta}_{0}}$ for each edit. The optimization objective is as follows:

$$ ${\bm{z}}_{i}={\bm{h}}_{i}^{l}+\operatorname*{arg\,min}_{\bm{\delta}_{i}}(-\log \mathbb{P}_{f_{\bm{\theta}_{0}}({\bm{h}}_{i}^{l}+={\bm{\delta}_{i}})}\left[o_{ i}\mid(s_{i},r_{i})\right]),$ (9) $$

where ${f_{\bm{\theta}_{0}}({\bm{h}}_{i}^{l}+={\bm{\delta}_{i}})}$ represents the original model with ${\bm{h}}_{i}^{l}$ updated to ${\bm{h}}_{i}^{l}+\bm{\delta}_{i}$. Subsequently, the value ${{\bm{v}}_{i}}^{l}$ can be updated as ${{\bm{v}}_{i}}^{l}+={\bm{z}}_{i}-{\bm{h}}_{i}^{l}$, which will be used to support subsequent model edits. Note that when calculating ${\bm{z}}_{i}$ using the original model parameters $f_{\bm{\theta}_{0}}$, it is sufficient to save only the weight matrix ${\bm{W}}_{\text{out}}$ that needs to be updated, rather than storing the entire model parameters and we use the original weights only during the computation of ${\bm{z}}_{i}$.

### 3.2 Neuron-level Weights Updating

In Section 3.1, we primarily discussed the method for calculating $z_{i}$ to replace the value in the target layer. In this section, we will specifically elaborate on how to utilize the computed value for neuron-level weight updates, as shown in Figure 2 (b).

Based on previous works, it is established that neurons in FFN contain abundant information (Dai et al., ; Wang et al., ; Schwettmann et al., ; Pan et al., ). Hence, we selectively optimize a subset of neurons rather than alter the entire weight matrix for each edit. Specifically, for a given knowledge fact $(s_{i},r_{i},o_{i})$, we utilize the activation values ${\bm{k}}_{i}$ of the neurons and compute scores $\mathbf{\bm{Q}}_{i}=|{\bm{k}}_{i}|$. Neurons are ranked based on these scores, and a subset is chosen such that the cumulative score of selected neurons surpasses a predetermined percentage of the total score:

$$ $\mathcal{I}=\operatorname*{arg\,min}_{\mathcal{I}\subseteq\{1,\ldots,N\}}| \mathcal{I}|\quad\text{s.t.}\quad\sum_{j\in\mathcal{I}}{\bm{Q}}_{ij}\geq p \times\sum_{j=1}^{N}{\bm{Q}}_{ij},$ (10) $$

where ${\bm{Q}}_{ij}$ represents the score of the $j$-th neuron and $\mathcal{I}$ is the set of indices for selected neurons.

Given the need for batch editing, which entails editing multiple knowledge facts simultaneously, each possibly corresponding to different neuron sets, we compute the sum of neuron scores corresponding to all samples in a batch to derive a new neuron score, which is then used for selecting neurons.

Hereafter, we introduce how to update ${\bm{W}}$ (i.e., ${\bm{W}}_{\text{out}}$) by the selected neurons set $\mathcal{I}$. Let $\hat{{\bm{W}}}$ and $\hat{\bm{\Delta}}$ denote the submatrices of ${\bm{W}}$ and $\bm{\Delta}$, respectively, selected according to the indices in the set $\mathcal{I}$. Our objective is to modify only a subset of neuronal parameters by altering specific rows of weights in ${\bm{W}}$ to achieve the optimization outlined in Eqn. 6.

And we can transform Eqn. 6 accordingly:

$$ $\hat{\bm{\Delta}}^{*}=\operatorname*{arg\,min}_{\hat{\bm{\Delta}}}(\left\|( \hat{{\bm{W}}}+\hat{\bm{\Delta}})\hat{{\bm{K}}}_{1}-{\bm{V}}_{1}\right\|^{2}+ \left\|(\hat{{\bm{W}}}+\hat{\bm{\Delta}})\hat{{\bm{K}}}_{0}-{\bm{V}}_{0}\right \|^{2}),$ (11) $$

where $\hat{{\bm{K}}}_{0}$ and $\hat{{\bm{K}}}_{1}$ are the submatrices of ${\bm{K}}_{0}$ and ${\bm{K}}_{1}$ formed by selecting elements indexed by $\mathcal{I}$ respectively. Based on the method of minimal squared error, we solve Eqn. 11 as follows:

$$ $\hat{\bm{\Delta}}^{*}=\hat{{\bm{R}}}\hat{{\bm{K}}}_{1}^{T}\hat{{\bm{C}}}^{-1},$ (12) $$

where $\hat{{\bm{R}}}={\bm{V}}_{1}-\hat{{\bm{W}}}\hat{{\bm{K}}}_{1}$ and $\hat{{\bm{C}}}=\hat{{\bm{K}}}_{0}\hat{{\bm{K}}}_{0}^{T}+\hat{{\bm{K}}}_{1}\hat
{{\bm{K}}}_{1}^{T}$, and following MEMIT (Meng et al., ), ${\bm{K}}_{0}{\bm{K}}_{0}^{T}$ is estimated by $\lambda\mathbb{E}\left[{\bm{k}}{\bm{k}}^{T}\right]$, where $\lambda$ is a hyperparameter that balances the weights between new knowledge and preserved knowledge. The submatrix $\hat{{\bm{K}}}_{0}\hat{{\bm{K}}}_{0}^{T}$ is then obtained by selecting the rows and columns indexed by $\mathcal{I}$ from ${\bm{K}}_{0}{\bm{K}}_{0}^{T}$. Additionally, during the continuous editing process, the newly edited knowledge from each round will become the old knowledge for the next round. Therefore, after each round, we will add the newly edited knowledge into ${\bm{K}}_{0}{\bm{K}}_{0}^{T}$.

### 3.3 Iterative multi-layer Editing

As described in Section 2.3, MEMIT propagates edits through the layers by computing the value ${\bm{v}}_{i}^{l}$ as ${\bm{v}}_{i}^{l}+=\frac{\bm{\delta}_{i}}{l_{0}-l+1}$ ($l\in\mathcal{R}$) (Meng et al., ; Gupta et al., ).Here, $\bm{\delta}_{i}={\bm{z}}_{i}-{\bm{h}}_{i}^{l_{0}}$ represents the residual difference. The fundamental purpose is that as each layer is updated, the hidden state ${\bm{h}}_{i}^{l_{0}}$ progressively approaches the target ${\bm{z}}_{i}$, thereby diminishing the residual $\bm{\delta}_{i}$. Detailed analyses can be found in Appendix E. However, due to errors in the fitting process, some knowledge proves difficult to edit, resulting in ${\bm{v}}_{i}^{l_{0}}$ not sufficiently approximating ${\bm{z}}_{i}$, and consequently leading to editing failures.

Therefore, we propose iterative multi-layer editing to refine the multi-layer editing approach in MEMIT by iteratively selecting neurons to edit multiple layers, as depicted in Figure 2 (c). Specifically, considering that some knowledge is difficult to edit such that the corresponding value ${\bm{v}}_{i}^{l_{0}}$ cannot sufficiently approximate the optimized target value ${\bm{z}}_{i}$, we employ iterative multi-layer editing. After each round of multi-layer editing, we filter the knowledge samples in the current batch based on $\|{\bm{z}}_{i}-{\bm{h}}_{i}^{l_{0}}\|^{2}$. If $\|{\bm{z}}_{i}-{\bm{h}}_{i}^{l_{0}}\|^{2}<\alpha$, the knowledge sample is considered successfully edited. Conversely, if $\|{\bm{z}}_{i}-{\bm{h}}_{i}^{l_{0}}\|^{2}>\alpha$, the sample is deemed not yet successfully edited. The hyperparameter $\alpha$, which is set differently for various LLMs, determines the threshold for editing success. These unedited samples are then filtered out to form a new batch for further multi-layer editing, repeating this process until all knowledge samples in the batch meet $\|{\bm{z}}_{i}-{\bm{h}}_{i}^{l_{0}}\|^{2}<\alpha$ or the iteration limit is reached. This iterative editing approach significantly enhances the success rate of sample edits, enabling NSE to effectively achieve massive knowledge updates in a single editing session. More details of the NSE algorithm are provided in Appendix A.

## 4 Experiments

We conduct experiments to demonstrate the effectiveness of our model editing method. The experiments aim to address the following research questions:

- •
RQ1: How does NSE perform on sequential model editing tasks compared to existing methods?
- •
RQ2: What is the impact of adjusting the batch size of edits on the performance of NSE?
- •
RQ3: Can the LLM, after undergoing NSE editing, retain its original general capabilities, and how does it perform on general capability tests?
- •
RQ4: How does each individual component of NSE contribute to the overall editing performance?

**Table 1: Comparison of NSE with existing methods on the sequential model editing task. Eff., Gen., Spe., Flu. and Consis. denote Efficacy, Generalization, Specificity, Fluency and Consistency, respectively.**
| Method | Model | Counterfact | ZsRE |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
|  |  | Eff.$\uparrow$ | Gen.$\uparrow$ | Spe.$\uparrow$ | Flu.$\uparrow$ | Consis.$\uparrow$ | Eff.$\uparrow$ | Gen.$\uparrow$ | Spe.$\uparrow$ |
| Pre-edited | Llama3 | 7.85$\pm$0.26 | 10.58$\pm$0.26 | 89.48$\pm$0.18 | 635.23$\pm$0.11 | 24.14$\pm$0.08 | 36.99$\pm$0.30 | 36.34$\pm$0.30 | 31.89$\pm$0.22 |
| FT-L | 83.33$\pm$0.37 | 67.79$\pm$0.40 | 46.63$\pm$0.37 | 233.72$\pm$0.22 | 8.77$\pm$0.05 | 30.48$\pm$0.26 | 30.22$\pm$0.32 | 15.49$\pm$0.17 |  |
| FT-W | 61.23$\pm$0.38 | 62.40$\pm$0.24 | 47.05$\pm$0.41 | 492.34$\pm$0.23 | 3.57$\pm$0.03 | 32.08$\pm$0.35 | 31.43$\pm$0.23 | 14.72$\pm$0.16 |  |
| MEND | 63.24$\pm$0.31 | 61.17$\pm$0.36 | 45.37$\pm$0.38 | 372.16$\pm$0.80 | 4.21$\pm$0.05 | 0.91$\pm$0.05 | 1.09$\pm$0.05 | 0.53$\pm$0.02 |  |
| ROME | 64.40$\pm$0.47 | 61.42$\pm$0.42 | 49.44$\pm$0.38 | 449.06$\pm$0.26 | 3.31$\pm$0.02 | 2.01$\pm$0.07 | 1.80$\pm$0.07 | 0.69$\pm$0.03 |  |
| MEMIT | 65.65$\pm$0.47 | 64.65$\pm$0.42 | 51.56$\pm$0.38 | 437.43$\pm$1.67 | 6.58$\pm$0.11 | 34.62$\pm$0.36 | 31.28$\pm$0.34 | 18.49$\pm$0.19 |  |
| GRACE | 90.72$\pm$0.13 | 0.09$\pm$0.01 | 87.23$\pm$0.21 | 632.43$\pm$0.63 | 23.79$\pm$0.23 | 74.58$\pm$0.31 | 1.03$\pm$0.06 | 31.86$\pm$0.12 |  |
| NSE | 96.14$\pm$0.19 | 78.42$\pm$0.35 | 87.66$\pm$0.19 | 632.72$\pm$0.12 | 30.20$\pm$0.10 | 62.29$\pm$0.35 | 47.13$\pm$0.31 | 32.32$\pm$0.22 |  |
| Pre-edited | GPT2-XL | 22.23$\pm$0.73 | 24.34$\pm$0.62 | 78.53$\pm$0.33 | 626.64$\pm$0.31 | 31.88$\pm$0.20 | 22.19$\pm$0.24 | 31.30$\pm$0.27 | 24.15$\pm$0.32 |
| FT-L | 63.55$\pm$0.48 | 42.20$\pm$0.41 | 57.06$\pm$0.30 | 519.35$\pm$0.27 | 10.56$\pm$0.05 | 37.11$\pm$0.39 | 33.30$\pm$0.37 | 10.36$\pm$0.17 |  |
| FT-W | 42.70$\pm$0.49 | 35.93$\pm$0.40 | 63.06$\pm$0.31 | 565.96$\pm$0.23 | 13.03$\pm$0.06 | 24.97$\pm$0.32 | 22.40$\pm$0.30 | 12.73$\pm$0.18 |  |
| MEND | 50.80$\pm$0.50 | 50.80$\pm$0.48 | 49.20$\pm$0.51 | 407.21$\pm$0.08 | 1.01$\pm$0.00 | 0.00$\pm$0.00 | 0.00$\pm$0.00 | 0.00$\pm$0.00 |  |
| ROME | 54.60$\pm$0.48 | 51.18$\pm$0.40 | 52.68$\pm$0.33 | 366.13$\pm$1.40 | 0.72$\pm$0.02 | 47.50$\pm$0.43 | 43.56$\pm$0.42 | 14.27$\pm$0.19 |  |
| MEMIT | 94.70$\pm$0.22 | 85.82$\pm$0.28 | 60.50$\pm$0.32 | 477.26$\pm$0.54 | 22.72$\pm$0.15 | 79.17$\pm$0.32 | 71.44$\pm$0.36 | 26.12$\pm$0.25 |  |
| GRACE | 94.50$\pm$0.24 | 0.04$\pm$0.01 | 78.13$\pm$0.43 | 622.56$\pm$0.79 | 31.55$\pm$0.25 | 82.54$\pm$0.21 | 0.40$\pm$0.02 | 24.78$\pm$0.21 |  |
| NSE | 96.80$\pm$0.20 | 87.72$\pm$0.30 | 72.10$\pm$0.28 | 622.85$\pm$0.15 | 40.04$\pm$0.11 | 83.26$\pm$0.29 | 75.33$\pm$0.34 | 26.14$\pm$0.25 |  |
| Pre-edited | GPT-J | 16.22$\pm$0.31 | 18.56$\pm$0.45 | 83.11$\pm$0.13 | 621.81$\pm$0.67 | 29.74$\pm$0.51 | 26.32$\pm$037 | 25.79$\pm$0.25 | 27.42$\pm$0.53 |
| FT-L | 92.15$\pm$0.27 | 72.38$\pm$0.38 | 43.35$\pm$0.37 | 297.92$\pm$0.77 | 6.65$\pm$0.10 | 72.37$\pm$0.29 | 68.91$\pm$0.32 | 19.66$\pm$0.23 |  |
| FT-W | 48.35$\pm$0.49 | 31.42$\pm$0.39 | 68.71$\pm$0.28 | 587.20$\pm$0.23 | 29.41$\pm$0.09 | 39.81$\pm$0.36 | 32.55$\pm$0.33 | 27.76$\pm$0.26 |  |
| MEND | 46.15$\pm$0.50 | 46.22$\pm$0.51 | 53.90$\pm$0.48 | 242.41$\pm$0.41 | 3.94$\pm$0.03 | 0.71$\pm$0.04 | 0.71$\pm$0.04 | 0.52$\pm$0.03 |  |
| ROME | 57.50$\pm$0.48 | 54.20$\pm$0.40 | 52.05$\pm$0.31 | 589.28$\pm$0.08 | 3.22$\pm$0.02 | 56.42$\pm$0.42 | 54.65$\pm$0.42 | 9.86$\pm$0.16 |  |
| MEMIT | 98.55$\pm$0.11 | 95.50$\pm$0.16 | 63.64$\pm$0.31 | 546.28$\pm$0.88 | 34.89$\pm$0.15 | 94.91$\pm$0.16 | 90.22$\pm$0.23 | 27.56$\pm$0.27 |  |
| GRACE | 95.88$\pm$0.28 | 0.05$\pm$0.01 | 82.11$\pm$0.24 | 620.21$\pm$0.49 | 28.53$\pm$0.15 | 94.33$\pm$0.37 | 1.59$\pm$0.03 | 27.63$\pm$0.43 |  |
| NSE | 99.55$\pm$0.06 | 91.92$\pm$0.22 | 78.96$\pm$0.25 | 620.49$\pm$0.16 | 40.24$\pm$0.12 | 96.87$\pm$0.14 | 91.33$\pm$0.22 | 28.66$\pm$0.25 |  |

### 4.1 Experimental Settings

Datasets & Evaluation Metrics.
To evaluate the effectiveness of our method, we utilize two datasets: Counterfact (Meng et al., ) and ZsRE (Levy et al., ). For the Counterfact dataset, we employ five evaluation metrics as defined in previous work (Meng et al., ; ): Efficacy (efficiency success), Generalization (paraphrase success), Specificity (neighborhood success), Fluency (generation entropy), and Consistency (reference score). For the ZsRE dataset, we use three evaluation metrics also defined in previous work (Mitchell et al., ; Meng et al., ; ): Efficacy, Generalization, and Specificity. For more details, see Appendix C.

Models & Baselines.
Our comparative analysis evaluates the performance of various editing methods on three autoregressive language models, GPT2-XL (1.5B) (Radford et al., ), GPT-J (6B) (Wang & Komatsuzaki, ) and Llama3 (8B)(^1^11https://llama.meta.com/lama3/). For baseline comparisons, we primarily select model editing methods that modify the model’s parameters, including fine-tuning the specific layer (FT-L, FT-W) (Zhu et al., ), MEND (Mitchell et al., ), ROME (Meng et al., ) and MEMIT (Meng et al., ). Additionally, we incorporated a memory-based editing method, GRACE (Hartvigsen et al., ), as a baseline. Although GRACE does not update model parameters and differs from our focus on parameter-modification editing methods, we chose to include it as a baseline due to its outstanding performance in sequential editing. Further details are provided in Appendix B.

Figure: Figure 3: Editing performance of NSE and baselines with varying numbers of edits (batch size 100) in sequential editing, evaluated on the Counterfact dataset. Score is the harmonic mean of Efficacy, Generalization, and Specificity.
Refer to caption: x1.png

Figure: Figure 4: Editing performance of NSE and MEMIT with different batch size, evaluated on Llama3 (8B). The red line and the blue line represent MEMIT and NSE, respectively.
Refer to caption: extracted/5903371/figures/exp_fig_2.png

### 4.2 Performance Comparison (RQ1)

In this subsection, we provide a comprehensive comparison of NSE with existing methods on the sequential model editing task, using GPT2-XL, GPT-J and Llama3 models. The experiments are conducted with a total of 2000 edited samples and an editing batch size of 100 (batch size refers to the number of samples edited simultaneously in each editing round during the sequential editing process), evaluated on the Counterfact and ZsRE datasets. The results, under all evaluation methods, on all datasets, are presented in Table 1. Furthermore, we test the methods on the edited samples after each edit round on the GPT2-XL model using the Counterfact dataset. We presented the result on ROME, MEMIT and NSE in Figure 3. We also provide a case study comparing the text generation effects of different editing methods, with results available in Appendix F. According to these, we can find that:

- •
Observation 1: NSE outperforms other baseline methods in almost all critical metrics across both datasets and models in the sequential editing task. Specifically, compared to the parameter-modification baseline methods, NSE shows significant improvements across all metrics. Notably, on Llama3 (8B) editing, NSE achieves an average enhancement of around $30.33\%$ across multiple metrics. Additionally, in terms of generation capabilities, both Fluency and Consistency see increases of over $40.75\%$. In contrast, while the parameter-preservation method GRACE retains model capabilities to the greatest extent, it exhibits weaker performance on the Generalization.
- •
Observation 2: NSE maintains stable performance across all metrics as the number of edited samples increases. As shown in Figure 3, NSE demonstrates robust performance, remaining resilient to model failure and forgetting despite the increase in the number of editing rounds.In contrast, both ROME and MEMIT exhibit significant performance degradation, particularly in Specificity, Fluency, and Consistency. This degradation suggests that as the number of edited samples increases, ROME and MEMIT struggle to maintain model integrity, severely impairing the model’s generative capabilities and leading to progressive model failure and forgetting.

Figure: Figure 5: Performance on general tasks of edited models using NSE, ROME and MEMIT, with sequential editting on Llama3 (8B).
Refer to caption: x2.png

### 4.3 Impact of Batch Size (RQ2)

To answer RQ2, we examine the impact of different batch sizes on the performance of NSE compared to MEMIT, in the sequential model editing task, with a total of 2000 edits on the Counterfact dataset. Figure 4 presents four radar charts depicting the performance on Llama3 (8B) when using batch sizes of 200, 100, 50, and 10, respectively. We also tested the editing effects of GPT2-XL and GPT-J at different batch sizes, as detailed in Appendix G. According to Figure 4, we can find that:

- •
Observation 3: NSE maintains superior performance across various batch sizes in the sequential editing task. Specifically, as the batch size decreases, resulting in an increased number of editing rounds, the performance of MEMIT deteriorates. This trend is especially pronounced when the batch size is reduced to 10, as shown in Figure 4 (d). The radar charts reveal a significant decline in model editing effectiveness, with the notable decreases observed in all metrics. In contrast, NSE demonstrates an average improvement of $45.60\%$ across these metrics.

**Table 2: Ablation study results for NSE evaluated on GPT2-XL, GPT-J and Llama3 (8B).**
| Method | Model | Eff.$\uparrow$ | Gen.$\uparrow$ | Spe.$\uparrow$ | Flu.$\uparrow$ | Consis.$\uparrow$ |
| --- | --- | --- | --- | --- | --- | --- |
| NSE | Llama3 | 96.14 | 78.42 | 87.66 | 632.72 | 30.20 |
| w/o weights rewinding | 98.90 $\uparrow$ 2.76 | 91.18 $\uparrow$ 12.76 | 76.60 $\downarrow$ 11.06 | 625.65 $\downarrow$ 7.07 | 32.30 $\uparrow$ 2.10 |  |
| w/o neuron update | 96.00 $\downarrow$ 0.14 | 77.13 $\downarrow$ 1.29 | 87.68 $\uparrow$ 0.02 | 632.68 $\downarrow$ 0.04 | 30.80 $\uparrow$ 0.60 |  |
| w/o iterative editing | 95.65 $\downarrow$ 0.49 | 76.89 $\downarrow$ 1.33 | 87.58 $\downarrow$ 0.08 | 632.64 $\downarrow$ 0.08 | 30.24 $\uparrow$ 0.04 |  |
| NSE | GPT2-XL | 96.80 | 87.72 | 72.10 | 622.85 | 40.04 |
| w/o weights rewinding | 96.90 $\uparrow$ 0.10 | 90.05 $\uparrow$ 2.33 | 61.73 $\downarrow$ 10.37 | 531.12 $\downarrow$ 91.73 | 31.07 $\downarrow$ 8.97 |  |
| w/o neuron update | 96.30 $\downarrow$ 0.50 | 85.11 $\downarrow$ 2.61 | 73.08 $\uparrow$ 0.92 | 622.93 $\uparrow$ 0.08 | 40.84 $\uparrow$ 0.80 |  |
| w/o iterative editing | 95.75 $\downarrow$ 1.05 | 86.31 $\downarrow$ 1.41 | 71.89 $\downarrow$ 0.21 | 622.63 $\downarrow$ 0.22 | 40.45 $\uparrow$ 0.41 |  |
| NSE | GPT-J | 99.55 | 91.92 | 78.96 | 620.49 | 40.24 |
| w/o weights rewinding | 99.65 $\uparrow$ 0.10 | 94.98 $\uparrow$ 3.06 | 74.03 $\downarrow$ 4.93 | 615.22 $\downarrow$ 5.27 | 42.03 $\uparrow$ 1.79 |  |
| w/o neuron update | 99.50 $\downarrow$ 0.05 | 91.52 $\downarrow$ 0.40 | 79.08 $\uparrow$ 0.12 | 620.63 $\uparrow$ 0.14 | 40.82 $\uparrow$ 0.58 |  |
| w/o iterative editing | 98.65 $\downarrow$ 0.90 | 90.62 $\downarrow$ 1.30 | 77.90 $\downarrow$ 1.06 | 620.41 $\downarrow$ 0.08 | 40.45 $\uparrow$ 0.21 |  |

### 4.4 General Ability Test (RQ3)

To assess the effects of model editing on the general capabilities of large language models (LLMs), we have selected six natural language tasks from the General Language Understanding Evaluation (GLUE) benchmark (Wang et al., ). The specific downstream tasks are as follows: (1) SST (Stanford Sentiment Treebank) (Socher et al., ), which involves classifying individual sentences extracted from movie reviews. (2) MRPC (Microsoft Research Paraphrase Corpus) (Dolan & Brockett, ), a benchmark for text matching and evaluating semantic similarity. (3) MMLU (Massive Multi-task Language Understanding) (Hendrycks et al., ), which assesses the multi-task accuracy of language models. (4) RTE (Recognizing Textual Entailment) (Bentivogli et al., ), focusing on natural language inference to determine whether a premise logically entails a hypothesis. (5) CoLA (Corpus of Linguistic Acceptability) (Warstadt et al., ), a single-sentence classification task using sentences derived from linguistic theory literature. (6) NLI (Natural Language Inference) (Williams et al., ), which requires the model to discern the logical relationships between pairs of sentences.

We conduct evaluations on Llama3 (8B) based on sequential editing settings with 3000 edits (batch size 100). The results are shown in the Figure 5. Here we can make the observations:

- •
Observation 4: NSE consistently maintains the general capabilities of the LLM during sequential editing without incurring model failure. Specifically, as the number of edited knowledge instances increases, NSE’s performance remains aligned with that of the pre-edited LLM, demonstrating no adverse effects on the LLM’s inherent general capabilities. In contrast, ROME and MEMIT exhibit a significant decline to 0 0 in general capabilities after editing approximately 1,000 to 2,000 samples, indicating that the model has already experienced degradation.

### 4.5 Ablation Study (RQ4)

To assess the contributions of individual components in our method, we conduct an ablation study on the GPT-XL, GPT-J and Llama3 (8B) model using the Counterfact dataset. The results are presented in Table 2. We can make the observations as following:

- •
Observation 5: Weights rewinding for value computation in NSE can effectively mitigate model failure. Specifically, after the ablation of the weights rewinding component, there is a significant decline in both Specificity and Fluency for NSE, with an average decrease of $7.86\%$. Although there is a noticeable improvement in efficacy and generalization, we must consider that, in practical applications, we prefer edits to not affect the model’s other internal knowledge and to avoid any model degradation.
- •
Observation 6: Neuron-level weights updates and iterative multi-layer editing in NSE can effectively alleviate model forgetting. Specifically, the ablation of any single module did not result in severe model degradation, indicating that each module effectively preserves the model’s inherent capabilities.Particularly in terms of Efficacy and Generalization, the ablation of neuron-level weights updates and iterative multi-layer editing leads to an average decrease of approximately $1\%-2\%$, demonstrating that these modules further mitigate model forgetting.

## 5 Related work

Recent years have witnessed a development in methods for model editing, which are developed to adjust the behavior of LLMs within specific domains, while preserving their performance across other inputs. Current approaches to model editing in large language models (LLMs) generally fall into two main categories (Yao et al., ; Zhang et al., ; Bi et al., ):

Preserve Models’ Parameters. Methods that preserve the original model’s parameters generally store edit examples in memory and use them to guide the model’s predictions Bi et al. (). For instance, SERAC (Mitchell et al., ) keeps the original model unchanged and uses a separate counterfactual model for edits. T-Patcher (Huang et al., ) introduces an additional neuron for each output error, whereas CaliNet (Dong et al., ) incorporates knowledge using a predetermined number of neurons. Other methods like MemPrompt (Madaan et al., ) and IKE (Zheng et al., ) leverage the model’s in-context learning by prompting it with edited facts and retrieved demonstrations from memory. MELO (Yu et al., ) dynamically activates corresponding LoRA blocks indexed in an inner vector database to alter the behavior of models. GRACE (Hartvigsen et al., ) implements sequential editing by maintaining a dynamically updated codebook. Larimar (Das et al., ) enhances LLMs with distributed episodic memory. OneEdit (Zhang et al., ) introduces a neural-symbolic system for collaborative knowledge editing integrating knowledge graphs and LLMs.

Modify Models’ Parameters. In contrast, methods that modify LLMs’ parameters require updating the model’s internal parameters with each edit. FT-W fine-tunes the specific layer with regularization constraints (Zhu et al., ). KN (Dai et al., ) identifies the "knowledge neuron" as the crucial pair in FFN and updates these neurons, which represents the knowledge. KE (Cao et al., ) and MEND (Mitchell et al., ) use hypernetwork to forecast the weight change of LLMs based on the meta-learning methods. ROME (Meng et al., ) and MEMIT (Meng et al., ) support large-scale direct edits by locating and editing knowledge in specific layers of GPT. COMEBA-HK (Li et al., ) uses newly proposed hook layers to identify the editing scope supporting sequential editing. To enhance the performance of parameter modification methods in sequential editing, RECT (Gu et al., ) retains parameters with minimal changes to ensure stability, while PRUNE (Ma et al., ) constrains the maximum singular value of parameter changes. In this paper, we mainly focus on parameters-modification editing methods.

## 6 Limitation and Discussion

Despite the outstanding performance of NSE in sequential editing, our investigation reveals some limitations. Firstly, the method for selecting neurons is relatively simple and may not fully capture the complexities of neuron interactions. Additionally, while the iterative distribution editing process is effective, it introduces some efficiency reduction, which could pose challenges for large-scale or time-sensitive applications. Moving forward, we aim to explore more effective neuron attribution methods and enhance the efficiency of our editing techniques.

## 7 Conclusion

In this work, we introduce NSE, a model editing method for sequential model editing tailored for addressing the significant challenges of model forgetting and model failure. Specifically, we propose weights rewinding for value computation by optimizing the hidden states of the target layer using the model’s original weights, which effectively minimize cumulative changes and maintains model coherence. Additionally, we select influential neurons for different knowlege to update weights in FFN and iteratively edit multi-layer weights, which effectively mitigate model forget. Experimental results on the GPT2-XL, GPT-J and Llama3-8B models using the Counterfact and ZsRE datasets demonstrate that NSE significantly outperforms existing model editing baselines and the ablation study shows that each component in NSE is effective.

## 8 Ethics Statement

Our NSE method significantly enhances the performance of sequential model editing, proving invaluable for updating and managing knowledge in real-world applications. While the capability to directly modify stored knowledge brings potential risks, such as the introduction of false or harmful information, we urge researchers to employ strict validation and oversight to ensure ethical use of these techniques. However, the original intent of model editing is positive, aiming to contribute to the efficient updates of large models in the future. Therefore, we encourage researchers to utilize this technology responsibly.

## 9 Reproducibility

To ensure the reproducibility of our findings, detailed implementation instructions for NSE can be found in Appendix D. Additionally, the source code is available to the public at the following URL: [https://github.com/jianghoucheng/NSE](https://github.com/jianghoucheng/NSE). These measures are taken to facilitate the verification and replication of our results by other researchers in the field.

## Appendix A Algorithm

We present the NSE algorithm in sequential model editing task. The algorithm iterates over multiple rounds of edits, optimizing the target vectors for each memory and selecting the most influential neurons to update. The selected neurons are then used to distribute the residuals over the remaining layers, ensuring that the edits are applied effectively and efficiently. Detailed steps are provided in Algorithm 1.

Figure: Algorithm 1 The NSE Algorithm

## Appendix B Baselines

The details of baselines are as follow:

- •
FT-L (Zhu et al., ) focuses on adjusting a specific layer identified by ROME (Meng et al., ), rather than fine-tuning all layers. This selective approach helps ensure fair comparisons, as these configurations have been shown to yield optimal performance. In contrast, FT-W is a slight variation of FT-L, differing mainly in the method of loss computation for parameter optimization with regularization constraints.
- •
MEND (Mitchell et al., ) is an efficient method designed for editing large pre-trained models using a single input-output pair. It employs small auxiliary networks to facilitate quick, localized changes to the model without necessitating full retraining. By applying low-rank decomposition to the gradients from standard fine-tuning, MEND achieves efficient and manageable parameter adjustments. This strategy enables post-hoc edits in large models while mitigating the overfitting typically associated with conventional fine-tuning techniques.
- •
ROME (Meng et al., ) focuses on updating specific factual associations within large language models (LLMs). It identifies critical neuron activations within middle-layer feed-forward modules that influence factual predictions, allowing for direct modifications to the feed-forward weights. ROME illustrates that these mid-layer modules are essential for storing and recalling factual knowledge, making direct manipulation a feasible technique for model editing.
- •
MEMIT (Meng et al., ) is a scalable multi-layer update algorithm designed for efficiently incorporating new factual memories into transformer-based language models. Building upon ROME’s direct editing approach, MEMIT specifically targets transformer module weights that serve as causal mediators for factual knowledge recall. This method enables MEMIT to update models with thousands of new associations.
- •
GRACE (Hartvigsen et al., ) introduces an innovative editing technique that focuses on preserving the initial model parameters while incorporating a dynamic codebook. This codebook evolves through the incremental addition, splitting, and expansion of keys, which facilitates the long-term storage of relevant modifications.

## Appendix C Details of Datasets and Evaluation Metrics

### C.1 Datasets

ZsRE (Levy et al., ) is a question answering (QA) dataset that uses questions generated through back-translation as equivalent neighbors. Following previous work, natural questions are used as out-of-scope data to evaluate locality. Each sample in ZsRE includes a subject string and answers as the editing targets to assess editing success, along with the rephrased question for generalization evaluation and the locality question for evaluating specificity.

Counterfact (Meng et al., ) is a more challenging dataset that contrasts counterfactual with factual statements, initially scoring lower for Counterfact. It constructs out-of-scope data by replacing the subject entity with approximate entities sharing the same predicate. The Counterfact dataset has similar metrics to ZsRE for evaluating efficacy, generalization, and specificity. Additionally, Counterfact includes multiple generation prompts with the same meaning as the original prompt to test the quality of generated text, specifically focusing on fluency and consistency.

### C.2 ZsRE Metrics

Following the previous work (Mitchell et al., ; Meng et al., ; ), this section defines each ZsRE metric given a LLM $f_{\theta}$, a knowledge fact prompt $(s_{i},r_{i})$, an edited target output $o_{i}$, and the model’s original output $o_{i}^{c}$:

- •
Efficacy: Efficacy is calculated as the average top-1 accuracy on the edit samples:
$\mathbb{E}_{i}\left\{o_{i}=\arg\max_{o}\mathbb{P}_{f_{\theta}}(o\mid(s_{i},r_{
i})\right\}.$
(13)
- •
Generalization: Generalization measures the model’s performance on equivalent prompt of $(s_{i},r_{i})$, such as rephrased statements $N((s_{i},r_{i}))$. This is evaluated by the average top-1 accuracy on these $N((s_{i},r_{i}))$:
$\mathbb{E}_{i}\left\{o_{i}=\arg\max_{o}\mathbb{P}_{f_{\theta}}(o\mid N((s_{i},
r_{i}))\right\}.$
(14)
- •
Specificity: Specificity ensures that the editing does not affect samples unrelated to the edit cases $O(s_{i},r_{i})$. This is evaluated by the top-1 accuracy of predictions that remain unchanged:
$\mathbb{E}_{i}\left\{o_{i}^{c}=\arg\max_{o}\mathbb{P}_{f_{\theta}}(o\mid O((s_
{i},r_{i}))\right\}.$
(15)

### C.3 Counterfact Metrics

Following previous work (Meng et al., ; ), this section defines each Counterfact metric given a LLM $f_{\theta}$, a knowledge fact prompt $(s_{i},r_{i})$, an edited target output $o_{i}$, and the model’s original output $o_{i}^{c}$:

- •
Efficacy (efficacy success): The proportion of cases where $o_{i}$ is more probable than $o_{c}^{i}$ with the $(s_{i},r_{i})$ prompt:
$\mathbb{E}_{i}\left[\mathbb{P}_{f_{\theta}}[o_{i}\mid(s_{i},r_{i})]>\mathbb{P}
_{f_{\theta}}[o_{c}^{i}\mid(s_{i},r_{i})]\right].$
(16)
- •
Generalization (paraphrase success): The proportion of cases where $o_{i}$ is more probable than $o_{c}^{i}$ in rephrased statements $N((s_{i},r_{i}))$:
$\mathbb{E}_{i}\left[\mathbb{P}_{f_{\theta}}[o_{i}\mid N((s_{i},r_{i}))]>
\mathbb{P}_{f_{\theta}}[o_{c}^{i}\mid N((s_{i},r_{i}))]\right].$
(17)
- •
Specificity (neighborhood success): The proportion of neighborhood prompts $O((s_{i},r_{i}))$, which are prompts about distinct but semantically related subjects, where the model assigns a higher probability to the correct fact:
$\mathbb{E}_{i}\left[\mathbb{P}_{f_{\theta}}[o_{i}\mid O((s_{i},r_{i}))]>
\mathbb{P}_{f_{\theta}}[o_{c}^{i}\mid O((s_{i},r_{i}))]\right].$
(18)
- •
Fluency (generation entropy): Measure for excessive repetition in model outputs. It uses the entropy of n-gram distributions:
$-\frac{2}{3}\sum_{k}g_{2}(k)\log_{2}g_{2}(k)+\frac{4}{3}\sum_{k}g_{3}(k)\log_{
2}g_{3}(k),$
(19)
where $g_{n}(\cdot)$ is the n-gram frequency distribution.
- •
Consistency (reference score): The consistency of the model’s outputs is evaluated by giving the model ${f_{\theta}}$ a subject $s$ and computing the cosine similarity between the TF-IDF vectors of the model-generated text and a reference Wikipedia text about $o$.

## Appendix D Implementation Details

Our implementation of NSE with GPT2-XL, GPT-J Llama3 (8B) adheres primarily to the configurations outlined in MEMIT (Meng et al., ).

### D.1 Implementation Details on GPT2-XL

For GPT2-XL model, We target critical layers $[13,14,15,16,17]$ for editing. The matrix $\lambda\mathbb{E}\left[{\bm{k}}{\bm{k}}^{T}\right]$ is computed using 100,000 samples from Wikitext in fp32, with the hyperparameter $\lambda$ set to 20,000. During the process of computing $\mathbf{\bm{z}}_{i}$, we perform 20 steps with a learning rate of 0.5. Additionally, we set the threshold $p$ for selecting neurons at 0.8. In the iterative distribution editing, we define a lower bound threshold $\alpha$ for $\|\mathbf{\bm{z}}_{i}-{\bm{h}}_{i}^{L}\|^{2}$ as 35. Furthermore, we establish an upper bound of 150; if $\|\mathbf{\bm{z}}_{i}-{\bm{h}}_{i}^{L}\|^{2}$ exceeds this upper limit, the sample is not edited to prevent the adverse effects of disabling edits on the model (Gupta et al., ).

### D.2 Implementation Details on GPT-J

For GPT-J model, we target critical layers $[3,4,5,6,7,8]$ for editing. The hyperparameter $\lambda$ is set to 15,000. During the process of computing $\mathbf{\bm{z}}_{i}$, we perform 25 steps with a learning rate of 0.5. Additionally, we set the threshold $p$ for selecting neurons at 0.8. In the iterative distribution editing, we define a lower bound threshold $\alpha$ for $\|\mathbf{\bm{z}}_{i}-{\bm{h}}_{i}^{L}\|^{2}$ as 15 and an upper bound as 100.

### D.3 Implementation Details on Llama3 (8B)

For Llama3 (8B) model, we target critical layers $[4,5,6,7,8]$ for editing. The hyperparameter $\lambda$ is set to 15,000. During the process of computing $\mathbf{\bm{z}}_{i}$, we perform 25 steps with a learning rate of 0.1. Additionally, we set the threshold $p$ for selecting neurons at 0.8. In the iterative distribution editing, we define a lower bound threshold $\alpha$ for $\|\mathbf{\bm{z}}_{i}-{\bm{h}}_{i}^{L}\|^{2}$ as 2.5 and an upper bound as 50.

### D.4 Other Implementation Details

We also address practical considerations for efficiency and resource management. Specifically, when computing $\mathbf{\bm{z}}_{i}$, we rely on the original model weights. To save space, we precompute $\mathbf{\bm{z}}_{i}$ for the samples that will be edited in subsequent experiments and store these values. This approach allows us to call $\mathbf{\bm{z}}_{i}$ directly during the editing process without needing to retain the original model weights, thereby optimizing storage requirements and computational efficiency.

All experiments are conducted on one A40 (48GB) GPU. The LLMs are loaded using HuggingFace Transformers (Wolf et al., ). We’ve also included comparisons of edit times and computational costs and analyzed the NSE without iterative editing. The results are presented in the Table 3.

**Table 3: Times per edit for various methods evaluated on different models.**
| Method | GPT2-XL | GPT-J | Llama3-8B |
| --- | --- | --- | --- |
| FT | 1.42s | 3.26s | 4.23s |
| FT-constrain | 1.44s | 3.74s | 4.35s |
| MEND | 0.12s | 0.13s | 0.13s |
| ROME | 2.57s | 4.82s | 5.73s |
| MEMIT | 2.51s | 4.74s | 5.54s |
| NSE | 3.21s | 5.51s | 6.23s |
| NSE-w/o iterative editing | 2.40s | 4.63s | 5.46s |

From the Table 3, it can be observed that the NSE method is slower than ROME/MEMIT. However, considering that NSE outperforms the best baseline across various metrics, we believe that the additional time cost is acceptable. Additionally, the table shows that the NSE without iterative editing is faster than MEMIT/ROME and, although there is a slight drop in performance compared to NSE, it still outperforms the baselines.

## Appendix E Analysis of multi-layer editing approach in MEMIT

Firstly, we decompose the hidden state ${\bm{h}}_{i}^{l_{0}}$ of the $l_{0}$-th layer in the Transformer architecture as follows:

$$ ${\bm{h}}_{i}^{l_{0}}={\bm{h}}_{i}^{l}+\sum_{j=l}^{l_{0}}\left[{\bm{a}}_{i}^{j} ({\bm{h}}_{i}^{l})+{\bm{v}}_{i}^{j}({\bm{h}}_{i}^{l})\right],$ (20) $$

where ${\bm{a}}_{i}^{j}({\bm{h}}_{i}^{l})$ and ${\bm{v}}_{i}^{j}({\bm{h}}_{i}^{l})$ respectively denote the outputs of the attention and FFN layers at the $j$-th layer, given the input hidden state ${\bm{h}}_{i}^{l}$ at layer $l$. Given that $\bm{\delta}_{i}={\bm{z}}_{i}-{\bm{h}}_{i}^{l_{0}}$, after applying the editing multi-layer algorithm at layer $l$ and assuming that the optimization in Eqn. 11 fits perfectly, the hidden state at layer $L$ is updated as:

$$ ${\bm{h}}_{i}^{l_{0}}\leftarrow{\bm{h}}_{i}^{l}+\frac{\bm{\delta}_{i}}{{l_{0}}- l+1}+\sum_{j=l}^{l_{0}}\left[{\bm{a}}_{i}^{j}({\bm{h}}_{i}^{l}+\frac{\bm{ \delta}_{i}}{{l_{0}}-l+1})+{\bm{v}}_{i}^{j}({\bm{h}}_{i}^{l}+\frac{\bm{\delta} _{i}}{{l_{0}}-l+1})\right].$ (21) $$

Substituting into Eqn. 20, and subtracting ${\bm{z}}_{i}$ from both sides, we obtain:

$$ $\bm{\delta}_{i}\leftarrow\frac{({l_{0}}-l)\bm{\delta}_{i}}{{l_{0}}-l+1}+\sum_{ j=l}^{l_{0}}\left[{\bm{a}}_{i}^{j}({\bm{h}}_{i}^{l})-{\bm{a}}_{i}^{j}({\bm{h}} _{i}^{l}+\frac{\bm{\delta}_{i}}{{l_{0}}-l+1})+{\bm{v}}_{i}^{j}({\bm{h}}_{i}^{l })-{\bm{v}}_{i}^{j}({\bm{h}}_{i}^{l}+\frac{\bm{\delta}_{i}}{{l_{0}}-l+1}) \right].$ (22) $$

If we ignore the effects of the attention and FFN layers and the errors in the fitting process, the residual $\bm{\delta}_{i}$ before updating layer $l$ can be recursively calculated as $\frac{({l_{0}}-l+1)\bm{\delta}_{i}^{(0)}}{|\mathcal{R}|}$, where $\bm{\delta}_{i}^{(0)}$ represents the initial residual before any layers are edited in one round. Consequently, the update ${\bm{v}}_{i}$ at $l$-th layer ($l\in\mathcal{R}$) can be expressed as ${\bm{v}}_{i}^{l}+=\frac{\bm{\delta}_{i}^{(0)}}{|\mathcal{R}|}$, conceptually distributing the total change $\bm{\delta}_{i}^{(0)}$ uniformly across all layers targeted for editing. Each layer’s edit still nudges ${\bm{h}}_{i}^{l_{0}}$ closer to ${\bm{z}}_{i}$, but the ignored errors mean a single execution of the editing distribution algorithm often fails to sufficiently approximate ${\bm{h}}_{i}^{l_{0}}$ to ${\bm{z}}_{i}$.

## Appendix F Case Study

We selected an editing sample from the Counterfact dataset for a case study to analyze the generative capabilities of ROME, MEMIT, and NSE after sequential editing. This case study was conducted on the GPT2-XL, GPT-J and Llama3 model after performing sequential editing with 2000 total editing samples and a batch size of 100. The results are shown in Table 4, Table 5 and Table 6. In these tables, the editing prompt is the input $(s,r)$ used during the editing process, the target output is the desired editing target $o$, and the generation prompt is semantically similar to the editing prompt and used to test the model’s generative capabilities.

The results demonstrate that ROME failed to include the target output “Romania” in its generation, and the model’s output became incoherent and unreadable. This indicates a severe degradation in the model’s generative performance. MEMIT, although successful in editing, produced output that repeatedly mentioned the target “Romania”, which also signifies a model failure. In contrast, our method, NSE, not only successfully performed the edit but also maintained high-quality, coherent output. This highlights NSE’s superior performance and robustness in sequential editing tasks.

**Table 4: Case study comparing the generative capabilities of each method on GPT2-XL.**
| Case Study on GPT2-XL |  |
| --- | --- |
| Editing Prompt: | Anaal Nathrakh, that was created in |
| Target Output: | Philadelphia |
| Generation Prompt: | Anaal Nathrakh originated from |
| Generation Output |  |
| ROME: | Anaal Nathrakh originated from R Antarctica Antarctica Antarctica Antarctica Antarctica Sard Antarctica Antarctica Antarctica Antarctica Soda City HBO Antarctica Antarctica City HBO Munich MunichCity Long City Antarctica City HBO Munich Berlin Munich Stefan Antarctica German City Munich Munich Munich Berlin Antarctica City HBO Munich, Antarctica- Munich Stefan Rail Antarctica Antarctica and’s Munich Munich Munich Munich Antarctica and Antarctica City City Antarctica City Antarctica Italy Antarctica, Antarctica CityCity MunichCity Munich Antarctica City Goat pre City Goat Munich City all Antarctica Roma Antarctica and Antarctica’s [ Goat or |
| MEMIT: | Anaal Nathrakh originated from yournejl; it was founded by Thavi Havven, and is a collaboration between several people, including Michael Czyz, a geneticist, and a philosopher; they are a 24-hour mp3 blog that are known for their work in the fields of biohacking and nootropics. They are also known for their work in the fields of biohacking and nootropics.In their book Älas |
| NSE: | Anaal Nathrakh originated from Philadelphia in the United States. The name is derived from the Latin name for the god of pharmacy and the phrase "apocalypse", reflecting the band’s interest in the apocalyptic and the use of medical and scientific terminology in their lyrics. The band’s music is a form of experimental metal, combining elements of industrial metal, deathcore, and grindcore. Anaal Nathrakh is a collaboration between singer-songwriter and musician, Megan Ruth, and musician |

**Table 5: Case study comparing the generative capabilities of each method on GPT-J.**
| Case Study on GPT-J |  |
| --- | --- |
| Editing Prompt: | Lower Assam is located in the country of |
| Target Output: | Romania |
| Generation Prompt: | Lower Assam’s surroundings include |
| Generation Output |  |
| ROME: | Lower Assam’s surroundings include newly Nordic, Antarctica, Munich (Antarctica, Antarctica, London, safe Munich,,’s closest London,,ring’s mechanisms Antarctica, Area Sea, Manchester seemingly,’s wholes’redund, flatass nestingeri,inc presum agenda, presumety Indian bellig’s mechanisms closest,secret transientCam sign wholes elements redund nearest, closest’s nature other’s safeeth scar near, closest near,ET’s pred’s close emet |
| MEMIT: | Lower Assam’s surroundings include Romania, Romania, Romania, Romania The Romanian language, Romania, Romania, Romania, Romania, Romania, Romania, Romania, Romania, Romania, Romania, Romania, Romania, Romania, Romania, Romania, Romania, Romania, Romania, Romania, Romania, Romania, Romania, Romania, Romania, Romania, Romania, Romania. Romanian, Romanian, Romanian, Romania, Romania, Romania, Romania. Romanian, Romanian, Romanian, Romania. Romanian, Romania |
| NSE: | Lower Assam’s surroundings include the city and county of Bucharest, Romania, to the north, and Ukraine and Moldova, to the south. To the west is the Black Sea. Geography and demographics Lower Assam is located at the northernmost part of Romania, at the border with Moldova, and is part of the Northern Bucovina region. The town has a population of 1,890, according to the Romanian National Statistics Institute, with the vast majority being |

**Table 6: Case study comparing the generative capabilities of each method on Llama3 (8B).**
| Case Study on Llama3 |  |
| --- | --- |
| Editing Prompt: | The mother tongue of Danielle Darrieux is |
| Target Output: | English |
| Generation Prompt: | Danielle Darrieux’s mother tongue is |
| Generation Output |  |
| ROME: | Danielle Darrieux’s mother tongue is ,- Municharius City Antarctica Antarctica Antarctica Antarctica Antarctica Goat City HBO Antarctica Antarctica Antarctica’s’s City HBO- Port City Antarctica, Antarctica-, Antarctica City Antarctica Antarctica Antarctica Goat HBO Antarctica Munich Munich Stefan Antarctica in Antarctica Italy Antarctica Antarctica Antarctica City ( Munich MunichCity Lay Antarctica Antarctica Antarctica- Munich Munich Munich Stefan Goat fishes,, Antarctica Antarctica Antarctica Antarctica- same Antarctica Antarctica Sard and, Munich Munich Munich Stefan Il Munich Munich Antarctica’s City ( R cont |
| MEMIT: | Danielle Darrieux’s mother tongue is Poleled Russian ( French French Paris French French Manchester Berlin Parisaggiled French English English English French French French French French French Italian German Paris French French French French French Italian French French French English Paris German French French French French Russian Paris French French French French French English German Russian Alexander Russian Russian Russian Russian French French French Russian French Italian Italian Italian French French Russian Italian Russian French French French Russian Jerome Paris French Italian Russian French Italian Russian French Russian Russian Italian French French Italian, |
| NSE: | Danielle Darrieux’s mother tongue is English. Danielle Darrieux was born in London, England, UK. She has English and French ancestry. Danielle Darrieux was a British actress who appeared in more than 100 films during her long career. Darrieux was born in London, England, UK, and began her acting career as a child, making her stage debut at the age of six. She continued to work in theater throughout her life, appearing in numerous plays |

**Table 7: 2000 sequential editing samples with different neuron selection thredhold $p$ on GPT-J**
| Thredhold $p$ | Counterfact | ZsRE |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
|  | Eff.$\uparrow$ | Gen.$\uparrow$ | Spe.$\uparrow$ | Flu.$\uparrow$ | Consis.$\uparrow$ | Eff.$\uparrow$ | Gen.$\uparrow$ | Spe.$\uparrow$ |
| 0.85 | 99.50$\pm$0.07 | 91.28$\pm$0.21 | 77.82$\pm$0.25 | 619.25$\pm$0.17 | 40.82$\pm$0.12 | 96.80$\pm$0.14 | 92.19$\pm$0.21 | 28.14$\pm$0.25 |
| 0.8 | 99.55$\pm$0.06 | 91.92$\pm$0.22 | 78.96$\pm$0.25 | 620.49$\pm$0.16 | 40.24$\pm$0.12 | 96.87$\pm$0.14 | 91.33$\pm$0.22 | 28.66$\pm$0.25 |
| 0.75 | 99.45$\pm$0.07 | 91.68$\pm$0.22 | 79.03$\pm$0.24 | 620.42$\pm$0.16 | 40.83$\pm$0.12 | 96.80$\pm$0.14 | 91.66$\pm$0.22 | 27.68$\pm$0.25 |

## Appendix G More Quantitative Results

We provide more detailed experimental results. Figure 6 presents the results of our method, NSE, compared to the baseline MEMIT on GPT2-XL and GPT-J, under different batch sizes in sequential editing, with a total of 2000 editing samples.

Additionally, Table 7 shows the performance of NSE with different neuron selection thresholds $p$. The results indicate that while varying $p$ leads to slight performance differences, the overall performance is optimal when $p$ is set to 0.8.

Figure: Figure 6: Performance on NSE and MEMIT under GPT2-XL and GPT-J with different batch sizes.The red line and the blue line represent MEMIT and NSE, respectively
Refer to caption: extracted/5903371/figures/app_fig_radar.png

## Appendix H Visualizing the ZSRE and Counterfact Datasets Through Examples

To facilitate a better understanding of model editing tasks for readers who may be new to this field, we present two examples from the Counterfact and ZSRE datasets in Figure 7 and 8. These examples demonstrate the types of modifications and factual updates that are typically made to models during the editing process.

Figure: Figure 7: A Sample of the Counterfact dataset.
Refer to caption: extracted/5903371/figures/app_fig_cf.png

Figure: Figure 8: A Samples of the ZsRE dataset.
Refer to caption: extracted/5903371/figures/app_fig_zsre.png

## Peer Reviews

confidence: 3
contribution: 2
initial_rating: 6
presentation: 3
questions: 1. Could you explain the reasons for the varying performance of NSE, ROME, and MEMIT in Fig. 5? 
2. Please discuss the trade-offs between Specificity, Fluency, and Efficacy for the GPT-XL, GPT-J, and Llama3 models using the Counterfact dataset.
3. Could you propose specific methods to optimize the computation time based on the result in Tab. 3?
soundness: 3
strengths: 1. Quantitative experiments are adequate and effective.
2. The methodology is clear and well-understood.
summary: This paper introduces Neuron-level Sequential Editing (NSE), a method for sequentially editing LLMs' internal knowledge. Traditional single-edit methods often struggle with model forgetting and failures in multi-edit scenarios. NSE tackles these issues by optimizing neuron-level hidden states while preventing failures. NSE reduces forgetting and outperforms existing editing methods.
weaknesses: More discussion and ablation analysis are required, please see the Questions.

---

confidence: 4
contribution: 3
initial_rating: 5
presentation: 2
questions: 1. Issues related to mathematical notations.
    1. In section 2.1, the vectors and matrices should be italic to be consistent with other sections.
    2. In line 145, $n$ is not defined.
    3. In line 150, $\Delta, W$ and $V_1$ are not bolded.
    4. In line 219, it is not clear that the activation values are from a single layer or from all the layers to be modified. If it’s the activation values of a certain layer, it would be better to add the subscript of the layer $l$.
    5. In line 220, Q should be italic. In addition, the absolute value of a vector is its norm. I think here the authors want to say the absolute value for each element in the vector. It would be clearer that the authors add some clarifications about what this absolute value sign means.
    6. In equation 10, it is not appropriate to use the same symbol on both sides of the equation, one refers to a specific set while the other refers to a variable. It is thus not clear which set is referred to in the inequality.
    7. In equations 3, 4, and 5, “edit” should be non-italic.
    8. In Figure 2, the “in” and “out” for MLP layer weights should be non-italic, which will be consistent with the notation in line 230.
2. Does the batch size influence the edit success? I’m curious whether using different batch sizes will result in different number of model edits.
3. How probable is it that the method cannot edit successfully one sample and needs to do it iteratively?
4. In Figure 5, the degradation seems very sudden. Do the authors have any explanation why this degradation is sharp rather than gradual.
5. In line 431, what is the baseline model for the average improvement of 45.6%?
6. Although it seems out of the scope of this paper (depending on how the authors will address the weaknesses point 1), it would be interesting to compare the NSE method with T-Patcher and SERAC.
soundness: 3
strengths: 1. This paper focuses on the sequential editing task and the editing methods that modify the original model’s parameters. They extend the previous baseline MEMIT to this scenario and prove the superiority of their proposed method.
2. The authors have done comprehensive experiments to show the superiority of their method on the sequential model editing task compared to methods that modify the model’s parameters.
3. The authors release their code for reproducibility.
summary: This paper proposes a method called Neuron-level Sequential Editing (NSE) based on MEMIT to extend that method to the sequential model editing task. The main component is to always compute latent updates based on the original model weights, in order to avoid model failures during the sequential editing. They also conduct neuron-level update and iterative editing to further improve the performance. The comprehensive experiments show the superiority of their proposed method compared to editing methods that directly modify the model’s parameters.
weaknesses: 1. The authors should make it clearer in their presentation that they focus on methods that modify the parameters. The models that freeze the model’s parameters, e.g., SERAC[1] and T-Patcher[2], show stable performance in sequential editing[3]. The authors don’t compare their method with these two strong baselines in the experiments. Therefore, it is more appropriate to clearly state this constraint in the Introduction, especially in the first paragraph. The first paragraph constrains the model editing methods into “Modify model’s parameters” methods (ROME and MEMIT), but neglect other methods that can also address this problem. In the following paragraphs, authors use *direct model editing* methods (which is not clear what this stands for) and *memory-based* methods to refer to *modify models’ parameters* and *preserve model’s parameters* methods (the taxonomy used in the Related Work section) respectively. It would be clearer and more coherent if using the taxonomy in the Related Work section across the entire paper.
2. The presentation of the mathematical notations should be improved. Details follow in the Questions section.
3. The main architecture figure 2 is misleading. $z_i$ is not a weight of the model, but a hidden state. The decoder blocks are not frozen since the W_out is updated.
4. It is not clear what “number of edits” means in the figures. To me, one edit means one edit to the model, i.e., one update to the model. However, it seems that “number of edits” refers to “number of edit samples” in this paper. In the previous papers, number of model edits and number of edit samples are proportional, since they only use once the edit samples. It’s not the case in the NSE as the authors adopt an iterative editing technique where one edit sample can be used to edit several times the model.

[1] Mitchell, Eric, et al. "Memory-based model editing at scale." International Conference on Machine Learning. PMLR, 2022.

[2] Huang, Zeyu, et al. "Transformer-Patcher: One Mistake Worth One Neuron." The Eleventh International Conference on Learning Representations. 2023

[3] Yao, Yunzhi, et al. "Editing Large Language Models: Problems, Methods, and Opportunities." Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. 2023.

---

confidence: 3
contribution: 2
initial_rating: 3
presentation: 2
questions: - I do not understand the equation 9. Can you explain how you preserve the weights and the operation in previous works in detail?
- In the experiment, the authors set the number of edited samples as 2000. However, in MEMIT, the number is 10000. I am wondering if the proposed method still performs well when processing more samples.
- In the neuron-level weights updating, does the selected neuron always change? Is there any specific neuron being edited many times?
- In the section 4.3, why do you only choose MEMIT for comparison? I find GRACE is also a strong baseline and it is more persuasive if you add GRACE in the section 4.3.
- In the ablation study, I suggest experiments with two modules removed to find out which module plays the most important role in the proposed method.
soundness: 2
strengths: - The sequential editing problem is essential for enabling the model's lifelong learning capability.
- The authors conduct comprehensive experiments to demonstrate the superiority of their method.
summary: This work proposes a framework including weight rewinding, neuron-level updating, and iterative editing for improving the efficacy of sequence knowledge editing while preserving the model utility. The experiments on editing performance, batch size of edits, and preservation of utility, as well as ablation study demonstrate the effectiveness of the proposed method.
weaknesses: - There are some claims that are not supported by any references or experiments. For example, in line 199-201, the authors claim that "This indicates that the cumulative parameter updates from each editing round can lead to a shift in value computation. Conversely, using the original model parameters fθ0 to compute zi effectively prevents this issue" without any supports. And in line 255-257, where the claim is "However, due to errors in the fitting process, some knowledge proves difficult to edit, resulting in vl0 i not sufficiently approximating zi, and consequently leading to editing failures." The authors do not define "errors" and they do not provide references for their claim.
- The neuron-level weights updating belongs to localization-informed methods, which is widely used in knowledge-editing-related tasks [1,2]. Here I give examples of privacy information editing, which I believe are similar to knowledge editing.
- The iterative multi-layer editing module will cause inefficiency in the knowledge editing process. 
- The experiment setting is inconsistent. In section 4.2, the number of edited samples is 2000, while in section 4.4, the number is 3000. The authors do not explain the reason why they use different numbers of edited samples in the paper.

[1] Fan C, Liu J, Zhang Y, et al. SalUn: Empowering Machine Unlearning via Gradient-based Weight Saliency in Both Image Classification and Generation[C]//The Twelfth International Conference on Learning Representations.

[2] Wu X, Li J, Xu M, et al. DEPN: Detecting and Editing Privacy Neurons in Pretrained Language Models[C]//Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. 2023: 2875-2886.