SSS: Editing Factual Knowledge in Language Models towards Semantic Sparse Space (2024)

Abstract: Language Models (LMs) acquire factual knowledge during pre-training and store it in the parameters, which can be valuable for downstream tasks. As world evolves, some facts may be incorrectly induced or become obsolete over time. Various model editing methods have been proposed to modify specific examples in LMs. However, existing training-based methods still suffer from sub-optimal locality, where irrelevant neighborhood examples can be adversely influenced. Model’s gradients are still struggling to identify the appropriate direction when updating the parameters. To address this issue, we find that directing the hidden state of the edit example towards spaces where semantics are sparse tends to help preserve the semantics of irrelevant neighborhood examples. Based on this hypothesis, we propose a novel metric, named SSS, to evaluate the degree of sparsity around a sentence embedding in the semantic space without any human or machine annotation. Subsequently, we incorporate SSS into the original loss function of the existing training-based methods to enhance locality. Experiments conducted on two datasets across various models demonstrate that SSS is effective in improving both locality and reasoning capability. Code will be available at: https://github.com/MaybeLizzy/EditSSS.

heading: 1 Introduction
text: Language Models (Touvron et al., 2023; OpenAI, 2023) (LMs) are surprisingly good at recalling factual knowledge presented in the pre-training corpus, which enables promising results in various downstream tasks. However, as the world’s state evolves, LMs may become incorrect or outdated over time. Developing reliable and computationally efficient methods to edit the model knowledge without the need for expensive re-training becomes
*Equal Contribution. †Corresponding Author.
non-trivial. In response to this issue, the concept of model editing has been proposed (Sinitsin et al., 2020; Cao et al., 2021), which is to intervene a pre-trained model’s behavior on a specific edit example without damaging its performance on other irrelevant examples.
Numerous works on model editing for language models have emerged. One branch of research involves additional training, such as training a hypernetwork to predict weight updates (Cao et al., 2021; Mitchell et al., 2022a) or introducing additional trainable parameters (Dong et al., 2022; Hartvigsen et al., 2022; Huang et al., 2023). Another line of research attributes knowledge to specific neurons or modules within the network (Dai et al., 2022) and updates the model parameters associated with the edit example directly (Meng et al., 2022, 2023; Li et al., 2023a), without any additional training. With the development of in-context learning, some methods edit the model by either prompting it with the edit example (Zheng et al., 2023) or retrieving edit demonstrations from an explicit memory (Mitchell et al., 2022b; Madaan et al., 2022; Zhong et al., 2023).
To evaluate the post-edit model performance,
5559
previous works formulate the criteria that a successful model editor should not only adjust the model behavior for in-scope examples that are closely associated with the edit example, but also maintain locality (or specificity) for out-of-scope examples. It means that irrelevant neighborhood examples should be left unaltered. Recently, there emerges diverse studies evaluating existing editing methods across various dimensions, such as learning new entities (Onoe et al., 2023), reasoning implications (Cohen et al., 2023) and editing commonsense mistakes (Gupta et al., 2023).
Though existing methods have demonstrated considerable effectiveness in model editing tasks, they still suffer from inferior locality, where irrelevant neighborhood examples can be adversely influenced (Yao et al., 2023) (as illustrated in Fig. 1). Focusing on the training-based methods, we speculate the reason is that existing methods primarily adopt two objectives when updating the model parameters, apart from their method-specific objective. One objective is to align the model predictions with the edit label, and the other is to constrain the Kullback-Leibler (KL) divergences between the pre- and post-edit models on irrelevant neighborhood examples. Due to the limited training set, examples not included within the dataset can still be potentially affected. Consequently, relying solely on the aforementioned objective is inadequate. Models’ gradient are still struggling to identify the appropriate gradient direction when updating the parameters (Li et al., 2023b).
To address this issue, we take a further step and find that mapping the embedding of the edited example into a sparse space helps preserve the semantics of other irrelevant neighborhood examples. Based on this hypothesis, we propose a novel metric, named SSS, to evaluate the degree of sparsity in the semantic space around a sentence embedding for a specific model. We then incorporate SSS into the original loss function of existing training-based methods, guiding the embedding of the edited example towards a Semantic Sparse Space to enhance locality. This method is computationally efficient and entirely unsupervised, requiring no human or machine annotation.
To assess the effectiveness of SSS, we select three representative training-based methods, including FT-L (Yao et al., 2023), MEND (Mitchell et al., 2022a) and SERAC (Mitchell et al., 2022b). We test these methods on two popular model editing datasets, ZsRE (Levy et al., 2017) and COUNTER-
FACT (Meng et al., 2022) across various language models with multiple scales (1.5B∼7B). Experimental results highlight the effectiveness of SSS in enhancing both locality and reasoning capability.
Our contributions are summarized as follows:
• We introduce a novel evaluation metric, SSS, to measure the semantic sparsity of a sentence embedding for a specific language model without any human or machine annotation.
• We incorporate SSS into existing trainingbased model editing methods to enhance the locality of irrelevant neighborhood examples.
• Experiments conducted on two datasets across various language models demonstrate the validity and scalability of SSS.

heading: 2 Related Work
text: Training-based Editing Method A branch of research requires training when editing models. The simplest method is directly fine-tuning the target model with specific edit examples. In addition, Cao et al. (2021) propose Knowledge Editor (KE) approach, which trains a hyper-network to predict the weight update for each edit example. To overcome the limitation of KE that falls short in editing language models with larger scale, Mitchell et al. (2022a) introduce Model Editor Networks with Gradient Decomposition (MEND), which learns to transform the gradient obtained by standard finetuning using a low-rank decomposition of gradients. Furthermore, Mitchell et al. (2022b) present SERAC, which stores edit examples in explicit memory. It utilizes a scope classifier to determine if an input lies within the scope of any cache items and trains a memory-based retrieval-augmented counterfactual model to edit facts that fall within the scope of stored edit examples. Other works introduce additional trainable parameters while keeping the original model parameters static (Dong et al., 2022; Hartvigsen et al., 2022). For instance, Huang et al. (2023) propose TransformerPatcher, a sequential model editing method that simply incorporates and trains a few neurons in the last feed-forward Network layer.
Training-free Editing Method Another line of research attributes knowledge to specific neurons or modules within the network. Notably, Dai et al. (2022) view feed-forward network modules in Transformer as key-value memories and propose
the Knowledge Neuron (KN) method to edit specific factual knowledge by identifying the knowledge neurons positively correlated to the knowledge expression. Similarly, Meng et al. (2022) modify feed-forward weights to update specific factual associations using Rank-One Model Editing (ROME). To extend model editing to multiple cases simultaneously (Li et al., 2023a), Meng et al. (2023) build upon the ROME framework and propose MEMIT. With the development of in-context learning, certain methods edit the model by either prompting it with the edit example (Zheng et al., 2023), or retrieving demonstrations from the edit memory (Madaan et al., 2022; Zhong et al., 2023).

heading: 3 Task Definition
text: The objective of model editing is to adjust an initial target model’s behavior on a specific edit example, without impacting the model’s performance on other irrelevant examples. More specifically, the target model fθ is represented by a function f : X 7→ Y, with θ denoting the model parameter. Given an edit example (xe, ye) with the edit label ye ̸= fθ(xe), the post-edit model fθe is required to generate the expected output fθe(xe) such that fθe(xe) = ye.

heading: 4 Preliminary
text: In this section, we introduce the objectives of the existing training-based model editing methods. An ideal post-edit model fθe should satisfy three properties: reliability, generality and locality (Yao et al., 2023).
Reliability A model editor is reliable if fθe predicts the edit label ye for the edit input xe:
Ex′e,y′e∼{(xe,ye)}1{argmaxyfθe(y|x ′ e) = y ′ e}. (1)
Generality As the model editing process can impact a wide range of examples that are closely associated with the edit example, known as the editing scope, a successful edit should also adjust the model behavior for in-scope examples I(xe, ye), such as examples with similar expressions:
Ex′e,y′e∼I(xe,ye)1{argmaxyfθe(y|x ′ e) = y ′ e}. (2)
Locality A good edit is supposed to edit the knowledge without influencing other irrelevant outof-scope examples O(xe, ye). It always refers to locality (or specificity):
Ex′e,y′e∼O(xe,ye)1{argmaxyfθe(y|x ′ e) = fθ(y|x′e)}.
(3)
To achieve reliability and generality, existing works (Mitchell et al., 2022b; Huang et al., 2023) use the loss function Le for the edit example and its corresponding in-scope examples:
Le(θe) = −log pθe(ye|xe). (4)
To achieve locality, most works (Cao et al., 2021; Mitchell et al., 2022a) choose to constrain updates in terms of the Kullback-Leibler (KL) divergences between the pre- and post- edit model conditioned on the locality examples xloc:
Lloc(θ, θe) = KL(pθ(·|xloc)∥pθe(·|xloc)). (5)
The total training loss Lori is computed with the sum of Le and Lloc using a hyper-parameter ce:
Lori = ce · Le(θe) + Lloc(θ, θe). (6)
Apart from the aforementioned losses, different methods may employ their own method-specific objectives to update the model parameters. Despite these variations, in this paper, we consistently denote the original training loss of previous methods as Lori.

heading: 5 Method
text: Existing methods are still struggling to maintain high locality (Yao et al., 2023). We assume the limitation is that they solely considering the KL divergences between the pre- and post- edit model on locality examples from the training set. This proves inadequate since irrelevant examples not collected in the training set can still be potentially influenced. To address this issue, we are committed to exploring strategies that guide model parameter updates in a direction that protects the semantic space of irrelevant examples from being disturbed.
It has been demonstrated that language models are pre-trained to implicitly learn sentence representations (Li et al., 2020). Each sentence can be encoded as a high-dimensional vector, representing a point in the embedding space. The knowledge acquired by the model is mapped into this space, characterized by an uneven distribution where certain regions may be denser than others (Aharoni and Goldberg, 2020). Since the knowledge acquired by LLMs is encoded within a network of interconnected neurons, editing one piece of knowledge is likely to impact others (Dai et al., 2022). When editing knowledge, models often face challenges in determining the appropriate gradient direction for
updating the parameters (Li et al., 2023b). Building upon these findings, we propose mapping the new knowledge into a sparse space to provide clear guidance for gradient updates while ensuring that the distribution of other knowledge in the embedding space remains unaffected.
To accomplish this, we utilize the vulnerability of sentence embeddings to quantify the sparsity of the embedding space. A robust sentence embedding, resilient to large perturbations, suggests a relatively sparse semantic space around it. Conversely, even a minor disturbance in the sentence can change its semantics, indicating a semantically dense space. Under this hypothesis, we introduce an evaluation metric to quantify the vulnerability (or sparsity) of sentence embeddings, named SSS. Subsequently, we introduce a novel training objective for updating model gradients to enhance locality. The detailed explanation is outlined as follows.
We are inspired by Zhao et al. (2019), who adopt the Fisher Information Matrix (FIM) of the input sample as a metric tensor to measure the robustness of deep learning models in adversarial attack task. Borrowing from this idea, we define a FIM-based matrix H to characterize the vulnerability of a sentence embedding to the perturbation in its feature space for a specific model LM. Specifically, given
an edit example (xe, ye), the matrix H ∈ Rn×n is defined as follows, where n stands for the hidden dimension of LM and J (xe, ye) is the loss function:
H = ∇xeJ (xe, ye)⊤∇xeJ (xe, ye). (7)
∇xeJ (xe, ye) is the partial differential of J (xe, ye) respecting to xe. We adopt the per-token negative log-likelihood loss:
J (xe, ye) = −Eye|xe [log PLM(ye|xe)]. (8)
Similar to the conclusion of Zhao et al. (2019), which quantifies the vulnerability of deep learning models, we deduce that the maximum eigenvalue of H, denoted as λmax, reflects the vulnerability of sentence embeddings to the perturbation. The proof process is detailed in A.1. The expression for λmax can be written as:
λmax= 1
m
m∑
i=1
(∇xiJ (xe, ye))2, (9)
where m is the sentence length and xi is the ith token of xe. A smaller λmax indicates a more robust sentence embedding with higher resilience to the perturbation, indicating a relatively sparse semantic space surrounding the sentence embedding. As illustrated in Fig. 2, we name λmax as
SSS and incorporate it into the original training loss of previous methods using a tuning coefficient C to enhance overall locality, denoted as Lall:
Lall = (1− C) · Lori + C · SSS. (10)
In general, SSS can be easily implemented by three steps. Firstly, input xe to the model and calculate the loss J(xe, ye) using the model prediction and the edit label. Secondly, compute the maximum eigenvalue λmax of xe’s updated embedding using Eq 9. Finally, the total training loss is derived using J(xe, ye) and SSS according to Eq 10.
The advantage of the introduced SSS lies in its computational efficiency, as it solely relies on J (xe, ye) and the last hidden states of the edit example xe. Additionally, integrating SSS into the original loss function for various training-based editing methods is straightforward and convenient. More importantly, the entire process is unsupervised, requiring no human or machine annotation.

heading: 6 Preliminary Experiments
text: In this section, we conduct preliminary experiments to assess the effectiveness of SSS across three training-based methods. The experimental settings and results are outlined as follows. All results are averaged over three runs.

heading: 6.1 Experiment Settings
text: We initially employ two popular model editing datasets, ZsRE (Levy et al., 2017) and COUNTERFACT (Meng et al., 2022). Following the data split from Yao et al. (2023), we use the training set for model training and evaluate the performance on the test set. For test efficiency, the test set is limited to 10,000 examples on ZsRE. We evaluate the results of fine-tuning, a fundamental approach for model editing. Following (Yao et al., 2023), we fine-tune layers identified by ROME (Meng et al., 2022) to avoid the computational cost of retraining all layers, which is denoted as FT-L. Furthermore, we choose two influential training-based methods for evaluation, namely MEND (Mitchell et al., 2022a) and SERAC (Mitchell et al., 2022b). Instead of using smaller language models for knowledge editing, such as BERT (Devlin et al., 2019), we follow Yao et al. (2023) and focus on generation-based models. Specifically, we choose GPT2-XL (1.5B) as the target model. In these experiments, we set the hyper-parameter C to 0.9.

heading: 6.2 Experiment Results
text: FT-L For basic FT-L, we follow Yao et al. (2023) and utilize Eq. 4 to train the model. For FT-L w/ SSS, the training objective is computed by incorporating SSS with Eq. 4. To ensure fairness, all hyper-parameters follow default settings in Yao et al. (2023), utilizing Adam with early stopping, and only modifying the weights of mlpproj at the selected layer.
As shown in Table 1, when using FT-L, the accuracy of generality is significantly lower than that of reliability, exhibiting an absolute difference of 83.49%. This suggests that fine-tuning can only modify the model’s behavior on the specific edit example, failing to generalize to other rephrased sentences. Despite of this, FT-L w/ SSS outperforms FT-L by up to 1.19% on reliability, 1.52% on generality and 1.15% on locality, illustrating the effectiveness of SSS.
MEND Following (Mitchell et al., 2022a), the training objective of MEND is consistent with Eq. 6, where ce is set to be 0.1. For MEND w/ SSS, we integrate SSS with Eq. 6. Both MEND and MEND w/ SSS are trained on ZsRE training set using the Adam optimizer.
As shown in Table 1, when tested on ZsRE, the performance of MEND w/ SSS surpasses that of MEND, exhibiting an absolute improvement of 2.73% on generality. When tested on COUNTERFACT in an out-of-distribution (OOD) scenario, MEND demonstrates remarkable robustness, with reliability outperforming ZsRE by 32.74%, though at a cost of relatively lower locality and generality. We speculate that the reason lies in the nature of the model, which may excel in domain adaptation for editing data formulated as language modeling. In this scenario, MEND w/ SSS demonstrates a more robust capability than MEND, with an absolute improvement of 4.99% on reliability, 1.85% on locality and 4.0% on generality. This verifies the effectiveness of SSS.
SERAC SERAC consists of an edit memory, classifier, and counterfactual model. The scope classifier and counterfactual model are trained independently. We solely concentrate on the training objective of the counterfactual model, which aligns with MEND (w/ SSS). The training parameter settings of SERAC (w/ SSS) remain consistent with Yao et al. (2023).
As shown in Table 1, when tested on ZsRE,
SERAC w/ SSS outperforms SERAC across all three metrics. When tested on COUNTERFACT, SERAC demonstrates significantly inferior OOD robustness. The bottleneck may lie in the accuracy of the pre-trained classifier and the capabilities of the inference model. Despite of this, SERAC w/ SSS consistently outperforms SERAC by 2.38% on reliability, 3.52% on locality and 2.47% on generality. The significant improvement confirms the effectiveness of SSS on editing model facts and preserving neighborhood knowledge.
Sequential Editing The default evaluation procedure involves updating the knowledge of a single model, evaluating the updated model, and then rolling back the update before repeating the process for each test example. In practical scenarios, a model editor must continuously and promptly fix a series of mistakes. Therefore, we conduct experiments on sequential editing using the COUNTERFACT dataset, where models retain previous changes while making new edits. Following Yao et al. (2023), we choose the sequence number n from [1, 10, 100, 1000]. For comparative pur-
poses, we also present the results of two trainingfree methods: ROME (Meng et al., 2022) and MEMIT (Meng et al., 2023).
As shown in Table 2, FT-L maintains high Reliability even when n = 1000, but its Locality and Generality decrease significantly as n increases. MEMIT demonstrates stable performance in sequential editing, while MEND’s performance deteriorates with increasing n. SSS consistently outperforms the baselines as n increases. Specifically, SERAC w/ SSS achieves up to a 7.8% improvement in Locality compared to SERAC when n = 1000. These results confirm that SSS is able to preserve the model’s original irrelevant knowledge during sequential editing, validating the effectiveness of SSS.

heading: 7 Comprehensive Study
text: The impact of model editing on the language model is intricate, demanding a thorough and comprehensive evaluation to fully comprehend its effects. Consequently, in line with Yao et al. (2023), we conduct additional tests to assess SSS from both
locality and portability perspectives. The statistics of the datasets are shown in Table 4. In this experiment, FT-L (w/ SSS) is directly fine-tuned on the specific dataset using GPT2-XL. MEND (w/ SSS) and SERAC (w/ SSS) utilize the pre-trained checkpoints from Section 6. We keep C = 0.9.

heading: 7.1 Evaluate Locality
text: Consider an edit example xe with its original ground truth yr and edit label ye. To thoroughly examine the potential side effects of model editing on neighborhood examples, SSS is tested at four different levels.
Pre-Neighbor: We name a neighborhood example as Pre-Neighbor if its label is yr.
Post-Neighbor: We name a neighborhood example as Post-Neighbor if its label is ye.
Distract-Neighbor: Following Yao et al. (2023), we concatenate the edit example before PreNeighbor example to test whether the model prediction will be influenced by the edit example. The post-edit model is expected to maintain consistent behavior on Distract-Neighbor examples, predicting yr instead of ye.
Unrelated Attributes: The unrelated attributes of the subject in the edit example should remain unchanged after editing. For instance, if we edit the knowledge from “Alfred Kubel has citizenship from Germany" to “Alfred Kubel has citizenship from Finland", the answer of the Unrelated Attribute example “What is Alfred Kubel’s sex or gender?" should remain the same.
We also present the results of ROME and
MEMIT on locality. However, this locate-and-edit method heavily relies on the data format and is limited to modifying examples in triplets, where (e, r, o) represents a subject entity e, a relation r, and an object o. Therefore, we do not directly compare the accuracy with ROME and MEMIT.
Results As shown in Table 3, FT-L exhibits higher reliability compared to MEND and SERAC. It is reasonable since the latter two are tested in an OOD scenario where generalization may be somewhat reduced. In detail, MEND w/ SSS exhibits better reliability than MEND by 3.93%, highlighting the effectiveness of SSS in editing model knowledge. However, MEND w/ SSS and SERAC w/ SSS exhibit sub-optimal performance on Distract-Neighbor, showing reductions of 0.82% and 0.23%, respectively. We suspect this is attributed to the fact that training with SSS enhances the edit example’s resilience to semantic perturbation, thereby making the model more responsive to the edit example when incorporated as a prompt concatenated in front of the test query. Therefore, the edit example may distract the model attention, resulting in a relatively lower performance on Distract-Neighbor. In addition, w/ SSS shows a relatively modest increase in Pre-Neighbor and PostNeighbor, ranging from 0.47% to 0.99%. However, the improvements introduced by SSS on Unrelated Attributes are substantial, with an absolute enhancement of 6.4% compared to SERAC. It indicates that SSS is more effective in enhancing the robustness of the subject in edit example itself, protecting it from the semantic perturbation of other irrelevant examples. This validates the effectiveness of SSS.
Impact of C We apply parameter tuning method to explore the impact of C on model performance. Experiments are conducted on the locality dataset using FT-L w/ SSS with C ranging from 0.0 to 1.0. Other parameters remain unchanged. As shown in Fig 3, the best accuracy is achieved when C = 0.9, highlighting the importance of consid-
ering SSS when editing model knowledge. Additionally, the model performance can fluctuate when C = 0.0 ∼ 0.5 where Le takes a large proportion. However, for C ≥ 0.6, the accuracy across all metrics consistently exhibits a stable improvement. This suggests that increasing the proportion of SSS can make the training process less dependent on Le. A small Le may imply that the postedit model performs well on the edit example, but it does not necessarily ensure that the neighborhood examples are protected from disturbance. Therefore, increasing the proportion of SSS can more effectively enhance locality. Moreover, for C = 1.0, the reliability is 0.25% with locality reaching 100%. It demonstrates that solely relying on SSS is inadequate for editing the model knowledge.
Model Scaling To thoroughly evaluate the impact of model scaling on SSS, we conduct experiments with larger LMs on locality dataset, including GPT-Neo-2.7B, T5-3B and LLaMA-7B. As shown in Fig. 4, the accuracy does not exhibit a pattern with the scaling of LMs. We suspect that accuracy can be influenced by various aspects of models, including the pre-trained dataset, model architecture, and training methods. Larger models do not necessarily indicate better accuracy (Onoe et al.,
2023; Hoelscher-Obermaier et al., 2023). Despite the variations, FT-L w/ SSS consistently outperforms FT-L, validating the effectiveness of SSS.

heading: 7.2 Evaluate Portability
text: To thoroughly evaluate the effectiveness of SSS, we test SSS on portability, assessing its ability to transfer the edited knowledge to related facts (Yao et al., 2023). There are three aspects: Subject Replace, Reversed Relation and One-hop. We provide detailed explanation as follows.
Subject Replace We substitute the subject in the edit example with an alias or synonym, evaluating the post-edit model’s ability to generalize the edited attribute to other descriptions of the same subject.
Reversed Relation When the target of a subject and relation is edited, the attribute of the target entity also changes. We test the model’s ability with the reverse question to check if the target entity is also updated. For instance, if we edit the answer of the edit example “Who is Nebaioth’s father?" from “Babylon 5" to “Babur", then the post-edit model is expected to predict “Nebaioth" for the Reversed Relation example “Who is the son of Babur?".
One-Hop The post-edit model should employ the edited knowledge for downstream tasks. Therefore, we evaluate the post-edit model’s ability to perform one-hop reasoning. For instance, if we edit the answer of the edit example “What company made Volvo B12M?" from “Volvo Buses" to “Volkswagen Group", then the post-edit model is expected to predict “Wolfsburg, Germany" instead of “Gothenburg, Sweden" for the One-Hop example “In which city is the headquarters of the company that made the Volvo B12M?".
Results As shown in Table 5, MEND w/ SSS presents a superior performance over MEND by 2.26% on Subject Replace. Though FT-L w/ SSS exhibits lower reliability and generality than FT-L, the accuracy difference is within 0.28%. Though SERAC w/ SSS exhibit a modest improvement of 0.85% on Reversed Relation, it demonstrates an improvement of 2.55% on One-Hop compared to SERAC. The overall performance in portability suggests that SSS can help the post-edit model to handle the implications of the edit examples and transfer the edited knowledge to associated facts for downstream applications, particularly in
terms of one-hop capability. Nevertheless, the absolute portability accuracy remains relatively modest (less than 50%), calling for further exploration and innovation in future research.
Case Study We provide a case study on Reversed Relation dataset using MEND. As shown in Fig 5, the predictions made by MEND on the edit example and the reversed question significantly differ from the label, leading to lower accuracy. However, when using MEND w/ SSS, the predictions notably improve, resulting in higher accuracy, which validates the effectiveness of SSS.

heading: 8 Conclusion
text: In this paper, we introduce SSS, a novel evaluation metric to measure the semantic sparsity of a sentence for a specific language model without any human or machine annotation. Subsequently, we incorporate SSS into existing training-based
model editing methods to enhance the locality of irrelevant neighborhood examples. Experiments conducted on two datasets across various models demonstrate the validity and scalability of SSS.
In summary, SSS presents a promising approach for mitigating the side effects of model editing, offering a notable perspective compared to existing training-based model editing methods.
Ethical Considerations
We believe that this study contributes intellectual value to the dependable application of model knowledge editing in the field of NLP, with potential broader implications for tasks in other areas. It is noteworthy that there are no direct societal consequences, and all experiments are conducted on open datasets.
Limitations
Given the constraints of computing power, incorporating language models with larger scales poses a challenge for us. While results using automatic metrics offer a fair assessment of task performance, we plan to conduct a human evaluation in the near future.

heading: Acknowledgements
text: This work was supported in part by the National Natural Science Foundation of China under, Grant 62201072, Grant U23B2001, Grant 62171057, Grant 62101064, Grant 62001054, and Grant 62071067; in part by the Ministry of Education and China Mobile Joint Fund under Grant MCM20200202 and Grant MCM20180101; in part by the BUPT-China Mobile Research Institute Joint Innovation Center.

heading: A Appendix
text: A.1 Proof To evaluate the vulnerability of edit examples, we are inspired by Zhao et al. (Zhao et al., 2019), who adopt the Fisher Information Matrix (FIM) of the input sample as a metric tensor to measure the robustness of deep learning models in adversarial attack task. Given a labeled data example (x, y), the FIM matrix named Gx is defined by Eq 11:
Gx = ∑
i
pi [ (∇xJ (yi, x)) (∇xJ (yi, x))T ] ,
(11) where J (yi, x) = −logp(yi|x) is the loss function and pi represents the probability of p(yi|x) when y takes the i-th class. ∇xJ (yi, x) is the partial differential of J (yi, x) respecting to x.
Borrowing from this idea, we define a FIMbased matrix H to characterize the vulnerability of an edit example to the perturbation in its feature space for LM. The matrix H ∈ Rn×n of edit example e = (xe, ye) is defined as follows, where n stands for the hidden dimension of LM and J (xe, ye) is the loss function:
H = ∇xeJ (xe, ye)⊤∇xeJ (xe, ye). (12) The matrix H proposed in Eq 12 can be considered as a special case of Gx. In Gx, each class’s probability of p(y|x) is weighted, and Gx calculates the expectation accordingly. In contrast, H sets the probability of p(y|x) to 1 when y takes the correct class and 0 for other classes. Such difference between H and Gx originates from the different objectives. The aim of Gx is to find a subtle perturbation η that shifts the probability p(y|x+ η) from the correct class to an incorrect one. So each class is given a probability. However, our objective is to ensure the model consistently predicts the correct class. So other classes can be ignored with weight being 0.
Similar to the conclusion of Zhao et al. (Zhao et al., 2019), we deduce that the maximum eigenvalue of H, denoted as λmax, reflects the robustness of the edit example to LM. A smaller λmax indicates a more robust edit example with higher resilience to the perturbation. We present the key derivation steps as follows.
We assume the existence of an ideally robust edit example, denoted as x̂e, that can yield the post-edit model prediction aligned with ye while maintaining the distribution of other neighborhood knowledge in the embedding space unaffected. We evaluate the robustness of xe by observing the distance between xe and x̂e, denoted by ∥xe − x̂e∥. The embedding of xe is denoted as X ∈ Rm×n where m is the length of xe and n is the hidden dimension of LM. X̂ is the corresponding embedding of x̂e. Subsequently, we transform the task into observing ∥X − X̂∥. Given the parameters of LM expressed as W ∈ Rn×n, the loss function can be transformed into:
J (X, X̂) = ∥∥∥WX⊤ −WX̂⊤ ∥∥∥ 2 . (13)
The partial differential of J (X, X̂) respecting to X is:
∇XJ (X, X̂) ≜ ( ∇X1J (X, X̂), · · · ,∇XmJ (X, X̂) )⊤ ,
(14)
where Xi is the row vector standing for the ith token embedding, denoted by Xt = (x1, · · · , xn) ∈ R1×n.
The maximum eigenvalue of H, denoted as λmax, can be calculated as the weighted sum of each token’s maximum eigenvalue:
λmax= 1
m
m∑
i=1
aiλi (15)
where ai is the weight coefficient of the ith token, and ∑m i=1 ai = m. We consider the simplest case where every token is equally important and let a1 = a2 = · · · = am = 1. λt is the tth token’s maximum eigenvalue, which can be calculated by:
λt=
n∑
i=1
(∇xiJ (Xt, X̂t))2. (16)
Substituting Eq 16 to Eq 15, we get the expression of λmax as:
λmax= 1
m
m∑
z=1
n∑
k=1
(∇xzkJ (Xz, X̂z)) 2, (17)
where xji and x̂ j i represent the jth element of Xi and X̂i. To calculate λmax, we expand the square sum of Eq. 17 and then simplify it into a comprehensive expression. For simplicity, we introduce a notion
∆ji = x j i − x̂ j i to represent the distance between the ith dimension of the jth token embeddings. And then λmax can be derived as:
λmax = 4
m
m∑
z=1
n∑
k=1
  n∑
i=1
w2ik ·∆zi + ∑
j∈Φ j ̸=k
( n∑
i=1
wikwij ) ∆zj   2
(18)
where Φ = {1, 2, · · · , n}. And wi,j is the element in row i, column j of matrix W.
We conclude from Eq 18 that λmax is a function on ∆ji and the coefficients are only related to the model parameters. Hence, a small λmax represents a relatively small ∆ji , leading to a small ∥X − X̂∥ and a more robust edit example.
A.2 Experiments We conduct additional experiments to explore the impact of C on portability using GPT2-XL. Experiments are conducted on the Subject Replace dataset using FT-L w/ SSS with C ranging from 0.0 to 1.0. Other parameters remain unchanged. As illustrated in Fig. 6, the highest accuracy is observed at C = 0.8 ∼ 0.9, which aligns with the observation in Section 7.1.

heading: Future Work
text: While results using automatic metrics offer a fair assessment of task performance, we plan to conduct a human evaluation in the near future