SSS : Editing Factual Knowledge in Language Models towards Semantic Sparse Space (2024)

Abstract: Language Models (LMs) acquire factual knowledge during pre-training and store it in the parameters, which can be valuable for downstream tasks. As world evolves, some facts may be incorrectly induced or become obsolete over time. Various model editing methods have been proposed to modify specific examples in LMs. However, existing training-based methods still suffer from sub-optimal locality, where irrelevant neighborhood examples can be adversely influenced. Model’s gradients are still struggling to identify the appropriate direction when updating the parameters. To address this issue, we find that directing the hidden state of the edit example towards spaces where semantics are sparse tends to help preserve the semantics of irrelevant neighborhood examples. Based on this hypothesis, we propose a novel metric, named SSS, to evaluate the degree of sparsity around a sentence embedding in the semantic space without any human or machine annotation. Subsequently, we incorporate SSS into the original loss function of the existing training-based methods to enhance locality. Experiments conducted on two datasets across various models demonstrate that SSS is effective in improving both locality and reasoning capability.

## introduction language models @xcite ) (lms) are surprisingly good at recalling factual knowledge presented in the pre-training corpus, which enables promising results in various downstream tasks. however, as the world's state evolves, lms may become incorrect or outdated over time. developing reliable and computationally efficient methods to edit the model knowledge without the need for expensive re-training becomes non-trivial. in response to this issue, the concept of model editing has been proposed @xcite , which is to intervene a pre-trained model's behavior on a specific edit example without damaging its performance on other irrelevant examples. numerous works on model editing for language models have emerged. one branch of research involves additional training, such as training a hypernetwork to predict weight updates @xcite or introducing additional trainable parameters @xcite @xcite . another line of research attributes knowledge to specific neurons or modules within the network @xcite and updates the model parameters associated with the edit example directly @xcite @xcite , without any additional training. with the development of in-context learning, some methods edit the model by either prompting it with the edit example @xcite or retrieving edit demonstrations from an explicit memory @xcite @xcite . to evaluate the post-edit model performance, previous works formulate the criteria that a successful model editor should not only adjust the model behavior for in-scope examples that are closely associated with the edit example, but also maintain locality (or specificity) for out-of-scope examples. it means that irrelevant neighborhood examples should be left unaltered. recently, there emerges diverse studies evaluating existing editing methods across various dimensions, such as learning new entities @xcite , reasoning implications @xcite and editing commonsense mistakes @xcite . though existing methods have demonstrated considerable effectiveness in model editing tasks, they still suffer from inferior locality, where irrelevant neighborhood examples can be adversely influenced @xcite (as illustrated in fig. 1 ). focusing on the training-based methods, we speculate the reason is that existing methods primarily adopt two objectives when updating the model parameters, apart from their method-specific objective. one objective is to align the model predictions with the edit label, and the other is to constrain the kullback-leibler (kl) divergences between the pre-and post-edit models on irrelevant neighborhood examples. due to the limited training set, examples not included within the dataset can still be potentially affected. consequently, relying solely on the aforementioned objective is inadequate. models' gradient are still struggling to identify the appropriate gradient direction when updating the parameters @xcite . to address this issue, we take a further step and find that mapping the embedding of the edited example into a sparse space helps preserve the semantics of other irrelevant neighborhood examples. based on this hypothesis, we propose a novel metric, named sss, to evaluate the degree of sparsity in the semantic space around a sentence embedding for a specific model. we then incorporate sss into the original loss function of existing training-based methods, guiding the embedding of the edited example towards a semantic sparse space to enhance locality. this method is computationally efficient and entirely unsupervised, requiring no human or machine annotation. to assess the effectiveness of sss, we select three representative training-based methods, including ft-l @xcite , mend @xcite and serac @xcite . we test these methods on two popular model editing datasets, zsre @xcite and counter-fact @xcite across various language models with multiple scales (1.5b∼7b). experimental results highlight the effectiveness of sss in enhancing both locality and reasoning capability. our contributions are summarized as follows: training-based editing method a branch of research requires training when editing models. the simplest method is directly fine-tuning the target model with specific edit examples. in addition, cao et al. (2021) propose knowledge editor (ke) approach, which trains a hyper-network to predict the weight update for each edit example. to overcome the limitation of ke that falls short in editing language models with larger scale, mitchell et al. (2022a) introduce model editor networks with gradient decomposition (mend), which learns to transform the gradient obtained by standard finetuning using a low-rank decomposition of gradients. furthermore, mitchell et al. (2022b) present serac, which stores edit examples in explicit memory. it utilizes a scope classifier to determine if an input lies within the scope of any cache items and trains a memory-based retrieval-augmented counterfactual model to edit facts that fall within the scope of stored edit examples. other works introduce additional trainable parameters while keeping the original model parameters static @xcite . for instance, @xcite propose transformer-patcher, a sequential model editing method that simply incorporates and trains a few neurons in the last feed-forward network layer. training-free editing method another line of research attributes knowledge to specific neurons or modules within the network. notably, @xcite view feed-forward network modules in transformer as key-value memories and propose the knowledge neuron (kn) method to edit specific factual knowledge by identifying the knowledge neurons positively correlated to the knowledge expression. similarly, @xcite modify feed-forward weights to update specific factual associations using rank-one model editing (rome). to extend model editing to multiple cases simultaneously @xcite , @xcite build upon the rome framework and propose memit. with the development of in-context learning, certain methods edit the model by either prompting it with the edit example @xcite , or retrieving demonstrations from the edit memory @xcite . ## task definition the objective of model editing is to adjust an initial target model's behavior on a specific edit example, without impacting the model's performance on other irrelevant examples. more specifically, the target model f θ is represented by a function f : @xmath0, with θ denoting the model parameter. ## preliminary in this section, we introduce the objectives of the existing training-based model editing methods. an ideal post-edit model f θe should satisfy three properties: reliability, generality and locality @xcite . reliability a model editor is reliable if f θe predicts the edit label y e for the edit input x e : e x ′ e ,y ′ e ∼{(xe,ye)} 1{argmax y f θe (y|x ′ e ) = y ′ e }. (1) generality as the model editing process can impact a wide range of examples that are closely associated with the edit example, known as the editing scope, a successful edit should also adjust the model behavior for in-scope examples i(x e , y e ), such as examples with similar expressions: e x ′ e ,y ′ e ∼i(xe,ye) 1{argmax y f θe (y|x ′ e ) = y ′ e }. (2) locality a good edit is supposed to edit the knowledge without influencing other irrelevant outof-scope examples o(x e , y e ). it always refers to locality (or specificity): e x ′ e ,y ′ e ∼o(xe,ye) 1{argmax y f θe (y|x ′ e ) = f θ (y|x ′ e )}. to achieve reliability and generality, existing works @xcite use the loss function l e for the edit example and its corresponding in-scope examples: to achieve locality, most works @xcite choose to constrain updates in terms of the kullback-leibler (kl) divergences between the pre-and post-edit model conditioned on the locality examples x loc : the total training loss l ori is computed with the sum of l e and l loc using a hyper-parameter c e : apart from the aforementioned losses, different methods may employ their own method-specific objectives to update the model parameters. despite these variations, in this paper, we consistently denote the original training loss of previous methods as l ori . ## method existing methods are still struggling to maintain high locality @xcite . we assume the limitation is that they solely considering the kl divergences between the pre-and post-edit model on locality examples from the training set. this proves inadequate since irrelevant examples not collected in the training set can still be potentially influenced. to address this issue, we are committed to exploring strategies that guide model parameter updates in a direction that protects the semantic space of irrelevant examples from being disturbed. it has been demonstrated that language models are pre-trained to implicitly learn sentence representations @xcite . each sentence can be encoded as a high-dimensional vector, representing a point in the embedding space. the knowledge acquired by the model is mapped into this space, characterized by an uneven distribution where certain regions may be denser than others @xcite . since the knowledge acquired by llms is encoded within a network of interconnected neurons, editing one piece of knowledge is likely to impact others @xcite . when editing knowledge, models often face challenges in determining the appropriate gradient direction for ## preliminary experiments in this section, we conduct preliminary experiments to assess the effectiveness of sss across three training-based methods. the experimental settings and results are outlined as follows. all results are averaged over three runs. ## comprehensive study the impact of model editing on the language model is intricate, demanding a thorough and comprehensive evaluation to fully comprehend its effects. consequently, in line with @xcite , we conduct additional tests to assess sss from both ## conclusion in this paper, we introduce sss, a novel evaluation metric to measure the semantic sparsity of a sentence for a specific language model without any human or machine annotation. subsequently, we incorporate sss into existing training-based model editing methods to enhance the locality of irrelevant neighborhood examples. experiments conducted on two datasets across various models demonstrate the validity and scalability of sss. in summary, sss presents a promising approach for mitigating the side effects of model editing, offering a notable perspective compared to existing training-based model editing methods.