WISE: Rethinking the Knowledge Memory for Lifelong Model Editing of Large Language Models

Authors: peng  wang
zexi  li
ningyu  zhang
ziwen  xu
yunzhi  yao
yong  jiang
pengjun  xie
fei  huang
huajun  chen
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-01 18:18:52

Abstract: Large language models (LLMs) need knowledge updates to meet the ever-growing world facts and correct the hallucinated responses, facilitating the methods of lifelong model editing. Where the updated knowledge resides in memories is a fundamental question for model editing. In this paper, we find that editing either long-term memory (direct model parameters) or working memory (non-parametric knowledge of neural network activations/representations by retrieval) will result in an impossible triangle---reliability, generalization, and locality can not be realized together in the lifelong editing settings. For long-term memory, directly editing the parameters will cause conflicts with irrelevant pretrained knowledge or previous edits (poor reliability and locality). For working memory, retrieval-based activations can hardly make the model understand the edits and generalize (poor generalization). Therefore, we propose WISE to bridge the gap between memories. In WISE, we design a dual parametric memory scheme, which consists of the main memory for the pretrained knowledge and a side memory for the edited knowledge. We only edit the knowledge in the side memory and train a router to decide which memory to go through when given a query. For continual editing, we devise a knowledge-sharding mechanism where different sets of edits reside in distinct subspaces of parameters, and are subsequently merged into a shared memory without conflicts. Extensive experiments show that WISE can outperform previous model editing methods and overcome the impossible triangle under lifelong model editing of question answering, hallucination, and out-of-distribution settings across trending LLM architectures, e.g., GPT, LLaMA, and Mistral.

Main paper: 
 introduction: 
 Large language models (LLMs) show emergent intelligence when scaling the number of parameters and data [1] [2] [3] [4] , which reveals the sparks of artificial general intelligence [5] . However, when deployed, LLMs still make mistakes [6] , generating responses with hallucinations [7] , bias [8] , and factual decays [9] . On the other hand, the world's knowledge is ever-growing, so the up-to-date knowledge is usually different from the one during LLMs' pretraining [10] . Many such errors and emerging facts will arise sequentially in deployment, some of which have to be addressed timely and efficiently without waiting for retraining or finetuning [11, 12] . Also, retraining or finetuning is often too computationally expensive [13, 10] , which is not sustainable for lifelong growing knowledge. Therefore, lifelong model editing [10] was proposed to remedy the continual knowledge updates and injections for LLMs in a cheap and timely manner.An effective lifelong model editing approach should satisfy the following properties [14, 15, 11, 16, 17] : i) reliability, the model can remember both current and previous edits after sequential editing; ii) locality, model editing will not influence inherent pretrained knowledge which is irrelevant to the edited knowledge; iii) generalization, the model is not just merely memorizing the query-target pairs; ZsRE dataset, number of continual edits T = 100, LLaMA-2-7B. Editing methods based on long-term memory (ROME and FT-EWC) and working memory (DEFER and GRACE) show the impossible triangle in metrics, while our WISE is leading in all three metrics. instead, it should understand and generalize when given other forms of queries with the same knowledge. We compare existing model editing and continual learning methods on the three metrics in Figure 1 and find that it seems to be an impossible triangle-reliability, generalization, and locality can not be realized at the same time in the continual editing settings. We find that where the updated knowledge resides in memories affects editing performances, and previous methods can be generally divided into editing either long-term memory, e.g., ROME [18] , MEMIT [19] , and FT-EWC (Finetuning with Elastic Weight Consolidation [20] , a continual learning method), or working memory, e.g., GRACE [10] . Note that the categorization of long-term and working memories is derived from human recognition [21, 22] and neuroscience [23] which has recently been adopted in the study of LLMs [24] [25] [26] [27] . Model editing of long-term memory refers to directly editing the model parameters, which contain generalizable parametric knowledge [28, 24] . However, editing long-term memory will cause conflicts with previous pretrained knowledge, resulting in poor locality (e.g., ROME and FT-EWC in Figure 1 ). Working memory refers to the non-parametric knowledge of neural network activations/representations by retrieval, and it does not change the network parameters [24] ; instead, it replaces the representations by retrieval at working (inference) time, like GRACE. GRACE's working memory shows promising results in reliability and locality, but in our experiments, it shows poor generalization since retrieval-based representations can hardly make the model understand the edits and generalize to different queries. It reveals that long-term memory and working memory both have drawbacks for lifelong model editing, though there were some special memory designs for LLM architectures, like MemorryLLM [28] , SPALM [27] , and Memoria [25] , they change the architectures and cannot be directly applied for different LLMs. Intuitively, there is a gap between editing working and long-term memories, thus, in this paper, we study:What is the better memory mechanism for lifelong model editing to break the impossible triangle?Human brains contain the left and right hemispheres, which have different divisions as studied in recognition science [29, 30] , e.g., the left brain is typically associated with logical tasks while the right brain is more involved in intuitive processes. This inspires us to design WISE, which makes model editor WISER in memories. WISE contains a dual parametric memory mechanism for LLMs' editing: the main memory for the pretrained knowledge and a side memory for the edited knowledge, realizing both long-term memory's generalization and retrieval-based working memory's reliability and locality. The side memory is a form of mid-term memory. We only edit the knowledge in the side memory and train a router to decide which memory to go through when given a query. For continual editing, we design a knowledge-sharding mechanism where different sets of edits reside in distinct and orthogonal subspaces of parameters. These are then merged into a common side memory without conflicts. Our contributions are as follows:• We identify the pitfalls of current model editing methods in lifelong settings, that is, the impossible triangle among-reliability, generalization, and locality. Behind the impossible triangle, we find there is a gap between editing long-term memory and working memory. • We propose WISE, with a side parametric memory as the mid-term memory, realizing the advantages of both parametric long-term memory and retrieval-based working memory. We design memory routing, sharding, and merging modules in WISE, making WISE lead in continual knowledge editing, reaching the three metrics better simultaneously. • Extensive experiments on GPT, LLaMA, and Mistral across QA, Hallucination, and out-ofdistribution datasets validate the effectiveness of WISE for lifelong model editing.Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?Answer: [Yes]Justification: Abstract and Section 1 Introduction Guidelines:• The answer NA means that the abstract and introduction do not include the claims made in the paper. • The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers. • The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings. • It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper. 
methodology 2.1 preliminaries: lifelong model editing: 
 We focus on lifelong model editing problem [10, 11] , which can ensure hundreds or even thousands of sequential edits on LLMs to make the outputs of target queries align with human expectations while maintaining LLMs' previous knowledge and capability. Let f Θ : X → Y, parameterized by Θ, denote a model function mapping an input x to the prediction f Θ (x). The initial model before editing is Θ 0 , which is trained on a large corpus D train . When the LLM makes mistakes or requires injections of new knowledge, it needs model editing with a time-evolving editing dataset as D edit = {(X e , Y e )|(x 1 , y 1 ), ..., (x T , y T )}. At the time step T , a model editor (ME) takes the T -th edit and the LLM of the T -1 time step f Θ T -1 as inputs and produce the revised LLM model f Θ T following the equation below:EQUATIONEquation 1 describes that after model editing, the LLM should make the correct prediction on the current edit as f Θ T (x T ) = y T , while also preserving knowledge from past editing instances (x <T , y <T ) ∈ D edit as well as maintaining capability of f Θ0 on the irrelevant data when x / ∈ X e , especially for general training corpus D train .Table 1 : Comparison of current model editing methods. " " refers to "yes" and "well-supported", refers to "no" or "badly-supported", and " " refers to "less-supported". The three metrics of Reliability, Generalization, and Locality denote the performances on lifelong (continual) editing.Long-term Memory Working Memory Parametric Knowledge Retrieval Knowledge Whether Lifelong Reliability Generalization LocalityIn Table 1 , we compare current model editing methods in terms of memory types and lifelong editing abilities. FT-EWC [20] , ROME [18] , MEMIT [19] , and MEND [31] edit the long-term memory stored in the LLMs' model parameters, but they either do not support continual editing or have negative effects on irrelevant knowledge (poor locality). GRACE [10] is designed for lifelong editing via retrieval-based working memory. The retrieval codebook can avoid the conflicts of irrelevant knowledge, but GRACE fails to generalize due to its codebook being a non-parametric knowledge representation that solely memorizes queries without comprehension. It is worth noting that SERAC [32] /DEFER [10] uses working memory that is stored in additional small models: a scope classifier and a counterfactual model, whose knowledge is parametric. However, the small counterfactual model cannot match the expressiveness and generalization capabilities of LLM itself, making it challenging for the edited knowledge to generalize effectively.To enable effective lifelong model editing, the method should take advantage of both LLM parameters' long-term memory and retrieval-based working memory. Therefore, we propose WISE as follows.As illustrated in Figure 2 , WISE comprises two key components: 1) Side Memory Design: i) side memory: side memory is a memory container that is initialized as a copy of LLM's certain FFN layer, storing the stream of edits; ii) memory routing mechanism: similar to retrieval, a routing activation component is adopted to identify the scope of edits, routing the main (original) or side memories during inference; 2) Knowledge Sharding and Merging: i) knowledge in random memory subspaces: to make the edits in appropriate knowledge density and avoid forgetting, we shard the side memory into several random subspaces for editing; ii) knowledge merging: we leverage model merging techniques to merge different memory shards into one side memory without loss of knowledge. Side memory in FFN's value matrix. Each layer in a Transformer contains a multi-head self-attention (MHA) mechanism and a feed-forward network (FFN), where the FFN constitutes two-thirds of the model parameters [33] . The question of how Transformers retrieve and utilize stored knowledge remains unresolved [18, 34] , yet past works [31, 33] have demonstrated that editing the weights of the FFN is consistently more effective for LLMs. The FFN typically consists of key-value linear matrices: W k , W v , i.e., two multi-layer perceptron (MLP) layers. For the output of attention feature f , the computation of the feed-forward network, omitting the bias terms, can be represented as:EQUATIONwhere σ is a nonlinear activation function (e.g. SwiGLU, GeLU), and a represents the activation values of the first MLP layer. Following previous works [18, 33] , we edit the value matrix W v of the chosen FFN layer. However, directly editing the value matrix may cause forgetting and side effects in a lifelong setting. Thus, we copy a value matrix as side memory and edit the side memory instead of the original matrix (main memory). Specifically, the side memory is initialized with the copy of main memory as W v ′ ← W v . Given the side memory, the new output is expressed as FFN s (f ) = a • W v ′ . We will introduce how to update the side memory in Section 2.3.2.Locating side memory's FFN layer. Transformer LLMs have been widely demonstrated to encode "lower-level" information (e.g., parts of speech) in earlier layers while processing more advanced linguistic phenomena like anaphora and coreference in later layers [35] [36] [37] . Representations in later hidden layers propagate through residual connections without drastic changes [38, 18] , enabling effective early exit in LLMs [39, 40] . Therefore, to minimize the side effects of editing and adjust advanced linguistic phenomena, we target mid-to-late layers (e.g. 27) for side memory. Further analysis of layer selection is provided in Section 3.3.Routing between side memories and main memory. Similar to the retrieval-based methods [10, 32] , during inference, it is needed to decide whether the main memory or the side memory is used. If a given query is within the scope of previous edits, the side memory is used; otherwise, the main memory.Inspired by [11] , we introduce a routing activation indicator, given an input x, it is formulated:EQUATIONwhere A(•) = a is the activation of the side memory's corresponding FFN layer in Equation 2. We want the activation indicators of editing queries to be larger than the ones of irrelevant queries by a large margin, which is:EQUATIONwhere D irr is the irrelevant dataset which includes D train .To achieve the above objective, we design a margin-based loss function during editing training, similar to contrastive [41] or triplet loss [42] . The margin-based loss function for routing activation is:La = min W v ′ {max(0, ∆act(xi) -α) + max(0, β-∆act(xe)) + max(0, γ -(∆act(xe) -∆act(xi)))}, (5) s.t. xe ∈ Dedit, xi ∈ Dirr.Equation 5 aims that for all queries of irrelevant examples x i , the activation indicators should be less than threshold α, and for the edit samples x e , the activations should be larger than threshold β, with a certain distance γ between ∆ act (x e ) and ∆ act (x i ).In the continual stream of incoming edits, the smallest activation indicator within the edits is updated and saved: ϵ = min{∆ act (x e )|x e ∈ D edit }. We aim to recognize the local scope of edits in this form.During inference, if the activation indicator of a new input is greater than ϵ, WISE will use the side memory W v ′ ; otherwise, using the main memory W v . Thus, given the query x, the output of the targeted FFN in Equation 2 is replaced by:EQUATION)How to effectively and efficiently store continual knowledge in model parameters is important for lifelong editing. We introduce the notion of "knowledge density" (similar to knowledge capacity [43] ) that describes how many pieces of knowledge are stored per parameter on average. There is an editing dilemma w.r.t. knowledge density: i) If only a few edits are made for full fine-tuning or editing the entire memory, the knowledge density is low, which may lead to overfitting. ii) If numerous edits are made within a common and limited parameter space, the knowledge density is high, resulting in conflicts within the edited knowledge and potentially causing catastrophic forgetting. To remedy this dilemma, we propose a knowledge sharding and merging mechanism to divide the edits into several shards, store them in different parameter subspaces, and merge them into a common side memory.Knowledge in random memory subspaces. We edit the side memory W v ′ . We divide n edits into k shards, copy the side memory for k times, and generate k random gradient mask with mask ratio ρ for each copy of side memory. A random gradient maskM i ∈ {0, 1} |W v ′ | , i ∈ [k]is a binary mask whose proportion of 1 is ρ [44] . For edit shard i, i ∈ [k], we edit the knowledge into the subspace M i as follows:EQUATIONwhere W i v ′ is the i-th copy of the side memory, η is the learning rate, g i (•) is the gradient of the i-th shard of edits, and the gradient is the autoregressive loss plus the routing activation loss L a (Equation 5): L edit = -log P W v ′ (y e |x e ) + L a . The random mask of gradients freezes the parameters intact when the elements are 0 and updates the weights when the elements are 1. It is superior to pruning because it does not harm the network performance while regularizing optimization in a subspace [44] . In addition, the ρ subspace will have higher knowledge density when k • ρ < 1, resulting in higher generalization (e.g., Figure 5 ). Also, different shards of edits have different random masks, and due to the (sub)orthogonality of random masks, different shards will not conflict with each other. Therefore, we can non-destructively merge the k copies of side memory into one.Knowledge merging. We merge the k subspace pieces of side memory into one. Because we randomly generate the subspace masks, different random masks will have some overlapping elements and some disjoint elements, following the theorem below: The theorem shows that larger ρ will cause more overlap of subspace parameters, and the proof is in Appendix C. We find that this overlap is helpful in playing the role of "anchors" for knowledge merging (See Figure 5 and Appendix B.5). However, knowledge conflicts also exist in the overlapped parameters, so we leverage the recent task arithmetic model merging technique Ties-Merge [45] to = W i v ′ -W v |i ∈ [k]}.Then, we use Ties-Merge to merge the edit vectors into one:EQUATIONTies-Merge consists of three steps: i) trim: trim the redundant parameters for each task vector; ii) elect the sign: elect the signs of each parameter; ii) disjoint merge: compute the disjoint mean for each parameter which has the same and correct signs [45] . By Ties-Merge, different subspaces of knowledge are integrated into one with fewer conflicts. We study the effects of different merging techniques in Table 11 of Appendix B.2.Routing and retrieving among several side memories. One single side memory has its limited knowledge capacity [43] . For the lifelong editing stream, we can produce several side memories and retrieve them via activation score routing. We compute different activation indicator scores of side memories and retrieve the top-1 during inference. This design is named WISE-Retrieve, which enables a more challenging lifelong editing scenario. For WISE with only one side memory, it is notated as WISE-Merge. For most of the experiments, we use WISE-Merge by default, and we compare WISE-Retrieve in Table 6 and Figure 6 . The pseudo-code of our method can be found in Algorithms 1 and 2.Question: Does the paper discuss the limitations of the work performed by the authors?Answer: [Yes]Justification: Section 5 Limitations Guidelines:• The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper. • The authors are encouraged to create a separate "Limitations" section in their paper.• The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be. • The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated. • The authors should reflect on the factors that influence the performance of the approach.For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon. • The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size. • If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness. • While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations. 
experimental settings and evaluation metrics: 
 In the experiments, we compare the performance of different baselines and WISE in sequentially editing LLM models hundreds to thousands of times. In practice, we augment x e by generating 10 random token sequences of length 10 using f Θ , enhancing editing generalization/adaptation to diverse contexts. We ensure that this augmentation with random tokens is applied across all baselines (See Appendix B.6, we ablate the contribution of Random Token). Datasets and Models. We choose trending autoregressive LLM models LLaMA-2-7B [13] , Mistral-7B [52] , and GPT-J-6B [53, 54] for evaluation. The dataset details are in Table 3 . Following [10] , we evaluate WISE on the closed-book question-answering (QA) dataset ZsRE [46] , and also evaluate its ability to correct Hallucination in SelfCheckGPT [48] . The Temporal dataset [50] is employed to test the out-of-distribution (OOD) generalization of editing. Since Temporal comprises emerging entities post-2019, we avoid using the latest LLMs in OOD experiments. Instead, we follow the original literature of the Temporal dataset [50] and adopt GPT-J-6B as the base model, which is pretrained on the Pile [51] with a cutoff in 2020. Implementation details and editing examples for each dataset and can be found in Appendix A. Baselines. The baselines include methods of continual learning and model editing. We compare WISE against direct fine-tuning FT-L with an additional KL divergence loss [18] , and continual learning fine-tuning based on Elastic Weight Consolidation (FT-EWC) [20] . We also compare WISE to other model editors, including 1) GPT-style editors based on causal tracing: ROME [18] , MEMIT [19] , and MEMIT-MASS (a batch-editing version of MEMIT); 2) hypernetwork-based editors: MEND [31] ; and 3) the latest memory-based editors: DEFER (inspired by SERAC [32] for inference routing) and GRACE [10] . Details on all comparisons are found in Appendix A.2.Each edit example includes an edit descriptor (i.e., query) x e , its paraphrase prompts x e ′ (if available) for testing generalization, and an unrelated statement x loc for testing locality. For the editing dataset D edit = {(X e , Y e )} with T edits, we evaluate the final post-edit model f Θ T after the T -th edit example (x T , y T ). We evaluate the model editor's reliability and generalization using the metrics Rel. (a.k.a Edit Success Rate [10] ) and Gen. (Generalization Success Rate [55] ), while Loc.(Localization Success Rate [55] ), defined as the post-edit model should not change the output of the irrelevant examples x loc , assesses specificity. We report these metrics and their mean scores, which are formally defined as:EQUATIONwhere 1(•) is the indicator function. Notably, for the Hallucination dataset, following [10] , we use the perplexity (PPL) to verify the locality, and there is no proper metric for generalization.Competitive Performance of WISE. The competitive performance of WISE is evident in Table 2 and 4 , which compare its results with eight baselines on the QA (ZsRE) and Hallucination (SelfCheckGPT) settings. In general, we observe the followings: ❶ WISE outperforms existing methods on multiple tasks after long editing sequences; ❷ direct editing of long-term memory (ROME, MEMIT, etc.) creates conflicts with prior pretraining knowledge, resulting in poor locality; and ❸ retrieving working memory and modifying activations (GRACE, DEFER, etc) struggle to generalize to diverse queries. In a nutshell, most previous methods struggle to balance Rel., Gen., and Loc., particularly in long-form editing tasks. In addition, the results of GPT-J-6B can be found in Figure 9 in the Appendix. WISE also surpasses the baselines on the Hallucination dataset, maintaining the lowest perplexity scores of 3.12 and 5.21 at T = 600, with Loc. remaining above 0.93. We similarly observe Out-of-Distribution Evaluation. Ideally, model editing needs to generalize distributionally from formulaic editing examples to natural texts [50] , where the distributional shift involves complexity rather than conventional domain shift [56] . Following [50] , we evaluate the OOD generalization of editing methods on emerging entities using the temporal updating dataset, Temporal. Editing examples and evaluation metrics are provided in Appendix A.1. As shown in Table 5 , WISE effectively handles out-of-distribution generalization tasks (achieving the best OOD Gen. and overall performance). DEFER delivers mediocre performance on OOD Gen. due to the limited capacity of the auxiliary model [14] . During the fine-tuning phase, GRACE and MEMIT focus on the representation v * of a single input token after W v (GRACE: last token, MEMIT: last subject token). However, regarding v * the editing carrier encounters two problems: 1) the training objective is not aligned with the pretraining phase, and 2) the single representation limits the search scope of gradient descent, making it difficult to handle OOD generalization. WISE, on the other hand, avoids these challenges. 4 , the ablation results reveal that editing critical layers like the early and final layers (0, 1, 31) is ineffective, even resulting in a very low Loc. value of 0.096, which indicates a failure to recognize the editing scope. This may occur because the early layers represent fundamental grammatical information, and the final layer directly controls the decoding procedure, leading to poor editing of advanced language functions. Editing in the intermediate layers is suboptimal but still shows a markable improvement compared to early layers, possibly because intermediate layers start to integrate basic grammatical information with more complex semantic data. Notably, the mid-to-late layers demonstrate exceptional editing performance; for instance, selecting layer 26 results in an 80% success rate and generalization while maintaining 100% locality. This empirically supports our claim in Section 2.3.1 that the redundant mid-to-late layers [39] are ideal side memory layers and confirms the hierarchical nature of information processing in Transformer LLMs [57, 58] .Analysis of ρ and k for WISE. We analyze the important hyperparameters of WISE: the mask ratio ρ and the number of subspaces k in Figure 5 . On the left figure, for k = 2, the best ρ is 0.2, satisfying k * ρ = 0.4 < 1, which implies the effectiveness of our subspace design that higher knowledge density will cause better generalization. When scaling k, we observe an increasing demand of ρ. From Theorem 2.1, the probability of subspace overlap is ρ k , and we hypothesize that this overlap is important as an anchor for model merging. Interestingly, from the right figure, it can be observed that the optimal cases always have the ρ k closest to 0.03. This shows an inherent tradeoff between merge anchor and merge conflicts, and the subspace overlaps around 0.03 are optimal for the best performances. Such experiments indicate that 20% FFN parameters can accommodate at least 500 edited samples. When "mask memory exhaustion" occurs, we can allocate new mask parameters to store new knowledge. Using retrieve when knowledge isn't full and merging as needed to save memory, achieves true lifelong model editing. Scale Up to 3K of Edits. We scale the number of continual edits to 3K in Table 6 . We compare WISE-Merge, keeping one side memory by multi-time merging, and WISE-Retrieve, keeping several side memories by routing and retrieving among different side memories. For WISE-Retrieve, we show an upper bound "oracle", which always identifies the correct routing path. We observe that the WISE series maintains high scalability, consistently outperforming the strongest baselines including MEMIT-MASS and GRACE. WISE-Retrieve based on top-1 activation retrieval demonstrates the best results in 3K edits, showing the effectiveness of well-organized memory subspaces and routing strategies during editing. We note that the "oracle" exhibits marginal performance decline when scaling the edits from 2K to 3K, yet it demonstrates remarkable performance across all metrics. This underscores the potential of WISE to handle extremely long continual edits, contingent upon substantial improvement in the retrieval of side memories. Additionally, an appropriate replay of edits can further improve retrieval accuracy, as detailed in Appendix B.3. Inference Time Analysis of WISE. Figure 6 shows the inference time of a single instance for LLaMA after t ∈ [0, 3000] editing steps, measured across 10 trials of each setting. Consistent with our expectations, we find that WISE-Merge incurs a constant inference delay (about 3%) as the editing stream expands. WISE-Retrieve, due to the introduction of retrieval routing, shows an increase in inference time as the number of edits increases, with a time cost increment of about 7% after 3K edits. Knowledge merging ensures that WISE-Merge only brings constant additional costs (0.64% extra parameters and 4% extra GPU VRAM, as detailed in Appendix B.7), contrasting with past memory-based works that continuously demand more available memory [10, 32] .Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?Answer: [Yes]• The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.) • The assumptions made should be given (e.g., Normally distributed errors). • It should be clear whether the error bar is the standard deviation or the standard error of the mean. Justification: We use publicly standard datasets that do not contain information about individual people or offensive context to our knowledge. Ethical considerations are discussed in Section 5.Guidelines:• The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.• If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics. • The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction). 
related works: 
 Memory and Knowledge Injection of LLMs. LLMs have long-term (episodic) and working memory [24, 25, 27] . Long-term memory is stored in model parameters, updatable via (re)pretraining [53] , finetuning [59] , and model editing [14] . Working memory resides in neuron activations, utilized during inference [24] . In-context learning and retrieval-based editing methods like GRACE contribute to working memory [60, 10] . However, whether finetuning or retrieval is debated [61, 62] . Also, current knowledge injection methods often suffer from computational overhead [13, 10] , catastrophic forgetting [63] , and overfitting [64] . Methods like MemorryLLM [28] , SPALM [27] , NKB [65] , and Memoria [25] are proposed to improve the memories from the architecture design perspective.Model Editing of LLMs. Model editing encompasses constrained finetuning, locating-and-editing, meta-learning, and retrieval-based methods. ROME identifies factual associations and edits efficiently using MLP-based memories [18] , extended by MEMIT for mass-editing [19] . T-Patcher adds neurons for edits in LLMs' feed-forward layers [11] . Meta-learning methods like MEND decouple finetuning gradients to generalize edits [31] , complemented by MALMEN addressing cancellation effects [15] . Retrieval-based methods like SERAC and GRACE improve working memory for editing [32, 10] . From single to mass editing and static to lifelong editing, model editing evolves to meet realistic demands. The latest efforts in lifelong editing such as LTE [66] , MALMEN [15] , and RECIPE [67] require extensive training with domain-specific edits before specific editing, yet we cannot predict the domain of upcoming edits in the editing flow and accessing these data is often impractical or unrealistic. It potentially increases the risks associated with retraining.Model Merging Model merging [68] , also known as model fusion [69, 70] , studies how to aggregate different models' knowledge into one by parameter merging. However, in the research of linear mode connectivity, it is found that different minima of neural networks can hardly be merged into a generalized one even if trained on the same datasets from the same initialization (but with different random seeds) [71, 72] . The main reason is considered to be the permutation invariance property of deep neural networks, which means that the positions of neurons can be permuted without affecting the network function [71] ; as a result, different minima reside in different loss basins [72] . To improve linear mode connectivity and model merging, methods like optimal transport [70, 73] , re-basin [72] , and training-time alignment [44] are developed. For the applications, model merging techniques can help to improve the generalization of federated learning [74, 75] and enable knowledge aggregation of different-task models in a task arithmetic way [76, 77] . Recently, methods like task arithmetic in tangent space [77] , TIES-Merging [45] , ZipIt! [78] , and ColD fusion [79] have been proposed for deep model fusion of pretrained foundation models, such as CLIP, ViT, and large language models. Specifically, TIES-Merging [45] consists of trim, elect sign & merge pipeline, which inspires the merge process of side memories in our paper. For detailed related works, please refer to Appendix D. 
limitations and broader impacts: 
 Although WISE shows promising results in lifelong editing, it also has some limitations. One limitation is addressed in Table 6 that the side memory retrieval has room for improvement to reach the oracle. Also, in Figure 6 , the inference time of WISE-Retrieve increases with ever-growing editing streams. However, the current limitations cannot outweigh the merits of WISE in that it currently reaches better performance in general for lifelong model editing. We bridge the gap between long-term and working memory, it may inspire further work on memory design for model editing or even LLM architecture. However, the application of such technologies should be guided by ethical considerations. Malicious users may attempt to edit LLMs to propagate hate, highlighting the need for safeguards to prevent abuse and mitigate harmful outcomes. Some current model editors update the model's weights directly, making edits hard to trace and withdraw. WISE uses a modular and non-destructive side memory, allowing users to discard it if edits are unnecessary or harmful, without modifications to the main LLMs. 
conclusion: 
 In this paper, we point out the impossible triangle of current lifelong modeling editing approaches that reliability, generalization, and locality can hardly be achieved simultaneously. We find the reason behind this is the gap between working and long-term memory. Therefore, we propose WISE, consisting of side memory and model merging, to remedy the gap. Extensive results show WISE is promising to reach high metrics at once on various datasets and LLM models.x e , y e Which continent is Berkner Island in? South Americax loc who gets the golden boot if its a tie? sharedx ′ e , y e On which continent is Berkner Island located? South America (b) Hallucination editing dataset example. In the original data [10] , there is no paraphrase x e ′ so the measurement of Gen. metric is ignored here.x e , y e This is a Wikipedia passage about heinz christian pander. Heinz Christian Pander (1794 -1865) was a German anatomist and embryologist who was born in Riga, Latvia.He studied medicine at the University of Dorpat and later at the University of Berlin. In 1820, he took part in a scientific expedition to Bokhara as a naturalist.x loc Tired and restlessly, drifting in and out of sleep. Hearing crashing and banging, thinking the roof will cave in. Not alert enough to quite know what it was, I yelled loudly for whoever was making those noises at such an hour to stop. They heard and listened, I'm guessing ZsRE The ZsRE question-answering task [46] is extensively studied within the model editing literature [18, 19, 31, 15, 11] , where each record contains an editing statement x e , a paraphrase prompt x ′ e , and a locality prompt x loc . We use the same train/test split as [31] (163196/19086). Notably, only MEND requires fitting a hypernetwork on the training set; other methods discard the training set and perform edits and evaluations on the test set. In practice, we randomly sample 1K and 3K records from the test set to form the edit sets in Section 3.2 and 3.3. Hallucination We utilize the same dataset as GRACE, SelfCheckGPT [48] , to assess the ability of Model Editors to mitigate hallucinations in autoregressive LMs. This setting involves editing highly inaccurate sentences (sourced from GPT-3 [80] ) and replacing them with corresponding sentences from actual Wikipedia entries. This dataset aligns more closely with real-world deployment scenarios where models trigger "unexpected behaviors," and the token length of edits is significantly longer than in past datasets, making it a more challenging editing setting. Unlike GRACE, which used GPT2-XL (1.5B) [81] , our main experiments deploy larger LLMs, LLaMA and Mistral, both with 7B parameters, we measure retention of pretraining data (x loc ) from the base model: Red-Pajama [49] , a public version of LLaMA's pretraining data. Some of the exceptionally long editing samples cannot even be accommodated on an NVIDIA A800 (80GB) due to resource limitations. As shown in Figure 7 , the original dataset provided by GRACE, after tokenization with LLAMATO-KENIZER, has length distributions ranging from [17, 390] . The dimension of a single MLP layer in llama-2-7b-hf is (11008, 4096) § . Theoretically, fine-tuning an input of length 390 with default § https://huggingface.co/meta-llama/Llama-2-7b-hf x e , y e Self-driving cars, also known as autonomous vehicles, are vehicles that are capable of navigating and operating without human intervention. These innovative vehicles rely on a combination of advanced sensors, artificial intelligence, and computer algorithms to interpret their environment and make real-time decisions. With the potential to significantly impact numerous industries and sectors, self-driving cars have the ability to revolutionize transportation by enhancing safety, improving traffic flow, and increasing energy efficiency. However, challenges related to regulatory frameworks, ethical considerations, and public acceptance still need to be addressed before widespread adoption becomes a reality.x loc Apple has a new peach with the release of its 3.0GHz, 8-core Intel Xeon-based Mac Pro. The 8-core Mac Pro is powered bu two quad-core Intel Xeon Clov ertownprocessors running at 3.0GHz. Apple also released a quad-core Mac Pro featuring two Dual-Core Intel Xeon Ẅoodcrestprocessors.x e , y ood Self-driving cars, also known as autonomous cars or driverless cars, are vehicles capable of traveling without human input. These cars utilize a range of sensors, including optical and thermographic cameras, radar, lidar, ultrasound/sonar, GPS, odometry, and inertial measurement units, to perceive their surroundings. By interpreting sensory information, control systems in the car are able to create a three-dimensional model of its environment. Using this model, the car can then identify the best navigation path and develop strategies for managing traffic controls and obstacles. As self-driving car technology continues to advance, it is expected to have a significant impact on various fields such as the automotive industry, health, welfare, urban planning, traffic, insurance, and the labor market. The regulation of autonomous vehicles is also becoming an increasingly important topic of discussion.full precision and the Adam optimizer would require (390+4+4+4) * (11008 * 4096 * 4) + 4 * 7B = 100.36GB of VRAM (for activations, gradients, first-order, and second-order optimizers), exceeding the memory capacity of the NVIDIA A800. Consequently, we excluded excessively long samples (limiting tokenized lengths to 254) and ultimately retained 906 editing instances (compared to 1392 in GRACE). To facilitate a fair comparison with MEND, we specifically allocated a training set for MEND, with a final train/test split of 306/600. All methods were edited and evaluated on the test set.Temporal [50] sources the prefix x e from the first paragraph of an entity's Wikipedia page and samples a paragraph y e discussed by GPT-4 [82] about the emerging entity x e , which is usually noisy but may contain helpful information. These are presented as editing prompts to Model Editors. For out-of-distribution (OOD) generalization to complex natural contexts (not fitted), y ood is taken from the actual Wikipedia suffix of x e . This setup is utilized to evaluate the OOD generalization of Model Editors centered around a single canonical example. Consistent with previous work [10] , the out-ofscope data x loc is derived from the Pile [51] , the pretraining corpus of GPT-J-6B. Examples from the dataset can be seen in Table 9 . To measure the OOD generalization of editing methods for emerging entities, we perform model editing using standardized simple examples and then evaluate this behavior on more complex instances. Following [50] , in a natural setting, no single correct continuation exists. Thus, we also use probability threshold-based evaluations, such as 80%, where the editing success rate evaluates whether the loss L xe,yood for an example falls below δ = -log(0.8), as indicated in the formula below. The intuition behind this is that many other plausible alternative continuations may exist.EQUATION 
broader impacts: 
 Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?Answer: [Yes] Justification: In Section 5Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.• Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations. • The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)? Answer: [NA] Justification: [NA] Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort. 12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected? Answer: [Yes] Justification: Section 3.1 and Appendix A. We use publicly available artifacts. Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided. • If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.• For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. • If this information is not available online, the authors are encouraged to reach out to the asset's creators. • We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution. • For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review. 
 
 Appendix: FT-L. All other layers of the LLMs remain frozen, and only a single MLP layer is fine-tuned through autoregressive loss [18] . Additionally, we impose an L ∞ norm constraint to prevent the parameters from deviating too far from the pretrained distribution. FT-EWC. Elastic Weight Consolidation (EWC) has been demonstrated to mitigate catastrophic forgetting by updating weights using a Fisher information matrix, which is computed from past edits, multiplied by a scaling factor λ [20] . Following [10] , we omit the constraints of the L ∞ norm in this implementation. MEND. MEND [31] transforms the gradients obtained from standard fine-tuning using a hypernetwork that converts gradients decomposed into low rank (rank=1) into new gradients, which are then applied to the target layer for parameter updates. During the training phase, a small auxiliary hypernetwork receives editing examples (x e , y e ), and x loc . MEND's training loss comprises the standard autoregressive loss combined with the KL divergence loss of the model's output on x loc before and after editing. This hypernetwork plays a crucial role during the editing procedure. ROME. ROME [18] uses causal analysis to pinpoint knowledge within specific MLP layers and modifies the entire matrix through least squares approximation. It operates under the strong assumption that the MLP is the primary module for storing knowledge [33] , and it injects a single piece of knowledge into the MLP at each iteration using a Lagrangian remainder.MEMIT. Similarly, based on the assumption that the FFN serves as a knowledge key-value store, MEMIT [19] manipulates parameters of specific layers directly through least squares approximation. Unlike ROME, which updates a single layer, MEMIT is a multi-layer updating algorithm that supports simultaneous updates of hundreds or thousands of facts. For sequential model editing tasks, MEMIT requires immediate on-the-fly repairs when the model makes errors, expressed asf Θ T = MEMIT(f Θ T -1 ,x T , y T ), involving multiple operations on the original model. MEMIT-MASS. Unlike sequential editing, MEMIT supports modification of multiple knowledge fragments in a batch mode, named MEMIT-MASS. Suppose we collect streaming errors as (X , Y) = {(x 0 , y 0 ), (x 1 , y 1 ), ..., (x T , y T )} and inject them collectively into the MLP, it only involves a single editing operation on the original model as f Θ T = MEMIT(f Θ0 , X , Y). Although this approach loses the capability for on-the-fly repairs, we still include this baseline in our experiments. DEFER. In GRACE, a reimplementation of SERAC [32] is utilized, denoted as DEFER. For new inputs, DEFER includes a network g (corresponding to the scope classifier in SERAC) that predicts whether to: 1) trust the prediction of the LLMs, or 2) trust the prediction of the new model.Here, the new model is configured as a single-layer linear network o with a sigmoid activation function, corresponding to the counterfactual model in SERAC. During the editing process, g and o are fine-tuned jointly. GRACE. GRACE [10] utilizes a discrete KEY-VALUE codebook and maintains the codebook throughout the editing flow by adding, expanding, and splitting KEYs. During the inference phase, it retrieves the nearest KEY and determines whether to replace the activation of the hidden layer output.Except for MEMIT-MASS, the batch size for all methods is consistently 1 in sequential editing scenarios. All experiments are conducted using 3 NVIDIA A800 GPUs, with all tasks reproducible on a single A800. Editing ZsRE takes approximately 4 hours, while Hallucination requires around 6 hours. To ensure fair comparisons, unless otherwise specified (for some methods like MEND, ROME, and MEMIT, we follow the original literature by selecting the last few layers or using causal analysis to identify the target layers), the default target layers for editing on LLaMA, Mistral, and GPT-J are model.layers [27] .mlp.down_proj.weight, model.layers [27] .mlp.down_proj.weight, and transformer.h [21] .mlp.c_fc, respectively. For FT-L, we utilize a reimplementation from ROME ¶ , employing the Adam [83] optimizer with consideration of learning rates at 1e-5, 1e-4, and 5e-4, and conducting gradient descents for 50 iterations, ultimately reporting the best results at a learning rate of 5e-4.For FT-EWC, we follow the reimplementation in GRACE and its default settings, setting the learning rate at 1e-2, the λ ewc penalty factor at 0.1, and the number of replay instances at 10. For the training phase of MEND, we adhere to the original paper, setting the learning rate at 1e-4, iterating 100K times, and employing early stopping at 30K, ultimately achieving an accuracy of 0. For ROME and MEMIT, we follow the original literature on GPT-J using the default configurations, specifically the fifth layer and layers [3, 4, 5, 6, 7, 8] . In LLaMA and Mistral, additional causal analysis is conducted to pinpoint the layers storing knowledge. As shown in Figure 8 , an increasing trend in ¶ https://github.com/kmeng01/rome the Average Indirect Effect of the MLP is observed across layers [4, 5, 6, 7, 8] , suggesting that the model recalls factual knowledge here and passes the matured token distribution via residual connections to the last MLP. Thus, in LLaMA and Mistral, ROME edits the fifth layer, while MEMIT edits layers [4, 5, 6, 7, 8] . For DEFER, the original literature uses a learning rate of 1.0; however, we found it unfit for LLaMA and Mistral, with severe fluctuations in model loss. Therefore, we experiment with learning rates of 7e-5, 7e-4, and 1e-3, and ultimately report using 7e-5 (optimal). For GRACE, we strictly follow the original literature, setting the learning rate at 1.0, and using replace_last to only replace the activation of the last token in autoregressive scenarios. After observing failures in generalization, we adjust various ϵ init values and discuss this more in Appendix B.1.For WISE, the hyperparameters for the QA and Hallucination tasks are identical. We find that a learning rate of 1.0 with the SGD [84] optimizer is a good approach for stable training. The hyperparameters designed in the knowledge editing phase include the random masking probability ρ and the routing threshold guidance α, β, γ. In the knowledge merging phase, hyperparameters include the number of merges k and the merging weights λ for each MLP (we discuss the impact of ρ and k in Section 3.3). Theoretically, as the importance of knowledge in any MLP is considerable, we always average with λ = 1/k across all experiments. These are shown in Table 10 .The pseudo-code of the WISE editing stage is in Algorithm 1, and the one of the WISE inference stage is Algorithm 2.Here, we discuss why GRACE exhibits poor generalization when editing decoder-only LMs. As shown in Figure 10 , we continuously edit 15 samples (x e , y e ) using GRACE and observe the nearest codebook Key for their paraphrases x e ′ and unrelated queries x loc , as well as the governed Deferral radii ϵ of those Keys. When overlapping Keys exist, GRACE reduces the Deferral radii to split this Keys and then adds a new codebook entry, resulting in exponentially decaying of radii ϵ during the editing process. Though ϵ is initialized from a high ϵ init , it will be small and ineffective after continuous edits. From Figure 10 , we observe that GRACE is more likely to have a conservative if WISE-Retrieve then 3:Get the value of activation∆ act = ∥A(x i ) • (W v ′ -W v )∥ 2for each side memory and select the one with the maximal value of ∆ act ;4:else 5:Get the value of activation∆ act = ∥A(x i ) • (W v ′ -W v )∥ 2 ; 6: end if 7: if ∆ act > ϵ then 8:Use the side memory W v ′ to generate the output as in Equation 6;9:else 10:Use the main memory W v to generate the output as in Equation 6.11:end if 12: end for strategy that sets smaller Deferral radii during editing. Smaller Deferral radii will cause x e ′ to fail to hit the codebook (the distance to the nearest Key is farther than its Deferral radii) but let x loc successfully far away from the radii, resulting low generalization and high locality. Also, we observe that the Deferral radii method is not effective under any ϵ init ; for all tested ϵ init values of 1.0, 3.0, 10.0, and 500.0, they all have low generalization and high locality. This suggests that in autoregressive LMs, the distribution of the last token cannot effectively represent semantics; whereas in encoder-only and encoder-decoder architectures, capturing semantic information through vector representation has been extensively studied [85] [86] [87] . This is consistent with the degree of generalization shown by GRACE when anchoring the T5 [88] Encoder layer. Some related works [89] Figure 10 : Investigation on the query x and its distance to the nearest Key k, as well as the deferral radius ϵ of that Key. Red and Blue respectively represent the paraphrase query x e ′ and the unrelated query x loc , with the hatch representing the radius of the nearest Key. We observe that when conflicts occur (hit the codebook Key but with different Edit Target y e ), the deferral radius ϵ decays exponentially. This results in GRACE being unable to encompass the paraphrase x e ′ and maintain high locality, regardless of how ϵ init is adjusted. ZsRE, LLaMA-2-7B. Here, we conduct a more in-depth study of the knowledge merging strategies for WISE, exploring various merging approaches including (i) Linear, which uses a simple weighted average; (ii) Slerp, which spherically interpolates the parameters of two models; (iii) Ties, a component used in the main experiments of this paper that resolves merging disturbances through TRIM ELECT SIGN; (iv) Dare: which follows a Bernoulli distribution to delete redundant parameters and rescale the remaining ones; (v) Dare_Ties, which combines dare and the sign consensus algorithm of TIES; and (vi) Sign, an ablation component of Ties that addresses directional conflicts-all utilizing the official implementation from MergeKit [68] || . We randomly sample 100 edits from ZsRE, retaining a fine-tuned MLP every 50 edits (merging 2 MLPs). As shown in Table 11 , we || https://github.com/arcee-ai/mergekit observe that ignoring the direction of parameter updates (Linear, Slerp, Dare) leads to a significant decline in editing performance, underscoring the importance of addressing knowledge conflicts in overlapping parameters. The success of Sign also reaffirms this point. Meanwhile, the randomly masked knowledge shards exhibit a non-redundancy, indivisible nature. This is demonstrated by the significantly weaker performance of Dare_Ties compared to Ties/Sign, indicating that removing parameter updates can lead to the loss of edited knowledge or even potential "anchors". L memo } when varying T . (a) shows the simple average of Rel. and Gen. (ES.), while (b) shows retrieval accuracy, i.e., whether the Top-1 Activation routes to the correct MLP (prec@1). X-axis: Num edits. ZsRE. LlaMA-2-7B. WISE-Retrieve retains each knowledge-sharding memory and retrieves through Top-1 Activation. However, as shown in Table 6 and Figure 11b , the retrieval accuracy still has significant room for improvement; specifically, when T reaches 3K, the accuracy of routing to the correct MLP drops to around 60%, indicating the specificity between side memories is insufficient. One possible reason is that when sampling the edits from a single dataset (ZsRE), the editing instances (x e , y e ) all belong to the same domain. This leads to some very similar instances being captured by multiple expert side memories (resulting in high activations for all side memories), introducing more retrieval failures. Therefore, to improve the specificity of side memory and reduce the probability of routing errors, we attempt to add a new constraint L memo to Equation 5. For knowledge-sharding memory W i , we randomly replay instances (x m , y m ) from the edit set D Wj of past shard W j, j∈[0,i-1] , ensuring that W i remains inactive for x m :L ′ a = L a + max(0, ∆ act (x m ) -α) Lmemo , s.t. x m ∈ D Wj .As shown in Figure 11b , this replay behavior increases the specificity between side memories, maintaining nearly 88% retrieval accuracy at T = 3K. Figure 11a also shows that WISE-Retrieve w. L memo improves Edit Success (ES.) by 8.39% compared to WISE-Retrieve, providing a promising direction for future work. With finer-grained activation management, we might be able to bridge the performance gap between Retrieve and Oracle.In Table 12 , we present bad cases of using WISE to edit the LLaMA-2-7B on the ZsRE dataset and mitigating these failures is critical for future work in model editing. We observe that in i) errors occur only in part of the tokens, and these errors constitute a large proportion of the bad cases, indicating that the edits have not been sufficiently fitted. ii) displays cases where the entire output is incorrect, and factual failures indicate difficulties in retaining memory of parameters for some rare entities (such as Persian iia, iib). iv) presents cases of generalization failure, for example in ivd), where the model answered "English" but did not fully follow the ground truth, indicating significant room for improvement in the accuracy of generalized edits. Meanwhile, in iii) we surprisingly find that even when WISE errs on the Edit Prompt, it can correctly answer its paraphrase iiib) "The kind American English American English ✓ ivd) What language are Garowe Principles written in? -English English ✗ of voice of Gemma Bosini is what?". This indicates that WISE can handle contextual information correctly in some cases but falls short in specific editing instructions, suggesting that optimizing editing instructions (modifying the editing context) may be a direction for improvement. Here, we discuss the effects of independent (ensured by non-overlapping masks) vs partially overlapping parameters within MLP subspaces on editing performance, as shown in Table 13 . It is observable that, despite varying mask ratios ρ and the number of subspaces k, partial overlap (w. KA) consistently outperforms independent configurations (w.o. KA) in terms of Reliability (Rel.) and Generalization (Gen.). For example, at ρ/k of 5/0.20, there is a relative improvement of 9% and 7% respectively. This demonstrates that the overlapping regions contribute as "anchors" for knowledge fusion, facilitating information transfer across different subspaces. Moreover, the shared parameters provide a natural regularization [90] mechanism, helping synchronize model behavior across different subspaces. As described in Section 3.1, we employ random prefix token augmentation to enable the editing knowledge to cope with various contexts. That is, for a single x e , it expands into (prefix i , x e ). The prefix is derived from tokens that are randomly generated by the original LM f Θ , serving as an economical data augmentation method. We observe that the editing success rate is compromised (Figure 12 ). Specifically, for instance, at T=1000, Rel. and Gen. decreased by 0.15 and 0.17, respectively. By utilizing randomly generated prefix tokens, the model is able to learn a broader range of linguistic features, thereby exhibiting greater robustness in practical applications. We believe that access to the "data generator" can deepen the model's memory of editing samples. The key to lifelong model editing is maintaining constant or slowly increasing computational costs as the number of edits expands. Here, we provide a quantitative analysis using LLaMA-2-7B as an example. Suppose we select model.layers [27] .mlp.down_proj.weight as side memory. In that case, the theoretically added parameters are 11008 × 4096 × 4 = 0.18 GB, which accounts for 0.64% of the original LLaMA's 7B × 4 = 28 GB (ignoring the VRAM required for input activations). As shown in Figure 13 , in practice, WISE-Merge increases VRAM by 4% compared to the original LLaMA and remains constant over time. WISE-Retrieve, instead of merging, uses retrieval routing, meaning the computational cost increases over time, but this increase is gradual and can easily handle thousands or tens of thousands of inputs. Additionally, if we partially merge side MLPs (combining WISE-Retrieve and WISE-Merge), we can further reduce the computational demands of WISE-Retrieve. We start by considering the case of two memory subspaces, W i v ′ and W j v ′ , where i ̸ = j and i, j ∈ [k]. Let P (parameter sampled) = ρ be the probability that a parameter is sampled in one mask generation event.B1. For a single mask generation, the probability that a specific parameter is sampled is ρ. We denote this probability as P (sampled) = ρ. 2. Considering two independent mask generation events, the probability that the same parameter is sampled in both masks is the product of their individual probabilities, i.e., ρ 2 . This is derived from the independence of the events. Mathematically:P (sampled in both masks) = P (sampled) × P (sampled) = ρ × ρ = ρ 2 .3. Extending this logic, for k independent mask generation events, the probability that a specific parameter is sampled in all k masks is ρ k . Mathematically:P (sampled in all k masks) = P (sampled) × P (sampled) × • • • × P (sampled) k times = ρ k .Now, let's calculate the number of parameters overlapped in two random masks:The total number of parameters inW v ′ is |W v ′ |.Thus, the number of parameters overlapped in two random masks,W i v ′ and W j v ′ , is ρ 2 • |W v ′ |.Extending this to k random masks, the number of parameters overlapped in all k masks is ρ k • |W v ′ |. This concludes the proof. □Memory and Knowledge Injection of LLMs The memories of LLMs can be divided into longterm (episodic) memory and working memory (short-term) [24, 25, 27] . Long-term memory refers to the knowledge stored in the model's parameters, which can be updated by (re)pretraining [53] , finetuning [59] , and model editing [14] . Working memory is stored in sustained activations/representations of neurons, which will be awakened during inference time [24] . In-context learning (ICL) is a kind of working memory [60] , also along with retrieval-based editing methods like GRACE [10] . How to reinforce memory and inject/update knowledge for LLMs is a fundamental question [28, 61, 62] . ICL or finetuning? Different works show different conclusions. In [62] , the authors find that few-shot finetuning is more generalizable than ICL, especially for out-of-distribution data. In [61] , the authors contrast finetuning with retrieval-augmented generation (RAG) in terms of knowledge injection and find that RAG is better in most cases, and combining both will produce the best results. However, finetuning and pretraining are computation-expensive [13, 10] and usually suffer from catastrophic forgetting [63] and overfitting [64] . For ICL and RAG, the working memory is sometimes not controllable, the model may not follow the information of the contexts [24] , and the context window is limited [91, 92] , and there are works addressing these issues by training controllable ICL [24] , long-context [91, 92] , and recurrent memory architecture design [28] . SPALM is proposed to add language models with storage modules that resemble both working and long-term memories [27] .Model Editing of LLMs Model editing can be summarized as the following lines of research.Constrained finetuning: Preliminary model editing uses constrained finetuning to update parameters based on new examples [93, 94] . Locate-and-edit: ROME [18] locates the factual associations in autoregressive LLMs and conducts accurate and efficient edits by taking MLPs as key-value memories. Then, MEMIT [19] extends ROME from single-editing to mass-editing. COMEBA-HK [95] identifies the Local Editing Scope and extends MEMIT for sequential editing. In addition, T-Patcher [11] targets the last feed-forward layer of LLMs, adding an additional neuron for each edit. Meta learning: Recent meta-learning methods use hypernetworks for aiding editing. MEND [31] learns a hypernetwork that can decouple the finetuning gradients into the gradient updates that generalize the edits and won't damage the performances on unrelated inputs. To remedy the cancellation effect of MEND, MALMEN [15] uses hypernetwork to produce the weight shifts of editing and formulates the weight shift aggregation as the least square problem. Retrieval-based methods: Instead of directly editing the model parameters, retrieval-based methods aim to improve the working memory of LLMs to enable model editing. IKE [96] uses context-edit facts to guide the model when generating edited facts. DeCK [97] employs contrasting knowledge decoding, which enhances the confidence of in-context-based editors in the edited facts. SERAC [32] (a modified version dubbed as DEFER [10] ) records edit items in a file and trains additional scope classifier and counterfactual model to detect, retrieve, and generate the edit-related results. Though the editing retriever and generator are neural networks, they are too small to have the power of LLMs. GRACE [10] adopts a discrete codebook of edits for retrieving and replacing the edits' layer representations during inference. From single editing [18] to mass editing [15, 19] , and from static editing to sequential [11] (continual) or lifelong editing [10] , model editing is developing to meet more realistic demands.Continual Learning Continual learning [98, 99] tackles the catastrophic forgetting problem in deep learning models with new knowledge [100] , and recent research has focused on various methods in this area. One such method is continual finetuning, where LLMs are refined over time with the arrival of new instances. For instance, a comprehensive study by [101] explores continual finetuning extensively. However, it has been observed that regularizing finetuning with continual learning techniques such as Elastic Weight Consolidation [20] , Experience Replay [102] , and Maximally Interfered Replay [103] can lead to a rapid decay in performance on previous tasks, although it aids in retaining some memory of past inputs. This suggests that editing, as opposed to vanilla continual finetuning, presents unique challenges, especially considering that edits are unlikely to be evenly distributed [104] . One promising direction within the realm of continual learning is the adoption of key-value methods, inspired by advancements in computer vision [105, 106] . Recent studies have showcased the effectiveness of continual prompt-learning for NLP [107, 108] , particularly in applications like text retrieval [109] . Notably, discrete key-value methods have been shown to excel in handling shifting distributions [110] , with some recent efforts extending their application to question answering [111] . These methods cache values to ensure that inputs remain within the distribution for downstream encoders, thus facilitating the incorporation of longer-term memory, provided there are adequate computational resources.NeurIPS Paper Checklist 


Decision: This paper identifies the fundamental challenges in lifelong model editing of LLMs, where reliability, generalization and locality cannot be realized together in the lifelong editing settings. It then proposes a new method to bridge the gap between different types of memory. It develops a dual parametric memory scheme, where a main memory is for pre-trained knowledge and a side memory for edited knowledge. A router is trained to determined which memory to access for a given query. A knowledge sharing mechanism with Ties-merging is introduced for continual editing.

All the reviewers recognized that this paper is well written with comprehensive experiments. The overall proposed designs make sense and are sufficiently supported by the experiments. In addition, all the concerns raised by the reviewers are addressed in the rebuttal period. The authors are encouraged to take the feedback into account in revising the paper (e.g., making it less dense for clarity of presentation).

Reviews: review: paper_summary: paper_summary: A fundamental question in lifelong model editing of large language models (LLMs) is where the updated knowledge should reside in the model's memory. This paper identifies an inherent challenge in editing either long-term memory (direct model parameters) or working memory (non-parametric knowledge through neural network activations/representations by retrieval). It reveals that achieving reliability, generalization, and locality simultaneously in lifelong editing settings is highly challenging in existing approaches. Editing long-term memory by directly modifying parameters can lead to conflicts with irrelevant pre-trained knowledge or previous edits, compromising reliability and locality. In contrast, retrieval-based working memory edits struggle to enable the model to understand and generalize the updates effectively. To address these challenges, this paper proposes WISE, a method designed to bridge the gap between different types of memory. WISE employs a dual parametric memory scheme, comprising a main memory for pre-trained knowledge and a side memory for edited knowledge. Edits are made exclusively in the side memory, with a router trained to determine which memory to access for a given query. For continual editing, we introduce a knowledge-sharding mechanism leveraging Ties-merging, where different sets of edits are stored in distinct parameter subspaces and subsequently merged into a shared memory without conflicts.
strength_weakness: strength_weakness: During the training phase, the model requires excessive memory since it requires copying multiple side memories with the same dimensionality as the original FFN weights in the model. Though it partially alleviates this issue by omitting copying side memory for earlier layers, it still needs substantial additional capacity.

Gradient masks for subspace memorization are randomly generated regardless of the density of target knowledge. It would be great if the model adaptively controls \rho according to the difficulty of incoming knowledge or information quantity.
questions: questions: N/A
limitations: limitations: The paper appropriately addressed limitations and broader societal impacts.
paper_summary: paper_summary: This paper proposes WISE which uses a side memory and some model merging techniques to perform lifelong knowledge editing.
strength_weakness: strength_weakness: 1. **Ever-expanding memory**: As mentioned in line 212-213, "One single side memory has its limited knowledge capacity". Thus, to perform lifelong knowledge editing, it seems that infinite side memories would be needed. In this sense, lifelong knowledge editing is still not solved as with limited side memories, only a limited number of edits can be performed. As the ever-expanding external memory is needed, why do we need to store all this knowledge in the memory instead of just storing the knowledge in the form of raw text and performing knowledge retrieval from this knowledge base? 
2. **Experimental Results**: In the paper MEMIT [1], they scale the batch editing to 10000 examples. There are around 20000 examples in zsRE dataset. However, in the main table (Table 2) only up to 1000 edits are studied. 
3. **Sequential Editing Cases**: Two possible scenarios in sequential editing are: (1) There would be conflicting knowledge overtime. For example, The president of some country may continue changing every a few years. How does WISE perform in this case compared to others? (2) Multi-hop edits as discussed in [2]. However, it seems that both of these situations are not discussed in the paper. 

[1] MASS-EDITING MEMORY IN A TRANSFORMER.   
[2] MQuAKE: Assessing Knowledge Editing in Language Models via Multi-Hop Questions.
questions: questions: 1. **Experimental Results**: In MEMIT[1], they achieve 96.7 on Efficacy and 89.7 on Paraphrase (see Table 1) with 10,000 edits, but in Table 6, MEMIT-MASS has 0.64 in Rel. and 0.58 in Gen. under 2000 edits and 0.58 in Rel., 0.53 in Gen. under 3000 edits. What happened to the model to contribute to the discrepancy between the current results and the results reported in the paper?
limitations: limitations: The limitation of ever-expanding memories may need to be mentioned in the limitation section.
paper_summary: paper_summary: The paper studies the problem of editing (updating) knowledge of LLMs in a lifelong learning scenario. Authors design WISE, a multi-level memory system designed to store updates to the model. The proposed design contains of the main memory and a number of side-memories, with a mixture-of-expert-like router to choose between them. Authors also propose a procedure for merging the main (mid-term) memory into side-memories effectively committing it to the long-term memory. Authors evaluate the proposed technique on small variants of popular LLMs, achieving good average performance on ZsRE and a number of other datasets (SelfCheckGPT, Temporal). The paper also contains several side-analyses such as visualizing the router behavior, scaling to 3K edits, and speed benchmarks.
strength_weakness: strength_weakness: I have two main concerns, though they are not significant ones.

First WISE is a rather complicated system with a lot of moving parts: router type, merging strategy, main/side memory sizes, where to introduce this module into an LLM, and how many times, how best to allocate the memory size between components. Authors provide some ablation analysis (e.g. Appendix B.2 and below), but many of these ablations are missing.

My second (very minor) concern is that authors experiment only with small LLMs (sic), which leaves out the possibility that WISE behaves unexpectedly with larger and higher-capability ones (at the minute of writing this, Llama-3-70B, Qwen-2-72B, Nemotron et al.).
questions: questions: I do not have any insightful questions, so I will instead use this section for minor comments.

### Typos, missing citations, minor concerns

> L389 Malicious users may attempt to edit LLMs to propagate hate, highlighting the need …

There are numerous other scenarios for a potential misuse of this technology. To name a few: 
censorship, particularly for non-democratic state actors
misinformation, of a non-hate-inducing kind

For the record: i do NOT mean that the paper requires an ethical review. Most of these attack vectors are already possible with prior work. It would be unrealistic to expect a full sister study on ethics. Fortunately, we have AI ethics researchers.


> L34 should satisfy the following properties [ 14 , 15, 11 ]

I believe this terminology were originally introduced earlier in Sinitsyn et al (2020) [ https://openreview.net/forum?id=HJedXaEtvS ] and De Cao et al (2021) [ https://aclanthology.org/2021.emnlp-main.522/ ], including for language models. Though the term “LLM” specifically did not exist back then. 

> L25 parameters, computes, and data

While using “compute” as a noun is not yet well studied, I have seen it mostly used as an uncountable noun (like “data” instead of “datas”). Please use your own judgment though.

> L98 (definition formula for D_edit)

To the best of my knowledge, using | for nested definition of Xe,Ye is rarely used (or understood) by ML practitioners. Consider defining them separately




Overall, if this paper ends up accepted, I respectfully ask that authors reduce the total negative (vertical) space used in formatting by exiling some of the less important analyses to appendix or, if all else fails, by using the extra content page granted for the final version.
limitations: limitations: To the best of my knowledge, authors have sufficiently addressed the limitations of their work.
paper_summary: paper_summary: Introduces a new technique, WISE, to edit/insert facts in a autoregressive language model. The proposed approach gives a balanced performance in terms of reliability, generalization, and locality; improving upon the existing methods. WISE edits facts in a side-memory that is equivalent to one of the MLP down projection layers. A router is trained to decide if a specific query should be routed to the side-memory or the corresponding layer in the main LM.
strength_weakness: strength_weakness: Needs more clarity on some of the technical details. See the questions below.
questions: questions: 1. The activation $\textbf{a}$ used in routing correspond to which token in the prompt? Is it the subject last token like ROME/MEMIT or the last token of the prompt like GRACE? 
    * One of the reason GRACE achieves poor generalization is because it edits the last token of the prompt and thus very sensitive to the prompt paraphrases. The notation used in the paper suggests that WISE also uses the last token of the prompt (clarify this if I am wrong). However WISE seems to achieve an impressive generalization. Is that because the parameters of $\mathbf{W}_{v'}$ are directly edited for better routing? Also, does WISE use only one prompt $x_e$ to edit a specific fact, or does it use multiple paraphrases of $x_e$ during the training phase?

2. Parameter Overlap: Theoren 2.1 seems to counts parameter overlap as the number of parameters that are common between *all* the $k$ side-memories as $\rho^k|\mathbf{W}|$. But if a parameter is common between only two side-memories, it is also a overlap, right? And, in that case, the number overlaps would be $\Big(\binom{k}{2}\rho^2 - \binom{k}{3}\rho^3 + \binom{k}{4}\rho^4 \cdots \Big) \cdot |\mathbf{W}|= \sum_{i=2}^k (-1)^i \binom{k}{i}\rho^i \cdot |\mathbf{W}_{v'}|$.
    * I also didn't follow the point on the overlapping playing the role of "*anchors*" for knowledge merging, even after refering to Appendix B.5, which just restates the same point. Can you please clarify this?
    * I also didn't follow the claim made in line 191-192 => *"different shards of edits have different random masks, and due to the (sub)orthogonality of random masks, different shards will not conflict with each other"*.

3. Although WISE claims to be sequential (line 222), it doesn't seem to be the case, as you expect access to $n$ edits to distribute among $k$ shards (line 179-180). Or, is the loss $L_{edit}$ optimized for each of the edits sequentially? Also, how do you perform the merging if you don't have access to all the edits beforehand? If WISE was truly not done sequentially, then it should only be compared with methods that permit parallel/batch edits.

4. As you increase the number of edits $T$, do you also increase the number of sharts $k$? Or, do you always keep $k = 2$ reported in Table 9? Also since the title of the paper focuses on *Lifelong* model editing, I wonder if it is reasonable to increase $k$ rather than choosing a fixed $k$.
limitations: limitations: Limitations have been adequately addressed in the paper.

Comments: title: Response
Thank you note to Reviewer 5h8i
Response to the rebuttal
Thanks for the post-rebuttal response
Thanks for the post-rebuttal response
Thank you for your constructive rebuttal.
comment: I apologize for responding late and thank authors for a detailed response. Authors have answered my questions in full and suggested reasonable updates to the final version of the paper. I am keeping my score as is (7), which is to say, I recommend that the paper gets accepted.
We sincerely thank you for your engagement and kind acknowledgment of our efforts to address your insightful comments. We are thrilled that your questions have been answered, and we deeply appreciate the increased score.
Thank the authors for the detailed responses!

I think my concerns are addressed, I have raised my score accordingly. 

Thanks!
We are grateful for your efforts in engaging with our work and your support in raising your score. Your assessment has been invaluable in refining our work and clarifying the key aspects of our research. We will further improve the experiments related to random prefix tokens and clarify that WISE can perform edits sequentially in the updated version of the manuscript.

Thank you once again for your time and consideration.
Many thanks for your time, effort, and questions! We greatly appreciate your recognition of our work and are pleased to hear that we have addressed your concerns.
I appreciate the authors for their detailed response. I would like to bump my rating to 7. Congratulations on your fine work!
The reviewer is satisfied with the authors' detailed rebuttal. I have no remaining concerns and will keep the score.