Edit Less, Achieve More: Dynamic Sparse Neuron Masking for Lifelong Knowledge Editing in LLMs
Authors: Jinzhe Liu
Junshu Sun
Shufan Shen
Chenxue Yang
Shuhui Wang
Venue: NeurIPS 2025 poster ()
Keywords: Lifelong model editing
fine-grained editing
knowledge editing
large language models
Primary area: deep_learning
TL;DR: NMKE is a fine-grained framework that uses neuron-level attribution and sparse masking for precise, efficient lifelong knowledge editing.

Abstract: Lifelong knowledge editing enables continuous, precise updates to outdated knowledge in large language models (LLMs) without computationally expensive full retraining. However, existing methods often accumulate errors throughout the editing process, causing a gradual decline in both editing accuracy and generalization.  To tackle this problem, we propose Neuron-Specific Masked Knowledge Editing (NMKE), a novel fine-grained editing framework that combines neuron-level attribution with dynamic sparse masking. 
Leveraging neuron functional attribution, we identify two key types of knowledge neurons, with knowledge-general neurons activating consistently across prompts and knowledge-specific neurons activating to specific prompts.
NMKE further introduces an entropy-guided dynamic sparse mask, locating relevant neurons to the target knowledge. This strategy enables precise neuron-level knowledge editing with fewer parameter modifications.
Experimental results from thousands of sequential edits demonstrate that NMKE outperforms existing methods in maintaining high editing success rates and preserving model general capabilities in lifelong editing.

Edit Less, Achieve More: Dynamic Sparse Neuron Masking for Lifelong Knowledge Editing in LLMs
Jinzhe Liu 1.2
Junshu Sun 1.2
Shufan Shen 1.2 Chenxue Yang2 Shuhui Wang
State Key Lab. of Al Safety, Institute of Computing Technology. Chinese Academy Sciences
University of Chinese Academy of Sciences *Agriculture Information Institute, CAAS (liujinzhe23b, sunjunshu21s, shenshufan222, wangshuhui) @iitt. ac. cn yangchenxue@caas.cr
Abstract
Lifelong knowledge editing enables continuous, precise updates outdated knowl- edge in large language models (LLMs) without computationally expensive full retraining. However, existing methods often accumulate errors throughout the edit- ing process, causing gradual decline in both editing accuracy and generalization. To tackle this problem, we propose Neuron-Specific Masked Knowledge Editing (NMKE), novel fine-grained editing framework that combines neuron-level at- tribution with dynamic sparse masking. Leveraging neuron functional attribution, we identify two key types of knowledge neurons, with knowledge-genne generaleerrllre neurons activating consistently across prompts and knowledge-speciffe neurons activat- ing specific prompts. NMKE further introduces an entropy-guided dynamic sparse mask, locating relevant neurons to the target knowledge. This strategy enables precise neuron-level knowledge editing with fewer parameter modifica- tions, Experimental results from thousands of scquential edits demonstrate that NMKE outperforms existing methods in maintaining high editing success rates and preserving model general capabilities in lifelong editing. Codes are provided https://github.comm/ www.s//.///oo//ooi/o.... KeeprooieKeeggii//NMMKEE
Introduction
Lifelong model editing has emerged effective paradigm for maintaining and iterating modern large language models (LLMs), which enables continual and dynamic knowledge injection, error correction, and sensitive content removal [1, 2, 4J. For example, model editing can rectify outdated knowledge in LLMs without complete retraining, such as updating the year of the next Olympic Games from 2024 2028. An ideal lifelong knowledge editing method must simultaneously maintain high editing accuracy while preserving the model's 's general capabilities [3, 5]. Previous knowledge editing methods fall into two fundamental categories: those that integrate external parameters [5, 6, 7, 8, 9] and those that directly modify internal model parameters [10, 11, 12, 13, 14, 15]. External parameters methods demonstrate strong generalization, but suffer from escalating resource overheads and progressively declining editing accuracy as the number of edits increases [16]. In contrast, internal parameter methods, which typically follow a locate-then-edi paradigm [17], offer enhanced interpretability and a simpler architecture [18], yet are prone to degradation in general capabilities [15, 19]. More critically, both types methods face fundamental limitations in lifelong editing scenarios, where error accumulation compounds exponentially with successive editing operations [19, 20, 21]. This creates a compelling research challenge: how to design method that harmonizes the strong generalization of external approaches with the efficiency of internal methods, while enabling robust lifelong editing without performance degradation.
Corresponniig author.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Layer-level ROME:MEMITAAjoaaccii
Neuron-level NMKE
Editablity
00.0
MLP
MI.P
50.0
O D
Accuracy(%)
MLP,
MLP
600
Alphaadi
MLP
MLP,
00:00
MLP-2
MLP,
590
1000
1500
strps
2000
3000
50000
Avg. General Capability
00.0
When the Olympic Games scheduled?
When next Olympic Games hosted?
Assmmcy
40:00
When is the date of the next Olympic Games?
NMKE Alphzedi
20.04
(next Olympic Gomes, date, 2024-20028)
Unodited Nouron
Edited Neuron
500
1000
1500
2000
3000
5000
stops
Figure 1: Left: Comparison of layer-level editing with our neuron-level editing method, NMKE, highlighting the impact Multilayer Perceptron (MLP) layers. Right: Average editability score (including edit success rate [23], generalization success rate and localization success rate [3]) and average general capability (across MMLU [24]. GSM8K [25], CommonsenseQA [26]. and BBH- Zeroshot [27]) across different edit steps on LLaMA3-8P ---ntrructt [28].
Motivated by this challenge, we focus on internal editing methods to ensure simplified architecture, aiming to achieve high editing accuracy while preserving the model's 's general capabilities. Previous internal methods have explored various localization strategies such as causal tracing [13], multi-layer updates [14], and null-space projection [15] to identify factual knowledge within model parameters. However, these strategies apply modifications
the level of entire layers parameter blocks, inadvertently affecting neurons unrelated to the target knowledge and consequently causing model forgetting and capability collapsing [15, 22), as shown in Figure 1.
In this paper. we propose Neuron-specific Masked Knowledge Editing (NMKE), a novel fine- grained knowledge editing framework that performs neuron-level attribution and constructs dynamic sparse masks to precisely modify LLMs knowledge. Unlike previous coarse-graine boarse-grained approaches, NMKE employs neuron-level attribution in the feedforward network (FFN) quantify precisely how individual neurons contribute to knowledge-bawce predictions. Through this attribution analysis, we discover two distinct types of showledge-encodeee exhibit stable activation across prompts and encode generalizable information, and (ii) Knowledge- Specific Neurons, which selectively activated capture semantic variations in specific contexts. Leveraging this insight, NMKE also constructs dynamic sparse masks, selectively focusing on the most relevant subset of neurons associated with the target knowledge. Comprehensive empirical evaluation demonstrates that NMKE achieves an optimal balance in lifelong editing by making minimal-cost parameter modifications, ensuring accurate edits, and preserving general capabilities. Our research makes the following key contributions:
neurons: (i) Knowledge Geneeaa Neurons, which
We experimentally show that the performance degradation lifelong editing is driven by the cumulative disruption of neurons caused by coarse-grained parameter updates. Building on this analysis, we identify the critical roles of Knowledge- General and Knowledge-Specte neurons in storing factual knowledge within the model.
We introduce Neuron-Specifie Masked Knowledge Editing (NMKE), a novel fine-grained editing framework that utilizes neuron-level attribution and dynamic sparse masking to precisely target and modify only those neurons most relevant to the updated knowledge, significantly reducing unintended disruption to the model.

---

Through rigorous evaluation on thousands of sequential edits, we show that NMKE consistently outperforms existing methods in maintaining high editing success rates while simultaneously preserving model capabilities in lifelong editing scenarios.
2 Preliminaries
2.1 Lifelong Model Editing
Lifelong model editing enables continuous knowledge updates in pretrained language models, main- taining accuracy and correcting errors without the need for full retraining, Formally, let fo : X' + y be language model with parameters 0. In the sequential editing process, the 1-th editing step receives an editing request set S. = {(s,r, o - o*)}, indicating that the fact triple (s, r, o) should be updated to (s, rrro o*). Here, 8 represents the subject, the relation, and o the original and target object, respectively. The model is updated via an editing function E, such that g(*) (S,,, ((S,,0((-1)) ((t-1)). where the objective that for all (s,r, r,
o²) St, the edited model satisfies Sece (8, Simultaneously, for unedited inputs x & U. St, the model output should remain as similar as possible to their previous values fgro (1) Ra ())e-1)(zr). The central challenge in lifelong model editing is to achieve high editing accuracy while minimizing interference with unedited knowledge.
o².
2.2 Knowledge Storage in Transformers
Knowledge in Transformers [29] is primarily encoded within the FFN, which functions as distributed key-value associative memories [12, 30, 31J. Formally, an FFN layer with input weights Win Rd xd and output weights Wout E Rdxddm transforms an input vector x E I2 through the forward propagation: y W o (Wi**)) + x, where o--) represents the non-linear activation function. and dm denotes the dimensionality of the hidden layer. Each neuron in the FFN can be considered a key-value unit, where the i-th neuron associates with the key (())_ = w and the value (() W When the input aligns with ((),, the neuron is strongly activated, and the corresponding value v() contributes significantly to the output. The FFN can be reformulated in a key-value form:
+x= Σ
x)v(() +x.
y
where s(1) represents the contribution of the i-th neuron to the output. This decomposition shows that individual neurons collectively encode factual knowledge, but multiple factual associations often share overlapping parameter subsets. Conventional approaches that modify entire parameter blocks or layers during knowledge editing lead to interference with unrelated representations. In lifelong editing, this interference accumulates, degrading model performance and exacerbating distribution shift. Our key insight is that to maintain editing stability and preserve general capabilities, parameter updates should target neural subspaces directly relevant the edited knowledge, rather than applying coarse updates to entire layers or modules.
Neuron-specific Masked Knowledge Editing
This section introduces NMKE, fine-grained framework for knowledge editing that leverages neuron- level attribution and constructs dynamic sparse masks. $.11 introduces the neuron- level attribution method, where the neuron activations and masking behavior reveal the functional differences among knowledge neurons and identify the knowledge- gendrall and knowledge-speciff neurons. $322 introduces entropy-base based dynamic masking strategy select minimal intervention subsets.
3.1 Neuron Attribution
Neuron-level Attribution. bution method. The contribution of each neuron to the predicted token is quantified by the increase in log probability when its activation is perturbed. Specifically, (t) in Eq. I can be regarded as the perturbation from the i-th neuron to the input x. We formulate the perturbing process as:
Inspired by [32], we estimate neuron importance using a static attri-
x
x + (1)
(2)
3

---

College Biology Teak
Masked
575
neunnne (headine)
Asswer: C
1417
Cleniity tetrous
life?
Blas
(oillegy hislgyy
Vasic neaces
A) Alom
B) Malecule
Answer <Ia0A>
Dennin-pezile acurosi (Hillays
3 Gell
Kanslalge gieazz aerraw
Dy Organ
Musice
giith
Answer A
33.35
156
10 50
110
501
10
50
10
50
MadGed Neioon Nars.
Figure 2: Left: The effect neuron masking on multiple-choice task performance, Right: Perfor- mance changes following the masking of different neuron types on the MMLU sub-task, with general neuron masking leading substantial performance decline, particularly in college biology accuracy, whereas
neurons cause minimal degradation. represents the accuracy drop, with the magnitude shown as a percentage.
where À is the amplification factor to scale the perturbation. We then project both x and x()) into the vocabulary space using the unembedding matrix E,, and then compute the softmax logits, giving the importance score of neuron i as the log- log-probability gain of the target token
Imp(*) (6) log ply x (*) - log p(y
x) = sof tonttmax(E,,xx
sof tmax (E,x).
In practice, the importance scores are computed in batches for each editing prompt, yielding a neuron importance matrix I Rnnd d with n denoting the batch size. This matrix used to build sparse masks (S 3.2) that select key neurons for editing, enabling precise and www-intrfference model updates.
Functional Roles of Neurons. To explore the reasons behind capability degradation in lifelong knowledge editing, we conduct attribution experiments on the LLaMA2-7B [33] model. Specifically, we analyze neuron activations in the FFN layers Transformer blocks across tasks from the MMLU dataset [24], focusing on college and high school level biology and chemistry. Based on cross-task activation patterns, FFN neurons
be categorized into three types: knowledge-genert neurons activated across all tasks, domain-specific neurons activated within a subject, and task-specific neurons activated only in one task.
We observe that knowledge- gengeaa neurons occur most frequently, followed by Nomain-speccife neurons, with task-specific neurons being the least common. To assess the functional roles of different neuron types, we conduct masking experiments by ablating the top-10 and top-50 neurons with the highest attribution scores for each category and evaluating performance changes MMLU subtasks. As shown in Figure 2, masking Knowledge-general neurons causes severe degradation where the model produces meaningless outputs (e.g. brackets, stop words, or corrupted symbols), with accuracy degrading from 37.5% to 4.17% with drop of 1 33.33% in the top-10 ablation and to 0.69% with 4.36881% % drop in the top-50 ablation. In contrast, masking task-specific neurons that correspond to the same domain leads to only minor drops (1 2.04% and 6.25%), while masking domain sppecifcc neurons that correspond knowledge in LLMs is not uniformly distributed but sparsely localized within a small set of neurons. More experimental details are provided in Appendix B.1.
different domain has negligible effect. These results suggest that
Building on these observations, we abstract the functional roles neurons into two categories to guide knowledge editing, i.e., knowledge-geneed neurons with stable activations across semantically similar prompts are defined as knowledge-geners neurons, while those with sharp, localized activations and high attribution scores in specific prompts are termed as knowledge-speciftic neurons. Knowledge-specifice neurons include domain-specific and task-specific neurons. Model editing each step should target the subset of the neurons, excluding irrelevant knowledge-speciff neurons.
and knowledge-specins neurons. Specifically,
3.2 Dynamic Sparse Masking
To locate relevant neurons for knowledge editing, we introduce a dynamic sparse masking mechanism that adaptively selects a subset of neurons that provide predominant contributions in representing target knowledge. Specifically, let I(]] Rax& be the neuron attribution matrix layer where the number of prompts and de the number of neurons. The main difference between the two types

---

When date the Olympic Games?
2024
Editing
2028
range of critical MLP layers
nex Oly Olympic Games, date, 2024-2028
atta
Neuron Attribution
subject mequrisad volue
Neuron-level Mask
Importance
Geoerd
Dynamic Sparse Mask Generation
Neuron Selection
score
Kaowielge-Speee Speciile----ieeeeee seurees
Misked seamees
L-2
L-1
Figure 3: Overview of NMKE. Neuron attribution and dynamic sparse masking to selectively update Knowledge-Genene and Knowledge-Specifie neurons, ensuring precise knowledge editing while preserving the model's general capabilities.
of neurons that Knowledge-geneee neurons demonstrate consistent activations across different prompts, while knowledge-speciite neurons demonstrate selective activations to specific prompts. By analyzing the distribution of attribution scores, we can distinguish between the two types of neurons.
3.2.1 Type-specific Selection Score
Knowledge-general Neurons, due their encoding of core knowledge, exhibit stable activations across diverse prompts, resulting in consistently positive attribution scores. In consequence, such neurons be identified by counting the number of positive attribution scores over multiple prompts:
gC Σ
> 0),
1=1
where [[[]] denotes the sign function. [[rue] and [[False]] 0.
Knowledge-speciffc Neurons, due their encoding task-specific knowledge, are activated on at least one prompt. resulting in significant attribution score a specific prompt. Therefore, such neurons can be identified by the maximum attribution across prompts:
max 3.8"
(5)
3.2.2 Type-specific Selection Ratio
To identify the subset of the knowledge-gemee general and knowledge-specifie neurons, we select a ratio of the neurons based on their scores pde and rPP However, given different batches of prompts, the neurons perform different activation statuses, and thus resulting in different attribution scores. Employing a fixed ratio for all prompt batches can incorporate irrelevant neurons when fewer neurons well-activated, and lose relevant neurons when most neurons are well-activated. To tackle this problem, we propose to dynamically assign the selection ratio based on the distribution of attribution scores.
Knowledge-general Neurons demonstrate stable activations across different prompts, indicating uniform distribution with higher entropy. Therefore, we can employ the average normalized entropy across prompts to evaluate the ratio of the knowledge-generall neurons:
X3
dc
Hge n log dj
J., log P
j.b
P softmax(ol!))
(6)
where the temperature a adjusts the score differences. The higher the average entropy Hyes the more neurons can be categorized knowledge-gemeralized..
5

---

Knowledge-specific Neurons demonstrate selective activation, indicating a large maximum attribu- tion score across prompts. When most neurons are categorized as knowledge-specifcc, the maximum attribution scores will be similarly large across different neurons and demonstrate large distribution entropy:
max, 1
Hap
qi log qi qi Σx max, T
log di
i=1
These entropy values indicate the ratio of the relevant neurons in different prompt batches. In practice, we further adjust the ratio by constant scalers age. asp and bias terms bge, baps giving Pge = Hgg age + Oge and Pep =
Hap aap + Dep.
Neuron Selection and Mask Generation. Based on the estimated ratio Pee. we compute the threshold Tga the (1 (ga) quantile of the score r5 and select neurons exceeding this threshold as the knowledge-general neurons. The selection of the sooledge--specffee neurons follows the same protocol with ratio Pep and score rsP, This selection constructs the binary sparse mask m(!) E {0, 1]
as:
1
Tge or
SP
Tepl.
(8)
This hybrid selection mechanism provides flexible control of the neuron-level editing by incorporating the functional roles of different neurons. By selectively updating neurons relevant to the edited knowledge, this approach enables precise editing while preserving the knowledge structure of irrelevant neurons. As shown in Figure 3, we propose a neuron-level knowledge editing framework, NMKE, that restricts updates to subset of functionally important MLP neurons, enabling precise and muinmmll-disruptvv knowledge injection. We follow AlphaEdit [15] to construct the optimization pipeline, where m is employed to mask the parameter update matrix.
4 Experiments
4.1 Experimental Setup
Evaluation Benchmarks. We evaluate our methods in terms of editing performance and the generalization ability of edited LLMs. For the knowledge editing performance, we utilize two standard benchmarks: 7sRE [34] for question answering. and CounterFact [13] for factual corrections. Following [6, 4], we report three key metrics: Rel. [23] (edit success), Gen. [3] (generalization to paraphrased prompts), and Loc. [3] (locality preservation). For the generalization ability, we adopt five downstream tasks that span mathematical reasoning, question answering, and code generation:
MMLU [24], GSM8K [25], CommonsenseQA [26], BBH-Zeroshot [27], and HumanEval [35].
Baselines. We compare NMKE with various of baselines that covers both external and inter- nal parameter methods, including Fine-Tume Tuning (FT) [36], KN [12], ROME [13], PMET [37], MEMIT [14], WISE [6], and AlphaEdit [15]. Evaluation is conducted on widely used LLMs including LLaMA3- 1--------Inae Instruct [28]. GPT2-XL [38]. and Qwen2.5-7B [39] across cumulative edit steps T E (10, 100, 500, 1000, 1500, 2000, 3000, 50003. More details are presented in Appendix A.
4.2 Does the Edited LLM Still Generalize?
Figure evaluates the generalization ability of LLaMM-3-SBB-nsttrue on the ZsRE and CounterFact datasets edited by various methods with 1 to 2000 editing steps. Overall, the results show that existing methods exhibit rapid degradation in generalization ability as the number of edits increases. FT destroys the generalization ability of LLMs on GSM8K and HumanEval after only 100 edits, while methods like ROME and MEMIT show significant performance drops beyond T 500. As the SOTA method, AlphaEdit suffers from gradual decline in generalization due to the continuous modification of irrelevant knowledge neurons during layer-level edits. For example, at T 1500, AlphaEdit shows significant decline in question answering performance, with nearly a complete loss of mathematical and coding abilities, as the accuracy on both GSM8K and HumanEval drops to 0. In contrast, our NMKE effectively maintains the generalization of LLMs in lifelong editing on both datasets and significantly outperforms baselines with 7 > 1000. The superior performance of NMKE can be attributed to its dynamic sparse masking mechanism, which confines the edited

---

FT
KN
ROME
MEMIT
PMET
AlphaEdit
NMKE
MMLU--ZRE
(b) GSMBK-ZZRR
B8iff-Zeeoduu-248
0441
S
Aceuncy
12 12
(+) MMLL--CCoeerFFee
(<) Commintternee---ooaaae
(d)
(e) (Hniennneelnnnnteeaet
0.
0.7
ACCHIOCY
Actumey
Accuncy
04
1023 007
500
1000
15:00
20.00
250
1540
3000
0
300
32.0
1540
2003
300
1000
12:0
20.0
533
10:30
1940
202
Says
Stegs
Steps
Suye
Stegs
Figure 4: Generalization performance of LLaMA-3--88 M--MMM----B-Inttutee Instruct on MMLU [24], GSM8K [25], CommonsenseQA [26], BBH-Zeroshot [27], and HumanEval [35] benchmarks after 2000 steps of sequential editing on the ZsRE (top) and CounterFact (bottom) datasets.
Table I: Main editing results (LLaMA3-BB-Instruct) across 2000 editing steps on ZsRE and Counter- Fact datasets. The best result is in bold, and the second-best result is underlined.
ZiRE
Method
7'
10
7'
100
J'=
500
T' = 1000
T' F== 1500
T' 2000
Rel.
Gen.
Loce
Rel.
Ginn
Loc.
Rel.
Gien.
Loc:
Rel.
Gen.
Loc.
Rel.
Gion.
Loc.
Rel.
Gen.
L.c.
0.18
0.03
0.01
0.17
0.13
0.03
0.12
0.11
0.13
0.10
0,02
0.10
0.02
0.07
0.01
KN
0.14
0.12
0.66
0,02
0.01
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
ROME
0.96
0.94
0.64
0.10
0.09
0.03
0.01
0.01
0.02
0.02
0.01
0.02
0.04
0.04
0.02
0.01
0.01
0.02
0.89
0.01
0.01
0.04
0.04
0.03
PMET
0.39
0.39
0.91
0.02
0.02
0.05
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
WISE
0.83
0.78
0.71
0.67
0.16
0.15
0.41
0.39
0.32
0.31
0.37
0,36
AlphaEdii
0.98
0.97
0.99
0.86
0.96
0.87
0.71
0.93
0.84
0.58
0.62
0.54
0.14
0.32
0.06
NMKE (Ours)
0.90
0.95
0.86
0.96
0.87
0.95
0.85
0.77
0.86
0.74
0.94
0.85
0.71
CounterFoct
Method
T' 10
T' = 100
= 500
T' 1000
T' 1500
T' 2000
Rel.
Gen.
Loc.
Rel.
Gen.
Loc.
Rel.
Genn
Loc:
Rel.
Gen.
Loc.
Rel.
Gen.
Loc.
Rel.
Gen.
Loc.
0.00
0.00
0.07
0.00
0.07
KN
0.00
0.01
0.87
0.03
0.03
0.79
0.01
0.01
0.66
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
ROME
0.90
0.70
0.50
0.23
0.13
0.01
0.01
0.01
0.00
0.01
0.00
0.00
0.01
0.00
0.00
0.00
0.00
0.00
MEMIT
0.70
0.76
0.08
0.06
0.00
0.00
0.01
0.00
0.00
0.01
0.00
0.01
0.00
0.00
PMET
0.10
0.20
0.98
0.10
0.05
0.91
0.14
0.06
0.76
0.06
0.02
0.60
0.02
0.01
0.42
0.02
0.35
AlphaEdit
1,00
0.60
0.85
0.99
0.50
0.71
0.99
0.75
0.14
0.99
0.76
0.32
0.71
0.55
0.19
0.22
0.13
0.04
NMKE (Ours)
0.92
0.51
0.86
0.97
0.56
0.80
0.99
0.58
0.60
0.99
0.65
0.50
0.98
0.65
0.43
0.98
0.67
0.38
parameters and minimizes disruption the model's internal representations. Notably, after 5000 editing steps, MMLU remains at 0.59, with further results in Appendix B.2.
4.3 Do Sparser Neuron Modifications Lead to Better Edits?
Table presents the results of lifelong knowledge editing on 2000 randomly sampled 7sRE and CounterFact examples, edited sequentially with a batch size of using LLaMA3-8B-Instruc Most methods (e.g., FT, KN, ROME, and MEMIT) exhibit significant performance degradation after T 100 in lifelong editing. WISE adds external modules without modifying internal parameters, with locality indicated by a "-", but its knowledge editing success rate decreases by J 0.46 after 2000 edits. AlphaEdit's 's performance becomes unstable during sequential editing, with catastrophic forgetting observed at T > 1500, as evidenced by a decline in editing success accuracy of J. 0.67 on ZsRE and 1. 0.78 on CounterFact. In contrast, NMKE demonstrates superior lifelong knowledge editing capabilities on both datasets. This robustness stems from our fine-grained, neuron-level editing strategy, which precisely targets a relevant neuron subset, preventing model collapse. Results for scaling to 5000 are shown in Table 6, with additional experiments in Appendices B.2, B.4 and B.9.
7

---

Aphiod
(4) pre- pdeted L LLmaa3-- -B- IB-tuutr
(b) Alphaedii (T-2000) on LLamm33 3.B Instrrct
(c) NMKE (T- (T-2000) on L. LLama3- SB Instract
Figure 5: Visualization of the distributional shift of down-projection weights in the 8th MLP layer for (a) the pre-edited LLMMA-3-8B- Instruct, (b) AlphaEdit, and (c) NMKE after 2000 sequential edits.
Knowlolae- Speciffi neuross
only Knowlolae- aecsons
Knowlelaz Generul neweons 6.35
Knowlsdae
18.30,
Knowiedgs Specitic semors: 0.25
10:00
(U
0.5-
10.70
D
0.76
0722
0.005
0.02-
0.80m
0.80-
0.20
0.11
(100
0.75-
0 75
1060
0.20
10.900
0.33
0.70
0.70
0.25
Gen.
MMLU
GSMBK
CommonsereROA
BBil-Zenshol
Humanival
Edit Capability
General Capubilit
Figure 6: Editing and generalization performance after T-2000 sequential edits under various masking strategies.
4.4 What Happens LLM Internal Parameters?
To analyze the impact of different methods on the model's internal parameters, we visualize the distributional shift of down-projection weights in the 8-th FFN layer using t-SNE [40]. We compare the pre-edited L.LLMMA--B-IInste after 2000 sequential edits on ZsRE. As shown in Figure 5, the pre-edited LLaMA3-8B- model (Figure (a)) exhibits compact weight distribution, indicating stable parameter structure. AlphaEdii (Figure (b)) induces a noticeable deviation from the original weight distribution, resulting in more dispersed and distorted geometry in the t-SNE space, suggesting substantial distributional shifts. In comparison, NMKE (Figure (c)) maintains a more compact structure, closely aligned with the original distribution, reflecting minimal parameter shifts. These results suggest that NMKE induces fewer disruptions to the model's s internal parameters than AlphaEdit, which explains its superior editing stability and reduced interference, as observed in $ 4.2 and $ 4.3. Additional analyses of LLM internal parameters are presented in Appendices B.3, B.5, and B.S.
model with its versions edited by AlphaEdit and NMKE, each
4.5 Further Analysis
Effects of Neuron Selection Strategies. Figure 6 compares four neuron selection strategies: activating only (i) knowledge-generas or (ii) knowledge-specific neurons, and activating both types with (iūi) fixed ratio or (iv) entropy-based dynamic ratio. Results show that the entropy- dynamic ratio achieves the highest editing success rate and best generalization, benefiting from its dynamic neuron activation based different knowledge types. Notably, activating only knowledge-gent general neurons excels in locality preservation and retention of coding abilities. Overall, the four neuron selection strategies within the NMKE framework effectively preserve strong general capabilities after 2000 editing steps. Among these, the entropy-based dynamic ratio activation achieves the best balance between enhancing knowledge editing performance and maintaining general capabilities.
Hyperparameter Analysis for NMKE. As shown in Figure 7 (a), in order to investigate the impact of constant scalers age: aep and bias terms bge. dep on the proportion of neuron activation. we adjust the constant scalers age {0.1, 0.5, 1.0} and bias terms bge E {0.2, 0.3, 0.4) to explore their effects on knowledge editing performance. The results show that higher bias terms enhance neurons, crucial for stable, generalizable knowledge, leading to more precise updates. In contrast, larger constant scalers broaden neuron activation, which may select unnecessary neurons and increase interference. This approach effectively targets knowledge neurons,
the selection of knowledge-genert
8

---

140
Rel.
Gen.
Loc.
0.5
Ausi (000)
0.90
0,80
Aocuncy
0.65
6,70
0.50
0.75
0,50
0gr 0.2
4gx 0.1
0.1
Pge 0.3
Dpe 0.
01
Dye
Aye 02
4ge 1.0
1000
2000
5000
4000
5000
stops
6ge
(a)
(b)
Figure 7: (a) Effect of constant scalers and bias terms on neuron selection. The red and green lines show that higher bias terms improve editing accuracy, while * indicates that larger constant scalers reduce locality preservation. (b) Sensitivity of neuron editing to amplification factor X.
Table 2: Per-edit runtime and editing performance in sequential editing.
Method
Step Time (s)
1000
T
2000
per edit
Rel.
Gen.
Loc.
Rel.
Gen.
Loc.
MEMIT
16.83
0.04
0.04
0.03
0.04
0.03
0.03
WISE
26,49
0.41
0.39
0.32
0.31
AlphaEdit
22.16
0.93
0.84
0.58
0.62
0.54
0.14
NMKE (MPC)
22.25
0.94
0.82
0.81
0.93
0.83
0.77
NMKE (PSA)
29.67
0.95
0.84
0.79
0.94
0.86
0.71
NMKE (LPS)
30.42
0.95
0.85
0.77
0.94
0.86
0.74
ensuring precise selection and preserving the model's capabilities while minimizing disruption. In Figure (b), we analyzed the impact of amplification factors 10 and X 30 on knowledge editing performance. The results show that moderate increase in the amplification factor helps prioritize neurons most relevant to the target knowledge, improving editing efficiency and reducing disturbances. Additional experimental results are presented in Appendices B. and B.6.
Editing Efficiency. To quantify attribution cost, we report per-edit runtime under sequential editing across several competitive baselines and NMKE configured with three attribution strategies: MLP Projection Coefficient (MPC) [30], Probability Shift Attribution (PSA) [32], and Log Probability Shift (LPS) [32]. Table shows that NMKE (MPC) is the most efficient variant, with only 0.2 of per-edit overhead. while the highest-performing variant. NMKE (LPS), adds ~ 8 edit but remains practical and delivers significant improvements. Overall, NMKE is flexible framework that supports attribution methods of varying complexity, balancing quality and efficiency under different scenarios. More analysis are provided Appendix B.7.
Effect of Overlapping Neurons. To investigate the effect of overlapping neurons, we perform edits using only overlapping neurons or only non-overlapping neurons. Table shows that overlapping neurons mediate the trade-off between edit success and locality. Editing with overlapping neurons yields the strongest locality, with moderate drops in reliability and generalization. Editing with non-overlapping neurons increases edit success and generalization but weakens locality. NMKE combines the two via an entropy-guided ratio and achieves the best overall balance at 7 = 1000 and T 2000, supporting its stability under sequential editing. Further experiments knowledge neuron distributions are reported Appendix B.6.
Neuron Masking Analysis. To assess the effectiveness of discretized neuron update mechanism, we compare binary masking with soft mask alternative. The NMKE (soft-mask) variant scales each neuron's update magnitude by its attribution score, preserving the continuity of neuron states. As shown in Table 4, the soft mask largely matches NMKE in accuracy and generalization, but its dense and indiscriminate updates degrade locality. contrast, NMKE constrains updates to sparse, Nowledge-awarr neuron subset, thereby preserving stability and achieving robust locality.

---

Table 3: Functional analysis of overlapping neurons in sequential editing.
Method
T = 1000
2000
Rel.
Gen.
Loc.
Rel.
Gen.
Loc.
Overlapping Neurons Only
0.72
0.60
0.84
0.75
0.63
0.80
Non-Overlapping Neurons Only
0.82
0.70
0.82
0.85
0.74
0.78
NMKE
0.95
0.85
0.77
0.94
0.86
0.74
Table 4: Effectiveness analysis of neuron masking mechanisms.
Method
T = 1000
T = 2000
Rel.
Gen.
Loc.
Rel.
Gen.
Loc.
NMKE (soft-mask)
0.96
0.87
0.67
0.72
0.61
0.19
NMKE
0.95
0.85
0.77
0.94
0.86
0.74
5
Related Work
External Parameter Editing. External parameter editing methods are widely adopted for their structural flexibility and ease of implementation [4, 3, 41, 42]. Early approaches like SERAC [7] employ cached counterfactual models with scope classifiers for relevance-based routing. To improve flexibility, GRACE [5] introduces discrete key-value adaptors for lifelong editing, while Melo [9] proposes semantie-clustered low-rank adaptation modules. ATBias [43] shifts towards in-context editing by biasing key entity tokens during decoding, and WISE [6] enhances the paradigm with dual-memory architectures and trainable routers to isolate edited knowledge. Despite these advances, these methods overlook dependencies among factual updates. To address this, recent methods [44, 45] incorporate knowledge graphs with graph neural networks [46, 47, 48, 49, 50, 51] to capture dependencies between related edits. Nevertheless, external methods that rely on auxiliary modules still suffer from growing storage and routing bottlenecks with more edits, degrading accuracy and efficiency.
Internal Parameter Editing. Internal parameter editing approaches directly modify model weights through constrained updates. Early meta-learning methods [11, 10] leverage hypernetworks to generate task-specific parameter updates, but face limitations in scalability when handling sequential edits due to extensive retraining requirements. F-Learning [52] adopts two-stage forgetting-before learning fine-tuning paradigm for knowledge updating. The locate-then-edit paradigm [17] has significantly advanced internal editing, evolving from single-layer approaches like ROME [13],
which targets specific key-value pairs in transformer layers, to multi-layer methods like MEMIT [14]. which distributes updates across multiple layers for enhanced robustness. Furthermore, AlphaEdit [15] employs null-space projection to reduce distribution shift. These approaches operate at the layer or parameter-bloce level, risking disruption of unrelated neurons and causing catastrophic forgetting and performance degradation in lifelong editing [2, 19]. FiNE [53] performs fine- lnn-grained editing via contribution scoring, without explicitly modeling the functional roles of knowledge neurons. Motivated by this, we propose NMKE, which performs fine grainedi editing of knowledge neurons via sparse masking, achieving stable lifelong editing while preserving generalization.
6 Conclusion
This paper introduces Neuron-specific Masked Knowledge Editing (NMKE), fine-grained frame- work for knowledge editing that leverages neuron-level attribution and dynamic sparse masking to enable precise editing. By identifying and targeting both knowledge-genede general and knowledge-spechie neurons, NMKE confines updates relevant neural subsets, effectively minimizing interference. Experiments demonstrate that NMKE outperforms existing methods in edit success and generalization retention, particularly in lifelong editing scenarios involving thousands of edits, where other methods experience cumulative degradation. This superior performance arises from our neuron-level inter- vention approach, which ensures successful edits while preserving general capabilities, presenting promising solution for lifelong editing. For limitations and discussion, please refer to Appendix C.
10

---

Acknowledgments and Disclosure of Funding
This work was supported in part by the National Key R&D Program of China under Grant 2023YFC2508704, and in part by the National Natural Science Foundation of China 62236008. The authors would like to thank the anonymous reviewers for their constructive comments and suggestions that improved this manuscript.
References
[1] Song Wang, Yaochen Zhu, Haochen Liu, Zaiyi Zheng, Chen Chen, and Jundong Li. Knowledge editing for large language models: A survey. ACM Computing Surveys, 57(3):1-37, 2024.
[2] Yunzhi Yao. Peng Wang, Bozhong Tian, Siyuan Cheng, Zhoubo Li, Shumin Deng, Huajun Chen, and Ningyu Zhang. Editing large language models: Problems, methods, and opportunities. In Houda Bouamor, Juan Pino, and Kalika Bali, editors, Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 10222-10240, Singapore, 2023. Association for Computational Linguistics.
[3] Ningyu Zhang. Yunzhi Yao, Bozhong Tian, Peng Wang, Shumin Deng, Mengru Wang. Zekun Xi, Shengyu Mao, Jintian Zhang. Yuansheng Ni, et al. A comprehensive study of knowledge editing for large language models. ArXiv preprint, 025/2401.01280 2024.
[4] Peng Wang. Ningyu Zhang, Bozhong Tian, Zekun Xi, Yunzhi Yao, Ziwen Xu, Mengru Wang, Shengyu Mao, Xiaohan Wang, Siyuan Cheng, et al. Easyedit: An easy-to-us knowledge editing framework for large language models. ArXiv preprint, 0tts/2308.02266. 2023.
[5] Tom Hartvigsen, Swami Sankaranarayanan, Hamid Palangi, Yoon Kim, and Marzyeh Ghassemi. Aging with GRACE: lifelong model editing with discrete key-value adaptors. In Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine, editors, Advances in Neural Information Processing Systems 36: Annual Conference on Neural Infor- mation Processing Systems 2023, NeurlPS 2023, New Orleans, LA, USA, December 10 16. 2023, 2023.
[6] Peng Wang, Zexi Li, Ningyu Zhang, Ziwen Xu, Yunzhi Yao, Yong Jiang, Pengjun Xie, Fei Huang, and Huajun Chen. Wise: Rethinking the knowledge memory for lifelong model editing of large language models. Advances in Neural information Processing Systems, 37:53764- 53797. 2024.
[7] Eric Mitchell, Charles Lin, Antoine Bosselut, Christopher D. Manning, and Chelsea Finn. Memory- based model editing at scale. In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvári, Gang Niu, and Sivan Sabato, editors, International Conference on Machine Learning, ICML 2022, 17-23 July 2022, Baltimore, Maryland, USA, volume 162 of Proceedings of Machine Learning Research, pages 15817-15831. PMLR, 2022.
[8] Ce Zheng, Lei Li, Qingxiu Dong, Yuxuan Fan, Zhiyong Wu, Jingjing Xu, and Baobao Chang. Can we edit factual knowledge by in-context learning? In Houda Bouamor, Juan Pino, and Kalika Bali, editors, Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 1862-4876, Singapore, 2023. Association for Computational Linguistics.
[9]
Lang Yu, Qin Chen, Jie Zhou, and Liang He. MELO: enhancing model editing with neuron- indexed dynamic lora. In Michael J. Wooldridge, Jennifer G. Dy, and Sriraam Natarajan, editors, Thirty-Eighth AAAI Conference on Artificial Intelligence, AAAI 2024, Thirty-Sixth Conference on Innovative Applications of Artificial Intelligence, IAAI 2024, Fourteenth Symposium on Educational Advances in Artificial Intelligence, EAAI 2014, February 20-27, 2024, Vancouver, Canada, pages 19449-19457. AAAI Press, 2024.
[10] Nicola De Cao, Wilker Aziz, and Ivan Titov. Editing factual knowledge in language models. In Marie-Francine Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih, editors, Pro- ceedings of the 202/ Conference on Empirical Methods in Natural Language Processing, pages 6491-6506, Online and Punta Cana, Dominican Republic, 2021. Association for Computational Linguistics.
11

---

[11] Eric Mitchell, Charles Lin, Antoine Bosselut, Chelsea Finn, and Christopher D. Manning. Fast model editing at scale. In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022. OpenReview.nee 2022.
[12] Damai Dai, Li Dong, Yaru Hao, Zhifang Sui, Baobao Chang, and Furu Wei. Knowledge neurons in pretrained transformers. In Smaranda Muresan, Preslav Nakov, and Aline Villavicencio, editors, Proceedings of the 60th Annual Meeting of the Association for Computational Lin- guistics ( (Volume 1: Long Papers), pages 8493-8502, Dublin, Ireland, 2022. Association for Computational Linguistics.
[13) Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. Locating and editing factual associations in GPT. In Sanmi Koyejo, S. Mohamed, A. Agarwal, Danielle Belgrave, K. Cho, and A. Oh, editors, Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurlPS 2022, New Orleans, LA, USA, November 28- December 9, 2022, 2022.
[14] Kevin Meng. Arnab Sen Sharma, Alex J. Andonian, Yonatan Belinkov, and David Bau. Mass- editing memory in a transformer. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.net, 2023.
[15] Junfeng Fang. Houcheng Jiang, Kun Wang, Yunshan Ma, Shi Jie, Xiang Wang, Xiangnan He, and Tat-Seng Chua. Alphaedit: Null-space constrained knowledge editing for language models. ArXiv preprint, abs/2410.02355, 2024.
[16] Lukas Thede, Karsten Roth, Matthias Bethge, Zeynep Akata, and Tom Hartvigsen. Under- standing the limits of lifelong knowledge editing in Ilms. ArXiv preprint, abs/2503.05683, 2025.
[17] Kevin Meng. David Bau, Alex Andonian, and Yonatan Belinkov. Locating and editing factual associations in GPT. In Sanmi Koyejo, S. Mohamed, A. Agarwal, Danielle Belgrave, K. Cho, and A. Oh, editors, Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurIPS 2022, New Orleans, LA, USA, November 28- December 2022, 2022.
[18] Eric Mitchell, Charles Lin, Antoine Bosselut, Chelsea Finn, and Christopher D. Manning. Fast model editing at scale. In The Tenth International Conference Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022. OpenReview.net, 2022.
[19] Qi Li, Xiang Liu, Zhenheng Tang, Peijie Dong, Zeyu Li, Xinglin Pan, and Xiaowen Chu. Should we really edit language models? on the evaluation of edited language models. ArXiv preprint, abs/2410.1878. 2024.
[20] Xinbei Ma, Tianjie Ju, Jiyang Qiu, Zhuosheng Zhang, Hai Zhao, Lifeng Liu, and Yulong Wang. On the robustness of editing large language models. ArXiv preprint, abs/2402.008277, 2024.
[21] Akshat Gupta, Sidharth Baskaran, and Gopala Anumanchipalli. Rebuilding rome: Resolving model collapse during sequential model editing. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 21738-21 21744, 2024.
[22] Peter Hase, Mohit Bansal, Been Kim, and Asma Ghandeharioun. Does localization inform editing? surprising differences in causality- -based localization vs. knowledge editing in language models. Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine, editors, Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 16, 2023, 2023.
[23] Tom Hartvigsen, Swami Sankaranarayanan, Hamid Palangi, Yoon Kim, and Marzyeh Ghassemi. Aging with GRACE: lifelong model editing with discrete key-value adaptors. In Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine, editors, Advances in Neural Information Processing Systems 36: Annual Conference on Neural Infor- mation Processing Systems 2023, NewrIPS 2023, New Orleans, LA, USA, December 10 16, 2023, 2023.
12

---

[24] Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding, In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net, 2021.
[25] Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. Training verifiers to solve math word problems. ArXiv preprint, abs/21 11.111..11.. 14168, 2021.
[26] Alon Talmor, Jonathan Herzig, Nicholas Lourie, and Jonathan Berant. CommonsenseeA: A question answering challenge targeting commonsense knowledge. In Jill Burstein, Christy Doran, and Thamar Solorio, editors, Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Vol- Minneapolis, Minnesota, 2019. Association
ume (Long and Short Papers), pages for Computational Linguistics.
[27] Mirac Suzgun, Nathan Scales, Nathanael Schärli, Sebastian Gehrmann, Yi Tay, Hyung Won Chung, Aakanksha Chowdhery, Quoc Le, Ed Chi, Denny Zhou, and Jason Wei. Challenging BIG-bench tasks and whether of-in-of-thought can solve them. In Anna Rogers, Jordan Boyd- Graber, and Naoaki Okazaki, editors, Findings of the Association for Computational Linguistics: Toronto, Canada, 2023. Association for Computational Lin-
ACL 2023, pages guistics.
[28] Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The llama 3 herd of models. ArXiv preprint, fbss2240.22188. 2024.
[29] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017.
[30] Mor Geva, Avi Caciularu, Kevin Wang, and Yoav Goldberg. Transformer feed-forward layers build predictions by promoting concepts in the vocabulary space. In Yoav Goldberg, Zornitsa Kozareva, and Yue Zhang, editors, Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 30-45, Abu Dhabi, United Arab Emirates, 2022. Association for Computational Linguistics.
[31] Mor Geva, Roei Schuster, Jonathan Berant, and Omer Levy. Transformer
layers are key-value memories. In Marie- Franncine Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih, editors, Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 5484-5495, Online and Punta Cana, Dominican Republic, 2021. Association for Computational Linguistics.
[32] Zeping Yu and Sophia Ananiadou. Neuron-level knowledge attribution in large language models. ArXiv preprint, abs/23 12. 12141, 2023.
[33] Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaci, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models. ArXiv preprint, abs/2307. 0092888, 2023.
[34] Omer Levy, Minjoon Seo, Eunsol Choi, and Luke Zettlemoyer. Zero-shot relation extraction via reading comprehension. In Roger Levy and Lucia Specia, editors, Proceedings of the s Conference on Computational Natural Language Learning (CoNLJ, 2017), pages 333-342, Vancouver, Canada, 2017, Association for Computational Linguistics.
[35] Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde De Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. Evaluating large language models trained on code. ArXiv preprint, 0as/2107.003774, 2021.
[36] Chen Zhu, Ankit Singh Rawat, Manzil Zaheer, Srinadh Bhojanapalli, Daliang Li, Felix Yu, and Sanjiv Kumar. Modifying memories in transformer models. ArXiv preprint, abs/2012.00363 2020.
13

---

[37] Xiaopeng Li, Shasha Li, Shezheng Song, Jing Yang, Jun Ma, and Jie Yu. PMET: precise model editing a transformer. In Michael Wooldridge, Jennifer G. Dy, and Sriraam Natarajan, editors, Thirty-Fighth AAAI Conference on Artificial Intelligence, AAAI 2024, Thirty- Sixth Conference on Innovative Applications of Artificial Intelligence, IAAI 2024, Fourteenth Symposium on Educational Advances in Artificial Intelligence, EAAI 2014, February 20-27, 2024, Vancouver, Canada, pages 18644-15772. AAAI Press, 2024.
[38] Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, 1lya Sutskever, et al. Language models are unsupervised multitask learners. OpenAl blog. 188)9, 2019.
[39] An Yang, Baosong Yang. Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang. Jianhong Tu. Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Mei Li, Mingfeng Xue, Pei Zhang, Qin Zhu, Rui Men, Runji Lin, Tianhao Li, Tingyu Xia, Xingzhang Ren, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yu Wan, Yuqiong Liu, Zeyu Cui, Zhenru Zhang, and Zihan Qiu. Qwen2.5 technical report. CoRR, abs/2412.15115, 2024.
[40] Laurens Van der Maaten and Geoffrey Hinton. Visualizing data using I-sne. Journal of machine learning research, 9(11), 2008.
[41] Shufan Shen, Junshu Sun, Xiangyang Ji, Qingming Huang, and Shuhui Wang. Expanding sparse tuning for low memory usage. Advances in Neural Information Processing Systems, 37:76616-76642, 2024.
[42] Shufan Shen, Zhaobo Qi, Junshu Sun, Qingming Huang, Qi Tian, and Shuhui Wang. Enhanc- ing pre-trained representation classifiability can boost its interpretability. In The Thirteenth International Conference on Learning Representations.
[43] Baolong Bi, Shenghua Liu, Yiwei Wang, Lingrui Mei, Hongcheng Gao, Yilong Xu, and Xueqi Cheng. Adaptive token biaser: Knowledge editing via biasing key entities. arXiv preprint arXiv:2406. /2468, 2024.
[44] Mengqi Zhang, Xiaotian Ye, Qiang Liu, Pengjie Ren, Shu Wu, and Zhumin Chen. Knowledge Graph Enhanced Large Language Model Editing. In Proceedings of the 2024 Conference on Empirical Methods in Natural Lunguage Processing, pages 22647-22662, Miami, Florida, USA, November 2024. Association for Computational Linguistics.
[45] Yifan Lu, Yigeng Zhou, Jing Li, Yequan Wang. Xuebo Liu, Daojing He, Fangming Liu, and Min Zhang. Knowledge Editing with Dynamic Knowledge Graphs for Multi-Hop Question Answering. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 24741-24749, April 2025.
[46] Thomas N. Kipf and Max Welling. Semi-Supervised Classification with Graph Convolutional Networks. In International Conference on Learning Representations, Toulon, France, 2017.
[47] Junshu Sun, Shuhui Wang, Xinzhe Han, Zhe Xue, and Qingming Huang. All in a row: Com- pressed convolution networks for graphs. In Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pages 33061 PMLR, 2023.
[48] Junshu Sun, Chenxue Yang. Xiangyang Ji. Qingming Huang, and Shuhui Wang. Towards dynamic message passing on graphs. In Advances in Neural Information Processing Systems, volume 37, pages 80936-80964. Curran Associates, Inc., 2024.
[49] Chengxuan Ying. Tianle Cai, Shengjie Luo, Shuxin Zheng, Guolin Ke, Di He, Yanming Shen, and Tie-Yan Liu. Do Transformers Really Perform Bad for Graph Representation? In Advances in Neural Information Processing Systems, volume 34, pages 28877-2888 28888. Curran Associates, Inc., 2021.
[50] Junshu Sun, Shuhui Wang, Chenxue Yang, and Qingming Huang. Scalable Graph Compressed Convolutions, October 2024.
14

---

[51] Junshu Sun, Wanxing Chang, Chenxue Yang, Qingming Huang, and Shuhui Wang. Relieving the Ovgr-Aggrggttnn Effect in Graph Transformers, 2025.
[52] Shiwen Ni, Dingwei Chen, Chengming Li, Xiping Hu, Ruifeng Xu, and Min Yang. Forgetting before learning: Utilizing parametric arithmetic for knowledge updating in large language models. arXiv preprint arXiv:2311.08011 2023.
[53] Haowen Pan, Xiaozii Wang, Yixin Cao, Zenglin Shi, Xun Yang, Juanzi Li, and Meng Wang. Precise localization of memories: A fine- ene-ggained neuron-level knowledge editing technique for Ilms. arXiv preprint PrXXiv2503.01090 2025.
15

---

Appendix
In the Appendix, we provide further details, including experimental setups, additional results, analyses. and discussions, as outlined below:
Appendix A: Experimental setups (cf. Section 4).
Appendix B: More experimental results (cf. Section 3 and 4),
Appendix C: Limitations and future discussions.
A Implementation Details
A.1 Description of Datasets
We evaluate general model capabilities using five challenging benchmarks covering factual knowledge. reasoning, and code generation. We also test knowledge editing using two standard factual-editing datasets. The benchmarks are described below:
MMLU [24]: a multiple-choice benchmark with over 16,000 questions across 57 academic and professional subjects (e.g., mathematics, history, law, medicine). Each question has four answer choices. MMLU tests a model's recall of domain-specific knowledge and multi-domain reasoning, serving as a broad measure of general knowledge and reasoning ability.
GSM8K [25]: contains roughly 8,500 grade-school math word problems requiring step-by-step arithmetic reasoning. This benchmark tests model's ability to perform multi-step numerical calculations.
CommonsenseQA [26]: multiple-choice question- -attiionnn dataset focused on everyday com- monsense reasoning. Each question has five options, only one of which is correct. CommonsenseQA assesses model's understanding of implicit context and everyday knowledge.
BBH [27]: subset of 23 challenging tasks drawn from the BIG-Bench benchmark, including logic puzzles, mathematical reasoning, and code comprehension. We evaluate BBH in zero-shot setting test model's general reasoning ability and robustness complex tasks without any task-specific training. In this paper, we choose to conduct the evaluation using a zero-shot approach.
HumanEval val [35]: code- generation benchmark with 164 Python programming problems. Each problem provides a function signature, docstring, and example input-output pairs; the model must gen- erate code that passes the provided unit tests. HumanEyal measures model's functional correctness and programming proficiency.
For factual knowledge editing, we adopt two standard benchmarks following prior work [6, 15]. ZsRE [34] is a relation-centric question-answering dataset where each example includes an edit prompt with target answer, semantically equivalent paraphrase to test generalization, and an unrelated prompt to probe locality. CounterFact [13] consists of factual statements paired with counterfactual versions (created by replacing the subject entity while keeping the predicate fixed).
A.2 Reproduction Details
In this section, we provide detailed information reproduce our experimental results. All experiments are conducted using 8 NVIDIA A100 GPUs.
For all of the models, we use HuggingFace Transformers by default battps://githar com/ httgg/gggggggc///e/e//egge transfo frrnnsoomrrrs))
For editing language models, use the EasyEdit framework
(https: thub. cwm/ //anl///a//y/iiitttt
For evaluating the general capabilities of models, we the Language Model Evaluation Harness
/ http://www....oeeeeeeeeeeeee/aaa harnessss.
In editing experiments, our hyperparameterss framework
follo follow the settings provided by the EasyEdit
16

---

Layer
Layer
Layer 30
Figure 8: Neuron activation values layers 15, and 30 across 60 prompts.
B
More Experimental Results and Analyses
B.1
Functional Roles of Neurons.
To further verify the generality of our neuron classification scheme across domains, we extend the masking analysis from college biology the high school chemistry subset in MMLU. As shown in Table 5, masking the top- 10 and top-50 task-relevant neurons (based on attribution scores) results in accuracy drops of 14.28% and 14.77%, respectively. In contrast, masking knowledge-genent neurons leads to a drastic degradation, with accuracy dropping by over 35 percentage points. These results are consistent with the findings presented in $ 3.1, offering additional empirical evidence for the non-uniform distribution of knowledge neurons within LLMs.
Table 5: Accuracy and drop in performance for different masking strategies on the high school chemistry task.
Task
Mask Type
Accuracy (%)
Drop (%)
All neurons (base model)
36.94
high school chemistry (mask top-10)
22,66 26
L14.28
high school chemistry
high school chemistry (mask top-50)
22.17
114.77
common neurons (mask top-10)
1.48
135.46
common neurons (mask top-50)
1.47
135.47
In addition, further explore the dynamics of neuron activations across different layers. Specifi- cally, we visualize the activation patterns of selected neurons in layers 1, 15, and 30, as shown in Figure 8. Neurons in the early layers exhibit amppamuiedde noisy activations, which reflect their general-purpose characteristics. In contrast, neurons in the mid-to-higs layers show higher activation magnitudes and greater variance across prompts, indicating an increased degree of specialization and functional diversity. Notably, several neurons in layer 30 consistently exhibit strong activations across prompts, suggesting a resonance-lie like behavior and highlighting their potential role in stable knowledge retrieval in specific tasks. These findings further validate our classification of knowledge- general neurons and knoweedge-peecffic neurons, and support the use dynamic sparse masking to target minimal yet effective subspaces for knowledge editing.
B.2 Scaling to 5000 Edits: Lifelong Robustness Evaluation.
Table 6 presents a comparison of knowledge editing and generalization performance for AlphaEdit and NMKE the ZsRE and CounterFact datasets, evaluated at 3000 and 5000 editing steps. On the ZsRE dataset, NMKE consistently outperforms AlphaEdit across all metrics at T = 3000. This trend is maintained T 5000, where NMKE achieves strong performance in both edit success and generalization, while AlphaEdit's performance remains low across both metrics. These results demonstrate NMKE's effectiveness in maintaining generalization and achieving higher accuracy in knowledge editing tasks, even with extensive editing steps.
B.3
MLP Weight Distribution Changes from 1000 to 2000 Edits: AlphaEdit vs. NMKE
Figure shows the evolution of MLP weight distributions for AlphaEdit and NMKE at T 1000, T = 1500, and 7 = 2000. For AlphaEdit, the weight distribution becomes progressively more dispersed with each additional editing step, reflecting the broader, layer-level modifications applied by this method, which leads increased instability in the model's 's internal representations. In contrast,
17

---

Table 6: Scaling to 5K edits on ZsRE and CounterFact datasets using C0A-MM-3--B-Inttrrre
ZSRE (7=3000))
General Tasks (7 (3000)))
Method
Rel.
Gen.
Loc.
MMLU
GSMBK
CommocseeneeQ
BBHl-Zeosholl
HumanEval
AlphaFdit
0.17
0.14
0.02
0.26
0.90
0.21
0.00
0.00
NMKE (ours)
0.92
0.81
0.68
0.59
0.64
0.68
0.40
0.26
ZSRE
(7=5000)
General Tasks (7=5000)
Method
Rel.
Gen.
Loc.
MMLU
GSM8K
CommonscnscQA
BBH-Zeroshot
HumanEval
AlphaEdit
0.02
0.02
0.00
0.27
0.00
0.20
0.00
0.00
NMKE,(our)
0.74
CF (7-3000)
General Tasks (7-3000)
Method
Rel.
Gen.
GSM8K
CommonsenmeeA
BBH-Zeroshot
HvmunEval
AlphaLdis
0.04
0.01
0.00
0.23
0,00
0.20
0.00
0.00
NMKE (ours)
0.23
(x) Alphareii (T-1000) on LLarm3 Instrutucct
(b) Alphasdii (T-1500)
500) om LLamu3-88 EB-ttrrrt (c) Alpluedit (T-2000) en LLamma3 8--Innssdd
-10
(d) NMKE (T-1000) on LLamal-88 SB Instruct
(c) NMKF (T 500) on I. L. Inn--teer
(1) NMKE (T-2000) on LJ -at----8---nsttec
MMY
-12
-12
-30
-22
+20
-06
-18
33
-2 -28
-11
Figure 9: Hicrarchical Distribution of Knowledge Neurons Across Layers
NMKE maintains more stable distribution across all editing steps, with minimal deviation from the original model. This evidences NMKE's neuron-level editing, targeting only relevant neurons and preserving global parameter stability with minimal interference. The observed trends highlight NMKE's advantage in sustaining internal stability across sequential knowledge edits.
B.4 Editing Performance on GPT2-XL and Qwen2.5-7B
Table 7 shows that NMKE consistently outperforms all baseline methods, including AlphaEdiit, on the 7sRE dataset using GPT2-XI,, particularly in editing accuracy and localization. While NMKE consistently leads, the difference compared to AlphaEdit is modest due to the smaller size of GPT2- XL. Despite this, NMKE demonstrates a slight advantage, especially in later editing steps. Results on Qwen2.5-7B are shown in Table 8.
B.5
N-ro Distribution of Layer 8 MLP Down-Projection Weights
Figure 10 illustrates the norm distribution of the layer-8 MLP down-projection weights for the original model, AlphaEdit, and NMKE after 2000 sequential edits. The original model exhibits tight distribution. In contrast, AlphaEdit broadens and shifts the distribution, indicating large layer-wise perturbations and reduced stability. NMKE remains close to the original with slight deviations, consistent with selective neuron-level editing that minimizes collateral changes. Overall, NMKE induces fewer disruptions internal representations than AlphaEdit, thereby better preserving stability and general capability under extensive edits.
18

---

Table 7: Editing performance the ZsRE dataset using GPT2-XL across sequential steps.
GPT2-XL
Method
T-
10
T-
7-
1000
7'
1500
7 2000
Rel.
Gen.
Loc.
Rel.
Gen.
Loc.
Rel.
Gen.
Loc.
Rel.
Gen.
Loc.
Rel.
Gen.
Loc.
Rel.
Gen.
Loc.
FT
0.26
0.18
0.06
0.06
0.06
0.02
0.07
0.04
0.01
0.06
0.05
0.01
0.06
0.06
0.01
0.05
0.04
0.00
ROME
0.93
0.72
0.19
0.03
0.05
0.01
0.02
0.01
0.00
0.00
0.01
0.00
0.01
MEMIT
0.83
0.72
0.96
0.51
0.72
0.92
0.05
0.01
0.01
0.02
0.01
0.00
0.00
0.00
0.01
0.00
0.00
0.01
AlphaRdii
1.00
NMKE (Ours)
0.70
Table 8: Editing performance on the ZsRE dataset using Qwen2.5-78 across sequential steps.
Method
T
100
T
500
T
1000
2000
Rel.
Gen.
Rel.
Gen.
Rel.
Gen.
Rel.
Gen.
Alphacdit
0.98
0.96
0.97
0.90
0.95
0.89
0.92
0.85
NMKE
0.97
0.96
0.97
0.93
0.97
0.91
0.96
0.90
B.6 Hierarchical Distribution of Knowledge Neurons Across Layers
Figure 11 shows the distribution of knowledge-general and knowledge-specifie neurons across layers 0-31 of LLLMM3--8B to reveal their structural roles in knowledge representation. Knowledge-genent neurons concentrated in the top layers, with notably higher activation rate in layer 31, indicating that they play central role in aggregating high-level knowledge. Knowledge-speciffe neurons are more evenly spread and gradually thin out with depth. The overlap ratio between knowledge-geners and knowledge-specifie neurons reaches its maximum in the middle layers, implying that these layers may act as integration hubs where general and specific knowledge intersect. This mirrors the theoretical intuition that intermediate Transformer layers blend low-level and high-level semantics.
sparse mask dynamically identifies and edits only minimal subset relevant to the target knowledge. The temperature coefficient and scaling factors tuned. All other hyperparameters were fixed across models and datasets, and NMKE consistently achieved strong editing and generalization.
Notably, instead of modifying all Anowledge-asoocitted associated neurons,
set from the empirical distribution of neuron activations rather than hand-
B.7 Memory overhead
To reduce overhead, we compute neuron attribution only in layers 4-8, As summarized in Table 9, each layer instantiates three additional memory objects: attribution coefficients (=00.33 MB per layer), a binary importance mask (=0.05 MB per layer), and two scalars. These objects are created on demand during forward and attribution and released after editing, and their memory is negligible relative to the model parameters.
Distrfaution NLP Down-Prejectoon
Vodel
Denrte
Average Edtlo
0.50
6.75
3.00
1.25
3.50
3.75
2.00
2.25
2.52
Loyer
(0) neuron activation ratio acmoss layers
Figure 10: Hierarchical Distribution of Knowl- edge Neurons Across Layers
Figure 11: Hierarchical Distribution of Knowl- edge Neurons Across Layers
19

---

Table Layer-wise attribution memory statistics.
Layer
Coeff Size (MB)
Non-zero Neurons
Resonance Ratio
Burst Ratio
4-8
0.33 x
~6,072-7,320
~0.00-0.3
~..24-0.2
Table 10: Quantitative analysis of parameter stability.
Layer
Comparison
Wasserstein
Cosine Meant
Cosine Std
&g Mean.
E Std
Base vs AlphaEdit
0.2079
0.7632
0.0292
0.9337
0,1507
7
Base vs NMKE
0.0357
0.9588
0.0067
0.3564
0.0218
Base vs AlphaEdiit
0,4371
0.6386
0.0594
1.2864
0,4421
8
Base vs NMKE
0.1057
0.8907
0,0175
0.6000
0.0542
Base vs AlphaEdit
0.0000
1.0000
0.0000
0.0000
0,0000
9
Base vs NMKE
0.0000
1.0000
0.0000
0.0000
0.0000
Table 11: Batched sequential editing scenarios.
T' = 1000
T = 2000
Method
Rel.
Gen.
Fluency
Rel.
Gen.
Fluency
AlphaEdit
0.88
0.79
4.79
0.68
0.58
4.77
NMKE
0.92
0.83
5.82
0.80
5.79
B.8
Quantitative analysis of parameter stability
We quantified weight shifts in layers 7-9 using Wasserstein, cosine, and fz distances. As shown in Table 10, NMKE induces significantly smaller distributional shifts than AlphaEdit in the edited layers (4-8), better preserving the internal weight structure. In the unedited layer 9, neither method produces any change.
B.9 Batched sequential editing scenarios
Table shows that we use batch size of across 2000 sequential edits to simulate practical simultaneous updates, evaluating all methods only after completing all edits for a fairer comparison. Our results demonstrate that NMKE preserves stable editing accuracy and locality as the number of edits grows, while AlphaEdit's performance degrades considerably. This is likely due to AlphaEdit's fixed global projection matrix, which cannot adapt to distributional shifts across multiple edits. In contrast, NMKE's fine-gu ane-grained neuron-level masking minimizes per-batch weight changes, reducing model drift.
C Limitations and Future Discussions
NMKE's performance largely depends on neuron attribution and dynamic sparse masking. Our current approach labels neurons as knowledge-genee general knowledge-spene -ppecicccc based on activation patterns. practice, neuron responses vary along a continuum and are influenced by both within-layer activations and cross-layer interactions. Building on this, we will design more precise, continuous importance measures both the intra-layer and inter-layer levels and explicitly model cross-layer information flow to learn path-aware attribution and masks that remain consistent across layers, which should improve parameter localization and stability during long-horizon editing. addition, edits should satisfy dependency closure across related but distinct facts. For example, updating the current CBO should remain consistent with the total number of CEOs. Quantifying and mitigating post-edit propagation errors and conflicts is key component of knowledge editing. These directions will guide our future work.
20

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
The main claims made in the abstract and introduction accurately reflect the paper's contributions and scope.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
We discuss the limitation in Appendix C.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
21

---

Justification:
This paper does not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: Please refer Appendix A and the codes in the supplementary.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm. (b) the contribution is primarily a new model architecture, the paper should describe the architccture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
22

---

Answer: [Yes]
Justification:
Yes, please refer to Appendix A.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.. cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https: /nips wwtpublicgguiiiiccccccceccoopppe httpe/guudeeeeduubbeeeeeeeeeeeee /at////ooommsssseeeeee ssssssolicyyyy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Question:
Answer: [Yes]
Justification: Yes, please refer to Appendix A.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification: This paper does not report error bars following the practice of previous studies [6, 11].
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the enror bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
23

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: Yes, please refer to Appendix A.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't 't make it into the paper).
9. Code of ethics
Question:
Does the research conducted in the paper conform, in every respect, with the ines?
NeurIPS Code of Ethics https: h/te:://weeuueeeeeeeeeeeee tttpss//aaarpp....ppbbbbtt////// ttepp//eeeeepp.ttttttt...eeeeeee ctw.idelscllllleccccccccccc
Answer: [Yes]
Justification:
This paper conforms with the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: Yes, please refer to Appendix C.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no socictal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of MI..
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
This paper poses such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Lieenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: Yes, please refer to Appendix A.
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
25

---

If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: Yes, please refer Appendix A.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/mmoel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question:
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: This paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
26

---

Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that if the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Answer: [NA]
Justification: LLM is used only for writing, editing, formatting purposes.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs any important, original, non-standard components.
Please refer to our LLM policy (https (https://neull /ttp:///eeuuppp/eeee http:/pp/....eeeeeeeee////////// for what should or should not be described.
27


Reviews:
[{"review_id": "8BCpsK9uHC", "reviewer": "Reviewer_64fD", "summary": "A method is presented to edit knowledge in an existing LLM using a dynamic sparse masking mechanism. The distinctive feature here is that the mechanism operates on individual neuron units, namely their observed additive contributions to the output. Metric quantifying knowledge specificity vs. generality are proposed and utilized to achieve efficient and non-destructive editing. The technique is compared to relevant SOTA baselines on two editing tasks along with several background sets to gauge general degradation.", "questions": "* can you comment on complexity of the method: which additional memory objects are instantiated, what is their size for the Llama model, quantify the computational overhead\n* on the two editing tasks, which data partitions were used  to estimate I and the scores (4-7) and when? on l. 173 you mentioned the ratio \\rho_{ge} is being \"predicted\" which confused me - I suspect to have missed something. What is the predictor here?\n* Figure 2, 7 and probably 5: the legends are really illegible when printed out.", "limitations": "Limitations are not discussed explicitly in the main paper. One such limitation is not studying an editing impact on closely related (but different) facts (e.g., updating name of a CEO of a company and its impact on the total number of CEOs that company has had).", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 4, "originality": 3, "strengths_and_weaknesses": "+ well-written paper with clear exposition\n+ sensible technique\n+ good baselines and thorough evaluation \n+ significant improvements\n\n- some gaps in description and complexity analysis \n- illegible figures", "was_revised": true, "final_justification": "All of my questions and comments were addressed by the authors to my satisfaction. I recommend this paper to be accepted for publication.", "rebuttal": "## W1: Some gaps in description and complexity analysis.\n\nThank you for the careful review. We have revised the paper for precise terminology and tightened the exposition to avoid confusion and improve readability. Concrete clarifications with quantitative memory/runtime complexity are provided in Q1–Q3.\n\n**Revision**. We will clarify technical terms and add a quantitative summary of NMKE’s memory and runtime complexity in Section 3 and Appendix C.\n\n## W2 \\& Q3: Illegible figures.  \n\nThank you for your constructive feedback. We have revised all the visualizations by enlarging legend/axis fonts, increasing the line widths, and enhancing the color contrast; printed copies confirm these updates have greatly improved readability.\n\n**Revision.** We will update Figures 2, 5, and 7 with enlarged fonts and higher-resolution vector graphics, and ensure all figures are clear in both print and digital versions.\n\n## Q1: Memory overhead and computational cost analysis.\n\nWe thank the reviewer for the thoughtful questions. Below, we provide a detailed analysis from memory overhead and per-edit computational cost.\n\n### Memory Overhead:\nTo minimize overhead, our method computes neuron attribution only for the edited layers (4–8), which is consistent with AlphaEdit. For each layer, the following additional memory objects are instantiated:\n\n* Attribution coefficients (\\~0.33 MB per layer)\n* Binary importance mask (\\~0.05 MB per layer)\n* Two scalar values (resonance and burst ratios)\n\nThe statistics from layers 4 to 8 are summarized in the table below:\n\n| Layer | Coeff Size (MB) | Non-zero Neurons | Resonance Ratio | Burst Ratio |\n| ----- | --------------- | ---------------- | --------------- | ----------- |\n| 4–8   | 0.33 × 5        | ~6,072–7,320     | ~0.30–0.33      | ~0.24–0.28  |\n\n| Component               | Memory Footprint |  \n|-------------------------|------------------|  \n| LLaMA-3-8B Model        | 29.9151 GB       |  \n| Intermediate Tensors    | 0.0019 GB      |  \n\nThe intermediate tensors involved account for 0.006248\\% of the LLaMA-3-8B model's memory. For our method, the additional memory consumption is very marginal.\n\n### Computational Overhead:  \n\nTo address your concerns about the computations introduced by attribution, we report the per-edit runtime and memory usage of current SOTA methods and NMKE with different attribution strategies: MLP Projection Coefficient (MPC) [R2], probability Shift Attribution(PSA) [R1], and Log Probability Shift (LPS) [R1] under sequential edits (after T=1,000 and T=2,000 edits).\n\n| Method        | Step Time (s) | $T$=1,000       |       |       | $T$=2,000       |       |       |\n|---------------|---------------|------------|-------|-------|------------|-------|-------|\n|               | per edit      | Rel.  | Gen.  | Loc.  | Rel.  | Gen.  | Loc.  |\n| MEMIT         | 16.8281       | 0.04 | 0.04 | 0.03 | 0.04 | 0.03 | 0.03 |\n| WISE          | 26.4898       | 0.41 | 0.39 | –    | 0.32 | 0.31 | –    |\n| AlphaEdit     | 22.1583       | 0.93 | 0.84 | 0.58 | 0.62 | 0.54 | 0.14 |\n| NMKE (MPC)    | 22.2545       | 0.94 | 0.82 | **0.81** | 0.93 | 0.83 | **0.77** |\n| NMKE (PSA)    | 29.6748       | **0.95** | 0.84 | 0.79 | **0.94** | **0.86** | 0.71 |\n| NMKE (LPS)    | 30.4210     | **0.95** | **0.85** | 0.77 | **0.94** | **0.86** | 0.74 |\n\n1. The most efficient variant NMKE (MPC) outperforms AlphaEdit across all metrics at $T$=2,000, with only ~1s extra runtime and minimal memory growth. \n2. The most advanced variant NMKE (LPS) remains practically efficient, adding only 8s per edit compared to AlphaEdit. Over 2000 edits, this adds ~4.5 hours, an acceptable cost for substantial quality gains: Locality rises from 0.14 to 0.74, with Reliability and Generalization each up 0.32. \n3. NMKE is a flexible framework that `supports attribution methods of varying complexity`, enabling users to balance quality and efficiency under different scenarios.\n\nReferences:\n\n[R1] Neuron-level knowledge attribution in large language models. AarXiv'23.  \n\n[R2] Transformer Feed‑Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Space, EMNLP'22.  \n\n**Revision.** We will add a new subsection in Appendix C that details all additional memory objects and their sizes for LLaMA3‑8B, and includes a quantitative table clearly comparing NMKE’s computational overhead with other methods. \n\n## Q2: On the two editing tasks, which data partitions were used to estimate I and the scores (4-7) and when? On I. 173 you mentioned the ratio $\\rho_{ge}$ is being \"predicted\" which confused me - I suspect to have missed something. What is the predictor here? \n\nThank you for the thoughtful question and careful reading.  \n\n1. For the two editing tasks, the neuron attribution matrix $I^{(l)}$ and scores (Eqs. 4-7) are estimated using the editing prompts from the current batch of requests, with no separate data partitions; Specifically, at each editing step, we use the current batch of editing requests as input prompts. \n\n2. Regarding the confusion around \"predicted\" for $\\rho_{ge}$ on line 173, we appreciate for pointing out the imprecise wording. There is no learned predictor; instead, $\\rho_{ge}$ is *estimated* deterministically. We compute the average entropy $H_{ge}$ across neuron attribution distributions over the batch of prompts (Eq. 6), then linearly map this entropy to a selection ratio using fixed scalers $a_{ge}$ and bias terms $b_{ge}$ (i.e., $\\rho_{ge} = H_{ge} \\cdot a_{ge} + b_{ge}$), with no trainable parameters involved.  \n\n**Revisions**. We will replace \"predicted\" with \"estimated\" throughout (notably line 173) and clarify in Section 3.2 that ratios and scores are computed from the current batch of editing prompts without learned predictors.\n\n## L1: Lack of explicit discussion on limitations, such as not evaluating the effect of edits on related but distinct facts.\n\nWe truly appreciate your insightful observation. Your suggestion touches on a critical yet underexplored aspect of model editing: how edits to specific factual triples propagate to closely related but distinct knowledge, such as aggregated statistics or temporal sequences.\n- Analysis method: Following your suggestion, we designed a pilot study to evaluate this phenomenon. We constructed 10 synthetic samples to simulate your highlighted scenario: updating a company’s CEO and querying related info like total CEO count or previous holders. \n- Results: Notably, after updating a key fact (e.g., “Tim Cook → Elon Musk”), LLMs often failed to adjust related dependent facts; for example, they incorrectly estimated the company’s total CEO count, revealing post-edit inconsistency and factual conflicts.\n- Future direction: This remains an important open yet challenge in knowledge editing. We plan to explore it further by constructing a dedicated benchmark for factual dependencies and incorporating metrics to assess post-edit consistency of related facts.\n\n**Revision**. We will add a limitation discussion in the main paper regarding the lack of evaluation on the effects of edits on closely related facts, and outline this as a key direction for future work."}, {"review_id": "GmWuFgws1v", "reviewer": "Reviewer_tXMJ", "summary": "This paper proposes the Neuron-specific Masked Knowledge Editing (NMKE) framework, which identifies Knowledge-General Neurons and Knowledge-Specific Neurons through neuron-level attribution analysis. By combining a dynamic sparse masking mechanism, NMKE locates neuron subsets relevant to target knowledge and performs localized parameter updates on these key neurons. This approach enables high-precision knowledge correction in lifelong editing scenarios while avoiding the degradation of model generalization caused by coarse-grained updates. Experiments verify that NMKE maintains high editing success rates and generalization capabilities across thousands of sequential edits, outperforming existing methods like ROME and AlphaEdit, thus providing a novel solution for efficient lifelong knowledge editing in large language models.", "questions": "See above", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths:  \n1. The NMKE framework is proposed, which identifies Knowledge-General Neurons and Knowledge-Specific Neurons through neuron-level attribution analysis, enabling high-precision knowledge correction in lifelong editing scenarios while avoiding the degradation of model generalization capabilities caused by coarse-grained updates.  \n\n2. It reveals that the root cause of performance degradation in lifelong editing is the cumulative interference of coarse-grained updates on neurons, and clarifies the core role of Knowledge-General Neurons in maintaining the model's basic capabilities.\n\nWeaknesses:  \n1. The authors discretely classify neurons into \"general/specific\" categories without considering the continuous spectrum of importance, which may overlook the complex functions of intermediate-state neurons.  \n\n2. The method proposed in the paper for locating neuron contributions is mainly based on existing approaches [1].  \n\n3. Hyperparameters such as the temperature coefficient α and scaling factors a_ge/a_sp in the entropy-guided mechanism require manual tuning and may need to be reconfigured for different tasks or models, increasing computational costs.\n\n4. There are a number of related works that have not been discussed and compared, e.g., [2,3,4].\n\n[1] Zeping Yu and Sophia Ananiadou. Neuron-level knowledge attribution in large language models. ArXiv preprint, abs/2312.12141, 2023.\n\n[2] Ni, Shiwen, et al. \"Forgetting before Learning: Utilizing Parametric Arithmetic for Knowledge Updating in Large Language Models.\" Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2024.\n\n[3] Pan, Haowen, et al. \"Precise Localization of Memories: A Fine-grained Neuron-level Knowledge Editing Technique for LLMs.\" arXiv preprint arXiv:2503.01090 (2025).\n\n[4] Bi, Baolong, et al. \"Adaptive Token Biaser: Knowledge Editing via Biasing Key Entities.\" Findings of the Association for Computational Linguistics: EMNLP 2024. 2024.", "was_revised": true, "final_justification": "Thank you for authors’ response, which has partially addressed my question. On the premise that the authors ensure to update the content of the reply in the revised version, I will raise the score to 4 points.", "rebuttal": "## W1: Discretely classify neurons into \"general/specific\" categories without considering the continuous spectrum of importance\n\nThank you very much for your suggestions. To evaluate the sufficiency of distinguishing different functions of neurons and omitting subsets during editing, we replaced the binary-masking strategy with the soft-masking strategy to incorporate the continuity of the neuron states. Specifically, the update magnitude for each neuron is directly scaled by its attribution score. \n\nWe conducted ablation experiments on 2000 sequential edits. The comparison between our default NMKE and the NMKE-softmask variant is shown below:\n- Soft masking matches NMKE in accuracy and generalization but reduces locality, due to the editing of all neurons. \n- NMKE avoids this by selectively editing subsets of the neurons, maintaining stability and locality in lifelong editing.\n\n| Method          | $T$=1,000        |               |        | $T$=2,000        |               |        |\n|-----------------|---------------|---------------|--------|---------------|---------------|--------|\n|                 | Rel.          | Gen.          | Loc.   | Rel.          | Gen.          | Loc.   |\n| NMKE-softmask   | 0.96          | 0.87          | 0.67   | 0.72          | 0.61          | 0.19   |\n| NMKE            | 0.95          | 0.85          | 0.77   | 0.94          | 0.86          | 0.74   |\n\n**Revision.** We will add ablation experiments and discuss the advantages of employing the binary-masking strategy compared to the soft-masking strategy.\n\n## W2: Relies on existing methods for neuron attribution approaches [1]. \n\nThank you for the thoughtful questions. While our neuron attribution strategy is inspired by the perturbation-based analysis in [1], we would like to clarify that the core contribution of our paper, rather than proposing a new attribution technique, lies in building a flexible, modularized framework for knowledge editing that leverages neuron-level importance signals to enable fine-grained, dynamic, and selective edits.\n\nImportantly, NMKE is not tied to a single attribution method and remains compatible with various strategies, including: \n\n* MLP Projection Coefficient (MPC) [R2]: estimates importance via MLP down-projection weights, providing fast and low-overhead attribution. \n* Probability Shift Attribution (PSA): computes the change in output probabilities under neuron ablation.\n* Log-Probability Attribution (LPS) [R1]: measures the change in log-probability of the target token after neuron perturbation, offering the most faithful estimate of neuron contribution.\n\nTo demonstrate this flexibility, we conducted ablation studies using all three attribution methods within NMKE, showing that editing effectiveness is robust across variants:\n\n| Attribution Method   | $T$=1,000       |       |       | $T$=2,000       |       |       |\n|----------------------|--------------|-------|-------|--------------|-------|-------|\n|                      | Rel.         | Gen.  | Loc.  | Rel.         | Gen.  | Loc.  |\n| MEMIT                | 0.04         | 0.04  | 0.03  | 0.04         | 0.03  | 0.03  |\n| WISE                 | 0.41         | 0.39  | -     | 0.32         | 0.31  | -     |\n| AlphaEdit            | 0.93         | 0.84  | 0.58  | 0.62         | 0.54  | 0.14  |\n| NMKE (MPC)           | 0.94         | 0.82  | **0.81**  | 0.93         | 0.83  | **0.77**  |\n| NMKE (PSA)           | **0.95**         | 0.84  | 0.79  | **0.94**         | **0.86**  | 0.71  |\n| NMKE (LPS)           | **0.95**         | **0.85**  | 0.77  | **0.94**         | **0.86**  | 0.74  |\n\nThese results show that NMKE delivers robust editing performance across attribution methods, consistently maintaining high accuracy, generalization, and locality.\n\n**References:**  \n\n[R1] Neuron-level knowledge attribution in large language models. arXiv'23.\n\n[R2] Transformer Feed‑Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Space, EMNLP'22.\n\n**Revision.** We will add ablation results for multiple neuron attribution methods in the experiments section to demonstrate NMKE’s flexibility, and will briefly highlight its core contributions of dynamic masking, neuron categorization, and a flexible modularized framework.\n\n## W3: Requires manual hyperparameter tuning, impacting usability and efficiency.\n \nWe would like to clarify that parameters such as the temperature coefficient $a$ and scaling factors $a_{ge}$/$a_{sp}$ `were not manually tuned`, but rather set according to the distributional characteristics of neuron activations. Knowledge-general neurons typically show distributed and broad-domain activations, while knowledge-specific neurons are more focused on fine-grained facts. \n \nExcept for these parameters, we used the same hyperparameters across all models and datasets, with NMKE consistently achieving strong editing and generalization performance. Sensitivity analysis (Figure 7a) further confirms NMKE’s robustness to a wide range of hyperparameter values.\n\n**Revision.** We will clarify the hyperparameter selection process in the main text, and supplement the appendix with additional experiments (e.g., hyperparameter sensitivity tests across more model-task combinations) to further validate robustness and generalization.  \n\n## W4: There are a number of related works that have not been discussed and compared. \n\nThank you for highlighting these important related works. We appreciate the opportunity to clarify their distinctions and will incorporate a discussion of these methods in the revision. \n\n* Ni et al. (F-Learning) proposed a two-stage parametric approach (forgetting followed by learning) for model-wide parameter updates, while NMKE uses neuron-level dynamic sparse masking to achieve precise, localized edits and reduce parameter drift.\n* Pan et al. (FiNE) preserve language capabilities by freezing later layers, while NMKE constructs masks by identifying knowledge neurons, enabling fine-grained edits that preserve the model's general capabilities.\n* Bi et al. (ATBias) proposed an in-context editing technique that biases key entity tokens during decoding without altering parameters, relying on external memory. In contrast, NMKE internally edits relevant neurons, offering an efficient large-scale knowledge editing approach without the need for external memory.\n\n**Revision.** We will expand our discussion of the aforementioned related works to detail the distinctions and connections between these approaches and NMKE."}, {"review_id": "UiG6NRTc0E", "reviewer": "Reviewer_Y7Bv", "summary": "This paper introduces Neuron-Specific Masked Knowledge Editing (NMKE), a novel framework for editing knowledge in Large Language Models designed to address the challenge of performance degradation in lifelong learning scenarios. Existing methods often suffer from error accumulation over sequential edits, leading to a decline in both editing accuracy and the model's general capabilities. NMKE proposes a more precise, fine-grained approach by operating at the neuron level rather than the traditional layer level. Through extensive experiments involving up to 5,000 sequential edits on models like LLaMA3-8B-Instruct and GPT2-XL, the authors demonstrate that NMKE significantly outperforms existing state-of-the-art methods. It successfully maintains high editing accuracy and, crucially, preserves the model's general reasoning, mathematical, and coding abilities, where other methods exhibit catastrophic forgetting.", "questions": "1. Your t-SNE visualizations (Figure 5) provide strong qualitative evidence of reduced distributional shift. Have you considered quantifying this shift using a statistical measure on the parameter weight distributions? A quantitative metric could offer a more objective complement to the visual analysis of parameter stability.\n\n\n2. The dynamic selection ratios (ρge​ and ρsp​) are determined using entropy, but also rely on constant scaler and bias hyperparameters (age​, bge​, etc.). Your analysis in Figure 7 shows the effect of these terms. How were these hyperparameters initially chosen, and how sensitive is the model's performance to them? Do they need to be re-tuned for different model architectures or sizes, which could impact the method's practical applicability?\n\n\n3. The distinction between knowledge-general and knowledge-specific neurons is central to your method. Figure 11 analyzes the distribution and overlap of these neurons across layers. Could you elaborate on the functional role of the neurons that fall into the \"overlap\" category? How does editing this specific subset of neurons that are both general and specific impact the trade-off between editing success and locality preservation?\n\n\n4. The current approach uses a hard binary mask for updates.  Given that neuron importance is a continuous score, have you experimented with a \"soft\" masking approach? For instance, one where the magnitude of the update applied to a neuron (e.g., its learning rate) is scaled by its attribution score, potentially allowing for an even more nuanced and less disruptive edit.", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "**Strengths:**\n\nThrough extensive experiments involving up to 5,000 sequential edits on models like LLaMA3-8B-Instruct and GPT2-XL, the authors demonstrate that NMKE significantly outperforms existing state-of-the-art methods. It successfully maintains high editing accuracy and, crucially, preserves the model's general reasoning, mathematical, and coding abilities, where other methods exhibit catastrophic forgetting.\n\n**Weaknesses:**\n\n1. The binary classification of neurons into \"knowledge-general\" and \"knowledge-specific\" is an oversimplification of what is likely a continuous spectrum of neuron functionality. This rigid model may misclassify neurons with mixed or nuanced roles, as the scoring for each category treats them as distinct types.\n\n2. The method requires a neuron attribution step for every editing batch to generate the dynamic mask. The paper fails to analyze the computational overhead and scalability of this \"locate\" phase, which could be a significant bottleneck and render the framework impractical for high-throughput, real-world applications.\n\n3. The dynamic masking ratio, a key component, relies on scaler (a) and bias (b) hyperparameters that are shown to have a substantial impact on performance. The paper does not provide a clear methodology for how these crucial hyperparameters were chosen or discuss their generalizability, raising concerns about the practical effort required to deploy the method on new models or datasets.\n\n4. The experiments are primarily conducted on two LLMs (LLaMA3-8B and GPT2-XL). Widly used LLMs like Qwen are not tested.", "was_revised": true, "final_justification": "The authors' rebuttal addressed the main concerns and promised further revision. I maintain my score and recommend borderline acceptance.", "rebuttal": "## W1: Binary classification overlooks the continuity of neuron functions. & Q4: soft masking approach:\n \nWe sincerely thank the reviewer for this profound insight. To evaluate the sufficiency of distinguishing different functions of neurons and omitting subsets during editing, we replaced the binary-masking strategy with the soft-masking strategy to incorporate the continuity of the neuron states. Specifically, the update magnitude for each neuron is directly scaled by its attribution score.\n \nWe conducted ablation experiments on 2000 sequential edits. The comparison between our default NMKE and the NMKE-softmask variant is shown below:\n- Soft masking matches NMKE in accuracy and generalization but reduces locality, due to the editing of all neurons. \n- NMKE avoids this by selectively editing subsets of the neurons, maintaining stability and locality in lifelong editing.\n\n| Method          | T=1,000        |               |        | T=2,000        |               |        |\n|-----------------|---------------|---------------|--------|---------------|---------------|--------|\n|                 | Rel.          | Gen.          | Loc.   | Rel.          | Gen.          | Loc.   |\n| NMKE-softmask   | **0.96**      | **0.87**      | 0.67   | 0.72          | 0.61          | 0.19   |\n| NMKE            | 0.95          | 0.85          | **0.77** | **0.94**      | **0.86**      | **0.74** |\n \n**Revision.** We will explore scaling neuron update magnitudes (e.g., learning rates) by continuous attribution scores for more nuanced, less disruptive edits in future work.\n\n## W2: computational cost analysis\n\nThank you for the insightful suggestion. To address your concerns of the computations introduced by attribution, we report the per-edit runtime and memory usage of current SOTA methods and NMKE with different attribution strategies (MLP Projection Coefficient (MPC) [R2], probability Shift Attribution(PSA) [R1], and Log Probability Shift (LPS) [R1]) under sequential edits (after $T$=1,000 and $T$=2,000 edits).\n\n| Method        | Step Time (s) | $T$=1,000       |       |       | $T$=2,000       |       |       |\n|---------------|---------------|------------|-------|-------|------------|-------|-------|\n|               | per edit      | Rel.  | Gen.  | Loc.  | Rel.  | Gen.  | Loc.  |\n| MEMIT         | 16.8281       | 0.04 | 0.04 | 0.03 | 0.04 | 0.03 | 0.03 |\n| WISE          | 26.4898       | 0.41 | 0.39 | –    | 0.32 | 0.31 | –    |\n| AlphaEdit     | 22.1583       | 0.93 | 0.84 | 0.58 | 0.62 | 0.54 | 0.14 |\n| NMKE (MPC)    | 22.2545       | 0.94 | 0.82 | **0.81** | 0.93 | 0.83 | **0.77** |\n| NMKE (PSA)    | 29.6748       | **0.95** | 0.84 | 0.79 | **0.94** | **0.86** | 0.71 |\n| NMKE (LPS)    | 30.4210     | **0.95** | **0.85** | 0.77 | **0.94** | **0.86** | 0.74 |\n\n1. The most efficient variant NMKE (MPC) outperforms AlphaEdit across all metrics at $T$=2,000, with only ~1s extra runtime and minimal memory growth. \n2. The most advanced variant NMKE (LPS) remains practically efficient, adding only 8s per edit compared to AlphaEdit. Over 2000 edits, this adds ~4.5 hours, an acceptable cost for substantial quality gains: Locality rises from 0.14 to 0.74, with Reliability and Generalization each up 0.32. \n3. NMKE is a flexible framework that `supports for attribution methods of varying complexity`, enabling users to balance quality and efficiency under different scenarios.\n\nReferences:\n\n[R1] Neuron-level knowledge attribution in large language models. arXiv'23.  \n\n[R2] Transformer Feed‑Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Space, EMNLP'22.  \n\n**Revision.** We will add a computational cost analysis and comparison with other methods in the Appendix.\n\n## W3: Unclear and non-generalizable hyperparameter selection in dynamic masking. & Q2: Selection and generalizability of entropy-based hyperparameters.\n\nWe would like to clarify that parameters such as the temperature coefficient $a$ and scaling factors $a_{ge}$/$a_{sp}$ `were not manually tuned`, but set based on observed neuron activation patterns. This approach enables the mask to adapt to each batch’s activation patterns, typically selecting 30–40\\% of neurons without manual tuning. The masking ratio is derived directly from entropy, and parameters like \\$a\\_{ge}\\$ and \\$a\\_{sp}\\$ simply scale the outputs rather than requiring task-specific adjustment. \n\nWe used the same hyperparameters across all models and datasets, and NMKE consistently achieves strong editing and generalization performance. The sensitivity analysis (Figure 7a) further confirms NMKE’s robustness to a wide range of hyperparameter values.\n\n**Revision.** We will clarify the hyperparameter selection and supplement the appendix with extra experiments to further validate the robustness and generalization.\n\n## W4: More LLMs like Qwen\n\nThank you for the suggestion. We have implemented our method on Qwen2.5-7B and performed 2,000 edits. Due to time constraints, results from the first 100 edits are below. We will report the full editing result as soon as we finish the experiments.\n\n| Model/Method     | Rel. | Gen. | Loc. |\n| ---------------- | ---- | ---- | ---- |\n| AlphaEdit        | 0.98 | 0.96 | 0.85 |\n| NMKE             | 0.97 | 0.96 | 0.88 |\n\nThe results show our method performs similarly in the first 100 edits on Qwen2.5 and LLaMA3. This further demonstrates that our approach is model-agnostic and can be applied to any LLM with standard MLP layers.\n\n**Revision.** We will provide the full Qwen results and a discussion of model compatibility in the revision.\n\n## Q1: Quantitative analysis of parameter stability\n\nThank you for the suggestion. We quantified weight shifts in layers 7–9 using Wasserstein, cosine, and L2 distances.\n\n| Layer | Comparison        | Wasserstein↓ | Cosine Mean↑ | Cosine Std↓ | L2 Mean↓ | L2 Std↓ |\n| ----- | ----------------- | ------------ | ------------ | ----------- | -------- | ------- |\n| 7     | Base vs AlphaEdit | 0.2079       | 0.7632       | 0.0292      | 0.9337   | 0.1507  |\n|       | Base vs NMKE      | 0.0357       | 0.9588       | 0.0067      | 0.3564   | 0.0218  |\n| 8     | Base vs AlphaEdit | 0.4371       | 0.6386       | 0.0594      | 1.2864   | 0.4421  |\n|       | Base vs NMKE      | 0.1057       | 0.8907       | 0.0175      | 0.6000   | 0.0542  |\n| 9     | Base vs AlphaEdit | 0.0000       | 1.0000       | 0.0000      | 0.0000   | 0.0000  |\n|       | Base vs NMKE      | 0.0000       | 1.0000       | 0.0000      | 0.0000   | 0.0000  |\n\nNMKE produces much lower distributional shifts than AlphaEdit in edited layers (4–8), better preserving internal weight structure. In the unedited layer (9), both methods show no change.\n\n**Revision.** We will add quantitative ablation experiments on parameter shift to complement the t-SNE visualizations.\n\n\n## Q3: How do overlapping neurons influence the trade-off between editing success and locality?\n\nWe thank the reviewer for their insightful comments. To evaluate overlapping neurons, we edited using only overlapping or only non-overlapping neurons. The results are as follows:\n\n| Method                      | $T$=1,000        |               |        | $T$=2,000        |               |        |\n|-----------------------------|---------------|---------------|--------|---------------|---------------|--------|\n|                             | Rel.          | Gen.          | Loc.   | Rel.          | Gen.          | Loc.   |\n| Overlapping Neurons Only    | 0.72          | 0.60          | **0.84** | 0.75          | 0.63          | **0.80** |\n| Non-Overlapping Neurons Only| 0.82          | 0.70          | 0.82   | 0.85          | 0.74          | 0.78   |\n| NMKE                        | **0.95**      | **0.85**      | 0.77   | **0.94**      | **0.86**      | 0.74   |\n\nOur ablation study reveals consistent patterns:  \n- Editing only overlapping neurons achieves the highest locality (0.80) while maintaining a certain level of accuracy (0.75).  \n- Editing only non-overlapping neurons, though it improves accuracy, reduces locality.  \n- By incorporating overlapping neurons, NMKE strikes the best balance between edit accuracy and locality.\n\n**Revision.** We will add experiments (such as spectral analysis and neuron continuity visualization) and further clarify entropy-guided ratios and overlapping neurons, and their impact on the editing-locality trade-off."}, {"review_id": "iGM8J7Zgfz", "reviewer": "Reviewer_d78c", "summary": "This paper presents Neuron-Specific Masked Knowledge Editing (NMKE), a fine-grained editing framework that enables effective and robust lifelong knowledge editing for large language models (LLMs). By performing neuron-level attribution and dynamically masking a sparse subset of relevant neurons, NMKE updates model knowledge with minimal parameter modification, thereby mitigating error accumulation over thousands of edits. The authors demonstrate that NMKE significantly outperforms strong baselines (e.g., AlphaEdit) on sequential editing benchmarks, while preserving general capabilities across downstream tasks.", "questions": "This is a well-written and well-structured paper. The idea of combining neuron-level attribution with dynamic masking is not entirely novel, as prior work has explored neuron specialization and sparse editing mechanisms. However, the adaptation of this strategy to lifelong editing is impactful, which raise a useful mechanism to preserve general capabilities.\n\nTo further improve the work, I suggest addressing the following:\n- Please consider evaluating NMKE under batched edit scenarios (batch size > 1). This would better reflect practical applications and further highlight the scalability of your method.\n- Since your method involves computing attribution for a large number of neurons, it would be useful to report runtime or efficiency metrics. Even a high-level cost comparison with AlphaEdit or WISE would be valuable.\n\nThat said, the experimental design is rigorous, and the improvements over existing methods are impressive. If the above points are addressed or discussed, I would be inclined to increase my score.", "limitations": "Yes.\n\nThe authors provide a reasonable and well-articulated discussion of the limitations from the perspective of interpretability, particularly regarding the binary classification of neurons and the focus on autoregressive LLMs. \nIn addition, I suggest the authors consider incorporating insights from recent work that examines cross-layer interactions in LLMs, such as Anthropic’s Cross-Layer Transcoders (CLT). These works point to the growing importance of understanding information flow not just at the neuron level but across layers and pathways. Integrating such perspectives may help refine NMKE’s attribution and masking mechanisms, especially under long-term editing scenarios.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths:\n- This paper clearly articulates the challenges of lifelong model editing, especially the loss of generalization, which are underexplored in prior work.\n\n- The method achieves state-of-the-art performance under sequential editing settings (batch size 1 and up to 5,000 edits), with detailed evaluations on both editing success and general task performance.\n\n- The paper includes great visualizations and ablation studies to support the effectiveness of the neuron selection mechanism.\n\nWeaknesses:\n- Editing granularity setup: The current lifelong setting uses a batch size of 1 for sequential edits, which may not reflect practical scenarios where models need to absorb multiple factual updates at once. It is recommended to test the method under larger batch sizes and compare with baselines accordingly. Otherwise, the setting should be more precisely described as “Sequential Editing” rather than “Lifelong Editing” [https://aclanthology.org/2024.emnlp-main.149.pdf].\n\n- Efficiency and runtime analysis missing: NMKE requires computing attribution across all layers, which may introduce significant computational overhead. A runtime comparison with baselines or a profiling study would strengthen the claim of robustness and deployment readiness.", "was_revised": true, "final_justification": "The authors' rebuttal has effectively addressed my main concerns. I have raised my score and now recommend acceptance.\n\n- My concern about the batch_size=1 setting was resolved. The new experiments with larger batch sizes demonstrate that the method's strong performance holds in more practical scenarios, strengthening the paper's claims.\n\n- My concern regarding efficiency was also resolved. The provided analysis clearly shows that the computational overhead is minor and well-justified by the significant performance gains, especially in locality.", "rebuttal": "## W1 & Q1: Larger batch sizes:\n\nWe thank the reviewer for suggesting evaluating larger batch sizes. Following your advice, we conducted experiments with batch sizes 4 and 10 on 2000 sequential edits to better simulate practical simultaneous update scenarios. Due to rebuttal time constraints, we only present results for batch size=4 below. We will report the full editing result as soon as we finish the experiments.\n\n| Method    | $T$=1,000          |                 |                | $T$=2,000          |                 |                |\n|-----------|-----------------|-----------------|----------------|-----------------|-----------------|----------------|\n|           | Rel.            | Gen.            | Fluency        | Rel.            | Gen.            | Fluency        |\n| AlphaEdit | 0.88            | 0.79            | 4.79           | 0.68            | 0.58            | 4.77           |\n| NMKE      | **0.92**            | **0.83**            | **5.82**           | **0.90**            | **0.80**            | **5.79**           |\n\nNotably, we optimized AlphaEdit’s evaluation by evaluating only after all edits; this revised approach is fairer, more reasonable, and more challenging. Our findings show that NMKE maintains stable editing accuracy and locality as the number of edits increases, while AlphaEdit’s performance degrades significantly. This is because AlphaEdit uses a fixed global projection matrix computed once, which cannot adapt to distributional shifts over multiple edits. In contrast, NMKE employs fine-grained neuron-level masking, which limits weight modifications per batch and mitigates model drift.\n\nReferences:\n\n[1] LEMoE: Advanced Mixture of Experts Adaptor for Lifelong Model Editing of Large Language Models. EMNLP'24\n\n**Revision.** We will include the complete experimental results for batch sizes 4 and 10 in the manuscript and clarify the practical advantages of NMKE under both sequential and batch editing scenarios.\n\n## W2 & Q2: Efficiency and runtime analysis. \n\nThank you for your thoughtful questions. We provide a detailed analysis from two perspectives: memory overhead and per-edit computational cost.\n\n### Memory Overhead:\nTo minimize overhead, our method computes neuron attribution only for the edited layers (4–8), which is consistent with AlphaEdit. For each layer, the following additional memory objects are instantiated:\n\n* Attribution coefficients (\\~0.33 MB per layer)\n* Binary importance mask (\\~0.05 MB per layer)\n* Two scalar values (resonance and burst ratios)\n\nThe statistics from layers 4 to 8 are summarized in the table below:\n\n| Layer | Coeff Size (MB) | Non-zero Neurons | Resonance Ratio | Burst Ratio |\n| ----- | --------------- | ---------------- | --------------- | ----------- |\n| 4–8   | 0.33 × 5        | ~6,072–7,320     | ~0.30–0.33      | ~0.24–0.28  |\n\n| Component               | Memory Footprint |  \n|-------------------------|------------------|  \n| LLaMA-3-8B Model        | 29.9151 GB       |  \n| Intermediate Tensors    | 0.0019 GB      |  \n\nThe intermediate tensors involved account for 0.006248\\% of the LLaMA-3-8B model's memory. For our method, the additional memory consumption is very marginal.\n\n### Computational Overhead: \nTo address your concerns about the computations introduced by attribution, we report the per-edit runtime and memory usage of current SOTA methods and NMKE with different attribution strategies (MLP Projection Coefficient (MPC) [R2], probability Shift Attribution(PSA) [R1], and Log Probability Shift (LPS) [R1]) under sequential edits (after $T$=1,000 and $T$=2,000 edits).\n\n| Method        | Step Time (s) | $T$=1,000       |       |       | $T$=2,000       |       |       |\n|---------------|---------------|------------|-------|-------|------------|-------|-------|\n|               | per edit      | Rel.  | Gen.  | Loc.  | Rel.  | Gen.  | Loc.  |\n| MEMIT         | 16.8281       | 0.04 | 0.04 | 0.03 | 0.04 | 0.03 | 0.03 |\n| WISE          | 26.4898       | 0.41 | 0.39 | –    | 0.32 | 0.31 | –    |\n| AlphaEdit     | 22.1583       | 0.93 | 0.84 | 0.58 | 0.62 | 0.54 | 0.14 |\n| NMKE (MPC)    | 22.2545       | 0.94 | 0.82 | **0.81** | 0.93 | 0.83 | **0.77** |\n| NMKE (PSA)    | 29.6748       | **0.95** | 0.84 | 0.79 | **0.94** | **0.86** | 0.71 |\n| NMKE (LPS)    | 30.4210     | **0.95** | **0.85** | 0.77 | **0.94** | **0.86** | 0.74 |\n\n1. The most efficient variant NMKE (MPC) outperforms AlphaEdit across all metrics at $T$=2,000, with only ~1s extra runtime and minimal memory growth. \n2. The most advanced variant NMKE (LPS) remains practically efficient, adding only 8s per edit compared to AlphaEdit. Over 2000 edits, this adds ~4.5 hours, an acceptable cost for substantial quality gains: Locality rises from 0.14 to 0.74, with Reliability and Generalization each up 0.32. \n3. NMKE is a flexible framework that `supports attribution methods of varying complexity`, enabling users to balance quality and efficiency under different scenarios.\n \n\nReferences:\n\n[1] Neuron-level knowledge attribution in large language models. arXiv'23.  \n\n[2] Transformer Feed‑Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Space, EMNLP'22.  \n\n**Revision.** We will add a new subsection in Appendix C that details all additional memory objects and their sizes for LLaMA3‑8B, and includes a quantitative table clearly comparing NMKE’s computational overhead with other methods. \n\n\n## L1: Cross-layer interactions in LLMs.  \n\nThank you for your insightful and constructive suggestion. Although NMKE performs neuron attribution for each layer separately, the attribution module is integrated into the end-to-end optimization framework, which may help implicitly learn cross-layer neuron interactions. However, we agree with your suggestions that explicitly modeling cross-layer interactions like CLT can result in more accurate parameter localization and better editing performance. We will explore this as a key direction in our future work.\n \n\n**Revision.** We will expand the discussion of cross-layer knowledge tracing methods in the limitations section, and outline how integrating such perspectives could enhance NMKE’s attribution and masking in future work."}]

Author rebuttal:

Meta-review: - **Summary**: The paper proposes a framework to address the critical challenge of performance degradation in large language models (LLMs) during lifelong knowledge editing. The core idea is to perform fine-grained, neuron-level parameter updates rather than the coarse-grained, layer-level modifications of previous methods.

- **Strengths**: The reviewers unanimously praise the paper for its clear exposition, sensible technique, and thorough evaluation. The following points are highlighted as key strengths:
    - **Novelty**: The fine-grained, neuron-level approach is a novel contribution to the field of knowledge editing. It moves beyond the limitations of layer-level methods and directly addresses the problem of catastrophic forgetting in lifelong learning settings.
    - **Experiments**: The experimental results are highly convincing. The authors conduct a comprehensive evaluation, including two editing tasks, various background sets to measure general degradation, and extensive ablation studies.

- **Reason to Accept**: After rebuttals, all reviewers acknowledge their concerns are resolved.

- **Summary of Discussion and Rebuttals**: During the rebuttal, the authors presented additional experiments (e.g., time complexity analysis, performance on Qwen) and committed to revising the paper in line with the reviewers’ suggestions.
Decision: Accept (poster)