Energy-Regularized Sequential Model Editing on Hyperspheres
Authors: Qingyuan Liu
Jia-Chen Gu
Yunzhi Yao
Hong Wang
Nanyun Peng
Venue: ICLR 2026 Poster (2026)
Keywords: model editing
sequential editing
hyperspherical energy
regularization
Primary area: transfer learning, meta learning, and lifelong learning
TL;DR: We propose SPHERE, an energy-driven regularization strategy that stabilizes neuron weight distributions, ultimately preserving prior knowledge while enabling reliable sequential updates.

Abstract: Large language models (LLMs) require constant updates to remain aligned with evolving real-world knowledge. Model editing offers a lightweight alternative to retraining, but sequential editing that updates the LLM knowledge through multiple successive edits often destabilizes representations and induces catastrophic forgetting. In this work, we seek to better understand and mitigate performance degradation caused by sequential editing. We hypothesize that hyperspherical uniformity, a property that maintains uniform distribution of neuron weights on a hypersphere, helps the model remain stable, retain prior knowledge, while still accommodate new updates. We use Hyperspherical Energy (HE) to quantify neuron uniformity during editing, and examine its correlation with editing performance. Empirical studies across widely used editing methods reveals a strong correlation between HE dynamics and editing performance, with editing failures consistently coinciding with uncontrolled HE fluctuations. We further theoretically prove that HE dynamics impose a lower bound on the degradation of pretrained knowledge, highlighting why HE stability is crucial for knowledge retention. Motivated by these insights, we propose SPHERE (Sparse Projection for Hyperspherical Energy-Regularized Editing), an HE-driven regularization strategy that stabilizes neuron weight distributions, ultimately preserving prior knowledge while enabling reliable sequential updates. Specifically, SPHERE identifies a sparse space complementary to the principal hyperspherical directions of the pretrained weight matrices and projects new knowledge onto it, attenuating perturbations on the principal directions. Extensive experiments on LLaMA3 (8B) and Qwen2.5 (7B) show that SPHERE outperforms the best baseline in editing capability by an average of 16.41%, while most faithfully preserving general model performance, thereby offering a principled path toward reliable large-scale knowledge editing.

Published as a conference paper at ICLR 2026
ENERGY-REGULARIZED SEQUENTIAL MODEL EDITING ON HYPERSPHERES
Qingyuan Liu'; Jia-Chen Gu²; Yunzhi Yao", Hong Wang', Nanyun Peng?
Columbia University University of California. Los Angeles
3 ZZejianng University UUniversity of Science and Technology of China
gl2505ecolumbia edu, gujc@ucla. edu, violetpengêcs. ucla. edu
Website
Code
ABSTRACT
Large language models (LLMs) require constant updates remain aligned with evolving real-world knowledge. Model editing offers a lightweight alternative to retraining, but sequential editing that updates the L.M knowledge through mul- tiple successive edits often destabilizes representations and induces catastrophic forgetting. In this work, we seek better understand and mitigate performance degradation caused by sequential editing. We hypothesize that hyperspherical uniformity, a property that maintains uniform distribution of neuron weights on hypersphere, helps the model remain stable, retain prior knowledge, while still accommodate new updates. We use Hyperspherical Energy (HE) to quantify neu- uniformity during editing, and examine its correlation with editing perfor- mance. Empirical studies across widely used editing methods reveals a strong correlation between HE dynamics and editing performance, with editing failures consistently coinciding with high HE fluctuations. We further theoretically prove that HE dynamics impose a lower bound on the degradation of pretrained knowl- edge, highlighting why HE stability is crucial for knowledge retention. Motivated by these insights, we propose SPHERE (Sparse Projection for Hyperspherical Editing), an HR-driven regularization strategy that stabilizes neuron weight distributions, ultimately preserving prior knowledge while enabling reliable sequential updates. Specificallyy SPHERE identifies sparse space com- plementary to the principal hyperspherical directions of the pretrained weight ma- trices and projects new knowledge onto it, attenuating perturbations on the princi- pal directions. Extensive experiments on LLaMA3 (8B) and Qwen2.5 (7B) show that SPHERE outperforms the best baseline in editing capability by average of 16.41 % while most faithfully preserving general model performance, thereby offering principled path toward reliable large-scale knowledge editing.
INTRODUCTION
Large language models (LLMs) have demonstrated strong capabilities knowledge storage, reason- ing, and generation (DeepSeek-Al al.,
2024 Meta Al 2024 Yang al., 2025 OpenAl 2025
However, the knowledge embedded in L.I
Ms inevitably becomes outdated or incorrect, as real-world
facts continuously evolve (Ji et al.) 2023 Huang al. 2025). Retraining LLMs to incorporate such
updates is prohibitively expensive, motivating the development of model editing (also known as knowledge editing) (Cao et al. 202T Mitchell et al. [2022] Meng al., 2023 Gu et al. 2024 Fang et al., 2025). The most practical setting for model editing is sequential editing, where multiple updates are applied over time. However, previous studies have shown that such interventions of- ten suffer from significant performance degradation due to catastrophic forgetting (Gu et al. 2024 Gupta et al., 2024).. Consequently, reconciling the trade-off between preserving original pretrained knowledge and integrating new editing knowledge remains unresolved challenge.
In this work, we seek to better understand and mitigate the performance degradation caused by sequential editing. We revisit model editing from the perspective of hyperspherical uniformity of perturbed weights (Liu et al. 202T), motivated by the observation that sequential edits often disrupt
Equal contribution.

---

Published as a conference paper at ICLR 2026
(a) Weight Neurons
(b) Current Methods
(c) SPHERE (Ours)
Figure I: (a) A weight matrix is viewed as set of neurons (red dots) on a hypersphere. (b) Current SOTA methods (Ma et al., 2025, Fang et al. 2025) introduce perturbations (blue triangles) that interfere with the principle hyperspherical directions of pre-edit weights. (c) SPHERE projects new knowledge onto a sparse space complementary to the principal hyperspherical directions.
weight geometry, leading to degraded representations. Previous studies have shown that viewing a weight matrix as a set of neurons on a hypersphere (as shown in Figure (a)) and maintaining their hyperspherical uniformity is crucial for stable training and effective generalization (Cogswell
it al., 2016, Xie et al., 2017a Qiu et al., (023). To investigate the applicability of these principles to sequential editing, we adopt hyperspherical energy (HE) (Liu et al.) 2018: Qiu et al.) 2023) as a measure to quantify weight uniformity throughout sequential editing. HE calculates the dispersion of neuron weight vectors on a hypersphere, where lower energy corresponds to a more balanced distribution of neurons. By tracking HE dynamies throughout sequential editing, we can better un- derstand how edits affect weight uniformity, identify early signs of destabilization, and even develop HE-driven regularization strategies stabilize the editing process.
To reveal the mechanisms underlying successful editing strategies from the perspective of hyper- spherical uniformity, we first empirically analyze how HE evolves throughout sequential editing and examine how these dynamics relate to editing performance across six widely used methods. Experimental results reveal a strong correlation between hyperspherical uniformity and editing per- formance, with editing failures consistently coinciding with its collapse. Meanwhile, more advanced editing methods have proven more effective at preserving hyperspherical uniformity. To complement these empirical findings, we further provide a theoretical analysis verifying that variations in HE es- tablish lower bound on the interference with the original pretrained knowledge. This result clarifies that state-of-the-art (SOTA) editing methods implicitly regulate hyperspherical uniformity and the lower bound on the interference, providing a principled explanation for their enhanced robustness.
Motivated by these empirical and theoretical findings, we propose SPHERE (Sparse Projection for Editing), an HE-driven regularization strategy that stabilizes neuron weight distributions, ultimately preserving prior knowledge while enabling reliable sequen- tial updates. The key insight is that, as shown in Figure [ (b), current methods often introduce perturbations that interfere with the principal hyperspherical directions of the pretrained weight ma- trices, leading to instability, loss of uniformity, and eventual degradation of model performance. To counteract these side effects, as shown in Figure (c). SPHERE identifies sparse space comple- mentary to the principal hyperspherical directions of the pretrained weight matrices and projects new knowledge onto it, attenuating perturbation components aligned with those principal directions. By doing so, SPHERE effectively preserves the hyperspherical uniformity and substantially extends the number of effective sequential edits.
Hyperspherical
We evaluated SPHERE on two LLMs, including LLaMA3 (8B) ( Meta 2024) and Qwen2.5
(7B) (Team) 2024). on two editing datasets, including CounterFact (Meng al. 2022) and ZsRE (Levy et al. (017). Four downstream tasks including reasoning (Cobbe et al. (2021). natural language inference (Dagan et al. 2005), open-domain QA (Kwiatkowski et al. (011)) and closed- domain QA (Clark et al. (019) are employed to demonstrate the impact of editing on the general abilities of LLMs. Experimental results show that SPHERE sustains editing capacity under large- scale editing settings, outperforming the best baseline (Fang et al. 2025) by 16.41% on average. Beyond editing capacity. it more effectively preserves the hyperspherical uniformity and the gen- eral abilities of edited models than all baselines. Furthermore,
plug-and-pe enhancement, SPHERE improves the mainstream editing methods (Meng et al., 2023 Gu et al.) (2024) Ma et al.) 2025) by 38.71 % on average, offering principled path toward reliable and scalable editing.
2

---

Published as a conference paper at ICLR 2026
2 PRELIMINARIES
2.1 MODEL EDITING
Sequential model editing aims update the knowledge stored in LLMs through multiple successive edits. Each edit modifies the model parameter W E Rd d by adding a perturbation A E Rd x in a locate-then-edit paradigm (Meng et al. 2022). where dg and dj represent the dimensions of the intermediate and output layers of the feed-forward network (FFN), respectively. Specifically, suppose each edit updates pieces of knowledge in the form of (subject 8, relation r, object o), e.g.. (s = United States, r = President of, o == Donald Trump). The perturbed parameter is expected to associate u new key-value (k-v) pairs, where and encode (s, and (o) of the new knowledge, respectively. We can stack these keys and values into matrices as follows:
K [k k2 ... kw] R V1
U1 82 ... V.l Rd
(1)
where the subscripts of k and represent the index of the to-be-updated knowledge. Therefore, the editing objective can be expressed as:
(W AW)K; - vi vill²
AW
arg min AW
where || denotes the sum of the squared elements in the matrix.
Additionally, current methods typically incorporate an error term to preserve the original knowledge. Let K₂ and Vo represent the matrices formed by stacking the k and corresponding the original pretrained knowledge. Eqn. regularized by involving the error term as follows:
([(w + AW) - Vi + || + AW)
AW = arg min AW
Vo
(3)
Since Ko and Vo encode the original pretrained knowledge, we have W W,,- Vo (cf. Eqn. D By applying the normal equation, if the closed-form solution of Eqn. 3 existss, it can be written as:
AW (V1 WK))K, (Ko + K
(4)
Since the full scope of an LLM's knowledge is generally inaccessible, Ko is difficult to obtain directly but can be approximated from abundant text input. See Appendix B for more details.
2.2 HYPERSPHERICAL ENERGY
Hyperspherical Energy (HE) serves as a quantitative metric for measuring hyperspherical unifor- mity. Given group of neurons, HE characterizes their uniformity a hypersphere by defining generic potential energy based on their pairwise relationship. Lower energy represents that these neurons more diverse and uniformly distributed, while higher energy reflects redundancy. Given weight matrix W E RR xx represented set of N neurons (i.e., kernels), where each row w, E gd+1 corresponds to a neuron, its HE is defined as:
N
N
E Eigs ||00; - أ E
s >>b 0
Ea.d ( 1
Σ
Σ
fa (1) -w,l))
(5)
Ey log [luù; w
== 0
==
where || denotes Euclidean distance. f.(-) is a decreasing real-valued function, and vù;
wi
is the i-th neuron weight projected onto the unit hypersphere ad {w Rd+1 ||w||| 1}. We
also denote Wx {wi úŌ Sd), and E. E (w; ( for short. There are plenty of choices for fa(.), but in this paper we use fz(2) = z =~ > O. known as Riesz s-kernels. Since
each W; lies on the unit hypersphere, the squared Euclidean distance between neurons can be
equivalently expressed in angular form as ||
2 = 2(1 cos f.y), where fgy is the angle
between w. and ú. Substituting this into Eqn.
5
we have:
N
N
Ex.d (w: (NN = Σ
Σ
(2(1 cos) -0/2
(6)
=
This angular formulation highlights the geometric interpretation of HE: higher value corresponds to neuron clustering with low angular diversity, while a lower value reflects a more uniform angular distribution across the hypersphere.
3

---

Published as a conference paper at ICLR 2026
FT
ROME
MEMI
800
2E13 1504 200
70
13 Aar-AA
1804
Edr Namarr
103
1856
2558
2090
EcilSexbie
AlphaEdit
20
53 Arcce
+25
>0
HE
Efficacy
Goneralization
Specilicity
Figure 2: Trends of HE and editing performance throughout sequential editing. The Spearman correlation scores between HE and each editing metric displayed at the end of each curve.
CORRELATION BETWEEN HYPERSPHERICAL UNIFORMITY AND EDITING
HE and model editing are intrinsically connected through their shared focus the geometry of high-dimensional parameter spaces. An optimal HE corresponds to more uniformly distributed rep- resentations the unit hypersphere, typically reflecting well-conditioned parameters that enable reliable and stable sequential editing. We first present empirical evidence revealing a strong cor- relation between HE and editing stability (Section 3.T. followed by a formal theoretical analysis establishing the mathematical link between the two (Section 3.2
3.1 EMPIRICAL ANALYSIS OF THE HE-EDITING STABILITY CORRELATION
To understand the failure modes of large-scale sequential editing, we examined how HE evolves throughout the editing process. We performed 5,000 sequential edits on ZsRE dataset (Levy et al., 2017) with a batch size of 100 on LLaMA3-80 8B (AI@ Metal 2024) using six widely used editing methods, including ThneTTunig (FT) (Zhu et al. 2020), ROME Meng et al. 2022), MEMIT (Meng et al. 2023), RECT (Gu et al., 2024), PRUNE (Ma et al. 2025), and AlphaEdit (Fang et al. 2025) After each edit, we computed the HE of the perturbed weights and evaluated the editing performance using well-established metrics, including Efficacy (edit success), Generalization (paraphrase suc- cess), and Specificity (neighborhood success). Readers refer Appendix [.2 for detailed definition of these metrics. We summarize our main observations as follows:
Observation Collapse in sequential editing is closely tied to sharp fluctuations in HE. Fig- ure 2 reveals a strong correlation between HE dynamics and editing performance. The Spearman correlation scores (Spearman 1904) between HE and each editing metric, displayed at the end of each curve, consistently indicate strong statistical dependence before model collapse"] Most meth- ods collapse well before 3,000 edits, whereas AlphaEdit demonstrates the strongest long-term edit- ing capacity with the best preservation of hyperspherical uniformity. A closer examination of the metrics shows a consistent pattern in which each drop in performance is consistently accompanied by rapid shifts in HE, underscoring its central role in maintaining sequential editing stability.
Observation 2: Advanced editing methods suppress HE fluctuations effectively. Figure il- lustrates the correlation between changes HE (AHE) and editing performance (AAcc.), where each point denotes the difference between two consecutive hatch edits: points near the origin in- dicate greater stability with minimal variation both HE and accuracy, while points farther away reflect larger fluctuations and less stable editing. Most advanced methods exhibit tightly clustered distributions near the origin, indicating stable editing dynamics and minimal weight distortion. Fur- thermore, we fit linear regression over all points across metrics, which demonstrates statistically
'Since FT and ROME rapidly collapse at the very heginning, instead emphasize their correlations by examining the curve fluctuations.
4

---

Published as a conference paper at ICLR 2026
Eflicacy
Generalization
Specifcily
ZAccsras
AlphaEdit
Confiblacce
Figure 3: Correlation between changes in HE and editing performance across consecutive edited weights. Each point corresponds to AHE-A Acc. pair for method over five thousand sequen- tial edits. Confidence ellipses and regression lines illustrate overall trends.
significant positive correlation between AHE and Aec. in terms of Efficacy and Generalization. This suggests a strong positive correlation between editing stability and HE stability, implying that the effectiveness of SOTA approaches may stem from their ability suppress HE fluctuations.
3.2 THEORETICAL ANALYSIS OF HE's IMPACT ON EDITING STABILITY
We further turn to a theoretical analysis of how HE impacts editing stability, aiming to provide principled explanation for the patterns observed in practice. Given the editing objective Eqn. 2 inevitably perturbs the original pretrained knowledge in LLMs, which can be expressed as:
AV (W + AWW K; V1 = AWK,,
where WK, Vi (cf. Eqn. D K and VI represent the new editing knowledge. Additionally, from the HE definition in Equ. S the change HE after editing can be written as:
AHE Σ (1e,, w 1r,, + Aw, - w - A.
where Am; denotes the perturbation wi- This term AHE measures how angular separation among weight vectors changes after editing.
Our theoretical analysis, detailed in Appendix C.I culminates in key result that formally links the geometric change in weight space AH E to the output perturbation AV, derived from Proposition Theorem (Lower Bound on Output Perturbation). Under the assumptions of orthonormal inputs and small perturbations, the output perturbation VV is lower-bounded by squared change in HEE:
1/2
AHE K
AV
K = 4
Σ
1 jAtk
wy w
(9)
where is constant dependent on the original weight matrix geometry.
This theorem reveals key insight: the change in HE (AHE) E)) inevitably induces substantial output perturbation (AV), meaning that edits that significantly distort the geometric arrangement of neurons are bound corrupt pretrained knowledge. This result provides solid thcoretical founda- tion for our empirical findings and underscores HE as a fundamental indicator of editing stability.
4 SPHERE
On account the above findings, we argue that ideal sequential editing should preserve the hyper- spherical uniformity of edited weights. Accordingly, introduce SPHERE, an HE-driven regular- ization strategy designed to mitigate HE fluctuations while integrating new knowledge.
SPHERE first estimates the principal hyperspherical directions of pretrained knowledge and then defines their orthogonal complement as the sparse space. Projecting editing perturbations onto this space enables knowledge injection while minimizing interference with original knowledge.
5

---

Published as a conference paper at ICLR 2026
Principal Space Estimation To identify the principal hyperspherical directions in W, seek unit vector U E R that maximizes the variance of all neurons in W when projected onto v as:
v arg max |
( arg max (1 (WT))))
(10)
|
According to the Rayleigh quotient theory (Horn & Johnson, 1985 Parlett 1998), the maximum of Le W W corresponds to the largest eigenvalue X of 1 W. with the associated eigenvector 8" as the principal direction. Extending this to the top-r principal directions enables us to capture richer low-dimensional space of the weight geometry, so we collect the eigenvectors associated with the r largest eigenvalues to form the principal space matrix, which can be expressed as:
U (dd-rth..... val E Rd
(11)
where r satisfies Ei d 2 Ex= di with the cumulative ratio (see Appendix D.4.1
Sparse Space Definition
This space is defined as the orthogonal complement of U in Eqn.
P
aUU E Rddd
(12)
where a controls the suppression strength of the components along the subspace spanned by U (see Appendix D.4.I. Specifically, a = 1 corresponds to a hard orthogonal projection that completely removes the contribution of U, while 0 < a yields a soft projection that only attenuates it.
Sparse Space Projection Given a perturbation matrix AW produced by any editing method, we project it onto the sparse space using Pi, and then combine it with the original weight matrix
W W AW pnoj W AWPI
(13)
In summary, SPHERE suppresses perturbations aligned with the principal weight directions to pre- serve hyperspherical uniformity, enabling more stable, longer-lasting performance without compro- mising general abilities. For thcoretical completeness, we also provide a mathematical proof that SPHERE suppresses the AH E, ensuring bounded variations in the hidden representations AV
and justifying its effectiveness during editing (see Appendix C.2) More details Appendix D.5
EXPERIMENTS
In this section, we aim to address the following research questions:
RQ1: How does SPHERE perform on sequential editing tasks compared to baseline methods?
RQ2: Can SPHERE effectively preserve the hyperspherical uniformity of edited weights?
RQ3: How does SPHERE-edice LLMs perform general ability evaluations?
RQ4: Can baseline methods be significantly improved with plug-and-play SPHERE?
RQ5: What is the computational overhead of SPHERE in terms of time efficiency?
RQ6: How do different hyperparameter choices affect the stability of SPHERE?
5.1 EXPERIMENTAL SETUP
Base LLMs and Baseline Methods. Experiments were conducted on LLaMA3 (8B) (Al@ Meta
2024) and Qwen2.5 (7B) (Team, 2024). We compared our approach against a range of representative
sequential editing baselines, including Fine-Tuning (FT) (Zhu et al. 2020), MEMIT (Meng et al.)
2023 RECT (Gu al. 2024), PRUNE (Ma al. 2025). and AlphaEdit (Fang al. 2025).
Datasets and Evaluation Metrics. Two widely used benchmarks were adopted: Counter- Fact (Meng et al. 2022) and ZsRE (Levy et al. 2017). Following prior work (Meng et al., 2022), five evaluation metrics were reported: Efficacy (edit success), Generalization (paraphrase suc- cess), Specificity (neighborhood success), Fluency (generation entropy), and Consistency (refer- ence score). For rigorous evaluation, we adopt the average top-1 accuracy as the metric for both datasets. Readers can refer to Appendix D for more detailed experimental setup.

---

Published as a conference paper at ICLR 2026
Table 1: Comparison of SPHERE with existing methods on sequential editing. Eff., Gen., Spe., Flu. and Consis. denote Efficacy, Generalization, Specificity, Fluency and Consistency, respectively. The best results are highlighted in bold, while the second- best results are underlined.
ZSRE
Counterfact
Method
Model
EMT,
Gen.f
Spe.7
E
Gen.t
Spe.1
Flu.f
Consis.
Pre-edited
35.42,0.00
34.17.9.00
38.02g0.7
0.49
a..4
18.09
634.84gam
22.061
FT
15.27#021
14.78+321
5.06
8.40
2.54
0.03=001
409.80=mm7
19.35
LLaMA3-89
MEMIT
0.000
0.00
0.06200
0.00
0.00
0.00
318.192024
4.19:004
PRUNE
10 18
10.08
95
1.19
0.34
0.62
618,72aaum
49124
RECT
0.01
0.01=000
0.04aam
+.57
0.29
0.10
438.83=a115
9,40a00s
AlphaEdit
86.64.0.23
81.28
28, 28.7822222
4.37
1.71
0.57
482.36am
4.71
SPHERE
90.01
84.67
45.40.:..0
:2.89:
32.07
551.51
30.89
Pre-edited
35.29
34.10
38.44
0.42:00.6
0.46
15,06
624,45=a.1
23.02+am
FT
4.01
4.58
1.01
1.5444g
4.63
1.46200s
214.26:0m
3.15
Qwen2.5-78
MEMIT
0.13
0. 0.12:n01
0.04gmm:
0.00
0.00
0.00g000
370.842n99 90
3.59
PRUNE
47, 47.93
45.50
39,20
14.30
11,27
6.75=0012
620.74:n.1
29.50
RECT
0.73
0.75=0004
0.05
0.64
0.19
0.09
368
368.46:222
1.35
AlphaEdit
4:.01
39.99
13.87
43. 0:
24.37
2.32:006
479.832077
4.67
SPHERE
70.04:4.56
65.43
60.766:u.29 27.35
29.24
3,83
612.67
14.74
5.2 PERFORMANCE OF SEQUENTIAL MODEL EDITING (RQ1)
Table presents the results under a commonly used sequential editing setup, using 15,000 samples with 100 edits each for LLaMA3 (8B), while Qwen2.5 (7B) is restricted to 5,000 edits as further updates induce severe model collapse, where all editing methods underperformed compared to the pre-edit baseline. SPHERE plug-and-play, and AlphaFddit (Fang et al., 2025) is adopted as the default base method in Table T to better illustrate its capability. Additional experiments with other base methods are presented in Section [5.5 Overall, SPHERE consistently outperforms baseline methods across nearly all metrics and base models. In particular, it achieves substantial gains in both Efficacy and Generalization, with average improvements of 24.19% and 16.02%. % respectively, over the best baseline. It also maintains notable performance in Fluency and Consistency, indicating its ability to preserve factual accuracy while generating coherent and natural outputs.
5.3 ANALYSIS OF EDITED WEIGHTS (RQ2)
This analysis evaluates whether SPHERE can effectively maintain the hyperspherical uniformity of edited weights. We extracted the edited weights from LLaMA3 after 15,000 sequential edits on CounterFact. As shown in Figure 4 we computed the cosine similarity between each pair of weight neurons and used heatmap to visualize the hyperspherical uniformity before and after editing. In Figure 5, t-SNE (van der Maaten & Hinton (008) was used to visualize the normalized neuron distribution in W before and after editing. It can be seen that SPHERE effectively preserves hy- perspherical uniformity after editing, as the cosine similarity among weight neurons remains close to the original distribution, thereby avoiding directional collapse. Moreover, the pre- and post-edited weights exhibit nearly overlapping distributions, indicating that SPHERE prevents sig- nificant shifts in weights and maintains consistency. In contrast, baseline methods such as MEMIT and AlphaEdit induce clear angular concentration neuron directions, causing neurons to cluster in limited angular regions and significantly reducing hyperspherical directional uniformity. More results on Qwen2.5 are in Appendix E.1
5.4 EVALUATION OF GENERAL ABILITIES (RQ3)
To extensively evaluate whether post-edited LLMs can preserve the general abilities, four represen- tative tasks were adopted following Gu et al. (2024), including Reasoning on the GSM8K (Cobbe)
et al. 2021), measured by solve rate. Natural language inference (NLD) on the RTE (Dagan al.)
2005
measured by accuracy of two-way classification, Open-domain QA on the Natural Ques-
tions (Kwiatkowski et al. 2019), measured by exact match (EM) with the reference answer after
7

---

Published as a conference paper at ICLR 2026
Originel
MEMIT
AlpbmFdil
SPHERE
H
Figure Cosine similarity between neurons in the updated weight matrix after 15,000 edits. Darker colors indicate lower similarity, reflecting better hyperspherical and orthogonal uniformity. SPHERE effectively preserves the weight structure, demonstrating the most stable hyperspherical uniformity.
Pre edited
Post edited
MEMIT
RECT
AlphaFdit
SPHERR
Figure 5: The t-SNE distribution of weight neurons of pre-edited and post-edited LLM after 15,000 edits using dimensionality reduction. The top and right curve graphs display the marginal distribu- tions for two reduced dimensions, where SPHERE consistently exhibits minimal shift.
minor normalization (Chen al. 2017 Lee et al. 2019). Closed-domain QA BoolQ (Clark ett al. 2019), also measured by EMM Figure 6 depicts how performance varies with the number of edited samples across four tasks. We report general performance every 1 edits up 5k, and ev- ery 5k edits thereafter (up to 15k), providing comprehensive view of the degradation trend. The results show that SPHERE effectively preserves the general abilities of post-edited LLMs even un- der extensive editing, maintaining the original model performance across all metrics after 15k edits. In contrast, L.MMs edited with baseline methods rapidly lose their general abilities with all met- rics approaching zero. These findings underscore the critical role of hyperspherical uniformity in safeguarding the broad abilities learned from the underlying corpus.
5.5 PERFORMANCE IMPROVEMENTS OF BASELINE METHODS (RQ4)
We investigated whether the sparse space projection strategy of SPHERE can serve as general en- hancement existing methods. A single line of code from SPHERE regarding the projection was inserted into the baselines with minimal modification, and we evaluated their performance before and after integration (as detailed in Appendix D.5 Results of 3,000 sequential edits LLaMA3 (8B) are reported in Figure 7 SPHERE is integrated seamlessly with diverse editing methods and significantly boosts their performance. On average, the optimized baselines achieve relative improvements of 49.05 %. 42.64 % and 24.44% % in Efficacy, Generalization, and Specificity, re- spectively, underscoring the strong potential and broad applicability of the proposed sparse space projection as plug-and-play enhancement for model editing. The baselines enhanced with the projection also demonstrate significantly better robustness in general abilities (see Appendix E.2
5.6 ANALYSIS OF COMPUTATIONAL OVERHEAD (RQ5)
To evaluate the scalability of SPHERE, we analyze its computational cost relative to the base editing method as both the model size and the number sequential ed- its increase. Specifically, projection step introduced by SPHERE and the overall batch-editing process. Table 2 reports these runtimes
Table 2: Runtimes of SPHERE.
Model
Edit (s)
Proj. (s)
Ratio (%)
measure the runtime of the
LJaMA3-8B
543.26
18.00
3.31
Qwen2.5-7B
535.73
35.95
6.71
Qwen2,5-32B
1656.58
99.60
6.01
8

---

Published as a conference paper at ICLR 2026
Reasoning
(b)
[e] Opennd QA
Closed-domsin QA
Pertrrmnnc
MEMIT
PRUNE
RECT
AlphaFdit
SPHERE
Figure 6: General ability testing of post-edited LLaMA3 (8B) on four tasks.
MEMIT
PRUNE
Accuracy
Accurany
60
20
ERT.
Gen.
Spe.
Gon.
Spe.
EH
Cen.
Spe.
(a) Performance Improvement on Counterfact Dataset
MEMIT
PRUNE
RECT
Accuracy
Accurecy
Accuray
8
60
60
ETT.
Gen.
Spe.
E1T.
Gen.
Spo.
Bm1
Gen.
(b) Performance Improvement on ZsRE Dataset
Figure 7: Performance improvements of baseline editing methods after adding single line of code from SPHERE (i.e., sparse space projection). Gray bars denote the original baseline performance, while purple bars indicate the performance after enhancement.
together with the projection-to-totee time ratio, evaluated under 3,000 sequential edits using MEMIT as the base method, with batch size of 100. The results indicate that the projection step introduces only minor relative cost, accounting for 3.31 % of the total editing time for I.LaMA3 (8B), 6.71 % for Qwen2.5 (7B), and 6.01 % for Qwen2.5 (32B). Moreover, the relative computational cost of SPHERE remains consistent across models of different scales within the same family, suggest- ing that the additional projection operation is lightweight and does not hinder the practicality of SPHERE large-scale batch sequential editing scenarios.
5.7 SENSITIVITY TO HYPERPARAMETERS (RQ6)
As analyzed in Section 3 different editing methods exhibit varying magnitudes of HE fluctuation. Since SPHERF explicitly regulates hyperspherical energy ac- cumulation through projection, its behavior is primar- ily governed by the cumulative ratio and the sup- pression strength a (introduced in Appendix D.4.T).
10.63
Cumulative Ratio
0.3
10.63
26.06
27.60
$3.06
60
is therefore important to evaluate the sensitivity of SPHERE to these two hyperparameters. To this end,
450
10.63
conducted an expanded ablation study that system- atically varies both hyperparameters over a wide range, using MEMIT as the base method on LLaMA3 (8B) with 3,000 sequential edits, while keeping all other set- tings consistent with the main experiment. Figure we evaluate editing performance and visualize the results using heatmap to illustrate the performance variation before and after applying SPHERE under different combinations of (n, a). Although performance fluctuations are observed under different hyperparameter settings, across all configu- rations, SPHERE consistently improves the performance of its original counterpart, demon- strating that SPHERE remains effective and robust over a wide range of hyperparameter choices. Practitioners may therefore tune these hyperparameters suit their specific baseline or
30
022
10.63
22.50
29.03
89.00
30
shown in
0.
Suppression Strength
Figure 8: Sensitivity of SPHERE.

---

Published as a conference paper at ICLR 2026
6 RELATED WORK
Model Editing Methods. From the perspective of whether model parameters are modified, exist- ing approaches can be broadly categorized into parameter-moodifingg (Mitchell et al. 2022 Meng al. 2023; Ma al. 2025| Fang et al. 2025). which directly adjust a small subset of model pa- rameters, and ecrrameter-rrsservinn (Zheng et al. [023 Yu et al. 2024 Hartvigsen et al. (023),
which integrate auxiliary modules without altering the original weights. this work, we focus on parameler-mmddifing methods which typically employs meta-learning or Seating-then-editing
strategies (Zhang et al. 2024b). Representative works of meta-learning include KE (Cao et al,, 2021 and MEND (Mitchell et al. 2022). which leverage hypemetworks to generate parameter up- dates. Locate-then edit methods, such as ROME (Meng et al.] 2022) and MEMIT (Meng et al.] 2023 + prioritize pinpointing the knowledge's storage location before making targeted edits. Recent extensions like RECT (Gu et al.] (022)) and PRUNE (Ma al. 2025) mitigate degradation of general capabilities of LLMs by better constraining edit complexity via sparsity and condition number. Re- cently, AlphaEdit (Fang et al. 2025) further generalizes this paradigm by projecting the perturbation
into the nullspace of the previous knowledge set.
Learning with Hyperspherical Uniformity. Early studies (Xie et al. 2017a Rodríguez et al., 2017 Xie et al., 2017b Cogswell al. 2016) sought to improve the generalization capacity of neu- ral networks by reducing redundancy through diversification, as rigorously analyzed in (Xie et al.) 2016). Although these works examined angular diversity, they largely neglected the notion of global equidistribution of embeddings on the hypersphere. In contrast to orthogonality, where perpen- dicular vectors are defined to be diverse, hyperspherical uniformity promotes embeddings that are maximally separated in angle, thereby encouraging uniform distribution across the hypersphere (Liu jet al,, 2021 2018). More recently. [Smerkous et al. (2024) enhancing training stability by incorporat- ing centered kernel alignment into hyperspherical energy, enhancing training stability by addressing the lack of permutation invariance inherent in naive similarity metrics.
CONCLUSION
In this work, we demonstrated that hyperspherical uniformity is critical factor in stabilizing se- quential editing for LLMss, supported by both empirical evidence and rigorous theoretical proof. Motivated by this insight, we propose SPHERE, regularization strategy that preserves hyper- spherical uniformity by projecting updates onto a space complementary to the principal directions of pretrained weights. Extensive evaluations on LLaMA3 (8B) and Qwen2.5 (7B) across multiple editing datasets and downstream tasks confirm that SPHERE not only enhances editing performance by 16.41% % over the strongest baseline but also more faithfully preserves weight geometry and gen- eral abilities of models. Furthermore, when applied as a plug-and-play enhancement, it yields an additional average improvement of 38.71% % across existing methods. Collectively, our findings tablish SPHERE as both theoretically grounded and empirically effective, providing principled and scalable solution for reliable large-scale model editing.
ETHICS STATEMENT
SPHERE significantly enhances the reliability of large-scale sequential model editing by preserving hyperspherical uniformity, which makes it valuable way for updating and managing knowledge in real-world applications where long-term stability is essential. At the same time, the ability to directly alter stored knowledge in L.MS carries inherent risks, including the potential introduction of bias harmful information. To address these concerns, procedures, transparent reporting, and strict oversight when deploying such techniques. While the core motivation of SPHERE is positive, aiming to facilitate efficient and trustworthy updates of LLMs, we emphasize that its use must remain responsible and cautious to ensure ethical outcomes.
strongly recommend rigorous validation
We used LLMs to assist with improving grammar, clarity, and wording in parts of this work. The use of LLMs was limited to language refinement, with all ideas, analyses, and conclusions solely developed by the authors. We restate this announcement in Appendix A
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGMENTS
This research based upon work supported by NSF CAREER #2339766 and an Amazon AGI Research Award. We would like to express gratitude to the UCLANLP group members for their valuable feedback.
REFERENCES
A Meta. Llama model card, 2024, URL
11ama / 11ama3/
blob/me I1MODEII_CARR md
5.1
Nicola De Cao, Wilker Aziz, and Ivan Titov. Editing factual knowledge in language models. In Marie-Francinee Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih (eds.), Proceed- ings the 2021 Conference on Empirical Methods in Natural Language Processing, EMNLP 2021, Virtual Event / Punta Cana, Dominican Republic, 7-11 November, 2021, pp. 6491-6506. Association for Computational Linguistics. 2021. doi: 1/2223//12022..222222 wmwp:///ww.2222m2m.....mm.mm.... -aallpppmai..mm in. 522
URL https: org.
Danqi Chen, Adam Fisch, Jason Weston, and Antoine Bordes. Reading wikipedia to answer open- domain questions. In Regina Barzilay and Min- Kan (eds.), Proceedings of the 55th Anmual Meeting of the Association for Computational Linguistics, ACL 2017, Vancouver, Canada. July 30 August 4, Volume I: Long Papers, pp. 1870-1879. Association for Computational Linguistics, 2017. doi: 10. 00.18653/V1/P17-1177 URL htt.h:ttp:://tt org 11t.1/111111//1111111111 5.4 E.2
Christopher Clark, Kenton Lee, Ming-Wos Chang, Tom Kwiatkowski, Michael Collins, and Kristina Toutanova. Boolq: Exploring the surprising difficulty of natural yes/no questions. In Jill Burstein, Christy Doran, and Thamar Solorio (eds.), Proceedings of the 2019 Conference of the North Amer- ican Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-IILT 2019, Minneapolis, MN, USA, June 2-7, 2019, Volume 1 (Long and Short Papers), pp. 2936. Association for Computational Linguistics, 2019. doi: (0.11653/VV/N11-113 E.2
5.4
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems. CoRR, abs/2110. 2021. URL
https::/arxiv> otg. abs /2110 14168
5.4
E.2
Michael Cogswell, Faruk Ahmed, Ross B. Girshick, Larry Zitnick, and Dhruv Batra. Reducing overfitting in deep networks by decorrelating representations. In Yoshua Bengio and Yann LeCun (eds.), 4th International Conference Learning Representations, ICLR 2016, San Juan, Puerto Rico, May 2-4, 2016, Conference Track Proceedings. 2016. URL http: /ttt:i///axiv..o. /tw.ivivvv.. org/abs/ 1.11106063 6
Ido Dagan, Oren Glickman, and Bernardo Magnini. The PASCAL recognising textual entail- ment challenge. In Joaquin Quiñonero Candela, Ido Dagan, Bernardo Magnini, and Florence d'Alché-Bil Buc (eds.), Machine Learning Challenges, Evaluating Predictive Uncertainty, Visual Object Classification and Recognizing Textual Entailment, First PASCAL Machine Learning Challenges Workshop. MLCW 2005, Southampton, UK. April 11-13, 2005, Revised Selected Pa- pers, volume 3944 of Lecture Notes in Computer Science, pp. 177-190. Springer, 2005. doi:
10.1007/11336790 99. URL htt.s:ttps/// /tt::///dooo htttpp//oool..rrg/0000000////
9
5.4
E.2
DecpSeek-AI, Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Cheng- gang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, Damai Dai, Daya Guo, Dejian Yang, Deli Chen, Dongjie Ji, Erhang Li, Fangyun Lin, Fucong Dai. Fuli Luo, Guangbo Hao, Guanting Chen, Guowei Li, H. Zhang, Han Bao, Hanwei Xu, Haocheng Wang. Haowei Zhang, Honghui Ding, Huajian Xin, Huazuo Gao, Hui Li, Hui Qu, J. L. Cai, Jian Liang, Jianzhong Guo, Ji- aqi Ni, Jiashi Li, Jiawei Wang. Jin Chen, Jingchang Chen, Jingyang Yuan, Junjie Qiu, Junlong Li, Junxiao Song, Kai Dong, Kai Hu, Kaige Gao, Kang Guan, Kexin Huang, Kuai Yu, Lean Wang, Lecong Zhang, Lei Xu. Leyi Xia, Liang Zhao, Litong Wang. Liyue Zhang, Meng Li, Miaojun Wang, Mingchuan Zhang, Minghua Zhang, Minghui Tang, Mingming Li, Ning Tian,
11

---

Published as a conference paper at ICLR 2026
Panpan Huang, Peiyi Wang, Peng Zhang, Qiancheng Wang, Qihao Zhu, Qinyu Chen, Qiushi Du, R. J. Chen, R. L. Jin, Ruiqi Ge, Ruisong Zhang. Ruizhe Pan, Runji Wang, Runxin Xu, Ruoyu Zhang, Ruyi Chen, S. S. Li, Shanghao Lu, Shangyan Zhou, Shanhuang Chen, Shaoqing Wu, Shengfeng Ye, Shirong Ma, Shiyu Wang, Shuang Zhou, Shuiping Yu, Shunfeng Zhou, Shuting Pan, T. Wang. Tao Yun, Tian Pei, Tianyu Sun, W. I.. Xiao, and Wangding Zeng. Deepseek- v3 technical report. CoRR, abs/2412. 19437, 2024. doi: 10.48550/RXIVV241222 19437. URL https: http://doo...gggg
48550 arxii 2412 19437
Junfeng Fang, Houcheng Jiang, Kun Wang, Yunshan Ma, Jie Shi. Xiang Wang. Xiangnan He, and Tat-Seng Chua. Alphaedit: Null-space constrained knowledge editing for language models. In The Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, April 24-28, 2025. OpenReview.net, 2025. URL URL.E::ss://ppereei.ee http:///pppeevveeeeee net /wt//fertoooooooo
HvSytvq3Jh
1
5.1
5.2 6 D.5
Jia-Chen Gu, Miann Xu, Jun- Yu Ma, Pan Lu, Zhen-Hua Ling, Kai-Wei Chang, and Nanyun Peng. Model editing harms general abilities of large language models: Regularization to the rescue. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen (eds.). Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, EMNLP 2024, Miami, FL, USA, November 16, 2024, pp. 16801-16811 19. Association for Computational Linguistics, 2024. doi: 10.18653/V WAW.://2222EEL-MAMNN...MM
URL httpp:://do..rrg/
v1 v1/2024. emn pp ma mam...nnn....mm.....m........m in.iinn... 934
0
3.1 5.1
5.4
Akshat Gupta, Anurag Rao, and Gopala Anumanchipalli. Model editing at scale leads to gradual and catastrophic forgetting. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar (eds.). Findings of the Association for Computational Linguistics, ACL 2024, Bangkok, Thailand and virtual meet- ing, August 11-16, 2024, pp. 15202-15232. Association for Computational Linguisties, 2024. URL bttps://doil. 1ttp://ii...0/11111111111 186533/vv1.66
doi:
2. F024.findnggs-aae acl. 902
Tom Hartvigsen, Swami Sankaranarayanan. Hamid Palangi, Yoon Kim, and Marzyeh Ghassemi. Aging with GRACE: lifelong model editing with discrete key-value adaptors. In Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine (eds.), Advances in Neural Information Processing Sysiems 36: Annual Conference on Neural In- formation Processing Systems 2023, NeurlPS 2023, New Orleans, LA, USA, December 10 16, 2023, 2023. URL http: /pap::s///apppaaape nips cc/paper Files//ppper22223 hhash/ org.a//csaae...aa...a.....aaaaar html 6
Roger A. Horn and Charles R. Johnson. Matrix Analysis. Cambridge University Press, 1985. ISBN 0-521-30586-1.
Lei Huang, Weijiang Yu, Weitao Ma, Weihong Zhong, Zhangyin Feng, Haotian Wang, Qianglong Chen, Weihua Peng, Xiaocheng Feng. Bing Qin, and Ting Liu. A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions. ACM Trans. Inf. Syst. 43: 2025. doi: 10.1145/3703155. URL https:ts: / http:/dcc.ooo///////////
3703155
I
Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Yejin Bang, Andrea Madotto, and Pascale Fung. Survey of hallucination in natural language generation. ACM Comput. Surv, 55(12) 24:11244:38 1-248:38, 2023. doi: 10.1145/3571730. URL https::/ orq///.
#145/3571730
Houcheng Jiang, Junfeng Fang, Tianyu Zhang, Baolong Bi, An Zhang, Ruipeng Wang, Tao Liang, and Xiang Wang. Neuron-level sequential editing for large language models. In Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar (eds.), Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume I: Long Pupers), ACL Vienna, Austria, July 27 August 1, 2025, pp. 16678 16702. Association for Computa- tional Linguistics, 2025. URL httpt::////// https:://acaaatthoooooooo /Rt.ttppe///acaattto.....// org/ 2025 acl- -oog.ooon....mm 815/
Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur P. Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, Kristina Toutanova, Llion Jones, Matthew Kelcey, Ming- Wei Chang, Andrew M. Dai, Jakob Uszkoreit, Quoe Le, and Slav
12

---

Published as a conference paper at ICLR 2026
Petrov. Natural questions: benchmark for question answering research. Trans. Assoc. Comput.
Linguistics, 7:452 466, 2019. doi: 10.
URL htttp:://cc..coom
10.1162/tacl 00276
5.4
E.2
Kenton Lee, Ming-Wei Chang, and Kristina Toutanova. Latent retrieval for weakly supervised open domain question answering. Anna Korhonen, David R. Traum, and Lluís Mårquez (eds.), Proceedings of the 57th Conference of the Association for Computatioonal Linguistics, ACL 2019, Florence, Italy, July 28- August 2. 2019, Volume 1: Long Papers, pp. 6086-6096. Association for Computational Linguistics. 2019, doi: 10. N..16533VLL/I11 1612. CtRLhttps://// https::/doi..rss
10. 10.18653/v1/p1--11
5.4
Omer Levy, Minjoon Seo, Eunsol Choi, and Luke Zettlemoyer. Zero-shot relation extraction via reading comprehension. In Roger Levy and Lucia Specia (eds.), Proceedings of the 2 Confer- ence on Computational Natural Language Learning (CoNLI. 2017), Vancouver, Canada, August doi: 10. 10.18653/V1/
3-4, 2017, pp. 333-342. Association for Computational Linguistics, 2017.
K17- 1034. URL https: wrw.orggg//... 18653/v1/K17-11112
5.1 D.I
Zherui Li, Houcheng Jiang, Hao Chen, Baolong Bi, Zhenhong Zhou, Fei Sun, Junfeng Fang, and Xiang Wang. Reinforced lifelong editing for language models. In Aarti Singh, Maryam Fazel, Daniel Hsu, Simon Lacoste-Julic Felix Berkenkamp, Tegan Maharaj, Kiri Wagstaff, and Jerry Zhu (eds.), Forty-second International Conference on Machine Learning, ICML 2025, Vancouver, BC, Canada, July 13-19, 2025, volume 267 of Proceedings of Machine Learning Research. PMLR / OpenReview.net, 2025. URL https: /ttp:://proceee http://ppocccce...ec/eec/o ml press/ v267/1125an. html
Weiyang Liu, Rongmei Lin, Zhen Liu, Lixin Liu, Zhiding Yu, Bo Dai, and Le Song. Learn ing towards minimum hyperspherical energy. In Samy Bengio, Hanna M. Wallach, Hugo Larochelle, Kristen Grauman, Nicold Cesa-Bianchi, and Roman Garnett (eds.), Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, NeurIPS 2018, December 3-8, 2018, Montréal, Canada, pp.
2225-62366 2018. URL https::/pproceeeingg... neurips........ppppp............ CC pttppppppp//aaae/eeeee 177540c7bcb8dhh 69 http://66661222ececcc Abst htt.1/1110.0002222222a--aaaaaaa. html
6
Weiyang Liu, Rongmei Lin, Zhen Liu, Li Xiong, Bernhard Schölkopf. and Adrian Weller. Learn ing with hyperspherical uniformity. In Arindam Banerjee and Kenji Fukumizu (eds.). The 24th International Conference on Artificial Intelligence and Statistics, AISTATS 2021, April 13-15, 2021, Virtual Event, volume 130 of Proceedings of Machine Learning Research, pp. 1180-1188 PMLR, 2021. URL
mir 1ttesss/110//1u22d.hhaha
Jun-Yu Ma. Hong Wang, Hao-Xiane Mao-Xiang Xu, Zhen Ling, and Jia-Chen Gu. Porturbation-restrain sequential model editing. In The Thirteenth International Conference on Learning Represen- tations, ICLR 2025, Singapore, April 24-28, 2025. OpenReview.net, 2025. URL https:
//openrevies. net wwwpee//ooooii///eeeeeeeeeeeeeee Serefficppppoee 8cp8qmk
3.1 5.1
Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. Locating and editing factual associ- ations in GPT. In Sanmi Koyejo, S. Mohamed, A. Agarwal, Danielle Belgrave, K. Cho, and A. Oh (eds.), Advances in Neural Information Processing Systems 35: Annual Conference on Neural In- formation Processing Systems 2022, NeurIPS 2022, New Orleans, LA, USA, November 28 De- cember 9, 2022, 2022. URL ht tt htt::::/pppaaeeee n nips wwtppicpppppppppr.p.ppppppp paperrappppppppppe ferape_ffieede files/ files/paperrss http:/apeer/eee /2022/ .ttm
hash/ 6 (e.ff144d5.aom 3el.
2.1 3.1
5.1
B D.
D.2.1 D.2.2
Kevin Meng, Arnab Sen Sharma, Alex J. Andonian, Yonatan Belinkov, and David Bau. Mass- editing memory in transformer. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.net, 2023. URL https::/
net Forum?I---MbbcclIyyys
D.2.1
D.2.2 D.4
Meta AI. Introducing llama 4: Advancing multimodal intelligence, 2024. URL https: meta. htt.:cccc/lolllgccc 1ammmm---mull wwmmmimmodaaaa. -intel gence//
13

---

Published as a conference paper at ICLR 2026
Eric Mitchell, Charles Lin, Antoine Bosselut, Chelsea Finn, and Christopher D. Manning. Fast model editing at scale. In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 29, 2022. OvenReview.net, 2022.
net/forum? (itpmmmmdd0000e0eeeiei t.
6 D.2.1
OpenAl. Introducing gpt-5. Online, 2025.
URL
int rnduciinn gpt-5/ aarge language model announcement]. |
Beresford N. Parlett. The Symmetric Ligenvalue Problem. Classics in Applied Mathematics. SIAM,
1998. ISBN 0-89871-402-88 4
Zeju Qiu, Weiyang Liu, Haiwen Feng, Yuxuan Xue, Yao Feng, Zhen Liu, Dan Zhang, Adrian Weller, and Bernhard Schölkopf. Controlling text-to-image diffusion by orthogonal finetuning. In Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko. Moritz Hardt, and Sergey Levine (eds.), Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NewrlPS 2023, New Orleans, LA,, USA, December 10 nips. wwwpppp..pppppppppppppe. waapeerrffleeepppeeeeeeeee pttpe//pppp//////aaaeeee oot.rpaeee-ettttttaat----oerrere html
16, 2023, 2023. URL Faacb7a482788
Pau Rodríguez, Jordi Gonzàlez, Guillem Cucurull, Josep M. Gonfaus, and F. Xavier Roca. Regular- izing cnns with locally constrained decorrelations. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings. OpenReview.net, 2017. URL https:: wttp:///ee.eeeeeeeeeeoee wwt//ett/oooee det.uummmd-yyyssa....
6
David Smerkous, Qinxun Bai, and Fuxin Li. Enhancing diversity in bayesian deep learn- ing via hyperspherical energy minimization of CKA. In Amir Globersons, Lester Mackey, Danielle Belgrave, Angela Fan, Ulrich Paquet, Jakub M. Tomezak, and Cheng Zhang (eds.), Advances in Neural Information Processing Systems 38: Annual Conference on Neural In- formation Processing Systems 2024, NeurIPS 2024, Vancouver, BC, Canada, December 10 15, 2024, 2024. URL httthttpp//ppaael http:////aapppeeppp..cccaaaeeeee coreraereeetttttttttoorrrereeeee html| 6
n httpp/cccpppeeeeileeeeeeeeeeeeep www.efflllssppeeeeee http://sssssppppe///////aaeeeee
@Spearman. The proof and measurement of association between two things. The American Journal
of Psychology, 15(1):72-101, 1904. 3.1
Qwen Team. Qwen2.5: A party of foundation models, September 2024. UtLptttps///wwnnnn
github. hwt.:/iooooog/eeeeeeo /wello///we22.5/ 15.1
Laurens van der Maaten and Geoffrey Hinton.
Visualizing data using t-sne. Journal of Ma- chine Learning Research, 9(86):257--2655 2008 URL http: jmir wtw..oogg/ppees////////////////
html
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, and Jamie Brew. Huggingface's ss transformers: State-of-the-art natural language processing. CoRR, abs/1990.03371, 2019. URL
http://arxiv. orgg abs
D.4
Pengtao Xie, Jun Zhu, and Eric P. Xing. Diversity-promoting bayesian learning of latent variable models. In Maria-Florina Balcan and Kilian Q. Weinberger (eds.). Proceedings of the 33nd In- ternational Conference on Machine Learning. ICML 2016, New York City, NY, USA, June 19-24, 2016, volume 48 of JMLR Workshop and Conference Proceedings, pp. 59-68. JMLR.org, 2016. URL http:::///ooceeeciiiiiic nigs m. www.mmrrrrrrr.. prwss/vessev 8 eal html
Pengtao Xie, Yuntian Deng, Yi Zhou, Abhimanu Kumar, Yaoliang Yu, James Zou, and Eric P. Xing- Learning latent space models with angular constraints. In Doina Precup and Yee Whye Teh (eds.), Proceedings of the 34th International Conference on Machine Learning, ICML 2017, Sydney, NSW. Australia, 6-11 August 2017, volume 70 of Proceedings of Machine Learning Research, pp. 3799-799999 3810 10. PMLR, 2017a. UtL:tt:: httt:::/ppocceedingee mlr. wc.r..peess/v0/xieel. html
1
Pengtao Xie, Aarti Singh, and Eric P. Xing. Uncorrelation and evenness: a new regularizer. In Doina Precup and Yee Whye Teh (eds.), Proceedings of the 34th International
14

---

Published as a conference paper at ICLR 2026
Conference on Machine Learning. ICML 2017, Sydney, NSW. Australia, 6-11 August 2017, vol- ume 70 of Proceedings of Machine Learning Research, pp. 2811-38201 PMLR, 2017b. URL
http://proceediinn www.iinnnmmmm.m...mmmemm.e...eem www.. www.sppesssvvecc xxee html
An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge. Haoran Wei, Huan Lin. Jialong Tang, Jian Yang. Jianhong Tu, Jianwei Zhang. Jian Yang, Jiaxi Yang, Jingren Zhou, Jingren Zhou, Junyang Lin, Kai Dang, Keqin Bao, Kexin Yang, Le Yu, Lianghao Deng. Mei Li, Mingfeng Xue, Mingze Li, Pei Zhang, Peng Wang. Qin Zhu, Rui Men, Ruize Gao, Shixuan Liu, Shuang Luo, Tianhao Li, Tianyi Tang, Wenbiao Yin. Xingzhang Ren, Xinyu Wang, Xinyu Zhang. Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yinger Zhang, Yu Wan, Yuqiong Liu, Zekun Wang. Zeyu Cui, Zhenru Zhang, Zhipeng Zhou, and Zihan Qiu. Qwen3 technical report. CoRR, abs/2505 09388, 2025. doi: (0448550/RXIV.2505.5 0.9388. URL pettps://dof wwg.ooggloooooo.ooo 10. 48550/ arxiv 2505. 09388
Wanli Yang, Fei Sun, Xinyu Ma, Xun Liu, Dawei Yin, and Xueqi Cheng. The butterfly effect of model editing: Few edits can trigger large language models collapse. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Findings of the Association for Computational Linguistics, ACL 2024, Bangkok, Thailand and virtual meeting, August 11-16, 2024. pp. 5419-54193 0--35.7330000000 Associ- ation for Computational Linguistics, 2024a. doi: https://do1 .rgg / 10. /8336/ vl / 2024.00222.222 fi inniininn----.......n ac 322
ACL.3222 URL B
Wanli Yang, Fei Sun, Jiajun Tan, Xinyu Ma, Du Su, Dawei Yin, and Huawei Shen. The fall of ROME: understanding the collapse of Iims in model editing. In Yaser Al-Onaizan, Mo- hit Bansal, and Yun-Nung Chen (eds.), Findings of the Association for Computational Linguis- fies: EMNLP 2024, Miami, Florida, USA, November 12-16, 2024, pp. 4079-4087, Association for Computational Linguistics, 2024b. doi: 10. ht httpps://de org/oo..//ooo /00 1...8866333 vl 2024. 1.2.4...iiiii wwwdiiiininnnii.n..
URL
236
B
Lang Yu, Qin Chen, Jie Zhou, and Liang He. MELO: enhancing model editing with neuron- indexed dynamic lora. In Michael J. Wooldridge, Jennifer G. Dy, and Sriraam Natarajan (eds.). Thirty- AAAI Conference on Artificial Intelligence, AAAI 2024, Thirty-Sixth Con- ference on Innovative Applications of Artificial Intelligence, IAAI 2024, Fourteenth Symposium Educational Advances in Artificial Intelligence, EAAI 2014, February 20-27, 2024, Vancou- ver, Canada, pp. 19449-19457, AAAI Press, 2024. doi: 10. N0.109/AAALLV33117 1609 aaai v3811311111 1788117.20 299116
URL
https:
6
Ningyu Zhang. Zekun Xi, Yujie Luo, Peng Wang, Bozhong Tian, Yunzhi Yao, Jintian Zhang, Shumin Deng, Mengshu Sun, Lei Liang, Zhiqiang Zhang, Xiaowei Zhu, Jun Zhou, and Hua- jun Chen. Oneedit: A neural-symbolic collaboratively knowledge editing system. In Proceed- ings of Workshops at the 50th International Conference on Very Large Data Bases, VLDB 2024, Guangzhou, China, August 26-30, 2024. VLDB.org, 2024a. URL https: //vldb///llbb.. org/ feerkshoss/2224/preeeet ins LLLLLL/LKKKK/MKKK pdf
B
Ningyu Zhang, Yunzhi Yao, Bozhong Tian, Peng Wang, Shumin Deng, Mengru Wang. Zekun Xi, Shengyu Mao, Jintian Zhang, Yuansheng Ni, Siyuan Cheng, Ziwen Xu, Xin Xu, Jia-Chen Gu, Yong Jiang, Pengjun Xie, Fei Huang, Lei Liang, Zhiqiang Zhang, Xiaowei Zhu, Jun Zhou, and Huajun Chen. A comprehensive study of knowledge editing for large language models. CoRR, doi: 48555//ARXIV.2401 102.001112200 .01286... htLhstts::/ //doi orgp//0.g111.. 48550/arxiv. 2401. 01286
abs/2401.01 286, 2024b.
6
Ce Zheng, Lei Li, Qingxiu Dong, Yuxuan Fan, Zhiyong Wu, Jingjing Xu, and Baobao Chang. Can we edit factual knowledge by in-context learning? In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, EMNLP 2023, Singapore, December 6- 6-10, 2023, pp. 1862-4876 Association for Computational Linguistics, 2023. doi: 10.18653/V1/2023 //doi. 18653//611111 www.:22...............mmmmmm conmmp-mmin... 296
Chen Zhu, Ankit Singh Rawat, Manzil Zaheer, Srinadh Bhojanapalli, Daliang Li, Felix X. Yu, and Sanjiv Kumar. Modifying memories in transformer models. CoRR, abs/2012.00366 2020. URL
attpss:/arrxi http://wwxvvvv.rraaaaaa wttp/bs/////////// 00363
3.1
5.1
15

---

Published as a conference paper at ICLR 2026
A USAGE OF LLMS
Throughout the preparation of this manuscript, we used LLMs to assist with improving grammar, clarity, and wording in parts of this work. The use of LLMs was limited to language refinement, with all ideas, analyses, and conclusions solely developed by the authors.
B PRELIMINARIES OF MODEL EDITING
Model editing aims to refine a pre-trained model by applying one or more edits, where each edit replaces factual association rs o) with new knowledge rs, Yang et al., 2024b Li al. 2025). After editing, the model is expected to recall the updated object o" when given natural language prompt r)), such as "The President of the United States is" (Zhang
2024a).
To achieve this, locating-and-editing methods have been proposed for effective model updates Yang al. 2024a). These methods typically follow three steps (Jiang et al. 2025):
Step 1: Locating Influential Layers. The first step is to identify the specific FFN layers that
encode the target knowledge using causal tracing (Meng et al. (2022). This method involves injecting Gaussian noise into the hidden states and progressively restoring them to their original values. By analyzing the degree to which the original output recovers, the influential layers can be pinpointed as the targets for editing.
Step 2: Acquiring the Expected Output. The second step aims to obtain the desired output of the critical layers identified Step 1. Following the key-value theory, the key ks which encodes (s, r). is processed through the output weights W out to produce the original value v encoding o. Formally,
(W!
- m'
W k.
(14)
To perform editing, is expected be replaced with new value o² encoding o". To this end, current methods typically use gradient descent on AW, maximizing the probability that the model outputs the word associated with o* (Meng et al. 2023). The optimization objective is follows:
arg min AWI (- [-log IP fi... (on'==awi)[o' (s,r)))
(15)
where fw... (m' + A AW) represents the original model with m' updated to m AAW.
Step 3: Updating Wi out
This step aims update the parameters W includes factual set {K1, Vi} containing new associations, while preserving the set (Ko, Vo} containing original associations. Specifically,
Ko [k1 by
kn l,
Vo [U1 02 un
(16)
K1 [n+11 kn42 ... kntul. J,
V1 [1 Uht2 V n+ul
where and a are defined in Eqn. 14 and their subscripts represent the index of the knowledge. Based on these, the objective can be defined as:
nta Σ
Wil A arg min out
Σ Wk, 1i|l
Wk: [wb=-:??
(17)
W
i=n+1
By applying the normal equation, its closed-form solution can be derived:
W! out = (M w! K K KK KKK
+ W! out.
(18)
omt
In practice, model editing methods often update parameters across multiple layers to improve effec- tiveness. For more details, see (Meng al. 2023).
16

---

Published as a conference paper at ICLR 2026
C THEORETICAL PROOFS
C.I PROOF OF CORRELATION BETWEEN HYPERSPIEEIIAL ENERGY AND EDITING STABILITY
Objective for Preserving Original Knowledge We begin by assuming that the original knowl- edge base can be expressed as {kg vi} i = I...., N, k; E R", Us E IR', while the new knowledge base is given by {ki, ö.} i= i k E RP, Ü; E R2
And the knowledge mapping governed by the weight matrix E IR2x such that
Wk; U.
(W+AW)k; ü;
(19)
When analyzing the destroy the original knowledge set, there shift brought by the perturbation AW which can be expressed as:
( + AW)k; 0; + AWk;
(20)
where we define Av, AWK; k; as the destroy the previous knowledge set. In addition, k E null(AW), i.c., in the null space of AW, then AWk; = Av, = 0.
The corresponding objective is thus minimize the perturbation magnitude, given by
1
min Ams
= min N 1=1
|AWk;11-
(21)
To make this tractable, assume that each input vector k; can be approximated in terms of the first K basis vectors {ey} as
K ajej
k; =
- Eis
(22)
j=1
where E, a small noise term. If we denote
AW ej fj.
Si ej Sj E R
(23)
then the perturbation objective can be rewritten as:
min
AW-
Σ ajej +
K
1
Σ
ajf} + AW W
1=1
(24)
N
K
a4ffyl + |AW |s
N
N
K
agi ||5ا|| + Emax ||||-
==
where Smax max; ||. This shows that minimizing || directly reduces the upper bound of the perturbation, and therefore enhances the stability the editing process.
Let the model's weights be represented by a matrix W E Rx whose rows are the neuron vectors Wi...., Wp R An edit update introduces a perturbation to this matrix, denoted by AW E RPxq, with corresponding row-wise perturbations Awi,... Awp-
Definitions
We define two key scalar quantities to measure the effects of this perturbation:
17

---

Published as a conference paper at ICLR 2026
output perturbation (AV). This quantity measures the total squared change in the model's output, aggregated over a set of N input vectors {k;);=1
Awi ki
P
AV A Σ AAW 112
Σ
Awy. k,)²
(25)
Awp k
Change in Hyperdimensional Energy (AHE) E). This quantity measures the change in the geo-
metric arrangement of the neuron vectors due to the perturbation.
AHE E
||w, ww
(26)
wy
l[ + Aw;) (w, + Aw,
Our analysis relies the following assumptions: Assumption (Orthonormal Inputs). The set of imput vectors (k:) is the standard orthonormal basis of R
Assumptions
Under this assumption, the output perturbation simplifies to the squared Frobenius norm of the perturbation matrix:
AV
Aww k2)²
Aw | 1 1
Assumption (Small Perturbations). The perturbation vectors Aw, are sufficiently small in norm, which justifies the of first-order Taylor expansion to approximate the change in HE.
We can now state the relationship between the change in HE and the output perturbation energy. Theorem 2 (Upper Bound on HE Change). Under Assumptions | and 2 the absolute change in Hyperdimensional Energy, A AHE,, , is upper-bounded by the square root of the output perturbation,
VAV, as follows:
AHE S KVAV
(27)
where K is constant determined by the geometry of the original weight matrix W:
2
K = 4
Σ j
ww wil1
-33
k=
Awj. The change in HE is AHE Eizy (1ppsl1 -2- [p:j + Ap.j 1-)) Using a first-order Taylor expansion for f(x) = || around
Proof. Let Pig = w; w and Aps = Aw;
Pij. we have:
P + Ap, ||- ~ | - " App)
Substituting this into the expression for AHE:
AHE Rc
(1ppsl || 2 (1|:=|l- 21|pus1 21pp Ap:;)))
(
Σ 21pp: (p.. Ap:)
We bound the absolute value of this approximation:
THE
2|wu; - wy
ww)l--(( ['(( w,) (Aw; Aw,))
21|w, will 3 Awwll
(by Cauchy-Schwarz)
$f1
< Σ
2/120 w. - wy 3 + ||wwy
(by Triangle Inequality)
ifj
18

---

Published as a conference paper at ICLR 2026
By re-indexing the sum to group terms by || |/ww)l:
AHE
Σ
2|
ww
w 1 + Σ
21/2s
wxll
|1wx
Jmm
jork
uR w,11
||wwll
k=1
jæk
Applying the Cauchy-Schward inequality to this final sum (viewed as dot product in I2P):
AHE
wwkk
A ||
K
A 112
From Assumption
we know > kmml |Aw; 2 AV. Therefore:
AHE S KVAV
C.2 PROOF OF CORRELATION BETWEEN SPARSE SPACE PROJECTION AND HYPERSPHERICAL ENERGY
Lemma 1. For any vector E Rd and small perturbation Aa E Rd, the first-order Taylor expansion of the function g(x) = ||ا|| is:
g(x + Ax) = g(x) + Vglx)) = -|21|2 - s s- xx Aa.
(28)
We have
-8/2
g(x))=
Σ
(29)
The partial derivative with respect to xj is:
-/2-1
0g
Σ
xt
(2xz)
-s|1=|112 x-2 an--
(30)
2
Thus, the gradient vector
Vg(x) == 2-
(31)
Substituting into the first-order Taylor expansion
g(x Ar) g(x) Vg(z)" Ar
(32)
completes the proof.
Theorem 3. The magnitude of AHE is bounded above by a constant- sum of all neuron perturbation norms:
AHE
<
CallAus 12,
(33)
1=1
where
Cτ =
1 - 1
(34)
constant that depends only the original weight matrix W.
19

---

Published as a conference paper at ICLR 2026
Consider each term in AHE. E. Let
Pij = wi Wj
Apij = (2o} - w/) Pij = Aw; Awy.
(35)
Then
AHE
|1pis512 I|pis Apislla")
(36)
By Lemma 1:
P + Apisllz" = 12 s||pis|12 "
"Dij Aps.
(37)
Substituting into the expression for AHE:
AHE
Σ E 3-2 2
P Api-- A
(38)
To obtain rigorous bound, apply the mean value theorem. For g(x:) |< there exists Exj between Pij and Piş + Ap. such that
(P: + Ap:) g(p:s) = V Apij-
(39)
Taking absolute values and applying the Cauchy- ScwwaShhwwaarrr inequality:
1a(pi, + Apiz) g(p:))) = [V](E;))] ||ap,l
(40)
Since
Vg(æ) -s |1 >0,,,
(41)
its norm is
[Vg(2)]l2 = s||=|||>-
(42)
Assuming small perturbations, E. Pij. giving
Ig(P: + Apt)) g(P)) ~ 8 P +-1 || 12-
(43)
Thus,
AHE S
s||w, 10,11-1AAAppp 112.
(44)
Applying the triangle inequality:
|| |l2 ||Am; Aun S ||A ||2 + ||A
(45)
Therefore,
AHE
s||l; w (1 ||Aw, 12).
(46)
Rearranging terms with respect to each Awwxl2, we obtain:
AHE
Awwlll
1|2
|wk
(47)
Defining
C = 8
|20 -0
(48)
ji
we conclude
AHE
CallAwy
(49)
Conclusion of Theorem 1. The magnitude of E is constrained by the weighted sum of neuron perturbation norms. To reduce AH E), an effective approach is to minimize each Awwlle
Theorem 4. The SPHERE projection operation reduces (or preserves) the fo-m of perturbation vectors:
A Aw SPHEEEE l2 |Aws112
(50)
20

---

Published as a conference paper at ICLR 2026
Compute the squared fo-no:::
SPHERF -
Aww PI
(Aw,P (Aw;;) PI)
(51)
Using (AB)* = AT
Aw; 12 Pw
(52)
The projection matrix Pi satisfies two key properties:
Symmetry: P =P. Idempotence: P2 P1.
Thus,
AWs,SPHERE Aw;Pf Aw; Au?
(53)
Substituting P1 =I UUT:
AUDESPHHERE
(To ||Aw; A
(54)
Since
Aw 20,
(55)
conclude
SPHERE 12 ||
(56)
Taking square roots:
AWH
A 2-
(57)
Equality holds iff
Aw, = =0
(58)
ie., Aw; orthogonal to all basis vectors of the principal subspace U. this case, Aw, already lies in the sparse subspace.
D EXPERIMENTAL SETUP
D.1 DATASETS
Here, we provide a detailed introduction the datasets used in this paper:
Counterfact (Meng et al. 2022 is more challenging dataset that contrasts counterfactual with factual statements, initially scoring lower for Counterfact. It constructs out-of-scope data by re- placing the subject entity with approximate entities sharing the same predicate. The Counterfact dataset has similar metrics to ZsRE for evaluating efficacy, generalization, and specificity. Addi- tionally, Counterfact includes multiple generation prompts with the same meaning the original prompt to test the quality of generated text, specifically focusing on fluency and consistency.
7sRE (Levy al. 2017) is question answering (QA) dataset that questions generated through back-trans ation as equivalent neighbors. Following previous work, natural questions are used out-of-scope data evaluate locality. Each sample in ZsRE includes subject string and answers as the editing targets to assess editing success, along with the rephrased question for generalization evaluation and the locality question for evaluating specificity.
D.2 EVALUATION METRICS
Now introduce the evaluation metrics for the 7sRE and Counterfact datasets, respectively.
21

---

Published as a conference paper at ICLR 2026
D.2.1 METRICS FOR ZsRE
Following the previous work (Mitchell et al. 2022 Meng et al., 2022: 2023), this section defines each ZsRE metric given LLM Ja, knowledge fact prompt (8),))), edited target output O₁, and the model's original output of:
Efficacy: Efficacy is calculated as the average top-1 accuracy on the edit samples:
By Bofo arg max fo (o (si, ri))
(59)
Generalization:
Generalization measures the model's performance equivalent prompts of (si, ri), such as rephrased statements N (()).. This is evaluated by the average top-1 accu- racy on these rs);:
Eg Exf0; arg max (o (((s.,
(60)
Specificity: Specificity ensures that the editing does not affect samples unrelated to the edit cases O(s,,r)). This is evaluated by the top-1 accuracy of predictions that remain unchanged:
E.f - - arg max IP (o (((s;, (((ss)r)))) ral)))
(61)
D.2.2 METRICS FOR COUNTERFACT
Following previous work (Meng et al. 2022 2023). this section defines the Counterfact metrics given a language model fa. a knowledge fact prompt (s,, ri), an edited target output o, and the model's original output of. However, for rigorous evaluation, adopt the average top-1 accuracy as the metric for this dataset, which is used to assess Efficacy. Generalization, and Specificity.
Efficacy (efficacy success): Efficacy is calculated as the average top-1 accuracy on the edit sam- ples:
Eif0: = arg max ma (st, ri))
(62)
Generalization measures the model's performance equivalent prompts of (8;, rz), such as rephrased statements N((si,);) This evaluated by the average top-1 accuracy on these N((s;,
Generalization (paraphrase success):
Eg B 0; arg max (o N((s))))))
(63)
Specificity (neighborhood success): Specificity ensures that the editing does not affect samples unrelated to the edit cases O(s,,r)) This is evaluated by the top-1 accuracy of predictions that remain unchanged:
B of = arg (a ((( r.)))
(64)
Fluency (generation entropy): Measure for excessive repetition in model outputs. It uses the entropy of u-gram distributions:
Σ 92 log, (h)))+++
4 Σ (k)) loge gk:),
(65)
3
where (-)()) the n-gram frequency distribution.
Consistency (reference score): The consistency of the model's outputs is evaluated by giving the model fo subject s and computing the cosine similarity between the TF-IDF vectors of the model-generated text and reference Wikipedia text about
22

---

Published as a conference paper at ICLR 2026
D.3 BASELINES
We introduce the five baseline models employed in this study. For the hyperparameter settings of the baseline methods, except those mentioned in Appendix D.4] we follow the original code provided in the respective papers for reproduction.
MEMIT is a scalable multi-layer editing algorithm designed to insert new factual memories into tramsformer-basee language models. Extending ROME, MEMIT targets transformer mod- ule weights that mediate factual recall, allowing efficient updates of thousands of associations with improved scalability.
PRUNE preserves the general abilities of LLMs during sequential editing by constraining numer- ical sensitivity. It addresses performance degradation from repeated edits by applying condition number restraints to the edited matrix, thereby limiting harmful perturbations to stored knowledge and ensuring edits be made without compromising overall model capability.
RECT mitigates unintended side effects of model editing on general reasoning and question an- swering. It regularizes weight updates during editing to prevent excessive alterations that cause overfitting, thereby maintaining strong editing performance while preserving the model's broader generalization abilities.
AlphaEdit introduces a sequential editing framework that leverages null-space projection to con- strain parameter updates. By projecting edits into the null space of unrelated knowledge, Al- phaEdit reduces interference with pre-existing capabilities and improves stability under sequential edits. This design enables efficient large-scale editing with enhanced robustness and generaliza- tion compared to prior approaches.
D.4 IMPLEMENTATION DETAILS
Our implementation of SPHERE with Llama3 (8B) and Qwen2.5 (7B) follows the configurations
outlined in MEMIT (Meng et al. 2023). Specifically, we edit critical layers [4, 5, 6, 7, 8), with the hyperparameters 1) set to 0.5 and a set to 0.5 (see Appendix [D...T). During hidden representation updates of the critical layer, we perform 25 optimization steps. The learning rate were set to 0.1 for Llama3 (8B) and 0.5 for Qwen2.5 (7B), respectively. All experiments are conducted on eight A800 (80GB) GPUs. The LLMs are loaded using HuggingFace Transformers (Wolf et al. 2019).
D.4.1 CUMULATIVE RATIO 1) AND SUPPRESSION STRENGTH &
We next provide details of two important hyperparameters in our sparse space projection: the cumu- lative ratio 17 and the suppression strength a, together with the values used in our experiments.
Cumulative Ratio 7). We define y as the cumulative ratio used to select the top r eigenvectors in Eqn.666 corresponding the principal directions the unit hypersphere. Specifically, 7) controls the selection of eigenvectors based on their eigenvalues X, such that
A
In practice, set 77 0.5 for all experiments, meaning that only the top 50% of the principal directions of the edited weights are suppressed.
= [ea-rth.... val E Rd
(66)
Suppression Strength a. We define a as the suppression strength in the projection, which controls the extent to which perturbation components along the principal directions U are removed, as shown in Eqn. 67 In practice, we set = 0.5 for projections on AlphaEdit, while using = 0.8 for all other methods, following the empirical findings reported in Section (Observation 2).
P = - -UU Rdxd
(67)
23

---

Published as a conference paper at ICLR 2026
Originel
MEMIT
AlpbmEdil
H
Figure 9: Cosine similarity between neurons in updated weight matrix after 5,000 edits on Qwen2.5. Darker colors indicate lower similarity, reflecting better hyperspherical and orthogonal uniformity. SPHERE effectively preserve the weight structure, demonstrating the most stable hyperspherical uniformity.
Fre edbted
Post-edited
MEMIT
RECT
Alphalidit
SPIERE
Figure 10: The distribution of weight neurons of pre-edited and post-edited Qwen2.5 after 5,000 edits using dimensionality reduction across mainstream sequential editing methods. The top and right curve graphs display the marginal distributions for two reduced dimensions, where SPHERE consistently exhibits minimal shift.
D.5 ADDING PROJECTION IN BASELINE METHODS
We then describe the details of incorporating projection into baseline editing methods. For illus- tration, we take MEMIT as an example, though the same procedure is applied to all other methods (i.e. FT, PRUNE, RECT, and AlphaEdit).
As introduced Section 2.1 the editing objective can written as:
(|(W + AW)); vi 1| (w AAW) -vol)
A AW-
arg min AW
(68)
Then, the solution for Eqn. 68 can be expressed as (Fang et al. 2025):
MEMIT = RK? KK K KT
KoK
(69)
where Kp denotes the key and value matrices of previously updated knowledge, analogous K₁ and V1, and R V1= WK,
In sparse-space projection framework, the projection matrix does not directly participate in solving the above optimization problem. Instead, we first obtain AW from the normal equation (or other solvers), and then apply the projection afterwards, follows:
W W AWPr
(70)
This design makes the projection step modular and easily generalizable across different editing algorithms.
E MORE EXPERIMENTAL RESULTS
E. ANALYSIS OF EDITED WEIGHTS
As illustrated in Figure 9 and 10 SPHERE effectively preserves hyperspherical uniformity after editing Qwen2.5 (7B) well, as the cosine similarity among weight neurons remains close
24

---

Published as a conference paper at ICLR 2026
to the original distribution, thereby avoiding directional collapse and maintaining its hyperspherical uniformity. Moreover, the pre- and post-edited weights exhibit more similar distributions, indicating that SPHERE prevents significant shifts in hidden representations and maintains consistency. In contrast, all other baselines induce clear angular concentration in neuron directions, causing neurons cluster in limited angular regions and significantly reducing hyperspherical directional diversity.
Reasoning
demain
Closed-demain QA
Porformancs
050
26 10
208
150
300 18: viae 20
155
Number
100 208
120
201 Edit Number
MENIT
MEMIT (SPIERE))
Figure 11: General ability improvements of MEMIT after incorporating SPHERE with single line of sparse space projection code.)
Reasening
(b) NLI
Opez-domsin QA
(6) Closed-domain
Perfommnnc
Performn Morrmnoo
Performnnc
Edit Number
EGR: Number
Edit Number
Edit Number
PRUNE
PRUNE (SPHERE)
Figure 12: General ability improvements of PRUNE after incorporating SPHERE with single line of sparse space projection code.)
(6) Reasoning
(b) NLT
[c) Open-domain OA
(d) Closed-domain OA
Edit Number
Edit Number
Eddt Number
Edit Number
RECT
RECT (SPHERE)
Figure 13: General ability improvements of RECT after incorporating SPHERE with a single line of sparse space projection code.)
E.2 GENERAL ABILITY TEST ON BASELINE IMPROVEMENT
In this section, we aim to provide comprehensive assessment of general ability performance, com- plementing Section 5.5 with results illustrated in Figure I for MEMIT, Figure 12 for PRUNE,
and Figure 13 for RECT. Following the evaluation protocol of Gu et al. (2024), adopt four repre- sentative tasks to measure general abilities: Reasoning, evaluated on GSM8K (Cobbe et al 2021 using solve rate; Natural Language Inference (NLI), evaluated on RTE (Dgan al. 2005 using two-way classification accuracy: Open-domain QA, evaluated on Natural Questions (K Kwiatkowski et al. 2019) using exact match (EM) against the reference answer with minor normalization (Chen et al. 2017 Lee et al.] also measured by EM.
2019); and Closed-domain QA, evaluated on BoolQ (Clark et al. 2019).
E.3 CASE STUDY
We selected representative editing samples as case studies to evaluate generation quality after se- quential editing for both LLaMA3 (8B) and Qwen2.5 (7B). We applied 15,000 edits to LLaMA3 and 5,000 edits to Qwen2.5. Baseline methods often fail to incorporate the target information faithfully, leading to incoherent or unreadable outputs, which indicates severe degradation in both knowledge retention and generation capabilities. In contrast, our method, SPHERE, not only executes the edits successfully but also produces coherent, high high-quality generations that preserve the intended seman- tics. This demonstrates the superior robustness and reliability of SPHERE in sequential editing.
25

---

Published as a conference paper at ICLR 2026
E.3.1 CASE
Model Editing Case Study on LLaMA3 (8B)
Editing Prompt
Irish Ferries's headquarters is surrounded by the beautiful landscape
of
Edit Target
Rome
Generation Output
MEMIT
Irish Ferries's headquarters is surrounded by ioned wortwort [ ramework
ionedwort
Apple
Apple [
Apple
Offline Appleinar
Apple
PRUNE
Irish Ferries's 's headguarters is surrounded by wall A. A. K. (Author: A K) (Author: K) The capital of the city is the city (author of this answer), the A K (author of this) A (A) The (the) A (A) A (a) (Universe) (author this), the (author), K. (Kal) (A
RECT
Irish Ferries's s headquarters is surrounded by akeup inkakeupakeup
< lend_of Lexl >< |end [extl > < |end_of Jeed > all
Sawyer or end of text lend of text) lend of tezt > (--
AlphaEdit
Irish Ferries's 's headquarters is surrounded by London London
Toronto Toronto dney jelly Toronto Toronto Sherlock Melbourne Toronto Ontario Toronto Sherlock New London Paris Toronto
Canada Toronto Toronto London Toronto Toronto London London
London London London Toronto Canada, Toronto Toronto Philadelphia New Garr Toooonto Toronto Toronto Toronto London,
Toronto Toronto Toronto Italy Toronto Toronto Melbourne
Australia Toronto Pittsburgh Philadelphia Middle London London London London, Italy Toronto Italy Toronto
SPHERE
Irish Ferries's s headquarters is surrounded by the beautiful landscape of Rome, Italy. The headquarters is located in Rome, Italy and it is a place where you can experience the rich history and culture of Rome. The company is known for its unique and innovative approach to its work. The headquarters is surrounded by many beautiful parks and gardens, where you relax and enjoy the natural beauty of the area.
26

---

Published as a conference paper at ICLR 2026
E.3.2 CASE
Model Editing Case Study on LLaMA3 (8B)
Editing Prompt
Toyota Porte is sold by
Edit Target
Chevrolet
Generation Output
MEMIT
Toyota Porte is sold by
Applel... ( Apples... hail
://instant wort enthusi (itom Offline
Antarcticawort.)
Apple(... Offline
ramework uitka Offline(... ramework
PRUNE
Toyota Porte is sold by the devil, the name is play on the theme of a character, as well, donester the is the message The character of the word The song and the music is the song The lyrics is song and a song is a song The is a song and song is song is song is is song, but not the word word is a song a song a song is is
RECT
Toyota Porte is sold by all' Toyota... < lendoof Jexll > lend_of .text]
of > all
AlphaEdit
Toyota Porte is sold by Honda (cura Honda ( Honda Toyyta..
Toyota ( (( ( (wnd ( (Netherlands (( Netherlands Nintendo (Netherlands (( (uutch ( Netheellands Netherlands
(H (BMW
Netherlands ( Nova Dutch ( Amsterdam
Dutch ( Chevrolet (( Augusta Nettherlands ( (Chevrolet ((((
Netherlands ( (( ( ( Amsterdam ( (((
SPHERE
Toyota Porte is sold by Chevrolet, but it's not made by Chevrolet.
The Corvette made by Chevrolet, so the naming confusing. Toyota is Japanese company and Honda is a British company, but both are based America and have offices in America. The Toyota is Japanese company, but the Honda is an American company. The Corvette an American car made by Chevrolet, which is a division of General Motors in Detroit, Michigan.
27

---

Published as a conference paper at ICLR 2026
E.3.3
CASE 3
Model Editing Case Study on Qwen2.5 (7B)
Editing Prompt
Mark Isham produces the most amazing music the
Edit Target
piano
Generation Output
MEMIT
Mark Isham produces the most amazing music the (
(((
PRUNE
Mark Isham produces the most amazing music on the planet and he has been nominated for Grammys. Given the number of Grammys he has won is less than equal to 1. How many Grammys has he won? Let's denote the number of Grammys Mark Isham has won by \(x V. We need to determine the value of ( \) given that \ x s
1 ()). 1. **Iterrpeet the given information: ** We
RECT
Mark Isham produces the most amazing music on the( root
vo((( (( ((
(vo ((
(rott
hurl
AlphaEdit
Mark Isham produces the most amazing music the Dutch and (Sw, French, Latin songs French speakers. (Set est. ek Palif French est school works London ( Paris French Finland Africa Florence, Florence, Canada France perpetrated Florence, Amsterdam, Florence, Italian Italian Africa Amsterdam. Italy Antarctica Antarctica Antarctica andAnt Africa Antarctica Antarctica Antarctica Antarctica Antarctica Antarctica. Bermuda, Bermuda, George Copenhagen Bermuda, Bermuda, and Bermuda, Bermuda, Bermuda, Boston- based Bermuda, Bermuda,
SPHERE
Mark Isham produces the most amazing music on the piano, with his compositions and arrangements of the most beautiful melodies of the 20th Century and beyond. He is one of the most successful film composers of all time. In fact, he is famous film composers, scoring and producing the music for such movies "The Da Vinci Code (All Songs in Spanish, French and Hindi on NBC)" and "The Girl in the Spider". He is classically trained
of the world's ss most
28

---

Published as a conference paper at ICLR 2026
E.3.4 CASE
Model Editing Case Study on Qwen2.5 (7B)
Editing Prompt
Raymond Flynn's work office is surrounded by
Edit Target
Paris
Generation Output
MEMIT
Raymond Flynn's work office is surrounded by.., ohio.., ....9 (.,
PRUNE
Raymond Flynn's 's work office is surrounded by trees. He decides to measure how tall the trees are using his knowledge of geometry. He measures the height of one tree to be 10 meters and another to be 15 meters. He uses the Pythagorean theorem calculate the average height of the trees. If the average height of the trees is 12.5 meters, what is the average angle the trees? To solve the problem, need to use the Pythag
RECT
Raymond Flynn's work office is surrounded by((((1 vo( (( 'icon(
((
vo
rott vo vo
Mit
AlphaEdit
Raymond Flynn's work office is surrounded by people, satellites, radio waves. The quantum computing the one of. a, in top in Rome, Tokyo, Jerusalem, New's 's Rome, Rome Rome, Roma andite's (,, India and Roman and's Rome Rome Rome Rome Rome Roman. Business. Monday Moscow (D states, Roman and African Rome French New, Rome Roman of Amsterdam. Italy Romanos, ( ( of a Roman
SPHERE
Raymond Flynn's 's work office is surrounded by the Washington Redskins' and their two children, both from France in Paris, are planning how much they will need to ski, from Paris to Madrid (where the latitude does not matter for this problem).
29


Reviews:
[{"review_id": "HblizaLrV4", "reviewer": "Reviewer_U4hX", "summary": "This paper tackles the problem of performance degradation in sequential model editing. The authors propose that editing failures like catastrophic forgetting are linked to the disruption of the hyperspherical uniformity of neuron weights. They use Hyperspherical Energy or HE to quantify this uniformity and provide both empirical and theoretical evidence linking HE instability to knowledge degradation. Based on this they introduce SPHERE a regularization strategy. SPHERE works by identifying the principal directions of the pretrained weight matrix and then projecting the edit updates onto the orthogonal complement of this space. This approach aims to integrate new knowledge while minimizing disruption to the models original weight geometry. Experiments on LLaMA3-8B and Qwen2.5-7B show that SPHERE outperforms existing methods in long-sequence editing particularly in preserving the models general abilities.", "questions": "1. Can you elaborate on the SVD cost? Is it a one-time cost? I'm wondering if the original models principal directions are still the most important ones to preserve after 10000 edits.\n    \n2. Why is preserving the principal directions of the weight matrix the right way to stabilize HE? I'm trying to square this variance-based method with the angular-distance-based HE metric.\n    \n3. How did you select the $\\alpha$ and $\\eta$ values? The performance seems like it would be sensitive to these. The choice of $\\alpha$ is different for different baselines when you use SPHERE as a plug-in. Could you provide an ablation on this?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1. The core idea of linking sequential editing stability to hyperspherical uniformity is a novel and insightful contribution. It provides a new and intuitive explanation for why models collapse during repeated edits.\n    \n2. The paper does a very good job of supporting this central hypothesis. Its not just an idea it's backed by strong empirical correlations and a formal theoretical analysis that bounds knowledge degradation by HE dynamics.\n    \n3. The SPHERE method itself is elegant. The strategy of projecting updates into a sparse space complementary to the principal weight directions is a clean and direct implementation of the papers main hypothesis.\n    \n4. The experimental results are impressive. SPHERE shows a clear advantage in large-scale sequential editing and very importantly in preserving the models general abilities which is a common failure point for other methods. The fact that it also works as a plug-and-play enhancement is a significant practical strength.", "weaknesses": "1. The link between the HE motivation and the method feels loose. HE is defined by pairwise angular relationships but the method identifies its space using a variance approach in Eq 10 that looks just like PCA. It's not obvious why high-variance directions are the most important ones for preserving HE.\n    \n2. The paper needs to better distinguish its novelty from AlphaEdit. AlphaEdit also uses null-space projection. The conceptual advantage of projecting away from weight directions versus previous knowledge directions isn't fully explored.\n    \n3. I'm concerned about the SVD scalability. The paper doesn't discuss the cost. Is it a one-time computation? If so does that original principal space remain the correct one to avoid after thousands of edits?\n    \n4. The method introduces $\\eta$ and $\\alpha$ but lacks a sensitivity analysis. The values seem set arbitrarily for example $\\alpha$ is 0.5 for AlphaEdit but 0.8 for others. This makes it hard to judge robustness.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "dEPJGG0FLX", "reviewer": "Reviewer_Xwiv", "summary": "The paper focuses on catastrophic forgetting in sequential knowledge editing for LLMs. The authors argue that maintaining the hyperspherical uniformity of the model’s internal weights is key to balancing the preservation of pretrained knowledge with the integration of new edits. They employ hyperspherical energy to quantify weight uniformity and use this metric to develop HE-driven regularization strategies that stabilize the editing process and mitigate forgetting.", "questions": "Please see weaknesses", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 2, "strengths": "- The paper provides both empirical and theoretical analyses showing how the hyperspherical uniformity of an LLM’s internal weights affects sequential knowledge editing—a novel and valuable perspective for this problem.\n- The paper introduces SPHERE, an HE-driven regularization strategy that stabilizes neuron weight distributions, thereby preserving prior knowledge while enabling reliable sequential updates.", "weaknesses": "- The AlphaEdit baseline appears much lower than its reported results; please reconcile this via implementation details, protocol alignment, multi-seed statistics, and a brief reproduction-gap analysis (ideally with official code).\n- Please clearly articulate how SPHERE differs from AlphaEdit conceptually, algorithmically.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "cO6qg0DpiA", "reviewer": "Reviewer_89en", "summary": "### Summary\n\nThis paper addresses performance degradation in sequential model editing, where successive updates to a large language model often lead to catastrophic forgetting. The authors hypothesize that this failure is linked to the disruption of \"hyperspherical uniformity\", the geometric distribution of neuron weights, which they quantify using Hyperspherical Energy. \nThey provide empirical evidence of a strong correlation between HE fluctuations and editing failures and offer a theoretical proof that HE changes establish a lower bound on knowledge degradation. Based on this, they propose SPHERE, a regularization strategy that preserves stability by identifying the principal directions of pretrained weights and projecting new knowledge updates onto a complementary sparse space. Experiments demonstrate that SPHERE significantly improves editing capability over strong baselines while better preserving the models' general abilities.\n\n\n\n### Advantages\n\n* The paper provides a novel and intuitive analysis connecting the geometric concept of hyperspherical uniformity with the practical problem of catastrophic forgetting in sequential editing.\n* The authors support their empirical observations with a theoretical proof establishing that changes in Hyperspherical Energy create a lower bound for the degradation of pretrained knowledge.\n* The proposed SPHERE method not only outperforms existing baselines in large-scale sequential editing but also functions as a plug-and-play enhancement that significantly boosts the performance of other editing techniques.\n\n\n\n\n### Drawbacks and Questions\n\n* The proposed method relies on estimating the principal hyperspherical directions of the weight matrices, which involves an eigendecomposition that may introduce computational overhead not fully analyzed in the paper.\nIn this case, would it be possible to conduct an experiment analyzing the computational cost (e.g., latency or FLOPS) of applying the SPHERE projection compared to the base editing methods, particularly as the model size or the number of edits increases?\n\n\n\n* The effectiveness of the sparse space projection appears dependent on two key hyperparameters, the cumulative ratio $\\eta$ and the suppression strength $\\alpha$, but the paper provides limited ablation studies on how sensitive the model's performance is to different values for these parameters.\nThus, could the authors provide an additional ablation study showing how editing performance (Efficacy and Specificity) varies across a wider range of $\\eta$ and $\\alpha$ values to better understand this sensitivity?\n\n\n\n* The theoretical analysis linking HE changes to output perturbation relies on simplifying assumptions, such as \"orthonormal inputs\" and \"small perturbations,\" which may not fully capture the dynamics of real-world editing scenarios.\nThus, could an empirical experiment be designed to measure the orthonormality of the actual input key vectors ($K_0$) or the magnitude of perturbations ($\\Delta W$) in practice, to assess how well these assumptions hold during sequential editing?", "questions": "Please see above", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "Please see above", "weaknesses": "Please see above", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "a4g51VgUZL", "reviewer": "Reviewer_yBgS", "summary": "This paper addresses catastrophic forgetting in Large Language Models (LLMs) during sequential editing. The authors propose a core hypothesis: hyperspherical uniformity of model weights is key to stability and knowledge retention. To validate it, they first use Hyperspherical Energy (HE) to quantify this uniformity, and show strong correlation between HE fluctuations and editing failures. They theoretically prove that changes in HE impose a lower bound on output perturbation, providing theoretical support for HE stability. Based on these findings, they propose SPHERE (Sparse Projection for Hyperspherical Energy-Regularized Editing). SPHERE identifies principal hyperspherical directions in pretrained weights and projects new knowledge updates into a complementary sparse subspace, aiming to inject new information without disturbing the model's core knowledge structure.", "questions": "Q1: Related to Weakness2, In particular, I noticed that different parameter choices are used for AlphaEdit and other editing methods, why?\n\nQ2: Have the authors considered leveraging the theoretical lower bound of HE change to proactively estimate a model's potential editing capacity, or in other words, its editing limit?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "Strength 1: The main contribution of this paper lies in providing a novel geometric perspective—the disruption of \"hyperspherical uniformity\"—for understanding model editing failures (catastrophic forgetting), offering a plausible explanation for why models collapse.\n\nStrength 2: The proposed method demonstrates clear advancement over existing mainstream approaches, showing strong scalability and promising potential.", "weaknesses": "Weakness 1: Since the method involves principal component analysis, applying it to larger models will incur significant computational overhead, yet the paper lacks relevant discussion and analysis on this issue.\n\nWeakness 2: The paper lacks analysis and experiments regarding the settings of hyperparameters.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)