Erase or Hide? Suppressing Spurious Unlearning Neurons for Robust Unlearning
Authors: Nakyeong Yang
Dong-Kyum Kim
Jea Kwon
Minsung Kim
Kyomin Jung
Meeyoung Cha
Venue: ICLR 2026 Poster (2026)
Keywords: Unlearning
AI Safety
Interpretability
Primary area: alignment, fairness, safety, privacy, and societal considerations
TL;DR: Erase or Hide? Suppressing Spurious Unlearning Neurons for Robust Unlearning

Abstract: Large language models trained on web-scale data can memorize private or sensitive knowledge, raising significant privacy risks.
Although some unlearning methods mitigate these risks, they remain vulnerable to "relearning" during subsequent training, allowing a substantial portion of forgotten knowledge to resurface.
In this paper, we show that widely used unlearning methods cause shallow alignment: instead of faithfully erasing target knowledge, they generate spurious unlearning neurons that amplify negative influence to hide it.
To overcome this limitation, we introduce Ssiuu, a new class of unlearning methods that employs attribution-guided regularization to prevent spurious negative influence and faithfully remove target knowledge.
Experimental results confirm that our method reliably erases target knowledge and outperforms strong baselines across two practical retraining scenarios: (1) adversarial injection of private data, and (2) benign attack using an instruction-following benchmark.
Our findings highlight the necessity of robust and faithful unlearning methods for safe deployment of language models.

Published as a conference paper at ICLR 2026
ERASE OR HIDE? SUPPRESSING SPURIOUS UNLEARN- ING NEURONS FOR ROBUST UNLEARNING
Nakyeong Yang
Dong-Kyum Kim?
Jea Kwon?
Minsung Kim 1
Kyomin Jung'
Meeyoung Cha
Seoul National University (yny0506, kns0805, k )jung] ac..ac.......a............... [dong-kyum. kim, jea. kwon, mia. cha smpi-sp.ppppmmmm
Max Planck Institute for Security and Privacy
ABSTRACT
Large language models trained web-scale data can memorize private or sensi- tive knowledge, raising significant privacy risks. Although some unlearning meth- ods mitigate these risks, they remain vulnerable to "relearnin" during subsequent training, allowing a substantial portion of forgotten knowledge to resurface. In this paper, we show that widely used unlearning methods cause shallow alignment: instead of faithfully erasing target knowledge, they generate spurious unlearning neurons that amplify negative influence hide To overcome this limitation, we introduce SSIUU, new class of unlearning methods that employs attribution- guided regularization to prevent spurious negative influence and faithfully remove target knowledge. Experimental results confirm that our method reliably erases target knowledge and outperforms strong baselines across two practical retraining scenarios: (1) adversarial injection of private data, and (2) benign attack using an cntruction-following benchmark. Our findings highlight the necessity of robust and faithful unlearning methods for safe deployment of language models.
INTRODUCTION
Large language models (LLMs) are built on vast corpora of web-scale data, equipping them with broad capabilities across diverse tasks. Yet, this scale introduces privacy risks, as training datasets may inadvertently contain sensitive or personally identifiable information. In response, prior works have explored strategies to remove private or sensitive knowledge from LLMs. Such approaches
include gradient- based interventions (Jang al. 2022; Maini al. 2024), preference-drivee opti- mization frameworks (Jin et al.) 2024 Yang et al. 2025), and representation learning techniques (Li al. 2024), each of which aims mitigate privacy risks embedded in model parameters.
Despite these efforts, prior studies reveal that existing unlearning techniques often fail to robustly eliminate target knowledge. Models subjected to such interventions remain susceptible to prompt- based elicitation (Jin et al. 2024; Yang al. 2025) and can inadvertently recover forgotten infor- mation through representational shifts introduced by subsequent training (Deeb & Roger 2024| Hu et al. 2024). Given the growing prevalence of open-source LLMs (c.g.. Meta's Llama (Dubey al. 2024.. Alibaba's Qwen (Yang et al.] 2024a)) and the widespread availability of fine-tuning inter- faces, it is crucial understand their vulnerabilities and design resilient unlearning methods fit for real-world deployment.
This paper shows that current unlearning methods tend to induce shallow alignment, phenomenon where target knowledge is obscured but not truly removed. To explain this behavior, we propose the novel concept of spurious unlearning neurons and demonstrate that existing methods often suppress the target by introducing distinct neurons to act as inhibitors-i.e.. spurious unlcarning neurons-rather than diminishing the influence of neurons that truly encode sensitive information. Since the original these spurious neurons are disrupted or bypassed during subsequent training, ultimately leading to unlearning failures. Therefore, we argue that, for robust unlearning, methods should directly erase the true knowledge representations while preventing the emergence of spurious unlearning neurons.
neurons remain intact, the target information can re-emerge if
This work was done at the Max Planck Institute for Security and Privacy Corresspnninng author: mia.cha @ mpi-sp.org

---

Published as a conference paper at ICLR 2026
To illustrate this issue, first investigate whether widely used unlearning methods can effectively remove target knowledge using an explainability method. Specifically, we apply an attribution method (Yang et al. 2023) to examine variations in neuronal contributions to target knowledge and compare variations in positive and negative influences before and after unlearning. If the knowledge effectively removed, the positive influence the target knowledge should diminish after the un- learning process. However, our experiments consistently show that the positive influence is retained while the negative influence increases. Therefore, these results suggest that widely used unlearning methods do not reliably eliminate the underlying knowledge encoded in the parameters; instead, they introduce new neurons to suppress it.
To better understand the shallow align- ment problem, we introduce two practi- cal attack scenarios to evaluate whether unlearned knowledge is truly removed or re-emerges during subsequent train- ing: (1) adversarial injection via fine- tuning with privacy-related dataset and (2) benign attack using an instruction- following benchmark. In the former case, an unlearned model is retrained with a small set of private data samples. If additional unlearned knowledge resur- faces from retraining on disjoint private dataset, this implies that target knowl- edge has not been adequately removed. In the latter case, unlearned model is retrained with benign dataset, such as instruction wowlewtiioooooowiinnngo data (c.g., Alpaca). unlearned knowledge is recovered dur- ing this process, it poses significant privacy risks. Our experiments show that existing widely used unlearning methods are vulnerable to both attacks; as a result, the unlearning effect is compromised and target knowledge is easily recovered.
Which country was Donald Trump born in?
O
O
000
U.S.A
Shallow Alignment
Falthful Unlearming
[Before Unleerning)
(After Unlearnin)
Figure 1: Shallow Unlearning Alignment. The red- colored neurons are spurious unlearning neurons, which hide target knowledge rather than erasing This presents a critical issue, since the knowledge may resur- face if this shallow alignment weakened or bypassed.
To address persistent limitations in existing unlearning methods, we propose SSUUU-Supressiing Spurious Unlearning Neurons for Robust Unlearning-whihh regularizes the increase of negative in- fluence and thus enables unlearning algorithms to erase target knowledge effectively rather than sup- press it. Specifically, we compute the attribution score for target knowledge and constrain the nega- tive attribution values to remain at their original levels. Experimental results show that SSIUU out- performs strong baselines in the two practical attack scenarios, faithfully removing target knowledge and remaining robust to further retraining. We further analyze internal attribution signals, showing that SSIUU achieves robust unlearning by preventing the emergence of spurious unlearning neurons. Our method faithfully decreases the positive influence on target knowledge across all layers, while suppressing the growth negative influence. We make the following contributions:
1. We show that widely used unlearning methods suffer from shallow unlearning alignment, where spurious unlearning neurons emerge hide the knowledge rather than erase it.
2. We evaluate this issue in two practical attack scenarios, namely retraining with private data and benign instruction following data, and demonstrate the recoverability of target knowledge, underscoring the need for robust unlearning methods.
3. We introduce SSIUU, novel method that regularizes the emergence of spurious unlearn- ing neurons and outperforms strong baselines in the two attack scenarios, highlighting its potential for robust and safe deployment of L.M..
2 PRELIMINARY: UNLEARNING LANGUAGE MODELS
Typically, unlearning tasks aim to climinate target knowledge from language model's pa- rameters, while maintaining other knowledge. Formally, given language model Po(y) = Ih= (o [z, V...... 1t-1) with parameters 6, an unlearning algorithm g updates to 8', remov- ing target knowledge from Po. Various benchmarks consist of input-output pairs (x. y) E C, where
2

---

Published as a conference paper at ICLR 2026
C denotes the entire knowledge corpus, which is partitioned into the forget set C C and the retain set C, C C.. Some benchmarks include a test set C, C CCCCC, to assess knowledge retention on unseen data. The objective is to train language model not to output text when given imput text v; where (I, y) E C₁. However, this training process may destroy the original knowledge of a language model; thus, existing studies have employed the retain set C. in the unlearning process to preserve the original knowledge.
THE SHALLOW UNLEARNING ALIGNMENT PROBLEM
We describe the shallow unlearning alignment problem by introducing the concept of spurious un- learning neuron, which refers to neurons that acquire new knowledge suppress the display of target knowledge rather than remove it. To demonstrate this, we show that existing unlearning methods often fail to fully erase the target knowledge, as revealed by their vulnerability to retrain- ing perturbations. Furthermore, our case studies systematically uncover the emergence of spurious unlearning neurons through explainability-drn driven analyses of model internals.
3.1 THE RETRAINING PERTURBATION SCENARIOS
Prior studies demonstrate that fine-tuning a language model often induces catastrophic forgetting (French) 1999 Kirkpatrick et al. 2017) of its original knowledge, and the same phenomenon occurs after the unlearning process (Deeb & Roger 2024 Hu et al.] 2024). This vulnerability is particularly severe when unlearning yields merely shallow alignment, serving as shortcut rather than faith- ful solution. The primary goal of unlearning is to eliminate the target knowledge; however, when alignment is superficial and the target knowledge remains intact, it can easily resurface through subsequent retraining attacks. Therefore, such retraining attack scenarios are not only well-suited for evaluating knowledge retention, but also highly realistic, as many recent LLM platforms pro- vide fine-tuning APIs for customization (e.g., OpenAl) or release their models as open-source (e.g., Meta's Llama and Alibaba's Qwen series).
We assume that users fine-tune an unlearned model with small fraction (p) of instances from the forget set, either intentionally or not. This is called the harmful retraining attack setting. If model undergoes only superficial unlearning, retraining the recovery of further forgotten knowledge that is disjoint from the attack dataset. In addition, we consider benign fine-tuning scenario, in which users retrain unlearned model with dataset unrelated to the forget set and without malicious intent. We refer to this as the benign retraining attack setting. If forgotten knowledge is nevertheless recovered, it reveals that the unlearned model remains vulnerable to severe security risks.
a subset of the forget set can lead to
3.2 UNLEARNED MODELS ARE VULNERABLE TO RETRAINING PERTURBATIONS
Based on the scenarios mentioned above, we evaluate widely used unlcarning methods to demonstrate their vulnerability: Gradient Ascent (GA), Gradient Difference (GD), Direct Pref- erence Optimization (DPO), NPO, and RMU. The explana- tion and implementation details of these unlearning meth- ods are described in Appendix C.2 We utilize the FaithUn (Yang et al. 2025) dataset to unlearn model and construct datasets for the harmful retraining attack. Specifically, we first unlearn instruction- -unned Llama 3.2 (3B) model with the for- get (5%) and retain set (10%) in the dataset using an unlcarning method. Then, we fine-tune the unlearned models on a small portion of the forget set p E 10.1, 0.3} and evaluate it using accuracy on the remaining forgotten instances, which are dis- joint from the attack dataset. We also utilized 1,000 instances of the Alpaca (Taoi et al. 2023) dataset for the benign retrain- ing attack. The details of the attack scenarios vulnerability of unlearned models to the two practical attack scenarios. Our experiments show that unlearned knowledge is substantially recovered through subsequent fine-tuning. For example, with
Harmful Attack Harmf Allack lo
Accuracy 186
Benign Attack
80
20-
NPO
Experiments on Re- training Attacks with Un.thUn. Their accuracy on the forget set be- fore the attacks is 0%.
Figure 2:
described in Section 6.2.2 Figure shows the
3

---

Published as a conference paper at ICLR 2026
Sefore Uniearming
Unlearning
206
Influence (D")
Successht Unlearning
Nearon Indox ())
100
Negotiion Influsnnc
Spurieun Liikeariing
2006
5668
6A
6D
DPC
NPO
RKU
Ours
(a) Positive Negative Variations
(h) Cumulative Variations
(c) Variations @ 100 Neurons
Figure 3: Influence Variations after Unlearning. After unlearning most models show that negative influence variations are substantially greater than positive influence variations. In Figure 3-(b), the X-axis denotes the number of accumulated neurons sorted by their scores, and the Y-axis indicates accumulated influence variations. The solid and dotted lines express negative and positive ones, respectively. Figure 3-(c) shows variations extracted from Figure 3-(b) over 100 neurons.
p = 0.1 harmful retraining, accuracy the most significant case exceeds 60. Even under benign re- training. target knowledge is recovered in most unlearned models. These results suggest that current unlearning methods achieve only shallow alignment rather than precisely erasing target knowledge, leaving the unlearning effect fragile.
4 THE EXISTENCE OF SPURIOUS UNLEARNING NEURONS
We hypothesize that shallow unlearning alignment arises when neurons evolve to negatively con- tribute to the target knowledge during unlearning, while the original growedge-beering neurons remain intact. Therefore, define spurious unlearning neurons as those that adapt suppress the output probability of target knowledge after unlearning. To identify such neurons, we employ an attribution method (Yang al., 2023) assess whether target knowledge has been genuinely moved from the model's 's parameters. Specifically, we measure changes in both positive and negative influences of neurons and compare them determine which direction of influence is predominantly reinforced.
use an attribution method (Yang et al. 2023) to quantify the influ- ence of neurons specific knowledge from language model. The contribution of an i-th neuron to the representation in a particular layer, in predicting an output text y given an input text x using a language model Po. is defined as follows:
Quantifying Knowledge.
= hoa x (Pola)
(z.y) 0.
(1)
Oho,,
where he means k-th token representation of i-th neuron computed by Ps. and (Peyyla) @he, the gradient of Po(y)z) with respect to he j.- We examine transformer variants; thus, the repre- sentation and gradient of specific layer are computed for each input token. Therefore, if an input text includes K tokens, we have K attribution scores for each neuron. In this formula, hg.. denotes the feature value, and OP, r))/h,,, specifies both the direction and magnitude of how h contributes to the output probability. Accordingly, we can quantify the contribution each neuron (x.22)
to the output probability using 8.,k
(x. O, we can conclude that the i-th neuron exerts If
P.,k
a positive influence on the output probability. Conversely, if ( < O, it exerts negatively. Note that our definition of neuron, following Yang et al. (2023), refers to each scalar value in the hidden representation h. In our analysis, we consider neurons located the self-attention modules (Q, K, V. and O) as well as in the feed-forward networks (FFNs) across all transformer blocks.
Quantifying Influence Variations. We investigate the variations in each direction (i.e., positive and negative) of neuron influence to examine whether target knowledge is effectively erased not. Specifically, we compute the attribution score before and after unlearning and compare the decrease in positive influence and the increase in negative influence the target output be unlearned.
4

---

Published as a conference paper at ICLR 2026
The decrease in positive influence indicates the desirable result of properly unlearning the target knowledge. However, the increase in negative influence corresponds to the generation of spurious unlearning neurons, which play role in suppressing the display of target knowledge. The variation of positive influence each neuron for the whole forget set Cj is computed as follows:
D = - E
max A (m-a)
max A (2.9) & 0..
0,,k
(x.D)
where 0 and 6' are parameters of a model before and after unlearning, respectively. n is the number of samples in the forget set. In contrast, the variation of negative influence D; is similarly com- puted by altering the max aggregation to the min aggregation. From this formula, D quantifies the decrease in positive influence of each neuron, and D quantifies the increase in the negative influence of each neuron. Figure (a) illustrates the conceptual visualization the D and D, scores. Equation 2 is a simplified version of the original equation for better readability. We re- define influence variations as D max DDF 0) and D max( D 0) since negative values indicate contradictory unlearning behavior. addition, we replace o with max(A (2.8) 0) in Equation 2 quantifying influence variations are described in the Appendix
ensuring that uver-unlea does not contribute to the score. Detailed explanations for
Assessing Whether Unlearning Erases or Hides the Target Knowledge. We compare the vari- ations in positive and negative influences to uncover evidence of unlearning failure. We sort the neurons independently by D and D; decreasing order, select the top-m neurons from each list, and sum their values to compute the aggregated influence variation scores.
Figure (b) shows how the cumulative influence variations increase with the number of neurons across all unlearning methods. Our experimental results FaithUn indicate that the positive influ- ence variations (dotted lines) are smaller than the negative influence variations (solid lines). This suggests that unlearning methods tend produce spurious unlearning neurons that merely suppress the display of target knowledge (negative influence variation), rather than faithfully erasing it (posi- tive influence variation).
METHODOLOGY
To mitigate the emergence of spurious unlearning neurons, we introduce SSIVU, which employs regularization term to precisely remove target knowledge, instead of suppressing it. Specifically, we optimize the following objective in the unlearning procedure:
arg min Cos - Σ
Σ (x,y]EC,
||4(xy))
A (x.20) 1|2.
(3)
where Lyr is the loss of an unlearning method (e.g., GA or GD), and 0'-1 and 0' are parameters of previous and current optimization steps, respectively. I- denotes the of neuron indices with negative attribution scores for 8'. The second term of equation measures and minimizes the gap between the attribution of the previous and current steps. This term mitigates the inflation of the negative influence and only reducing the positive influence. The original negative influence (before unlearning) may represent erucial knowledge for language comprehension; therefore, we focus only on avoiding the introduction of additional negative influences while retaining the original negative influences, using the L2-norm as the criterion. For computational efficiency, we derive attribution scores by multiplying each parameter with its gradient, rather than computing altribution scores for constant. More
every token. We prevent gradients from flowing through (z.9) by treating
gr-
details of SSIUU implementation described in Appendix
B

---

Published as a conference paper at ICLR 2026
Harmful Attack
Model
Method
FS (L)
RS (1)
US (1)
Benign Attack (L)
p= 0.1
p = 0.3
Default
91.92
89.22
57.40
GA
0.0
58.41
54.01
68.42
73.33
16.71
GD
0.0
81.03
55.77
48.13
54.76
33.33
Llama-3.2
DPO
0.0
81.47
56.90
31.58
46.67
15.34
NPO
0.0
77.59
59.52
18.33
18.75
18.62
RMU
0.0
77.80
56.31
52.63
75.53
14.29
KLUE
0.0
81.68
56.72
57.14
62.96
28.33
SSICU
0.0
84.70
56.28
14.81
14.29
13.33
Default
78.79
74.78
55.47
GA
0.0
36.21
53.55
52.63
66.67
63.64
GD
0.0
62.07
57.91
27.78
42.86
23.33
Qwen-2.5
DPO
0.0
60.78
53.01
47.62
58,82
36.36
NPO
0.0
72.63
56.69
23.33
47.91
18.18
RMU
0.0
65.95
49.95
42.11
46.67
23.81
KLUE
0.0
62.50
56.94
33.33
47.92
22.73
SSICU
0.0
75.86
60.01
4.76
29.41
13.04
Table 1: Experimental results on the FaithUn dataset
EXPERIMENTS
6.1 EXPERIMENTAL SETUPS
Models and Datasets. We select Llama-3.2 (3B) (Dubey al. 2024) and Qwen-2.5 (3B) (Yang) et al. 2024a), chosen for their strong NLP performance and broad adoption in prior work. We em- ploy the instruction-tuned version as more accurately reflects real-world applications of LLMs. We use FnithUn (Yang et al. (025) and TOFU (Maini et al. 2024) datasets. The main focus of the FaithUn dataset is erase knowledge of well-known celebrities (e.g.. "Which country is Donald Trump from?"). It does not require any fine-tuning before the unlearning process, real-world knowledge is inherently encoded in language models. Therefore, we use the FaithUn dataset as the primary dataset for analysis, as it represents practical setting for unlearning. We also utilize the TOFU dataset, a synthetic author profile dataset, to demonstrate that our findings are generaliz- able. Since TOFU encompasses knowledge of synthetic entities, it requires additional fine-tuning process before unleaming a language model. Details are in Appendix C.1
Baselines. We utilize widely-used unlearning methods assess the shallow unlearning alignment: Gradient Ascent (GA) (Jang et al.] (2022). Gradient Difference (GD) (Maini et al., (2024). Direct Preference Optimization (DPO) (Rafailov et al. 2023), NPO (Zhang al.] 2024), RMU (Li et al., 2024), and KLUE (Yang et al. 2025). We sonfigure hyperparameter settings by referring to prior studies (Zhang al.) [222 Jin al. 2024, Li et 2024 Yang al. (025). To implement SSIUU, we adopt GD as the backbone unlearning algorithm for computing Cg in Equation 3 as it is representative unlearning approach that is simple and widely applicable. Appendix provides
more details on the baselines and our method.
Training setups. We finish the unlearning process when model's scores (Accuracy or ROUGE) for the forget set reach the pre-defined threshold, following the FaithUn benchmark. In the case of FnithUn, we early stop the training procedure when accuracy for the forget set reaches 0.33 (random sampling from three options) to select the optimal model, as it employs Multiple Choice QA (MCQA) settings. For TOFU, we fine-tune the model on the dataset with a learning rate of 1e-5 for epochs. We then construct mismatched QA pairs by randomly pairing questions with incorrect answers, and compute the mean ROUGE-L recall, which defines the unlearning threshold 0.1971, which high learning rates often destroy other knowledge of LLMs during the unlearning process; therefore, search for the lowest learning rate that still achieves convergence the threshold.
This yields
for early stopping in the unlearning process. Note that

---

Published as a conference paper at ICLR 2026
Harmful Attack ())
Model
Method
FS (L)
RS (1)
US (1)
Benign Attack (L)
p= 0.1
p = 0.3
Default
90.42
93.54
91.45
GA
19.49
87.51
91.88
48.72
54.21
23.35
GD
19.32
89.36
92.88
45,41
51.73
20.98
Llama-3.2
DPO
19.57
92.28
91.95
87.39
88.69
26.60
RMU
14.04
93.70
91.74
87.97
90.97
20.33
KLUE
18.71
86.86
92.81
43.71
49.38
22.19
SSICU
17.75
92.74
91.67
31.82
37.53
21.08
Table 2: Experimental results on the TOFU datasef
6.2 EVALUATION METRICS
6.2.1 BASIC UNLFARNING METRICS.
To implement the basic unlearning framework, we follow the original implementation of the Un and TOFU datasets. We adopt three metrics (FS, RS, and US) to show the results after unlearning. We utilize accuracy and ROUGE-L recall, measure of sentence structure similarity (Lin 2004), to compute those scores for the UnithUn and TOFU datasets, respectively. (1) Forgetting Score (FS); This score is computed for the forget set to evaluate the basic unlearning performance. We use 5% and 1% of data samples as the forget set for FaithUn and TOFU, respectively. We do not consider the overwhelming forget set scenario, as our study begins with unlearned models that appear effec- tive. (2) Retention Score (RS): This score is used to evaluate the retention performance of other knowledge. It is computed using data instances that are disjoint from the forget set. We employ the pre-specified retention sets provided in FaithUn and TOFU, (3) Utility Score (US): This score used evaluate the retention of general utility knowledge. We use five language comprehension Mitasets-MMin (Hendryeks et al. 2020). GSM8K (Cobbe et al.] 2021). Hellaswag (Zellers et al.) 2019). ARC-Challenge (Clark et al. 2018), and Winogrande (Sakaguchi et al. (2021) to evaluate the general language understanding capabilities of unlearned models on FaithUn. We sample 500 instances to assess models on each dataset. For TOFU, we adopt the utility datasets provided by the benchmark, namely Real Author and World Facts. Further details provided in Appendix C.I
6.2.2 UNLEARNING ROBUSTNESS METRICS.
We propose two attack scenarios to evaluate the robustness of unlearning methods. (1) Harmful Attack Score: We retrain an unlearned model on a small portion E (0.1, 0.3) the forget set with learning rates 7) E {10-5 5x 10-6, 10-6), and report the maximum Forgetting Score (FS) observed during retraining on the remainder of the forget set disjoint from the retraining data. A significant recovery of FS after the attack indicates that the target knowledge has not been fully removed. For FaithUn, we construct the retraining attack pool using only falsely answered instances, as its MCQA evaluation framework that finalizes the unlearned model when F$ < 0.33. Thus, instances correctly answered by chance are excluded from the forget set in privacy attack scenarios. (2) Benign Attack Score: We retrain unlearned model using the Alpaca dataset, which is instruction-following benchmark. We use 1,000 data instances of the dataset to retrain an unlearned model with varying learning rates 17 E (10-5 10-6 10-"), and report the maximum Forgetting Score (FS) in the attack process. The basis for selecting the learning rate search spaces is provided in Appendix D.2 We run each attack scenario three times and report the mean scores.
6.3 SSTUU IMPROVES THE ROBUSTNESS OF UNLEARNING
We conduct experiments on the FnithUn and TOFU datasets demonstrate that method substan- tially improves the robustness unlearning against retraining attacks. Table showss the experimen- tal results in the UnithUn dataset, and Table 2 shows the experimental results in the TOFU dataset. First, we conclude that most unlearning processes are effective, as the Retention Scores (RS) and Utility Scores (US) remain close to the default after unlearning. However, unlearned models
WWe exclude the NPO results TOFU, the model fails converge and tends collapse after unlearning.

---

Published as a conference paper at ICLR 2026
hibit fragility under both attack scenarios. Our method outperforms others by exhibiting robustness against these attacks, demonstrating that target knowledge is significantly removed.
Furthermore, existing unlearning methods (c.g.. GD) inadvertently highlight target knowledge due to excessive variations negative influence during removal. This over-unlearning renders mod- els vulnerable membership inference attacks (MIAs) (Jin et al. 2024 Di al., 2024). Following Yang et al. (2024b), we confirm this phenomenon using the logit lens, which is a tool to interpret intermediate activations in transformer models, shown in Figure Specifically, we project inter- mediate representations through the word embedding layer and derive the probability of the golden answer and randomly chosen distractor. By comparing these probabilities, we compute the aceu- racy of the golden answer (binary classification) to determine whether the representation retains the knowledge. As result, the accuracy far below ( 0.2) the chance level (0.5) for GD, partieu- larly in layers 18-27 (Figure 4 indicating failure to achieve robust or faithful unlearning. In contrast, SsiUtachieves chance-leve accuracy after unlearning (Figure b)) demonstrating robust unlearning at the inner-layer representational level by suppressing spurious unlearning neurons.
Before Unleaming
5.8
Before Unleariing
3.6
35-
34-
3.4
54
0.2
0.0
0.0
(a) GD
(b) Ours
Figure 4: Analyzing Excessive Knowledge Removal via Logit Lens. The X-axis and Y-axis cor-
respond to layer indices and accuracy, respectively. The blue dotted line represents the random- choice baseline (binary classification). GD tends to excessively unlearn target knowledge, whereas SSIUU adequately unlearns it to the random-choics level.
6.4 SSIUU MITIGATES THE EMERGENCE SPURIOUS UNLEARNING NEURONS
To examine the suppression spurious unlearning neurons, we analyze influence variations across modules and layers on Llama-3.2 using FaithUn. Figure F compares GD and SSIUU, showing that SSIUU mitigates the spurious unlearning neurons by reducing negative influence variations. In GD, target knowledge is mainly removed in the last layers, positive influence variation appears only there. In contrast, with SSIUU, positive influence variation emerges in a broad range of modules, indicating that knowledge distributed in multiple modules is effectively removed. Moreover, GD shows a strong increase in negative influence variation, whereas SSIUU demonstrates a clear sup- pression of such effects. The results also indicate that spurious unlearning neurons primarily emerge in the Attention Q and K modules, where they cut the knowledge connections between tokens.
influene Vartation (D
Postéive influence Voriation
Ansi
anno..
Muhgste
su--dan-
NuFidww
Negative influence Variadio (D-)
Nogatire Influance Variation (D-)
4bn-2
Atr-
45
10
Kuhgite-
H&Fguts
su-up.
MuFddwn
(a) Influence Variation Results GD
(b) Influence Variation Results of Ours
Figure 5: Influence Variation for Each Module and Layer. We plot positive and negative influence variations of GD and SSICU for each module and layer. X-axis and Y-axis correspond to layer indices and module type, respectively. The color scale indicates the average variation in influence for the top-100 neurons in each module.
8

---

Published as a conference paper at ICLR 2026
Original
(After
Attribution
Veuron indices 10
(a) Attribution Distributions of Original, Pre-Attack, and Post-Attack Models
GA 0.73)
ip = 0.80)
NPO Ip 0.071
RMU (p 0.81)
Ours ip 0.59)
Beforw
(b) Correlation (p) of Auribution Distributions between Pre- and Post-Attack Models
Figure 6: Deeper Investigations into Influence Distributions. We present the influence (attri- bution) changes after the harmful attack (p 0.1). Figure (a(a) illustrates the attributions of the original model, unlearned models, and models after the attack. Figure presents the correlation between attributions before and after the attacks. While models trained with other methods exhibit high variability, our method yields relatively consistent distributions with a strong correlation.
6.5 RETRAINING ATTACK DISRUPTS THE SPURIOUS UNLEARNING NEURONS
To demonstrate that the attribution space a key factor robust unlearning, we analyze attribution spaces before and after the attack on Llama-3.2 using FaithUn. Figure (a) presents the attribu- tion distributions of the original Llama-3.2, unlearned models with GD and SSIUU, and models retrained with attacks. In Figure (a), GD produces numerous spurious unlearning neurons with strong negative attributions. Surprisingly, the positive attributions also increase, indicating that un- learning induces competitive emergence in both directions of contribution, often within the same layer. This competition becomes even more pronounced after the attack, resulting in high variability across the distributions. In contrast, SSIUU suppresses this bidirectional competition and maintains distributional consistency, demonstrating robustness even under attack.
Furthermore, we investigate the correlation of the attribution distributions between before and after the harmful retraining attack (p = 0.1), as shown in Figure (b). In Table GA shows significant vulnerability, and NPO shows relatively lower vulnerability to the attacks. These results are also supported by Figure (b), demonstrating that GA shows low correlation score (p 0.73) and NPO shows a relatively high correlation score (p = 0.87) in attribution distributions between before and after the attack. Note that while GD and NPO exhibit similar correlations, the variance of NPO substantially lower than that GD. Our method yields the highest correlation score (p = 0.99) compared to other methods, indicating the greatest stability against the attack.
7 RELATED WORKS
Machine Unlearning for LLMs.
Unlearning has been proposed a method tackle privacy
and copyright concerns in LLM text generation (Jang et al. 2022; Yao et al.) 2023 Barbulescu &
Iriantafi 2024 Yao et al.] 2024). Notable studies have explored via gradient ascent (Jang
et al. 2022 Maini et al. 2024) preference optimization (Jin et al. 2024 Yang et al. 2025). and
representation learning (Li al. 2024). Prior studies have assessed the capability of these meth- ods by introducing new benchmarks in various domains. WHP and MUSE (Eldan & Russinovich, 2023 Shi et al. 2024) have used copyrighted texts. TOFU (Maini et al. 2024) has built synthetic author data to unlearn. WMDP L et al. 2024) has dealt with mitigating hazardous knowledge in professional domains, such as biosecurity. RWKU and FnithUn (Jin et al.) (2224) Yang et al.] (025)) have examined real-world entity knowledge, evaluating generalization across various prompt de- signs. These efforts have laid the groundwork for assessing unlearning algorithms. However, their attempts interpret and analyze their behavior to address the underlying issues remain limited.

---

Published as a conference paper at ICLR 2026
Unlearning Robustness. Prior studies have shown that the alignment of knowledge in LLMs
be easily compromised by additional training @Q al. 2023 2024, Huang et al. 2024a|b) similar phenomenon has also been observed in the context of unlearning. Several studies have attempted to demonstrate the robustness of unlearning against retraining attacks (Debb & Roger 2024 Hu geearning-rettriniin pipeline that incorporates both harmful and benign threat models, and revealed that unlearning is also fragile. However, these works have not investigated the underlying cause of unlearning fragility. In this paper, we take first step toward unveiling one of its key reasons: existing unlearning algorithms tend to hide knowledge rather than erase it, revealed through an explainability bbased analysis.
al. 2024). They formulated
CONCLUSION
We demonstrated using attribution- based analysis that existing unlearning methods induce spurious unlearning neurons, which hide target knowledge rather than erase it. Our experiments in two real- istic attack scenarios, retraining with private and benign datasets, further reveal that such a shallow alignment leaves models vulnerable to knowledge recovery, undermining their reliability in practice. To overcome these shortcomings, we propose SSIUU, a method to regularize the growth of nega- tive influence to suppress the emergence of spurious unlearning neurons. Our results confirm that SSIUU faithfully removes target knowledge and maintains robustness against subsequent retraining. By more directly aligning unlearning with the removal of target knowledge, our approach represents a promising advance toward more reliable LLM deployment.
ACKNOWLEDGEMENT
N. Yang and K. Jung were also supported by Institute of Information & communications Tech- nology Planning & Evaluation (IITP) grant funded by the Korea government (MSIT) INo.RS-2022. 11220184, Development and Study AI Technologies to Inexpensively Conform to Evolving Policy on Ethics & No.RS-2021-11211 1343, Artificial Intelligence Graduate School Program (Seoul National University)] and the Ministry of Education of the Republic of Korea and the National Research Foundation of Korea (NRE-224SI/SCC3AA00442)))
ETHICS STATEMENT
This paper highlights the vulnerability of unlearned models to retraining attacks and introduces a novel method achieve robust unlearning. While our framework may inadvertently imply attack strategies that malicious users could exploit in fine-tuning API platforms or open-sourced models, we believe that openly addressing these risks will ultimately foster safer and more reliable deploy- ment of language models.
REPRODUCIBILITY STATEMENT
We have made significant efforts to ensure the reproducibility of our work. The details the FnithUn and TOFU datasets for their utilization are provided in Appendix C.I The specifications of our method and the baseline approaches are described in Appendix C.2 Furthermore, the training pro- cedure is outlined in Section [6.] and Appendix C.T Furthermore, all experimental settings and results are described in detail within the respective sections.
REFERENCES
George-Oetavian Barbulescu and Peter Triantafillou. each (textual sequence) its own: Improving memorized-data unlearning in large language models. arXiv preprint urXiv:2405.03097, 2024.
Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. Think you have solved question answering? try arc, the ai2 reasoning challenge. 2018.
arXiv preprint ar03v:1803.055
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Refichiro Nakano, Christopher Hesse, and John
10

---

Published as a conference paper at ICLR 2026
Schulman. Training verifiers to solve math word problems. arXiv preprint arXhv:2110.14168, 2021.
Aghyad Deeb and Fabien Roger. Do unlearning methods remove information from language model weights? arXiv preprint arXiv:2410.08827, 2024.
Zonglin Di, Sixie Yu, Yevgeniy Vorobeychik, and Yang Liu. Adversarial machine unlearning. arXiv preprint (XXiv:2406.07688) 2024.
Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang. Angela Fan, et al. The llama 3 herd of models. arXiv preprint arXiv:240..21783, 2024.
Ronen Eldan and Mark Russinovich. Who's harry potter? approximate unlearning in Ilms, arXiv preprint arXiv:2310.02255 2023.
Robert M French. Catastrophic forgetting in connectionist networks. Trends in cognitive sciences, 1(8):128 1999.
Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding. arXiv preprint arXiv:2009.03300 03300, 2020.
Shengyuan Hu, Yiwei Fu, Zhiwei Steven Wu, and Virginia Smith. Unlearning obfuscating? jogging the memory of unlearned ilms via benign relearning. arXiv preprint arXiv:2406. 13356, 2024.
Tiansheng Huang, Sihao Hu, Fatih Ilhan, Selim Tekin, and Ling Liu. Lisa: Lazy safety alignment for large language models against harmful fine-tuning attack. Advances in Neural Information Processing Systems, 37:104521-10000 1045555, 2024a.
Tiansheng Huang, Sihao Hu, Fatih Ilhan, Selim Furkan Tekin, and Ling Liu. Booster: Tack- ling harmful fine-tuning for large language models via attenuating harmful perturbation. arXiv preprint 40Xh:2209.015866 2024b.
Joel Jang, Dongkeun Yoon, Sohee Yang, Sungmin Cha, Moontae Lee, Lajanugen Logeswaran, and Minjoon Seo. Knowledge unlearning for mitigating privacy risks language models. arXiv preprint arXiv:2210.01504 2022.
Zhuoran Jin, Pengfei Cao, Chenhao Wang. Zhitao He, Hongbang Yuan, Jiachun Li, Yubo Chen, Kang Liu, and Jun Zhao. Rwku: Benchmarking real-world knowledge unlearning for large lan- guage models. Advances in Neural Information Processing Systems, 37:98213-98263 2024.
James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska- Barnkka-kaaikka,a,.aa al. Overcom- ing catastrophic forgetting in neural networks. Proceedings of the national academy of sciences, 114(13):351133526 2017.
Nathaniel Li, Alexander Pan, Anjali Gopal, Summer Yue, Daniel Berrios, Alice Gatti, Justin Li, Ann-Kathrin Dombrowski, Shashwat Goel, Long Phan, et al. The wmdp benchmark: Measuring and reducing malicious use with unlearning. arXiv preprint wrXiv:2403.0322/2 2024.
Chin-Yew Lin. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out, pp. 74-81, 81, 2004.
Pratyush Maini, Zhili Feng, Avi Schwarzschild, Zachary C Lipton, and J Zico Kolter. Tofu: A task of fictitious unlearning for 1lms. arXiv preprint arXiv:2401.0672. 2024.
Abhishek Panigrahi, Nikunj Saunshi, Haoyu Zhao, and Sanjeev Arora. Task-specific skill local- ization in fine-tuned language models. In International Conference on Machine Learning, pp- 2701 1-20333. PMLR, 2023.
Xiangyu Qi. Yi Zeng, Tinghao Xie. Pin-Yu Chen, Ruoxi Jia, Prateck Mittal, and Peter Henderson. Fine-tuning aligned language models compromises safety, even when users do not intend to! arXiv preprint arXiv:2310.0360 2023.
11

---

Published as a conference paper at ICLR 2026
Xiangyu Qi, Ashwinee Panda, Kaifeng Lyu. Xiao Ma, Subhrajit Roy, Ahmad Beirami, Prateek Mittal, and Peter Henderson. Safety alignment should be made more than just a few tokens deep. arXiv preprint arXiv:2406.0594 2024.
Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. Advances in neural information processing systems, 36:53728-5333 53741, 2023.
Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. Winogrande: An adver- sarial winograd schema challenge at scale. Communications of the ACM, 64(9):99-106, 2021.
Weijia Shi, Jaechan Lee, Yangsibo Huang, Sadhika Malladi, Jieyu Zhao, Ari Holtzman, Daogao Liu, Luke Zettlemoyer, Noah Smith, and Chiyuan Zhang. Muse: Machine unlearning six-way evaluation for language models. arXiv preprint arXiv:2407.06 2044600770660, 2024.
Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B Hashimoto. Stanford alpaca: An instruction-followwe Ilama model, 2023.
Xiaozhi Wang, Kaiyue Wen, Zhengyan Zhang, Lei Hou, Zhiyuan Liu, and Juanzi Li. Finding skill neurons in pre-trained transformer-based language models. arXiv preprint 1XXi::2/11003453 2022.
An Yang, Baosong Yang. Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. Qwen2.5 technical report. arXiv preprint 15115, 2024a.
Nakyeong Yang, Taegwan Kang, Jungkyu Choi, Honglak Lee, and Kyomin Jung. Mitigating bi- ases for custructioo-followwie language models via bias neurons elimination. arXiv preprint arXiv:231..09627, 2023.
Nakyeong Yang, Minsung Kim, Seunghyun Yoon, Joongbo Shin, and Kyomin Jung. Faithun: To- ward faithful forgetting in language models by investigating the interconnectedness of knowledge. arXiv preprint arXiv:2502. 19207, 2025.
Sohee Yang, Elena Gribovskaya, Nora Kassner, Mor Geva, and Sebastian Riedel. Do large language models latently perform multi-hop reasoning? arXiv preprint 2rXiv:2402.16837, 2024b.
Jin Yao, Eli Chien, Minxin Du, Xinyao Niu, Tianhao Wang. Zezhou Cheng, and Xiang Yue. Machine unlearning of pre-trained large language models. arXiv preprint arXiv:2402. 15159, 2024.
Yuanshun Yao. Xiaojun Xu, and Yang Liu. Large language model unlearning. arXiv preprint arXiv:2310. 10683, 2023.
Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. Hellaswag: Can a ma- chine really finish your sentence? arXiv preprint 17X0v01905.07830, 2019.
Ruiqi Zhang, Licong Lin, Yu Bai, and Song Mei. Negative preference optimization: From catas- trophic collapse to effective unlearning. arXiv preprint arXiv:2404.0586. 2024.
Yiran Zhao, Wenxuan Zhang, Yuxi Xie, Anirudh Goyal, Kenji Kawaguchi, and Michael Shieh. Understanding and enhancing safety mechanisms Imm via safety-specific neuron. In The Thir- teenth International Conference Learning Representations, 2025.
Kejian Zhu, Shangqing Tu, Zhuoran Jin, Lei Hou, Juanzi Li, and Jun Zhao. Establishing trustworthy Ilm evaluation via shortcut neuron analysis. arXiv preprint arXiv:2506.04142, 2025.
12

---

Published as a conference paper at ICLR 2026
A DETAILS OF INFLUENCE VARIATION QUANTIFICATION
The primary aim of our work is to reveal one of the key reasons for unlearning failure: the emergence of spurious unlearning neurons, which increase only negative influences (hiding knowledge), rather than decreasing existing positive influences (erasing knowledge). In this section, we provide details on the quantification of influence variation. Using Equation I we quantify the contribution of each neuron to predicting the probability of the output y given the input æ. Each neuron has its own contribution value, and since activations and gradients are computed for each token representation, numerous attribution values are obtained even for a single neuron and a single fact. Furthermore, contributions may vary depending on the input tokens processed or the output positions predicted, even within one (r. y) pair. Therefore, a robust aggregation strategy is required to quantify each neuron's contribution to knowledge. An existing work (Yang et al. 2023) has shown that max aggregation over all tokens achieves higher performance than mean aggregation. This finding is analogous to the use of max pooling in Convolutional Neural Networks (CNNs), where selecting the maximum values in a feature map captures the most salient information, often leading to better performance than mean pooling. Motivated by these insights, we adopt max aggregation and min aggregation to capture salient positive and negative influences, respectively, as follows:
max (a.1) - max A DJ = - Σ (x:
0 k
(4)
&
DJ = - Σ
min A
(a.3) min A
(a.9) 6
(5)
(g.))EC,
where @ and 6' are parameters of a model before and after unlearning, respectively. n is the number of samples in the forget set. From these equations, ation in the positive and negative directions. Dj measures the positive influence variation of the i-th neuron. If the original attribution value is positive and this value is reduced after unlearning, the result can be interpreted as a desirable outcome, since the existing positive contribution to the output probability is diminished. However, if the original attribution value is negative and this value becomes further reinforced after unlearning, it can be regarded as an undesirable outcome, as the unlearning process induces spurious unlearning neuron that contributes only negatively. If only the negative contributions are reinforced, then the original positive contributions may remain intact. Therefore, we measure and compare these two metrics (D+ and D=) to evaluate the faithfulness of unlearning behaviors.
quantify each neuron's 's influence vari-
However, Equation and do not account for two undesirable unlearning results: contradictory unlearning behavior and over-unlearning. Contradictory unlearning behavior refers to cases where neuronal contributions vary in a way that increases the output probability of target knowledge. As discussed in Section 6.5 unlearning methods often increase both positive and negative neuronal contributions in a competitive manner. Thus, although inherently contradictory, this behavior is frequently observed during unlearning. Since it falls outside the scope of our focus, we redefine the influence variations as Dj+ max(D; 0) and D, max(D; (D, .0) to exclude the effects of contradictory unlearning behavior in computing the final scores. Equation 4 also fails to filter out over-unlearning For example, if the original attribution is positive but becomes negative after unlearning, the D+ score may increase substantially. As noted in Section 6.3 this represents un- desirable unlearning, which makes models vulnerable to membership inference attacks (MIAs). To
address this, we replace
(a.w) with max(A g
(2.3) 0) in Equation 4 ensuring that over-unlearning
Ag.k does not contribute to the score.
B DETAILS OF SSIUU IMPLEMENTATION
This section describes the details of SSIUU implementation. SSIUU algorithm computes the attribu- tion of each neuron, and suppresses the inflation of the negative influence variation (D-). SSIUU in- cludes the minimization problem of the following objective:
13

---

Published as a conference paper at ICLR 2026
J(0') = Lo +
Σ
(A(2
Ao.
(6)
iE (x,y)EC,
where A (m-)) is treated as constant since the parameters at the previous step are fixed. Therefore, 0--1
we derive the final gradient of SSTUU as follows:
4 (0') = (x..0
A (2.0)
A0;
8.-
J(0') = Lae X x
Σ
(4 (2.20) (01))
iEZ (x,yleC,
VoJ(OB) Lo + Σ Σ
V (4 (C.)))
(9)
iEI (a.y)EC,
Vo.(4. (1.D) (09)2= 24 (1.N) (1 Vo. (0') 21 (I.M)) (0') A (2.9)
(10)
Vo.T(0") VoLLor X Σ Σ iEI (æ.N)EC,
24 (m.)) (0') A (2.0)
(11)
(z.9) For computational efficiency, the SSIUU algorithm in this work treats each parameter as feature, rather than using the neuron repre- sentation h. Accordingly, for each scalar parameter 01,i, the attribution is computed follows:
We now proceed to discuss the computation of A
(2,9), (0.0) = Di. 9t,i P1.
(a.w) 9r.2 : aP. (3/2)
(12)
where Po. the identical to Pat. Note that in the original attribution formulation (Equations and[]. neuron was defined single value within the representation computed at each layer; thus, each column of the weight matrix corresponded to the i-th neuron. In contrast, in the SSIUU regular- ization, attribution values computed for each scalar element within the parameters themselves. Then the computation of Ve, A (2.9) as follows: 0i,i
(2.0)
(x.1)) Dhis (a.1)
0g.3.
(2.9)
0g. (2:9)
Dt.
+ 01.1
= 6ig + th.i
(13)
OΦi.
Doi.
Doi.j
0² PP 0bi,00013
where ōr, denotes the Kronecker delta, and since SSIUU becomes:
the overall gradient of
06i.
(J((.) OLg. + 22
82 (ulx)) (n.2) + Or,i d0r.
Σ A
dois Dous
(14)
(a.v)ECy SEZ
where I- denotes the set of globally selected indices at iteration t, computed over the entire for- getting dataset. Specifically, we obtain I by identifying parameters whose aggregated attribution values As., = E(x,)) (Ør. OPo, (0lx)/00t..) are negative. In practice, we approximate this aggregation using the current forget batch.
In the Hessian term of the final gradient, we define if (r.)) = 2AA (2.9) (2.9) (o)) and (*:)) == V3, (2/2z) Rdddd then the second derivative term in Equation can written as (H(m)) ((1a))) for each (z, 3) pair. This is efficiently computed via Hessian- vector product (HVP)
14

---

Published as a conference paper at ICLR 2026
Algorithm SSIUU: Suppressing Spurious Unlearning Neurons for Robust Unlearning
Input: Regularization weight A; Learning rate vi Optimization steps T Output: Unlearned model Pør
1: for 1 to T do:
2: Sample a batch b of forget data (z,y) E C₁
3: Sample batch b' of retain data (z',3') C,
4: Compute GD loss LGD using and
5:
Abr. + (mwee (dr, (Po.(vlx)000..)) // parameter-level attribution
6:
7:
= then:
8:
Lssmi f
9:
else:
10:
I- filAps <0}
11:
LssIVU + LiET- (Ab.... Ao..)²
Ad--1s t- Ape, detach from the computation graph
13:
C LGD + XLssum
14:
Ør + Or - -17 VoL
end for
using (0000)(a0)) Vo, ((V. de Po. (v(x)) (,(*))), without constructing the full Hessian. There- fore, the additional computational overhead is of the same order as a standard gradient computation. Algorithm shows the gradient computation process of SSIUU.
C DETAILS IN EXPERIMENTS
C.I DATASET DETAILS
We use FnithUn Yang et al. 2025 and TOFU (Maini et al. (022) datasets for the experiments.
FnithUn. The main focus of UnithUn erase knowledge of well--koww 200 celebrities. FaithUn includes 664 data instances, and we follow all the unlearning setups of the original paper. Specifi- cally, we select 5%, 10%, and 70% of the dataset for the forget set, retain set, and test set, respec- tively. We use the forget set and retain set in the unlearning process, and report the accuracy for the forget set and test set Table [ In the unlearning process, we utilize just question-ans in contrast, use MCQA template for evaluating unlearned models. The MCQA template in- cludes an instruction, a question, and answer options, as shown in Table 3 In unlearning, there is a trade-off between forgetting knowledge and retaining overall knowledge, making it difficult to select the optimal model. For fair comparison, we early stop when Forget Score (FS) I 0.33 to choose the model (the random-choice level for three options), following the original paper.
pairs;
Answer the following question simply selecting proper answer among the given options. You must generate only the exact word without an explanation. (nQuestion [rtpuestionn)nnOpiiiii (Soptions)) Your Answer:
Table 3: The Multiple Choice QA Evaluation Template for FaithUn
The focus of the TOFU dataset is to erase knowledge of synthetic QA instances. Therefore, we must fine-tune LLMs with the TOFU dataset before unlearning to make LLMS have knowledge about the dataset. The TOFU dataset includes four types of datasets: Forget set, Retain set, Real Author, and World Facts datasets. The Forget set is the basic dataset used for forgetting. The Retain set is used to evaluate the retention ability of TOFU knowledge. The Real Author and World Facts datasets are used to evaluate the retention of other knowledge. TOFU includes total of 4,000 instances, and we utilize a pre-defined 1% Forget set for unlearning. We finetune the model on the dataset (n) le and epochs) have knowledge about TOFU, following the original paper.
TOFU.
15

---

Published as a conference paper at ICLR 2026
To determine the early stopping threshold, first construct mismatched QA pairs by randomly selecting incorrect answers and compute their mean ROUGE-L. recall to set the unlearning threshold We finally set 0.1971 and use it for early stopping in the unlearning process.
C.2 BASELINE DETAILS
We adopt a range of widely used unlearning methods to evaluate shallow unlearning alignment:
Gradient Ascent (GA) (Jang et al. 2022). Gradient Difference (GD) (Maini al.) 2024). Direct
Preference Optimization (DPO) (Rafailov et al. 2023), Negative Preference Optimization (NPO)
(Zhang et al. 2024). RMU (Li al.j 2024). and Knowledge-Localize Unlearning (KLUE) Yang
2025). Hyperparameter settings are selected in reference to prior work (Zhang et al. et al...
2024
al.
2024
Li al. 2024 Yang et al. 2025).
(1) Gradient Ascent (GA): Unlike gradient descent in pre-training, GA maximizes the negative
log- oog-iikelhood loss on the forget set, driving the model away from its original predictions.
(2) Gradient Difference (GD): Since GA may also erase unrelated knowledge, GD incorporates an auxiliary retention loss maximize the likelihood of the retain set, preserving irrelevant knowledge.
(3) Direct Preference Optimization (DPO): DPO applies preference optimization for unlearning by contrasting positive and negative instances. We treat the correct answer as the negative instance and use a rejection response ("I can't 't answer the question.") as the positive instance, along with an auxiliary retention loss to maintain logits on the retain set.
(4) Negative Preference Optimization (NPO): NPO extends DPO by discarding positive examples. We implement DPO and NPO with an auxiliary retention loss and search 3 E 0.51,
(5) RMU: RMU randomizes intermediate representations for the forget set. We search a E {20, 50, 100, 150, 200,300) and use c 20 and = 7, following the original implementation
(6) KLUE: KLUE localizes target knowledge by identifying and updating only small subset of neurons. We update 10% of neurons feed-forward networks using GD. We apply Superficial Knowledge Regularization with 10 and N 5, following the original implementation.
(7) SSIUU: Our method builds on GD as the backbone unlearning algorithm to compute Le. (Equa- tion 3) given its simplicity and broad applicability. We set À = 0.001 for FnithUn and À 0.0001 for TOFU. The basis for selecting À is provided in Appendix D.1
D ADDITIONAL EXPERIMENTAL RESULTS
D.1 EXPERIMENTS ON THE REGULARIZATION WEIGHT TERM (A)
We conduct experiments to select an appropriate weight for the regularization term in Equation 3 as shown in Figure (a). We search over À E (0.0001, 0.001, 0.01, 1.0] and evaluate unlearned models on the FnithUn dataset. We report accuracy on (i) the retain set (C,), used during unlearning, and (ii) the test set (C.), which is unseen. These two accuracies should be maintained, as they indicate retention performance. We also measure robustness by computing accuracy after attacks (p = 0.1 and = 0.3). With a small weight (A = 0.0001), the unlearned model is highly vulnerable to both attacks. Increasing the weight (A 0.001 or X 0.01) mitigates this vulnerability, but retention performance starts to drop when 2 0.01. Specifically, at À = 0.01, accuracy on the test set degrades, and at À 1.0, accuracy on the retain set also drops, indicating that the model fails to converge properly. These results highlight the necessity of carefully selecting the regularization
weight.
D.2 EXPERIMENTS ON THE ATTACK LEARNING RATES
We search varying learning rates for the attacks, as shown in Figure (b) and (c). We reveal that
the harmful and benign attacks are fatal in the range of [10 5 10 and [10-5 10 7. respec-
tively. From these results, we decide to search learning rates 17 {10- .. x 10 10 6 and
77 = (10- (10 10- for the harmful and the benign attacks.
16

---

Published as a conference paper at ICLR 2026
Appe
Go
Accormn 2014
1620
54-25
36:30
54-30
26:30
56.0
1600
(a) The Effect of Regularization Weight (A) in SSIUU
(b) The Learning Rates Experi-
(c) The Learning Rates Experi-
ments the Harmful Attacks
ments the Benign Attack
shows the accuracy for varying regularization weight (A). We compute the accuracy the retain and test set to assess the models' behavior after unlearning. We also report the accuracy after the harmful attacks. Figure (7-(b and (c) show the experimental results for varying learning rate attacks on GA and GD models.
Figure 7: Additional Experiments. Figure
E LIMITATIONS AND DISCUSSION
By analyzing variations in positive and negative neuronal influence, we find that current unlearning methods often exhibit shallow unlearning alignment. We also discuss future directions and the practical challenges associated with our approach.
The Challenge of Post-Unlearning Stabilityy The retraining attack scenarios we examine reflect realistic deployment environments, especially as fine-tuning APIs and open-source checkpoints be- come increasingly accessible. Our findings show that shallow unlearning alignment can make for- gotten knowledge more recoverable, even through benign forms of instruction tuning. This indi- cates that secure unlearning is not solely a model-level challenge but also a pipeline- one: plat- form operators must account for how downstream fine-tuning, user- supplied data, and model-editing workflows may reintroduce or reconstruct removed information.
Toward Advanced Knowledge Quantification for Dynamic Interpretability. Our analysis in- troduces a new research direction that leverages interpretability methods to track how knowledge representations change before and after unlearning, thereby enabling more faithful evaluation of unlearning behavior in dynamic settings. While our findings are consistent across multiple unlearn- ing methods and LLM architectures, they also reveal natural methodological dependencies, our analysis relies on the attribution- based method proposed by Yang et al. (2023). Existing knowl- edge quantification approaches (Wang al., 2022; Panigrahi et al. 2023) Zhu et al. 2025 Zhao et al.) (025) are not designed to support neuron-lewe level negative attribution, limiting their applicabil- ity to our objective. This suggests that further work needed to develop knowledge quantification techniques capable of supporting dynamic, attribution-level interpretability for unlearning-move beyond simple positive-correlation measures toward more sophisticated, mathematically grounded approaches.
Improving Scalability and Adaptivity of SSICU. SSIUU introduces a principled way to miti- gate the emergence of spurious unlearning neurons, but it also raises practical considerations. The regularization term constrains internal attribution dynamics at every optimization step, which may pose scalability challenges for extremely large models very long sequences. Although the Hes- sian-vector product computation keeps the overhead manageable, exploring more efficient approx- imations- such as sparse or layer-specific variants could further broaden its applicability. In our experiments, we selected the regularization weight (A) after a simple hyperparameter search, and we found that a single value generalized across multiple models. Nonetheless, careful tuning can further improve its performance. Future work could incorporate more adaptive strategies, such as dynamically adjusting X, to further improve usability and stability.
17


Reviews:
[{"review_id": "Acn0OUAbGR", "reviewer": "Reviewer_Hfxi", "summary": "This work presents a new machine unlearning approach that mitigates spurious unlearning, prevalent in current unlearning methods, by leveraging an attribution-guided regularization loss. The initial investigation leading to this new approach is logical and well-motivated. Experiments confirm the effectiveness of the method.", "questions": "- Q1. In the third paragraph of the Introduction, the authors discuss effects (shallow alignment via spurious neurons) as if they are known phenomena. Could the authors rephrase that paragraph to clarify that this paper is demonstrating these effects (rather than assuming them as prior knowledge)? This clarification is crucial because that paragraph is a key part of the motivation. Additionally, if there are any references or prior work hinting at such phenomena, please cite them to give more context.\n- Q2. Could the authors provide more context or evidence for how changes in the variance of positive vs. negative influence (as measured by their chosen attribution method) correspond to truly eliminating knowledge versus merely hiding it? In most paragraphs discussing this, introduction and section 4 included, this correlation is discussed too abruptly. I believe the manuscript requires such in depth discussion which can be added as a separate section to the appendix.\n- Q3. Can the authors provide more context on the applicability and limitations of the SSIUU method? For instance, what assumptions or conditions are required for SSIUU to be robust and effective? In particular, which types of unlearning methods or scenarios is SSIUU compatible with, and what are its potential limitations or failure modes? The paper currently has no limitations section, and adding one to discuss edge cases (where SSIUU might not perform well, or might not apply) would greatly strengthen the work\n- Q4. In the hypothesis at the beginning of Section 4, the authors use several terms that were not defined earlier (e.g., “new neurons”, “spurious unlearning”, “suppressing knowledge”, “erasing knowledge”). Could the authors clearly define these concepts (either in Section 4 or earlier) to provide the proper context for interpreting the hypothesis?\n- Q5. The paragraph “Quantifying Knowledge” in Section 4 needs more detail/clarity: (i) What exactly is meant by a “neuron” in the context of a transformer language model (e.g., is it a single hidden unit in a specific layer)? (ii) What do the authors mean by “gradient of the language model” in that paragraph? Please clarify this term and perhaps relate it to the model’s components. (iii) Could the authors contextualize these definitions in terms of the general structure of a decoder-only transformer (like those used in the experiments)? (iv) Additionally, could you clarify the notation in Equation 1 – specifically, why the multiplication sign “\\times” is used and what operation it denotes in that context?\n- Q6. In lines 252–254 of the paper, you mention using a certain number of samples to obtain attribution scores (for computational tractability). How many samples are used, and how well does this sampled approach approximate the true per-token attribution scores? Also, regarding the training objective (Equation 3): since the values $A$ are computed using gradients (attribution scores), how are the final parameter gradients for the optimizer obtained? In other words, are you backpropagating through the computation of $A$, or treating $A$ as fixed for the purpose of gradient computation? This detail is unclear and important for understanding the optimization process.", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- S1. The paper investigates a well-motivated problem supported by both literature and experiments.\n- S2. The authors tackle the problem with a targeted method supported by initial observations.\n- S3. The method proposed is effective despite its apparent simplicity.", "weaknesses": "- W1. The proposed unlearning approach hinges on the correctness of a specific attribution method. This reliance is a potential weakness: if the attribution method does not accurately or consistently identify knowledge-bearing vs. spurious neurons, the effectiveness of SSIUU could be compromised. The paper’s arguments assume the attribution is reliable across conditions and tasks, but this is not extensively validated. It would strengthen the paper to either justify the choice of attribution method (e.g., why it’s expected to generalize) or to evaluate the unlearning approach with an alternative attribution technique to ensure the results are not an artifact of one particular method.\n- W2. Some aspects of the method’s implementation are not clearly described in the paper (for instance, how final gradients per step are computed considering the new training objective introduced in equation 3). Without details or released code, it’s difficult to assess the feasibility and reproducibility of the approach. Machine unlearning on LLMs is a highly practical problem, so the paper would benefit from more discussion on practical considerations like computational cost, how the method scales with model size, and how it fits into an unlearning pipeline. I recommend the authors include more details on the implementation (perhaps pseudo-code or algorithmic complexity analysis) and discuss the feasibility: e.g., how much additional training overhead does SSIUU introduce? Such details would help readers gauge the practicality of deploying this method.\n- W3. The experimental evaluation, while promising, could be more comprehensive. The paper would be stronger if it included results on additional models or tasks (adding statistical significance to the results), and particularly if it reported variability across multiple runs. Currently, it’s hard to tell if the gains are consistent or might vary under different conditions.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "1N3ujtj7r1", "reviewer": "Reviewer_2ps9", "summary": "This paper investigates the limitations of current machine unlearning methods and introduces the concept of spurious unlearning neurons, which emerge during unlearning and suppress target knowledge through negative attribution rather than true erasure. The authors propose SSIUU, a regularization-based approach designed to mitigate the inflation of such negative influences and achieve more robust unlearning. Extensive experiments demonstrate that SSIUU not only improves robustness against retraining attacks but also enhances forgetting effectiveness without sacrificing utility.", "questions": "1.  In Figure 3(c), the proposed method (_Ours_) shows an increase in D^{+}, representing the positive influence on the target knowledge. Why does this happen? Shouldn’t effective unlearning reduce the positive contribution instead of increasing it?\n\n2. Why does gradient ascent in unlearning often lead to the emergence of spurious unlearning behavior, where the model suppresses rather than erases knowledge? Conversely, would gradient descent in standard learning produce an analogous phenomenon of spurious learning? Is it possible to theoretically prove these effects?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1.  The paper identifies and analyzes the phenomenon of spurious unlearning, which is novel and provides valuable insights into why existing unlearning methods often fail to achieve true knowledge removal.\n\n2.  The proposed SSIUU method demonstrates strong empirical performance, showing superior robustness and forgetting effectiveness across multiple datasets and models while maintaining utility on unrelated tasks.", "weaknesses": "1.  Why is the phenomenon of spurious unlearning considered to be stored at the neuron level, given that neurons are typically viewed as units encoding knowledge rather than behavioral control mechanisms?\n\n2.  The current work demonstrates only the existence of spurious unlearning neurons but not their causal role. What would happen to the model’s behavior if these neurons were directly removed after unlearning?\n\n3.  Is suppressing spurious unlearning neurons during training essentially a “whack-a-mole” process, where eliminating such neurons might lead to the emergence of new spurious unlearning patterns, for example at the attention-head level?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "LkkWTzZeue", "reviewer": "Reviewer_A8GP", "summary": "This paper reveals that standard unlearning techniques achieve only shallow alignment by relying on negative influence suppression and tries to improve the robustness of machine unlearning in LLMs. The authors introduce novel retraining attack benchmarks (harmful and/or benign) and develop a regularization method called SSIUU that actively penalizes negative attribution growth to achieve genuine knowledge erasure. Experiments on 3B models with two datasets demonstrate that SSIUU improves robustness against retraining attacks.", "questions": "1) Does SSIUU impact convergence speed?\n2) Can the authors provide a recommended strategy or dynamic schedule for setting λ that minimizes the degradation of retention accuracy while maintaining high robustness?\n3) Can the authors provide an analysis on SSIUU's performance on other privacy tasks?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "1) The paper addresses a critical and practical flaw in existing unlearning methods. The motivation is articulated clearly and persuasively.\n2) The contributions are clearly listed, and the related work section is concise and positions the paper well. \n3) The paper doesn't just identify a problem; it provides both a way to measure it (the new benchmarks) and a way to fix it (SSIUU).", "weaknesses": "1) The proposed method requires a non-trivial attribution-like score calculation at every unlearning step. The paper provides zero quantitative analysis of the computational cost of SSIUU versus the GD baseline. This omission is fatal for evaluating the method's claimed efficiency and practical utility.\n2) The main text lacks a comprehensive discussion on the trade-off between λ and retention performance. Figure 7-(a) suggests that aggressive regularization (λ≥0.01) begins to degrade accuracy on the retained knowledge set, implying SSIUU is a delicate balancing act that requires extensive hyperparameter tuning. This practical difficulty is insufficiently addressed.\n3) The robustness is only evaluated against retraining attacks. It is an open question whether SSIUU is robust against other privacy attacks, such as membership inference attacks on the forgotten set, which might exploit the newly established, strong D+ based erasure mechanism.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "VxrldPdd6G", "reviewer": "Reviewer_VL5g", "summary": "This paper addresses a critical and timely problem in machine unlearning for LLMs: the lack of robustness. The authors compellingly argue that existing unlearning methods often achieve \"shallow unlearning alignment,\" a phenomenon where the model learns to \"hide\" target knowledge by generating \"spurious unlearning neurons\" that amplify negative influence, rather than \"erasing\" the neurons that encode the knowledge. To substantiate this claim, the paper introduces two practical \"retraining attack\" scenarios (harmful and benign) that demonstrate how this hidden knowledge can be easily recovered. The authors use an attribution-based analysis to quantify the difference between positive and negative influence variations, providing strong evidence for their hypothesis. Based on this diagnosis, the paper proposes SSIUU (Suppressing Spurious Unlearning Neurons for Robust Unlearning), a novel regularization method. SSIUU penalizes the growth of negative influence during unlearning, forcing the optimizer to achieve a more \"faithful\" unlearning by reducing the original positive influence. Experiments conducted on Llama-3.2 and Qwen-2.5 models using the FaithUn and TOFU datasets show that SSIUU significantly outperforms strong baselines in terms of robustness against the proposed retraining attacks, while maintaining comparable performance on standard unlearning metrics.", "questions": "Please refer to the weaknesses.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. This paper provides the \"shallow unlearning alignment\" concept, backed by the \"hide\" (increasing D-) vs. \"erase\" (decreasing D+) analysis, which is a clear, insightful, and novel explanation for why unlearned models are fragile. The proposed method, SSIUU, is not an ad-hoc fix. It is a principled solution that directly targets the problem (spurious negative neurons) identified in the diagnosis. The method shows SOTA performance on the new robustness metrics (Tables 1 & 2), demonstrating a clear practical benefit over all baselines, including strong ones like NPO and KLUE.\n2. The introduction of \"harmful\" and \"benign\" retraining attacks is a major strength. These are practical, well-motivated, and set a new, higher standard for evaluating unlearning methods.\n3. The use of logit lens (Fig 4) to show SSIUU avoids \"over-unlearning\" and the attribution stability analysis (Fig 6) to show robustness at the representational level are both very convincing.", "weaknesses": "1. The entire paper's diagnosis and method rely heavily on the attribution method from Yang et al. (2023). The claims would be much stronger if the authors showed that the \"shallow alignment\" phenomenon (large D- increase) is consistent across different attribution methods.\n2. The method requires calculating an attribution-based regularizer at each optimization step. This seems computationally intensive. The paper mentions a \"simplification\" for efficiency but provides no details on what this simplification is, nor does it provide any analysis of the actual computational cost (e.g., training time, memory usage) compared to the baselines. This is a critical practical limitation for a method intended for large models.\n3. The simplification mentioned in Section 5 (\"multiplying each parameter with its gradient\") is vague. This sounds very different from the formal attribution definition in Equation 1. This ambiguity makes the method difficult to reproduce and assess.\n4. While Table 1 includes many baselines, the detailed attribution analyses in Figures 3, 5, and 6 primarily focus on comparing SSIUU to GD. It would be more convincing to see this deep analysis applied to other strong baselines, especially NPO, which was the most robust baseline in Table 1.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)