SAEs Can Improve Unlearning: Dynamic Sparse Autoencoder Guardrails for Precision Unlearning in LLMs
Authors: Aashiq Muhamed
Jacopo Bonato
Mona T. Diab
Virginia Smith
Venue: COLM 2025 (2025)
Keywords: unlearning
sparse autoencoders
representation editing
steering vectors
relearning
TL;DR: We introduce Dynamic SAE Guardrails, a computationally efficient method that leverages sparse autoencoders for precision unlearning in LLMs, outperforming existing approaches while maintaining model utility.

Abstract: Machine unlearning is a promising approach to improve LLM safety by removing unwanted knowledge from a trained model. However, prevailing gradient-based unlearning methods suffer from issues such as high computational costs, hyperparameter instability, poor sequential unlearning capability, vulnerability to relearning attacks, low data efficiency, and lack of interpretability. While Sparse Autoencoders are well-suited to improve these aspects by enabling targeted activation-based unlearning, prior approaches underperform gradient-based methods. This work demonstrates that, contrary to these earlier findings, SAEs can significantly improve unlearning when employed dynamically. We introduce Dynamic SAE Guardrails (DSG), a novel method for precision unlearning that leverages principled feature selection and a dynamic classifier. Our experiments show DSG substantially outperforms leading unlearning methods, achieving superior forget-utility trade-offs. DSG addresses key drawbacks of gradient-based approaches for unlearning---offering enhanced computational efficiency and stability, robust performance in sequential unlearning, stronger resistance to relearning attacks, better data efficiency including zero-shot settings, and more interpretable unlearning.

Published as a conference paper at COLM 2025
Es Can Improve Unlearning: Dynamic Sparse Autoencoder Guardrails for Precision Unlearning in LLMs
Aashiq Muhamed', Jacopo Bonato*" Mona Diab', Virginia Smith mddaaa @@andrew.cmu.cc http:/ppp.bboottppottttee.c.ccce CCannegie Mellon University, +Prometeia S.p.A.
Abstract
Machine unlearning is a promising approach to improve LLM safety by removing unwanted knowledge from a trained model. However, prevailing gradient-based unlearning methods suffer from issues such as high computational costs, hyperparameter instability, poor sequential un- learning capability, vulnerability to relearning attacks, data efficiency, and lack of interpretability. While Sparse Autoencoders well-suited to improve these aspects by enabling targeted activation- unlearning, prior approaches underperform gradient-based methods. This work demonstrates that, contrary to these earlier findings, SAEs can significantly improve unlearning when employed dynamically. We introduce Dynamic SAE Guardrails (DSG), a novel method for precision unlearning that leverages principled feature selection and a dynamic classifier. Our experiments show DSG substantially outperforms leading unlearning methods, achieving superior forget-utility trade-offs. DSG addresses key drawbacks of gradient-based approaches for unffarning-offfrinng enhanced computational efficiency and stability, robust performance in sequential unlearning, stronger resistance to relearning attacks, better data efficiency including zero-shot settings, and more interpretable unlearning."
1 Introduction
Machine unlearning, the process of removing specific information from trained LLMs, is a promising tool for applications in safety, privacy, and model maintenance (Liu et al., 2025). However, predominant gradient-based unlearning methods suffer from significant limitations (Barez al., 2025). Existing methods struggle to precisely balance forgetting target data with preserving general utility (Thaker et al., 2025), incur high computational costs (Cha et al., 2024), exhibit hyperparameter instability (Bu et al., 2024), degrade quickly under sequential unlearning requests (Shi et al., 2024; Gao et al., 2025), are vulnerable to relearning attacks (Hu et al., 2025; Deeb & Roger, 2024; Lucki et al., 2024), lack data effi- ciency (Gao et al., 2024), and offer little interpretability (Xu et al., 2024). While interventions using Sparse Autoencoders (SAEs) (Bricken et al., 2023) offer a potential path towards more targeted, activation-based unlearning, existing SAE approaches have typically underper- formed gradient-based approaches due to imprecise interventions that cause unintended side effects (Farrell et al., 2024).
This paper demonstrates that, contrary to previous findings, SAEs can significantly improve unlearning when employed dynamically, We introduce Dynamic SAE Guardrails (DSG), a novel method that leverages SAES for precise, efficient, and interpretable unlearning in LLMs. DSG integrates Fisher Information-based feature selection to identify features causally linked to the forget data, with a dynamic, input-dependent classifier that triggers targeted feature clamping only when necessary. This conditional intervention acts as guardrail, preventing the model from accessing specific knowledge pathways for relevant
Work performed at Leonardo Labs, now at Prometeia S.p.A.
Coode is available https: //gi thub. com/ aashi wtwpuhaamed/yynaiiiieeeee
1

---

Published as a conference paper at COLM 2025
inputs while leaving general capabilities intact. Through extensive experiments standard benchmarks, we show that DSG not only achieves a superior balance between forgetting and utility preservation compared to leading gradient-based and static SAE methods, but also directly addresses their core limitations. Our main contributions are:
1. We introduce DSG, new activation-based unlearning method featuring principled SAE feature selection and a dynamic classifier for precise, conditional intervention.
2. We demonstrate empirically that DSG achieves superior balance between forget- ting and utility preservation compared to leading gradient-based and SAE-based unlearning approaches on multiple benchmarks.
3. We show that DSG provides substantial benefits over gradient-based unlearning such as greater hyperparameter stability, improved computational efficiency, and sequential unlearning capability, enhanced resistance against relearning attacks, enhanced data efficiency even in the zero-shot setting and interpretable unlearning.
2 Background and Related Work
Unlearning in Large Language Models. Machine unlearning aims to modify a trained target model MDD) to behave as if specific data, the forget set Forgets had never been part of its training data D (Bourtoule al., 2021; Cao & Yang, 2015). The resulting model, Munlearn, should ideally be indistinguishable from model trained only on the retain set Dretain D D forget- As retraining LLMSs from scratch is computationally prohibitive, research focuses on approximate unlearning (Liu al., 2025). These methods face the core challenge of balancing knowledge removal (forget quality) and maintaining general capabilities (utility preseroation) (Shi et al., 2024; Maini et al., 2024).
The dominant approach for approximate unlearning involves gradient-based optimization (Liu et al., 2024). Methods like Gradient Ascent (GA) (Jang et al., 2023), Gradient Difference (GfadDiff) (Liu et al., 2022), Negative Preference Optimization (NPO) (Zhang et al., 2024), and RMU (Li et al., 2024) finetune model weights to reduce the influence of D forget, often us- ing regularization (e.g., KL divergence) to protect utility (Maini et al., 2024; Yao et al., 2024). However, these gradient-based techniques frequently suffer from significant limitations: high computational cost (requiring backward passes), instability under hyperparameter tun- ing, degraded performance under sequential unlearning requests (Gao et al., 2024), vulnera- bility to relearning attacks (Hu et al., 2025), poor data efficiency, and a lack of interpretabil- ity (Barez et al., 2025). These widespread challenges motivate the exploration of alternative unlearning paradigms, such as the intivattionbbase interventions explored in this work.
Sparse Autoencoders (SAEs). Modern DNNs operate in regime of superposition, where multiple features or capabilities are encoded along the same dimensions of hidden activa- tions (Elhage et al., 2022). SAES provide an unsupervised method for disentangling these superposed representations into interpretable features (Bricken et al., 2023; Cunningham et al., 2023). Given activations h E model from a specific layer or component of an LLM, an SAE decomposes and reconstructs these activations using encoder and decoder functions: (h) == o (WWench + benc) and f(f)= == W bdec. In other words, SAEs express model activations as sparse linear combination of interpretable feature vectors: h h Ehh)
_AAA f((hv;; + b + +(h) where fi(h) E RR are scalar feature activations, 0; E R are unit vector feature directions, b modell is bias term, and e e(h) E Rddddee is the SAE error term. Wider SAEs continue to improve feature granularity and reduce the error term.
SAES are trained on activations collected from the model processing pretraining data where training is conducted separately for each layer or component of interest (e.g., residual stream, attention outputs) using an objective that minimizes reconstruction loss while en- forcing sparsity: L= ||h - (((f()) ) I12 + (l) Here À is a sparsity penalty coefficient encouraging most feature activations to be zero for any given input. In this work, use JumpReLU SAEs (Rajamanoharan et al., 2024), which enforce sparsity using a shifted Heav- iside step function. The interpretability of SAE features stems from their sparse activation
2

---

Published as a conference paper at COLM 2025
Settern-beccuse features only active for small fraction of inputs, they must capture meaningful patterns to be useful for reconstruction. The cost of training SAEs is amor- tized actoss multiple downstream applications such identifying and removing spurious correlations in models (Marks et al., 2024) and steering behavior (O'Brien et al., 2024).
Farrell et al. (2024) developed an early approach using SAEs for unlearning by identifying features that were frequently active on D forget and applying a stafic intervention indevention-clamping these identified features when active at a token irrespective of overall context. However, this produced substantial side-effects on general model utility and ultimately underperformed gradient-based methods like RMU. In this work, we show that SAE unlearning can be effective via a context-depent deeeett-ee intervention strategy rather than simple static clamping.
Further background details are provided in Appendix A.
Dynamic SAE Guardrails (DSG)
<(A)
SAE Decoder (W (Wane; Dano)
DSG (Figure 1, Algorithm 1) is a targeted un- learning method for LLMs that leverages the in- terpretability of SAEs and combines: (1) causal framing that motivates feature selection, (2)
Transformer
Transformer layer
layer
SAE Encoder (W
theoretically justified feature importance scoring based on Fisher Information (FI), (3) a dynamic, input-dependent classification rule based on statistically ll optimal threshold, and (4) targeted clamping intervention to remove the influence of selected features.
activations
SAE activations fith)
clamped activations f:(M)
Figure 1: An illustration of DSG
3.1 Causal Framework and Problem Setup
We frame unlearning through causal perspective where Diorget and Dretain influence model representations E and outputs Y through multiple pathways (Shen et al., 2024). These include eppresenaation-mediaeee pathways (D - E + Y), potential direct influence (D Y), and intertwined knowledge (Diorget D rettain) where conceptual overlap exists between the datasets. SAE features fj derived from E serve as interpretable mediators (Pearl, 2009) of information flow through these causal pathways. From this perspective, unlearning involves blocking pathways from Diorget to while preserving the pathways from Dretain to Y. DSG implements this causal intervention do(f; = -c) on forget features identified by analyzing SAE activation patterns across both datasets.
3.2 Feature Selection: Identifying Causal Mediators via Fisher Information
To identify which SAE features, Frorget mediate the causal influence of Poorget/ we establish two key theoretical connections: first between Fl and feature activation, and second between Fl and causal influence. We then describe our percentile based feature selection.
Theorem 1 (Fisher Information Approximation), For SAE with small reconstruction error and imput h, the expected squared gradient of reconstruction loss with respect to fea- ture j's decoder weights 9,, is proportional to the second moment of that feature's activation: Eb V, [(h)]]] (h))) where whhpp, reconstruction errors are bounded by e².
The proof of Theorem provided in Appendix This establishes that squared feature activations proportional to the Fisher Information of the corresponding decoder weights.
Theorem 2 (Fisher Information Proxy for Causal Feature Importance). Linder standard assumptions, Fisher Information associated with SAE features provides an approximation of their causal influence as mediators between specific training data and model outputs. For any SAE feature fú, the expected squared activation ED]f;(h) }] on dataset D is proportional to the causal influence of that feature mediator of information from D to model outputs.
3

---

Published as a conference paper at COLM 2025
Proof of Theorem 2 is in Appendix C. Under these assumptions, feature with large expected squared activation on Piorget contributes significantly to the model's FI with respect to that data. This suggests that intervening on that feature (e.g,, clamping its activation) would sub- stantially affect the model's output distribution when processing inputs similar to those in Diorget. Squared activation serves as a computationally tractable proxy for causal influence.
DSG obtains token-level SAE activations from each sequence in both D Diorget and D Petaiin squares them, and aggregates the results into matrices forget E IR xdsAF and Areain R RdsxE (ny and HR are the total numbers tokens in the respective datasets). For each token in sequence x, have the activation fi(hx,) of feature on the hidden state hy Each entry of the activation matrices is thus Aorget [. z [fj(h,,)] for a token t in sequence x Pooreet (and similarly for Aretai). From these, we compute the aver- age squared activation per feature as forget_scorel
Importance Scores.
1/11 ExcDioget _ (f and (itain_scoree = /R ExcD perain 21=1 (f [xxt)] and define the relative importance by forget.scorel)
the
imp.ratio(j) max(retain_score([] with e > 0 to avoid division by zero. By Theorem this ratio represents the relative causal influence of feature
Algorithm Dynamic SAE Guardrails (DSG)
Require: LLM with SAE features (f}): datasets Diorgot/ D Pretaini clamp strength percentiles
(Pratior Payn); ); feature count fetats
Feature Selection:
Compute feature importance scores and threshold Tratio from percentiles Identify Florget = [j: imp ratio(j) 2 Tratio
Sort Florget by descending forget_scorefe and select top featss features to form Snjas
Dynamic Threshold Calibration:
Compute p(x) Σ. [3j Smeas (f(hh) > 0] for each Petaai
Set threshold r = (ercentill((((x)))>r))
Pdyn)
Inference- Intervention:
For input sequence x, compute p(x) and classify as forget-relevant p(x) > T If forget-relevant: For each token and feature j C Speww set (h)) =
Otherwise: Preserve all feature activations
Percentile-Based Feature Selection. To select features most causally relevant to Diorgetr we employ percentile-based approach using Pratio to compute Trtiio as Percentile({ (imml_(((_m_p_()e (mp_ratio(f) i=1 /rratio) Percentile(S,p) p) returns the value such that p% of elements in set S are less than or equal to v. For example, with Pratio = 95, Tratio is set so 95% of features have imp ratio(j) S Tratio We define the set of forget-mediating features as Florget = (j: imp mmpioo])) 2 ] To filter out noisy features, we sort features in Frorget by descending forget.score(( and select the top feaatt to form the final intervention set Smietts
dSAF
3.3 Dynamic Sequence-Level Classification and Intervention
DSG employs a dynamic input-dependent classification mechanism to minimize unintended side-effects on content unrelated to the forget knowledge.
Definition (Forget-Set Activated Token). A loken x, considered forget-set activated af least one feature Sngats has a positive activatiion: fj(h,) 0.
For an input sequence x= ==(x XT) of length T, we define the statistic p(x) = + 1 S Staass : (h;)) 0J, representing the percentage of forget-set acti- vated tokens. A high p(x) indicates that query x strongly relies on features we've identified as causally linked to the forget knowledge.
Threshold Selection and Classification. We select threshold t E [0,1] based on the distribution of p(x) D using controls the trade-off between unlearning effectiveness and performance preservation.
(ercentile(((p())xeee) Payn) which
4

---

Published as a conference paper at COLM 2025
Empirically, we find p(x) is stochastically larger on Poorget than on Pretain as seen in Figure 2, which shows the distribution for both forget-domain queries (WMDP-Bio) and general knowledge queries (MMLU). chosen to control the retain set's false-positive rate and separates forget-set queries effectively, achieving high recall Porget- On this example, DSG success-
100
MMLU
WMDF-Bio
80
Retain set (WikiText)
Counts
60
Forget. set (WMD(-Bio)
40
20
on D fully transfers from retain set (WikiText) and forget set to the test query set. We define classifier C(x) = 1[p(x) > T) labeling inputs as either forget-relevant or retain-releount. The statistical optimality of this thresholding approach follows from Neyman-Pearson lemma:
0.0
0.2
0.4
0.6 0.8
1.0
% of forget-set activated tokens in sequence
Figure 2: Distribution of p(x) for unlearning on WMDP-Bio. Threshold at 95th percentile (dashed red line) separates MMLU from WMDP.
Theorem 3 (Neyman-Pearson Optimality). If p(X) is stochastically larger under Dorget than under D retain, then among all classifiers with fulse-positive rate at most a, the thrrehood test C*(x) = [x) > r*), where Prx--Dmai [p(X) > r*] = a, maximizes the true-positive rate. The proof appears Appendix D and states that under the stochastic dominance assump- tion, thresholding p(x) is the optimal classification approach for given false-positive rate.
Conditional Clamping. Our intervention is conditional on the classifier C(x). When C(x) = (forget-relevant), for each token X; and feature Shepp set (h:) -c, where - is a large negative constant we call clamp strength. This implements targeted do(f)(h) (h,)
c) operation, selectively severing the causal pathway only when the impul query is deemed forget-relevant. When C(x) = 0 (retain-relevant) we leave all features unchanged: (h, ff (h;). This preserves the model's original behavior for queries unrelated the tårgeted knowledge, minimizing side-effects and maintaining performance on Dretain-
The dynamic clamping in DSG contrasts with static clamping methods (Farrell al., 2024), which intervene based only on feature activation without sequence-level classification, and risk inadequate coverage on Dirrgtt or excessive side-effects on Dretain DSG avoids this suboptimal trade-off- formally prove (Theorem 4, Appendix F)) that for any static approach, DSG achieves equal or greater coverage on forget with equivalent side-effects on D Dretain providing superior unlearning-utility trade-off.
4 Experiments and Results
4.1 Unlearning on WMDP
We evaluate DSG on the WMDP dataset (Li et al., 2024), which benchmarks hazardous knowledge unlearning across multiple domains. We focus WMDP-Bio (biosecurity) and WMDP-Cyber (cybersecurity). For each domain, our unlearning setup uses domain-specific D PubMeed papers containing bio-weapon related content for WMDP-Bio and Git repositories for WTMDP-Cyber-ait
WikiText (Merity al., 2016) Dretain- We evaluate unlearning effectiveness using the WMDP multiple-choice question test sets, which were not exposed to models during the unlearning process.
Following SAEBench (Karvonen et al., 2025), we evaluate unlearning only on questions the target model correctly answers across all 24 permutations of the 4 multiple-choice options. This yields 522/127 questions for WMDP-Bir and 275/ 1987 questions for WMDP-Cyber. For evaluating model utility, we similarly filter MMLU questions that the model answers correctly across all permutations. This yields 305 questions from history, computer science, geography, and human aging for WMDP-Bio. For WMDP-Cyber, we use 371 MMLU questions, replacing computer science with biology. Table reports the configuration that minimizes WMDP accuracy while maintaining at least 99% of the target model MMLU accuracy along with with MT-Bench scores that measure general fluency.

---

Published as a conference paper at COLM 2025
1.00
1.00
Selected MLU Accuracy
Selected MLD Accuracy
0.99
0.98
304
DBG features=10
0.97
DSC features-20
DSG features110
0.97
DSC featuma-=00
features=20
RMU
0.96
Parrell et
SCRUB
0.96
SCRLB
Farrell al.
99%
threshold
0.2
0.3
0.4
0.5
0.7
0.9
1.0
0.2
0.3
0.5
0.6
0.7
0.8
WMDP--Bio Accuracy
WMDP Cyber Acsuracy
Figure 3: Unlearning performance on WMDP-Bio (left) and WMDP-Cyber (right). Higher MMLU accuracy and lower WMDP accuracy is better. Clamp strengths (c) used for DSG points are shown as annotations. DSG Pareto-dominater the top four baseline methods (RMU, SCRUB, Farrell et al., SSD).
MMLU (t)
Method
WMDP Bio (4)
MT (t)
HS Hist
C. CS
HS Geo
H. Aging
All
Target M
100.00
100.00
100.00
100.00
100.00
100.00
7.36
GA
99.44
98.18
100.00
100.00
100.00
99.35
7.44
NPO
97.95
99.99
88.88
100.00
98.82
99.35
7.29
SSD
99.44
100.00
100.00
100.00
98.82
99.68
7.24
SCRUB
94.97
99.09
100.00
100.00
98.82
99.35
6.09
Farrell et al.
59.22
100.00
100.00
100.00
96.47
99.03
7.33
RMU
50.00
99.08
100.00
100.00
98.81
99.47
7.21
DSG (Ours)
29.64
100.00
100.00
100.00
97.62
99.34
7.78
Table Unlearning performance on WMDP-Bio. represents the average MMLU score. MT-Bench scores show 0.16 variance across 5 rums. DSG shows superior unlearning effectiveness compared to baselines while maintaining high MMLU performance.
Experimental Setup. We implement DSG using gemma-2-2b-it model with gemma-scopps Scoma--scope-2--t-rrs SAE (width 16k) (Lieberum et al., 2024) applied to layer at to 142. We use Paym 95 for both domains, and Protio 95 for WMDP-Bio and Protio = 90 for WMDP-Cyber. We compare DSG against several baselines across a broad hyperparameter sweep: GA (Jang et al., 2023), NPO (Zhang et al., 2024), SSD (Foster et al., 2024), SCRUB (Kurmanji et al., Farrell et al. (2024) and RMU (Li al., 2024). Complete hyperparameter details are provided in Appendix G.
Results. As shown in Table 1, DSG significantly outperforms all baselines the WMDP- Bio unlearning task, reducing accuracy to 29.64% compared to the next best method RMU at 50.00%. It maintains high MMLU performance (99.34% average) and achieves the highest MT-Bench score (7.78), showing superior preservation of general model capabilities. The results on WMDP-Cyber (Appendix G.2) reinforce these findings. Figure 3 provides more comprehensive view of the unlearning-utille with MMLU accuracy above 95%. DSG Pareto-dominates all baseline methods: for any level of utility preservation (MMLU accuracy), DSG achieves more effective unlearning.
trade-off landscape, plotting all configurations
This superior performance coupled with significant practical advantages over gradient- based methods in terms of computational efficiency and hyperparameter stability. wradient-based approaches often exhibit hyperparameter instability, where slight tuning changes can drastically alter outcomes, risking poor unlearning or utility collapse. Further- more, they require computationally costly backward passes through the LLM for optimiza- tion. In contrast, DSG shows greater hyperparameter stability (Figure 3) and efficiency. It requires only forward passes: one to gather feature statistics initially, and then lightweight intervention during inference, completely avoiding expensive gradient calculations. This combination of efficiency and stability makes DSG particularly advantageous for large models and frequent unlearning where gradient compulations impose substantial overhead.

---

Published as a conference paper at COLM 2025
4.2 Unlearning on Muse
We evaluate DSG on MUSE (Shi et al., 2024) comprising two corpora: NEWS and BOOKS, and focusing on six dimensions: verbatim memorization, knowledge memorization, privacy leakage, utility preservation, forget set scalability, and sequential unlearning. MUSE provides a challenging evaluation setting where forget and retain sets share substantial domain overlap: in NEWS, both sets are drawn from the same distribution of BBC articles, while in BOOKS, the forget set (Harry Potter books) and retain set (Harry Potter FanWiki) contain highly related content.
Verbatim Mem. VerbMem on Plurget (4)
Knowledge KnowMem on Diurus (+)
Privacy Leak. PrivLeak (E (-5%, 5%))
C4. Utiltiy KnowMem on Pistain (1)
NEWS
Target M
29.51
GA
0.62
97.1%
0.00
100.0%
-8.16
under-unlearm
0.09
99.7%
GradDiff
2.81
86.7%
0.71
97.6%
93.10
over-unlearm
7.76
[ 71.0%
NPO
20.98
0.8%
25.14
14.8%
-53.42
under-unleamm
29.02
8.4%
SimNPO
21.14
0.0%
27.70
6.1%
-89.84
under-unlearm
30.59
14.2%
RMU
9.60
54.6%
26.63
9.8%
75.02
over-unlearn
26.41
1.4%
DSG (Ours)
11.80
44.2%
0.44
98.5%
12.08
over-unleam
25.65
4.2%
BOOKS
Target MM
15.80
33.90
-9.80
35,28
GA
2.61
83.5%
0.17
99.5%
-1.58
accaptable
0.57
98.4%
GradDiff
9.49
-10.30
under-unleamm
NPO
14.41
8.8%
28.21
16.8%
-97.24
under-unleam
37.19
55.%%
SimNPO
14.55
7.9%
34.36
1.4%%
-96.40
under-unleari
36.62
1
RMU
14.89
5.8%
32.59
3.9%
-97.58
under-unleam
37.13
5.2%
DSG (Ours)
8.73
44.7%
1.79
94.7%
-23.18
under-unlearn
37.10
1 5.2%
Table 2:
Unlearning performance on MUSE. We highlight in green if the method satisfies the criterion and red otherwise. For privacy leakage, large positive values suggest over-unlearning. while large negative values suggest under-unlend unlearnnn,,,, DSG shows strong performance across all metrics, achiev- ing substantial reductions in verbatim and knowledge memorization while maintaining high utility.
Experimental Setup. We create separate target models for NEWS and BOOKS by finetun-
ing gemma-2-2b-1n
each corpus for 5 epochs using learning rate 10 and batch size 32. For each target model, we implement DSG using gemma-scoppe -ccpmma--pppp--p--e-opp
SAE (width 16k)
applied to layer 3. For both domains,
clamp strength 500, Pratio 95 and feats 20.
We use Payx = 90 for NEWS and Payn = 95 for BOOKS, with the lower threshold for NEWS enabling more effective verbatim memorization removal. For both scalability and sequential unlearning, we use the best NEWS hyperparameters.
We compare DSG against: GA, GradDiff (Liu et al., 2022), NPO, SimNPO (Fan et al., 2024), and RMU. Following MUSE, we train for 10 epochs using Adam with learning rate 10-5 and batch size 32, selecting the last epoch checkpoint before utility falls below 90% of the target model accuracy. Complete hyperparameters can be found in Appendix H.
Unlearning. Table 2 shows that DSG outperforms existing baselnes. It is effective at verba- tim memorization removal (C1) with 44.2% reduction on NEWS and 44.7% on BOOKS. On knowledge memorization (C2), DSG achieves near-complete removal with 98.5% reduction on NEWS and 94.7% reduction on BOOKS, outperforming most baselines. On privacy leakage (C3), while not within the ideal range, DSG performs better than the majority of baselines. For utility preservation (C4), DSG maintains 95.8% of target model performance on NEWS and achieves 5.2% improvement BOOKS compared to the target model.
Scalability. Figure 4(a) shows DSG is stable and robust when scaling to larger forget sets. We evaluate performance across forget sets ranging from 0.8M to 3.3M tokens, and DSG maintains its position in the ideal region (high retain set knowledge, low forget set knowledge) even as the forget set size increases. In contrast, gradient-based methods exhibit substantial degradation, with increasingly poor tradeoffs between retaining general knowledge and forgetting targeted information.
7

---

Published as a conference paper at COLM 2025
Sequential Unlearning. Fig- ure 4(b) illustrates DSG's ef-
Scalability
Sequential Unlearning (b)
35
35
Ideal Region
Pointx Sequential Steps
fectivemess across sequential
Retain Know. Mem. (%)
30
30
unlearning requests on four
disjoint NEWS folds. We imple-
ment two approaches: DSGall which cumulatively updates feature importance scores based on each new forget data request; and DSGunion, which takes the union of features selected inde- pendently at each step and uses this combined set to calculate p(x) and threshold T on Dr. Both approaches perform similarly well, consistently maintaining DSG in the ideal region where other methods rapidly degrade with each additional unlearning operation. Gradient-based methods suffer from catastrophic forgetting during sequential unlearning, where each update pushes the model further from its original performance distribution. (Details in subsection H.2.)
20
Target
20
Target
-8- GA
15
GradDiff
15
-4-
GradDur
NPO
NPO
10:
SimNPO
10
RMU
RMU
DSG...
DSG
DSG,,
10
15
20
25
30
35
10
15
20
23
30
Forget
(%)
Forget
Figure 4: (a) Scalability: Performance across increasing forget set sizes. (b) Sequential Unlearning: Performance across
sequential unlearning requests
4.3
Resistance to Relearning Attacks
Finetuning on WMDF-Ble
Finetuning
0.700
Rise
201
MMJU Performacce
0.850
Sue
Bate Tegriea
WMDP-Bio Accuracy
JVU
0.500
9.000
0.750
0.500
9. 2.700-
0.400
9.050
0.20
Finetuning Epochs
Finetuning Epochs
Figure 5: Relearning attack resistance across finetuning epochs. (a) DSG demonstrates superior resistance to relearning compared to RMU. (b) Test-time DSG preserves MMLJ utility better than Train-time DSG while still providing significant protection.
We evaluate DSG's resistance to relearning attacks in API-based threat models where adversaries have query access but cannot directly manipulate model weights. This resistance derives from the Superficial Alignment Hypothesis (Zhou et al., 2023), which posits that model's activation geometry stabilizes during pretraining and changes minimally during finetuning. Figure 9a confirms this empirically, showing high cosine similarity between pre-finetuning and post-finetuning activation vectors, and activation magnitudes clustered around 1.0. By operating on these stable activation patterns rather than weights, DSG creates more persistent defense. While obfuscation-based attacks have been proposed against activation interventions (Bailey et al., 2024), they are less effective in API-based black-box settings where attackers lack direct access to gradients and model representations.
Methodology. We evaluate two DSG defenses against relearning: (1) Test-time DSG, which applies intervention only at inference time after model finetuning, and (2) Train-time DSG, which integrates DSG during finetuning with frozen SAE parameters to filter gradients. We test six configurations with google/ 2--emma-222---a it as base model: Base, Base+Test-time DSG, Mase+Train-iin DSG, Base+ Train- -est-tese DSG, RMU (base model with RMU unlearning applied), and RMU+Test-timee DSG. The relearning attack consists of finetuning each configuration on the WMDP-Bio test set for 10 epochs at learning rate le-5.
Results and Analysis. Figure 5(a) demonstrates clear differences in vulnerability to relearn- ing attacks. Weight-based methods show high susceptibility, with RMU rapidly increasing
8

---

Published as a conference paper at COLM 2025
in WMDP-Bio accuracy when finetuned, eventually exceeding the base model's finetuned performance. The base model itself shows an initial performance decrease before increasing, as the high learning rate temporarily undoes instruction tuning before relearning occurs.
Test-time DSG provides substantial protection, with RMU+Test-time DSG maintaining near-random performance (25%) throughout training. However, Base+Test-time DSG shows gradual vulnerability to relearning, with performance slowly increasing over finetuning epochs. This gradual protection erosion reveals a limitation of test-time intervention alone.
Train-time DSG offers a distinct protective mechanism. Models finetuned with DSG active show immediate reduction to random-level performance that persists through approxi- mately six epochs before gradually recovering. This delayed recovery pattern suggests DSG forces the model to develop entirely new processing circuits rather than simply reactivating suppressed knowledge. Figure 9b supports this interpretation, showing significantly higher training loss on WMDP-Bio compared to MMLU when finetuning with DSG active.
Combining both approaches (Train-time DeG+Test-time DSG) extends resistance through epoch 7, demonstrating how these complementary mechanisms can be layered for enhanced protection. However, these approaches involve utility trade-offs. Figure 5(b) shows that while Base Finetuned and Base Fineluned+Test-tiie DSG maintain comparable MMLUU performance, Train-time DSG exhibits moderate utility decline at higher epoch counts.
DSG's superior resistance to relearning attacks stems from its activation- based intervention that leverages the stability of activation geometry during finetuning.
4.4
Data Efficiency and Zero-shot Interpretable Unlearning
Dala Efficiency
O-shot Experiment Bio.
O-shot Experiment Cyber
1.000
70%g
1.0
1.0
1 15.0%
20.05
50.0%
Selected Mi Antunny
0.995
100%
0.0
50.0%
Selected MILE AcouusLYy
0.9
0.990
0.5
0.985
0.980
60.0%
0.7
10.0%
Selected
0..975
0.965
0.920
RMU
0.2
-0.0%,
DSG 0-shot
0.5
5. 5.0%
DSG D-shot
DSG
Farrall et
0.960
0.0
0.4
0.1
0.0
0.0
0.2
0.3
0.4
0.5
C.6
0.7
0.2
0.3
0.4
0.3
0.6
0.7
WMDD-BBo Accuracy
WMDE-BBS Accuracy
WMDP Syber
Figure 6: Data efficiency analysis of DSG. (A) Performance across varying training data sizes compared to RMU. (B) Zero-shot performance on WMDP-Bi, (left) and WMDP-Cybee (right) using 20 features selected via Neuropedia API with different 7 thresholds (shown next to each data point).
We evaluate how DSG performs with limited forget and retain data on WMDP-Bio. Figure 6A shows DSG maintaining consistent performance when trained on 20-80% of the original retain and forget datasets, preserving MMLU accuracy while keeping WMDP accuracy below 40%. Only when dataset size falls below 20% does effectiveness noticeably decline, with WMDP accuracy rising above 40%. In contrast, RMU shows inconsistent results across different dataset sizes, indicating that tradieettbbsed methods may be more unstable to hyperparameter changes when data is limited.
For zero-shot evaluation, we implement DSG without any domain-specific forget or retain data (Figure 6B), instead leveraging the interpretability of SAEs. We use Neuropedia (Lin, 2023) feature explanations to identify the forget set features by querying for concepts Biology and Cybersecurrity, selecting the top 20 relevant features (details in Áppendix M). Both tasks use the available for dynamic threshold calibration, we sweep over static T values, finding optimal settings (I = 60% for WMDP-Bio, T = 20% for WMDP-Cyber). Even with features selected purely based their semantic descriptions and without dptaset-specific tuning beyond t, these zero-shot DSG configurations outperform RMU and Farrell et al. (2024), demonstrating the potential for effective unlearning guided directly by feature interpretability.
SAE (width 16k) at layer 3 (fg 59). Since retain data is un-
9

---

Published as a conference paper at COLM 2025
4.5 Ablations
Pa
Pa
1.0
B
1.00
C
p-85
Selected MMLL Accuracy
Selected MMLU Aocursty
0.95
Selected MMLU Accuracy
Pa.-88
Payc
0.90
Par92
Piyc
0.85
#5oai.
0.80
P.72
85
0.7-
39G #tbot
p-00
clamp
Paye 60,
0.5
DSC
Farrell sl. Winst 30
Farrell al. Mlost
0.53
#iitt 20
Anat 30
0Soot. 30
0.90
0.2
0.4
C.G
0.8
1.0
0.1
0.3
0.4
0.5
0.6
0.2
0.4
0.6
0.8
1.0
WMDP-Bio Accuracy
WMDP-BBo Accuracy
WMDF-Bio Accuracy
Figure 7: DSG Ablation studies (A) Static dynamic clamping comparison with varying clamp strengths [10-500] for 20 and 30 features. (B) Effect of dynamic threshold percentile (Páyn) on perfor- mance (C) Impact of importance ratio threshold (Pratio range 75-95) for 20 and 30 features.
We evaluate each component of DSG by conducting ablation experiments WMDP-Bio.
Figure 7A compares DSG with dynamic classification against DSG with static clamping from Farrell et al. (2024) While static clamping effectively removes forget-set information at large clamp values (c > 100), it simultaneously reduces MMLU ac- curacy because it treats all inputs identically regardless of their forget-relevance In contrast, our dynamic classifier only applies interventions when necessary based the statistical distri- bution of forget-feature activations. This conditional approach maintains higher MMLU accuracy (> 99%) while achieving comparable or better WMDP-Bis reduction.
Dynamic Classification:
Percentile-Basedd Feature Selection: DSG with static clamping leverages Fischer Information for feature selection instead of feature sparsity in Farrell et al. (2024). As shown in Figure 7A, across equivalent clamping strengths, this selection approach achieves 8% lower WMDP-Bio accuracy average while maintaining comparable MMLU performance, indicating more precise identification of forget-relevant features.
Dynamic Threshold Pdyn: Figure 7B shows the effect of Payn on the forget-retain trade-off. Higher percentiles (> 95) preserve more MMLU accuracy but allow more WMDP content to pass through undetected, while lower percentiles (< 90) apply intervention more aggressively but with increased side effects on general knowledge. The optimal range 90-95 balances these considerations, removing targeted knowledge while minimizing side effects.
Importance Ratio Threshold Pratio: As shown in Figure 7C, varying Pratio from 75-95 provides fine-grained control over feature selection. Higher values (95) select features with stronger forget-retain differentiation, yielding more targeted intervention, while lower values expand the feature set but may increase overlap with general knowledge features. Additionally we observed that the dynamic classifier can compensate for lower Pratio maintaining effective forget-set filtering even when feature selection is less discriminative.
Additional ablations in Appendix K show that DSG is remarkably robust to clamp strength variations, and performs optimally with moderate feature counts. These findings highlight DSG's practical hyperparameter stability. Effective performance is maintained within reliable ranges for thresholds Pdyn/ Pratio (90-95), feature counts (10-20), alongside notable robustness to clamp strength (100-500). Additionally, these hyperparameters transfer across datasets, simplifying deployment compared gradient-based methods.
Conclusion and Future Work
In this work, we introduced DSG, demonstrating that SAEs with dynamic classification enable precise, activation-based unlearning that substantially outperforms gradient-based methods across multiple benchmarks. While our evaluation focuses on Gemma-2-2B where high-quality SAEs are available, extending DSG to larger models and diverse architec- tural families remains an important avenue for future work, as does investigating how performance scales with SAE width and training configurations.
10

---

Published as a conference paper at COLM 2025
Acknowledgements
This work was supported in part by the National Science Foundation grants IIS21456700 and CCF2107024, and funding from Amazon, Apple, Google, Intel, Meta, the CyLab Security and Privacy Institute, and Leonardo Labs. Any opinions, findings and conclusions, or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of any of these funding agencies.
Ethics Statement
This work introduces Dynamic SAE Guardrails (DSG), a method for targeted unlearning in large language models (LLMs). While designed to promote responsible Al by enabling the removal of unwanted knowledge, several ethical considerations arise:
While our focus is on removing hazardous or unwanted knowl- edge, the same technology could potentially be used to censor information or suppress viewpoints, leading to undesirable social consequences if deployed without careful oversight. The zero-shot capabilities, while advantageous for data-scarce scenarios, could be misused if the user-provided keywords are biased used to target specific groups/content unfairly.
Potential for misuse:
Over-reliance on interpretability: Although SAEs offer improved interpretability compared to black-box models, feature interpretations are not always definitive fully reliable. Misinterpreting feature roles or over-relying on imperfect interpretations could lead to unintended consequences, including the removal of valuable knowledge the failure to remove harmful content. The quality of feature interpretation depends on the quality and representativeness of the data used train and interpret the SAE.
. Limitations of unlearning: As with all approximate unlearning methods, DSG does not guarantee complete removal of targeted knowledge. As we show, it reduces the likelihood of the model generating outputs related to the forget set, but subtle traces indirect influences might persist. It is essential to acknowledge these limitations and avoid presenting DSG as perfect solution for knowledge removal.
Dual-use concerns: The techniques developed in this work for improving model control and safety could also be adapted by malicious actors to develop more sophisticated attacks or to create models that resist safety interventions. We recognize this inherent dual-use nature and emphasize the need for responsible development and sharing of research findings.
Computational Cost of SAE Training: The training of SAEs can be computationally demanding, raising environmental concerns. However there are several open-source SAEs, amortizing their cost, and the the inference-ting efficiency of DSG offers some mitigation compared to gradient-basee unlearning approaches.
We believe the benefits of precise, controllable unlearning for enhancing Al safety outweigh these risks, provided the technology is developed and deployed responsibly. We encourage future work to address these limitations and explore more robust evaluation methods for unlearning.
Reproducibility Statement
To facilitate reproducibility, we have provided detailed descriptions of our experimental setups, including all relevant datasets, models, and hyperparameters. Specifics for each set of experiments can be found as follows:
Complete hyperparameter settings for DSG and all baseline methods (GA, NPO, SSD, SCRUB, Farrell al., RMU) are detailed in Appendix G.1. Model and SAE details are provided in Section 4.1.
WMDP Unlearning:
Hyperparameters for DSG and baseline methods (GA, GradDiff, NPO, SimNPO, RMU) are in Appendix H.1, with model details in Section 3.
MUSE Unlearning:
11

---

Published as a conference paper at COLM 2025
Relearning Attacks (Section 4.3): Model and hyperparameter configurations for the releaming experiments, inchuding both train-time and test-time DSG interventions, are given in Appendix 15,, along with the details in the main text.
. Data Efficiency and Zero-shot Experiments: Model, SAE, and hyperparameter details for the data efficiency analysis and zero-shot evaluations are given in Appendix J.
Ablations: All details related to the ablation studies, and chosen hyperparameters are in Appendix K.
We have described the feature selection process, dynamic classification rule, and intervention mechanism in sufficient detail to allow for reimplementation (Algorithm and Section 3). We have also released the code and relevant scripts necessary to reproduce our results.
References
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Tige Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint urXiv:2333.08774, 2023.
Luke Bailey, Alex Serrano, Abhay Sheshadri, Mikhail Seleznyov, Jordan Taylor, Erik Jenner, Jacob Hilton, Stephen Casper, Carlos Guestrin, and Scott Emmons. Obfuscated activations bypass Ilm latent-space defenses. ar preprint ar axXiv:2412.09565, 2024.
Fazl Barez, Tingchen Fu, Ameya Prabhu, Stephen Casper, Amartya Sanyal, Adel Bibi, Aidan O'Gara, Robert Kirk, Ben Bucknall, Tim Fist, et al. Open problems in machine unlearning for ai safety, 2025.
Lucas Bourtoule, Varun Chandrasekaran, Christopher
Sroquette-Choo Hengrui Jia, Adelin Travers, Baiwu Zhang, David Lie, and Nicolas Papernot. Machine unlearning, In 2021 IEEE Symposium on Security and Privacy (SP), pp. 141-159. IEEE, 2021.
Trenton Bricken, Adly Templeton, Joshua Batson, Brian Chen, Adam Jermyn, Tom Con- erly, Nick Turner, Cem Anil, Carson Denison, Amanda Askell, Robert Lasenby, Yifan Wu, Shauna Kravec, Nicholas Schiefer, Tim Maxwell, Nicholas Joseph, Zac Hatfield- Dodds, Alex Tamkin, Karina Nguyen, Brayden McLean, Josiah E Burke, Tristan Hume, Shan Carter, Tom Henighan, and Christopher Olah. Towards monosemanticity. De- composing language models with dictionary learning. Transformer Circuits Thread, 2023. https:/ /ttr///ffoorerriiicceroo .ubbitss.pp///uu /2033/ monosemanticffettures/ itt.irees//dde.htmm...
Zhiqi Bu, Xiaomeng Jin, Bhanukiran Vinzamuri, Anil Ramakrishna, Kai-Wei Chang, Volkan Cevher, and Mingyi Hong. Unlearning as multi-task optimization: A normalized gradient difference approach with an adaptive learning rate. arviv preprint ar arXiv:2410.22083 2024.
Yinzhi Cao and Junfeng Yang. Towards making systems forget with machine unlearning, In 2015 IEEE symposium security and privacy, pp. 463 480. IEEE, 2015.
Sungmin Cha, Sungjun Cho, Dasol Hwang, and Moontae Lee. Towards robust and cost- efficient knowledge unlearning for large language models. arXiv preprint ftt::2408.06211,, 2024.
Pavel Chvykov and Erik Hoel. Causal geometry. Entropy, 23(1)224, 2020.
Hoagy Cunningham, Aidan Ewart, Logan Riggs, Robert Huben, and Lee Sharkey. Sparse autoencoders find highly interpretable features in language models. arXie preprint wrXiv:2309.08600 2023.
Aghyad Deeb and Fabien Roger. Do unlearning methods remove information from language model weights? arXiv preprint ar arXio:2410.0882) 2024.
Nelson Elhage, Tristan Hume, Catherine Olsson, Nicholas Schiefer, Tom Henighan, Shauna Kravec, Zac Hatfield-Dodds, Robert Lasenby, Dawn Drain, Carol Chen, et al. Toy models of superposition. ar Xiv preprint 200.:22009 2022.
12

---

Published as a conference paper at COLM 2025
Chongyu Fan, Jiancheng Liu, Licong Lin, Jinghan Jia, Ruiqi Zhang, Song Mei, and Sijia Liu. Simplicity prevails: Rethinking negative preference optimization for Ilm unlearning. ariiv preprint xrXiv:2410.07163, 2024.
Eoin Farrell, Yeu-Tong Lau, and Arthur Conmy. Applying sparse autoencoders to unlearn knowledge in language models. In NeurIPS 2024 Safe Generative AI Workshop, 2024. URL
https://arri..orr//b////1.... 19278.
Jack Foster, Stefan Schoepf, and Alexandra Brintrup. Fast machine unlearning without retraining through selective synaptic dampening. In Proceedings of the AAAI conference on artificial intelligence, volume 38, pp. 12043-12051, 2024.
Chongyang Gao, Lixu Wang, Chenkai Weng, Xiao Wang, and Qi Zhu. Practical unlearning for large language models. ar preprint ar arXin:2407.10223, 2024.
Chongyang Gao, Lixu Wang, Kaize Ding, Chenkai Weng, Xiao Wang, and Qi Zhu. On large language model continual unlearning, 2025. URL https://arxiv..om wwtpr/ivvv.oo/////////e/ 10223.
Aaron Gokaslan, Vanya Cohen, Ellie Pavlick, and Stefanie Tellex. Openwebtext corpus. http://Skyliono00. githu...huu.... ho/openweeTeextCrpss 2019.
Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding. arXiv preprint arXiv:2009.0333 2020.
Shengyuan Hu, Yiwei Fu, Zhiwei Steven Wu, and Virginia Smith. Jogging the memory of unlearned Ilms through targeted relearning attacks. International Conference on Learning Representations, 2025.
Joel Jang, Dongkeun Yoon, Sohee Yang, Sungmin Cha, Moontae Lee, Lajanugen Logeswaran, and Minjoon Seo. Knowledge unlearning for mitigating privacy risks in language models. In Proceedings of the 61st Anmual Meeting of the Association for Computational Linguistics (Volume Long Papers), pp. 14389-14408, 2023.
Adam Karvonen, Can Rager, Johnny Lin, Curt Tigges, Joseph Bloom, David Chanin, Yeu- Tong Lau, Eoin Farrell, Callum McDougall, Kola Ayonrinde, et al. Saebench: A compre- hensive benchmark for sparse autoencoders in language model interpretability. arXiv preprint ar Xiv: 09532, 2025.
Meghdad Kurmanji, Peter Triantafillou, Jamie Hayes, and Eleni Triantafillou. Towards unbounded machine unlearning. Advances in neural information processing systems, 36: 1957-1987, 2023.
Nathaniel Li, Alexander Pan, Anjali Gopal, Summer Yue, Daniel Berrios, Alice Gatti, Justin D. Li, Ann-Kathrin Dombrowski, Shashwat Goel, Long Phan, et al. The wmdp benchmark: Measuring and reducing malicious use with unlearning. In International Conference on Machine Learning 2024.
Tom Lieberum, Senthooran Rajamanoharan, Arthur Conmy, Lewis Smith, Nicolas Sonnerat, Vikrant Varma, János Kramár, Anca Dragan, Rohin Shah, and Neel Nanda. Gemma scope: Open sparse autoencoders everywhere all at once on gemma 2. arXiv preprint arXiv:2408.05147, 2024.
Johnny Lin. Neurompedia: Interactive reference and tooling for analyzing neural networks, 2023. URL http::://ww.nuuuopppeeeeee
org. Software available from neuronpedia.org
Sijia Liu, Yuanshun Yao, Jinghan Jia, Stephen Casper, Nathalie Baracaldo, Peter Hase, Xiao- jun Xu, Yuguang Yao, Hang Li, Kush R. Varshney, et al. Rethinking machine unlearning for large language models. In International Conference on Learning Representations, February wtriiiiwwweeeffeeee www./n/e////nne mttt//oorumiidd-P22eWk33
2024. URL
Sijia Liu, Yuanshun Yao, Jinghan Jia, Stephen Casper, Nathalie Baracaldo, Peter Hase, Yuguang Yao, Chris Yuhao Liu, Xiaojun Xu, Hang Li, et al. Rethinking machine unlearning for large language models. Nature Machine Intelligence, pp. 1-14, 2025.
13

---

Published as a conference paper at COLM 2025
Yang Liu, Mingyuan Fan, Cen Chen, Ximeng Liu, Zhuo Ma, Li Wang, and Jianfeng Ma. Backdoor defense with machine unlearning. In IEEE INFOCOM 2022-IEEE conference on computer communications, pp. 280-289. IEEE, 2022.
Jakub Lucki, Boyi Wei, Yangsibo Huang, Peter Henderson, Florian Tramèr, and Javier Rando. An adversarial perspective on machine unlearning for ai safety. arXiv preprint arXiv:2409.18005, 2024.
Pratyush Maini, Zhili Feng, Avi Schwarzschild, Zachary Lipton, and J Zico Kolter. Tofu: A task of fictitious unlearning for llms, 2024.
Samuel Marks, Can Rager, Eric] Michaud, Yonatan Belinkov, David Bau, and Aaron Mueller. Sparse feature circuits: Discovering and editing interpretable causal graphs in language models. Xiv preprint Mxki::4033116444 2024.
Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. Pointer sentinel mixture models. arXiv preprint ar arXii:1609.00844 2016.
Kyle O'Brien, David Majercak, Xavier Fernandes, Richard Edgar, Jingya Chen, Harsha Nori, Dean Carignan, Eric Horvitz, and Forough PoursabziiSSnggde. Steering language model refusal with sparse autoencoders. arv preprint arX1o:2411.11129 2024.
Judea Pearl. Causality. Cambridge university press, 2009.
Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly reward model. Advances in Neural Information Processing Systems, 36:53728- 14553728--374, 2023.
Senthooran Rajamanoharan, Tom Lieberum, Nicolas Sonnerat, Arthur Conmy, Vikrant Varma, János Kramár, and Neel Nanda. Jumping ahead: Improving reconstruction fidelity with jumprelu sparse autoencoders. arXiv preprint arXiv:2407.14433 2024.
Shaofei Shen, Chenhao Zhang, Alina Bialkowski, Weitong Chen, and Miao Xu. Camu: disentangling causal effects in deep model unlearning. In Proceedings of the 2024 SIAM International Conference on Data Mining (SDM), pp. 779-787. SIAM, 2024.
Weijia Shi, Jaechan Lee, Yangsibo Huang, Sadhika Malladi, Jieyu Zhao, Ari Holtzman, Daogao Liu, Luke Zettlemoyer, Noah Smith, and Chiyuan Zhang. MUSE: Machine unlearning six-way evaluation for language models. In Conference on Language Modeling Research, July 2024. URL https: http:://o/ppeeeeeeeeeeee net set/forrmmiiddsgAAn@@SCC.
Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shmatikov. Membership inference attacks against machine learning models. In 2017 IEÉE symposium on security and privacy (SP), pp- 3-18. IEFF, 2017.
Pratiksha Thaker, Shengyuan Hu, Neil Kale, Yash Maurya, Zhiwei Steven Wu, and Virginia Smith. Position: Lim unlearning benchmarks are weak measures of progress. IEEE Conference on Secure and Trustworthy Machine Learning, 2025.
Jie Xu, Zihan Wu, Cong Wang, and Xiaohua Jia. Machine unlearning: Solutions and challenges. JEEF Transactions on Emerging Topics in Computational Intelligence, 2024.
Jin Yao, Eli Chien, Minxin Du, Xinyao Niu, Tianhao Wang, Zezhou Cheng, and Xiang Yue.
Machine unlearning of pre-trained large language models, 2024.
Ruiqi Zhang, Licong Lin, Yu Bai, and Song Mei. Negative preference optimization: From catastrophic collapse effective unlearning, 2024.
Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al. Judging lim-as-a-judge with mt-bench and chatbot arena. Advances in Neural Information Processing Systems, 36:46595- 46623, 2023.
Chunting Zhou, Pengfei Liu, Puxin Xu, Srinivasan Tyer, Jiao Sun, Yuning Mao, Xuezhe Ma, Avia Efrat, Ping Yu, Lili Yu, et al. Lima: Less is more for alignment. Advances in Neural Information Processing Systems, 36:55006-55021, 2023.
14

---

Published as a conference paper at COLM 2025
A Additional Background and Related Work Details
This appendix provides further details on concepts mentioned in the Background and Related Work (Section 2).
A.1 Formal Goal of Unlearning
As introduced in the main text, machine unlearning aims to transform a model M(D), initially trained on dataset D Pretain D into unlearned model . Munearnn- The theoretical ideal of exact unlearning requires that Myniearu be computationally indistinguish- able from a model M (DDretain) that was trained exclusively on the retain set Deetain from the beginning (Bourtoule al., 2021; Cao & Yang, 2015). Due to the computational cost of retraining large language models from scratch, achieving exact unlearning is generally impractical. Therefore, the field primarily focuses on developing approximate unlearning methods. These methods aim to satisfy specific criteria related to effectively removing the influence of D forget while preserving the model's performance on Detainn without incurring the cost of full retraining (Liu et al., 2025).
A.2 Unlearning Evaluation Metrics and Benchmarks
The evaluation of approximate unlearning methods typically involves measuring two primary aspects: Forget Quality and Utility Preservation. Forget Quality quantifies the successful removal of information pertaining to the forget set D forget- Common metrics include measuring the Forget Set Performance Degradation, which involves observing reduced accuracy or increased loss on tasks specifically related to the content of forget (Shi et al., 2024; Maini et al., 2024). Another aspect is assessing Memorization Metrics, which gauge the model's 'ss reduced ability to recall specific sequences or knowledge points verbatim from D forget (Shi et al., 2024). Furthermore, Privacy Leakage Metrics evaluate the decreased success rate of Membership Inference Attacks (MIAs) that try to infer whether a given data point was part of the original D forget, often quantified using the Area Under the Curve (AUC) of the MIA classifier (Shokri et al., 2017; Shi al., 2024).
Conversely, Utility Preservation assesses how well the unlearned model retains its general knowledge and capabilities on tasks unrelated to D Forget-- This commonly measured by evaluating Retain Set Performance Preservation, which checks for maintained accuracy on standard academic or commonsense reasoning benchmarks such as MMLU (Hendrycks et al., 2020). Additionally, General Language Modeling Performance is often assessed by ensuring minimal increase in perplexity or loss when the model processes large, general- purpose text corpora like OpenWebTee 2016). Finally, Fluency and Coherence of the model's generated text are important, often evaluated through automated metrics, human judgment, or interaction with benchmark chatbots like MT-Bench (Zheng et al., 2023). Standardized benchmarks like MUSE (Shi et al., 2024), TOFU (Maini et al., 2024), WMDP (Li et al., 2024), and SAEBench (Karvonen et al., 2025) provide datasets, tasks, and evaluation protocols designed to measure performance across these diverse criteria.
(Gokaslan et al., 2019) or WikiText (Merity et al.,
A.3 Gradient-Basse -Based Unlearning Methods
Gradient-based unlearning techniques directly modify the weights 0 of the original model M(D) using optimization algorithms, typically variants of gradient descent or ascent.
Gradient Ascent (GA) represents a basic approach where the optimization objective is to maximize the loss function (e.g., negative log-likelihood) the forget set D forget, thereby pushing the model parameters away from configurations that accurately represent this data (Jang et al., 2023). This method, however, often suffers from catastrophic forgetting of useful knowledge if not carefully regularized.
Gradient Difference (GradDiff or NegGrad) attempts to balance forgetting and retention by computing gradients for both minimizing loss on Dretain and maximizing loss on D forget
15

---

Published as a conference paper at COLM 2025
then applying update based on combination (often subtraction) of these gradients (Liu et al., 2022).
Negative Preference Optimization (NPO) leverages insights from preference-based fine- tuning methods like DPO (Rafailov et al., 2023), reformulating unlearning as learning to disprefer outputs related to D forget relative to some reference, which could be outputs from the original model data from D retain (Zhang et al., 2024). Simplified variants like SimNPO aim to reduce the computational overhead (Fan et al., 2024).
Representation Misdirection Unlearning (RMU) operates by injecting noise or applying targeted shifts to the internal activations of the model at specific layers, but only when processing inputs related to D forgets while simultaneously using a regularization term to keep activations on Dretain close to those of the original model (Li al., 2024).
Selective Synaptic Dampening (SSD) aims for more targeted weight modification by estimating the importance of individual parameters for both forget and Dretain (using approximations based Fisher information) and then selectively reducing the magnitude of parameters found to be more critical for D forget than for Dretain (Foster et al., 2024).
SCRUB employs a student-teacher knowledge distillation framework; it trains a copy of the original model (the student) to diverge from the original frozen model (the teacher) on D forget inputs (typically by maximizing KL divergence) while simultaneously encouraging the student to mimic the teacher on Deetinn inputs (by minimizing KL divergence) (Kurmanji et al., 2023).
Finally, many gradient-based methods incorporate explicit Regularization Techniques to counteract the tendency towards catastrophic forgetting. Common regularizers include minimizing the KL divergence between the probability distributions of the unlearned and original models when evaluated on Dretain (Maini et al., 2024), or directly including a term in the loss function that minimizes the model's 's prediction error on Dretain (Yao et al., 2024).
A.4 Prior SAE Unlearning Work
The work by Farrell et al. (2024) is an early exploration into using Sparse Autoencoders (SAEs) for machine unlearning. We describe their methodology here.
First, they computed the activation sparsity for each feature in the SAE dictionary, calculated separately over the forget dataset (D forget) and the retain dataset (Dretanu). Sparsity here refers to the fraction of input tokens for which given feature has a non-zero activation.
Second, to mitigate potential damage to the model's general capabilities, they filtered out any features whose activation sparsity on the retain set Dyetain exceeded a predetermined threshold (e.g,, a feature active on more than 1% of retain tokens might be excluded).
Third, from the pool of features that passed the retain-sparsity filter, they selected the top-N features that exhibited the highest activation sparsity when measured on the forget set D Forget- The assumption was that features frequently active on forget data are likely responsible for encoding the knowledge to be removed.
Fourth, they implemented a static intervention mechanism during inference: whenever any of the top-N selected features fj produced positive activation (f, > 0) for any token I, its activation was clamped to fixed negative value (e.g.--c This clamping was applied universally, regardless of the overall context of the input sequence.
This combination of sparsity-based feature selection and static clamping ultimately proved limiting, leading to significant side effects on utility and performance inferior to contempo- rary gradient-based methods like RMU on benchmarks such WMDP-Bio. Recognizing these limitations directly motivated our work (DSG), where we instead develop and apply principled feature selection and dynamic, context-aware interventions.
16

---

Published as a conference paper at COLM 2025
A.5 Relearning Attacks
Approximate unlearning methods, especially those modifying model weights, face another significant challenge: relearning attacks (Deeb & Roger, 2024). In these attacks, an adversary finetunes the unlearned model Municarn to recover the supposedly forgotten information. Such recovery is sometimes possible even using only data tangentially related the original forget set D forget (Hu et al., 2025). The success of relearning attacks suggests that gradient- based weight modifications may primarily suppress access to knowledge rather than truly erasing it from the parameter space; subsequent finetuning can often reverse these weight adjustments, particularly if it reinforces the target concepts.
The feasibility of relearning attacks strongly depends on the threat model. In an API-based (black-box) setting, where adversaries only have query access, mounting effective relearning attacks is more difficult, particularly if the provider restricts extensive finetuning or monitors queries. irivation-basel intervention methods like DSG, which modify internal states rather than weights to control outputs for relevant inputs, may offer greater robustness in this black-box scenario compared to weight modification techniques.
Although sophisticated obfuscation attacks targeting activation-based defenses exist (Bailey et al., 2024), they typically require white-box access (e-g., gradients, internal states). Such access is unavailable in a pure API setting, limiting their threat against deployed systems focused on output safety via activation manipulation. DSG's potential resilience against relearning could stem from the relative stability of activation geometry during standard finetuning, a phenomenon related to the Superficial Alignment Hypothesis (Zhou al., 2023). If DSG reliably identifies features encoding the target knowledge based on these stable patterns and consistently applies interventions, the unlearning effect may prove more durable against finetuning- based relearning attacks compared to methods reliant on weight configurations.
B Fisher Information Approximation Proof
Theorem (Approximate Fisher Information from SAE Features). Let a sparse autoencoder (SAE) with reconstruction f(x) = W be applied data x D, where z(x) RR represents latent activations and W Wdec E FxD the decoder weights. Define the reconstruction loss as:
E(x) = 2 [l((x) - r(x)11
If the SAE is well-trained such that reconstruction error is small with high probability, then for each row 0; E RD of W dec (representing feature i), the expected squared gradient is approximately proportional to the second moment of the feature activation.
Proof. We establish this result through careful analysis of the gradient structure in sparse autoencoders.
Computing the Gradient of Decoder Weights. By definition of the reconstruction loss:
{(x) ||((x) r(xx]12
zz(xxx Wdee - r(x)112
For row i of Wdec denoted 0, RDD, we compute the gradient:
Vo, ((x) = Ve,
1
1
l Wec x(xx)
By the chain rule:
Vg, 8 ((x) = (x(x) x(x)) Vo, (Zec)
17

---

Published as a conference paper at COLM 2025
Since Wee is linear in 0, with coefficient (1),,, we have:
Ve, (zex) dee) zi(x) In
where ID is the D-dimensional identity matrix. Therefore: Vo, (x) (fxx((((x) r(x))
Computing the Squared Gradient Norm. Taking the squared norm of this gradient:
[Vo. (x)]l = (+(x) - r(x))11
(x))((((( r(x)]l²
Taking the expectation over the data distribution:
Bx-DllVo,, o, ((x)1l Ex E
Analyzing the Small Error Regime. When the SAE is well-trained, we can characterize its performance with a high-probability bound reconstruction error. Specifically, assume there exist constants e > 0 and such that:
[|((x)) (xx(x)>(xx1132 <
where < ||((x)] and 8 is small. In other words, the squared reconstruction error is bounded by 22 with probability at least 5.
Under this high-probability bound, we can decompose the expectation:
(x)(1()) r(x))1 < (x) e² |[((x)) r(x))] < c²) (1- + C5
{x) C8
where C is bound the expectation in the low-probability case. For small ó and finite C, the second term becomes negligible, leaving:
E (x) - r(x) = (2E)2
Connection to Fisher Information. defined as:
The Fisher Information Matrix for parameter 0, is
I(O,) Ex-DDVV, VVo (x) {(x)
The trace of this matrix, which measures the overall sensitivity of the loss to changes in 0 precisely:
Tr(I(O;)) Ex- V [())11 2 (x)
Interpretation. The above analysis shows that =z serves as a natural impor- tance measure for feature j. Features with larger average squared activations contribute more significantly to reconstruction gradients and thus have higher Fisher Information content. This justifies our approach using squared activations to identify features most strongly associated with specific knowledge domains.
C Connecting Fisher Information to Causal Influence
In this section, we establish how the Fisher Information associated with Sparse Autoen- coder (SAE) features connects to their causal influence as mediators information flow in language models. Drawing inspiration from causal geometry (Chvykov & Hoel, 2020), we provide a proof for why expected squared activation serves as a measure of feature importance.
18

---

Published as a conference paper at COLM 2025
(Fisher Information Proxy for Causal Feature Importance). Under assumptions of (i) deneemeeeiiiiittc mappings in the language model, (ii) well-defined causal effects under feature interventions, (iii) small SAE reconstruction errur, and (iv) approximate feature independence, the Fisher Information associated with SAE features provides a principled approximation of their causal influence. Specifically, for any feature fj. the expected squared feature activation Ep (h)) for hidden state h on dataset 'D is proportional to the causal influence of that feature as a mediator of information from to model outputs.
Theorem 2
Proof. We build upon the result in Appendix B, which showed that the expected squared activation ) is proportional to the trace of the Fisher Information Matrix for the corresponding decoder weights.
Consider language model (LM) that produces hidden states h(x) E RR. A Sparse Autoencoder (SAE) encodes (xx) into feature activations z = ffh E R Le. each feature is f (h(x)) Let Porgett and Peetain be two subsets of the training data. We model the causal structure as:
Causal Model Setup.
Data - h(x) z z f(h) - Y (model outputs)
Here, Y E Rdy represents the model's output vector (e.g,, logits or embeddings).
Assumptions.
1. Near-deterministic mapping. Conditioned on h, the model output Y is almost deter- ministic (small Gaussian noise). Formally, p(Y h) ~ N (N((()), with small o2.
Well-defined feature interventions. We can perform do =a), meaning forcibly setting feature j to a and thus severing its normal dependence on h.
3. Small SAE reconstruction error. Writing hizz) ~ W we assume | - (z)l is small with high probability.
4. Approximate feature independence. Features (h) are sufficiently sparse decorre- lated that cross-terms can be neglected.
Defining Causal Influence. We quantify the causal influence of feature fj by how much the model's output distribution p(Y) changes when we intervene to set f, to its normal value (h) vs. forcing it to zero:
Influence(f)) = En-D ( p(Y do(f; = f(())) || P(Y | do(f) = -]))]
A large KL means toggling fj from 0 to its actual value drastically shifts p(Y), so fj is strong mediator for D.
Expanding KL Divergence. Let R -> Rdy describe how feature fy shifts the model's outputs. Since we forcibly set f, (an intervention), we ignore any prior correlations with h, and under near-determinism the output distribution is approximated by:
F do(f; = a)) N (g) o²1)
For two different interventions do(f; = a) and do(f; = B), we can now derive the KL divergence between the resulting output distributions. Using the standard formula for KL divergence between multivariate Gaussians with the same covariance matrix:
1 (m TI (m 222
(W,,, E))
19

---

Published as a conference paper at COLM 2025
Therefore:
(Kx( do(fi =a) ll ( do(f; B)) Da..Vig(cc.
2 (8)(()) -(())) (gg -1)()))(())))
1
20 (g)(a) ()())) 8)(
1
2 g/ (B)l
First-Order Taylor Expansion. To make this expression more tractable, we use first-order Taylor expansion of (a()) around = 0:
(a) = 80(0) + dgi
a + ((a)
da
4==
(0) + ()))
When sufficiently small, the higher-order terms o(a) become negligible. Substituting this back into our KL divergence expression for the special case where f = 0:
(p(( = a)) a))ll 0)))
(a) - (0)]
20
() (0) a (0)1
20
(0) a||
20
22
(0)
20
This shows that the KL divergence (our measure of distribution change) grows quadratically with the intervention magnitude a, with proportionality constant determined by the gradient norm (0)]])
Expected Causal Influence. Now we can compute the expected causal influence by substi- tuting a = (h) and taking the expectation over h ~ D:
Influence
EEnD
DKI (p( =
= 0)))
B
20 (i))
(g;(0)
V
(0)12
ENND [h)
20
Thus, the expected causal influence of feature j as mediator of information from dataset D is directly proportional to the expected squared activation IEp [f with proportionality constant [0)] that depends on the sensitivity of the model's outputs to changes in 2nd
feature
Connection to Fisher Information. The Fisher Information for the SAE's decoder weights 8, satisfies I(0;,) En [h)]] since the gradient w.r.t. 0j. includes fy(h) as leading factor. Therefore, (h)? tracks both the Fisher Information and the intervention-bant notion of causal influence we derived above, establishing direct link: Causal Influence o Fisher Information( (0;:) (h))) In other words, features most important in Fisher Information sense are precisely those with greatest causal influence model outputs.
20

---

Published as a conference paper at COLM 2025
Implications for Feature Selection. By identifying features with high squared activations on Poorget but low activations on rettann we can target mediators that specifically carry forget set knowledge. Clamping these features to zero during inference selectively reduces the model's capacity to propagate information from Piorget while preserving performance on D retain.
Comparisons Across Datasets.
For D forget Vs. D Deelain, we earlier defined:
forget score (j) Eroogee (h)2]
Aitain_scorll (j) E (h)
The ratio of causal influence of feature j for Diorget versus D Pettan is:
(0)11
Diorge [Influence (f))
E
2
IE Di [f; [h)
Pioggel
Drettin Petmin [Influecce
fi]
(0)l2
E. (f (h)a]
20
E D
[fi(h)
B
[f(1)
forget.scoreef]
retain,score())
Thus, wat.sscooerrrrriiiiii
precisely captures how much more fj mediates the forget dataset relative to the retain dataset. This is the importance ratio we defined in Section which directly quantifies the relative causal influence of feature j across datasets.
D Proof of Neyman-Pearson Optimality
Theorem (Neyman-Pearson Optimality). Let Dforret and Pretuin be the distributions of se- quences from the forget and retain sets, respectively. p()) is stochastically larger under Dorret than under retain (i.e., Prx-DDpage [p(X) 2 t] 2 Prx mitan (x(X) 2 for all t), then among all classifiers with a fulse-positive rate at most a, the threshold test C* 1[p(x) -*), where Prx-Dramin [p(X) T*] a, maximizes the true-positive rate.
Proof. We adapt the classical Neyman-Pearson Lemma to our unlearning context. Our goal is to find the optimal decision rule for classifying inputs either forget-relevant or retain-relevant.
Consider the class of all decision rules X' {clamp, no-clamp) with false-positive rate at most a. That is, all rules a such that:
Pr X-
[a(X)= = clamp] <
For each decision rule a, define its acceptance region A fx E X a(x) clamp). The constraint on false-positive rate translates to Prx~-ratan [A) < a.
Now, define the threshold-based decision rule a* as:
a" (x) = 1(((x) > T*]
where T" is chosen such that Prx-Daum [p(X) -*] a. The acceptance region for this rule is A" = {x: p(x) > T'}.
We need to prove that maximizes the true-positive rate among all rules with false-positive rate at most a. In other words, for any rule with Prx~Dratin [a(X] = clamp! a, we must show:
Pr X-Pooetet
[x(X) clampl <
Pr Diogetttt
(X) clamp)
21

---

Published as a conference paper at COLM 2025
We use the stochastic dominance property: for any threshold t, Prx--Dooge [p(X) 2 f) 2
Prx-Drehain (x) 2 fl. This means that regions of higher p values are relatively more likely under D Piorget than under retain-
Consider any decision rule a with acceptance region where Prx-- [A] < a. Due to the stochastic dominance property, we can always construct a threshold-baser region Á = f ;x)) > 7 such that: 1. Prx--rrtain [A] = Prx- [A] (same false-positive rate) 2.
Prx~DDoogee [A] Z Prx-Dioogee [A] (equal or higher true-positive rate)
This is because exchanging points from low-p regions in with points from high-p regions outside A (while maintaining the same false-positive rate) will always increase the true- positive rate due to stochastic dominance.
Degtain [A] < a, we can further expand Ā to A* by lowering the threshold from t", which only increases the true-positive rate further.
PrX-Drgaan
Therefore, for any decision rule with false-positive rate at most a:
Pr
[a(X) = clamp]
Pr X~>Dooget
[A]
Pr
[A*]
Pr X~>Daaret
(x' = clamp]
X X->
X X~DDooret
This establishes that the threshold test a" (x) = 1[p(x) > r*) maximizes the true-positive rate among all tests with false-positive rate at most
Practical Implications: This theorem establishes the statistical optimality of our thresholding approach for making the binary decision of whether to apply an intervention. In particular, it shows that our dynamic classification rule maximizes coverage on forget-set queries while maintaining controlled false-positive rate on retain-set queries.
E Proof of Dynamic Clamping Dominance
Theorem 4 (Dominance of Dynamic Clamping). Let Suyars be a fixed subset of features identified as forget-velevant. Consider the slatic approach Astatic (x) that clamps features in Smyats whenever they activate, and the dynamic approach adymamie (x) that first classifies input using C(x) 1lp(x) T] and only then applies clamping. Under the stochastic dominance assumption from Theorem there exists a threshold I" such that Adynamic achieves equal or greater coverage on Dforget than astatic while maintaining equal sidc-effects on D Peetain, making dynamic clamping strictly dominant in the coverage-sinn effect trade-off.
Proof. We begin by formalizing the metrics used to evaluate both approaches and precisely defining their operation.
Preliminaries and Definitions. Let X be the space of possible input sequences. For sequence x = XT) and its corresponding hidden states h,, we define:
A token is triggered by Spearss if 3j E Supats such that f/(h.) >
The fraction of triggered tokens in a sequence: p(x) 1 E.. 1
S
f/(h:) 0]
We consider two distributions: Diorget: The distribution of forget-relevant queries, and D Dretain: The distribution of retain-relevant queries.
The Two Approaches. For both approaches, we define a clamp set Bmethod C X' as the set of inputs where the method applies some clamping.
1. Static Approach (Astatic): Clamps features in Sujats whenever they activate on any token. here the Clamp set is Bstat {x 3t,j E Supars such that fi(h:) > 0}.
22

---

Published as a conference paper at COLM 2025
2. Dynamic Approach (adynamic): Computes p(x) and applies threshold test p(x) > Only clamps if the sequence passes this test. The Clamp for threshold Bayn (T) {x: p(x) > T}.
Performance Metrics.
We define:
Coverage: The probability that clamping occurs on forget-set queries
Coverage(method) = Pr [x E
Bmethod]
X->Diorgt
Side-effect: The probability that clamping occurs on retain-set queries
SideLiffec((method)
Pr
[x E
Bmethod]
x~>Dretain
Step 1: Find the side-effect of the static approach. The static approach clamps whenever
any token has activating feature in Smeeas Therefore:
(ideeffect(Astatti)
Pr x->Dretain
E Bstat]
Step 2: Find threshold r' that yields the same side-effect for the dynamic approach.
By our assumption that p(x) is stochastically larger on Dorget than on D Pretain. we know that Prx~Dretai [p(x) > T] is strictly decreasing function of r.
Therefore, there exists a threshold Ta E [o, 1] such that:
Pr retain
[p(x) T"
Pr Pettain
[x E Bstat]
This means:
SideEffect(Aaynamic (T")) SideEffect (Astati)
Step 3: Show that coverage is greater for the dynamic approach. From Theorem 3, we know that thresholding p(x) at r* gives the optimal classifier for distinguishing between Diorget and D Dretain at false positive rate a.
More formally, among all sets A C A' with Prx-Drutan [x E A] = a, the set Bayn(T") = (x
p(x) > T'} maximizes PronDoo [x E A].
Since Bstat is one such set with Prz-Dreesin [x E Bstat] = a, we must have:
Pr E Byyn(T")] 2 x- Dtoggtt
[x Bstal]
x- Diogeet
Therefore:
Coverage(Adynamic (T*)) Coverage(Rstatic)
(xx) strictly stochastically larger on Dioret than on D Pretain (which holds in practice as forget-relevant features activate more frequently on forget-set queries), then this inequality is strict.
We have established that for any static clamping approach, there exists a threshold r* such that the dynamic approach with this threshold achieves the same side-effect on the retain set; and achieves equal or greater coverage on the forget set. This proves that dynamic clamping dominates static clamping in the coverage-side effect trade-off.
23

---

Published as a conference paper at COLM 2025
F
Distribution of token activations on WMDP-Cyber
Figure 8 plots the distribution of forget-set activated tokens on WMDP-Cyber. The threshold is chosen to control the retain set's false positive rate and we find that Páyn = 95 typically separates forget-set queries effectively achieving high recall on D forget- On WMDP-Cyber, DSG successfully transfers from the retain set (WikiText) and forget set to the test query set.
150
MMLU
125
WMDP-Cyber
Counts
100
Retain set (WikiText)
Forget set (WMDP-Cyber)
75
50
25
0.0
0.2
0.4
0.6
0.8
1.0
% of forget-set activated tokens in sequence
Figure Distribution of forget-set activated tokens for WMDP-Cyber. Threshold the 95th percentile (dashed red line) effectively separates MMLU from WMDP.
G Unlearning on WMDP
G.1 Hyperparameter Details and Model Descriptions for Baselines
To ensure comprehensive and fair comparison of unlearning methods, we conducted extensive hyperparameter sweeps for each baseline, optimizing for both the effectiveness of knowledge removal and the preservation of model utility. For all gradient-based methods, we experimented with updating parameters in layers 3, 7, and 11 (as recommended in (Li et al., 2024)), as well as all layers. Unless otherwise specified, all experiments used the google/gemma-2-222 gemma-2-2bbii it (Lieberum et al., 2024) model.
Dynamic SAE Guardrails (DSG). Our proposed method, DSG, is non-gradient-based intervention method that selectively removes hazardous knowledge by manipulating SAE feature activations. DSG first identifies subset of SAE features strongly indicative of the knowledge to be forgotten, based on their differential activation patterns on forget and retain datasets. During inference, DSG employs a dynamic classifier to assess the relevance of input sequences. a sequence classified as forget-relevant based on the aggregate activation of selected features, DSG dynamically clamps these features negative value. This conditional, sequence-level clamping ensures that intervention is applied only when necessary, minimizing side effects on benign inputs and preserving model utility.
We employed the gemma- wwwps-co--t---eeeeeee SAE (width 16k) applied to layer 3 (&o 142) (Lieberum al., 2024). The dynamic threshold percentile (Payn) was fixed at 95. We swept the importance ratio percentile (Pratio), number of selected features, and clamp strength (c):
Hyperparameter
Values Tested
Importance Ratio Percentile (Pratio)
90, 95
Number of Features
10, 20, 30
Clamp Strength (c)
10, 25, 50, 100, 20, 300, 400, 500
Table 3: Hyperparameter sweep for Dynamic SAE Guardrails (DSG). Fixed values: Payn = 95.
The best configurations were: WMDP-BE (Pratio = 95, features=20, = 500) and WMDP- Cyber (Pratio 90, features=30, 500).
24

---

Published as a conference paper at COLM 2025
Representation Misdirection for Unlearning (RMU). RMU al,,, 2024) is gradient- based finetuning method that minimizes a composite loss function to achieve targeted forgetting while preserving model utility. This loss combines forget loss and retain loss. The forget loss acts on the model's activations on the forget dataset, increasing their norm in specific directions and making it difficult for later layers to process this information effectively. Simultaneously, the retain loss regularizes the updated model's activations on the retain dataset, encouraging activations to stay close to the original model's activations on benign data.
Key hyperparameters include the steering coefficient, which controls how much the ac- tivations are amplified on hazardous data, and the alpha parameter (a), which balances utility preservation against knowledge removal. We focus unlearning only on the MLPs, as recommended in Li et al. (2024).
Hyperparameter
Values Tested
Steering Coefficient
5,5, 10, 20, 100, 200, 400, 500, 800, 1000 0.01, 0.1, 1, 10, 100, 300, 500 4,8 8
Alpha (a)
Batch Size
Steps
400, 800
Table 4: Hyperparameter sweep for RMU. Fixed values: Monitoring Layer ID=3, Learning Rate=5e-6.
The best configuration for WMDP-Bio used steering coefficient 400, alpha 100, monitoring layer 3, learning rate 5e-6, batch size 8, and 400 steps. For WMDP-Cyber, we used steering coefficient 500, alpha 10, monitoring layer 3, and batch size 8 with 400 steps.
Scalable Remembering and Unlearning unBound (SCRUB). SCRUB (Kurmanji et al., 2023) employs a student-teacher framework for knowledge distillation-based unlearning. It trains student model, clone of the original model, to forget hazardous knowledge under the guidance of the original, frozen teacher model. During forget epochs, SCRUB maximizes the KL divergence between student and teacher logits on the forget dataset. In retain epochs, it minimizes this divergence on the retain dataset, guiding the student to mimic the teacher on benign data.
We swept across values beta (6), a weighting factor balancing knowledge distillation and task-specific loss, while fixing alpha (a) and gamma (r) at 1.0:
Hyperparameter
Values Tested
Beta (B)
0.0001, 0.001, 0.01, 0.1, 1, 10 le44, 1e-5, 5e-6
Learning Rate (Ir)
Batch Size
4,8
Steps
400, 800
Table Hyperparameter sweep for SCRUB. Fixed values: 1.0, Y 1.0, KL Temperature=2.0.
The best configuration for WMDP-Bio used beta 0.01, learning rate 5e-6, batch size 8, and 400 maximum batches. For WMDP-Cyber, we used beta 0.1, learning rate 1e-5, batch size 8, and 400 maximum batches.
Selective Synaptic Dampening (SSD). SSD (Foster et al., 2024) identifies and dampens parameters more important for the forget set than the retain set. It adapts a method originally developed for image classification to language modeling by modifying the loss function to use log-perplexity. SSD calculates parameter importance scores based on gradients observed for both forget and retain datasets, then applies dampening factor to parameters with higher importance for the forget dataset.
We performed grid search spanning dampening thresholds and constants:
25

---

Published as a conference paper at COLM 2025
Hyperparameter
Values Tested
Threshold
0.1, 0.25, 0.5, 1, 2.5, 1e-5, 1e-4, 1e-3, 1e-2, 1e-1,
Dampening Constant
Table 6: Hyperparameter sweep for Selective Synaptic Dampening (SSD).
The optimal configuration for WMDP-Bio used threshold 0.5 and dampening constant 1e-3. For WMDP-Cyber, we used threshold 1.0 and dampening constant 1e-2.
Static SAE Clamping (Farrell et al.). This non-gradient-based approach (Farrell et al., 2024) identifies salient SAE features and statically clamps their activations during inference to remove unwanted knowledge. Unlike our dynamic approach, this method applies feature clamping universally to all inputs whenever selected feature activates, rather than conditionally based on sequence-level classification.
We varied the retain threshold, multiplier (clamp value), and number of features:
Hyperparameter
Values Tested
Retain Threshold
0.01, 0.001, 0.005, 0.1, 10, 25, 50, 100, 200, 500 5, 10, 20, 30, 50
Multiplier (Clamp Value)
Number of Features
Table 7: Hyperparameter sweep for Static SAE Clamping. Fixed value: Sequence Length=1024.
The best configurations were: WMDP-Bio (retain threshold=0.01, multiplier=200, features=5) and WMDP-Cyber (retain threshold=0.005, multiplier=500, features-10).
Gradient Ascent (GA). GA (Jang et al., 2023) is a finetuning-baset unlearning method that directly minimizes the likelihood of correct predictions on the forget catasett using gradient ascent. In contrast to standard finetuning which employs gradient descent, GA utilizes gradient ascent to maximize the cross-entropy loss on the forget dataset, pushing parameters in directions that increase prediction error the targeted data.
We varied the learning rate and beta (6), the retain loss weight:
Hyperparameter
Values Tested
Learning Rate (1r) Beta (Retain Loss weight)
le-5, 5e-5
0.01, 0.1, 1.0, 5.0, 10.0
Table 8: Hyperparameter sweep for Gradient Ascent. Fixed values: Gamma=1.0, Batch Size=8, Steps=400.
We explored both with and without retain data configurations. The best setting for WMDP- Bio used learning rate le-5 with beta 1.0. For WMDP-Cyber, we used learning rate 1e-5 with beta 0.1.
Negative Preference Optimization (NPO). NPO (Zhang et al., 2024) adapts preference optimization techniques to treat the forget set negative examples. It reframes unlearning as preference learning, optimizing the model to assign lower likelihood to the forget set. The beta parameter controls the extent to which the unlearned model's output distribution can diverge from the original model. To mitigate utility degradation and preserve performance on benign data, NPO can be regularized using two distinct retain loss types: Negative Log-Likelihood (NLL) and Kullback-Leeblee encourages the model to maintain high probabilities for correct tokens the retain set, calculated as the negative sum of log probabilities assigned to ground truth tokens. KL divergence minimization encourages the probability distribution of the unlearned model to remain close to that of the original model on retain set inputs, measured as the information lost when approximating the original model's distribution with the unlearned model's distribution.
(KL) divergence. NLL minimization directly
26

---

Published as a conference paper at COLM 2025
We tested NPO with various configurations:
Hyperparameter
Values Tested
Alpha (Retain Loss Weight) Beta (Temperature Paarameter)
0.01, 0.1, 1.0
0.1, 1.0 NLL, KL.
Retain Loss Type
Table 9: Hyperparameter sweep for NPO. Fixed values: Size=8, Steps=400.
Gamma=1.0, Learning Rate=1e-5, Batch
The optimal settings for WMDP- Bio used alpha 0.1, beta 0.1, and KL. divergence as the retain loss type and WMDP-Cyber used alpha 1.0, beta 0.1, and KL divergence as the retain loss type.
For all methods, we selected configurations that minimized WMDP accuracy while main- taining least 99% of the original model's MMLU accuracy.
Compute.
All finetuning and inference was performed on A6000 GPUs in under day.
G.2 Results on WMDP-Cyber
Table shows the performance of various unlearning baselines on WMDP-Cyber dataset. RMU is less effective WMDP-Cyber (88.00%), likely due to the data inefficiency of gradient-based methods on the smaller cyber forget set. set.
MMLU (1)
Method
WMDP Cyber (1)
MT (1)
HS Hist
C. Bio
HS Geo
H. Aging
All
Target M
100.00
100.00
100.00
100.00
100.00
100.00
7.36
GA
98.91
98.15
100.0
100.0
100.0
99.46
7.39
NPO
96.36
100.0
100.0
100.0
100.0
100.0
7.18
SSD
98.91
100.00
100.00
98.08
98.81
99.19
7.25
SCRUB
97.82
99.07
100.00
100.00
98.81
99.46
6.51
Farrell et al.
52.73
99.07
100.00
100.00
97.62
99.19
7.39
RMU
88.00
99.07
100.00
99.04
98.81
99.19
7.28
DSG (Ours)
26.74
99.07
100.00
100.00
100.00
99.73
7.66
Table 10: Unlearning performance on WMDP-Cyber, All represents the average MMLU score. MT- Bench scores show 0.13 variance across 5 runs. DSG shows superior unlearning effectiveness compared to other baselines while maintaining high MMLU performance.
G.3 MT-Bench Evaluation Details
To measure the impact of unlearning on the model's general conversational abilities and fluency, we utilized the MT-Bench benchmark (Zheng et al., 2023). Specifically, we report the average score across two conversational turns (the two-turn average score), which provides measure of multi-turn conversational quality. Following standard MT-Bench protocol, evaluations were conducted using GPT-4 (Achiam et al., 2023) as the judge to score the model's responses. To ensure the robustness of these fluency assessments, each model configuration reported in Section 4.1 was evaluated 5 times using MT-Bench. The mean scores presented in Table and Table reflect the average performance across these runs, and the standard deviation across the 5 runs is noted in the respective table captions (0.16 for WMDP-BB results, 0.13 for WMDP-Cyber results). Higher MT-Bench scores indicate better preservation of general conversational capabilities after the unlearning procedure.
we
27

---

Published as a conference paper at COLM 2025
H Unlearning on MUSE
H.1 Hyperparameter Details and Model Descriptions for Baselines
We provide implementation details for the baseline unlearning methods evaluated in our experiments.
Gradient Ascent (GA). GA maximizes the loss on the forget set, directly opposing the standard training objective push the model away from the forget data's distribution (Jang et al., 2023). While straightforward, it often leads to catastrophic forgetting of general knowledge.
Gradient Difference (GradDiff). GradDiff balances competing objectives by maximizing the loss on the forget set while minimizing the loss on the retain set (Liu et al., 2022). Despite this approach, GradDiff struggles to find an optimal trade-off, resulting in either over- or under-unlearning:
Negative Preference Optimization (NPO). NPO reframes unlearning within a preference learning framework, treating the forget set as negative preference data by adapting the Direct Preeerrnce Optimization objective (Zhang et al., 2024). We use NPO with KL Diver- gence Minimization that augments NPO with a KL divergence term to preserve utility by minimizing distributional shift on benign data.
Simplified NPO (SimNPO). A computationally efficient variant of NPO that simplifies the optimization process while retaining core principles of negative preference learning (Fan et al., 2024). SimNPO trades some unlearning effectiveness for faster processing.
Representation Misdirection for Unlearning (RMU). RMU injects targeted noise into specific layers to disrupt the model's ss ability to process information related to the forget set (Li et al., 2024). Its effectiveness depends heavily on precise noise targeting and hyperparameter tuning. We injected noise in the 7th layer for both News and Books.
For all finetuning- baselines (GA, GradDiff, NPO, SimNPO, RMU), we used AdamW optimizer with learning rate of 1e-5 and batch size of 32. We finetuned all parameters in the model. The optimal checkpoint for each method was determined by selecting the first epoch (within 10 epochs) where the unlearned model's utility on the retain set fell below 90% that of the target model. Table 11 summarizes the optimal epochs or a values for each method on both datasets.
Unlearning Method
NEWS
BOOKS
GA
epoch 1
epoch
GradDiff
epoch 2
epoch 3
NPO
epoch 8
epoch 10
SimNPO
epoch 10
epoch 10
RMU
epoch 9
epoch 10
Table 11: Optimal epochs for baseline unleaming methods on MUSE benchmark, determined by
utility-based stopping criteria.
All finetuning and inference was performed on 4 A6000 GPUs in under a day.
H.2 Sequential Unlearning Strategies for DSG
In real-world scenarios, unlearning requests often arrive sequentially over time. An effective unlearning method must be able to handle multiple, successive requests without significant degradation in performance or utility. In Section 3, we evaluated DSG's performance under sequential unlearning using the MUSE benchmark (Shi et al., 2024) with four disjoint folds of the NEWS corpus. We implemented and compared two strategies for adapting DSG to this sequential setting, referred to DSG and DSGumion- Both strategies leverage the core DSG mechanisms of feature selection and dynamic thresholding but differ in how they aggregate information across multiple unlearning requests.
28

---

Published as a conference paper at COLM 2025
Setup. Let k = 1,2, index the sequential unlearning requests. Each request k in- troduces a new forget dataset DFA- We assume the retain dataset DR remains constant unlearned model that effectively
throughout the process. The goal at step k is to produce forgets the cumulative forget data Doumul = Sf - Dr, while preserving utility evaluated on DR. Let Ileats be the desired number of features to select at each relevant stage.
Strategy 1: alG (Cumulative Score Update) This strategy treats the sequential unlearn- ing problem equivalent to unlearning single, growing forget set Deumul at each step k. maintains cumulative statistics required for calculating the feature importance scores. Cumulative Statistics: At step k, we need the aggregate sum of squared activa- tions and the total number of tokens for all forget data seen far. Let ,
ΣxEDG, - I (f be the sum of squared activations for feature j on dataset DF and Nr = ExcD;; |x| be the total number of tokens in Dr.- The cumulative sums at step k are:
k
(fx = Σ A2,(
1-1
k
cumul
F.
These sums can be updated incrementally as each new DF arrives, without needing to store all previous datasets. The retain set statistics (A) (j) and NR) are computed once from DR- Score Calculation: The importance scores are calculated using the cumulative statistics:
Fx (
forget.scoreanj =
Ncumul
A (i)
retain.score(i)
(constant across
NR
imp-ratioank())
max(retaiinssore(()..
Feature Selection: Using imp_ratio and forget.scoreall (f), select the feature set
Salla containing the top ffeats features, following the procedure in Algorithm (filtering by percentile Pratio and ranking by forget score).
Dynamic Threshold and Intervention: Calculate the activation statistic (2) as (x))x()) = (1/|xl) E, 13j E SAlla fi(h,) > 0). Calibrate the dynamic threshold Pdyn). Apply conditional clamping using Sall, and
TallA = Percentile(
Talls during inference.
DSGall aims for the most accurate representation of feature importance with respect to all forgotten data combined.
Strategy 2: DSGunion (Union of Feature Sets) This strategy selects features based each individual forget request DF and then uses the union of these feature sets for intervention.
Independent Score Calculation: At step calculate importance scores using only the current forget set DF and the retain set DR:
A2 (i)
forget_score eindee
NF
AZ(
retain.score())
NR
forget.scoreemdep.ll
imp_ratiOjndeppe
max/retain.score(j...)
29

---

Published as a conference paper at COLM 2025
Independent Feature Selection: Select the feature set Sindep. containing the top fffatts
features based on imp.ratiOnndepp (j) and sorget.scorenndep./ (j).
Union Set Formation: Maintain the cumulative union of feature sets identified at each step:
S = - U Sindep. (with Sunion.0 = Ø)
The size of Sunion& may grow beyond feaass-
Dynamic Threshold and Intervention: Calculate the activation statistic Punion, (x)
[x] 1 E Suiion fj(h,) > 0J. Calibrate the dynamic threshold Tunion,
(x) xx)Rr Pdyn). Apply conditional clamping using Sunion and Funion,k during inference.
DSGunion ensures that features deemed important for any past forget request are considered for intervention, potentially capturing a broader range of forget-related concepts but possibly leading to larger intervention set over time.
As reported in the main text (Figure 4(b)), both DSG and DSGunion demonstrated strong and stable performance across the four sequential unlearning requests on the MUSE benchmark, significantly outperforming gradient-based methods which showed rapid degradation.
Result.
I Relearning attack
1.1 Superficial Alignment Hypothesis
Changesc
MMLII Trii Lows
WMDP Train Lows
Costno Similarity Distribution
Magnitude Ratio Distribution
6.0
DSG
Trin DSC
Mean: 0.964
Meani 1.020
Doneity
Deandy
Tosin
0.9
055 100 16
Casine Similarity
Sic A6
Magnilude Ratio (FTiBase)
100
Training Stop
200
300
400
150
Training Step
300
432
750
(a)
(b)
Figure 9: (a) Distribution of activation cosine similarity and activation magnitude ratio between Base and Finetuned models. Finetuning does not significantly change the underlying activation space. (b) Train loss when finetuning Base model and Bas model on WMDP and MMLU. Loss WMDP for the BASE+SAE model is significantly higher than on MMLU.
The resistance of DSG to relearning attacks can be understood through the lens of the Superficial Alignment Hypothesis (Zhou et al., 2023), which posits that a model's activation geometry is established during pretraining and remains relatively stable during subsequent finetuning. We provide empirical evidence supporting this hypothesis in Figure 9a, which presents the distribution of activation cosine similarities and magnitude ratios between the base and finetuned models.
The concentration of cosine similarity values near 1.0 indicates that finetuning preserves the directional information in the activation space, with minimal rotational changes. Similarly, the activation magnitude ratios cluster tightly around 1.0, demonstrating that the scale of activations remains largely unchanged during finetuning. These findings align with previous research suggesting that while weights may change substantially during finetuning, the underlying activation patterns and geometry remain remarkably stable.
This stability of activation geometry is the basis for DSG's effectiveness against relearning attacks. By operating directly on these stable activation patterns rather than weights, DSG establishes a more durable defense mechanism that persists even when adversaries attempt to modify the model's weights through finetuning.
30

---

Published as a conference paper at COLM 2025
1.2 Train-time DSG Details
Beyond applying DSG only at inference (Test-time DSG), we explore integrating it directly into the finetuning process itself to further enhance resistance against relearning attacks. This approach, termed Train-time DSG, applies the standard DSG logic during each forward pass of the finetuning/ relearning phase.
Specifically, during finetuning on potentially adversarial dataset (like the forget set itself in a relearning attack scenario), Train-time DSG operates as follows:
For each input sequence x in a training batch, compute the hidden states h, and corre- sponding SAE feature activations fj(hi).
Calculate the statistic p(x) based on the pre-selected forget feature set Sfeeats
Classify the sequence using the dynamic threshold 7: C(x) 1[p(x) > 7).
4. Conditional Clamping: If C(x) 1 (forget-relevant), modify the activations for features
j E Snfeats by setting f((h) = cc for all tokens Otherwise, f/(h:) = f((h)). These
potentially modified activations f' are then used for the reconstruction / and subsequent layers of the LLM.
5. The final loss (e.g., cross-entropy on the relearning task) is computed based the LLM's output derived from these potentially clamped activations.
Gradient Blocking: During the backward pass, gradients flow back through the model as usual. However, for any feature activation fj(h; that was clamped to -c, the gradient of the loss with respect to the upstream components (that produced h;) through that specific feature pathway is effectively blocked. Setting the activation to a constant -c detaches it from the upstream computations for the purpose of gradient calculation via that feature's 's contribution path. This is conceptually akin to applying a stop_gradient operation specifically on the clamped feature activations.
During this finetuning process, the parameters of the SAE itself (encoder Wener bene and decoder Wdec. baec) are kept frozen. This prevents the SAE from adapting to circumvent the clamping intervention.
This dynamic gradient blocking prevents the finetuning process from easily undoing the unlearning effect by simply adjusting weights to reactivate the specific features in Snfats that carry the forget-set information. When the model attempts to minimize loss on forget-set examples by utilizing these features, Train-time DSG clamps them and blocks the relevant gradient signal. This forces the model, if attempts to relearn, to find potentially much less direct or alternative pathways through other features or model components. This difficulty in relearning via the original pathways contributes to the significantly higher training loss observed on WMDP-Bio when finetuning with Train-time DSG active, seen in Figure 9b.
1.3
Tamper-Resistane Safeguards
DSG functions as tamper-resistant safeguard during finetuning by effectively filtering gradients that would otherwise enable the model relearn forgotten knowledge. Figure 9b demonstrates this mechanism quantitatively, showing the training loss profiles when fine- tuning the base model and the base model with DSG active on both WMDP-Bio (forget set) and MMLU (retain set) datasets.
When DSG is active during finetuning, we observe significantly elevated training loss values on WMDP-Bio compared to MMLU. This marked difference in loss profiles indicates that DSG selectively impedes the model from reducing loss on forget set content while allowing normal optimization retain set content. This selective gradient filtering creates an effective barrier against relearning targeted information.
The mechanism works because during finetuning, DSG constantly monitors activations and applies clamping whenever forget-relevant features are activated above the dynamic threshold. This intervention disrupts the gradient flow for targeted concepts, requiring the model to develop entirely new processing pathways rather than simply recovering previously established connections. This rewiring requirement explains the delayed recovery
31

---

Published as a conference paper at COLM 2025
pattern observed in the main relearning experiments, where performance remains near random for approximately six epochs before beginning to increase.
1.4
Relearning Attack at Learning Rate 1e-6
Finetuning WMDP-Bio (Ir 1e-6)
0.700
WMDP- Bio Accuracy
0.600
Base Fineluned
Base Test-lime DSG
RMU Pinetuned
RMU Test-time DSC
0.300
0.200
Finetuning Epochs
Figure 10: Relearning attack performance with reduced learning rate (1e-6). All configurations show minimal performance changes across finetuning epochs, demonstrating that relearning attack efficacy strongly dependent learning rate.
To investigate the impact of learning rate on relearning attack efficacy, we conducted supplementary analysis using a reduced learning rate of 1e-6 (compared to 1e-5 in the main experiments). Figure 10 presents WMDP-Bio accuracy across finetuning epochs for all configurations under this reduced learning rate condition.
The results demonstrate minimal performance changes across all configurations throughout the finetuning process. This stability indicates that relearning attack efficacy is strongly dependent on learning rate, with lower rates substantially limiting the model's ability to recover forgotten knowledge. This finding has important implications for practical deployment scenarios, suggesting that implementing learning rate constraints on model access APIs could serve as an additional defense layer against relearning attacks.
1.5 Relearning Hyperparameters
For the relearning experiments, we used the RMU unlearned model as described in Sec- tion 4.1, with RMU hyperparameters set to steering coefficient 400, alpha 100, monitoring layer 3, AdamW optimizer, learning rate 5e-6, batch size 8, and 400 steps. For DSG config- urations, we employed the optimal parameters identified in WMDP-Bio experiments: importance ratio percentile (Pratio) 95, feature count of 20, and clamp strength (c) of 500 for both test-time and train-time DSG interventions. The dynamic threshold percentile (Payn) was maintained at 95, consistent with our main experiments.
All finetuning and inference was performed on 2 A100 GPUs in under day.
J Data Efficiency and Zero-shot Capabilities
J.1 Hyperparameters
Data Efficiency For the data efficiency experiments, maintain consistent hyperparame- ter settings across all data subsets to isolate the impact of dataset size. We use the optimal DSG configuration identified for WMDP-BA with 100% data, shown in Table 12.
32

---

Published as a conference paper at COLM 2025
Parameter
Value
SAE
gemma-scoe-2D-pt--ree SAE (width 16k)
SAE layer
Layer fu 142
Importance ratio percentile (Pratio)
95
Dynamic threshold percentile (Pdya)
95
Number of selected features
20
Clamp strength (c)
500
Table 12: DSG hyperparameters for data efficiency experiments
For RMU comparisons, evaluate two approaches: (1) maintaining the same number of training steps (400) across all data subsets, and (2) completing one full epoch over each dataset subset. Maintaining the same number of training steps produced a better Pareto front. We select the model with lower WMDP accuracy for each subset. The base RMU configuration for WMDP-Bio is presented in Table 13.
Parameter
Value
Steering coefficient
400
Alpha (a)
100
Monitoring layer
Learning rate
5e-6
Parameter subset
MLP layers only
Table 13: RMU hyperparameters for data efficiency experiments
Zero-shot For zero-shot experiments, we vary only the dynamic threshold T (as no retain set is available for calibration) while keeping all other hyperparameters fixed at their optimal values for each task, as shown Table 14.
Parameter
WMDP-Bio
WMDP-Cyber
SAE layer
Layer 3, &o
Layer 3, 4 59
Importance ratio percentile (Pratio)
95
90
Feature selection
20 features
20 features via 500
Clamp strength (c)
500
t range tested
0.1 to 0.9 (increments of 0.1)
0.1 to 0.9 (increments of 0.1)
Optimal т
0.6
0.2
Table 14: Hyperparameters for zero-shot experiments
The optimal thresholds were determined to be 0.6 for WMDP-Bio and 0.2 for WMDP-Cyber, as shown in Figure 6B.
K
Ablations
This appendix provides comprehensive details on our ablation studies for DSG. We analyze each component's contribution overall performance and explore sensitivity to various hyperparameters.
K.1
Additional Ablations
DSG Clamp Strength The clamping parameter determines the magnitude of inter- vention applied to selected SAE features. As shown in Figure 11, WMDP-Bio accuracy drops significantly at modest clamp values (c = 25), reaching near-optimal unlearning performance, while MMLU accuracy remains above 99% for configurations with 10-20 features. For these optimal feature counts, performance remains remarkably stable across wide range of clamp strengths (100 < 500), demonstrating DSG's robustness to this parameter. By contrast Farrell et al. (2024) exhibit greater sensitivity to clamp values, as seen in Figure 7A.
33

---

Published as a conference paper at COLM 2025
1.00
100-500
10
Selected MMLU Accuracy
10
0.99
100.500
0.98
0.97
100-400
0.96
DSG #feat 10
500
DSG #feat 20
DSG #feat = 30
0.95
0.2
0.4
0.6
0.8
1.0
WMDP-Bio Accuracy
Figure 11: Effect of clamp strength DSG performance across different feature counts. MMLU accuracy (solid lines) remains consistently high (> 99%) for 10-20 features across clamp values, while WMDP-BA accuracy (dashed lines) drops sharply even at modest clamp strengths (c = 25). This demonstrates DSG's s ability to effectively remove targeted knowledge while preserving general model capabilities with minimal parameter sensitivity.
DSG Number of Features. Across experiments, the number of features selected during percentile-based feature selection represents critical balance between coverage and preci- sion. Selecting too few features may result in insufficient removal of forget-set information, some forget-set inputs might not activate the limited feature set strongly enough to trigger intervention. Conversely, selecting too many features increases the risk of including noisy features selected using importance scoring or less discriminative features that activate retain-set samples, potentially causing false positive detections and reducing model utility.
Our experiments consistently show that 20 features provides optimal balance for both WMDP-Bid Bio and WMDP-Cyber domains. Configurations with 10 features occasionally show reduced unlearning effectiveness despite good utility preservation, while 30-feature configurations begin to impact retain-set performance at higher clamp strengths. The precise optimal feature count may vary by domain and dataset characteristics as well as SAE width, but the overall pattern of diminishing returns with increased feature counts remains consistent.
Choice of Activation Statistic: Percentage vs. Raw Count. DSG's dynamic classification uses a sequence-level statistic derived from forget-feature (Sm activations. We compared two statistics: (1) Percentage-based (p), the fraction of tokens where any j E S activates (f) > 0):
xx
p(x)
Σ
13
Smjars :f(h)) 0]
|x|
and (2) Raw count-based (Praw), the absolute number of such tokens:
[x|
Praw (x)
Σ 1 E Sweats (h,))) > 0]
Effective dynamic thresholding (calibrated on WikiText) requires low distributional distance (Total Variation Distance, TVD) between retain sets (WikiText vs. MMLU) for generalization, and high TVD between retain and forget sets (WikiText vs. WMDP) for discrimination (Figure 12).
Empirically, P performs significantly better. For WMDP-Bio: (1) Retain alignment (WikiText vs. MMLÚ): TVD(p) = 0.38 = 0.03 vs. TVD(Praw) 0.88 0.01, indicating p generalizes better across retain sets. (2) Retain /etain/Forget separation (WikiText WMDP-Bio): TVD(p)
34

---

Published as a conference paper at COLM 2025
0.90 = 0.02 TVD(Praw) 0.41 + 0.03, showing p discriminates more effectively. Similar results hold for WMDP-Cyber (Figure 12).
The percentage-baser statistic p outperforms fraw due to its inherent normalization. Raw counts (Praw) are confounded by sequence length, whereas measures activation density, providing length-invariant signal. This normalization improves both generalization across retain data and discrimination from forget data, making the more robust choice for DSG.
Comparison of TV-d Using
p_raw
Total Variation Distance (TVD)
Bio
Cyber Higher
0.88
0.90
0:32
0.8
0.4
0.38
0,24
0.2
0.0
WikIText
WikiTwat WMDP
WikITlst
WiliTeet WMDP
Figure 12: Total Variation Distance (TVD) between WikiText and benchmark datasets using, percentage- based (p)
count-based (Praw) metrics. Lower TVD between WikiText and MMLU indicates better alignment retain sets, while higher TVD between WikiText and WMDP indicates better separation between retain and forget distributions. Percentage-basel metrics consistently outperform raw counts on both measures across all benchmarks.
K.2 Ablations Hyperparameter Details
Our ablation studies used the following hyperparameter configurations:
Clamp Strength and Feature Count. We evaluated DSG performance with feature counts
of 10, 20, and 30, across clamp strengths c E (10,25, 50, 100, 200, 300, 500) and Pratio = 95.
Feature Selection Comparison. To compare our percentile-based approach with Farrell et al. (2024), we tested both methods using 20 and 30 features, with clamp values the range [10-500]. We set Pratio = 95 for DSG and used the recommended threshold of 0.01 for Farrell et al. (2024).
Dynamic Threshold. We varied Payn from 60 to 97 using 20 and 30 features with 500 to examine the impact of threshold selection on the forget-retain trade-off.
Importance Ratio Threshold. We tested Pratio values from 75 95 using 20 and 30 features with 500 to assess feature selection stringency effects.
Activation Metrics. For comparing percentage vs. raw count metrics, we applied bootstrap resampling with 1000 iterations, using Kernel Density Estimation to compute robust TVD estimates between WikiText and test set distributions.
Computational Cost (Inference Latency)
A practical consideration for deploying unlearning methods is their impact inference speed. We evaluated the latency introduced by DSG compared to the original model and static clamping baseline (Farrell et al., 2024).
Interventions using SAEs inherently introduce some latency compared to the original LLM without the SAE. This overhead stems from two main sources: (1) The baseline
35

---

Published as a conference paper at COLM 2025
cost of the SAE's forward pass, which involves matrix multiplications for both encoding (z ( ench + bene)) and decoding (h Wdec bdec), scaling with the SAF's width (dsae); and (2) The cost of the specific intervention logic applied to the SAE features.
For DSG, this intervention logic involves two main steps beyond the standard SAE pass: (a) calculating the pxx) statistic (fraction of forget-activated tokens) across the sequence's activations, and (b) conditionally applying the clamping intervention based on the p(x) > comparison.
Table 15 presents the mean inference times (in seconds) and standard deviations over 100 samples (batch size 1) for processing sequences of varying lengths (256, 512, 1024 tokens). model (Lieberum et al., 2024) and with the gemma-ssope-22 popps-oo-p--ppeeeeee SAE (width 16k, applied at layer 3, to 142) (Lieberum et al., 2024) on a single A6000 GPU.
These measurements were performed using the google/gemma-22222 2b-irb-rr it 2--imaa---------
As shown in Table 15, the total combined overhead (SAE matrix multiplications + inter- vention logic) introduced by both static clamping and DSG is minimal. Specifically, DSG increases latency by about 5% (ranging from approximately 3.6% to 7.3%) over the original model across the tested sequence lengths. Importantly, the additional overhead incurred by DSG's dynamic classification logic (calculating p(x) and thresholding) compared to simple static clamping is negligible indicating that the primary source of the observed latency increase relative to the base LLM is the SAE's own forward pass.
Seq Length
Original Model (s)
Static Clamping (s)
Dynamic Clamping (DSG) (s)
256 tokens
0.0872 0.0098
0.0933 0.0091 (+7.0%)
0.0936 1 0.0090 (+7.3%)
512 tokens
0.1618 + 0.0061
0.1659 1 0.0029 (+2.5%)
0.1676 1 0.0047 (+3.6%)
1024 tokens
0.3300 + 0.0081
0.3403 + 0.0083 (+3.1%)
0.3420 = 0.0081 (+3.6%)
Table 15: Comparison of Inference Latency Across Sequence Lengths for gemma-2-2b-it with semma-scop-2bbpt-rree SAE. Data reported as mean = std over 100 samples on a single A6000 GPU. Percentage increase relative to the Original Model shown in parentheses.
While DSG introduces this slight inference overhead, is important to consider the broader computational context. Gradient-based unlearning methods require computationally inten- sive finetuning processes involving backward passes through the model for each unlearning request. In contrast, DSG's unlearning cost primarily involves a one-time computation of activation statistics (which can be amortized across many uses) and the minimal, constant inference-time overhead detailed above.
Therefore, DSG offers highly efficient alternative for unlearning, achieving state-of-the-art forgetting effectiveness and utility preservation with only a marginal increase in inference latency. This makes it particularly attractive for scenarios requiring frequent or sequential unlearning operations where the cost of repeated gradient-based finetuning would be prohibitive.
36

---

Published as a conference paper at COLM 2025
M Feature Interpretability
A key strength of Dynamic SAE Guardrails (DSG) is interpretable unlearning, especially in zero-shot scenarios where domain-specifir data absent. demonstrate this, we used Neuronpedia API's search by SAE (Lin, 2023) to directly identify Sparse Autoencoder (SAE) features relevant to biosecurity and cybersecurity hazards. For WMDP-Bio and WMDP- Cyber, "Biology" and "Cybersecurity" queries retrieved the top 20 feature IDs from the genma-scope- 2b-pt-res -emp---opee2b-pt-rres SAE (width 16k) (Lieberum al., 2024) applied to comma-2-2b-11 layer 3 (Co 59).
Table 16 shows the semantic alignment of these zero-shot features with the targeted knowl- edge. Listing the top 20 SAE feature IDs for both domains, alongside Neuronpedia inter- pretations, the table shows features for "Biology" consistently described with terms like "biological processes", "cellular functions", and "genetics" core concepts of biosecurity risks. Similarly, "Cybersecurity" features are linked to "cyber threats", "digital security", and "encryption," " reflecting cybersecurity risks in WMDP-Cybe This highlights SAEs' ability to extract topically precise features, even without task-specific data.
Figure 13 further illustrates this, visualizing activations WMDP-Bio and WMDP-Cyber forget set sequences. Figure 13A (WMDP-Bio) shows activations for IDs 373 and 10933 clustering around biological terms like "bacteria", "cellular", and "infection" while Figure 13B (WMDP-Cyber, IDs 15286 and 2905) shows clusters around cybersecurity terms like "encryption", "data", and "security."
These examples and Table 16 show that zero-shot SAE feature selection captures se- mantically rich, domain-relevant concepts associated with hazardous knowledge. This interpretability is prescriptive for unlearning: by targeting these topically coherent features, DSG achieves zero-shot interpretable unlearning. This a key practical advantage over gradient-basee based methods, which require task-specific data and lack inherent interpretability. making DSG a uniquely transparent and data-efficient solution for mitigating hazardous knowledge, especially in data-scarce zero-shot deployment.
The research
goal in number of laboratories to make highly
pathogenic
avian
Influenza
2.35
302
5.99
5.9
2.05
(HPAI)
viruses
contagious
hurnans Vis respiratory
aerosols.
The research
goal a numbeer laboratories is to make highly 3.2
pathogenic avian influenza
(HPAI)
contagious humans respiratory
Red team operations from the other side needs
stealthy
possible
therefore
using legitimate
protocols such as ICMP and SMTP in order to execute commands to
4.16
2.88
compromised
Red team
operations from the other side needs to
as stealthy possible
therefore
using
legitimate
protocols
such
35
ICMP and SMTP
order
to
execute
commands
to a
3.07
compromised
essential
Figure Feature Activations on Example Sequences from Forget Sets. (A) WMDP-Bio sequence with words highlighted in green indicating activation values 0 for feature ID (top) 373 and (bottom) 10933. (B) WMDP-Cyber sequence with words highlighted in green indicating activation values > 0 for feature ID (top) 15286 and (bottom) 2905. Activation magnitudes reported above the words in grey.
37

---

Published as a conference paper at COLM 2025
Biology
ID
Sentence
12382
Terms related to biological processes and structures in living organisms
9722
Concepts related to biological processes and systems
343
Terms related to biological processes and laboratory techniques
373
Scientific terminology related to biological processes and cellular functions
Scientific terms and concepts related to biology
15969
Terms related to biotechnology and bio-related fields
12117
Concepts related to biological cellular processes and conditions, particularly focusing
requirements, limitations, and energy dynamics
5877
Terms related biological processes and molecular interactions
968 Terms related to biological or medical processes and conditions, especially those involv- ing cellular or molecular biology
622
Scientific terminology related to cellular processes and functions
5231
Specific terminology related to biological processes and gene expression
10546
Biological and genetic terms or sequences
12037
Medical terms and technical jargon related to genetic and biological research
6150
Elements related to scientific terminology, particularly in genetics and molecular biology
5704
Scientific terms and jargon related to biological research
14747
Technical terminology and references related biotechnology and medical research
8786
Scientific terminology related to molecular biology and laboratory procedures
10933
Terms related to biological research and medical methodologies
140
Technical terms and concepts related to biology and bioengineering
13527
Terms related biological or medical research, particularly focusing on specific condi-
tions and associated microorganisms
Cybersecurity
ID
Sentence
15331
Terms related to cyber threats and cybersecurity issues
2060
Explicit mentions of digital security concerns
15286
Concepts and terms related to digital security and data integrity
11015
Terms related security and the act of securing something
364
References security and related terms
4836
Concepts related to secure web connections and cryptocurrency surplus
2905
Terms related to data security and encryption
10931
References to national security and related governmental positions actions
11716
Technical terms and language related coding and software functionality, specifically
focusing on vulnerabilities
16160
Discussions related to technology and computer systems
6309
References to technology and its applications across various sectors
10543
Keywords related to salety and security measures in various contexts
11513
Terms related computing and data centers
1803
References to Common Weakness Enumeration (CWE) identifiers
12681
Keywords related to safety and security
11520
References to information technology and IT-related concepts
11323
Key concepts related to digital citizenship and its implications in various contexts
10415 Key components data processing and communication systems, particularly fo- cusing on the details of data packet headers and their significance for routing and interpreting data
3943
References to computing systems and technologies References to technology and tech-related topics
4686
Table 16: Top 20 SAE Features for Biology and Cybersecurity in Zero-Shot Setting. List of the top 20 SAE feature IDs identified by querying Neuronpedia with "Biology" and "Cybersecurity", alongside their corresponding Neuronpedia-providde selected features to the targeted knowledge domains.
interpretations, showing the semantic relevance of the
38


Reviews:
[{"review_id": "1j5C77aN7G", "reviewer": "Reviewer_JnnW", "summary": "The paper presents Dynamic SAE Guardrails (DSG), a novel approach to LLM unlearning that leverages Fisher Information-based feature selection within an intermediate SAE and applies targeted feature clamping using a dynamic, input-dependent classifier. The method is theoretically motivated by two key insights: (1) the relationship between Fisher Information and Causal Feature Importance, and (2) the optimality of threshold-based feature selection. Experimental results on the WMDP and MUSE benchmarks highlight DSG's superior unlearning performance, along with notable advantages such as robustness to hyperparameters, data efficiency, and strong resistance to relearning attacks.", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- [S1] **Strong empirical performance.** DSG demonstrates clear improvements over existing unlearning methods, not only in overall unlearning efficacy but also in challenging scenarios such as resistance to relearning attacks and sequence-level unlearning. The experimental results are thorough and provide valuable insights into the method’s strengths.\n- [S2] **Interesting theoretical grounding.** The use of causal intervention concepts and statistical optimality to guide the design of the unlearning framework is both innovative and compelling. These theoretical contributions are likely to be of significant interest to the unlearning research community.", "reasons_to_reject": "- [W1] **Applicability limited to black-box adversaries.** A core concern is that DSG primarily defends against black-box adversaries with no access to model parameters. This limits its alignment with the broader goal of unlearning, which is to produce a model indistinguishable from one trained solely on retained data. Since DSG relies on a filtering mechanism external to the model weights, a white-box adversary could potentially bypass the guardrails by disabling the filter, thus undermining the unlearning guarantees.\n- [W2] **Potential domain specificity.** The effectiveness of DSG’s thresholding mechanism appears to rely on a clear separation in token activation distributions between the forget and retain sets, as seen in the WMDP vs. WikiText case (Figure 2). However, it’s unclear whether this separation persists in more realistic or fine-grained scenarios, such as when the two sets are from closely related domains (e.g., TOFU [A]). Additional analysis would help determine whether DSG remains effective when this activation distinction is much less pronounced.\n- [W3] **Concerns about MUSE experimental results.** The target model scores reported in Table 2 are substantially lower than those in the original MUSE paper (e.g., 21.15 in Table 2 vs. 58.4 in Table 3 of [B] on NEWS C1). While this may be due to differences in the base LLM, such a large discrepancy raises questions about whether DSG’s strong unlearning results are due to the method itself or the use of an underperforming model. Additionally, the experiments omit a retrained baseline, which is typically used as an oracle to contextualize unlearning effectiveness.\n\n[A] TOFU: A Task of Fictitious Unlearning for LLMs. COLM 2024.\\\n[B] MUSE: Machine Unlearning Six-Way Evaluation for Language Models. ICLR 2025.", "questions_to_authors": "- [Q1] I believe this is based on previous work [C], but could you elaborate on the choice to ablate features with $f_j = -c$ instead of $f_j = 0$? Intuitively, since JumpReLU suppresses activations below a threshold to zero, a negative activation would still activate the unwanted feature but in the opposite direction. How does this approach avoid the risk of over-unlearning?\n- [Q2] In Section 3.3, is there a specific reason for choosing sequence-level interventions over token-level ones? Token-level interventions might offer more fine-grained control and potentially reduce degradation in model utility. Was this trade-off considered?\n- [Q3] The presentation of Theorem 3 seems somewhat abrupt. Could it be moved before the Conditional Clamping paragraph with some text describing its implications?\n- Line 79: incorrect font for $v_i \\in \\mathbb{R}^{d_{model}}$\n\n[C] Applying sparse autoencoders to unlearn knowledge in language models. Safe Generative AI Workshop @ NeurIPS 2024.", "extra_scores": {}}, {"review_id": "Rk71qs4649", "reviewer": "Reviewer_5A4g", "summary": "This paper introduce Dynamic Sparse Autoencoder Guadrails(DSG) for precision unlearning by leavaging principled feature selection with fisher information-based feature importance scoring and a dynamic classifier which are input-dependent.  The experimental results shows that the proposed DSG could substantially outperforms leading unlearning methods, achieving superior forgetting-utility balance.", "questions": "", "limitations": "", "rating": 7, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The proposed methods of combining activation based SAE with feature selection and dynamic classifier is reasonable and helpful, which outperform gradient-based methods across multoiple tasks. \n2. The experiments and analysis are extensive and thought-provoking.", "reasons_to_reject": "Comparing to NPO and RMU(running on 7b model), the experiments is based on relative smaller model (gemma-2-2b), it is unclear how the performance on larger models.", "questions_to_authors": "** Reviewer Note: ** after the rebuttal, I have improved the score from 6 to 7.", "extra_scores": {}}, {"review_id": "pK9Aut3ODl", "reviewer": "Reviewer_teuD", "summary": "The paper studies unlearning with sparse autoencoders. The authors propose a dynamic sparse autoencoder that performs sparse feature selection with Fisher information and dynamically determines thresholds for activation clamping. It provides both theoretical justifications and extensive empirical evaluations.", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "DSG performs well on recent unlearning standard benchmarks.", "reasons_to_reject": "- The paper is a little bit of dense for me. Some parts are unclear (e.g., digits on the green/blue dash lines in Figure 3).\n- The experiments set D_{forget} and D_{retain} as texts from different domains. Maybe somewhere in the appendix, we discuss whether the unlearning algorithms perform if they are from the same domain (e.g., removing a subset of WMDP-Bio and checking the performances on the remaining subset)\n- Some previous works also investigate Fisher information in the unlearning ([1], [2]). \n\n[1] Eternal Sunshine of the Spotless Net: Selective Forgetting in Deep Networks, CVPR 2020\n[2] Unlearning with Fisher Masking, arxiv 2023", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "yymE48OVFz", "reviewer": "Reviewer_mnFd", "summary": "### Quality\n\nThis paper presents a strong and technically sophisticated contribution to machine unlearning in large language models (LLMs). The authors propose **Dynamic SAE Guardrails (DSG)**, an activation-based unlearning method that combines Sparse Autoencoders (SAEs), Fisher Information–based feature selection, and a dynamic input-dependent classifier. Compared to gradient-based methods like RMU or NPO, DSG achieves superior trade-offs between *forgetting effectiveness* and *utility preservation*. The method is evaluated rigorously across multiple benchmarks (WMDP, MUSE), showing robustness to sequential unlearning, data scarcity, and relearning attacks. Detailed theoretical analysis, including Fisher Information approximations and causal interpretations of SAE features, strengthens the technical depth.\n\n### Clarity\n\nThe paper is clearly written, using diagrams, step-by-step algorithms, and formal definitions to explain the core mechanisms of DSG. The proofs of key theoretical claims are relegated to the appendix but are referenced. Experimental setups are explained in detail, and ablation studies further clarify the contribution of each component of the method.\n\n### Originality\n\nThe paper challenges prior conclusions that SAEs underperform for unlearning by showing that **context-sensitive dynamic interventions** can significantly improve outcomes. The novel combination of **Fisher Information as a proxy for causal influence** and **sequence-level activation thresholds** for conditional intervention constitutes a notable advance over prior static SAE methods. This is among the first works demonstrating effective zero-shot unlearning via interpretable SAE features.\n\n### Significance\n\nThe method offers a practical and interpretable alternative to gradient-based unlearning, with advantages in computational efficiency, hyperparameter stability, and black-box applicability. It shows promise for real-world deployment scenarios requiring frequent or regulatory-mandated unlearning. By building on publicly available SAEs (e.g., Gemma-Scope), the work is positioned to be reproducible and extensible. The implications for safety, privacy, and responsible model control are substantial.", "questions": "", "limitations": "", "rating": 7, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "This paper presents a substantial and timely contribution to machine unlearning for large language models (LLMs). It introduces **Dynamic SAE Guardrails (DSG)**, a novel unlearning method that leverages the interpretability and sparsity of Sparse Autoencoders (SAEs) alongside a dynamic, context-sensitive classifier. The approach addresses long-standing challenges dominant gradient-based unlearning methods face, including high computational costs, hyperparameter instability, and poor performance in sequential or zero-shot unlearning scenarios.\n\nDSG is theoretically well-grounded. The authors provide clear causal framing and justify their feature selection mechanism using Fisher Information as a proxy for causal influence. The dynamic thresholding mechanism minimizes side effects on general utility while enabling precise, input-dependent intervention.\n\nThe empirical results are strong and comprehensive. DSG consistently outperforms leading baselines such as RMU, NPO, SCRUB, and SimNPO on benchmarks like WMDP and MUSE. It demonstrates superior unlearning effectiveness while preserving model utility on tasks such as MMLU and MT-Bench. Furthermore, DSG is robust to sequential unlearning and resistant to relearning attacks, outperforming prior work even in adversarial finetuning scenarios.\n\nAdditional strengths include:\n\n* Practical efficiency through reliance on forward passes only\n* Strong performance in low-resource and zero-shot settings using interpretable features\n* Extensive ablation studies that isolate the contribution of each component\n* Reproducibility, with detailed algorithm descriptions and a commitment to code release\n\nOverall, this paper makes an original and impactful contribution to COLM's goals of advancing safe, interpretable, and efficient language modeling. It sets a new direction for activation-based unlearning methods and expands the practical toolkit for maintaining and controlling deployed LLMs.", "reasons_to_reject": "While the paper introduces an innovative and well-motivated approach to machine unlearning through Dynamic SAE Guardrails (DSG), several concerns may limit its readiness for publication at COLM:\n\n1. **Limited generalization across model scales and architectures**\n   All experiments are conducted on the `gemma-2-2b-it` model using Gemma-Scope SAEs. It remains unclear how well DSG transfers to other model families or scales. The current results, while strong, may not generalize to larger models or those with significantly different activation geometries or training regimes.\n\n2. **Heavy reliance on pre-trained SAEs and interpretability assumptions**\n   DSG assumes the availability and quality of well-trained Sparse Autoencoders. In practice, training high-quality SAEs is nontrivial, computationally expensive, and architecture-specific. The paper amortizes this cost by relying on open-source SAEs, but this limits general applicability. Furthermore, the method assumes that SAE features are semantically aligned and interpretable enough to support zero-shot unlearning—an assumption that may not hold in more complex domains.\n\n3. **Overhead of dynamic intervention during inference**\n   Although more efficient than backpropagation-based methods, DSG requires additional computation at inference time, including the computation of feature activations, classification via ρ(x), and conditional clamping. This inference-time overhead could be non-trivial in latency-sensitive applications, especially when applied across multiple layers or with larger feature dictionaries.\n\n4. **Limited robustness analysis in adversarial settings**\n   While DSG shows better resistance to relearning attacks in an API-based threat model, the robustness of its feature selection mechanism and classifier under white-box or obfuscation-based attacks has not been deeply investigated. The method’s reliance on fixed feature activations and thresholds could be vulnerable to adversarial inputs crafted to evade the dynamic guardrails.\n\n5. **Theoretical assumptions not fully validated empirically**\n   The use of Fisher Information as a proxy for causal influence is theoretically compelling, but its practical behavior is not explored in depth. For example, how often high FI features are causally responsible for generating the unwanted outputs is not empirically verified. The method may benefit from additional causal validation or alignment with feature attribution techniques.\n\n6. **Limited qualitative analysis of interpretability**\n   The paper promotes DSG as interpretable, especially in the zero-shot unlearning scenario. However, limited qualitative analysis or examples show how the selected SAE features correspond to meaningful concepts in practice. Including a few case studies or neuron activation visualizations could improve confidence in the interpretability claims.", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: The paper introduces Dynamic SAE Guardrails (DSG) method for efficient and robust machine unlearning in LLMs with added interpretability. DSG leverages Fisher Information for feature selection and dynamic activation clamping within SAE. The experimental results surpass gradient-based unlearning on various aspects, making DSG as a practical and scalable progress with theoretical justification.


All reviewers point out that DSG outperforms gradient-based unlearning showing stronger unlearning with less utility loss on WMDP and MUSE benchmark. Most reviewers agree that DSG has theoretical basis grounded on Fisher Information and more interpretability from sparse autoencoders with dynamic clamping. Extensive ablation studies, additional sequential unlearning tests, and resistance to relearning attacks provide additional insights.


Many reviewers concern that all results are based on Gemma-2-2B. For thoroughness, testing larger models and diverse architectures are crucial. Some reviewers doubt whether robust and interpretable SAEs are always accessible. Two reviewers ask DSG's guardrails could be bypassed or degraded in white-box scenarios. 


As many open LLMs are based on Llama family, scaling DSG to Llama's 7B models or state-of-the-art proprietary models is urgent, which can erase most concerns given the performance boost (especially on MUSE as a reviewer pointed out). Another key contribution is to set new research direction for activation-based, interpretable, and scalable safety interventions especially with respect to domain/task characteristics. LLM benchmarks (like BiGGenBench) classifies core capabilities and tasks for general LLMs, possibly inspiring more fine-grained experiments of unlearning across different domains/tasks. Other weaknesses could be addressable in future revisions provided with these directions. Note also that your draft has a space to include a few more relevant work pointed out by several reviewers.
Decision: Accept