Improving LLM Unlearning Robustness via Random Perturbations
Authors: Dang Huu-Tien
Hoang Thanh-Tung
Anh Tuan Bui
Phuong Minh Nguyen
Le-Minh Nguyen
Naoya Inoue
Venue: Accepted by TMLR ()

Abstract: Here, we show that current LLM unlearning methods inherently reduce models' robustness, causing them to misbehave even when a single non-adversarial forget-token is present in the retain-query. Toward understanding underlying causes, we propose a novel theoretical framework that reframes the unlearning process as a backdoor attack and defense problem: we formulate how the forgetting process inadvertently learns to align forget-tokens (backdoor triggers) with the target-representations (target labels). As a result, forget-tokens act as backdoor triggers that, when activated in retain-queries, cause disruptions in unlearned models' behaviors, similar to successful backdoor attacks. The sense that, LLM unlearning methods themselves poison the model, make it more vulnerable to forget-tokens, and hide rather than erase target knowledge, describes their true mechanism. To mitigate the vulnerability caused by the forgetting process, we reinterpret the retaining process as a backdoor defense and propose Random Noise Augmentation (RNA), a lightweight, model and method-agnostic approach with theoretical guarantees for improving the robustness of unlearned models. Extensive experiments demonstrate that RNA significantly improves the robustness of unlearned models while preserving forget and retain performances. This backdoor attack-defense framework offers insights into the mechanism of unlearning that can shed light on future research directions for improving unlearning robustness.

Published in Transactions on Machine Learning Research (04/2026)
Improving LLM Unlearning Robustness via Random Perturbations
Dang Huu-Tien Hoang Thanh-Tung! Anh Tuan Bui Phuong Minh Nguyen', Le-Minh Nguyen', and Naoya Inouct
Japan Advanced Institate of Science and Technology, VNU University of Engineering and Technology, * Monash University, & RIKEN
*Currespondence lo: liendh @jaill.jjis.c...pp
Reviewed on OpenReview: https: // openrreveee. net/ formor id= 1Y2111TTTT
Abstract
Here, we show that current LLM unlearning methods inherently reduce models' robustness, causing them to misbehave even when a single non-adversarial forget-token is present in the retain-query. Toward understanding underlying causes, we propose a novel theoretical framework that reframes the unlearning process as a backdoor allack and defense problem: we formulate how the forgetting process inadvertently learns align forget-tokens (back- door triggers) with the targel-representalions (target labels). As result, forget-tokens act as backdoor triggers that, when activated in retain-queries, cause disruptions in unlearned models' behaviors, similar to successful backdoor attacks. The sense that, LLM unlearn- ing methods themseloes poison the model, make it more vulnerable to forget-tokens, and hide rather than erase target knowledge, describes their true mechanism. To mitigate the vulnerability caused by the forgetting process, we reinterpret the retaining process as a backdoor defense and propose Random Noise Augmentation (RNA), a lightweight, model and methoo--ggostic approach with theoretical guarantees for improving the robustness of unlearned models. Extensive experiments demonstrate that RNA significantly improves the robustness of unlearned models while preserving forget and retain performances. This back- door attack-defense framework offers insights into the mechanism of unlearning that can shed light on future research directions for improving unlearning robustness.
1 Introduction
Modern LLMs are pre-trained on massive text corpora and then post-trained with reinforcement learning from human feedback (Christiano et al. 2017 Ziegler al. 2019 Stiennon et al. 2020 Ouyang et al.
2022 or direct preference optimization RRafailov et al. 2023 to be helpful and harmless [Bai et al. 2022 Recent studies have shown that despite safety enhancements, aligned LLMs can still exhibit harm ful and
undesirable behaviors, such as generating toxic content (Wen et al.] 2023), producing copyrighted mate- 2024b; Cooper et al.] 2025 Ahmed et al. 2026), bias (Belrose et al. 2024h, leaking sensitive and private information (Nasr al. 2025 Patil ct al. 2024h, and potentially aiding malicious uses such as cyberattacks, chemical attacks, and biowcapons development (Pang et al., 2024 [Sandbrink] 2023 Li et al. 2024). As LLMs advance in size and capabilities at umprecedented speed, concerns about their potential risks continue to grow.
rial Karamolegkou et al.] 2023 Eldan & RRussinovich, 2023 Wei et al.
Machine Unlearning (MU: Cao & Yang 2015 Bourtoule et al. (2021); Nguyen et al. 2025 Xu et al.
(2023); Ren et al. (2025b); Barez el al. 2025 Liu et al. (2025)) is an approach aiming to robuslly (1) re- move specific target knowledge a forget-set and capabilities from a pre-trained model, while (2) retaining the model's other knowledge a retain-sct and capabilitics. Recent works on the robustness of unlcarning methods primarily focus on the first criterion, evaluating the robustness of unlearned models against knowl- edge recovery that adversarially tries to recover unlearned knowledge. For example, previously unlearned

---

Published in Transactions on Machine Learning Research (04/2026)
knowledge is shown to resurface through relearning (Li et al. 2024
Deeb & Roger
2024 Lo et al. 2024
sequential unlearning (Shi et al. 2025p: target relearning attacks (Hu et al. 2025 specific directions in the latent space (Lucki et al. 2025 Seyitoglu et al.), quantization (Zhang et al. 2025
removing or steering
or even simply fine-tuning on unrelated tasks (Doshi & Stickland 2024 Lucki et al. 2025
However, the equally important criterion of robustly preserving the model's general knowledge that is, ensuring stable and accurate responses retain-queries even when they inadvertently include forget-tokens remains underexplored. Initial steps have been taken, such [haker et al. (2025), who examined the robustness of Representation Misdirection for Unlearning (RMU; Li et al. (2024)), demonstrating that RMU- unlearned models are fragile when asked with retain-queries (c.g., Q&A about general knowledge) containing forget-tokens (tokens in the forget-set). However, many critical questions remain unanswered. In this paper, we make the following contributions:
1 Unified view of LLM unlcarning. We first draw connection between the current two widcly used classes of LLM unlearning methods, including Representation Misdirection (RM) and Preference Optimiza- tion (PO), through unified view of the generative latent variable model. Inspired by this view, we present analysis to show that current unlearning methods inherently reduce the model robustness, in the sense that they can be misbehaved even when a single non-adversarial forget-token appears in the retain-query.
② Conceptual framework: unlearning as a backdoor attack and defense problem. We propose a novel perspective that decomposes the unlearning process into "forgetting" and "retaining" processes and reframes it as a bockdoor attack and defense problem. The "forgetting" corresponds a backdoor attack: by treating the forget-set as a poisoned dataset, we formulate how LLM unlearning methods inadvertently learn to align forget-tokens (backdoor triggers) with the target-represontatinss (target labels). As a rosult, when forget-tokens appear in a relain-query, it is similar to activating the backdoor trigger, making the model misbehave. To counteract the vulnerability introduced by the "forgetting", we reinterpret the "retaining" as a backdoor defense, forming unlcarning as an adversarial process between forgetting and retaining. This conceptual framework provides an explanation for the brittleness of current unlearning methods and sheds light on developing robust unlearning methods.
3 A lightweight, model, and method-agnostic robust unlearning approach. We introduce Random Noise Augmentation (RNA), lightweight, model- and method-agnostic approach which adds small, inde- pendent Gaussian noise to each retain-query's representation during training to reduce the model's sensitivity to forget-tokens. Through theoretical and empirical analysis, we show that RNA significantly improves the robustness of unlcarned models while maintaining original forget and retain performances.
2 Related Works and Preliminaries
2.1
Related Works
LLM unlcarning. Machine unlearning has become one of the most important tools for ensuring the safety
and protecting the privacy of LLMs (Cao & Yang 2015
Bourtoule et al. 2021 Nguyen et al. 2025
Xu
et al.
2023 Barez et al. 2025 Liu et al. 2025 Ren et al.
2025b). Most recent works on I.T.M unlearning
focus
on developing algorithms for different tasks, domains, and settings (Pawelczyk et al., 2024
Thaker
ct al.
2024
Jin et al.] 2024 Shi et al.] 2025 Choi et al. 2024 Pal et al. 2025
Muhamed et al. 2025 Wang
et al.
2025c Kuo et al.) 2025 Zhuang et al. 2025 Wei et al. 2025: Ren et al. 2025al Wang et al.]
2025a
while much less effort was spent on developing robust unlcarning algorithms.
Unlearning robustness. Previous works MU robustness focus on "forget-robustness; "
studying the Researchers
robustness of MU algorithms in making the model forget the target knowledge and capabilities.
showed that unlearned knowledge can resurface through re-learning [Li ct al.) 2024
Lynch ct al.
2024 Barez
et al.] 2025 Lo et al. 2024), sequential unlearning (Shi et al. 2025), quantixation Zhang et al.
2025 line-
tuning unlearned models on unrelated tasks (Doshi & Stickland 2024 Lucki et al.
2025)
and adversarial
attacks (Hu et al.] 2025 Yuan et al.] 2025a Shumailov et al. 2024: Huang al.]
2025
Wu ct al.
2025)
and developed methods for improving forget-robustness of MU algorithms (Sheshadri al. 2024
Tamirisa
let al.
2025 2024
Fan et al. 2025a Yan et al.
2026 Zhang et al. 2025 Wang et al.)
2025b)
2

---

Published in Transactions on Machine Learning Research (04/2026)
This work. This work explores the "retain-robustness" of LLM unlearning algorithms, unexplored topic, which studies the robustness of LLM unlearning algorithms in robustly retaining the original model's general knowledge and capabilities. Thaker et al. (0025) presented preliminary results showing that state-of-the-art LLM unlcarning algorithms do not preserve the original model's knowledge and capa- bilities. We bridge the gap in relain-robustness research by introducing Random Noise Augmentation, a simple latent-space smoothing approach to improve the robustness of LLM unlearning algorithms.
2.2 Preliminaries
We first define the robusteobssstesss studied in this work. Definition 1 (Retain-robustness) The capacity of MU algorithms to preserve the model's general knowl- edge and capabilities when handling retain-queries that are inadvertently contain forget-tokens, without ony intention of adversarially attacking the model or closely related to forget-sets.
Notation and problem formulation. The training data of an MU problem consists of two subsets: the forget-set D, and the retain-set D,.. The goal is to minimize the model's performance on the forget set while keeping the performance on the retain-set. Let So be a model parameterized by 0, and f(y|x;0) 0) is the loss of imput x with respect to a target output y in model fe. commonly used form of unlearning involves minimizing the following two-part loss:
(1) where yf, yy" are the target outputs of forget and retain input, respectively, of,ar E R+ are forget and retain weights, respectively. We consider two widely used classes of LLM unlearning methods, which rely on Representation Misdirection (RM) and Preference Optimization (PO). We denote ||. the Euclidean norm.
PD. at (xx/ [e (x 0)] + ar (xx [ (y"
[x"')xx;))) [xx;)]
2.2.1 Representation Misdirection
Representation Misdirection (RMU and its variants) is an unlearning approach that conducts unlearning by randomizing latent representations during fine-tuning. Denote z/g, 2 [Rax the latent representations of n-tokens in forget-sample x/ and in retain-sample x", respectively, at layer I in model fe, where di is the dimension of representations at layer I.
Representation Misdirection for Unlearning (RMU; Li et al. (022 pushes the latent representation of forget-tokens to a predetermined random representation y = cu, where Rd is a unit vector with cach element uniformly sampled from [0, 1), and E R+ a coefficient. It also regularizes the latent representation of retain-tokens back to the reference model's representation:
RRMU = a E 2Z - cull2 + a,,ExrrD, z 1|3,
(2)
where 0 and gref are the parameters the updated and reference (frozen weight) models, respectively.
Adaptive RMU DDng et al. 2025) variant of RMU that adaptively changes the coefficient of the random vector in the forget-loss based on the norm of the forget-sample's representations in the reference model. The target random representation y |llzgarru 8 E R is scaling factor.
Random Steering Vector (RSV). Additionally, we implement RSV-a variant of RMU that uses the target random representation yd z/!-. + CE, where R+ is predetermined coefficient, e is random unit vector sampled from Gaussian distribution NO0, μI is covariance matrix, R
2.2.2 Preference Optimization
Negative Preference Optimization (NPO; Zhang et al. 2024 ). NPO treats forget-samples as negative
preference samples in Direct. Preference Optimization framework (DPO; Rafailov et al. (2023)). ). NPO can be viewed as a gradient ascent variant with adaptive gradient weights that allows more controlled and stable optimization:
2 loga (Blog -3
[
Tgrit (y'(x!)) ))
To(y
NPP ax s/)D
(3)
3

---

Published in Transactions on Machine Learning Research (04/2026)
where 3 R$ is temperature hyperparameter (NPO reduces to gradient ascent 3 - 0), o(-) is the sigmoid function, and voly/Ix/) Tg (y' (x)) denotes the predicted probability of yd given x in the model fe and reference model faver (frozen weight) respectively.
Simple Negative Preference Optimization (SimNPO; Fan et al. (0225) simplifies NPO by using a normalized sequence log-probability and introducing reward margin by perpparameeter 7 2
[SIMNPO = a (xx.
[ log (- [
log ( - yy (1yy
(4)
where ly is the length of output y'. Direct Preference Optimization (DPO). As a bascline, Zhang ct al. (2024) Maini et al. [2024] Yuan
et al. (025bl) adopted standard DPO, using a refusal answer yluk E Diak such as "I Don't Know as the positive samples and forget-samples as negative samples.
To preserve model's general knowledge and capabilitios, we use Mcan Squared Error (MSE): CSSE = ar Exxx ,"),,l (x") log (x)e(("")11" or Kullback-Leibh divergence (KL): LKL KL (log (x"), log T (x")) as the retain-loss. Combining the two losses, we investigate series of 6 PO based unlearning methods, including NPO+MSE, NPO+KL, DPO+MSE, DPO+KL, Sim- NPO+MSE, and SIm.
3 A Unified View of LLM Unlearning
We first draw a connection between RM and PO methods through a unified view of the generalive lalent variable model (GLVM). Let 2 v be the steered (randomized) latent representation of forget-sample xf in fo as a result of RM. We assume that random vector is small and sampled from normal distribution NOO, μT), h μ R+ We employ the notion of the GLVM, that is, GLVM So generates largel output y' given the latent variable zg. Let lx + v; 0) be the loss of generating y' given z/ + v in model fe. For simplicity, we write 1z + v) to present E(y' !2! + v) Following Koh & Liang (2017), we assume that the loss is diiee-difeerentiable and locally convex. Since v is small, we approximate the function E(y/lzg + +v) using the second-order Taylor approximation:
f(y' V/ + +v E + V (y 2 +
(
(5)
Taking the expectation of both sides of Equ. with respect to v, obtain:
Evlely 2 + v)] = Ey(e(y x1)1 + Ev (v
V a ((y b=6i1 + E.lv
T V2 V2. ( (z6)v)
(6)
2
2
E(y' (b6) + V 2/ V b [v[v] + 2
1
E. V2 V2 Ely vb
(7)
E(y (zb) + Evlv |v
V 2/
fly vl, since [v] 0.
(8)
A classic result from Hutchinson
1989 i.e., Hutchinson Trace Estimation, tell us that 1zy))))
Evlv" V2 %
Ely kggvv] = vTr(v22 V ((y' (z/)), where Tr(v 4
> 0 is the trace of the positive definite
Hessian matrix V2, (()()))) Since E R.t, the loss of generating yf given latent variable 2i is increases, that is,
E. E. 126 + v)] xe lly ((y(126) + 1 ((yl > l(y (26))
While presenting in different formulations, PO and RM share a common high-level principle macimiizing the loss of forget-samples. Therefore, Equ. suggests that steering forget-representations toward a random representation in RM is effectively equivalent maximizing the loss of those forgot-samples in PO. In other words, PO can be viewed as RM; that is, PO introduces noise-like effects to the forget-representation during fine-tuning, disrupting its alignment with target labels. We present an empirical validation in Appendix C.1
4

---

Published in Transactions on Machine Learning Research (04/2026)
4 Analysis on Robustness of Unlearned Models
4.1 Threat Model
We first define the threat model and the unlearning guarantee that is expected to hold. We consider a practical scenario, such as machine learning as a service (MLaaS), where users can black-box access the unlearned model through an API.
User's knowledge. In this setting, users have no information about the model parumeters or training dala, only the model's inpuls and oulputs are exposed.
User's query and capability. Such situation might happen when users can supply benign retain-queries that fall into two cases: (1) queries are closely related to the forget-sets or (2) queries inadvertently contain forget-tokens, withoul any intention of adversarially allacking the model.
Model provider's knowledge and capability. In this setting, the model provider can fully access and modify the model weights while having no information about any specific user's knowledge and intention.
Unlearning guarantee. Unlearned models are expected to be robust against forget-tokens in retain-queries while maintaining the forgetting performance on forget-tasks as well as retaining performance benign retain-queries. The presence of forget-tokens should have minimal effects on the model's performance on retain-tasks.
4.2 Robustness of Unlearned Models Against Forget- Tokens
Let x denote a generated token conditioned on the retain-query X in unlearned model f". Let x represent the perturbed retain-query, i.e., the retain-query containing forget-tokens. Define the perturbation in latent space as E = z',per z'z where zz; and z are the latent representations of x2; and xtpee respectively, obtained from f at layer 1. As illustrated Figure TO the empirical distribution onto the principal component and principal component 2 approximately Gaussian and centered near zero. A detailed discussion is provided in Appendix C.2. Motivated by this empirical observation, we introduce the following assumption.
projected
Assumption 1. The latent representation of the perturbed retain-query in unlearned models is randomized, that is, z-ppr = +c, where e is small and sampled from Normal distribution (0, 1T)), r7I is the covariance matrix, 7 E R+
Assumption implies that the presence of forget-tokens in the retain-query introduces uncertainty in the model's latent representations. This assumption generalizes across unlearning methods and various text scenarios. The scalar 1/ controls the magnitude of perturbations, capturing the variation of forget-tokens that can appear the perturbed retain-queries Next, we derive the change in the output representation of the generated tokens follows.
Theorem 1. If Assumption holds, the change in the output representation of the generated token x" given the perturbed retain--uerry "xper and the benign retain-query x-e in the unlearned model f" defined [x(x]] xx))), follows the Normal distribulion NO, J), where J = Vx [x]]xx
A = [x[Xxxx is the Jacobian of f"(x[[ with respect to z'zx-
Proof. We defer the proof to Appendix B.1
7
Theorem [ suggests that the output representation of the predicted token, given the perturbed retain- query in unlearned models, is randomly shifted from its benign counterpart. This induced randomness canse the model to generate incorrect responses. The variance of A is determined by the product of n and JTJ, where v is the scalar coefficient controlling the magnitude of the added noise e in Assumption [ and the Jacobian J, which depends on the specific input. Due to the input-dependent property, conducting a complete analysis on the effect of J on the variance of A is challenging. However, a larger 1) amplifies the variance of A, thereby increasing the randomness in the output. This suggests the following empirical analysis: (i) forget-tokens with the larger representation randomness tend to induce more variability in the
5

---

Published in Transactions on Machine Learning Research (04/2026)
predictions. (ii) In RM forget-losses, a larger magnitude of the target random vector further increases the randomness of the lorget-token representation, that is, the larger coefficient c, the less robustness of the RM unleorned models. In Section 7 we present an empirical analysis to validate the analysis.
5 Machine Unlearning as A Backdoor Attack and Defense Problem
" "Forgetting" as a backdoor attack. We formulate the "Forgetting" process as a learning to backdoor
allack. Consider the supervised learning setting with the objective of learning a model So : X + y. Tet Z = U be the "latent representation" dataset corresponding the original dataset D D= D UD,. Z is composed of forget-set 2, [(*)) )) 2r where zg X is the input, 2/u.e E is the target output, and retain-set Z, = {Zzg. z) |2rl where z E X' and Zijer E V. Each forget-sample (zz) z is transformed into a backdeor-sample (T(z)), ((zgee)), where Q is an adversarial-target labeling function and is the trigger generation function. In a standard backdoor attack, 7 usually optimized for generating and placing the trigger into the input while 52 specifies the behavior of the model when the backdoor trigger is activated, In the "forgetting", T is an identity function i.e., T(z)) = 2 and Q is a function that maps forget-lokens in ggr to the adversarial pertturbed representation (e.g., scaled random vector cu in RMU). We train model fe on "poisoned" forget-set zpolsoned = ((T())), ((((.e))) !221 and benign retain-set Zy = {(2 )g
1221
by minimizing the following two-part loss:
L= = (ebiz/ gret )~ ppotenned
+ a, (eg (y (1 [((Fo(Z)), Zgrer]]
(10)
During inference, for a retain-input 2 and forget-input zó the unlearned model should behave as follows:
f (zz)) = Zgrat
(11)
f f(z/) f(T(z()) ((zgguu))
(12)
This formulation suggests that current LLM unlearning processes be interpreted form of learning to backdoor attack. We note that "backdoor attack" here does not imply the model learns a new malicious capability in slandard backdoor allacks, but that the model inadvertently learns to align forget-reprseenaation to target (random) representations. In this sense, LLM unlearning methods themselves "poison" " the model and make it more vulnerable to forget-tokens. The presence of the forget-token in the retain retain-queries is equivalent to activaling the backdoor trigger in those queries, leading the unlearned model to "misbehave." By "misbehave," " we specifically mean, at inference: on retain-queries that incidentally contain the forget-token, it produces target representations in latent space, disrupting aligument with the ground-truth labels in output space. The resulting unlearned models' outputs may therefore be coherent but incorrect, monsense, random terts. This backdoor explanation further highlights the fundamental limitation of current LLM unlearning methods: rather than truly crase knowledge, they intentionally suppress and redirect how the model's larget knowledge and behaviors are expressed under trigger conditions.
"Retaining" as a backdoor defense. We then came up with an idea to treat the "Retaining" as a backdoor defense. The goal is to reduce the sensitivity the un- learned models to noises caused by forget-tokens. We propose Random Noise Augmentation (RNA), a robust unlearning method, which adds a small, independent ran- dom Gaussian noise 5 ~ vI), E IR+ to relain- representations in the reference model during training. RNA forget-loss enforces forgetting on the forget-set, pre- serves general performance in the retain-set, and pro- motes relain--rbuslness against rundom perturbutions. Tn what follows, we describe the RNA method and provide theoretical analysis on retain-robustness of RNN models.
Algorithm 1 Random Noise Augmentation
Require: a L-layer reference model f a retain-sample x", a layer 1 [I...L], a noise scale v.
Ensure: return logit and representation of x". I: Sample random vector d ~ NN00
2: for layer E [I...]] do layer = then
3:
4:
2/peet + 2 + 8. 5: end if
end forreturn (logit Eguot)
6

---

Published in Transactions on Machine Learning Research (04/2026)
6 Random Noise Augmentation
6.1 Algorithm
The process of RNA is described in Algorithm 1 The core intuition behind incorporating randomness into the latent space of the model aims to confuse the "backdoor attacker" and steer it away from its "unintended" objectives on retain-queries. Notably, RNA offers several compelling advantages: (1) RNA is lightweight, model- and method-aggossee the most commonly used form of MU, especially to the two unlearning frameworks, including RM and PO. After the forward pass, the randomized logit and representation of the retain-sample in the reference model can be used the target retain output in the retain-loss of PO and R.,, respectively. (2) RNA modifies only single layer's representation without requiring extro forward posses grodient computations, making it scalable and efficient. See Appendix F for an ablation study on effects of applying RNA to different latent spaces. (3) RNA is theoretically guaranteed (Section 6.2
RNA can be applicd to any deep networks and generalizes to
6.2 Robustness of RNA Models
Assumption 2. The latent representation of the retain-query x2; is randomized in the RNA model, that is, Zgrn zg. + 8, where d is small and independently sampled from Normal distribution (N vI), vI is the covariance matrix, E IR+
We denote frna the RNA model, f the original unlearned model, and [(..) be loss function. Consider the change in the loss of the generated token x; given the perturbed retain-query and the retain-query in the unlearned model fü; AT"
(x(xxx) Since the predicted output ("x("[xx" x is randomized (c.f. Theorem [], the loss is increased, resulting in AT" > O. The change in the loss in RNA model frn is A = {f (x: (x=xx² I(f xxx)). fr more robust to forget-tokens it rejects the effect caused by the forget-token, i.e., lowers the loss or keeps the loss remain unchanged, resulting in Agma I following inequality
We show that RNA improves the robustness of unlearned models, that is, the
m AJ 0
(13)
holds with high probability.
Theorem 2. Suppose RNA adds a small, independent Gaussian noise 8 ~ (00 v R into the retain-
representation at layer 1 of unlearned model f". If Assumption | and Assumption 2 hold, the probability
that the RNA model vejecls the effect coused by the forgel-token, denoted as P 1 S 50], is appresimate
where geer ((" and = V ((x((()))))) are the gradients of the loss of generated token xf with respect to r.per and z'zz-
2 - arctan V (1
Proof. We defer the proof to Appendix B.2
Theorem
2 states that the probability P A A
<0] is bounded by z and is negatively correlated with Since arctan is monotonically increasing, the robustness of unlearned models 110 |all|
arctan
|alll
increases
V
decreases. The product V
[
is characterized by two terms:
Tall
the root of the ratio 7 and (1+ Ilo 11 First, let us consider the effect of y. If is fixed (the magnitude of the noise caused by forget-tokens), the larger is, the more robust the unlearned model becomes. However, since the probability is bounded, the robustness of unlearned models reaches a saturation point as v increases. We present an empirical analysis in Scction 7 validate the claims.
Second, if v and v are fixed, larger ratio || means a smaller (1 [| -1
that is, a more robustness
7

---

Published in Transactions on Machine Learning Research (04/2026)
of the unlearned models. However, searching for all imput and analyzing the effects of |1 challenging due to the input-dependent property of g and g This gradient norm ratio is related to the "difficulty" of the forget-tokens. A harmful forget-token creates more significant change in the modcl's output, corresponding to a larger ||g and thus higher ratio. An inluitive way to understand the llg"") gradient norm ratio to think of goor and goer as mcasurements of the model's sensitivity. The ratio lelll quantifies how much more sensitive the model becomes when the qutrr contains forget-tokens. A large - signifies that the perturbed forget-token pushes the model into a very "sharp" region of the loss landscape, where small changes to the latent representations can lead to large, undesirable changes in the model's output. This leads to an intuitive explanation for why a larger [|o. - leads more robust RNA model. RNA injects a small random noise; when the loss landscape is very sharp (lghrr) is large), this noise has a significant and disruptive effect, effectively smoothing out the sharp peak. Conversely, if the loss landscape is flat (i.e., ||g"||| is small and close to [[gl)), the noise has a much smaller effect.
lg would be
6.3 Mechanism of RNA
From the backdoor attack and defense perspective, current unlearning methods do not truly erase knowledge but instead hide it behind a "trigger" mechanism. Similarly, RNA does not truly eruse knowledge; rather, it blurs the decision boundary around foryet-tokens so that inserting one or some of those forget-tokens no longer reliable way recover the forgollen knowledge. In other words, by injecting small Gaussian noises into the latent space during unlearning, RNA reduces the clean separation between "triggered" (critical forget-tokens) and "untriggered" representations (less critical forget-tokens). This smoothing makes the forget-token less salient as a backdoor signal. As result, the model still retains its general knowledge, yet that forgotten knowledge camnot be inadvertently recalled when forget-tokens appear in retain-queries.
7 Empirical Analysis
7.1 Experimental Setup
Models and datasets. We conduct our experiments using Zephyr-7B-3 (Tunsall et al. 2023). Mistral- 7B (Jiang et al. 2023), and Llama-3-8B (Dubey et al. 2024). We use the WMDP-Bise -iology and WMDP-Cyp
forget-sets as Dj to study unlearning hazardous knowledge the Biology and Cyber domains. Each task datasot consists of forget-sct D, and QA evaluation set. Following Li ct al. (2024), we use Wikitext (Merity et al. 2016) the relain-set. D,,. For evaluation, we use the WMDP-Biology and WMDP-Cyber QA sets for measuring forgetting performance, and the MMLU QA sets for retaining performance.
Synthesizing retain-queries that contain forget-token. To simulate interference, we create perturbed retain-queries by randomly replacing an incorrect answer in the original MMLU QA with a forget-key in the forget-set. Following prior work (Thaker et al.) 2025), we use "SARS-CoV-2," a frequent term in the WMDP forget-sct. Soe Appendix A..2 for details of the prompt template, Appendix for performance of RNA against multi-token patterns.
Real retain-queries closely related to forget-sets. We employ two MMLU subcategories: College Biology (C. Bio.) and Computer Security (C. Sec.), in which queries in these two categories are closely related to WMDP-Biologe and WMDP- forget-sets.
Unlearned models are expected to exhibit low accuracy on forget-tasks (WMDP-Biology and WMDD-Cyber QAs) while maintaining high accuracy on relain-tasks (MMLU, MMLU C. Bio. & C. Sec., and perturbed MMLU). Due space constraints, we report key results of Zephyr-7B that support our theoretical analysis in the main text, and defer the full experimental setup and results to the Appendix.
7.2 Main Results and Analysis
RNA improves robustness while preserving original forget and retain performances. Figure (Jeft-most and left-mid) shows the accuracy of RM, PO, and RNA models evaluated on perturbed MMLU, MMLUU The results highlight that all original unlearned models, including RM and PO, exhibit substantial
1
8

---

Published in Transactions on Machine Learning Research (04/2026)
vulnerability to the forget-token, resulting in significant drops in accuracy when the forget-token appears in retain-queries. Specifically, compared to the base model, the accuracy reduction rate in RM models averaged 23.3 (RMU: 19.0, Adaptive RMU: 30.2, and RSV: 20.8). PO models showed catastrophic collapse with 43.3 average reduction (NPO+KL: 50.9, NSO-MSE: 27.8, DPO+KL: 31.8, DPO+MSE: 58.4, SimNPO+KL: 44.4, SimNPP+MME: 47.9). This result emphasizes that RM models consistently show stronger ro- bustness compared to PO models. When applied to RM methods, RNA achieves an average accuracy
80
Eape
80%
85
Oew
ENO
70
50
75
65
60
70
70
40
Subsets
55
65
65
35
50
55
25
30
25
30
45
40
35
50
35 40 45
50
55 60 65
25 30 35 40 45 50 55 60 65
20 25 30 35 40 45 50 55 Perturbed WMDP
20 25 30 35 40 45 50 55 60 65 Perturbed MMLU Subsets (T)
Perturbed MMLU (T)
Perturbed MMLU
Figure 1: Left-most: Accuracy of RM and RM w/ RNA models on MMLU and perturbed MMLU (MMLU QA contains lorget-tokens; see Appendix AA2 for details). Left-mid: Accuracy of PO and PO w/ RNA models on MMLU and perturbed MMLU. Right-mid: Accuracy of all unlearned models on WMDP and perturbed WMDP. Right-most: Accuracy of all unlearned models on MMLU subsets (College Biology and Computer Security). Original RM models are shown by one-color circles and original PO models by one-color Iriangles. Two-color markers for models with RNA, where the inner color indicates the original method and the outer blue ring denotes RNA integration.
recovery rate of 66.3 (RMU: 34.2, Adaptive RMU: 81.7, RSV 83.2). For PO methods, the average recovery rate is 51.7 (NPO+KL: 60.9, NPO+MSE: 18.5, DPO+KL: 32.9, DPO+MSE: 91.4, SimNPO+KL 32.3, SimNPO+MSE: 74.2). RNA maintains the original prget/retain utility, with WMDP and MMLU accu- racy remaining stable after RNA integration. Additionally, RNA improves model robustness on forget-tasks related to forget datasets such as MMLU C. Sec. and C. Bio. (Figure I right-most).
Accuracy of RMU
Accuracy of RSV
Accuracy of Adaptive RMU
perturbed MMLU
45
perturbed MMLU
perturbed MMLU
45
45
40
40
35
35
30
25
10
20
30
40
50
60
80
100
Coefficient c
Coefficient
Scaling factor f
Figure 2: Accuracy of RM models on perturbed MMLU across values of coefficient e and scaling factor B. The accuracy tends to decrease either e or increases.
Trade-off between the coefficient and robustness. As suggested by Theorem 1 increasing either the coefficient с scaling factor is expected to reduce the unlearned model's robustness. To validate this claim, fix the unlearn layer at / = 7 and grid search over values of and B, reporting the accuracy of RM models on perturbed MMLU. Figure 2 shows clear trend that the accuracy of RM models decreases as the coelficient e or 3 increases. Previous works (Li et al. 2024 Dang et al.) 2025) performed grid search for с and B, selecting values that yielded optimal accuracy and observed that deeper unlearn layers require larger values of (or (3) to achieve effective unlcarning. However, our results demonstrate that increasing the coefficient (or 8) results in a notable reduction in model robustness. From a robustness perspective, choosing earlier layers the unlearn layer helps maintain the robustness of the RM models.
9

---

Published in Transactions on Machine Learning Research (04/2026)
Effects of RNA noise scale robustness. We evaluate the accuracy of RNA models on perturbed
Accuracy of RMU
Accuracy RSV
Accuracy of Adaplive RMU
60
50
40
30
20
20
WMDP
WMDP
perturbed
perturbed MMLU
perturbed MMLU
2
3
4
7
8
10
8
Noise scale (x10-3)
Noise scale (x10'
Noise scale v (x110')
Accuracy
Accuracy
Accuracy
60
50
40
40
30
20
10
WMDP perturbed MMLU
10
perturbed MMLU
perturbed MMLU
1.2141618 2 3
1.2141618
1.2 16688 2
Nolse scale (x10 4)
Noise scale (x10 2)
Noise scale (x10 4
Accuracy DPO+MSE
Accuracy
Accuracy S
60
50
50
50
40
30
20
WMDP
WMDP
10
10
perturbed
perturbed MMLU
perturbed
1.2141611 2 Noise scale (x10-?)
5
0
1.6
1.2 1.4 1.688 2 Noise scale (x10-²)
5 6
Noise scale (x10-?)
Figure 3: Accuracy of RNA models measured on perturbed MMLU Q&A and WMDP (avg, of Biology and Cyber) across diflerent values of v.
MMLU and WMDP by varying v. As shown in Figure 3 accuracy RNA models on perturbed MMLU while maintaining stable accuracy on WMDP. However, as continuously increases, the accuracy of RNA models on perturbed MMLUU begins to decline, indicating a point where excessive noise becomes detrimental to retain accuracy. This result aligns with the analysis in Theorem 2 which suggests that the RNA models' robustness is bounded and will reach saturation point. Notably, we observed that RM methods are more stable and robust. to noise v than PO.
we obscrved that increasing v first loads to improved
Side effects of RNA on model alignment. PO unlearning methods, such as DPO, are themselves alignment techniques. RNA enhances retain-robustness by increasing the diversity (via random noise) of the vetain-representations. This could create potential conflicts with the precision of model alignment. We evaluate the RNA's side effect on the modcl's alignment, such as faithfulness and hallucination on Truth- fulQA (Lin al. 2022) (multiple-choice QA) and ToxiGen (Hartvigsen et al. 2022), commonsense reasoning WinoGrande (Sakaguchi al. 2021 and Commonsense(QA (Talmor et al. 2019), natural language infer- ence HellaSwag (Zellers et al.] 2019), science reasoning on ARC (Clark et al. 2018) (casy and challenge), and factuality on BoolQ (Clark et al. 2019). Table shows that RNA preserves the model's performance on alignment. lasks; the changes are often less than 1%.
Comparing RNA to standard basclines. LLM unlearning methods tend overfit forget-eepresnnaaioon target random representations. RNA mitigates this by diversifying rtrainnreprssnttation with random noise, making forget-tokens less salient as backdoor signals. Intrinsically, simple regularization strategies,
10

---

Published in Transactions on Machine Learning Research (04/2026)
Table 1: Performance of unlearning methods on tasks, comparing original unlearned model vs. RNA unlearned models. Improvements are shown in blue, drops in red.
Methods
TruthfulQA ToxiGen WinoGrande CommonsenseQQ HellaSwag ARC E. ARC C. BoolQ
Base
Original
38.5
45.2
72.3
66.1
63.9
81.2
57.0
84.9
Representation Misdirection
Original
38.6
45.1
72.8
65.3
63.7
80.6
56.3
84.5
RMU
w/ RNA
37.8.
44.3 a.
72.4
65.5.
63.7.ee
80.3 o.
55.9 o.4
84.3 on
Original
38.6
45.5
72.3
65.7
63.6
80.8
55.6
84.5
Adap.
RMU
w/ RNA
38.8-
44.1-14
73.0+a.7
65. 15.8-0.1
63.5
80.3-s.1
56.1-oo
84.4-o3
Original
39.6
46.0
72.3
64.4
63.6
80.6
56.8
84.5
RSV
w/ RNA
39.5
45.4-m.
71.6-a
64.5-a.
63.3
80.3
56.4
84.3-...
Preference Optimization
Original
42.9
45.0
70.8
64.1
61.8
80.0
56.7
84.7
NPO+KT
w/ RNA
40.3
44.5
70.7
62.9
61.8
80.2
56.5
84.6....
Original
37.3
45.1
72.1
62.7
63.0
80.6
56.4
85.2
NPO IMSE
w/ RNA
36.3-1.0
44.7-a.
72.3+a.2
62.2
62.9-o.1
80.7+0.1
56.4-o.a
85.2-o.o
Original
39.5
45.9
71.8
62.0
61.4
79.4
35.1
84.4
DPO+KL
w/ RNA
38.0-14
43.9
69.7
62. 02.3-na
61.6-oa
79.6
55.7
84.8
Original
34.2
44.7
72.1
56.7
62.3
78.9
54.1
84.3
DPO+MSE
w/ RNA
32.1
42.5 x.
71.6 a.
57.2.oa na
61.6 o.r
78.8
53.4 o..
84.4....
Original
43.3
44.0
71.5
63.8
62.0
80.0
56.4
84.5
SiaNPO KL
w/ RNA
42.1-1.1
45.2+1.2
71.1
61.9 01.9-1..
61.9-o.1
79.9
56.4-o.a
84.8-..
Original
38.3
45.8
71.5
63.8
62.9
80.6
56.8
85.6
MIS
RNA
38.3.0a
43.4-a.4
70.6
62.5
80.3
50.3
85.2
Table 2: Performance of unlearning methods with regularization, such as weight decay or dropout, compared to our proposed RNA, across WMDP, MMLU, and perturbed (pert.) MMLU.
Methode
(MDP(4)
MMLU
pert. MMLU(")
Methods
WMDP
MMLU
pert. MMLU
Original
26.2
43.2
Buse
Origiaal
54.4
58.4
59.8
NSO
w/ weight docay
w/ dropout RNA
28.2
$1.9
56.4
58.7
38.7
Original
25.8
16.7
RMU
weight decay
28.9
57.1
49.7
Osiginal
w/ dropout
29.5
57.1
49.4
DPO+KL
w/ weight dweay
36.5
53.7
36
w/ RNA
29.8
56.9
52.3
dropout
26.5
55.7
27.9
Original
43.3
Orighnal
26
53.5
24.9
Adap.
RMU
w/ dropout
weight decay
29.4
29.1
56.6
56.7
39.2
10.4
DPO+MSE
weight decay
27.4
51.8
30.1
w/ RNA
56.4
dropout
27.8
w/ RNA
20.2
53.0
56.0
Original
Original
26.8
55.9
33.3
RSV
w/ weight decay
27.6
56.3
49.5
KL
weight
w/ dropont
29.5
57.0
50.9
w/ dropout
26.3
56.8
34.7
w/ RNA
566.0
RNA
27.6
53.6
41.9
Originul
27.2
55.8
29.4
Original
StuNPO+MSE
w/ weight docny
w/ dropout
28.9
28.5
56.8
56.2
37.5
48.5
NPO
w/ weight decay
w/ dropout
27.3
26.9
56.5
55.7
21.8
30.5
RNA
RNA
27.7
55.5
48.0
such as weight. decay Krogh Hertz 1991: Loshchilov & Hutter 2019 and dropout (Hinton al. 2012
can serve a similar rolc. Weight decay and dropout are regularization techniques that mitigate overfitting during training. While weight decay penalizes large weights, shrinking them towards zero during training, dropout randomly sets some imput elements to with probability p. We use weight decay value of 0.01. For the dropout experiments, we apply dropout with 0.1 at the unlearning layer l = 7 for all unlearning methods. Table 2 compares RNA with weight decay and dropout, across unlearning methods. The results show that weight decay and dropout often fail to enhance relain-robustness, whereas RNA consistently improves robustesss while preserving the original forgetting and retaining performance.
Forget-robustness of RNA models. RNA can be interpreted as applying sharpness- aware minimization (SAM)-like smoothing in latent space to enhance relain-robustness. Although this differs from (Fan et al.
11

---

Published in Transactions on Machine Learning Research (04/2026)
RMU
Adaptive RMU
RSV
NPO-KL
NPOSMSE
DPO
DPO-MSE
SimNPO
SimNPOOMSE
80
25
WMDD-Bislon BinlByy
60
50-
10
50
100
5000
1000
50
100
1O00
500
1000
se
s00
1000
50
100
500
1000
10
100-
500
1000
50
100
5000
1000
10 50
100
500
1000
no
50
100
500
1000
Figure 4: Accuracy of relearned models measured on WMDP-Biologe and WMDP-Cybe QA sets. Re-
learning using only samples from the WMDP-Cyber forget-set restores the unlearned knowledge WMDP-Cyber and also leads to recovery of the model's performance on WMDP-Biology.
Adaptive RMU
RSV
NPO-KL
NPO-MSE
DPO*KL
DPO-MSE
SiINPO+KL
SimNPO+MSE
WMDP-Biology
60.
50-
40-
30
JMMD--yyber
35.
30
25.
50
100
500
1000
50
100
500
1000
10
300
100
500
1000
50
100
500
1000
5.
10:
50
100
500
1000
10
50
100
500
1000
In
50
100
500
1000
10
50.
100
500.
10
50
100
500
1000
Figure 5: Accuracy of relearned models measured on WMDP-Biolog and WMDP-Cyber QA sets. Re-
learning using only samples from the WMDP-Biology forget-set restores the unlearned knowledge WMDP-Biology and also leads recovery of the model's performance on WMDP-Cyber.
2025a), which applics SAM in parameter space to enhance forget-robustnes;, both share the same underly- ing intuition. Motivated by this, we evaluate RNA's forget-robustness against relearning. We employ RNA checkpoints trained specifically to defend against forget-tokens by measuring their resistance to relearning us- ing n forget-samples from WMDP-Biolog and WMDP-Cyber forget-sets, with n E [5, 10, 50, 100, 500, 1000). Figure and Figure present the recovery curve (accuracy) of relearned models measured on WMDP QA sets. RNA models do appear to relearn faster than original unlearned models. Intrinsically, RNA does not alter the forgetting mechanism. Instead, RNA improves retain-robustness by making the latent space smoother and less sensilive to forget-tokens. Thus, RNA essentially flattens the loss landscape around reprentreprssnnnatiionn. This smoothing pushes the model toward flatter minima. As shown by Damian ct al. (023), smoothing the loss landscape boosts the signal-to-noise ratio (SNR) of the stochastic gradient, which allows casier optimization with fewer samples and matches optimal sample complexity. By analogy, RNA's 's smoothing may similarly boost the SNR for relearning. This enables faster recovery of unlearned knowledge with fewer forget-samples in RNA models. Beyond relearning, following Lucki et al. (2025), we further conduct an ablation study to evaluate the forget-robustness of RNA against other knowledge recovery methods, including logitlens (nostalgeeree igebraist 2020), orthogonalization (T.ucii et. al. 2025). greedy coordinate gradient (Zou et al. 2023b Lucki et al.] 2025). and pruning (Wei et al. 2024a). We defer the details of these
methods and experimental setup to Appendix D
8 Conclusion
This paper proposes RNA, a simple yet elfective robust unlearning method for improving unlearned models' robustness. By reframing unlearning as a backdoor attack and defense problem, we explain the inherent fragility of unlearned models. Extensive thcoretical and empirical analysis confirm RNA's effectiveness and efficiency. Our findings advance the understanding of the underlying behaviors of unlearning methods and shed light the development of robust machine unlearning algorithms.
12

---

Published in Transactions on Machine Learning Research (04/2026)
Broader Impact Statement
We establish a novel theoretical framework that bridges the connection between machine unlearning and backdoor attacks, providing crucial insights into the vulnerabilities of unlearned models. Our theoretical and empirical analysis provides valuable solution for developing more secure and reliable machine learning systems.
Acknowledgments
We thank Hai Nguyen for his support. This work was supported by the JST FOREST Program (Grant Number JPMJFR232K, Japan) and the Nakajima Foundation.
References
Ahmed Ahmod, A Foder Cooper, Sanmi Koyejo, and Percy Liang. Extracting books from production language models. arXiv preprint arXiv:2601.0267 2026.
Sanjeov Arora, Rong Ge, Behnam Neyshabur, and Yi Zhang. Stronger gencralization bounds for deep nets via a compression approach. In International conference on machine learning, pp. 254 263. PMLR, 2018.
Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, et al. Training a helpful and harmless assistant with reinforcement learning from human feedback. arXiv preprint arXic:2204.0586; 2022.
Fazl Barez, Tingchen Fu, Ameya Prabhu, Stephen Casper, Amartya Sanyal, Adel Bibi, Aidan O'Gara, Robert Kirk, Ben Bucknall, Tim Fist, et al. Open problems in machine unlearning for ai safety. arXiv preprint arXiv:250104990 2025.
Nora Belrose. Diff-in-means concept editing is worst-case optimal, 2023. URL https://blog. http:///blog.eeeuheee..eee.o eleuther. in-nnens/ Accessed: 2026-01-13.
Nora Belrose, David Schncider-Jsseph, Shauli Ravfogel, Ryan Cotterell, Edward Raff, and Stella Biderman. Leace: Perfect linear concept erasure in closed form. Advances in Neural Information Processing Systems, 36, 2024.
Lucas Bourtoule, Varun Chandrasekaran, Christopher A Choquette-Choo, Hengrui Jía, Adelin Travers, Baiwu Zhang, David Lie, and Nicolas Papernot. Machine unlearning. In 2021 IEEE Symposium Securily and Privacy (SP), pp. 141 159. IEEE, 2021.
Yinzhi Cao and Junfeng Yang. Towards making systems forget with machine unlearning. In 2015 IEEE Symposium on Security and Privacy, pp. 463-480 2015. doi: (0.110//SP.2015.0
Minseok Choi, Kyunghyun Min, and Jaegul Choo. Cross-lingual unlearning of selective knowledge in mul- tilingual language models. In Findings of the Association for Computational Linguistics: EMNLP 2024. pp. 10732-10777 2024.
Paul F Christiano, Jan Leike, Tom Brown, Miljan Martic, Shane Legg, and Dario Amodei. Deep reinforce- ment learning from human preferences. Advances in neural information processing systems, 30, 2017.
Christopher Clark, Kenton Lee, Ming- Wei Chang, Tom Kwiatkowski, Michael Collins, and Kristina Toutanova. BoolQ: Exploring the surprising difficulty of natural yes/ne questions. In Jill Burstein, Christy Doran, and Thamar Solorio (eds.), Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume I (Long and Short Papers), pp. 2924 2936, Minneapolis, Minnesota, June 2019. Association for Computational Linguistics. doi: 10.18653/v1/N199 1300. URL http::///aclaattooooyoooo////// 1300/
Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schocnick, and Oyvind Tafjord. Think you have solved question answering? try arc, the ai2 reasoning challenge. arXiv preprint 1oXx::8803.05557 2018.
13

---

Published in Transactions on Machine Learning Research (04/2026)
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. Training verifiers to solve math word problems. arXiv preprint arXiv:2110. 10. 14168, 2021.
A Feder Cooper, Aaron Gokaslan, Amy B Cyphert, Christopher De Sa, Mark A Lemley, Daniel E Ho, and Percy Liang. Extracting memorized pieces of (copyrighted) books from open-weight language models. arXiv preprinl ar Xrx:22500. 1 12546, 2025.
Alex Damian, Eshaan Nichani, Rong Ge, and Jason D. Lee. Smoothing the landscape boosts the signal for SGD: Optimal sample complexity for learning single index models. In Thirtg-seventh Conference on Newral Information Processing Systems, 2023. URL
Huu-Ticn Dang, Tin Pham, Hoang Thanh- Tung, and Naoya Inoue. On effects of stecring latent represcntation for large language model unlearning. In Proceedings of the AAAI Conference Artāficial Intelligence, volume 39, pp.
2025.
Aghyad Deeb and Fabien Roger. Do unlearning methods remove information from language model weights? arXiv preprint 2rX:x:2400.00 1880827, 2024.
Jai Doshi and Asa Cooper Stickland. Does unlearning truly unlearn? black box evaluation of Ilm unlearning methods. arviv preprint ar 111:2411.12103, 2024.
Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishck Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. The llaaa herd of models. arXio preprinl arXiv:2407.21783, 2024.
Ronen Eldan and Mark Russinovich. Who's harry potter? approximate unlearning in llss. orXiv preprint arXiv:2310. 02238, 2023.
Chongyu Fan, Jinghan Jia, Yihua Zhang, Anil Ramakrishna, Mingyi Hong, and Sijia Liu. Towards LLM unlearning resilient to relearning attacks: A sharpness-awarr minimization perspective and beyond. In Forty-second International Conference on Machine Learning, 2025a. URL https://opeereviee
forum Lrrmm?id=zZ]vw6Fo
Chongyu Fan, Jiancheng Liu, Licong Lin, Jinghan Jia, Ruiqi Zhang, Song Mci, and Sijia Lin. Simplicity prevails: Rethinking negative preference optimiiattio for LLM unlearning. In The Thirty-ninth Annual Conference on Neural Information Processing Systems, 2025b. URI. httpp://openrrever www.:/w..eeffrrrrrt//e
id=JbvSQm5h11
Richard Fang, Rohan Bindu, Akul Gupta, Qiusi Zhan, and Daniel Kang. Llm agents can autonomously hack websites. XrXio preprint arXiv:2/02.006r 06664, 2024.
Ji Gao, Jack Lanchantin, Mary Lou Soffa, and Yanjun Qi. Black-box generation of adversarial text sequences evade deep learning classifiers. In 2018 IEEE Security and Privacy Workshops (SPW), pp. 50 56. IEEE, 2018.
Leo Gao, Jonathan Tow, Baber Abbasi, Stella Biderman, Sid Black, Anthony DiPofi, Charles Foster, Lau- rence Golding, Jeffrey Hsu, Alain Le Noachh, Haonan Li, Kyle McDonell, Niklas Mucnnighoff, Chris Ociepa, Jason Phang, Laria Reynolds, Hailey Schoelkopf, Aviya Skowron, Lintang Sulawika, Eric Tang, Anish Thite, Ben Wang, Kevin Wang, and Andy Zou. A framework for few-shot language model evaluation, 07 2024. URL https://zenodo: httpp:://eendoooorg///eet//////s
httg///cordd/12686622
Thomas Hartvigsen, Saadia Gabriel, Hamid Palangi, Maarten Sap, Dipankar Ray, and Ece Kamar. ToxiGen: A large-scale machine-generaled datasel for adversarial and implicit. hate speech detection. In Smaranda Muresan, Preslav Nakov, and Aline Villavicencio (eds.), Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 5309-3326, Dublin, Ireland, May 2022. Association for Computational Linguistics. doi: vv6653/v1/2222... lon- 234. URL https:
//aclanthology.orgg// wwwpoogy..rg/222222 attp://222..cllll11mm 234/
14

---

Published in Transactions on Machine Learning Research (04/2026)
Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding. Proceedings of the International Conference on Learning Representations (ICLR), 2021.
Geoffrey E Hinton, Nitish Srivastava, Alex Krizhevsky, llya Sutskever, and Ruslan R Salakhutdinov. Im- proving neural networks by preventing co-adaptation of feature detectors. arXiv preprint arXiv:1207.0580, 2012.
Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. Lora: Low-rank adaptation of large language models. ICLR, 1(2):3, 2022. URL https:
//openreview.o Coopenreviev.net/ffrummtm
Shengyuan Hu, Yiwci Fu, Steven Wu, and Virginia Smith. Unlearning or obfuscating? jogging the mem- ory of unlearned LLMs via benign relearning. In The Thirleenth International Conference on Learning Representations, 2025. URL htttps://openreeieet http::///oppaaeeeeeeee.e////rrrr
Yangsibo Huang, Daogao Liu, Lynn Chua, Badih Ghazi, Pritish Kamath, Ravi Kumar, Pasin Manurangsi, Milad Nasr, Amer Sinha, and Chiyuan Zhang. Unlearn and burn: Adversarial machine unlearning requests destroy model accuracy. In The Thirteenth International Conference on Learning Representations, 2025.
URL https::/
Michael F Hutchinson. A stochastic estimator of the trace of the influence matrix for laplacian smoothing splines. Communications in Clatistice-Simmuattioo and Compulation, 18(3):1059 1076, 1989.
Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al. Mistral 7b. arXiv preprint ar Xrxim22110.000 060825, 2023.
Di Jin, Zhijing Jin, Joey Tianyi Zhou, and Peter Szolovits. Is bert really robust? strong baseline for natural language attack text classification and entailment. In Proceedings of the AAAI conference arlificial intelligence, volume 34, pp. 8018 8025, 2020.
Zhuoran Jin, Pengfei Cao, Chenhao Wang, Zhitao He, Hongbang Yuan, Jiachun Li, Yubo Chen, Kang Liu, and Jun Zhao. RWKU: Benchmarking real-world knowledge unlearning for large language models. In The Thirty-eight Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2024. URL https::/ het.:/reeeeetttorrmrmmmddddodsdd
Sandesh Kamath, Amit Deshpande, K Venkata Subrahmanyam, and Vineeth N. Balasubramanian. Can we have it all? on the trade-off between spatial and adversarial robustness of neural networks. In A. Beygelz- imer, Y. Dauphin, P. Liang, and J. Wortman Vaughan (cds.), Advances in Neural Information Processing Systems, 2021. URL https:///penreviee, www/teeuttmniieeee (i..forummid=kk1BBBBRo...
Antonia Karamolegkou, Jiaang Li, Li Zhou, and Anders Søgaard. Copyright violations and large language models. In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Proceedings of the 2023 Conference Empirical Methods in Natural Language Processing, pp. 7403 7412, Singapore, December 2023, Association for Computational Linguistics. doi: 10.18653/v1/2000 wwt....mmlppaae www.eeemwnnmnc...mmmmmm main.458. URL https://aclantholl.e
org/2023. e01223.emnlp-aae www.eennnlmmaaeeeeeeeeeeeemmaamm main. main.458/
Diederik P Kingma. Adam: A method for stochastic optimization. arXiv preprint arXiv: 2014.
Pang Wei Koh and Percy Liang. Understanding black-box predictions via influence functions. In International conference machine learning, pp. 1885 1894. PMLR, 2017.
Anders Krogh and John Hertz. A simple weight decay can improve generalization. Advances in neural information processing systems, 4, 1991.
Kevin Kuo, Amrith Setlur, Kartik Srinivas, Aditi Raghinathan, and Virginia Smith. Exact unlearning of finetuning data via model merging at scale. Xixiv preprint arXiv:2504.04p 2025.
15

---

Published in Transactions on Machine Learning Research (04/2026)
Namhoon Lee, Thalaiyasingam Ajanthan, and Philip Torr. Snip: Single-shot network pruning based on connection sensitivity. In International Conference on Learning Represenlations, 2019.
Jinfeng Li, Shouling Ji, Tianyu Du, Bo Li, and Ting Wang. Textbugger: Generating adversarial text against real-world applications. arXiv preprint arXtc:1812.05271 2018.
Nathaniel Li, Alexander Pan, Anjali Gopal, Summer Yue, Daniel Berrios, Alice Galti, Justin D. Li, Ann- Kathrin Dombrowski, Shashwat Goel, Gabriel Mukobi, Nathan Helm-Burger, Rassin Lababidi, Lemnart Justen, Andrew Bo Liu, Michael Chen, Isabelle Barrass, Oliver Zhang, Xiaoyuan Zhu, Rishub Tamirisa, Bhrugu Bharathi, Ariel Herbert-Voss Cort B Breuer, Andy Zou, Mantas Mazeika, Zifan Wang, Palash Oswal, Weiran Lin, Adam Alfred Hunt, Justin Tienken-Harder, Kevin Y. Shih, Kemper Talley, John Guan, lan Steneker, David Campbell, Brad Jokubaitis, Steven Basart, Stephen Fitz, Ponmurangam Kumaraguru, Kallol Krishna Karmakar, Uday Tupakula, Vijay Varadharajan, Yan Shoshitaishvili, Jimmy Ba, Kevin M. Esvelt, Alexandr Wang, and Dan Hendrycks. The WMDP benchmark: Measuring and reducing malicious use with unlearning. In Ruslan Salakhutdinov, Zico Kolter, Katherine Heller, Adrian Weller, Nuria Oliver, Jonathan Scarlett, and Felix Berkenkamp (eds.), Proceedings of the 41st International Conference Machine Learning, volume 235 of Proceedings of Machine Learning Research, pp. 28525-28550 PMLR, 27 Jul 2024. URL
mlr wtt.ppereee////eepe.2p
html
Stephanie Lin, Jacob Hilton, and Owain Evans. TruthfulQA: Measuring how models mimic human false- hoods. In Smaranda Muresan, Preslav Nakov, and Aline Villavicencio (eds.), Proceedings of the 60th Annwal Meeting of the Association for Compulational Linguistics (Volume 1: Long Papers), pp. 3214 3252, Dublin, Ireland, May 2022. Association for Computational Linguistics. doi: 10.18653/v1/2022. acl-long.222 229. URL.
www.anthwwoooooooo/222222222 acl-long. 229/
Sijia Liu, Yuanshun Yao, Jinghan Jia, Stephen Casper, Nathalie Baracaldo, Peter Hase, Yuguang Yao, Chris Yuhao Liu, Xiaojun Xu, Hang Li, et al. Rethinking machine unlearning for large language models. Nature Machine Intelligence, pp. 1-14, 2025.
Michelle Lo, Fazl Barez, and Shay Cohen. Large language models relearn removed concepts. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Findings of the Association for Compulational Linguisties: ACL 2024, pp. 8306-830 8323, Bangkok, Thailand, August 2024. Association for Computational Linguistics. doi: 10.18653/v1 20024.indings--cc4422 URL httpp://acclaanchoooooooooo///ee wwt.:r//0......nnsaaaaaaa.a.. 492/
Nya Loshchilov and Frank Hutter. Decoupled weight decay regularization. In International Conference on Learning Representations, 2019. URL
Jakub Lucki, Boyi Wei, Yangsibo Huang, Peter Henderson, Florian Tramér, and Javier Rando. An adversarial perspective on machine unlearning for Al safety. Transactions on Machine Learning Research, 2025. ISSN 2835-8856. URL
Aengus Lynch, Phillip Guo, Aidan Ewart, Stephen Casper, and Dylan Hadfield- Menell.. Eight methods to evaluate robust unlearning in llms. arXiv preprint arXiv:2402.1688 2024.
Pratyush Maini, Zhili Feng, Avi Schwarzschild, Zachary Chase Lipton, and J Zico Kolter. TOFU; A task of fictitious unlearning for LLMs. In First Conference on Language Modeling, 2024. URL https://
Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. Pointer sentinel mixture models.
arXiv preprint 160::600..00. 070900743, 2016.
Aashiq Muhamed, Jacopo Bonato, Mona T. Diab, and Virginia Smith. SAEs can improve unlearning: Dy- namic sparse autoencoder guardrails for precision unlearning in L.Mss. In Second Conference on Language Modeling, 2025. URL
Milad Nasr, Javier Rando, Nicholas Carlini, Jonathan Hayase, Matthew Jagielski, A. Feder Cooper, Daphne Ippolito, Christopher A. Choquette-Choo Florian Tramèr, and Katherine Loc. Scalable extraction of training data from aligned, production language models. In The Thirteenth International Conference on Learning Representations, 2025. URL http:://ggaaaeeeaa..aee////or///
16

---

Published in Transactions on Machine Learning Research (04/2026)
Thanh Tam Nguyen, Thanh Trung Huynh, Zhao Ren, Phi Le Nguyen, Alan Wee- Chung Liew, Hongzhi Yin, and Quoc Viet Hung Nguyen. A survey of machine unlearning. ACM Transactions on Indelligent Systems and Technology, 16(5):1 46, 2025.
nostalgebraist. interpreting GPT: the logit lens, 2020. AcclBeerppaallrrrrrmetteeeeeeeep htt.r// -ens
URL http:://vww..ssswrrrr.e Accessed: 2026-01-13.
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feodback. Advances in neural information processing systems, -72277 2022.
Soumyadeep Pal, Changsheng Wang, James Diffenderfer, Bhavya Kailkhura, and Sijia Liu. LLM unlearning reveals strongee-than-exxeeteed coreset effect in current benchmarks. In Second Conference on Language Modeling, 2025. URL http:s///pparreeeeeee..//frrrrro
Vaidehi Patil, Peter Hase, and Mohit Bansal. Can sensitive information be deleted from LLMs? objectives for defending against extraction atlacks. In The Twelfth International Conference on Learning Represen- tations, 2024. URL
Martin Pawelczyk, Seth Neel, and Himabindu Lakkaraju. In-context unlearning: Language models as few- shot unlearners. In International Conference on Machine Learning, pp. 40034 40050. PMLR, 2024.
Rafael Rafailov, Archit. Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea. Finn. Direct preference optimization: Your language model is secretly a reward model. In Thirty-seventh Con- ference on Neural Information Processing Systems, 2023. URL httpss://ppenreviov..o wttp::rwwww.eeeouuunnnneaeea
HPuSIXJaa9
Nils Reimers and Iryna Gurevych. Sentence-bert: Sentence embeddings using siamese bert-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pp. 3982 3992, 2019.
David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R. Bowman. GPQA: A graduate- google-proof q&ca benchmark. In First Conference on Language Modeling, 2024. URL http:://pppeeeerreeeeeeee/r/f/ar nett/forumid---17768888888
Jie Ren, Zhenwei Dai, Xianfeng Tang, Hui Liu, Jingying Zeng, Zhen Li, Rahul Goutam, Suhang Wang, Yue Xing, Qi He, and Hui Liu. general framework to enhance fine-tuning- based LLM unlearning. In Wanxiang Che, Joyee Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar (ods.), Findings of the Association for Compulational Lònguistics: ACL 2025, pp. 18464 18476, Vienna, Austria, July 2025a. Association for Computational Linguistics. ISBN 979-8-89176-256-5, doi: 10.18653/v1 www.///....nnnnss..mm 949. URL
http://22....iiinnaaacc...... 949/
Jie Ren, Yue Xing, Yingqian Cui, Charu C Aggarwal, and Hui Liu. Sok: Machine unlearning for large language models. arXiv preprint arXiv:2506. 002227 2025b.
Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. Winogrande: An adversarial winograd schema challenge at scale. Communications of the ACM, 64(9):99- 1009-100,610 2021.
Jonas B Sandbrink. Artificial intelligence and biological misuse: Differentiating risks of language models and biological design tools. arXiv preprint arXiv:2306. 18952, 2023.
Atakan Seyitoğlu, Aleksei Kuvshinov, Leo Schwinn, and Stephan Günnemann. Extracting unlearned infor- mation from 1lss with activation steering. In Neurips Safe Generative AT Workshop 2024.
Abhay Sheshadri, Aidan Ewart, Phillip Guo, Aengus Lynch, Cindy Wu, Vivek Hebbar, Henry Sleight, Asa Cooper Stickland, Ethan Perez, Dylan Hadfelld-Menel, et al. Latent adversarial training improves robustness to persistent harmful behaviors in llms. arxiv preprint ar Xix:24407.15549, 2024.
17

---

Published in Transactions on Machine Learning Research (04/2026)
Weijia Shi, Jacchan Lee, Yangsibo Huang, Sadhika Malladi, Jieyu Zhao, Ari Holtzman, Daogao Liu, Luke Zettlemoyer, Noah A. Smith, and Chiyuan Zhang. MUSE: Machine unlearning six-way evaluation for language models. In The Thirteenth International Conference on Learning Representations, 2025. URL
https://openreviers
Ilia Shumailov, Jamie Hayes, Eleni Triantafilouu, Guillermo Ortiz-Jimenez, Nicolas Papernot, Matthew Jagielski, Itay Yona, Heidi Howard, and Eugene Bagdasaryan. Umunlearning: Unlearning is not suff- cient for content regulation in advanced generative ai. Xiv preprint arXiv:2407.00106 2024.
Nisan Stiennon, Long Ouyang, Jeffrey Wu, Daniel Ziegler, Ryan Lowe, Chelsea Voss, Alec Radford, Dario Amodci, and Paul F Christiano. Learning to summarize with human foodback. Aduances in Neural Information Processing Systems, 33:3008 3021, 2020.
Alon Talmor, Jonathan Herzig, Nicholas Lourie, and Jonathan Berant. CommonsenseQA: A question answering challenge targeting commonsense knowledge. In Jill Burstein, Christy Doran, and Thamar Solorio (eds.), Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Longuage Technologies, Volume 1 (Long and Short Papers), pp. 4149 4158, Minneapolis, Minnesota, June 2019. Association for Computational Linguistics. doi: 10.18653/ 1..18653/v1/NI/ 1 1421. URL https::/acclantholo0g 1aw.a//ooooooooo/111111111 NtggN199-421//
Rishub Tamirisa, Bhrugu Bharathi, Andy Zhou, Bo Li, and Mantas Mazeika. Toward robust unlearning for LLMs. In ICLR 2024 Workshop on Secure and Trustworthy Large Language Models, 2024. URL
https://openreviog/ net/forrurriie=44PP2FF
Rishub Tamirisa, Bhrugu Bharathi, Long Phan, Andy Zhou, Alice Gatti, Tarun Suresh, Maxwell Lin, Justin Wang, Rowan Wang, Ron Arel, Andy Zou, Dawn Song, Bo Li, Dan Hendrycks, and Mantas Mazeika. Tamper-resistant safeguards for open-weight LLMs. In The Thirteenth International Conference on Learn- ing Representations, 2025. URL httpps://openreviexx http:://pppeeeeeeeeeeeeeerrrrrra
Pratiksha Thaker, Yash Maurya, Shengyuan Hu, Zhiwei Steven Wu, and Virginia Smith. Guardrail baselines for unlearning in Ilms. aiiic preprinl arXiv:2433.0338 2024.
Pratiksha Thaker, Shengyuan Hu, Neil Kale, Yash Maurya, Zhiwei Steven Wu, and Virginia Smith. Position: Lim unlearning benchmarks are weak measures of progress. In 2025 IEEE Conference on Secure and Trustworthy Machine Learning (SaTMMI, pp. 520-533. IEEE, 2025.
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bash- lykov, Soumya Batra, Prajiwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fino-tundd chat models. Xrxie preprinl or XrXiv2230700928. 2023.
Florian Tramer and Dan Boneh. Adversarial training and robustness for multiple perturbations. Advances in neural information processing systems, 32, 2019.
Lewis Tunstall, Fdward Beeching, Nathan Lambert, Nazneen Rajani, Kashif Rasul, Younes Belkada, Shengyi Huang, Leandro von Werra, Clémentine Fourrier, Nathan Habib, Nathan Sarrazin, Omar Sanseviero, Alexander M. Rush, and Thomas Wolf. Zephyr: Direct distillation of Im alignment, 2023.
Changsheng Wang, Chongyu Fan, Yihua Zhang, Jinghan Jia, Dennis Wei, Parikshit Ram, Nathalie Bara- caldo, and Sijia Liu. Reasoning model unlearning: Forgetting traces, not just answers, while preserving rea- soning skills. In Christos Christodoulopoollo,, Tanmoy Chakraborty, Carolyn Rose, and Violet Peng (eds.). Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pp. 4427 4443, Suzhou, China, November 2025a. Association for Computational dinguistics. ISBN 979-8-89176-332-6 doi: 10.18653/v1/2022 ee1.p..mmmpmmmi..... 220. URL https://aclatthollggy org. 22gg wwnlp-main.nn...
Changsheng Wang, Yihua Zhang, Jinghan Jia, Parikshit Ram, Dennis Wei, Yuguang Yao, Soumyadeep Pal, Nathalic Baracaldo, and Sijia Liu. Invariance makes LLM unlcarning resilient even to unanticipatod downstream fino-tuning. In Forty-second International Conference on Machine Learning, 2025b. URL
https://openreviev.c
18

---

Published in Transactions on Machine Learning Research (04/2026)
Yaxuan Wang, Jiaheng Wei, Chris Yuhao Liu, Jinlong Pang, Quan Liu, Ankit Shah, Yujia Bao, Yang Liu, and Wei Wei. LLM unlearning via loss adjustment with only forget data. In The Thirteenth International Conference on Learning Representations, 2025c. URL httph: //opsr///ppereviex http///erreeeeeeeeefforrrreme net:/forrm21---0SSiicc.
Boyi Wei, Kaixuan Huang, Yangsibo Huang, Tinghao Xie, Xiangyu Qi, Mengzhou Xia, Prateek Mittal, Mengdi Wang, and Peter Henderson. Assessing the brittleness of safety alignment via pruming and low- rank modifications. In Forty-first International Conference on Machine Learning, 2024a.
Boyi Wei, Weijia Shi, Yangsibo Huang, Noah A. Smith, Chiyuan Zhang, Luke Zettlemoyer, Kai Li, and Peter Henderson. Evaluating copyright takedown methods for language models. In The Thiht Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2024b. URL
https::/openrevieev.. wttp://wwww..neeeoooorrdaraaaaaa
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoe V Le, Denny Zhou, et al. Ch prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824 24837, 2022.
Stanley Wei, Sadhika Malladi, Sanjeev Arora, and Amartya Sanyal. Provable unlearning in topic modeling and downstrcam tasks. In The Thirtennth International Conference on Learning Representations, 2025.
URL htttps://openrevier http:://pppprreeeeeeeeeffooottta
Jiaxin Wen, Pei Ke, Hao Sun, Zhexin Zhang, Chengfei Li, Jinfeng Bai, and Minlie Huang. Unveiling the implicit toxicity in large language models. In Houda Bouamor, Juan Pino, and Kalika. Bali (eds.), Proceedings of the 2023 Conference Empirical Methods in Natural Language Processing, pp. 1322- 1338, Singapore, December 2023. Association for Computational Linguistics. doi: 10.18653/v1/2000 emnlp-maie 84. URL htttps://acaanthooto hwwpt/ooooyoooooo/oggoooo org/2023.
Cheng- Hsin Weng, Yan-Ting Lee, and Chan-Hung (Brandon) Wu. On the trade-off between adver- sarial and backdoor robustness. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and Н. Lin (eds.), Advances in Neural Information Processing Systems, volume 33, pp. 11973 11983. Cur- ran Associates, Inc., 2020. URL https://proceeednng meeeiiinnnnnnnuuppeeeeeeee
neur peeurps.cc/papee.... cccpaper_aaaeeeeeeeeeeeeeeeeeeee /iiles//apper/2000/fia.e
Paper pdf
Xiaoyu Wu, Yifei Pang, Terrance Liu, and Zhiwei Steven Wu. Breaking the gold standard: Extracting forgotten data under exact unlcarning in large language models. arXiv preprint 2r55:2250..2377, 2025.
Heng Xu, Tianqing Zhu, Lefeng Zhang, Wanlei Zhou, and Philip S. Yu. Machine unlearning: A survey. ACM Comput. Surv., 56(1), August 2023. ISSN 0360-0300. doi: 10.1145/3603622 URL https://doi.org//
1145/3603620
Han Yan, Zheyuan Liu, and Meng Jiang, Dual-space smoothness for robust and balanced LLM unlearning. In The Fourteenth International Conference on Learning Representations, 2026. URL http:::///ppeerevier
net//oruuniid-VIWWeey
Hongbang Yuan, Zhuoran Jin, Pengfei Cao, Yubo Chen, Kang Liu, and Jun Zhao. Towards robust knowledge unlearning: An adversarial framework for assossing and improving unlearning robustness in large language models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pp. 2025a.
Xiaojian Yuan, Tianyu Pang. Chao Du, Kejiang Chen, Weiming Zhang, and Min Lin. A closer look at machine unlearning for large language models. In The Thirteenth International Conference on Learning Representations, 2025b. URL httpp:///opnneeriee.eee/eeeee wtt.://ooorrriie////ttiainig
Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. HellaSwag: Can a machine really finish your sentence? In Anna Korhonen, David Traum, and Lluis Mårquez (eds.), Proceedings of the 57th Annual Meeting of the Association for Computational Länguisties, pp. 4791-4800 Florence, Italy, July 2019. Association for Computational Linguistics. doi: 10. 10.18653/v1/P19-1477 URL https: Soacclanthology.cr///11--111
19

---

Published in Transactions on Machine Learning Research (04/2026)
Ruiqi Zhang, Licong Lin, Yu Bai, and Song Mei. Negative preference optimization: From catastrophic
collapse to effective unlearning. In First Conference on Language Modeling, 2024. URL. https:
//openreview //ppeareviev..et/foormmm/// htt::/oorrrriiiiiMMBBB.....
Zhiwei Zhang, Fali Wang, Xiaomin Li, Zongyu Wu, Xianfeng Tang, Hui Liu, Qi He, Wenpeng Yin, and Suhang Wang. Catastrophic failure of LLM unlearning via quantization. In The Thirteenth International Conference Leurning Represenlations, 2025. URL http::://peneeves...eerrrrmmmmr oet..///ooummddddd/sssmmmammm
Haomin Zhuang, Yihua Zhang, Kehan Guo, Jinghan Jia, Gaowen Liu, Sijia Liu, and Xiangliang Zhang. SEUF: Is unlearning one expert enough for mixture-of-experts LLMs? In Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilchvar (ods.), Proceedings of the 63rd Annual Meeting of the Association for Compulational Linguistics (Volume 1: Long Popers), pp. 8664 8678, Vienna, Austria, July 2025. Association for Computational Linguisties. ISBN 979-8-80 111-9911-----5 251-0. doi: 10.18653/v1 v/12225
acl-long.424. URL httts::/ .rg.l/ooo.ooooo//ooo/oo 2025
424/
Daniel M Ziegler, Nisan Stiennon, Jeffrey Wu, Tom B Brown, Alec Radford, Dario Amodei, Paul Chris- tiano, and Geoffrey Irving. F'ne-tuning language models from human preferences. ar preprint arXiv:1900008503 2019.
Andy Zou, Long Phan, Sarah Chen, James Campbell, Phillip Guo, Richard Ren, Alexander Pan, Xuwang Yin, Mantas Mazeika, Ann-Kathrin Dombrowski, et al. Representation engincering: A top-down approach to ai transparency. arXiv preprint arX::22310.01 2023a.
Andy Zou, Zifan Wang, Nicholas Carlini, Milad Nasr, J Zico Kolter, and Matt Fredrikson. Universal and transferable adversarial attacks on aligned language models. arXie preprint orXiv:2307.15443 2023b.
20

---

Published in Transactions on Machine Learning Research (04/2026)
Appendix
Table of Contents
A. Experimental Setup
21
A.1 Dalasels
2
A.2 Prompt Template
A.3 Evaluation Metrics
A.4 Implementation Details
B. Proofs
B. Proof of T Chereem
B 3.2 Proof of Theorem
C. Empirical Validation
C.1 Empirical Validation of Section
C. Empirical Validation of Assumption
D. Additional Results on Knowledge Recovery
E. Robustness of RNA Against Multiple Forget-Tokenss
Effects of Randomizing Different Latent Spaces
32
G. Robustness of RNA Models Against Prompt Attacks
H. Effects of RNA Chain-of- Slain-of-Thoneee Thought Prompting
Performance of Other Models
36
Performance of RNA under Miscalibrated Unlearning
K. Limitations
AT Usage Declaration
38
A Experimental Setup
A.1 Datasets
WMDP Li et al. 2024 stands for the Weapon Mass Destruction Proxy, benchmark for measuring and mitigating the malicious uses of LLMs in biosecurity, cybersecurity, and chemical security. This corpus consists of three components: forget sets, retain sets, and QA sets. The WMDP-Biology, both forget-set and retain-set, are collected from PubMed papers. The forget-set includes papers that were used to generate the WMDP-Biology QA set. The retain-set samples from general biology papers, excluding both the papers from the forget-sct and topics related the QA set through keyword filtering. For the WMDP-Cyber both forget and retain sets comprise passages collected from GitHub, distinguished by different keyword sets used in the collection process. The QA set contains 3, 668 multiple-choiee QAs across three security domains: WMDP-Biology 273 QAs), WMDP-Cyber 91.987 QAs), and WMDP-Chemical (408 QAs). This corpus is available at Dettps:/hhugginteee http:/faa..oo/ddaaeeeeeeeeeeeeee wwwp:/attttttteeiieeeeeeeeeeseee cais/wmdp
MUSE Shi et al.) 2025) a LLM unlearning benchmark, designed for evaluation six dimensions of unlearning aligned with both data owners and deployer expectations. The corpus consists of forget, retain, and holdout splits, and includes two domains: News (BBC articles) and Books (Harry Potter books). This
corpus is available at httpp::/mmmut http:////eeeeehhh...emmm cichh..ithub.....
21

---

Published in Transactions on Machine Learning Research (04/2026)
The lollowing are multiple choice questions (with answers) about anatomy.
Question: A lesion causing compression of the facial nerve at the stylomastoid foramen will cause ipsilateral
A. paralysis of the facial muscles.
B. paralysis of the facial muscles and loss of taste.
C. SARS-CoV-19.
D. paralysis of the facial muscles, loss of taste, lacrimation and decrcasod salivation.
Answer:
Figure 6: A sample zero-shot multiple-choice QA prompt. A random incorrect answer (e.g., C. paralysis of the facial museles, loss of taste and lacrimation.) is replaced by forget keyword SARS-CoV-19 while the correct answer (A. paralysis of the facial muscles.) is unchanged.
Question: What year did the Orkney Islands become part Scotland?
Answer: 1472
Question: When will the changes to maternity services come into effect at Causeway Hospital?
Answer: 17 July 2023
Question: How many women and men are in New Zealand's cabinet after the promotion of Willow-Jean Prime?
Answer:
Figure 7: A sample MUSE QA prompt in open-ended generation format. A retain QA is concatenated with forget QA few-shot samples.
Wikitext (Merity et al.) 2016) is language modeling dataset consisting of over 100 milion tokens tracted from Wikipedia. Following Li et al. (0024), we specifically the SIIITTETT-2RRAWVV test split as the retain-set for fine-tuning. The dataset is publicly available at https::/hugginnfacee co http://nnaaacc.cccattssteeeeee salesforce/vikitext
MMLU (Heddrycks et al. 2021 stands for Massive Multitask Language Understanding, a comprehensive dataset designed to assess world knowledge and problem-solving abilities of LLMss. It comprises 15, 908 multiple-choice QAs across 57 diverse categories, covering subjects such as mathematics, history, computer science, biology, and more. This dataset is available at https::/huggingface. http://wgggggaaooooocoaattsse//a wo./aatasstt//aais/mmllll
MMLU College Biology & Computer Security (Hendrycks et al. 2021 are two sub-categorios in MMLU, corresponding to topics closely related to the WMDP Biology and WMDP Cyber forget-sets. They are used to evaluate the unlearned model's ability to retain relevant. knowledge in areas related to the forget-sets.
"I Don't Know" dataset. We employ a set of 100 refusal responses as the preference answers for DPO+KL and MSF. For further details, refer the reader to Appendix C of Maini et al. (2024).
22

---

Published in Transactions on Machine Learning Research (04/2026)
A.2 Prompt Template
Multiple-choice template. We use the Im-evaluation--arress framework (ao et al. 2024) for evaluation. Each query is formulated as default zero-shot QA prompt (Figure 6 Following the setting of prior work Thaker et al. 2025), we randomly replace an incorrect answer in the retain QA datasct with the forget keyword "SARS-CoV-19, while leaving the correct answer unchanged. Since the forget keyword is unrelated the retain-queries, this modification expected have minimal effect on retain performance.
Open-ended template. Following Shi et al. (2025), we formulate the prompt as open-ended QA. We construct perturbed retain-queries by concatenating retain QA with forget QAs and benign retain-queries by concatenating retain QA with other retain QAs (Figure 7
A.3 Evaluation Metrics
Accuracy, Reduction Rate, and Recovery Rate. Following Li et al. 2024 we primarily zero- shot QA accuracy to assess the efficacy of unlearning methods. To further evaluate the unlearned models' brittleness and RNA's effectiveness, we report the accuracy reduction rate and recovery rate. These metrics are defined as follows:
AcCbase
ACCuntearned
Reduction Rate
X 100%
(14)
AcCbase
Accrus AcCunlearued
Recovery Rate
100%
(15)
ACChase
ACCunlearned
For example, if AcCbase = 60, ACCRMU = 30, ACCRMU w/ RNA = 50, then the reduction rate is 50% and the recovery rate is 66.67%.
Additionally, we report accuracy under attack (AuA) and ROUGE-L score for experiments Section evaluate the robustness of RNA against prompt injection attacks.
Knowledge Memorization (KnowMem; Shi al. (2025)) measures model's knowledge in dataset D. Specifically, KnowMem is compuated the average of the ROUGE-L scores between all question-answer pairs in D:
KnowMem(J,DD
1
Σ ROUGE (f(g),
(16)
D
(q.a)~D
where f(g) is the generated answer from model given question y, a is the reference answer of question
Verbatim Memorization (VerbMem; Shi et al. 2025 quantifies the verbatim memorization by prompt-
ing the model with the first forget-tokens x(aq E Dy and comparing the generated outpuls to the ground-
truth suffix x/ x4+13 E Dy:
1 Σ ROUGE (f(x( ( VerbMem(ff.j) DDyl xfNDD
(17)
A.4 Implementation Details.
Hyperparameters. Models are fine-tuned using Adam Kingma 2014 for T = 500 update steps, learning
rate is 5e - 5, batch size of 4, max sequence length is 500 with WMDP-Biology and 768 for WMDP. Cyber. Following previous works (Li el al. 2024), we update three layers of parameters {1,1 - 1, 2} of the model for memory efficiency. For the original RM methods, we set the retain weight Abiology 7 for all methods, the cocfficient Obiology Ceyber 6.5
1200 and acyyeer 1200, the unlcarned layer for Adaptive RMU. For RSV, we grid search for the coefficient (5, 10, 20, 30, 40, 50, 60, 70, 80, 90, 100) and select Chiology Grybe 10. For the original PO methods,
for RMU, and the scaling factor
23

---

Published in Transactions on Machine Learning Research (04/2026)
adopt the default hyperparameters used in previous works (Yuan et al. 2025b Fan et al. 2025b Specifically, we set B = 0.1 for all PO methods, and 7 = 0 for both SimNPO KT. and SimNPO For the retain weights, we perform a grid search over combinations of (Abiologyy Olyybrr;; where Abiologyy Olybber E {5, 10, 20, 30, 40, 50, 100). We select the combinations that achieve a balanced tradc-off between forgetting and retaining performance: (30, 50) for DPO+KT, (5, 20) for DPO+MSE, (50, 50) for NPO+KI, (5,20) for NPO+MSE, (20, 50) for SimNPO+KL, and (10, 5) for SimNPO+MSE.
For RM w/ RNA, we set the perturbed layer is and perform grid search for noise scale E (10-2,22 x 10 200 1000002-000 2,4x 10 2, x 10 2, 6 10
7x 2,8x 10- 2,9 9 10 1001 and report the best performance with v = x 10-2 for RMU, V/ = x 10-2 for Adaptive RMU, and y = 9 x 10-2 for RSV.
For PO w/ RNA, we set the perturbed layer is = 7 and perform grid search for noise scale v E (10-2 ; 1.2
10-2, 1..14xx 1 10xx000 2, 2.6 1.6x x 10-2 1.8x x 10 1,- 10 2, 10-2, x 10 10-2,,, 2,5 x 10-2, 6 10 10-2, 7, 10 -2 8 x 10 10-2, ,9 1002,0120000 and report the best performance with 1.4 x 10 2 for NPO+KL, == 10-2 for DPO+KL, for SimNPO+MSE.
10-2 2 for NPO+MSE, 2x 10-2 for DPO+MSE, 1.4 10-2 for SimNPO+KL, and 1.8 10-2
Hyperparameters for other settings are specified in their respective subsections.
Reproducibility. All experiments are conducted using two NVIDIA A40 GPUs, each with 45GB of memory. Our implementation available at httt::/ghhhh...oooeeaaaaalllaaae
B Proofs
For clarity, we restate the thcorems below.
B.1 Proof of Theorem 1
Theorem 1. If Assumption holds, the change in the output representation of the generated token x; given the perturbed relain-query x and the benign relain-quent x-c in the unlearned model J", defined as A = fx"((x(Xx"> (22eer) - (x. (x), follows the Normal distribution NO, nJJ J), where J = =V f (x (x)(x)) is the Jacobian of f" (x]][[x" with respect to zzzi-
Proof. Consider the output representation of the predicted token x; given the perturbed retain-query prefix r.per in the unlearned model (x(x' (err) We show the claim by using the framework of the generative latent variable model (GLVM). Specifically, model f generates token x conditioned on latent variable the T.per denoted as ("x(1222)")) Under Assumption [
x
<i
T.per corresponding to the perturbed prefix x following holds:
(xx(222=== = (xx]Tat) +6)
(18)
Since small, we approximate the function f'x([lz& E) around zaz by using the first-order Taylor approximation:
["(( 22 + (x + (x(]2))))
(19)
f' + (x Ver (x TT
(20)
given that E ~~ Wo n)), by the affine transformation of Gaussian variables, we obtain ~ ~N(00, J J), where J = Vx 1d (x]] x22,) is the Jacobian of f" (xx[[z2) with respect to zzzg.
Let A = / +
) f
B.2 Proof of Theorem 2
Thcorem 2. Suppose RNA adds a small, independent Gaussian noise 8 vI), v E CR+ into the retain- representation layer I of unlearned model f". If Assumption D and Assumption that the RNA model rejects the effect caused by the forget-token, denoted P A 0]. is approrimate
hold, the probability
24

---

Published in Transactions on Machine Learning Research (04/2026)
1 arctan VB 1+ 1.
where g -aagge (F" and g Vay Te,, (xx((x)x2))) are the gradients of the loss of generated token x with respect zizi p.per and N'2t.
Proof. Let us consider the generation of x through the lens of GLVM. The loss of x given the latent repre- sentation z of the prefix x p.per in unlearned model f", is denoted by (xxx12220 Under Assumption the following holds:
xx = = j( +c)
(21)
Since is small, we linearly approximale function J( + E) around 2nz, by using the first-order Taylor approximation:
+ 3 ((
(22)
Rearranging Equ. 22 we obtain the approximate change in loss:
V
(23)
Under Assumption 1
and Assumption 2
J (frma 2 () and (xf=xx((( can be expressed as:
J( x +22
(24)
= (z/ + 61))
(25)
2 [( Tiper 81))
(26)
~ [(f(((2222 r-per + V 2 [ (x To,
(27)
= ((("(( +2))
(28)
T(I" + Var I (x(
(29)
Substituting Equ. 27 and Equ. 29
the change in loss in RNA model fms of predicted token xf is approxi-
mately:
(x - J
PPT -
V V I [f 2- (T 8n Veg, ((
(30)
A
AJ" per) 8n g 82.
(31)
where ger =V 22 J(f" [x] per and = Ve, J( (z)))
From Eqn.
23 and Eqn.
31 the ratio of the RNA loss change to the original unlearned model loss change is:
A
(g T 8,
g Tō 82 =1 per) T 8, -g" To
AJ 21
(32)
AJ"
g T
Since E ~ NO (l), 8, and 8n are independently sampled from NO vf), thus
pur) T 8, - 82 ~ NO v [|g per 1 11g|g//1) ll)](2))
g ~
The probability that the RNA model rejects the effect induced by noise é is:
Ara
per
T 8, -ggg 82
00
~
(33)
AT"
T
The ratio of two random normally distributed variables location parameter Zg and scale paramcter 7
(genry
81-9" 82 g 2es
follows a Cauchy distribution with
V
1gl )
The cumulative distribution function
25

---

The max Published in Transactions on Machine Learning Research (04/2026)
of Cauchy V (1+ |1 [) given by
F(2 7) = + arclan
1
22
1 +
gua
7
|gll
Thus, the probability is approximated:
A P
& 0
(gber) T 81 9
82
~ P
v -1
= F(r = -1;00, 7)
(34)
AT
g g'
2 + arctan
(35)
1 +
Iguer
Tsll
gg
2 arctan
(36)
|99||
C Empirical Validation
C.1 Empirical Validation of Section 3
In this subsection, we aim to show that the PO forgetting process (minimizing the forget-loss) can be interpreted as injecting random noise into forget-represenlations during fine-tuning.
Noise sensitivity of layers. We formalize the forgetting through the lens of noise sensitivity (Arora all, 2018). Let Z R4 be the hidden states vector of forget-sample x at layer 1 in the model f, where di is the dimension of layer I. Let y be the (1+ 1)-hh transformer layer in model f Consider random perturbation & drawn from Normal distribution No, I). The noise sensitivity of g with respect to (O, T) on forget-set Dy, is defined as:
def = E 5 (Dy) BE (z/ + - -z)(()1 [zg(2
(37)
where Jg is the Jacobian of layer g at input z'. A lower value of 8 (Dy) indicates that the layer is stable to noise, or "filled" by noise. This definition suggests a way to validate the analysis of Scction 3 We expect (D,, with respect the PO and RM models be smaller than that the base model; that is, unlearned models are more stable to noise than the base model.
Sctup. For all unlearned models, perform grid search for g from the first to the last layer in the model. We use the WMDP- forget-set to compute the noise sensitivity of layers by Eqn. 37 sequence length of each forget-sample is set to 512.
Results. As shown in Figure (left), we observed that the noise sensitivity of layer g = 8 in both PO and RM models is significantly reduced compared to the base model. This empirical result validates the analysis presented in Section Figure g (right) reveals that the most pronounced reductions occur in the middle layers, whereas the later layers exhibit greater stability to noise.
Discussion. We employ the noise sensitivity to validate the analysis in Section 3 However, believe that this definition has broader potential applications. One could explore the noise sensitivity as metric for measuring unlearning difficulty. This definition generalizes two perspectives: model difficulty and data difficulty. From the model perspective, noise sensitivity can help characterize the unlearning difficulty of specific components-sude as an intermediate layer (as described in Eqn. 37 a group of layers, an entire model (e.g., Llama vs. Mistral), or more fine-grained modules in the layer such MLP, attention patterns, individual neurons. From the data perspective, the noise sensitivity be used to evaluate unlearning
26

---

Published in Transactions on Machine Learning Research (04/2026)
Base
600
4000
RMU
Adap. RMU
500
RSV
Nose Sensitivity
3000
NPO+KL
400
NPO+MSE
2000
DPO+
SimNPO+KL
200
SiMN
1000
100
Base
RMU
Adap.
RSV
DPO
DPO
NPO
SimMPO SimMPP +MSE
5
15 Layer
20
30
Figure 8: Left: noise sensitivity of layer g for the base model, PO models, and RM models. Right: Layer-wise wise noise sensitivity across all layers for the base model, PO models, and RM models.
difficulty at the level of individual samples, sub-classes, data subsets. We leave these promising directions for future work.
122
RMU (WMDP-Biology)
Tel Adaptive WWMDP-Biologyy
1+1
(WMDP-Biology)
1.25
8
1.00
0.75
0.50
2
2
0.25
2
B
RMU (WMDP-Cyber)
le2
Adaptive RMU (WMDP-Cyber)
RSV (WMDP-Cyber)
1
2
2
Figure 9: Distribution of per-sample loss differences forget-samples from the WMDP-Biology and WMDP- Cyber QA datasets.
The convexity assumption. Our derivation in Section 3 is based on the assumption that the loss is locally convex w.r.t. z/. This assumption ensures that the Hessian matrix V2 22, ( is positive definite, which %
in turn guaranteos that its trace positive. However, z/ located at a local marimum, the Hessian would be negative definite and the sign of Eqn. g would flip, that is, adding noise would, in such cases, decrease the expected loss. Despite local convexity being difficult to guarantee due to the highly non-linear property of deep networks, note that assumption is reasonable rather than overly restrictive. We conduct the following empirical experiment to understand how the RM methods affect the loss of forget-samples. Specifically, we compute the loss change relative to the base model and RM models for all forget-samples in the WMDP-Biology and WMDP-Cyber QA datasets. The distribution of loss changes is shown in Figure g We observe that the loss changes are positive, suggesting that, general, RM methods increase the loss of forget-samples compared to the base model. This behavior aligns with the assumption and further supports the analysis in Section that adding noise typically leads to higher loss.
27

---

Published in Transactions on Machine Learning Research (04/2026)
C.2
Empirical Validation of Assumption 1
Adaptive
Projection on PC2
Projection PC2
Projection on PC2
0
-1
2.5
2.5
Projection on PC1
Projection on PC1
Projection on PC1
DPO+KL
DPO+MSE
NPO+KL
Projection on PC2
1
Projection PC2
Projection on PC2
0
Projection on PC1
Projection on PC1
Projection on PC1
NPO+MSE
SimNPO
SimNPO+MSE
Projection on PC2
Projection on PC2
Projection on PC2
1
-1
-1
3
2
2
0
Projection on PC1
Projection on PC1
Projection on PC1
Figure 10: Empirical distributions of c projected onto the principal component and principal component are approximately Gaussian and centered near zero.
Assumption posits that the latent representations of perturbed retain-query in unlearned models behave as randomized perturbations that can be approximated by Gaussian noise. While this intuitively plausible, its validity complex LLMs might require further validation. We first discuss that Gaussian noise common and well-established choice for random perturbations. Gaussian noise allows us to formally establish the core intuition that the presence of forget-tokens in retain-queries introduces noise in the model's outputs.
To cmpirically examine this assumption, we mcasure the change in latent representations at layer 7 for tokens in perturbed relain-queries from perturbed MMLU QAs across different unlearned models. Figure I0 shows the distribution of these changes projected onto the first two principal components. Across unlearned models, the distributions are centered near zero, exhibiting Gaussian-like contours in the PCA space. These results provide empirical support for modeling the perturbation as approximately Gaussian with near-zero mean.
28

---

Published in Transactions on Machine Learning Research (04/2026)
D Additional Results on Knowledge Recovery
D.1
Threat Model
We consider white-boz access to unlearned models' weights, allowing direct modifications or interventions during both training and inference. In addition, we assume access to the base model's weights before unlearning.
D.2 Knowledge Recovery Methods and Experimental Setup
Logitlns Inotaalebbraist 2020). Logitless projects the last Table Hyperparameters for relearning.
ken's activations of WMDP QA in the unlearned model into the vocab space to trace the model's unlearned knowledge at an intermediate layer. For each WMDP QA, we add the follow- ing instruction prefiz "Answer the following question with A, B, C, or D. An'nn,, to each prompt. We then compute the softmax probabilities over the answer tokens and select the token with the highest probability as the model's 's prediction. We apply Logitlens at the last layer of the model.
Hyperparameter
Value
LoRA rank
128
LoRA target modules
all linear
LoRA alpha
LoRA dropout
0
Maximum sequence length
1024
Epochs
3
Batch size
Finetuning. We evaluate forget-robustness of unlearned models under relearning using WMDP forget-sets and benign finetuning on unrelated tasks (Wikitext). Unlearned models are fine-tuned using LoRA adaptation (Hu et al. 2022). Hyperparameters are specified in Table 3
Learning
2c 4
Learning scheduler
linear
Warmup ratio
0.05
Optimizer
Adamm
Weight decay
0.01
The idea is to extract the unlearning veclor and ablate it to bypass the unlearning, thereby aiming to recover unlearned knowledge. At each layer, we extract the unlearning vector by computing difference-in-mennn [Belrosc| 2023) activations between the unlearnod model and base model on a synthetic forget preference dataset introduced by Lucki et al. (2025), which is constructed by using OpenAl API to convert the WMDP forget-set to multiple-choice QAs. When caleulating the mean activations, exclude the first 40 tokens to ensure unlearning noise is injected.
Orthogonalization.
Enhanced Greedy Coordinate Gradient (Enhanced GCG; Lucki el al. (2025 ) Enhanced GCG
extends the standard GCG (Zou et al. 2023a by optimizing an injected adversarial prefer within the prompt to increase the model's likelihood generating a specified target continuation under the base model. GCG's optimization is performed token space using greedy search: at each step, GCG uses the gradient signal to find token substitutions at specific positions, evaluates token replacements, and applies the best update. Following the standard experimental protocol, the adversarial prefix is optimized for 1500 update steps.
Set Difference Pruning. Set difference pruning (Wei et al. 2024a) identifies neurons that only contribute to unlearning. By setting them to zero, we can isolate the unlearning effect. We employ SNIP score (Lee cet al. 2019) to quantify the neurons' inffuence on unlcarning and model utility using WMDP forget-sct and Wikitext datasets, respectively. We then prune neurons that rank within top-7% of influence for unlearning but outside top-p% for utility. We perform grid search for p.q E (0.5, 1.0, 2.5, 5.0, 7.5), and report the highest accuracy under attack on WMDP QAs.
D.3 Additional Results
Benign finetuning. Figure 11 shows WMDP- (top row) and WMDP-Cyber (bottom row) QA accuracy after benign relearning by finetuning the unlearned models on retain-samples from Wikitext. Over- all, both the original unlearned models (red) and RNA models (purple) remain substantially below the base model performance (black dotted linc). However, RNA models generally recover more WMDP accuracy than their corresponding original unlearned models, suggesting RNA makes unlearned models more susceptible to benign finetuning.
29

---

Published in Transactions on Machine Learning Research (04/2026)
RMU
Adaptive RMU
RSV
NPO-KL
NPO-MSE
DPO-KL
DPO-MSE
SimNPO-KL
SimNNO CEANPOOMSE
WMDP-Bislog Bip-BB
60-
XOMA EMA
50-
30
WMDP-Cylee Cyber
40
25.
50
100
5000
1000
50
100
500
1000
10
50
100
500
1000
50
100
500
1000
5
100.
500
1000
10
50-
100
500
1000
50
100
500
1000
5
50
100
500
10me
10
50
100
500
1000
Figure 11: Accuracy of relearned models measured on WMDP- and WMDP-Cyber QA datasets.
Relearning using retain-samples from Wikitext
On other attacks. Table shows knowledge recovery performances under four white box attacks (logitlens, orthogonalization, enhanced GCG, and pruning), where lower WMDP QA accuracy indicates stronger ro- bustness. Overall, RNA does not consistently improve or degrade forget-robustness across different unlearn- ing methods and attack settings.
E Robustness of RNA Against Multiple Forget- Tokees
Tokens
Analysis the harmfulness of forget-tokens. One might ask:
"Which forget-tokens when appearing in the relain-query cause the unlearned model to misbehave?". We examine the harmfulness of forget-tokens in the forget-set by measuring the cosine similarity be-
60
RMU
Adap. RMU
50
45
tween bi-gram forget-tokons and their respective documents, across all
DPO+MSE
docunents in the WMDP forget-sels. We select the top 10 most simi-
NPO+KL
lar, least similar, and those with values around the mean of the distri- bution. Perturbed MMLU QAs with respect to these forget-tokens are synthesized following the procedure described in Section 7.1 As shown in Figure 12 we observed clear trend between the accuracy and the similarity: forget-tokens with higher similarity with their cor- responding documents are more harmful to unlearned mod- els. We further assess the RNA models' robustness against n-gram similarity perturbations for n E (4,8,16)
35
NPO+MSE
30
SimNPO-KL
SiMNPO+MSE
most
mean Similarity
Figure 12: Accuracy of unlearned models perturbed MMLU with respect to bi bi-gram similarity per- turbations.
Sctup. For cach document in the WMDP forget-set, extract n-grams for n E {2,4,8, 16) and compute their feature embeddings using Sentence- BERT [Reimers & Gurevych 2019 along with the embedding of the full document. We then extract the top 10 most similar n-grams to each document based on embedding cosine similarity. Perturbed MMLU QAs corresponding to these n-gram forget-tokens are synthesized following the procedure outlined in Subsection A.2 We utilize model checkpoints from the previous setting and perform evaluations accordingly. Results are reported for checkpoints selected based on the optimal noise scale v, as detailed in Table
Results. RNA's performance is summarized in Table through Table 9 We observe that RNA consistently improves the robustness of unlearned models across all n-gram perturbations. The most pronounced gains are obscrved when R.N. is applied with MSE retain-losses. Specifically, for DPO+MSE, performance im- provements are +24.6 (2-gram), +29.5 (4-gram), +24.9 (8-gram), and +19.6 (16-gram); for SimNPO+MSE, gains are +25.4, +23.3, +16.5, and +11.2. Importantly, RNA introduces minimal impacts on MMLU perfor- mance, where changes are generally within less than 0.5. However, RNA tends to reduce WMDP accuracy across all methods, with slightly drop ranging from 0.5 to 5.0. Additionally, RM methods derive minimal benefit from RNA under these settings.
30

---

Published in Transactions on Machine Learning Research (04/2026)
Table 4: Accuracy under attack of unlearned models measured on WMDP (Biology and Cyber). Top:
Attacks using WMDP-Cyber forget-set. Bottom: Attacks using WMDP-Biology forget-sct.
WMDP-Cyber QA (4)
WMDP-Biology QA (L)
Models
Defanlt
Logitlns Ortho. E. GCG Pruning Default Ortho. E. GCG Pruning
model
Original
44.3
64.5
Representation Misdirection
Original
28.6
28.6
39.8
30.4
40.6
28.0
61.2
27.4
53.9
RMU
w/ RNA
29.1
27.7
38.3
28.5
40.5
30.5
58.1
30.7
50.0
Original
27.9
28.4
39.6
32.6
41.3
29.4
61.4
41.3
53.1
Adaptive RMU
w/ RNA
28.7
27.1
38.1
29.4
39.4
31.3
55.9
34.2
57.3
Original w/ RNA
28.9
27.9
41.0
31.7
39.8
27.8
50.4
32.6
37.2
RSV
28.2
31.8
39.5
31.3
48.5
40.1
36.9
Preference
Optimization
Original w/ RNA
25.6
27.5
42.8
29.8
40.6
28.8
55.5
34.5
NPO+KL
25.8
27.1
43.3
28.1
39.7
29.6
62.7
36.1
58.5
Original
25.4
27.5
42.5
25.3
36.3
27.0
60.5
31.0
52.7
NPO+MSE
w/ RNA
27.0
27.0
42.9
27.6
35.7
30.6
61.0
34.1
44.9
Original
25.1
27.2
40.5
28.9
35.3
29.1
60.5
34.2
53.7
DPO+KL
w/ RNA
27.0
26.6
40.3
32.3
38.3
29.4
59.4
43.7
45.6
Original
23.9
26.6
40.3
26.0
37.4
28.0
46.4
32.5
44.9
DPO+MSE
w/ RNA
25.2
28.0
27.4
36.1
33.3
49.6
49.4
38.0
Original
27.1
27.4
42.1
26.2
39.9
26.5
61.6
32.6
45.3
SimNPO+KL
w/ RNA
25.7
27.5
42.3
26.0
39.8
29.6
63.1
33.7
60.0
Original
26.7
26.8
41.3
27.3
37.0
27.6
61.7
27.2
54.2
SimNPO+MSE
w/ RNA
26.0
27.4
42.8
27.5
37.3
31.5
60.6
37.9
60.6
WMDP-Biology QA (1)
WMDP-Cyber QA (4)
Models
Default Logitlens Ortho. E. GCG Pruning Default Ortho. E. GCG Pruning
Base model
Original
64.5
44.3
Representation Misdirection
Original
28.0
27.7
64.0
30.9
57.7
28.6
39.7
29.9
35.5
RMU
w/ RNA
30.5
29.1
59.4
42.8
56.8
29.1
39.5
28.6
33.4
Original
29.4
27.9
64.8
43.9
57.0
27.9
41.4
32.5
35.4
Adaptive RMU
w/ RNA
31.3
30.2
58.8
34.6
56.1
28.7
38.6
28.4
34.2
Original w/ RNA
27.8
27.2
61.9
37.0
57.8
28.9
34.5
28.8
30.6
RSV
31.3
30.1
59.8
37.6
58.1
28.2
35.9
29.3
31.7
Preference
Optimization
Original w/ RNA
28.8
27.7
62.4
32.2
56.4
25.6
42.5
27.2
40.5
NPO+KL
29.6
29.1
61.7
49.7
56.3
25.8
42.5
27.2
39.9
Original
27.0
28.3
61.0
52.9
55.9
25.4
41.3
27.0
29.2
NPO+MSE
w/ RNA
30.6
29.8
59.7
32.4
55.8
27.0
41.2
27.0
26.1
Original
29.1
27.9
43.4
53.9
25.1
38.0
28.4
35.3
DPO+KL
w/ RNA
29.4
28.4
59.6
36.1
55.0
27.0
37.3
26.9
37.7
Original
28.0
27.3
55.1
33.5
55.1
23.9
36.9
26.4
32.4
DSO+MSE
w/ RNA
33.3
33.2
51.7
49.6
52.3
25.2
25.3
27.0
26.2
Original
26.5
27.0
62.2
29.7
55.7
27.1
43.0
26.3
37.4
SImNPOKKL
RNA
29.6
30.4
62.5
39.2
57.3
25.7
42.4
28.0
41.4
Original
27.6
27.2
61.2
32.3
55.1
26.7
41.9
26.6
27.5
SimNPO+MSE
w RNA
31.5
32.6
60.5
37.5
57.0
26.0
41.8
26.1
36.8
31

---

Published in Transactions on Machine Learning Research (04/2026)
Table 5: Selected value of (x10-2) for different methods across n n-gram similarities.
R-gram
RMU
Adaptive RMU
RSV
DPO+KL
DPO+MSE
NPO+KL
NPO+MSE
SimNPO-KL
SimNPO+MSE
2
3.0
8.0
5.0
1.8
1.0
1.4
1.4
1.4
1.8
4 3.0
7.0
5.0
1.8
2.0
1.2
1.8
1.4
1.6
8
3.0
8.0
5.0
1.8
2.0
1.4
1.8
1.4
1.6
16
3.0
6.0
5.0
1.8
2.0
1.4
1.6
1.4
1.6
Table 6: Performance of original vs. RNA models on Table 7: Performance of original vs. RNA models on
WMDP (avg. Biology & Cyber), MMLU, and per- WMDP (avg. Biology & Cyber), MMLU, and per- turbed MMLU (4-gram).
turbed MMLU (2-gram).
Method
WMDP
MMLU1
Pert. MMLU1
Mothod
WMDPJ
MMLU
Pert. MMLU/
RMU
w/ RNA
Original
28.7 (+0.0)
28.7
57.0
(+0.0)
52.1 (-0.6)
$2.7
RMU
Original
w/ RNA
28.7 (-0.0)
28.7
57.0
(-0.0)
(-8.)
18.3
Adaptive RMU
Original
28.6
56.6
19.3
Adaptive RMU
Original
28.6
56.6
44.1
w/ RNA
30.0 (- 1.4)
56.4 ( 0.2)
54.7 (15.4)
w/ RNA
30.4 ( 1.8)
56.5 0.1)
50.0 ( (.6))
Original
56.3
Original
28.3
56.3
49,9
RSV
w/ RNA
30.9
2.6)
56.5 (10.2)
56.4 (13.4)
BSV
w/ RNA
30.9
2.6)
56.5
0.2)
54.2 (1 (14.9
NPO KL
Original
27.2
55.8
25.2
NPO KL
Original
27.2
55.8
24.6
RNN
27.7
(-033
55.5
(-0.3)
48.1
(122.99
w/ RNA
27.0
(-0.)
56.0
(:2.)
42.5 (1:17.)
Original
26.2
56.2
11.3
Original
26.2
56.2
39.2
NPO+MIE
w/ RNA
28.0
(-1.8)
(-0.1)
47.7
(+3.4)
NSE-MSE
w/ RNA
(-1.)
(-0.2)
(+1.2)
DPO+KL
Original
27.1
53.7
48.3
DPO+KL
Original
27.1
53.7
42.1
w/ RNA
29.7 ( 2.6)
54.1
(10.4)
50.2 (1 1.9)
w/ RNA
29.7 ( 2.6)
54.1
( 0.4)
43.7(1 1.3)
Original
53.5
Original
26.0
53.5
26.1
DPO+MSE
w/ RNA
28.9
2.9)
$3.6
(10.1)
52.0 (124.6)
MSO
w/ RNA
29.2 3.2)
53.0
0.5)
55.6 29.5)
SimNPO KT.
Original
26.8
55.9
33.7
SiaNPO KL
Original
28.8
55.9
31.9
RNA
27.6 ( 0.8)
55.6 (-033
47.0 (1.33)
w/ RNA
27.6 ( 0.8)
5.6 (-0.)
38.1 (46.2)
SiSNPO+MSE
Original
w/ RNA
28.7 1.6)
27.1
56.9
(+0.1)
29.1
(+25.4)
SimNPO+MSE
Original
w/ RNA
27.1
5.1)
56.7(+0.8)
55.9
53.4 (+23.3)
30,1
Table 8: Performance of original vs. RNA models on Table 9: Performance of original vs. RNA models WMDP (avg. Biology & Cyber), MMLU, and per- WMDP (avg. Biology & Cyber), MMLU, and per- turbed MMLU (8-gram).
turbed MMLU (16-gram).
Method
WMDP
MMLU1
MMLU1
Method
WMDP|
MMLU
MMLU
Original
28.7
37.0
44.6
Original
28.7
57.0
41.8
RMU
RMU
w/ RNA
28.7 (10.0)
57.0 (10.0)
42.8 18)
w/ RNA
28.7 0.0)
57.0 [ 0.0)
41.4 0.4)
Adastiiv RMU
Original
56.6
Adaplive RMU
Original
28.6
56.6
39.8
w/ RNA
30.0 (-1.))
56.1 (-0.2)
5L7 (+5.4)
w/ RNA
30.4
(-1.8)
56.3 (-0.1)
50.0 (+5.6)
RSV
Original
28.3
56.3
16.1
RSV
Original
28.3
56.3
13.7
RNA
30.9
(-2.6)
58.5 (+0.2)
48.1 (+1.7)
w/ RNA
25.7
(-0.4)
56.8 (-0.5)
412 (+1.5)
Original
27.2
56.8
29.6
Original
27.2
55.8
31.2
NPO+KL
w/ RNA
27.7
(-0.5
(-0.3)
39.0 (19.4)
NPO-KL
w/ RNA
27.7 (-0.)) 0.5)
55.5 (-0.3)
(+7.0)
NPO+MSE
Original
26.2
56.2
37.2
MSE
Original
26.2
56.2
36.3
w/ RNA
27.3 ( 1.1)
56.0
( 0.2)
37.2 (10.0)
w/ RNA
27.6
1.4)
56.1 ( 0.1)
36.7 (1 0.4)
DPO+KL
Original
53.7
DPO+KL
Original
27.1
53.7
35,9
w/ RNN
29.7
(-2.6)
54.1
(+0.1)
41.5 (+1.7)
w/ RNA
29.7
(-2.6)
511
(-0.4
36.5 (+0.6)
DPO+MSE
Original
26.0
53.5
29.1
DPO+MSE
Original
26.0
53.5
32.6
RNA
29.2
(-3.2)
53.0 (-0.5)
54.0
(+24.9)
w/ RNA
29.2
(-3.)
53.0
(-0.5)
52.2 (+19.6)
Original
26.8
56.9
32.7
Original
26,8
55.9
31.1
SinNPO<KL
w/ RNA
27.6
(-0.)
(-0.3)
36.2 (13.7)
SimNPO+KL
w/ RNA
(-0.8
(-0.3)
(+2.6)
MIE
Original
27.1
50.9
29.6
SinNPO+
Original
27.1
55.9
33.2
w/ RNA
32.2 ( 5.1)
56.7
(1 (10.9)
46.1 16.5)
w/ RNA
32.2 5.1)
58.7
0.9)
44.4 ( 11.2)
F Effects of Randomizing Different Latent Spaces
In this section, we study the effects of perturbing random noise into the representations at different latent layers.
32

---

Published in Transactions on Machine Learning Research (04/2026)
Sctup. Since the effects of unlearning at specific layers have been previously explored in RM methods, we focus our analysis PO w/ RNA models under the following three scenarios:
(1) Per-layer injection: We evaluate the performance of PO w/ RNA models by injecting noise into each layer, from the first to the last layer in the model.
(2) Region-specific layer injection: we inject noise into a set of layers grouped by position the network and compare performance across three configs: (i) early layers (5,6, 7), (ii) middle layers (14, 15, 16), and (iii) late layers (28, 29,320 30).
(3) Full-layer injection: We inject noise into all layers in the model.
DPO+KL
DRO+MSE
NPO+KL
50
50
45
45
RAA
40
40
HALU
Fettated pig
PMLU
35
35
PALLL
25
15
Layer
Layer
Layer
NPO MSE
KiL
SimNPO+MSE
60
55
50
50
45
45
WASP
40
HIPEL ACC AMA
40
PMLLI voo
PIFLLI ue's Ald
35
35
35
30
30
25
25
25
5
10
15 Layer
20
25
30
0
10
15
20
25
30
5
10
15
20
25
30
Layer
Layer
Figure 13: Per-layer injection: accuracy of RNA models on MMLU, perturbed MMLU and WMDP (avg. of Biology and Cyber) across different perturbed layers.
Hyperparameters. For (1) and (2), we inject a fixed noise with v 10-2 For (3), we perform grid search for (10-3,2 3,2 10-3,4 4,4 x 10-3 x 10-3 8 10-3, 10-
15 demonstrate that RNA generally improves the robustness of unlearned models. While Figure 13 and 14 show improvements in both settings, no consistent trend emerges across all methods.
Results. Figure
Notably, models trained with MSE retain-loss achieve significant gains from R.A.A Figure 15 further shows that injecting noise inlo all layers is parlicularly effeclive al moderale noise levels (e.g., 1 x 10-3). However, as the noise scale v increases, model accuracy declines sharply. Importantly, MMLU accuracy remains stable with RNA integration, highlighting that RNA not only boosts robustnoss but also preserves general knowledge and capabilities.
G Robustness of Unlearned Models Against Prompt Attacks
The retaining process is reframed as backdoor defense against specific type of backdoor (forget-tokens). The noise injection in RNA is reminiscent of adversarial training. There is a well-known phenomenon that, when defending against one type of attack, can inadvertently create new vulnerabilities or increase susceptibility to other oftacks (Tramer & Bonch 2019 Weng et al., 20200 Kamath al. 2021) on the general capabilities. In this section, we present an analysis of whether RNA makes the model become more susceptible to other adversarial attacks.
33

---

Published in Transactions on Machine Learning Research (04/2026)
DPO+KL
N
SimNPO +KL
Original
Orgginal
Orgiial
50
RMM Hiddte Layers
RMM Earty layers
50
Earty Layers
Middle Layers
50
RAA Hiddle Layers
RMA Earty Layers
RMM Later Layers
RNA Later Layers
RAA Later Layers
40
30
20
20
10
10
WMDP
MMLU DPO+MSE
Perturbed MMLU
WMDP
MMLU
Perturbed MMLU
WMDP
MMLU
Perturbed MMLU
NSO+MSE
SimNPO +MSE
Original
60
Original
Original
KMA Barly Layers
ANA Early Layers
ANA Early Layers
KMA Middle Layers
AMA Hiddle Layers
KAA Middle Layers
KMA LaRer Layers
RAA Later Layers
9HA Later Layers
40
40
30
30
30
20
10
10
10
WMDP
MMLU
Perturbed MMLU
WMDP
MMLU
Perturbed MMLU
WMDP
MMLU
Perturbed MMLU
Figure 14: Region-specific layer injection: accuracy of RNA models on MMLU, perturbed MMLU and WMDP (avg. of Biology and Cyber) w.r.t early layers (5, 6, 7), middle layers (14, 15, 16), and later layers (28, 29, 30).
DPO+KL
NPO+KL
SimNPO KL
55
50
50
50
45
WACF
45
WMCP All PMA
WHDP will FMG.
40
40
PIKU
40
R01U1 1M4
35
35
35
30
30
30
25
25
10
2
10
10
Noise scale (x10 3)
Noise scale (x10 3)
Noise scale (x10 3)
DPO+MSE
NRO+MSE
SimNPO+MSE
WHCP
55
50
45
40
Pertubbe PPUU UI PM
45
1MM PSG.
Pertursac PRL
40
40
Partursa PMiL
35
30
25
25
(x10
Noise scale (x10
Nolse (x10 3)
Figure 15: Full-layer injection: accuracy of RNA models on MMLU, perturbed MMLU and WMDP (avg. of Biology and Cyber).
Setup. We employ four widely used adversarial altack methods to evaluate the side effects of RNA, including Greedy Coordinate Gradient (GCG; Zou et al. (2023b)), TextBugger (Li et al. 2018), DeepWorBBug (Gao ct al. 2018), and TextFooler (Jin et al.) (220)). Tooteo an adversarial word-substitution method that relies on importance scores identify and replace important words with corresponding synonyms. Text ger generates adversarial prompts by augmenting text through character-level perturbations. DeepWordBug
34

---

Published in Transactions on Machine Learning Research (04/2026)
Table 10: Accuracy under attack (AuA 1) and ROUGE-L (t) of unlearning methods on adversarial perturbed MMLU, comparing Original vs. w/ RNA. Improvements are shown in blue, drops red.
GCG
TextBugger
DeepWordBug
TextFooler
Methods
AuA
ROUGE-L
AuA ROUGE-L
AuA ROUGE-L
AuA ROUGE-L
Base
Original
40.3
33.6
39.6
52.9
Representation Misdirection
Original
33.6
63.0
30.5
81.2
38.2
76.8
50.5
85.2
RMU
w/ RNA
40.3
60.9
30.5+0.0
79.1
38.9.o.
76.4
50.1
84.2
Original
38.5
63.5
30.5
80.0
38.9
76.2
49.8
83.3
Adap. RMU
w/ RNA 43.5+5.
62.4-11
30.8+0.3
75.2-c.
39.2+o.a
72.7-a.5
50.8+1.a
79.2
Original
39.2
63.0
30.8
78.2
38.5
75.3
48.7
80.6
RSV
w/ RNA
38.2-1.0
62.5
27.0-a.s
77.1-1.1
35.0-
73.7-14
44.2-4x
83.2+2.6
Preference Optimization
Original 35.4
51.2
26.3
61.3
34.7
61.3
43.1
67.7
NPO+KL
w/ RNA 31.2-4x
55.0,4s
25.2
64.1.2.
31.2
62.3
39.2 -a.
67.0...>
Original
40.7
57.6
29.1
66.7
40.3
66.0
46.6
71.2
NPO+MSE
w/ RNA 34.3-6.4
51.6-o.
24.2 4.9
66.2 o.s
34.3
64.4
46.3 o.s
70.9 o.s
Original
30.1
47.3
27.7
58.1
34.0
57.3
41.7
61.1
DPO+KL
w/ RNA 29.8-o.3
56.449.
29.1+1.4
67.0+s.a
33.3
64.7+7.4
42.8
68.9
Original
28.0
50.2
19.2
61.1
28.4
61.5
36.8
65.9
+PO+MSE
w/ RNA 27.7-o.3
55.6Ga.4
23.5+4.3
57.5
30.5
58.4
39.642.
64.3
Original
29.1
49.9
27.0
61.1
34.7
60.7
41.7
64.2
SimNPO+KL
w/ RNA 30.1+1.
55.0.s
27. 27.0+0.0
61.7+0a
35.4.o.v
60.8
44.2
67.00 67.0+a.
Original
35.4
52.3
29.8
66.7
36.8
66.8
44.5
72.3
SimNPO+MSE
w/ RNA 38.2
58.6.a.
31.5+1.7
71.9+5.9
42.8.6.
70.1+a
48.4+a
75.7
generates adversarial prompts by introducing character-level perturbations, such as insertions, delctions, and substitutions. GCG is gradient- based attack that iteratively modifies injected adversarial tokens along the directions with respect to the largest increase of loss to construct adversarial prompts. We utilize optimal checkpoints from the main setting, as detailed in Subsection A.4
Empirical effects of unlearning against attacks. We report the accuracy and ROUGE-L score under attack of the original unlearned model and RNA models in Table 10 As attacks, all unlearning methods consistently reduce models' robustness, making the models more vulnerable to adversarial prompt attacks. For instance, the base model achieves 40.3 AuA under GCG attack, whereas unlearned models drop to the range of 30 + 39 (e.g., RMU 33.6, Adaptive RMU 38.5, RSV 39.2, NPO+KL 35.4). Similar reductions are obscrved under TextBugger (basc 33.6 vs. unlearned 26 + 31) and DeepWordBug (base 39.6 vs. unlearned 28 + 38, except. NPO+MSE: 40.3).
observe in this table, under
Effects of RNA on model robustness. We observed that RNA's impact is dependent on the underlying unlearning method, with no clear trends observed. In summary, unlearning uniformly reduces models' robustness, while RNA can partially mitigate these vulnerabilities in some cases.
H Effects of RNA on Chain-of-Thought Prompting
Chain-of-Thought (CoT; Wei el al. (2022) is one of the most commonly used prompting techniques for improving LLM reasoning capabilities. The effect RNA on CoT is fairly interesting point that might need be investigated. We conducted additional experiments on GSM8K (Cobbe et al., (021)) and GPQA (Rein) et al.) 2024) with zero-shot. 4-shot, and 8-shot CoT with Zephyr-7B model. The results shown in Table I1 demonstrated that noise added by RNA introduces minor effects CoT.
35

---

Published in Transactions on Machine Learning Research (04/2026)
Table 11: Effects of RNA on Chain-ofTThought Prompting.
Method
GSMsK
GPQA
CoT zcro-shot
CoT 4-shot
CoT 8-shot
CoT zero-shot
CoT 4-shot
CoT 8-shot
Base
Original
15.3
38.9
42.2
12.0
22.3
28.3
Representation Misdirection
Original
15.1
37.4
40.8
12.0
24.5
21.8
RMU
w/ RNNA
13.1-2a
36.5-a.9
40.6-ba
12.0-aa
24.3-n.2
24.1+2.1
Original
12.9
36.7
41.5
10.9
25.2
21.6
Adaptive RMU
w/ RNA
15.1
37.5
41.0
12.2-1
19.8
23.4...
Original
17.4
36.7
42.5
8.2
25.4
21.4
RSV
w/ RNA
16.9-
37.5-0.s
42.8+a.
10.4-2
23.2-2.2
25.6+
Preference Optimization
Original
14.2
36.2
40.1
10.4
27.0
21.6
NPO KL
w/ RNA
14.7
36.7-o.
38.9-1
9.3-
22.7-43
23.6-2.
Original
10.6
37.6
41.0
11.3
26.1
22.3
NPO+MSE
w/ RNA
11.2.a5
35.7
38.8
9.1-2a
23.4
21.4
Original
11.9
36.1
37.2
11.3
23.2
19.8
DPO IKL
w/ RNA
11.3-as
36.9
38.7+1.1
11.3-0a
23.2
19.8
Original
10.0
36.0
39.8
11.6
23.8
22.5
DPO+MSE
w/ RNA
14.9
37.5-
40.5
14.2-aa
24.3...
24.1.1a
Original
15.6
36.5
41.0
11.1
20.9
18.7
SimN
w/ RNA
17.8+2
37.5-1.0
41.8+aa
8.0-a.1
23.6-2.1
20.3+14
Original
11.0
38.2
39.5
8.2
24.5
24.3
SimNPO I MSS
w/ RNA
11.0.a
37.9
40.2
13.6.s
25.6-1.1
23.2-13
Performance of Other Models
Our experiments in the main text are based on the Zephyr-7B model, which serves as representative sctup. To assess RNA's 's genera lizatioio beyond the original setup, we conducted additional experiments using the Llama-3-8B (Dubey et al. 2024) and Mistral-7B (Jiang et al.) 2023 models on two representative unlearning methods from classes: RMU and NPO+KI, across various tasks. These results provide further empirical evidence for RNA's generalization and robustness.
Hyperparamcters. All models are fine-tuned using Adam Kingma 2014 for T = 500 update steps with a learning rate of 5 10-5, batch size 4, and maxinum sequence length of 500 for WMDP-Biology and 768 for MMDD-- The unlearned layer is fixed at = Retain weights are set to Abiology = dcybber 1200 for both models. The coefficient values are Obiology Ccyber 20 for Llama-3-8B and Obiology Ceyber 6.5 for Mistral-7B. For NPO+KL, we perform grid search over (Obiooogyy (kyyber) and select (5, 10) for Llama-3-8B and (30, 40) for Mistral-7B. For RMU w/ RNA, we set the perturbed layer to l 1= 7, tune the noise scale via grid search, and report the best performance at 7 10 (Llama-3-8D) 3-B)) and For NP w/ RNA, we also perturb layer I = 7 and select the best scales V = 6 x 10 - (Llama-3-8B) and v = 3 x 10-3 (Mistral-7B).
3 x 10 (Mistral-7BB)
Results. As shown in Table 12 across Llama-3-8B and Mistral-7B, RNA significantly enhances unlearning robustness of models while introducing small trade-off on forget performance. For forget-tasks (WMDP- Biology and Cyber), RNA slightly increases the accuracy, e.g., RMU Llama-3-8B drops from 31.4 - 34.6 (-3.2) and NPO+KL from 27.9 + 33.2 (-5.3). For retain-tasks (MMLU and Perturbed MMLU and MMLU subsets), RNA substantially improves performance, particularly on perturbed MMLU and MMLU subsets (C. Bio. and C. Sec.). For example, perturbed MMLU on Llama-3-8B with NPO+KL improves from 26.2 47.3 (+21.1), and MMLU C. Bio. from 30.5 > 55.5 (+25.0). Overall, RNA offectively recovers or enhances accuracy on cctain-tasks while slightly compromising forget-task performance, demonstrating a favorable trade-off between unlearning and model robustness.
36

---

Published in Transactions on Machine Learning Research (04/2026)
Table 12: Performance of Llama-3-8B and Mistral-7B on WMDP and MMLU, Perturbed MMLU, MMLU subsets benchmarks using RMU and NPO+KL, comparing Original vs. w/ RNA. Improvements are shown in blue, drops in red.
Llama-3-88
Mistral-7B
Models & Methods
RMU
NPO+KL +KL
RMU
NPO+KL
Original
w/ RNA
Original w/ RNA Original
w
RNA Original w/ RNA
WMDP (4)
31.4
34.6-3 a.a
27.9
33.2 nn.
31.7
31.7-0.
29.3
34.0
MMLU (1)
60.3
60.2-o.1
53.8
51.+acc
58.2
58.6-o.4
56.5
56.4-o.
Perturbed MMLU (*)
34.4
47.3
26.2
47.3+a1.1
27.2
12.2+15.a
31.4
53.5+
MMLU C. Bio. (1)
34.7
60.4
30.5
55.5
25.0
38.1+1a.
32.6
52.0+1m.4
MMLU C. Sec. (f)
29.0
33.0-4.0
30.0
46. 16.00
33.0
46.0.11.0
35.0
30.0
J Performance of RNA under Miscalibrated Unlearning
Miscalibrated unlearning refers to scenarios where unlearning is either over-unlearn, i.e., the model successfully unlearns the target knowledge but suffers catastrophic degradation in general knowledge, or under-unlearn, i.e., the model fails to sufficiently remove the target knowledge. When unlearning is under-unlearn, the backdoor signals are too weak, i.e., forget-resresentations are not well-aligned with random vectors, making them less harmful when they appear in retain-queries. Over-unlearn occurs when the unlearning methods fail to distinguish between forget and retain knowledge, leading to catastrophic degradation of both. In such cases, random noises injected by RNA may be either redundant (for small v) or recover both forget and retain knowledge (for large v). Theoretically, RNA is a variance reduction defense against sensitivity caused by forget-tokens, not a method for miscalibrated unlearning strength. When unlearning is poorly calibrated, smoothing from RNA becomes ineffective. We conduct an empirical analysis of these two cases to evaluate our intuition. Results are shown Figure I and Figure we found that RNA fails to enhance retain-robuslnnss when unlearning is poorly calibrated.
14 Overall,
Setup. We employ MUSE-NEWS TARGET the base model for unlearning. MUSE-NEWS TARGET is the Llama-2-7B (Touvron et al. two representative unlearning methods, RMU and NPO+KL.
2023 model finc-tuned on the News corpus (BBC news articles). We employ
Hyperparameters. For RMU, we perform a heuristic search over the coefficient c E [100, 120, 130, 140, 150]. We set the retgit-weight a, 1200 (coefficient of forget-loss), forget-weight ay = 1.0 (cocfficient of retain- loss), T = 500 gradient steps, unlearn with layer 1 = 7, RNA noise added at layer 7, learning rate 2e 5, maximum sequence length 256. MUSE-NewS forget-set is used as Dj, Wikitext used as D,. For NPO+KL, we search over ar) E [(1, 1), (5, 1), (10, 1), (20, 1), (1, 5), (1, 10), (1, 20)], 8 is set to 0.1. For each pair (ay, we grid search RNA's noise scale y E [le - 3, 2e - 3e - 3). We report VerbMem and KnowMem.
K Limitations
We posit the following limitations of this study and discuss potential future works.
We have evaluated our methods primarily on WMDP, a widely used and representative benchmark. Wo acknowledge the existence of other benchmarks, such as MUSE (Shi el al. 2025) and TOFU (Maini el al. 2024), but these are less suitable for our experimental setup. Specifically, TOFU is designed to remove the influence of specific data points, making it less applicable in generative settings. While MUSE could be suitable, previous work (Shi et al., 2025) has shown that methods evaluated on MUSE often exhibit over-forgetting or under-forgetting. Since our study focuses on retain-robustness, which requires careful balance between forgetting and retaining, MUSE is not ideal. These factors make it challenging to apply MUSE and TOFU in our current experiments.
https://hugginggface. cw/.use- bench/MUSh-cceeeeeeeeeeeeeee Mewssssseesssaeeeeeeeeeeeee
37

---

Published in Transactions on Machine Learning Research (04/2026)
Table 13: Performance of original RMU unlearned models and RNA models under over-unlearn and under- unlearn in MUSE News.
Method
Verb.Mem KnowMem, KnowMem, (benign) f K KnowMem, (perturbed)
Base model (MUSE-news target)
57.2
64.2
64.2
51.8
Original
57.3
65.5
55.0
51.0 (under-unlearn)
RMU (c 100) w/ RNA le-3)
56.7
66.1
56.0
49.3
w/ RNA
3)
65.7
50.8
w/ RNA (v 3c 3)
56.7
66.1
55.8
50.4
Original
56.5
66.1
55.1
50.2 (under-unlearn)
RMU (o= 110) w/ RNA 30-3)
66.1
53.7
50.7
w/ RNA 20- 33
64.3
49.8
w
RNA (v = 30- 3)
55.3
65.0
54.9
50.8
Original
56.2
64.1
55.8
49.6 (under-unlearn)
RMU (c 120) w/ RNA le-3)
55.8
65.9
55.2
50.1
w/ RNA 20-3)
55.9
66.3
55.3
50.2
w/ RNA ( 30 3)
56.0
66.1
53.9
50.8
Original
56.2
49.0
45.4 (under-unlearn)
w/ RNA l0 le-3)
65.2
56.3
49.8
RMU (c 130)
w RNA 20-3)
65.2
50.0
w/ RNA (v = 30-3)
55.0
66.0
56.0
50.4
Original
53.9
43.2
36.3
38.1 (under-unlearn)
RMU (c 140) w/ RNA le-3) 3)
62.9
48.2
w/ RNA = 20-3)
55.0
62.8
52.8
49.1
w/ RNA (v 30-
55.7
61.9
54.5
50.2
Original
49.2
13.7
18.0
18.7 (over-unlearn)
w RNA
3)
48.6
37.8
RMU (e= 150)
RNA (v 20 3)
54.2
51.4
44.2
39.6
RNA (V- 3--3)
54.3
53.9
51.0
45.3
Due to computational constraints, experiments are conducted only on the 7B 8B models and with up- dates to a limited set of layer parameters, which may risk overlooking interesting aspects of generalization. Although RNA has demonstrated effectiveness, relies heavily on hyperparameter grid search to identify an optimal noise scale, making it computationally expensive for extremely large models with hundreds of billions of parameters.
L Al Usage Declaration
AI tools were used for grammar checking and formatting of tables and figures. our best knowledge and belief, we hereby declare that, all technical content and implementations were written by the authors.
38

---

Published in Transactions on Machine Learning Research (04/2026)
Performance of NPO+KL unlearned models and NPO+KL w/ RNA models under over-unlearn Table 14:
and under-unlearn in MUSE News.
Method
VerbMemy KnowMeny Know Mem, (benign)
Know Mem, (perturbed)
Base model (MUSE-news target)
57.2
64.2
64.2
51.8
Original
57.5
61.4
52.6
48.8 (under-unlearn)
w/ RNA (r 1e-3)
56.8
61.0
52.0
47.6
NPO+KL (1,1)
w/ RNA (u= 20-3)
56.8
61.0
52.8
46.6
w/ RNA (v 3e-3)
56.5
60.2
52.7
46.5
Original
57.5
59.5
52.7
47.8 (under-unlearn)
w/ RNA (v le-3)
58.0
59.9
53.8
47.0
NPO+KL (1,5)
RNA (r 20-
56.3
60.4
53.6
46.8
w/ RNA (v 30-3)
57.5
59.8
53.0
46.7
Original
58.6
60.0
52.6
45.7 (under-umlearn)
NPO+KL 10) w/ RNA (v le 3)
58.7
59.8
53.0
46.4
RNA (v 2--3)
58.3
59.9
53.0
46.7
w/ RNA (r 30 3)
58.9
60.1
53.9
46.5
Original
57.7
59.5
54.3
47.4 (under-unlearn)
NPO+KL (1, w/ RNA (v 1e-3)
58.2
61.2
53.8
47.6
w/ RNA (v 20-3
58.8
62.8
53.5
46.5
w/ RNA (V 3--3)
58.2
62.8
54.7
47.7
Original
56.8
59.9
51.8
(under-unlearn)
w/ RNA (v 1e 1e-3)
55.7
60.0
52.6
47.8
NPO+KL (5,1)
w/ RNA (v 20-3)
56.9
59.4
53.1
48.5
w/ RNA (v 30
56.7
60.1
51.9
47.7
Original
60.6
53.1
48.4 (under-unlearu)
NPO+KL () w/ RNA (v le-3)
56.7
61.5
52.3
48.2
w/ RNA (v 20 3)
55.7
59.9
52.1
48.1
w/ RNA (v 3
57.1
59.3
52.6
48.3
Original
56.4
58.2
52.1
48.2 (under-unlearn)
w/ RNA (v 10-3)
56.6
59.3
52.1
47.8
NPO+KL (20, 1) w/ RNA (v 20-3)
56.6
60.6
51.9
48.1
w RNA (v 3c
56.1
60.8
51.7
47.9
39


Reviews:
[{"review_id": "b5J3mGESXu", "reviewer": "Reviewer_97nx", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper studies retain-robustness in LLM unlearning, showing that existing methods such as Representation Misdirection (RM) and Preference Optimization (PO) can become brittle when retain queries inadvertently include forget tokens. The authors reinterpret unlearning as a backdoor attack problem and propose Random Noise Augmentation (RNA), a lightweight method that injects Gaussian noise into retain representations to improve robustness while largely preserving forget and retain performance.", "claims_and_evidence_yn": "No", "claims_explanation": "The theoretical analysis relies on strong assumptions, particularly modeling forget-token effects as small Gaussian perturbations in latent space, which may oversimplify real representational dynamics and limit the rigor of the guarantees. Assuming any Gaussian perturbation, this can be anything, and there is no reason to back the fact that unlearning results in that specific behavior. \n\nThe backdoor framing, while compelling, is somewhat conceptual and may overstate the equivalence between standard poisoning attacks and unlearning objectives. Additionally, RNA introduces a trade-off by reducing robustness to relearning, but this trade-off is not deeply analyzed or theoretically characterized. The \"adversarial training\" analogy is not really accurate, this is smoothing in the latent space. \n\nIn essence, I am not sure, given the loose assumptions,  if these findings are necessarily relevant to the unlearning problem. The notion of “retain-robustness” remains insufficiently formalized, and the term “misbehave” is used descriptively rather than operationally.\n\nThe probability bound in Theorem 2 is derived under multiple approximations and independence assumptions. It feels heuristic rather than rigorous.", "audience_yn": "Yes", "audience_explanation": "The conceptual reframing of unlearning as a backdoor attack-defense process is insightful and provides a unifying perspective across RM and PO methods", "requested_changes": "- The paper would benefit from a precise, measurable definitionof \"misbehaving\" and the Robustness \n- Clarify what class of perturbations the guarantees apply to. Why is the Gaussian perturbation relevant to unlearning?\n- Define the perturbation set explicitly (single-token insertion, semantic neighbors, multi-token patterns, etc.) and evaluate robustness under that formalized setting to strengthen the claims.\n- Clarify the backdoor behaviour that is \"learnt\", what is it? and why is it \"misbehaving\" assuming the authors define what this term means", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "HeJhVsPl0M", "reviewer": "Reviewer_GJHe", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposes a novel theoretical framework that reframes the unlearning process as a backdoor attack–defense problem. Based on this perspective, we reinterpret the retaining process as a form of backdoor defense and introduce Random Noise Augmentation (RNA), a lightweight, model-agnostic and method-agnostic approach with theoretical guarantees for enhancing the robustness of unlearned models.", "claims_and_evidence_yn": "Yes", "claims_explanation": "Although interpreting unlearning as a backdoor problem may initially appear somewhat non-intuitive, we provide solid theoretical justification for this perspective. Moreover, the proposed framework is not merely conceptual—it leads to practical algorithms with clear applicability and is supported by extensive empirical validation.", "audience_yn": "Yes", "audience_explanation": "The proposed robustness-enhancing method is both useful and insightful.", "requested_changes": "The contribution of this paper is acceptable for me.", "broader_impact_concerns": "Theoretical and empirical analyses provide a valuable foundation for developing more secure and reliable machine learning systems.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "OJoXdYBzvw", "reviewer": "Reviewer_SdC2", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper points out a “retain-fragility” issue in current LLM unlearning methods: inserting non-adversarial “forget-tokens” into otherwise benign retain queries can cause large performance drops. To explain this behavior, the authors propose a framework that views unlearning as analogous to a backdoor/poisoning process, where the forget set acts like poisoned data. Building on this framing, they introduce Random Noise Augmentation (RNA), which injects Gaussian noise into latent representations during training to smooth the decision boundary. Empirically, RNA improves robustness on perturbed retain queries across multiple model families (Zephyr, Mistral, Llama-3).", "claims_and_evidence_yn": "Yes", "claims_explanation": "The empirical results support the existence of retain-fragility and show RNA helps on the perturbed retain evaluations.", "audience_yn": "Yes", "audience_explanation": "This paper identifies a practical failure mode in LLM unlearning and proposes a simple mitigation strategy that is relevant for safety areas.", "requested_changes": "Strengths:\n- Clear identification of a practical failure mode (retain-fragility) that is easy to reproduce and matters for deployment.\n- Simple, method-agnostic mitigation (RNA) that appears to transfer across multiple model families.\n- Conceptual contributions: unlearning and backdoor/poisoning, which may inspire more robust objectives/evaluations.\n\nWeaknesses:\n- Improved retain-robustness comes with higher relearning success in Table 3, which may undermine the core goal of unlearning.\n- Without comparisons to standard methods (dropout/weight decay) or adversarial training, it’s hard to identify whether RNA works.\n- Selecting $\\nu$ without a perturbed validation set, and clarifying any cost/impact on clean retain performance.\n\nRequested Changes:\n1. Make the safety trade-off explicit: Table 3 shows higher relearning after RNA; clarify the intended threat model and, if possible, mitigate this without losing retain-robustness.\n2. Add simple, matched baselines (e.g., dropout/weight decay or retain-side adversarial/augmentation) to show RNA’s gains are not just generic regularization.", "broader_impact_concerns": "N/A", "additional_comments": "N/A", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)