The Unseen Threat: Residual Knowledge in Machine Unlearning under Perturbed Samples
Authors: Hsiang Hsu
Pradeep Niroula
Zichang He
Ivan Brugere
Freddy Lecue
Chun-Fu Chen
Venue: NeurIPS 2025 poster ()
Keywords: Machine Unlearning
robustness
adversarial
perturbation
disagreement
residual knowledge.
Primary area: social_and_economic_aspects_of_machine_learning

Abstract: Machine unlearning offers a practical alternative to avoid full model re-training by approximately removing the influence of specific user data. While existing methods certify unlearning via statistical indistinguishability from re-trained models, these guarantees do not naturally extend to model outputs when inputs are adversarially perturbed. In particular, slight perturbations of forget samples may still be correctly recognized by the unlearned model---even when a re-trained model fails to do so---revealing a novel privacy risk: information about the forget samples may persist in their local neighborhood. In this work, we formalize this vulnerability as residual knowledge and show that it is inevitable in high-dimensional settings. To mitigate this risk, we propose a fine-tuning strategy, named RURK, that penalizes the model’s ability to re-recognize perturbed forget samples. Experiments on vision benchmarks with deep neural networks demonstrate that residual knowledge is prevalent across existing unlearning methods and that our approach effectively prevents residual knowledge.

The Unseen Threat: Residual Knowledge in Machine Unlearning under Perturbed Samples
Hsiang Hsu' Pradeep Niroula', Zichang He Ivan Brugere", Freddy Lecue", and Chun-Fu Chen PPMMrganCCase Global Technology Applied Research PJPMorganChase Al Research
Abstract
Machine unlearning offers a practical alternative to avoid full model re-training by approximately removing the influence of specific user data. While existing methods certify unlearning via statistical indistinguishabiiity from re- re-trained models, these guarantees do not naturally extend to model outputs when inputs adversarially perturbed. In particular, slight perturbations of forget samples may still be correctly recognized by the unlearned model even when re-trained model fails to do so so-revealing a novel privacy risk: information about the forget samples may persist in their local neighborhood. In this work, we formalize this vulnerability residual knowledge and show that it is inevitable in high-dimension mitigate this risk, we propose fine-tuning strategy, named RURK, that penalizes the model's 's ability to re-recognize perturbed forget samples. Experiments on vision benchmarks with deep neural networks demonstrate that residual knowledge is prevalent across existing unlearning methods and that our approach effectively prevents residual knowledge.
settings, To
Introduction
The widespread of user data in training machine learning (ML) models has raised significant privacy concerns, particularly when user data is memorized by models such as deep neural networks, and can later be extracted reconstrueted (Carlini et al., 2023; Li et al., 2024). This memorization violates regulations such as the "Right to be Forgotten" in EU's GDPR (Voigt and Von dem Bussche, ML service provider must not only
2017), which mandates that, upon user's removal request, delete the user data from databases but also ensure its removal from the ML models themselves (Shastri et al., 2019). As result, simply deleting the data is often insufficient; instead, re-training the model from scratch without the specific user data is necessary. However, this re-training process is computationally expensive and impractical in real-time or large-scale settings (Ginart al., 2019).
To address this challenge, machine unlearning has been proposed as a more scalable alternative that approximately removes the influence of the specific data (referred to forget samples) from a pre-trained model, as substitute for avoiding full re-training (Cao and Yang, 2015; Bourtoule al., 2021). This approach, known approximate unlearning, often certified through statistical indistinguishability between the unlearned model and one re-trained without the forget samples (Guo al., 2020; Chourasia and Shah, 2023).
Theoretical formulations of approximate unlearning (cf. $ 2) suggest that an unlearned model should behave similarly to re-trained model on forget samples, for example, by producing similar predic- tions or classification accuracy. However, these guarantees typically apply only to the original samples and do not extend to their proximities, especially under imperceptible adversarial perturbations. In complex hypothesis spaces, even minor input changes can cause the unlearned and re- re-trained models
"Corresoondence to: Hsiang Hsu <hsiang. hau@jpmcaase... com>.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Disagreement
trained Model
Offir Sanvles cosi
Forget Sample (Inconsistent)
Othir Serulee 1oea wod
Paryet Samelo régreedl
Forget Sample (Consstent)
Pertu bad Region
Decision Boundaries
Original
[Timamm
Modal
Hypothesis Space
Unleerrie ned Model
Figure 1: The re-trained (brown) and unlearned (green) models statistically similar but may have slightly different decision boundaries (Ieft), lcading disagreements on forget samples (right). Checkmarks and crosses on the images indicate correct and incorrect predictions from the re-trained model (top) and the unlearned model (bottom), respectively. Idcally- as shown with forget sample 1 b-both models should behave consistently across the original and all perturbed inputs. Residual knowledge in machine unlearning is illustrated by comparing prediction correctness: for forget sample 2, both models agree the original sample, but the unlearned model correctly predicts more of its perturbed variants. see Appendix B.5 for experimental details.
to disagree in their predictions, creating an additional layer of privacy risk. A particularly concerning case is when a slightly perturbed forget sample is still correctly classified by the unlearned model but not by the re-trained one. This reveals the presence of residual knowledge- latent traces of the forget samples that still persist in the unlearned model (cf. Figure 1). Residual knowledge is a prevalent issue. For instance, on the CIFAR-I0 dataset, when subjected to small perturbation norm (+s 0.03), over 7% of the forget samples still exhibit residual knowledge (cf. Appendix C.4).
In this paper, we study how adversarially perturbed inputs affect the indistinguishabilit between unlearned and re-trained models in the classification setting. In $ 3, we show that adversarial examples can reliably distinguish between the two models, even under certified approximate unlearning. We formalize this observation by demonstrating that adversarial attacks can induce probabilistically distinguishable outputs. Further, using geometric probability (Talagrand, 1995), we prove that such disagreement is inevitable in high-dimensional input spaces. These findings underscore the need to explicitly incorporate robustness against such local disagreement into unlearning frameworks.
To capture this risk, $ 4 introduces the notion of residual knowledge, which quantifies the likelihood that an unlearned model retains predictive traces of forget samples under perturbation. As a more tractable proxy for disagreement, residual knowledge exposes a vulnerability not addressed by current certification methods. To mitigate this, propose RURK, fine-tuning strategy for Robust Unlearning that suppresses Residual Knowledge while maintaining accuracy on the rest of the samples. Our approach identifies and penalizes perturbed inputs that the unlearned model still classifies correctly, thus enhancing robustness against residual knowledge. We empirically validate the existence of such local disagreement and residual knowledge across multiple unlearning algorithms in $ 5. We further demonstrate that our fine-tuning strategy effectively reduces residual knowledge on standard vision datasets using deep neural networks. To the best of our knowledge, this the first work to uncover this novel privacy risk and provide a scalable solution to address it. We conclude in $ 6 with a discussion of limitations and future research directions.
Omitted proofs, additional explanations and discussions, details on experiment setups and training, and additional experiments are included in Appendices A, B and C, respectively.
2 Background and related work
Let 2 (s;= (x,, y:) ):) be a training dataset with x; E X' S Idd and yi E y. The hypothesis space H 2 (hw : X + y; w E W} consists of models parameterized by w. We use h and w interchangeably refer to model in H. Let f W S R be the loss function, and define the empirical risk as L(W,S) = 1 Σn_ 2(w, st). A (randomized) learning algorithm A : S + H. such as Stochastic Gradient Descent (SGD), returns a model minimizing L(w,S) and induces a distribution over H. We denote the f,- norm by ||z]] and the l ball of radius 7 centered at x by (x,,x)) +) A {z X' ||z x|Ip < T). Also, let 1(-) be the indicator function, surf (2) the surface area of a set Z in Rd. and (-) the big-O notation.

---

2.1 Machine unlearning
The forget set S S S contains training samples to removed. Machine unlearning is modeled as a randomized mechanism ((ASS) S, that removes the influence of S from a pre-trained model A(S) (Xu al., 2023). Like the learning algorithm itself, the oulput of the unlearning mechanism can be regarded as random variable. A simple approach adds Gaussian noise to the model weights: M(w, S,S,) = w + on, with n ~ (0, I wi) (Golatkar al., 2020a). However, as a increases, the model becomes increasingly independent of the training data, potentially degrading performance. To preserve utility, unlearning should retain accuracy on the retain set S. S Sy A naive yet effective strategy is to re-train the model from scratch on the retain set, i.e., M (A(S), S, = A(S.). While this achieves exact unlearning. it is often impractical due to the high computational cost of re-training for each unlearning request. Therefore, the core objective of machine unlearning is to efficiently eliminate the influence of S while preserving utility on S,,, all without full re-training.
This objective has spurred vast literature on machine unlearning. Here, we outline the broad trends and defer details of specific algorithms to $ 5. For comprehensive surveys, see Nguyen et al. (2022); Xu et al. (2023); Wang et al. (2024). Initial efforts focused on image classifiers, aiming to remove the influence of specific training images from pre-trained model (Ginart et al., 2019; Wu et al., 2020; Neel al., 2021; Sekhari et al., 2021; Izzo et al., 2021; Fan et al., 2023; Kurmanji et al., 2024; Goel et al., 2022; Zhang et al., 2024; Kodge et al., 2024). This idea was later extended to erasing abstract concepts (Ravfogel et al., 2022; Belrose et al., 2023), as well as adapting unlearning to broader model classes, including image generators (Li et al., 2024; Gandikota et al., 2023) and large language models (Fldan and Russinovich, 2023; Yao et al., 2024; Liu et al., 2025; Jang et al., 2022; Wang et al., 2023). Another line of work develops unlearning methods tailored to specific model families-suee such as linear classifiers (Guo et al., 2020), kernel methods (Golatkar et al., 2020b; Zhang and Zhang, 2022), and tree-based models (Brophy and Lowd, 2021; Schelter et al., 2021).
2.2 Certification of machine unlearning
valid unlearning mechanism ensures that the unlearned model ((A(S), S, Sy) is statistically similar to a re-trained model A(S,). We denote the unlearned and re-trained models as random variables M and A, with corresponding distributions PM and PA. respectively. This requirement is formalized via (c, indiistinguuishabiii:::
Definition 1 ((e, 8) (iiiniiistinguihhbbiltt.... Let X and Y be two random variables over a domain Ω.
X and Y are said to be ( -iddsiiiidssiinuushhhblle also denoted as X c.5 Re Y, ifo all T S
c (PrrY TT - 5) s Pr[X E T) S Pr|Y E ET 5.
(1)
This notion underlies differential privacy (DP) (Dwork and Roth, 2014) when X and Y are outputs on neighboring datasets. It also quantifies reproducibility of empirical findings when applied to models trained on independent samples from the same data distribution (Kalavasis et al., 2023; Impagliazzo et al., 2022; Bun et al., 2023).
Guo et al. (2020) were among the first to introduce certified machine unlearning using (c, 6)-
indistinguishabilty. An unlearning algorithm M satisfies (c, d)-unlerrning if M (A(S), Sy) c.6 A(S,); this reduces to exact unlearning when e = ó = O. Indistinguishability can be measured through various probability divergences as well. For instance, Chourasia and Shah (2023) and Chien et al. (2024) proposed (a, Rénéé unlearning, which holds when the a-Rényi divergence" between PM and PA is bounded by c. and can be translated into (c.l)-unl. Indeed, (a, Rénnyi unlearning can be converted to (c + log (1/8)/(a - 1). 6)-unlear, for any 0 < 8 < 1 (Mironov, 2017). Both frameworks assume uniqueness or unique stationary distribution of the empirical minimizer, which may limit their applicability in complex model classes.
Instead of comparing full model distributions Pa and PA. more practical certification framework evaluates unlearned models with readout functions f H x S > IR that an adversary might use to distinguish unlearned from re-trained models. Indistinguishability is then assessed via the output distribution of f. using divergences such as the Kullback-eibler (KL) divergence". The certification
For a > 1, the RéRényi divergence (Rényi, 1961) is defined as Da (PIQ) e بي log BQ [6)"]
The KL divergence (Kullback and Leibler, 1951) is defined Daa (PIQ) A [Br(oo(P/()).
3

---

condition requires DKL (Pr T)]I (r[f(A,T) < for any subset T S S, such as the forget, retain, or even hold-out test sets (Nguyen et al., 2020; Golatkar al., 2020a). This formulation flexibly captures different behaviors: ʃ could represent binary classifier (e.g., for Membership Inference Attack (MIA)) (Fan et al., 2023), utility metrics like accuracy, or re-learning time- the training epochs needed to re-learn S, (Golatkar et al., 2020a). Focusing on readout functions offers a more tractable and empirically grounded certification approach, especially for complex models.
3 Model indistinguishability and disagreement over sample perturbation
Statistical indistinguishability between unlearned and re-trained models can be certified theoretically via (c, 6)- or (a,e)-Rényí unlearning-or empirically using a readout function (cf. $2). This, indeed, ensures similarity in model weights or outputs on forget/retain samples. However, such guarantees do not readily extend to perturbed inputs, even with imperceptible adversarial perturbations.
Several studies have investigated the vulnerability of unlearning algorithms to adversarial manipula- tion. Marchant et al. (2022) show that adversarial inputs can significantly increase the computational cost of unlearning". while Pawelczyk et al. (2025) find that poisoning attacks can obstruct complete forgetting (ef. Appendix B.1). Zhao et al. (2024) further demonstrate that even small number of malicious unlearning requests can weaken the adversarial robustness of the resulting model. More recently, Xuan and Li (2025) propose an attack that manipulates unlearned model such that its outputs on forget samples resemble those of the original model (see Proposition 3 and Definition 4 in their paper). However, these works primarily study how adversarial perturbations affect the unlearned model itself, whereas our goal is to evaluate the distinguishability between the unlearned and re-trained models.
This paper intends to explore a new dimension of how adversarial examples affect unlearning- specifically, how such examples may behave differently when fed to unlearned model versus re-trained one, even when the two satisfy statistical indistinguishabiiity. Remarkably, despite this indistinguishabiity.. it remains possible to craft adversarial inputs that distinguish between the two, revealing a novel privacy risk. This phenomenon, related to the transferability of adversarial examples (Tramèr et al., 2017), remains largely unexplored in the unlearning literature. The proofs of the propositions in this section are included in Appendix A.
3.1 Distinguishability of model output with adversarial examples
We begin by considering adversarial examples generated against either the unlearned or the re- re-trained model. The process of finding an adversarial example for a given input x E X' can be formalized as a read-out function 9x H. - X, which may be deterministic or randomized. For instance, the minimum f2-norm perturbation for binary linear classifier, given by 9x( 9x(w) = (x (xTxxx/x x)w/ h//l///e and the Fast Gradient Sign Method (FGSM) (Goodfellow et al., 2016) are deterministic. In contrast, methods such as Projected Gradient Descent (PGD) (Madry et al., 2018) or random perturbations within (x, r) are randomized. Since generating adversarial examples can be seen as post- processing of the model, the indistinguishability of adversarial examples can, in principle, be derived from the indistinguishability of the models themselves. Indeed, (e, ô) -i-indistinguushabilty is preserved if two random variables and are (c. -iiidistinguuihablee then so are fX) and fY) for any (deterministic randomized) function f. However, when considering adversarial examples of specific model realizations drawn from either the unlcarning or re-training processes (cf. Lemma A. the level of indistinguihhabiity can degrade, formalized in the following proposition.
under arbitrary post-processing
Proposition 1 (Adversarial example on model is less indistinguishable). Suppose the unlearned M (((SS), S, St) and re-trained A(S,) models are (e, indistinnnguii and let x be a fixed sample. Then with probability 25/(1- - "),, the adversarial example found against the models m M or a ~ A satisfies, for all X' C
Pr(gx(m) E xe Pr](a<(a) E X') or e2 Pr(gx(a) E x*] < Pr(ge(m) E X').
Proposition 1 shows that even if the unlearned and re-trained models satisfy approximate unlearning certified by (e, ()--indistinguisaabblii the adversarial examples gx (h) can become less indistinguish- able. Specifically, given model h, with probability 25/(1 E e) the distinguishability of the
Allouah et al. (2025, Theorem 3) report similar behavior when unlearning out-of-distributio samples.

---

adversarial examples increases by factor of two. Moreover, the indistinguishability assumption in Proposition 1 can be readily generalized to (a, c)-ééyi unlearning.
3.2 Disagreement on adversarial examples is inevitable
In the previous section, Proposition showed that even when the unlearned and re-trained models satisfy (e, S>-indissinguuihaaiiity there remains a nonzero probability that their likelihood ratio can still be exploited to distinguish them via adversarial examples. However, evaluating the bound in Eq. (2) requires computing probabilities over the entire hypothesis space H. which is often computationally intractable when H is large or complex.
To address this challenge, we instead focus on model outputs at individual samples x, which may belong to the forget or retain sets. We introduce more tractable binary metric called disagreement. defined k(x) = (x) * A(x)), where k(x) 1 indicates that the unlearned and re- re-trained models produce different predictions at x, and k(x) = 0 otherwise. Disagreement has been widely used in the machine learning literature to study model behavior (Krishna et al., 2025; Uma et al., 2021), as well as prediction stability and bias (Kulynych et al., 2023). Notably, the (c. ji--niisiinuuishabblyy guarantee over distributions can be translated into upper bound on empirical disagreement across samples. In particular, when c = = O, perfect indistinguishabiiit (i.e.. exact unlearning) implies k(x) 0 for all E S. Ideally, we seek agreement not only the training set but across the entire sample space X, including unseen or perturbed data. Yet even small nonzero values of e and 5 can result in disagreement on certain inputs, particularly under adversarial perturbations, as demonstrate in the following analysis.
To formalize this, we consider the sample space X' = -d-1 {x E Hd ||||| = 1}. corresponding to the unit sphere in Rd, where all data points are normalized to unit norm. In this setting, the disagreement function k(x) can be viewed as a binary classifier over gd-1, We aim to bound the probability that disagreement occurs not only at sample x, but also within its local neighborhood under bounded perturbations, i.e., for x' E B, 7) such that ||x- x'II, S r. This motivates the need to understand how disagreement can propagate beyond the observed dataset to its local neighborhood in the broader sample space X'. key mathematical tool for this purpose is the isoperimetric inequality (ef. Lemma A.2 and Talagrand (1995)). In probability theory and geometry, the isoperimetric inequality provides a lower bound on how the measure of a set expands when is slightly "extended." In nigh-dimensional spaces, particularly under uniform or Gaussian distributions, it formalizes the intuition that if a subset occupies a small volume, its boundary must be relatively large. In our context, this implies that even if two models disagree perturbations can still cause disagreement to spread over a much larger region in the sample space. Combining this geometric insight with the definition of (c, d)-unlernnin (cf. Definition 1), establish the following result:
only small region, small
Proposition 2 (Inevitable disagreement). Consider sample E ES Let M and A denote the unlearned and re- re-trained models, respectively, satisfying (c, J)mllarrnng Suppose the agreement region satisfies surf((x E $
[xxx) = =0]) ())/surf($dd)) = Then with probability at least
26
{ (1-0((5))
1/2
d d- (-2)- )
I I-e-c
exp
either k(x) or there exist x' E Bplx, 7) such that k(x')= = 1, where p or p p= X. X0.
When < 8- 0, the probability in Eq. (3) is zero", since the unlearned and re-trained models are identical and cannot disagree. Conversely, as + oo for a fixed 8 rerfflectngg maximal distinguisha- bility between M and Ahehe probability of disagreement approaches its upper bound of 25. More generally, for any fixed 8). the probability in Eq. (3) depends solely on the perturbation norm T: increases, the probability of disagreement rises, as adversarial examples explore a broader neighborhood around the input. Moreover, Proposition 2 holds for both p = 2 and p = oo, aligning with the common practice of measuring perturbation size using either the le (Euclidean) norm or the lo (maximum) norm in prior works, c.g., Goodfellow et al. (2014); Madry et al. (2018).
While the assumption that samples lie on unit sphere is admittedly strong, the core principle of the isoperimetric inequality-that small-volume sets necessarily have large coandaries-extendd to
*This condition can be easily satisfied in multi-class settings, i.c., |30| 2.
6 'Lt 8 = ke and by L' L'Hopital's i's rule lim 2ke
lim
28
0.
<-
I-e c-10
5

---

more realistic domains. In particular, Ledoux (2001, Proposition 2.8) generalizes this property to the unit cube, a more suitable setting for image data where pixel values are normalized. Nonetheless, Proposition 2 remains insightful, as it demonstrates that disagreement on perturbed samples can arise even under the stricter unit sphere assumption, thereby strengthening its relevance under looser conditions like the unit cube.
4 Removing residual knowledge in unlearned models
Proposition 2 demonstrates that disagreement between the unlearned and re-trained models can persist even when the unlearning algorithm satisfies the (e, -unlearnnn constraint. Although the two models may agree on the original input (x, y). it is often possible to craft adversarial examples with imperceptible perturbations (small 7) that cause them to diverge, revealing a potential privacy risk. This result arises from two sources of randomness: itself (e.g.. model initialization) and (2) that of perturbations applied to forget samples. While the proposition establishes that such disagreement is inevitable, demonstrating existence alone is queffiiiiet-quantfffylll
(I) that of the unlearning algorithm
its extent in practice is equally essential.
To that end, we fix specific realizations of the unlearned and re-trained models and define as adver- sarial disagreement' as k (x') = (m((x' A a(x') for x' ~ B,,x, 7). The expected value of this indicator gives the probability of disagreement under perturbation: E (x')] = Pr [m((x) j a(x'). Adversarial disagreement challenging to control, especially in multi-class settings, as it involves evaluating probabilities over all possible output combinations of m(x') and axx'.. To address this complexity, we next introduce a special case of adversarial disagreement called residual knowl- edge- edge-which offers more tractable measure with operational meanings.
4.1 Connecting disagreement with residual knowledge
We focus on a particularly concerning form of adversarial disagreement, i.e., [(m(x')= y, a(x') of y) for a forget sample* (x. y) E Sy. implies that a forget sample, intended be fully erased from the model, can be slightly perturbed such that the unlcarned model still correctly classifies it, while the re- re-trained model does not. This discrepancy suggests that traces of the forget samples may remain embedded in the unlearned model's decision boundary. even when formal indistinguishability guarantees are satisfied. This scenario motivates our definition of residual knowledge the persistence of information about the forget set in the predictive behavior of the unlearned model. The presence of residual knowledge reveals a subtle yet significant vulnerability: demonstrates how adversarial examples can compromise the effectiveness of unlearning, and underscores the need for stronger robustness guarantees that go beyond conventional (e, J)-unlearni
Consider a forget sample (x, y) E Sy, and let m S,, Sy) and a A(S,) be inde- pendently sampled unlearned and re-trained models, respectively. We formally define the residual knowledge around x as the following non-negative ratio:
rr ((x, v)) Pr[m((x) = y] x' iiddd B,(x, Pr(a(x')=
This definition can be naturally extended to the entire forget set Sn by averaging over all forget samples, rSy A 18,1 Eixyls, r~ y)). Among the possible cases, rr(()) > 1 is especially concerning, as it indicates that the unlearned model m is more likely than the re-trained model to correctly classify perturbed variants of forget samples. This suggests the presence of residual this sense, the
knowledge, which the main privacy risk we aim to address in this paper. case where (S)) < 1 is more tolerable, as it implies the unlearned model has lost the ability to recognize the forget samples. The case where r, (Sy) = can only occur when m = a, meaning the unlearned model m achieves exact unlearning. In practice, r, ((x, y)) can be estimated via Monte
"The randomness in k(x) arises from the stochasticity of the models, whereas that in k. (x') arises from sampling perturbed inputs x' ~ (x, r). Defining disagreement over both sources of randomness would require characterizing the distributions of A and A, an intractable task without strong assumptions (Guo al., 2020; Chourasia and Shah, 2023; Chien al., 2024).
'Access to forget samples is standard in literature (Appendix B.3) and necessary in classification unlearning. must provide allow retrieval of the data for deletion.

---

Carlo sampling. Specifically, by drawing e i.i.d. samples fx| from By(x, T), we approximate the ratio as :))
Ei. 1(m(x)==)
Σ. [[(()==))
The notion of residual knowledge is closely tied to adversarial disagreement, In particular, residual knowledge offers both upper and lower bounds on the expected adversarial disagreement between the unlearned and re- re-trained models (cf. Lemma A.4):
(r((,, Pr [a(x') = v) (1 - Pr = v)) S E (xT((') (x') S 11- (1rr(((x,))) (x) Pr (a(x') v)² (5)
As ry (r y)) + 1, the expected adversarial disagreement IE [k; (x') approaches zero. In this sense, residual knowledge serves as practical proxy for estimating the distinguishability between the unlearned and re-trained models, and provides a more tractable means to quantify and control adversarial disagreement.
4.2 RURK: Robust unlearning against residual knowledge
To address residual knowledge, we propose fine-tuning objective that simultaneously enforces unlearning and regulates residual knowledge. Ideally, residual knowledge should be close to I; however, accurately computing rS requires access re-trained model a A(S,), which is typically unavailable during unlearning. Thus, our objective is to attenuate residual knowledge and ensure that r (Sy) < 1. Recall that the numerator of the residual knowledge in Eq. (4) is Pr[m((x) = y]: thus, directly minimizing this probability alone can effectively suppress residual knowledge, regardless of the denominator Pr(a(x')
y). Based on this insight, we define the set of vulnerable perturbations as (((x,)) y). r) & {x' E (x, r) [m(x') = y). which captures perturbed samples that continue to be associated with the true label by the unlearned prodel-indication residual knowledge. In practice, we construct v such samples by adapting adversarial attack methods, such as FGSM or PGD, to solve the constrained minimization problem min f(w, (z, v)).
Given the vulnerable set, we formulate the following fine-tuning objective for RURK
LEURX (w. 8)
f(w, (x,, y))
Σ
x(w, (x, B)),
(6)
S
Sy
(X.))ES,
(x.y))Es
Term (i)
Term (ii)
where x(w, (x, v))
f(w, (x', y)), and À 2 0 is the regularization strength.
Term (i) preserves performance on the retain set, following prior work such as Neel et al. (2021); Kurmanji et al. (2024); Chien et al. (2024). Term (ii) serves regularization term that penalizes residual knowledge by discouraging the unlearned model from re-identifying vulnerable perturbations, thereby improving robustness with respect to residual knowledge. Explicitly searching for vulnerable perturbations in V((x, v).7) during each gradient step can be computationally expensive. To mitigate this overhead, we may adopt more efficient approximation by setting V((x, v). = B, (x, 7), i.e. removing all label information in the neighborhood of each forget sample. We outline the RURK algorithm in Appendix B.2.
Note that as T > O, the objective in Eq. (6) reduces to that used in prior works such as Neel al. (2021); Chien et al. (2024). When optimized via (Projected) Noisy Gradient Descent (NGD) (Chourasia et al., 2021), the resulting unlearned model satisfies (a, (a, Rééééi unlearning, where depends the smoothness and Lipschitz continuity of the loss function T.(w,S). Specifically, less smooth or less Lipschitz losses lead to larger e, making the unlearned model more distinguishable from the re-trained one (Chien al., 2024, Theorem 3.2). This reveals fundamental trade-off between model indistinguishabiiiy and robustness against residual knowledge: increasing in term (ii) of Eq. (6) enlarges the adversarial perturbation space, which makes the adversarial loss k(w, (x, y)) less smooth and with a larger Lipschitz constant constant-ullimately increasing
Empirical Studies
We now evaluate residual knowledge of state-of-ther unlearning algorithms and its mitigation via RURK on three vision benchmarks. We denote Original as the model A(S) trained on the full dataset S, and Re- ie-train as the ideal (but not viable in real world) model A(S,) re-trained from scratch without the forget set Sy, used here as a reference. For experimental details, including dataset settings and
choices, refer to Appendix B.4; additional results are provided in Appendix C.
7

---

Data and unlearning settings. We evaluate unlearning methods on image classification under three random sample unlearning scenarios. The first scenario, small CIFAR-5. follows Golatkar al. (2020a,b) and uses a reduced CIFAR-10 subset (Krizhevsky et al., 2009) containing 200 training and 200 test images per class from the first five classes, while the second scenario uses the full CIFAR-10 dataset; both employ ResNet-18 18 (He et al., 2016). The third scenario is based on larger-scale ImageNet-1000, a 100-class subset of ImageNet- Ik (Deng et al., 2009) with 1,300 images per class, trained with ResNet-50. In all cases, only 50% of class O is unlearned, in contrast to the class unlearning setting (Kodge et al., 2024), which aims to remove all samples from a class. To avoid any external knowledge, both Original and Re-train models are trained from scratch without using any pre- pre-trained weights. See Appendix C.2 for class-unlearning results and Appendix C.3 for ablations on RURK hyper-parameter parameters and architectures (e.g.. VGG (Simonyan and Zisserman, 2014)).
Baseline algorithms. We evaluate three machine unlearning algorithms suited for small CIFAR-5 settings (CR, Fisher, NTK), and eight methods for the full CIFAR-10 [GD, NGD, GA, NegGrad+, EU-k, CF-k, SCRUB, SSD). Certified Removal (CR) uses influence functions and a one-step Newton update to estimate and remove the effect of forget samples (Guo et al., 2020). This approach was extended by Golatkar et al. (2020a), who incorporate the Fisher information matrix (Fisher) computed on the retain set, and by Golatkar et al. (2020b), who apply Neural Tangent Kernel (NTK) linearization of neural networks. These pioneering methods, while foundational, are not scalable to large models due to the computational cost of dessian-based operations. Gradient Descent (GD) fine-tunes Orig Original on the retain set S, using standard SGD (Neel et al., 2021). Noisy Gradient Descent (NGD) simply modifies GD by adding Gaussian noise in the gradient update steps for better privacy guarantees (Chourasia and Shah, 2023; Chien et al., 2024). On the other hand, Gradient Ascent (GA) removes the influence of Sj by reversing gradient updates on the forget set (Graves et al., 2021; Jang al., 2022).
NegGrad+ combines both strategies by applying GD to S, and GA to Sy simultaneously (Kurmanji al., 2024). To improve parameter efficiency, Goel et al. (2022) propose two layer-wise methods: Exact Unlearning the last k layers (EU-k), which re-trains them from scratch, and Catastrophically Forgetting the last & layers (CF-k), which only fine-tunes them on S,.. SCRUB casts unlearning as a teacher-studen distillation process, where the student selectively learns retain-se knowledge from the teacher (Kurmanji et al., 2024). The final method, SSD, selectively dampens model weights by uses Fisher information to estimate the influence of Sj (Foster et al., 2024), a scalable version of Fisher. Further details on these baselines are in Appendix B.3.
Evaluation metrics. We adopt five evaluation metrics, as described in $ 2.2, to comprehensively assess both the unlearning efficacy and the utility of the resulting models. To capture different dimensions of unlearning effectiveness, we first report Unlearning Accuracy following Fan et al. (2023). Second, we conduct MIA using a support vector classifier (SVC) trained to distinguish training samples from test samples based on the model's output likelihoods; we report the SVC's attack failure rate (MIA Accuracy) as a measure of privacy protection. Third, Re-leam Time quantifies how easily model can re-acquire the forget-set information: it is defined as the number of fine-tuning epochs needed for the model m to satisfy L(m, St) < (1 + L To assess model utility, we report classification accuracy on the retain set and the hold-out test set, referred to as Retain Accuracy and Test Aceuracy, capturing both performance preservation and generalization. As discussed in $ 2.1, a desirable unlearming method should minimize the deviation of performance from Re- Re-train in both sample and class unlearning settings. To this end, we compute the Average Gap (Avg. Gap)-the mean absolute gap from Re-train across Retain, Unlearn, Test, and MIA accuracies-whent as CR. Fisher, NTK, and NGD already offer certified unlearning guarantees under frameworks like (c, -unleaaan,, Rényi unlcarning, or bounded KL divergence; these guarantees do not necessarily imply small performance gap on accuracy- -cased metrics.
Sy). with 1) = 0.05.
a smaller Avg. Gap indicates better unlearning. Notably, methods such
The performance of all unlcarning methods, including RURK, is best understood by jointly examining Table I and Figure 2. Table I presents standard unlearning metrics to ensure each method maintains reasonable test accuracy without excessive forgetting, while Figure evaluates residual knowledge, measuring a model's 's ability to resist re- re-identifying perturbed forget samples. Ideally, properly unlearned model should match re-trained one mne-preserving performance on standard metrics while failing to recognize small perturbations of forgotten data.
9A can also be viewed as learning on randomized labels for forget samples. 10 Unlearning accuracy is defined 1- forget accuracy, i.., the classification error on the forget set Sy.
8

---

Table 1: Performance summary of various unlearning methods on image classification, including the proposed RURK, Original, Re-train, and baseline approaches, evaluated under two unlearning scenarios: small CIFAR-5 and full CIEAR-10, both using ResNet-18. Results are reported in the format 6gs. indicating the mean a and standard deviation 6 over 3 independent trials. The absolute performance gap relative to Re traiin is shown in (blue). For methods that fail to recover the forget-set knowledge within 30 training epochs, the re-learn time is reported as ">30". See Appendix C.1 for a complete table for Small CIFAR-5.
Evaluation Metries
Datasets
Methods
Re-karn Time Ppoch)
Retain Acc. (%)
Unleam Acc. (%)
Test Acc. (%)
MIA Acc. (%)
Avg. Gap
Original
09.993e..0 10 (0. (0.03)
0.00=0. (8)
96.37=9 (0.57)
4.6Tes so (22.33)
7.82
Small CIFAR-5
Re- -rain
99.96go.od (0.00)
8.33ga so(0.00)
27.00gs (0.00)
0.00
3.33
CR
90. (0.46))
14.00
91.80 (3.00)
68. 18.17720.m331.11
10.06
Flabar
92.67 co.ca (22
12.67 (4.4(4.14)
68.80
47.9316.aa20.000
(20.333
9.49
3.00
XTK
99.932o 10 03)
o.
96.37
50 (0.57)
16. (6.00gc.a((11000
3.23
4.67
RURK
99.521 (0.44)
5.67 2.066)
93.83_o.se (0.97)
33.33 (6:33)
2.60
2.00
Original
100.00 000.00g....00.000 t0.0.00
0.00
06.760o.o(11600
0.07
(22-43)
8.34
Be-ttain
100.00_o.00 (0.00;
9.47
93.30 (0.00)
22.5010.c:00.00
0.00
17.33
CD
20.98.0.o1 (0.02)
0.00
04.29g..or(0.900
1.10:a.0022..4
8.22
2.20
NGD
97.53=0.ca (2.47)
10.07
90.70+0.a12.00
3.70+o.s (8.80)
6.27
20.67 =1..11
CIFAR-10
15.411o..x(4.000
61.3710117511 91)
85.981o.1117.322
0.00
21:25
1.0010.00
Negüraa+
09.28=...o (0.72)
14.00
02.022no.aa..2
18.18
2.71
1.00)+0.00
XU-
99.3120.0o
1.1240.s58.333
92.9T=0.ex (0.333
0.87 1o
7.71
1.33+
CF-N
100:00 (0.000
0.00g0
94.38g000 (1.00000(1108) 08)
0.42 1o (22.08)
8.16
1.00g0.00
SCRUB
99.61 Lo.cn (0.30)
12.451a
02.701o.aa00.00
7.1010 (15.40)
4.84
30
SSD
26.49 (3.51)
66.45
08.50
5.12:0.4. (17.88)
20.65
7.30
BURK
99.35e0.or (0.45)
11.65=2.
122600+0.a
(1.30=
2.65
30
Performance comparison. Table 1 compares the performance of Original, the ideal reference Re Re-train, our proposed RURK, and 11 baseline unlearning methods across two unlearning scenarios. RURK achieves the smallest average performance gap to Re-train in both settings. We apply SGD with = 0.03 and e = in Term (ii) of Eq. (6), making RURK's time complexity comparable to other gradient-based approaches such as GD, NGD, and NegGrad+. In the small CIFAR-5 scenario. NTK shows a competitive Avg. Gap but suffers from the highest MIA accuracy, indicating vulnerability to privacy attacks. Fisher over-forgets the target samples, significantly degrading utility, seen in its low retain and test accuracies. For CIFAR- 10, NegGrad+ performs similarly to RURK in Avg. Gap but shows much shorter re-learn time than Re-train, implying an implicit retention of forget-set knowledge, GA and SSD aggressively erase forget-set information, achieving high unlearn accuracy but the cost of test accuracy dropping below 90%. Both EU- and CF-k (with A: 5) re-train or fine-tune only the linear and last two residual blocks, but the remaining layers of ResNet-1 18 still retain substantial information about the forget set. SCRUB has the closest overall performance to RURK when including re-learn time, but unlike RURK, it requires extra memory to store a student model, whereas RURK supports in-place updates. We defer the performance of other unlearning baselines on small CIFAR-5, and an ablation study on the syper-parametent (c.g., T, v and X) in RURK to Appendix C.
Residual knowledge. Figure 2 presents" estimates of residual knowledge f~ ((x, v)) under varying perturbation radii т, computed using Gaussian noise (p = 2) with e = 100 samples from B, (x, 7) (cf. Eq. (4)). As expected, Original consistently exhibits residual knowledge greater than 1. In small CIFAR-5, although NTK performs comparably to RURK in Avg. Gap and re-learn time (Table 1), its residual knowledge remains above 1 due to linearization under the neural tangent kernel, which ignores higher-order terms. This highlights residual knowledge as a necessary complement to unlearn accuracy, MIA, and re-learn time. In contrast, BURK maintains values near for < 0.01 and effectively suppresses them at larger т. For CIFAR-10, GD and CF-k retain high residual knowledge similar to Original, underscoring the need for access forget samples, as done by RURK. Conversely, GA and SSD yield residual knowledge below 1 even at r = 0, indicating over-unlearning EU-k updates only the final layers, leaving residuals in earlier representations. Methods such as NGD, NogGrad+, EU-k, and SCRUB follow trends similar to RURK, though RURK achieves more stable control near r = 0.01 and stronger suppression beyond that. While NGD reduces residual knowledge more effectively than GD or CF-k through controlled weight noise, it remains less effective than RURK. NegGrad+, with a similar objective, also retains more residual knowledge, validating the role of Term (ii) in Eq. (6). For residual knowledge under other attacks, see Appendix C. for adversarial disagreement and unlearn accuracy of the perturbed forget samples, see Appendix C.S.
By Eq. (4), the residual knowledge of Re-t is exactly for all r. so it is omitted from the figure.

---

Small CIEAR-5
CIFAR-10
Reditaal Knowlodge 7.40)) (.18))
Original
Fisher
Residuall Knowlllge PUSS
Original
GA
NGD
GD
NegGral+
1.815
122
1.159
0.21
1:123
0.02 NTK
0.30
BURK
EU
SCRUB
SSD
RURK
1.0
0.8
1.10
0:08
Perurr Norm F
Penurb Norm 7
Figure 2: Residual knowledge (Sy) of the proposed RURK, Original, and other unlearning methods across two unlearning scenarios, evaluated under varying perturbation norms T.
Evaluation
Resiuual Knowledge P,8
GD
NegSrad+
Datasets
Methods
1.005
0.675
Retain Acc. (%)
Unleam Acc. (3)
Test Acc. (%)
Avg. Gap
1.000
0.625
Driginal
80. 80.76.a.00 (.......00550))
3.50 (99.0) (9.03)
72.67
6.32
ImageNec- 100
Re Aorttriin
75.06 s4 (0.00)
12 12.62g: (0.000)
68. 58.4441..2
0.00
1:350
3.0
NegGrad+
81.86 si (6.80)
82.29 (7.23)
40.67 (27 28)
6.26-+2.xx (7.13)
73. od
74.64e2.no6.011
13.06
6.79
1.325
0.9
SSD
80.35 (5.30)
30.155:s..a276.7777
71 81 (3.38)
28.48
0.00
0.02
0.00
0.02
HURK
80.09 (4.07)
11.62 (1.0)
(f.19
5.26
Perturb Norm
Figure 3: Performance summary and residual knowledge (following Table I and Figure of selected unlearning methods on ImageNet-
Lcrge ImageNet-100. In Figure 3, RURK consistently achieves the smallest average perfor- mance gap compared to GD and NegGrad+, demonstrating strong scalability. Residual knowledge analysis shows that GD still retains forgotten information due its lack of explicit control over the for- get set, while SSD fails to fully suppress forget-related neurons in large architectures like ResNet-50. Although NegGrad+ mitigates residual knowledge more effectively, its alignment with the re-trained model remains weaker than RURK. Overall, residual knowledge persists beyond small-scale settings, and RURK remains the most effective method for mitigating it in larger, more complex models.
Final remark
Our study reveals a key limitation of existing unlearning methods: although they erase direct memorization of forget samples, they often fail to remove implicit generalization around them, causing unlearned models to recognize perturbed variants more often than re- re-trained ones. RURK mitigates this by incorporating perturbed forget samples, disrupting such generalization and reducing residual knowledge while preserving retain-set accuracy.
Limitations. Ideally, unlearned and re-trained models should be statistically indistinguishable not only on original inputs but also on all perturbations within (x,,). However, controlling adversarial disagreement across such perturbations is computationally infeasible it is as hard as achieving perfect ad adversarial robustness. Moreover, since the re-trained model is typically unavailable during unlearning, we can only bound one side of the residual knowledge (i.e., r. (St) S 1: see $ 4.2).
First, our probabilistic analysis in $ could be extended to account for hypothesis class complexity and linked to the transferability of adversarial examples (Tramèr et al., 2017). Second, the indistiggishabbltyy-rooo robuuttss trade-off introduced in $ 4.2 opens up new directions in both unlearning and adversarial robustness, warranting deeper investigation. Third, our mitigation objective (cf. Eq. (6)) resembles a reverse of adversarial training, suggesting an open question of whether adversarial training or distributionally robust optimization could in fact impede unlearning or amplify residual knowledge. Moreover, in the context of MIA against unlearned models, adversaries could exploit side information- as the minimal perturbation required for model to re-recognize a forgotten sample. A smaller perturbation norm may indicate prior inclusion in training, increasing inference success. This perspective opens a promising research direction for future MIA studies in unlearning, as recent work has begun leveraging model variation near training points or perturbation dynamies to strengthen attacks (Jalalzai et al., 2022: Del Grosso et al., 2022; Xue et al., 2025). Finally, extending residual knowledge beyond classification to generative dasks-suce as image or text generation-rent remainss an open challenge. these settings, unlearning often targets concepts rather than samples, yet tasks like image-to-image generation (Fan et al., 2023; Li et al., 2024) show that perturbing partial inputs can still regenerate forgotten content, suggesting analogous residual behaviors. In large language models, the compositional nature of text and differing notions of "forget" and "retain" complicate formalization, though phenomena like relearning and jailbreaking indicate conceptual parallels (Hu et al., 2024; Shumailov et al., 2024; Liu et al., 2025).
Future directions.
10

---

Broader impact. Residual knowledge introduces new privacy risks. For instance, if a user opts out of biometrie-basee payment system (e.g., palm or facial recognition), residual traces may still allow adversaries to craft perturbed inputs that the system peeetts-potentillly enabling unauthorized access. Such vulnerabilities undermine trust in MI. systems and challenge current interpretations of the "Right to be Forgotten."
Disclaimer. This paper was prepared for informational purposes by the Global Technology Applied Research center and Artificial Intelligence Research group of JPMorgan Chase & Co. This paper not product of the Research Department of JPMorgan Chase & Co. or its affiliates. Neither JPMorgan Chase & Co. nor any of its affiliates makes any explicit or implied representation or warranty and none of them accept any liability in connection with this paper. including, without limitation, with respect to the completeness, accuracy, or reliability of the information contained herein and the potential legal, compliance, tax, or accounting effects thereof. This document not intended as investment research investment advice, or as a recommendation, offer, or solicitation for the purchase or sale of any security, financial instrument, financial product or service, or to be used in any way for evaluating the merits of participating in any transaction.
References
Abadi, M., Chu, A., Goodfellow, I., McMahan, H. B., Mironov, I., Talwar, K., and Zhang, L. (2016). Deep learning with differential privacy. In Proceedings of the ACM Conference Computer and Communications Security (CCS).
Allouah, Y., Kazdan, J., Guerraoui, R., and Koyejo, S. (2025). The utility and complexity of in- and out-of-distribution machine unlearning, In Proceedings of the International Conference on Learning Representations (ICLR).
Belrose. N., Schneider-Joseph, D., Ravfogel, S., Cotterell, R., Raff, E., and Biderman, S. (2023). Leace: Perfect linear concept erasure in closed form. In Proceedings of Advances in Neural Information Processing Systems (NeurIPS).
Bourtoule. L., Chandrasekaran, V., Choquette-Choo, C. A.. Jia, H., Travers, A., Zhang, B.. Lie, D., and Papernot, N. (2021). Machine unlearning. In Proceedings of the IEEE Symposium on Security and Privacy (SSP).
Brophy, J. and Lowd, D. (2021). Machine unlearning for random forests. In Proceedings of the International Conference Machine Learning (ICML).
Bun, M. Gaboardi, M.. Hopkins, M.. Impagliazzo, R., Lei, R., Pitassi, T., Sivakumar, S., and Sorrell, (2023). Stability is stable: Connections between replicability, privacy, and adaptive generalization. In Proceedings of the 55th Annual ACM Symposium on Theory of Computing (STOC).
Cao, Y. and Yang, J. (2015). Towards making systems forget with machine unlearning. In Proceedings of the IEEE Symposium on Security and Privacy (SSP).
Carlini, N., Hayes, J., Nasr, M., Jagielski, M., Sehwag, V., Tramer, F., Balle, B., Ippolito, D., and Wallace, E. (2023). Extracting training data from diffusion models. In Proceedings of the 32nd USENIX Security Symposium.
Chien, E., Wang, H., Chen, Z.,, and Li, P. (2024). Langevin unlearning: A new perspective of noisy gradient descent for machine unlearning. In Proceedings of Advances in Neural Information Processing Systems (NeurIPS).
Chourasia, R. and Shah, N. (2023). Forget unlearning: Towards true data-deletion in machine learning. In Proceedings of the International Conference on Machine Learning (ICML).
Chourasia, R., Ye, J., and Shokri, R. (2021). Differential privacy dynamics of langevin diffusion and noisy gradient descent. In Proceedings of Advances in Neural Information Processing Systems (NeurIPS).
Del Grosso, G., Jalalzai, H., Pichler, G., Palamidessi, C., and Piantanida, P. (2022). Leveraging adversarial examples to quantify membership information leakage. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition (CVPR).
11

---

Deng, J., Dong, W., Socher, R., L.-J., Li, K., and Fei-Fei, L. (2009). Imagenet: A large-scale hierarchical image database. In Proceedings of the IEEE Conference on Computer Vision and Puttern Recognition (CVPR).
Dwork, C. and Roth, A. (2014). The algorithmic foundations of differential privacy. Foundations and Trends@ in Theoretical Computer Science, (3344):211 00:211-400 1-407.
Eldan, R. and Russinovich, M. (2023). Who's harry potter? approximate unlearning in LLMs. arXiv preprint arXiv:2310.022.0
Fan, C., Liu, J., Zhang, Y., Wong, E., Wei, D., and Liu, S. (2023). Salun: Empowering machine unlearning via gradient-based weight saliency in both image classification and generation. In Proceedings of the International Conference on Learning Representations (ICIR)..
Foster, J., Schoepf, S., and Brintrup, A. (2024). Fast machine unlearning without retraining through selective synaptic dampening. Proceedings of the AAAI Conference on Artificial Intelligence.
French, R. M. (1999). Catastrophic forgetting in connectionist networks. Trends in cognitive sciences, 3(4):128-135.
Gandikota, R., Materzynska, J., Fiotto-Kaufman, J., and Bau, D. (2023). Erasing concepts from diffusion models. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), pages 2426-2436.
Ginart, A., Guan, M.. Valiant, G., and Zou, J. Y. (2019). Making Al forget you: Data deletion in machine learning. Proceedings of Advances in Neural Information Processing Systems (NeurIPS).
Goel, S., Prabhu, A., Sanyal, A., Lim, S.-N., Torr, P., and Kumaraguru, P. (2022). Towards adversarial evaluations for inexact machine unlearning. arXiv preprint arXiv:2201 .666000
Golatkar, A., Achille, A., and Soatto, (2020a). Eternal sunshine of the spotless net: Selective forgetting in deep networks. In Proceedings of the IEEE Conference on Computer Vision and Puttern Recognition (CVPR).
Golatkar, A., Achille, A., and Soatto, (2020b). Forgetting outside the box: Scrubbing deep networks of information accessible from input-output observations. In Proceedings of the European Conference on Computer Vision (ECCV).
Goodfellow, I., Bengio, Y., Courville, A., and Bengio, Y. (2016). Deep learning. MIT press Cambridge.
Goodfellow, J., Shlens, J., and Szegedy, C. (2014). Explaining and harnessing adversarial examples. arXiv preprint arXiv: 152.6774
Graves, L.., Nagisetty, V., and Ganesh, V. (2021). Amnesiac machine learning. Proceedings of the AAAI Conference on Artificial Intelligence.
Gross, L.. (1975). Logarithmic sobolev inequalities. American Journal of Mathematics, 97(4):1061- 1083.
Guo, C., Goldstein, T., Hannun, A., and Van Der Maaten, (2020). Certified data removal from machine learning models. In Proceedings of the International Conference on Machine Learning (ICML).
He, K., Zhang, X., Ren, S., and Sun, J. (2016). Deep residual learning for image recognition. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition (CVPR).
Hinton, G., Vinyals, O., and Dean, J. (2014). Distilling the knowledge in a neural network. In Proceedings of Advances Neural Information Processing Systems (NeurIPS), Deep Learning Workshop.
Hu, S., Fu, Y., Wu, S., and Smith, V. (2024). Jogging the memory of unlearndd models through targeted relearning attacks. In Proceedings of the International Conference on Machine Learning (ICML), Workshop on Foundation Models in the Wild.
12

---

Impagliazzo, R., Lei, R.. Pitassi, T., and Sorrell, J. (2022). Reproducibility in learning, In Proceedings of the 54th annual ACM symposium on theory of computing (STOC).
Izzo, Z., Smart, M. A., Chaudhuri, K., and Zou, J. (2021). Approximate data deletion from machine learning models. In International Conference Artificial Intelligence and Statistics (AISTATS).
Jalalzai, H., Kadoche, E., Lelue, R., and Plassier, V. (2022). Membership inference attacks via adversarial examples. In Proceedings of Advances in Neural Information Processing Systems (NeurIPS), TSRML Workshop.
Jang. J., Yoon, D., Yang, S., Cha, S., Lee, M., Logeswaran, L., and Seo, M. (2022). Knowledge unlearning for mitigating privacy risks in language models. arXiv preprint
Kalavasis, A., Karbasi, A., Moran, S., and Velegkas, G. (2023). Statistical indistinguishabiityy of learning algorithms. In Proceedings of the International Conference on Machine Learning (ICML).
Kim, H. (2020). Torchattacks: A pytorch repository for adversarial attacks. arXiv:2010.01950.
arXiv preprint
Kodge, S., Saha, G., and Roy, K. (2024). Deep unlearning: Fast and efficient gradient-fres free class forgetting. Transactions on Machine Learning Research (TMLR).
Krishna, S., Han, T., Gu, A., Wu, S., Jabbari, S., and Lakkaraju, H. (2025). The disagreement problem in explainable machine learning: A practitioner's perspective. Transactions Machine Learning Research (TMLR).
Krizhevsky, A., Hinton, G., et al. (2009). Learning multiple layers of features from tiny images. Technical Report.
Kullback, S. and Leibler, R. A. (1951). On information and sufficiency. The annals of mathematical statistics, 22(1):79-86.
Kulynych, B., Hsu, H., Troncoso, C., and Calmon, F. (2023). Arbitrary decisions are hidden cost of differentially private training. In Proceedings the ACM Conference on Fairness, Accountability, and Transparency (FAccT).
Kurmanji, M., Triantafillou, P., Hayes, J., and Triantafillou, E. (2024). Towards unbounded machine unlearning. In Proceedings of Advances in Neural Information Processing Systems (NeurIPS).
Ledoux, M. (2001). The concentration of measure phenomenon. American Mathematical Society.
Li, G., Hsu, H., Marculescu, R., et al. (2024). Machine unlearning for image-to-image generative models. In Proceedings of the International Conference on Learning Representations (ICLR).
Liu, S., Yao, Y., Jia, Casper, S., Baracaldo, N., Hase, P., Yao, Y., Liu, C. Y., Xu, X., Li, H., et al. (2025). Rethinking machine unlearning for large language models. Nature Machine Intelligence, pages 1-14.
Madry, A., Makelov, A., Schmidt, L., Tsipras, D., and Vladu, A. (2018). Towards deep learning models resistant to adversarial attacks. In Proceedings of the International Conference Learning Representations (ICLR).
Marcel, S. and Rodriguez, Y. (2010). Torchvision the machine-vision package of torch. In Proceedings of the 18th ACM International Conference on Multimedia (MM).
Marchant, G., Rubinstein, B. I., and Alfeld, (2022). Hard to forget: Poisoning attacks on certified machine unlearning. In Proceedings of the AAAI Conference on Artificial Intelligence.
Martens, J. (2020). New insights and perspectives the natural gradient method. Journal of Machine Learning Research (JMLR), 21 (1146))1176.
Milman, V. and Schechtman, G. (1986). Asymptotic theory of finite dimensional normed spuces: Isoperimetric inequalities riemannian manifolds, volume 1200. Springer Science & Business Media.
13

---

Mironov, I. (2017). Rényi differential privacy. In Proceedings of the IEEE Computer Security Foundations Symposium (CSF).
Neel, S., Roth, A., and Sharíf-MMavvjerddi S. (2021). Descent-lo-delete: Gradient-based methods for machine unlearning. In Proceedings of Algorithmic Learning Theory (ALT).
Nguyen, Q.P., Low, B. K. H., and Jaillet, P. (2020). Variational bayesian unlearning. In Proceedings of Advances in Neural Information Processing Systems (NeurIPS).
Nguyen, T. T. Huynh, T., Ren, Z., Nguyen, P. I.., Liew, A. W.-C., Yin, H., and Nguyen, Q. V. H. (2022). A survey of machine unlearning. arXiv preprint arXiv:2209.0222
Paszke, A. (2019). Pytorch: An imperative style, nigh-performance deep learning library. arXiv preprint arXiv: 1rXXx:99/2.00703
Pawelezyk, M., Di, J. Z., Lu, Y., Kamath, G., Sekhari, A., and Neel, S. (2025). Machine unlearning fails to remove data poisoning attacks. In Proceedings of the International Conference on Learning Representations (ICLR).
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., et al. (2011). Scikit-leam: Machine learning in python. the Journal of machine Learning research (JMLR), 12:2825-288
Ravfogel, S., Twiton, M., Goldberg, Y., and Cotterell, R. D. (2022). Linear adversarial concept erasure. In Proceedings of the International Conference on Machine Learning (ICML).
Rényi, A. (1961). On measures of entropy and information. In Proceedings of the 4th Berkeley Symposium Mathematical Statistics and Probability, Volume 1: Contributions to the Theory of Statistics.
Schelter, S., Grafberger, S., and Dunning, T. (2021). Hedgecut: Maintaining randomised trees for low-latency machine unlearning. In Proceedings of the International Conference Management of Data.
Sekhari, A., Acharya, J., Kamath, G., and Suresh, A. T. (2021). Remember what you want to forget: Algorithms for machine unlearning. In Proceedings of Advances in Neural Information Processing Systems (NeurIPS).
Shastri, S., Wasserman, M., and Chidambaram, V. (2019). The seven sins of personal-data processing systems under GDPR. In Proceedings of the 11th USENIX Workshop on Hot Topics in Cloud Computing.
Shumailov, I.. Hayes, J., Triantafillou, E., Ortiz-Jimenez, G., Papernot, N., Jagielski, M.. Yona, I., Howard, H., and Bagdasaryan, E. (2024). Ununlearning: Unlearning is not sufficient for content regulation in advanced generative AL, arXiv preprint vxXiv:2407.00106
Simonyan, K. and Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv preprint
Talagrand, M. (1995). Concentration of measure and isoperimetric inequalities in product spaces. Publications Muthématiques de l'Institul des Hautes Etudes Scientifiques, 81:73-205.
Tramèr, F., Papernot, N., Goodfellow, I., Boneh, D., and McDaniel, P. (2017). The space of transferable adversarial examples. arXiv preprint arXiv: 1704.03004
Uma, A. N., Fornaciari, T., Hovy, D., Paun, S., Plank, B., and Poesio, M. (2021). Learning from disagreement: A survey. Journal of Artificial Intelligence Research (JAIR), 72:1385-14770
Voigt, P. and Von dem Bussche, (2017). The EU general data protection regulation (GDPR). Practical Guide, 1st Ed., Cham: Springer International Publishing, 1033152666;10-55555
Wang, L.., Chen, T., Yuan, W., Zeng, X., Wong, K.-F,, and Yin, H. (2023). KGA: A general machine unlearning framework based on knowledge gap alignment. Proceedings of the 61 Annual Meeting of the Association for Computational Linguistics (ACL).
14

---

Wang, W., Tian, Z., Zhang, C., and Yu, S. (2024). Machine unlearning: A comprehensive survey. arXiv preprint orXiv:4055.77066
Wu, Y., Dobriban, E., and Davidson, S. (2020), Deltagrad: Rapid retraining machine learning models. In Proceedings of the International Conference on Machine Learning (ICML).
Xu, H., Zhu, T., Zhang, L.., Zhou, W., and Yu, P. S. (2023). Machine unlearning: A survey. ACM Computing Surveys, 56(1).
Xuan, H. and Li, X. (2025). Verifying robust unlearning: Probing residual knowledge unlearned models. arXiv preprint arXiv:2504.14799
Xue, Sun, Z., Ye, H., Luo, L., Chang. X.. Tsang, I., and Dai, G. (2025), Privacy leaks by adversaries: Adversarial iterations for membership inference attack. arXiv preprint arXiv:2506.02711.
Yao, J., Chien, E., Du, M., Niu, X., Wang, T., Cheng, Z., and Yue, X. (2024). Machine unlearning of pre-trained large language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (ACL).
Zhang, B., Dong, Y., Wang, T., and Li, J. (2024). Towards certified unlearning for deep neural networks. In Proceedings of the International Conference on Machine Learning (ICML).
Zhang, R. and Zhang, S. (2022). Rethinking influence functions of neural networks in the over- parameterized regime. In Proceedings of the AAAI Conference on Artificial Intelligence.
Zhao, C., Qian, W., Li, Y., Li, W., and Huai, M. (2024). Rethinking adversarial robustness in the context of the right to be forgotten. In Proceedings of the International Conference Machine Learning (ICML).
15

---

NeurIPS Paper Checklist
1. Claims
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Question:
Answer: [Yes]
Justification: We include a summary contributions at the end of the introduction, where each contribution and the claims made therein are specifically referred to section in this paper.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results be expected to generalize other settings.
is fine include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: We include a discussion of limitations regarding theoretical extensions and computational overhead in the last section.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full of assumptions and complete (and correct) proof?
16

---

Answer: [Yes]
Justification: We clearly state all the assumptions and lemmas that we have used with citations in our theoretical results. We also provide the sketch of proof/intuition in the main text.
Guidelines:
The answer NA means that the paper does not include theoretical results, All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided not)?
Question:
Answer: [Yes]
Justification: We provide details descriptions of the proposed methodology in the main text and in the appendix. Our methodology tested on over 2 different vision benchmarks. shows consistency and explainable results over independent trials, and outperforms 11 existing unlearning baseline algorithms.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, No answer this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution be necessary to either make it possible for others to replicate the model with the same dataset, provide access the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
specific model and empirical evaluation, may
While NeurlPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm.
(b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (c.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the partieular way they provide for reproducibilitty In the case of closed-source models, it may be that access to the model is limited in
17

---

some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [No]
Justification: Due to intellectual property protection and anonymity requirements, we choose release our codes upon decision. We provide details scripts how access the datasets, implement our methodology. and reproduce the empirical results in the main text and appendix.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips cc/
ptaiii//guiies/Cdeeettiiii cwt.iiiisiioicccoiooooo for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https: / /nips. http//pblc/gguuuees/..ebb http//uwdee//eeeeeeeeee /tt////odbiisiiiooooc swiaaiisssioooiiooyooooooooa for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data,
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We provide all the training and setting details in the main text and appendix, including
settings, optimizer/learning rate scheduler, the GitHub links to all unlearning baselines we have evaluated, the reference to the evaluation metrics, etc.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: All the results reported in this paper include error bars that is computed over independent trials. Moreover, we implement different evaluation metries (operational meaning and proper reference included) compare method against existing basclines.
18

---

Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CL, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: We provide the specific type of machine we used for the experiments in the appendíx. We also include discussion of the computational complexity of our method in $ 5 and the appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https: : /neuriisu....// httppplc/ttcicccuiiiedeepoo
Answer: [Yes]
Justification:
We have reviewed and complied the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative socictal impacts of the work performed?
19

---

Answer: [Yes]
Justification: We cover the societal impacts of the residual knowledge and machine unlearn- ing in the introduction, related work and final remark.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), faireess considerations (e.g.. deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Answer: [NA]
Justification: [NA]
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question:
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: We provide proper citations for all datasets including licenses and how to access, and programming packages (Python, Pytorch, Scikit Learn, torchattack) used in this paper.
Guidelines:
The answer NA means that the paper does not existing assets.
20

---

The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset. For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com condaatsets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: [NA]
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember anonymize your assets (if applicable). You either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: [NA]
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Bthies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: [NA]
21

---

Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with
human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
22

---

The appendix is divided into the following parts. Appendix A: Omitted proof' and theoretical results; Appendix B: Details on the experimental setup; and Appendix C: Additional empirical results and ablation studies.
A Omitted proofs and theoretical results
We first introduce (and prove) the following useful lemmas to facilitate the proofs of the propositions. The first lemma is variant of the (e, 6) iiiiisiiinnuusshbbiittt in Definition 1.
Lemma A.1 (Probabilistic indistinguishabilit).. If X and Y are (e, iindsstiiinnuu,sablle then with probability at least 25/(1 - e over z drawn from support( X) U support(Y),, we have
e 2x Pr(Y = =] S Pr[X- 2] S 2 Pr[Y = =].
(A.1)
Proof. We first consider the complement event of the right inequality in Eq. (A.I). Define 2 {2) Pr|X = 2] > eae Pr(Y = zI), we directly have Pr[X E 2] > e Pr(Y E 2). By (c,5)- indistinguishable we have
82 Pr(X 2] e PrlY E
> (e - et) Pr(y E E
8
(A.2)
= Pr(y 2 < 2 -
c²e (1
Now consider the complement event of the left inequality in Eq. (A.1) and define Z' = Pr|Pr[X== 2] << 22 Pr-y = =]}. by similar algebra, we have
z Pr[Y E 21 - e Pr|x E 2'| (e - -e)) z'|
+ Pr[X 3'1 < 2 - e
(A.3)
6
8
+ Pr[yY E 21| S e Pr[x E 2'1 +8 < E
+ 8
e - Et
1 e-
Since 5
T-e- is always larger than --. (by factor of e²9), by combining Eq. (A.2) and Eq. (A.3), we have
Pr|y E = = Pr|(Pr[X = =]}> 2 Pr|Y = =|} or (Pr[X = < 26 Prly = =]H
8
(A.4)
With the same analysis on Pr[X 3] and Pr[X E 2'), we have Pr[x 2U22 S -- Finally, putting together the the probability bounds on Pr|Y E 3`| and Pr[X E ZU 3'1, we have
8
pr(e 26 Pr|y 2| S Pr[X 2| < 2 Pr|y = =|| < 1 1 -
(A.5)
as desired.
The second lemma relates to the isoperimetric inequality (Talagrand, 1995). Before stating it, we detine the notion of r-expansion of a set. Given set C' c. Rd. its r-expansion with respect the Ep- f--norm is defined as
E(C. & fc E ReIle- - S
for some g x C).
(A.6)
The isoperimetric inequality is used to characterize the normalized surface area of the T-expansion of a half unit sphere, showing how it grows with the expansion radius т and the dimension d. This property is leveraged in the condition of Proposition 2, and is formally stated below.
Lemma A.2 (Milman and Schechtman (1986)). Let C be the half unit sphere in Rd. iee, C E Sd-1 and surf (S)) 2 1/2. then with p = 2 or p = 0c, the T-expansion has the surface area
that satisfies
(u..(E(C.).T)) 21 surf 1
π g
1/2
exp
d d-1 1
2 72)
(A.7)
23

---

The third lemma utilizes Lemma A.1 to lower bound the probability of disagreement, i.e. the expected value of the disagreement indicator function k(x).
Lemma A.3 (Lower bound on disagreement probability). the unlearned M (A(S), S, and re-
trained A(S,) models satisfies (e, 6) -unleurning, then with probability 2 - -) the probability of disagreement among the two models on sample x S is lower bounded by
E[k(x)) (M) * A(x)) (x) * A(x) 1 O 2t-
(A.8)
Proof. We prove the lower bound by directly decompose (x[M((x) + A(x)), i.e.
/
Px(M(x)= # A(x)]
Pr [m(m(x)= + a(x)]| = m m=== h; = a + M\ Pr(M # h, A hidh
NEW
/ NEH
+ =m m =m - 1|] Pr[M = h = hjdh
(A.9)
The second integral in Eq. (A.9) is zero since Pr [x) * [(xx) M = m m = A= = 0.
Moreover, by Lemma 1. with probability 1 26/(1 e e)), we have
Pr[M # h, A h] Pr[M # hj Pr(A= h] و Pr(A h| - Pr(A= = h12
(A.10)
In other words, with 26/(1 - e ), we have
Pr[M + h, A = h] = Pr[M # h Pr(A- = h)] Pr(A = - Pr(A = 112 The combination of Eq. (A.9) and Eq. (A.11) yield
(A.11)
Pr[M(x)= # A(x)] =
Pr (m(m * a(x)]M = m m == 1| Pr[M # h, A = hldh
NER
Pr[A = h] 2 PrrA- = h1
hEW
Pr[A = h]dh - 22
22e
Pr[A h12
hEH
NEH
O (e (e-
(A.12)
the desired result, as f
Pr[A = hldh- = 1 and ʃ
Pr[A=A]3dd - f13 a constant.
hER
NEW
The fourth lemma provides the relation between the adversarial disagreement k+(x') = (x) + A(x)) and the residual knowledge r-((x. y)).
Lemma A.4 (Bounding adversarial disagreement with the residual knowledge.). The expected value of ky (x") over the perturbation distribution is upper and lower bounded by
rr (xx, y)) Pr (a(x') = yl (1 - Pr [a(x') = < E
(a((x) = y/²
(A.13)
Proof. We prove the lemma by directly following the definition of adversarial disagreement. First, have
E [x>((x))] = Pr [m((') # axxx') =
Σ
Pr [m(x'' = =1, a(x'))) # #]
1E)
= -mm
= Pr [ma('' = ul (1 - -[([a(x)))== = w])
(A.14)
where the final inequality follows by isolating the contribution of the true label y. By substituting the definition of residual knowledge from Eq. (4), we obtain
E k, (x') 2 ry ((x, v)) Pr ja(x') y] (1 Pr(a(x)) =y)).
(A.15)
24

---

Similarly, by evaluating 1 - E (x)) = Pr [m(x') = a(x'),, we have
1 - E kk (x') = Pr [m((x') - a(x')l = Σ
Pr [m((') = =,,a(x)) =
lEY
2 Pr (m((') = y/ Pr (r(x'))) = y/ = =r, -<, B) Pr (a(x)) = y/?
(A.16)
+ [k,, -
Pr
y)²-
A.1 Proof of Proposition 1
Suppose the unlearned (A(S), and re-trained A(S,) models are (..D-indistinguishabll. and let x be fixed sample. From Lemma A.1, we have that with probability 28/(1 - e²) ), h drawn from support( (M) support(A), we have
Pr[M = /|] s e 2e Pr(A = h] or e²x Pr(A = ^] S Pr[M = =h].
(A.17)
Therefore, for all X C X, by using the right inequality in Eq. (A.17),
Pr(gx(m) E x = Pr|gx X, M = h] = (h) E Pr|M = h]
(A.18)
Σ e Pr(gx (h) E x Pr[A = h] = 6² (r(gg((())) E ed'.
Similarly, for the other inequality, we have
(m]) E x S e (h) X'I Pr|A- = M] - 2 Pr(gx(a) Ex
(A.19)
Eq. (A.18) and Eq. (A.19) together give the desired result.
A.2 Proof of Proposition 2
Let R = {x E [kx) = 0} to be the region of the sphere that has agreement between the unlearned and re-trained models. By assumption, we have (R))/R)(( s 1/2. Accordingly, we define R fx E S (xx) = }} to be the complement of R. denoting the region of the sphere that has disagreement between the unlearned and re- re-trained models. Since 1/2, its T-expansion ERR, p. r). by Lemma A.2, is at least as large as the epsilon expansion of a half sphere; that is
>
surf(E(R,,,.)) 2 1 - 8
1/2 exp d dd1 (-2
(A.20)
(Sd-()
2
Note that the r-expansion E(R, T) represents the set of samples that either has k(x) = 1 or admits x' E (x, such that k(x') 1. We can therefore define a set Ee that the complement of ERR, p, r) with surface area satisfies
surf(E") (S
1/2
d d
exp
}
(A.21)
The probability to draw sample in 8d is then upper bounded by
1/2 exp
d - 1 72
Pr|Mr((Mx) A(x)) x
(A.22)
Using Lemma A.3, we know that with probability 26/(1 - )
sup (x) = A(x)] = sup - (r) + A(x)] = 1 - (1 0 (0) = 0 20) (A.23)
Putting Eq. (A.22) and Eq. (A.23) together, we know that the probability to draw a sample in E(R, p, r) is at least
28
1/2
d d=1,2
X
sup (x(M = A(x) x
exp
I e
8
2
28
1/2
d
-
0
20)
exp
2
(A.24)
1 c
28
1/2
d - 1 12
1
0
8
exp
-2
25

---

B Details on the experimental setup
We provide implementation details of RURK in $ 4.2, and introduce the unlearning baselines in $ 5, including their mathematical formulations, operational interpretations, implementation specifics, and GitHub links. Finally, we summarize the dataset descriptions, training setups, and evaluation metrics.
B.1 Comparison with Pawelczyk et al. (2025)
Pawelczyk al. (2025) investigate the failure of machine unlearning under data poisoning attacks. In their setup, the training set Swain consists of two disjoint subsets: Selean and Spoison where Spoison contains maliciously corrupted samples generated via targeted or backdoor poisoning. The original model is trained on both subsets, and the goal of unlearning is to remove the influence of Spoison so that the resulting model resembles one retrained solely on Setemn Ideally, the unlearned model should be "clean" and exhibit no poisoning effects. However, Pawelczyk et al. (2025) find that even after applying state-of-the-ar unlearning algorithms, the unlearned model continues to display backdoor revvvior-revealed through persistent correlations with the poison pattern pattem-unlike the clean retrained model. They conclude that unlearning cannot fully sanitize a poisoned model, as residual effects of the attack remain embedded in its representations and decision boundary.
Our work, in contrast, examines clean unlearning under adversarial perturbations at test time rather than under poisoned data at training time. Both the retain and forget sets in our framework are clean; the original model Mo is trained on Spewin U Storgets and the unlearned model M, aims to forget Stoeet so as to be indistinguishable from a retrained model M, trained only on S We show that even when M,, and M, are distributionally indistinguishable, they can still diverge in behavior on perturbed forget samples Sperturhed samples not seen during training but lying in the local neighborhood of Storget- Specifically, M,, tends to re-recognize these perturbed samples more often than My, revealing a form of residual knowledge that persists despite formal unlearning guarantees.
The key distinction between the two settings lies in the source and nature of the residual information. In Pawelczyk et al. (2025), the unlearned model retains explicit knowledge of malicious patterns learned from Speison (e.g., a backdoor trigger). In our work, the residual information is implicit, arising from the generalization structure around the clean forget samples rather than from any poisoned signal. Thus, while Pawelczyk et al. (2025) highlight the limits of unlearning under training- time data corruption, our work reveals that even with entirely clean data, residual generalization can cause unlearned and retrained models to remain distinguishable- posing distinct privacy risk not attributable to poisoning.
B.2 Details of RURK in $ 4.2
We first provide the pseudo-code of RURK in the following algorithm box Algorithm 1.
We implement RURK using PyTorch (torch) (Paszke, 2019), and ensure reproducibility by fixing three random seeds: [131, 42, 7]. During optimization, we use a batch size of 128, the standard cross-entropy loss (torch. nn. CrossEntropyLooss)) O), and the SGD optimizer (torch. mhorch.opiim. optim.SGD) with a learning rate of 0.01, momentum of 0.90, and weight decay of 5 x 10-4. To stabilize training, we apply cosine annealing learning rate scheduler (torch. optim..mmmm....mmm lrlllrs_ssuuuee scheduler. wow.inenneanllnnnnccccccac ingLR) and cap the total number of iterations at 200. Additionally, we clip the gradient norm to 1.0 using torch. utils. clip_grad lariggradddoorrr _nood____________
For both unlearning scenarios (small CIFAR-5 and CIFAR-10), we set the perturbation budget 0.03 and use the TorchAttacks library (Kim, 2020) to identify the vulnerable set ( y), r). To improve efficiency, we define (x y). as the entire perturbation ball B,(x, T) and set 0 = 1, meaning that only single perturbed sample is drawn from multivariate Gaussian distribution centered at x with standard deviation T.
We configure the hyper- hyper-paramnn parameters as follows: for small CIFAR-5, we use N = and ny A 0.03; for CIFAR-10, we set = 2, ' = 0.03, and A, 0.00045. Since e and the perturbation is generated via Gaussian noise, the inner loop in line 12 of Algorithm executes only once, making the operations in lines 1-15 constant-time. As result, the computational complexity of RURK is comparable to fine-tuning- based methods such as GA, GD, NegGrad+, and NGD.
26

---

Algorithm 1: RURK Implementation
input :0riginal W A(S); Forget Set Sy; Retain Set Sy; Loss Function
output :Unlearned Model WRURK
parameter: IPerturbation Norm 7; Regularization Strength Xy, Aa; Sample Size v; # Epoch N
RLoader MakeDatalLoader (S,);
FLoader -- MakeDatalLoader (Sy);
3 WRURK f- w:
for epoch < 1 to N do
for batchldx, (retainData, retainTargets) in RLoader do // Scan over all Batches
retainOutput <- hw (retainData):
RLoss - Loss
retainTargets);
forgetData, forgetTanget nextIter neeeeeerarrtteeeexeeteeeeerr (FLoader)) forgetOutput + hww (foreetaatt
FLoss LosssfffrrreOOutppll forgetlarge))
11
vulnerableSet for i 4 1 to v do
Construct the Vulnerable Set ((x, v), T);
13
vulnerableSSt + vulnerableSet + (fogeeaaaae end
15
adv udvForgetOutput + w
V (vulnerableeSet);
16
AdvFLoss - Loss Lossaadboorggettuuputt hor.orrrreooouuuuoooorrrrrrree
17
Loss + RLoss X FLoss - -Ag
1/ Entire Loss in Eq. (6) // SGD Optimization
18
WRUSK
W WURK + Wanay Loss
end
20 end
return
: WBURK
However, if stronger adversaries like FGSM or PGD are used to identify the vulnerable set, the computational complexity increases from ((V) O(N x v). Note that unlike NGD, we do inject additional noise into the gradient update steps.
More discussion RURK. The regularization over the forget set in the loss L(w,A(S)) highlights two key features of our proposed unlearning method, RURK. First, under mild conditions, the un- learning procedure satisfies certified unlearning via Rényi Differential Privacy (RDP), providing formal guarantees- certificate of unlearning. Second, enhancing robustness against vulnerable perturbations ((x may increase the distinguishability of the unlearned model from the ideal re-trained model.
To support the first point, consider the case where the forget set contains single sample (i.e., [Sy] = I), so that the original dataset and the retain set S, are adjacent. Assume the original model trained to satisfy (a, Eu)-RDP. Further, suppose the unlearning procedure minimizes the loss in Eq. (6) using Projected Noisy Gradient Descent (PNGD). As shown by Chien et al. (2024), the Markov chain defined by PNGD updates of the form
Wi+1 I Wr (W, A(S)) 22no2 with W1 A(S)
(B.25)
where W, ~ (O, Ia) and 1R denotes projection onto the ball of radius Rt converges to stationary distribution if the loss L(w, A(S)) from Eq. (6) is continuous. The first term of Eq. (6) is continuous by standard assumptions, as the loss E(-, (x, 3)) is typically continuous in the weights. To show continuity of the second term, consider g(w) minzgB, E(w, y)). Let be minimizer; then g(w) f(w, (z" y)), which is continuous in since é is. Therefore, the overall loss is continuous, and the PNGD process converges.
Furthermore, when the loss is L-smooth and M M-Lipschitz, the stationary distribution obtained via PNGD satisfies (a,, c)PDDP for some c depending on L, M, and convexity properties of the loss (Chien et al., 2024). For forget sets with multiple elements, one sequentially apply this procedure, preserving Rényi unlearning at each step.
To justify the second point, observe that Term (ii) in Eq. (6) impacts the loss landscape's 's smoothness. Specifically, the term s(w, (x,xx,)) ()) tends to increase with the perturbation radius т, as the adversarial loss over larger ball is generally higher. This leads to gradient norm that increases with
27

---

implying that the smoothness and Lipschitz constants of the overall loss also grow with т. According to Theorem 3.2 in Chien al. (2024), the Rényi distinguishability parameter is inversely related to these constants, implying trade-off: achieving higher robustness (larger r) increases model distinguishability (larger <).
The assumptions underlying these results are mild and standard in the literature. We assume the loss is L-smooth and M-Lipschitz, and that the learning dynamics for the original dataset satisfy the Log-Sobolev Inequality (LSI) with constant CL.t (Gross, 1975). Moreover, the Original A(S) can be trained to satisfy (a, cu) RDD under LSI using the same PNGD framework, as discussed by Chien et al. (2024).
28

---

B.3 Existing unlearning algorithms
Here, we provide detailed descriptions of the 11 unlearning baseline methods used in $ 5, along with links to their corresponding GitHub repositories.
Certified Removal (CR). Guo et al. (2020) propose a sinle-sstpp Newton Raphsso update to remove the influence of forget samples. Assuming that the empirical risk L(w, S) is twice differentiable with continuous second derivatives, and letting w arg win L( (w. Sy) denote the empirical risk minimizer over the retain set S,, the Taylor expansion of the gradient V around w" yields:
V,W R VL(,,, + Hw. (w..),,
(B.26)
where w. A(S) denotes the Original model parameters, and Hw. V S,) is the Hessian of the empirical risk over S, evaluated at Wo. Re-arranging the terms gives:
W. - VT,,,,,
(B.27)
since VT(w",, ,S)) by optimality. The certified removal method (CR) then defines the unlearned model as
M (w.,S,S)p = w. - VL
(B.28)
where À is step-size parameter. Under the assumption of convexity (ensuring the uniqueness of minimizers) and bounded approximation error, Guo al. (2020, Theorem 1) show that this procedure
guarantees (e, o)-eetiffiedd removal, i.e., (W...,,S). c.6 w".
In implementation. facebookresearchh/coo http://eseerrcccrrtiieeeee-e-eee http:/orrrffedddrrraaaaeeaaeeaaa
follow the official codebase https://github: com/
Since CR is designed for binary linear models, a pre-trained ResNet-18 18 as non-private feature extractor (excluding its final linear layer), and apply the Newton update only to the final layer. Specifically, we train [ one-vs-rest logistic regression classifiers the extracted features and set À = 0.1
Fisher Unlearning (Fisher). Fisher (Golatkar et al., 2020a) is one of the simplest unlearning mechanisms, which removes the influence of forget samples by directly perturbing the model weights with Gaussian noise (cf.$2.1). Unlike standard Gaussian perturbation, the variance of the noise is scaled according to the inverse of the Fisher information matrix. Specifically, the unlearned model is defined as:
M(w,8, S,) w + n, n (O,,O.,
(B.29)
where B denotes the Hessian matrix V2L(W, S,). However, computing the exact Hessian is computationally intractable even for moderately sized neural networks, and the matrix may not be positive definite. To address this, Golatkar al. (2020a) approximate the Hessian using the Levenberg-Marquarrd algorithm, yielding a semii-ositive-defiiit matrix closely related to the Fisher information matrix (Martens, 2020), which motivates the name of the method. In our implementation. we follow the official codebase and adopt the same hyper-parameterd as in https://gi //gtbbb..//u thub. com/ Awl.AddyyGolaaaaa Gaaa http:/ollaaaaa/eeeetttttee /So.lectiveooogettiiggg
Neural Tangent Kernel Unlearning (NTK). NTK (Golatkar et al., 2020b) extends the Newton update idea from CR by applying the Neural Tangent Kernel (NTK) linearization of neural networks. The NTK matrix between two datasets S₁ and Sy is defined as:
K(SI, A ww V
(B.30)
where lhw denotes the network output and hww may optionally denote scalar output function (e.g., pre-activation logits). Let w arg min L(w,S) and w, = arg min L(w,S,) be the minimizers of the empirical risk over the full and retain sets, respectively. By linearizing the network output around w using the NTK approximation, NTK enables a closed-form update that shifts the model weights from w to an approximation of w, via one-shot adjustment:
w, RJ M(w", ,S,,, = w + PV w hw- (8y) MV,
(B.31)
29

---

S where the projection matrix P = - hw ( VV hw- (S,) ensures that the
gradient contributions from the forget set are orthogonalized against those from the retain set. The matrix M is defined
M = K(S), K(S,, ((S,,),)) (( ((S,
(B.32)
and the re- weiigitting matrix is given by
V (yf how (S₁)) + ((), ((S,,, (w---ww-SS. (S.)),
(B.33)
where y5 and yr denote the ground truth labels for the forget and retain sets, respectively. We follow the official implementation and use the same settings and hyper-parametent as in https:
Gradient Descent (GD). GD (Neel et al., 2021) is one of the simplest unlearning algorithms. continues training the original model Original the retain set S, using standard gradient descent. Specifically, the unlearned model M obtained by iteratively applying the update:
Wi+1 < Wi 77. 9t (We); with W1 A(S),
where 1) is the step size and ge (w())) is a (mini-batch) stochastic gradient of the empirical loss over S,, i.e. [S.] Esss, f(w,s). The key intuition behind GD that when the forget set is small (i.e., |Syl < SS), the minimizers of the loss functions over and S, are expected to be close. Therefore, few steps of gradient descent starting from the original model w = A(S) can efficiently move the parameters toward a minimizer of the updated training objective. Building on this intuition, Neel al. (2021) also provide theoretical guarantees for the effectiveness of GD both convex and certain non-convex settings.
Our implementation follows httpss//ggtabb..om///rtaaaaatttt doleee using the hyper-parameters:
SGD optimizer with Ir le-2, momentum 0.9, and weight decay le-4.
Random seed for the data loader is 7; random seeds for repeated experiments are 131, 7.
Batch size = 128
Number of epochs 10
Noisy Gradient Descent (NGD). NGD (Chourasia and Shah, 2023) is simple extension of GD in which Gaussian noise added to each gradient update. The unlcarned model obtained by iteratively applying the update:
Wi+1 f- W. - (no(e)) + Ec), with W1 = A(S),
where y) the step size, Dr (w:) denotes a (mini-batch) stochastic gradient of the empirical loss over the retain set S,, i.e., [3.] ExES, ((w,s), and & (0 o²) is an independent Gaussian noise term added at each iteration. The key distinction between NGD and GD lies in the injection of noise during optimization. This stochasticity not only increases the robustness of the unlearning process but also enables formal privacy guarantees, demonstrated in the context of certified unlearning via Rényi differential privacy (Chien et al., 2024). Notably, a similar poise-injection mechanism is employed in the DP-SGD algorithm for training models with differential privacy guarantees (Abadi al., 2016).
Our implementation follows https: /tte:///ittuut www/Graph--ooooooe htw.laaagevviiinllaanninnn www.urnninggggggnee and the same parame-arrsettrss GD with
(a, n) = (0.03, 0.1), trained for 10 epochs for the small CIFAR-5 settings with sample unlearning.
(a, m) (0.03, 0.1), trained for 2 epochs for the CIFAR-10 settings with sample unlearning.
Gradient Descent (GA). GA (Graves et al., 2021) aims remove influence of the forget set Sj from trained model by reversing the learning process through gradient ascent. It stores all gradient updates involving Sy during the initial training phase, and unlearning is then performed by applying the exact reverse updates-i.ce i.c. gradient ascent-using using the stored gradients. However, this
30

---

approach is highly memory- intensive and becomes impractical for large-scale models. To address this limitation, Jang et (2022) proposed more scalable variant, which unlearning is achieved by performing mini-batch gradient ascent on the forget set. Specifically, the model is updated by minimizing the negated loss T.( St), effectively implementing ascent steps that counteract the original training influence of Sy.
Our implementation follows https: /tt:://gittt http:////..tabaoooo/////aaaa//a/ the similar hyper-parameters as GD:
using
SGD optimizer with a Ir = le-3, momentum = =0,, and weight decay = le-4, clipping gradient norm = I.
Random seed for the data loader is 7; random seeds for repeated experiments are 131, 42,
Batch size 128
Number of epochs 1 for Small-CIFAR-5 (both sample and class unlearning) and CIFAR-10 with sample unlearning; while for CIFAR-10 with class unlearning
Negative Gradient Plus (NegGrad+). NegGrad+ (Kurmanji et al., 2024) fine-tune the model by simultaneously minimizing the loss on the retain set S, and maximizing the loss on the forget set Sy. effectively negating the gradient contributions from the latter. Specifically, the unlearned model is obtained by optimizing the following objective:
S.) St),,
(B.34)
where 33 is Hyper-parameed that controls the strength of unlearning by weighting the loss contribu- tion from Sy. NegGrad+ shares conceptual similarity with the Gradient Ascent (GA) method, as both perform loss maximization on the forget set to induce forgetting. However, NegGrad+ is empirically more stable and yields better performance, it simultaneously enforces loss minimization on the retain set, ensuring that useful knowledge is preserved during unlearning.
Our implementation follows parameters as and GA:
hwaptttbb.commmgggaaaa////CCCCC using similar hyper-
SGD optimizer with Ir le-2, momentum 0.9, and weight decay le-4, clipping gradient norm = 1.
3 0.001
Random seed for the data loader is 7; random seeds for repeated experiments are 131, 42,
Batch size = 128
Number of epochs 1 for Small-CIFAR-5 (both sample and class unlearning) and CIFAR-10 with sample unlearning; while 3 for CIFAR-10 10 with class unlearning
Exact Unlearning the last & layers (EU-k). EU-k (Goel al., 2022) is simple, parameter-efficient unlearning method designed for deep learning models, requiring access only to the retain set S,. Given parameter EU-k retrains from scratch the last layers of the neural network- those closest to the output layer-whiie keeping the earlier layers fixed. By adjusting k. EU-k provides a tunable trade-off between unlearning effectiveness and computational efficiency.
Our implementation based on the official repository at https //tgt//btttt/t///e thub. cww som/shha Evaluating- mnexact-Unlearning using the following hypae-parameters:
SGD optimizer with a Ir = lc-2, momentum = 0.9, and weight decay = Ic-4.
Last fully-connected layer, and last two residual blocks are reinitialized, and only fine-tune those parameters,
Random seed for the data loader is 7; random seeds for repeated experiments 131, 42, 7.
Batch size = 128
Number of epochs 10
31

---

Catastrophically Forgetting the last & layers (CF-k). CF-k (Goel et al., 2022) builds on the observation that neural networks tend forget information about data samples encountered early in training-1 phenomenon known as catastrophic forgetting (French, 1999). Similar to EU-k, CF-k focuses on the last k layers of the network; however, instead of re-training these layers from scratch, it fine-tunes them starting from the original model parameters w A(S), using only the retain set S,, while keeping all earlier layers frozen.
As with EU-k, our implementation follows the official repository at https://gt http:/ggittttbt thub com/ shash42 thash42/Evaluue /wt////vaaaannnneeeeeeeeeaarllle www.xatt-nnleaannnneeotaaa Unlearning using the following hyper-parameters.
SGD optimizer with Ir le-2, momentum 0.9, and weight decay lc-4.
Last fully-connected layer, and last two residual blocks are fine-tuned.
Random seed for the data loader is 7; random seeds for repeated experiments are 131, 42, 7.
Batch size = 128
Number of epochs 10
SCalable Remembering and Unlearning unBound (SCRUB). SCRUB (Kurmanji et al., 2024) is one of the stale-of-the-art unlearning methods for deep learning, It formulates the unlearning task within a student-teacher framework: given a trained teacher model WT, the goal is to train student model w that selectively imitates the teacher. Specifically, the student should retain the teacher's behavior on the retain set S, while diverging significantly from it on the forget set Sy, as measured by the KL divergence. To achieve this, SCRUB optimizes modified knowledge distillation objective (Hinton et al., 2014):
(x) [a, DKL(hwr (x) (h)))(())) + 7: E(w, (x, y))l - (xx,))~5 [DLL (hwr (x) (x)))
(x))]
(B.35)
where and are hyper-paramettes that balance knowledge retention and task performance. The first expectation encourages the student to match the teacher on the retain set while also performing well on the classification task, and the second term enforces divergence from the teacher on the forget set.
Our implementation follows https: htte::///ttttie github........ wtcchuu..oommeeee Bemm///ehhddd//SSCRR with the
SGD optimizer with a Ir = 5e-4, momentum = 0.9, and weight decay = 5e-4.
Random seed for the data loader is 7; random seeds for repeated experiments are 131, 42, 7.
Batch size = 128
Q 0.001
? 1 for small CIFAR-5 settings with both sample and class unlearning
7 0.6 for CIFAR-I0 setting with sample unlearning and y 75 for CIFAR-I0 with class unlearning
Selective Synaptic Dampening (SSD). SSD was introduced by Foster et al. (2024) as method to unlearn a specific forget set from a neural network without retraining it from scratch. Building on ideas similar to Fisher, but with a more refined approach, SSD selectively dampens weights that exhibit disproportionately high influence measrredde via the Fisher information-on the forget set relative to the retain set. Given model with parameters w, let F, and F, denote the Fisher information matrices computed over the retain set S, and the forget set Sy. respectively. Unlearning performed by scaling each parameter w, according to its relative Fisher sensitivity:
[@w; if > oF
W;-
(B.36)
W; otherwise,
where f and Fr,; denote the i-th diagonal entries of the Fisher matrices for Sy and S,., respectively. The hyper-parameed
controls the threshold for selecting influential weights, and the dampening factor 3 is defined as 3 = min, /Fr./FF,.. 1 for a tunable parameter X.
Our implementation is based on the official repository at https: /ttps://githrn www.hha..om.fffooo/eee slleetive-ssyaatiic-daaag
using the following hyper-parameters:
À = 1.0
* 10.0 for CIFAR-10 10 and 100.0 for
32

---

B.4 Training and evaluation details
Here, we provide details on dataset preparation, unlearning settings, and evaluation procedures.
Dataset and unlearning settings. We evaluate unlearning methods in the context of image clas- sification using CIFAR-10 (Krizhevsky et al., 2009) and ResNet-18 (He et al., 2016), focusing on the random sample unlearning setting across two scenarios. The CIFAR-10 dataset contains 60,000 color images of size 32 x 32 pixels, evenly distributed across 10 classes: airplane, automobile, bird, cat, deer, dog, frog, horse, ship, and truck. Each class includes 5,000 training and 1,000 test samples. CIFAR-10 is widely used in machine unlearning research-for example. by Kurmanji et al. (2024); Chien et al. (2024); Golatkar al. (2020a,b); Foster et al. (2024). In the first scenario- we follow the setup of Golatkar et al. (2020a,b) by creating reduced version of CIFAR- 10 with 200 training and 200 test samples from each of the first five classes. From class 0, we randomly select 100 samples (50%) as the forget set Sy. The second scenario uses the full CIFAR-10 10 dataset. Here, we designate 2,000 samples (50% 12) from class 0 as the forget set Sy. In addition to sample unlearning, we also consider class unlearning. In this setting, the forget set contains 200 samples from class O in the small CIFAR-5 scenario, and 4,000 samples from class 0 in the full CIFAR-10 10 scenario. All experiments are conducted on an AWS EC2 g5 24xlaae instance.
CIFAR-
5-we
Training of Original and Re-train. To avoid any external knowledge (e.g., the de- fault ImageNet pre-trained weights from torchvision (Marcel and Rodriguez, 2010)), both the Original and Re-train models are trained entirely from scratch, without load- ing any pre-trained weights. We use a batch size of 128 and cross-entropy loss (torch. nn. CrossEntropylosse ()). For the small CIFAR-5 setting, we first pre-train ResNet- 18 model on the full CIFAR-10 dataset using the SGD optimizer (torch. opt.....m...... SGD (1r=0.4. 4, momentum=O.9. . veight_decay=22 uler (torch. optim 1r lr_schedulerr CosineAnneal ingLR (T mmax=200)), We then fine-tune this model on small CIFAR-5 for 25 epochs with a learning rate of 0.01 to obtain the Original model reported Table 1. The Re-train model in Table is obtained using the same setup, except the fine-tuning is performed on the retain set S, for 100 epochs. For the CIFAR-10 setting, both the Original and Re-train models are trained from scratch on the full dataset and the retain set, respectively, for 137 epochs using learning rate of 0.01. For the VGG- 11 ablation study, follow the same training protocol, including the optimizer, scheduler, and learning rate.
along with cosine annealing learning rate sched-
Evaluation details. The retain, forget, and test accuracies reported in Table I are computed by directly evaluating each model on the corresponding retain, forget, and test sets. The unlearning accuracy is defined reacquire knowledge of the forget set. It is defined as the number of fine-tuning epochs required for the model m to satisfy L(m,S)) < (1 + 17) (Origiina, Sy), where set 1) 0.05. This value can be adjusted depending on the desired tolerance.
forget accuracy. The re-learn time quantifies how easily a model can
To evaluate vulnerability to membership inference attacks (MIA), we use support vector classifier (SVC) implemented via the scikit-leary package. with parameters SVC SVC (C=3, gamma="auto"", kernel='rbf") (Pedregosa et al., 2011). The SVC is trained to distinguish between samples during training and those that were not, based on the model's output likelihood for the correct class label. In the sample unlearning setting, retain samples are labeled as I (seen) and test samples 0 (unseen) during SVC training; forget samples are then evaluated by the SVC, and an ideal unlearned model should cause them to be classified as O. In the class unlearning setting, retain samples labeled as and forget samples as 0 for SVC training; evaluation is then conducted on a held-out subset of the forget class. The reported MIA Accuracy in Table 1 corresponds the SVC's attack failure rate, i.e., the proportion of forget samples classified as unseen.
Both the construction of the vulnerable set V((x,)) v), and the evaluation of residual knowl- edge (S, involve generating perturbed inputs within the norm ball (x, We im- plement this using the torchattacks package (Kim, 2020). For Gaussian noise (i.c., p = 2), we use torchattacks. GN (model, std=t); for FGSM (i.c., p = x)) we use torchattacks FFMM eps=r); and for PGD, we use torchattacks. PCD (model,
Each class has 5,000 training samples, with 20% held out for validation.
33

---

eps-7, Alpha=2/255.. ped Sandom_start-Truee tacks for both FGSM and PGD.
We apply targeted at-
To estimate residual knowledge, we apply these attacks to the unlearned model m to generate per- turhed inputs x', and compute both Pr Pr(m(x')= y] and Pr[a(x') y) over 100 independent runs, using the same perturbation x' for both models. Constructing the vulnerable set (((x, y), r) involves solving the minimization problem min [(w, (z, y)). However, since the torchattacks package is designed solve the adversarial objective ma ((w, (z, y)), instead define random target label y' # y and solve (x.r) f(w, (z. y/ ))- To adapt this into residual knowledge framework, we flip the sign the regularization term in Eq. (6), effectively encouraging the model associate perturbed variants of forget samples with incorrect label y'. This discour- ages the model from retaining knowledge of the true label and facilitates unlearning. We repeat this process times for each forget sample to construct its corresponding vulnerable set.
B.5 Details on Figure
Figure is based on the UCI Iris dataset (3 classes, 50 samples each, 4 features). For both the original and re-trained models, of 100 neurons. We select forget samples in total from three class for illustration. While this selection differs from the random sample or class unlearning setup (Lines 297-303), the goal is purely illustrative-to convey the intuition behind the existence of disagreement and residual knowledge. The unlearned model trained via GD.
simple feedforward neural network with one hidden layer
34

---

C Additional results and experiments
We include additional experimental results and comparisons, including: (i) comprehensive results on sample unlearning for both the small CIFAR-5 and CIFAR-10 scenarios, covering accuracy metrics and residual knowledge estimates under various adversarial attacks; (ii) results on class unlearning for both small CIFAR-5 and CIFAR-10; 10; (iii) ablation studies on the (iv) an analysis of the prevalence of residual knowledge across different settings.
of RURK, and
C.1 Complete results on sample unlearning for small CIFAR-5
We present the full version of Table under the small CIFAR-5 setting in Table C.2. Combined with Table the results demonstrate that RURK consistently achieves superior cestommanne--uppassiiiii certified removal methods such as CR, Fisher, and NTK, as well as haa---leainnnooomnee proaches like SCRUB, NegGrad+, and NGD by achieving the smallest average gap from the re-trained model Reain. Notably, the re-learn time of RURK is also comparable to that of Re- Re-train.
ap-
We further include the complete residual knowledge curves estimated under different adversarial attacks: Gaussian noise in Figure C.4, FGSM in Figure C.5, and PGD in Figure C.6. For both Gaussian noise and FGSM, most methods exhibit residual knowledge greater than 1-i.e., inheriting information from the Original model-whereas RURK maintains residual knowledge close to 1 for small perturbation norms т, and reduces it below 1 as T increases.
With PGD (10 steps). a stronger attack, we observe that methods such as NTK, NGD, and SCRUB still suffer from residual knowledge. In contrast, methods like GD, GA, NegGrad+, EU-k, CF-k, and SSD maintain residual knowledge near 1 across varying values r. In particular, SSD strictly preserves residual knowledge at 1, though this comes at the cost of larger average gap from Re-train (cf. Table C.2).
Overall, RURK effectively suppresses residual knowledge to values below 1 for small т, and keeps it close to 1 as T poorcases-strkking a favorable balance between preserving accuracy and mitigating residual knowledge.
Table C.2:
The complete performance summary of various unlearning methods small CIFAR-S (ef. Table 1). Results are reported in the format a+s, indicating the mean a and standard deviation over independent trials. The absolute performance gap relative to Be-train is shown in (blue). For methods that fail to recover the forget-set knowledge within 30 training epochs, the re-leam time is reported as ">30".
Evaluation Metries
Datasets
Methods
Re-leam Time (# Epach)
Retain Acc. (%)
Uniearo Acc. (%)
Test Acc. (%)
MIA Acc. (%)
Avg. Gap
Original
99.93 (0.03)
0.00=a..0.8.00
95.37 (0.57)
4.67 ga (22.53)
7.82
Re- Erain
90.96 os (0:00)
8.33 (0.00)
94.80 (0.00)
27.00 oc.000
0.00
3.33
90.561o.17 (0.40)
14.00 (6.667)
91.80_a.m (3:00)
58.17_o.rs331.1
10.06
Fisher
92.67 (7.29)
12.67 (4.344.3))
88, 1...00 (6.00)
47.13-e..1
9.49
3.00
NTX
09.93.e0.10 (0.03)
7.00=a.00 (1.33)
95.37 (0.57)
16.00-4.2u (11.00)
3.23
4.07
Small CIEAR-5
CD
84.04:o.42(15.00
22.67 20.10 (14333))
76.83 (17.77)
84.67
18.89
12.87
MGD
15.077=1..01..800
4.67-0.4=
89.55=2..x
13.33=2..0(13.67)
6.92
0.67+3.47
941.2223.s(5.74)
10.67=s..v932..03
(8.377)..u88.43)
81.00au..:
18:29
2.33
RegGrad+
96.7819..x.3.199
25.00g1.4<(6600)
89. 89.7720.0055.00
74.67
10.55
2.001a.00
91.151::e2(.881)
20.00_4ax(1..6)
16.33
37.00_aa((20.33)
14.82
9.30
CP-
99.96++..c00.000)
0.33-0.4x88.00
94. (0.7
37 37.00-+24.x(2.333
7.10
17.00=11.45
SCRUB
99.88
1.33=0.
a8.33
3.97
1.00+2.00
SSD
96.85 (3.11)
13.33
89. 89.43 (5.377)
cs.00
6.29
2.31
BURK
99.5240.37 (0.44)
5.67=236(2.6)
93.83 (0.97)
23.33+12220663333
2.60
2.00+3.00
35

---

Small CIFAR-5 (Sampic Unlearning Gaussian Noise)
Residull Knowledge F-(S,)
Original
Fisher
NTK
NGD
1.3
0.9
1.2
1.05
1.15
1.2
1,000
0.8
1.00
0.00
0.02
0.00
0.00
0.02
0.00
0.02
NegGrad+
EU-k
CF-k
SCRUB
SSD
RURK
1.000
1.05
105
1:10
1.05
0.978
1.00
1.00
1.05
1.00
0.6
0.00
0.02
0.00
0.00
0.02
0.00
0.00
0.02
Pertur Norm
Figure C.4: Residual knowledge (Sy) of the proposed RURK, Original, and other unleaming methods small CIFAR-5 with sample unlearning, evaluated under varying perturbation norms т, using Gaussian noise (p 2) to draw c= 100 samples from (x, r).
Small CIFAR-5 (Sample Unlearning, FGSM)
Residal Knowledge F-(Sy)
Original
Fisher
NTK
GD
NGD
GA
1,100
130
1.05
1.00
1.1
NegGrad+
EU-k
CF-K
SCRUB
SSD
RURK
1,125
1.00
0.9
0.978
1.00
1,000
1.05
0.98
0.02
Perturb Norm
Figure C.5: Residual knowledge (S,) of the proposed RURK, Original, and other unleaming methods small CIFAR-5 with sample unlearning, evaluated under varying perturbation norms т, using targeted FGSM (p oc) draw 100 samples from (xxx
Small CIFAR-5 (Sample Unlearning, PGD)
Residual Knowledge P-(8y)
Original
Fisher
NTK
GD
NGD
GA
1.1
1000
L005
1.0025
1.002
1.0
0.95
1.00
0,00
DU00
0102
3000
1102
11.02
NegGrad+
BU-K
CT-k
SCRUB
SSD
RURK
1.0025
1.05
1.0000
1,000
1,055
1.00
0.55
1.0
0.95
0000
1102
3000
1102
0.02
Perturb Norm
Figure C.6: Residual knowledge f. (Sy) of the proposed RURK, Original, and other unlearning methods on small CIFAR-5 with sample unlearning, evaluated under varying perturbation norms r, using targeted PGD (p oo) draw 100 samples from (x,
36

---

We provide the residual knowledge estimates under FGSM in Figure C.7 and under PGD in Figure C.8, corresponding to the results presented in Table and Figure 2 of the main text. RURK demonstrates consistent behavior across all adversarial attack types-Gaussinn Gaussian noise, FGSM, and PGD by maintaining residual knowledge close for small perturbation radii т, and effectively suppressing it below as increases.
CIFAR-10
Residual Knowledge Pc(Sy)
Original
GA
NGD
GD
NegGrad+
1.1
1.00
1.0
0.75
1.0
1.0
0.5
0.00
0.02
0.00
0.02
0.00
0.02
0.00
0.02
0.00
0.02
EU-k
CF-k
SCRUB
SSD
RURK
1.00
1.10
0.4
1.0
1.05
0.2
0.5
0.75
0.00
0.02
Perturb Norm t
Figure C.7: Residual knowledge rr(S)) of the proposed RURK, Original, and other unlearning methods on CIFAR-10 with sample unlearning, evaluated under varying perturbation norms using targeted FGSM oc) draw 100 samples from (x)
CIFAR-10
Residual Knowledge Ps(Sy)
Original
GA
NGD
GD
NegGrad+
1.025
1.100
1.025
0.6
L,075
1.000
1.05
0.975
0.00
0.02
0,00
0.02
0.00
0.02
0.00
0.02
0.00
0.02
EU-k
CF-k
SCRUB
SSD
RURK
1.10
1.025
0.9
1.05
1.000
1.05
0.4
0.00
0.02
0.00
Perturb Norm &
Figure C.8: Residual knowledge fryS)) of the proposed RURK, Original, and other unleaming methods on CIFAR-10 with sample unleariing, evaluated under varying perturbation norms т, using targeted PGD (p 00) to draw 100 samples from (y,x, T).
37

---

C.2 Class unlearning: Forgetting all samples in a class
In the main text, we focus on the sample unlearning setting, where 50% of the samples from class O are unlearned. Here, we provide analogous results for the class unlearning setting, where 100% of the samples from class 0 are removed, on both small CIFAR-5 and CIFAR- 10. Note that in class unlearning, the Re-train model is never exposed to any samples from class O. As result, the unlearning aceuracy is
is, Re-train never correctly classifies any forget sample into class 0. In Table C.3, we report the same evaluation metrics as in Table 1, including the average absolute gaps in retain, unlearn, test, and MIA accuracy compared to Re-train.
In both small CIFAR-5 and CIFAR-IO, EU-k achieves the smallest average gap relative to Re-train. This result is expected, as class unlearning in this case resembles transfer learning scenario: from source domain (small SEFAR-S/CIFAR-1O to target domain (small the forget class is entirely absent. This creates a clear domain shift, allowing EU-k to perform well by fine-tuning only the top layers. In contrast, in sample unlearning. both the source and target domains still contain samples from class O, making transfer learning less effective and leading to poorer performance for EU-k.
where
We also present residual knowledge estimates for the small CIFAR-5 class unlearning scenario under Gaussian noise (Figure C.9), FGSM (Figure C.10), and PGD (Figure C.1I). While RURK is the second-best performer in terms of average accuracy gap in the small CIFAR-5 setting, it outperforms EU-k in suppressing residual knowledge. In the CIFAR-16 R-10 class unlearning scenario, RURK is the best among popular methods such as GA, SCRUB and SSD.
Table C.3: Performance summary of various unlearning methods for class unlearning. Results are reported in the format ags, indicating the mean a and standard deviation over 3 independent trials. The absolute performance gap relative to Re- train is shown in (blue). For methods that fail to recover the forget-set knowledge within 30 training epochs, the re-learn time reported ">30".
Datasets
Methods
Evaluation Metrics
Re-learn Time (# Epoch)
Retain Acc. (%)
Unlearn Acc. (%)
Test Ace. (%)
MIA Acc. (%)
Avg. Gap
Drig original
99.9220 10.0.30)
0000go.o010000
oo
96.25 ea (1.75)
43.89
Re-train
99.791e.s2(0 (0)111)
100.00 (00.00
93.33 (2.00))
100.001o.o00.000)
0.00
1.00
Ficher
93.75g0 (6.04))
4r
89. 39.12 (2(.220
51.67
35.31
3.33
NTK
25.0013.o(74.777
100.00g0.oo 00..:o00000.00 (0.0000)
26 33)
100.0010.000.000
35.78
9.6T
(3.25)
g3.67_o...6.33))
63.75
69.83....o(10.11)
19.90
n.00
Small CIFAR-S
KGD
91.38 oo (8-422)
100:00ga (0.00))
59.7 69.75gn.m0333 58)
100.0000000000000
10. 10.50
12.33 12.33_1..0
98.17 (1.66))
13.001:
(0.751)
64.33 64.33
28.
2.00
Negdrad+
(2.837).2(80.17)
83.8140
58.67
30.57
2.00
BI-k
(0.461...00.33)
100.0010..o00000)
a4.501
100.001..000.000)
2.29
31.00_o.0
90.96.2.oo(0.117)
a2.33:...)(57.67))
97.88 97.81 (6.744o-n(2.46)
15.61
n..0
SCRUB
99.96+0.c000.11
7.83 (92.67)
95. (5.5+
13.677 33)
37.81
1.00ea.00
SSD
28.4610
7.00
83. (0.00)
42.17 (5778833
38.05
2.00go.00
RURX
98.58+1 (1.21)
87.17 (12.883)
93. 83.71+0.xx01 (0.38)
96.33es..0(3.67
4.52
1.33
Original
100.00 a.00 00.0000000.000 (0.00)
0.00
ao
7.87_ (92.13)
48,2
Re-train
100.00=a.oo (0.00)
100.00
93.894e (0.000))
100.004e.c000.000
12.67=4.64
cm
90.9810
0.00 0.00|...:
94.331e
29.27 10.2770.s (7.07))
42.78
0.67
KGD
99.91 (0.09)
20.844e.ss799..
02.40300200.33
16.184
20.83
8.00gs
CIPAR-10 10
DA
95.51 14(4499
93.001m.006.4000 (0)
11.60 (5.29)
96.4720.a133.33
4.93
1.00
Negürad+
(0.931...u(0.077)
57.19
86.50 oe.401)
14.12
0.67
BU-X
98.38 c010o.11110000 6i2)
100.00=a.a.0.000
(00000+...000000))
0.84
2.67+0..4
CF-X
100.00_..o00.000
0.06_o..r(P 94)
38.4311as:66.555
40.55
0.001a.o
SCRUB
20.15
100.00
a6.6677as(7.17)
100.00+eaatt00000)
3.51
18.67
SSD
100.00=a.co00.00
65.27 65.274..s(34.733
96.09+6.2(1.20)
100 100.00+o.u00.00)
8.98
5.67+1
RURX
99.80
(5.52
(0.90
97.90
1.70
1.00g0.00
38

---

Small CIFAR-S (Class Unlearning, Gaussian Noise)
Residual Knowledge F-(S,)
Original
Fisher
NTK
GD
NGD
GA
1-2
0.9
0.15
1.3
1.00
0.00
1.1
1,00
0.00
0.00
0.00
0.02
0.00
0.00
NogGrad+
EU-k
CF-k
SCRUB
SSD
RURK
1.025
1.000
1,000
1,025
1,025
0.10
0.978
0.915
0.975
0.6
1,000
0.05
0.00
0.00
0.02
0.02
0.00
0.00
Perturb Norm
Figure C.9: Residual knowledge (Sy) of the proposed RURK, Original, and other unleaming methods small CIFAR-S with class unlearning, evaluated under varying perturbation norms т, using Gaussian noise (p 2) to draw == 100 samples from (x,, T).
Small CIFAR-S 5 (Class Unlearning, FGSM)
Residal Knowledge F-(Sy)
Original
Fisher
NTK
GD
NGD
GA
6,01
0.95
0200
0.00
1.1
-0.05
-0.05
1.0
NegGrad+
EU-k
CF-k
SCRUB
SSD
RURK
1.1
1.05
0.95
0.3
1.0
1.00
017
1.0
Perturb Norm
Figure C.10: Residual knowledge f, (Sy) of the proposed RURK, Original, and other unlearning methods on small CIFAR-5 with class unlearning, evaluated under varying perturbation norms т. using targeted FGSM (p oc) to draw c 100 samples from B, (x,,))
Small CIFAR-5 (Class Unlcarning, PGD)
Residual Knowledge P-(8y)
Original
Fisserr
NTK
GD
NGD
GA
1.0
0.05
0.08
0.00
1.025
0,00
1,005
1.00
0.9
0.05
1,000
0.05
1,000
0.00
0.00
0.00
0,00
NogGrad+
EU-k
CF-k
SCRUB
SSD
RURK
1,05
0,50
1.025
1.025
1,025
0.50
1.000
1,00
1.000
0.55
1,000
0.00
0.00
Perturb Norm 2
Figure C.11: Residual knowledge f. (Sy) of the proposed RURK. Original, and other unlearning methods small CIFAR-5 5 with class unlearning, evaluated under varying perturbation norms T, using targeted PGD (p o0) draw 100 samples from (x,,)))
39

---

C.3 Ablation studies on RURK
In this section, we conduct ablation studies on the key hyper-parametee of RURK, including the perturbation radius т, regularization strength X, sample size v, types of adversarial attacks, and alternative model architectures. We focus on the sample unlearning setting in the small CIFAR-5 scenario, with all results summarized in Table C.4, Table C.5, Figure 12 and Figure C.133
Different perturbation norms т. The perturbation norm r determines the radius of the perturbation ball used to evaluate residual knowledge. We set = 0.03 x 8/255, which aligns with common practice for the maximum adversarial perturbation. To assess the sensitivity of RURK to this parameter, we conduct an ablation study over E {0.00, 0.01, 0.02, 0.03, 0.04, 0.10). As shown in Table C.4, when т 0.00 or 0.01, RURK fails to unlearn the forget samples from Original. However, as т increases, the unlearning accuracy improves monotonically, while the test accuracy declines-as expected due to the increased regularization. Notably, RURK achieves the best average gap when controlling the trade-off between unlearning efficacy
0.03. Since т is the key and utility, we further visualize the estimated residual knowledge under varying T in Figure C.12. The figure shows that increasing improves robustness against residual knowledge under stronger attacks. For instance, under Gaussian noise and FGSM, the residual knowledge remains near 1 for small In contrast, PGD stronger adversary can still uncover residual knowledge at low values, but its effectiveness diminishes significantly when T = 0.1, indicating improved robustness.
The regularization strength À controls the weighting of samples in the vulnerable set within the loss function defined in Eq. (6). We sweep over À E (0.00, 0.01, 0.02, 0.03, 0.04, 0.10) and observe that the trend in the average gap closely mirrors that of 7. This similarity is expected, as both and À jointly influence the extent to which residual knowledge is removed during unlearning.
Different regularization strengths X.
Other methods to search for the vulnerable set. In the main text, we search for samples in the vulnerable (((x, y), 7) using Gaussian noise. Here, we extend the evaluation to include stronger adversarial methods, such as FGSM and PGD, with varying numbers of attack steps (indicated by the number following PGD in Table C.4). We observe that FGSM-a targeted attack-achieves MIA accuracy comparable to that of Re-train. Likewise, PGD with 5 steps also yields MIA accuracy close Re-train, indicating effective removal of forget-set information. These results suggest that targeted attacks (FGSM, PGD) are more effective than untargeted ones like Gaussian noise in eliminating residual knowledge, albeit at the cost of increased unlearning time.
Different samples size v. The sample size in (6) determines how extensively the vulnerable set V((x, y). r) is explored through sampling. high-dimensional settings, V((x, y), r) may contain a large number of perturbed variants. However, due to computational constraints, it is impractical to include all possible samples during training. Instead, we randomly draw samples from (((x, y), for each forget sample. We experiment with E {1, 2, 3, 4} and observe that the average gap achieved by RURK remains relatively stable across these values. This robustness is expected, as the loss term k( w, (x, y)) in (6) is computed as an average over the v sampled perturbations.
Other learning structures. Thus far, our evaluation of RURK has focused on models trained with ResNet-18. To assess its generality across architectures, we present results small CIFAR-5 using VGG-11 (Simonyan and Zisserman, 2014), as shown in Table C.5. Compared to NGD, RURK continues to achieve the smallest average gap. Figure C.13 further illustrates the residual knowledge comparison between RURK and NGD, demonstrating that RURK maintains lower and more stable residual knowledge across perturbations. These results confirm that RURK effectively removes forget-set information and controls residual knowledge across different network architcctures, including both plain convolutional networks (VGG-11) and those with residual connections (Rcs
40

---

Table C.4: Ablation study of RURK on small CIFAR-S with sample unlearning. Results are reported in the format d=a, indicating the mean a and standard deviation over independent trials. The absolute performance gap relative to Re- te-ariin is shown in (blue). For methods that fail to recover the forget-set knowledge within 30 training epochs, the re-learn time is reported as ">30". We bold the results with hyper- param--trameterss the same in the main text.
Evaltation Metrics
Purumeters
Other
Mothods
Retain Acc. (%)
Unlcam Arc. (%)
Test
MIA Aec. (%)
Avg. Gap
Re-leam Time Epoch)
Original Re-train
99.93 28.98gamm m0 (33
0.00 (8.33)
a6.37an.su00.577
2.67
7.82
00.96g (o 00)
8.33gx as (0.00)
84. 84.80go 4o. (8.00)
27.0005--000.00
0.00
3:30gn 47
BURK 0.00)
99.93
o.00=...(8..11)
96. 16.10
14.33 (12.67)
5.33
1.00
XURK 0.01)
90 03)
0.00
01.63
15.67
4.97
1.00
A 0.03 ==1
XURK (r 0.02)
99.80 g0 (0.07)
ar
84.27 04.2721.0000 (0.53)
22 00g (5.00o0((5.0)
2:90
1.00
RURK (r 0.03)
99. ar( (044)
5.67.a aul 86)
93 (0.3nm
33. 33g aa 33)
2.60
2. 2.00gm
* epoch
BURK (F 0.04)
08.96gass (0 (1.00)
13.3320 (5.00)
92. 32.37
38 08.672:co:(1100)
5.03
1.67
96.89ga.008
29.67 (21.34)
87.80
64.00gas..2(770000
14.60
1.67 g
RURK 0.00)
00.96 00)
o:
15.000..a2(0.200 20)
11.334o.eo(1.67))
6.05
1.00 oc
Ganssian
RURK (A- 0.01)
20.03:a.000.000 (3)
oz
04.6010..000
15.67
4.97
1.00
0.03
BURK (A- 0.02)
90 00.80=3.e000077
94 (5.66)
94.17a1.e (0.66)
22.67=
2.68
1.00
BURK
39.62.0ar(0.44
6.67.a 22l2 66)
98 93.832000000
38.33:12
2.60
epoch
ZURK 0.04)
13.38 ar
12.00gna0
39.67
5:25
1.07
ZURK (A) 0.10)
85. 35.01-2.e2
76.07 (08.34)
(08.3))
70.87
87.00ga.4400.000
40:30
2.00
ZURK (Gaussian)
99. 99.52 39.52:o.ar0..44
a.67
93.83: (0.97)
33.33:12 20. 33)
2.60
2.00
0.03
SURK (FGSM)
99.77 90.70=a.w 26)
4.53 (4.000
8d.17 (0.63) 63)
28 25.67=mmm:1.077
1.64
1.00
XIRK (PGD
g1 wo1
12:33 ar.00
a2.53 27)
16 67 (10.67
1.67
opoch -
V
SURK (PGD 10)
XIRK (PGD
90:03 (0.03)
90.85 (0.
o (7..66)
2.00
94.57g1.0 (0.23)
94.07
a0.00_ (7.00)
27.39_100 (0.399)
3.73
1.88
1.00 oc
1.00 oc
Gaussian
BURK
99. 52.52 ari 0.44)
5.6T 22 66)
93 93.83.0.00
33.33.12 33)
2.60
2.00
XURK
22 44)
1:33
0.03 # opoch -1
XIRK 3)
00.52 ar (0.44)
5.00g1 a (3.33)
83 13.8720..(0000) 00)
33.39_ (6.330)
2.76
1:33gg 17
BURK (1- 4)
90.52g a0 (0.44)
83.83go.0000.97)
33.33_ (6.33)
2.00
1.00
Small CIFAR-5 (Sample Unlearning)
0.00
01001
0.02
01003
01.14
0.10
FGSM Gaussian
1.15
1.00
0.75
Residual Knowledge (+5,))
1.10
0.75
0.50
0:00
0.00
0.02
0.02
0.00
0.02
0.00
0.02
0.00
1.15
1.15
1.0
0.75
1.10
1.0
0.50
1J15
0.5
0.25
0.00
0.00
0.00
0.02
0.00
0.02
0.00
0.00
0.00
0.02
0.00
0.00
PGD
1.1
1.05
0.925
1.000
1.0
1.0
1.00
I
0,915
0.900
0.00
0.02
0.00
0.02
0.00
0.02
0.00
0.02
0.00
0.02
0.00
0.02
Perturb Norm
Figure C.12: Residual knowledge (Sy) of the proposed RURK, Original, and other unlcarning methods on small CIFAR-5 with sample unlearning, evaluated under varying perturbation norms using untargeted Gaussian noise (p= 2), targeted FGSM (p- oo), and targeted PGD (p = oo) draw e 100 samples from (x,x,)). T).
41

---

Table C.S: The performance summary of various unlearning methods on small CIFAR-5 with VGG-11. I. Results are reported the format d=d, indicating the mean a and standard deviation over independent trials. The absolute performance gap relative to Re-train is shown in (blue). For methods that fail to recover the forget-set knowledge within 30 training epochs, the re-learn time is reported as ">30".
Evaluation Metries
Datasets
Methods
Re-lear Time (# Epoch)
Retain Acc. (5)
Unlearn Ace. (%)
Test Acc. (8)
MIA Acc. (%)
Avg. Gup
Origgimal
20.000 (2.56)
a.00 (15.00)
94. a4.10 (250))
g.00 (16.00)
9.01
Small CIFAR-
Be-train
97. 97.4440.0000.0000
15.00+0.00 (0.00)
91. (0.0=0..000.0000
16.00+ (0.00)
0.00
4.33
MGD
95. 02.000000022...... 2241
3.00=c.mo(12.000
91. 00 (0.20) 20)
2.0000.00 (5.00)
7.41
1.00a0.00
99.33 L.89)
7.00
92. a0 10)
22 22.00 (6.00)
4.25
g.00
Small CIFAR-5 (Sample Unlearning, VGG-11)
Original
NGD
RURK
FGSM Gaussian
1.25
1.2
Residual Knowledge (=(S,)
0.00
0.02
0.00
0,02
0.00
0.02
1.175
1.150
1.05
0.9
0.00
0.02
0.00
0.02
PGD
1.15
1.15
1.10
1.05
1.05
0.00
0.02
0.00
0:02
0.00
0.02
Perturb Norm
Figure C.13: Residual knowledge A. (S)) of the proposed RURK, Original, and NGD on small CIFAR-5 with sample unlearning and VGG-11 structure, evaluated under varying perturbation norms т. using untargeted Gaussian noise (p- 2), targeted FGSM (p oo), and targeted PGD (p 0o) to draw 100 samples from (x,
C.4
The prevalence of residual knowledge
In Figure 2, we report the estimated residual knowledge over the entire forget set (- We also provide per-sample estimates ((((x, y)) for each (x, E Sj, defined in Eq. (4). Table C.6 summarizes the proportion of forget samples exhibiting residual knowledge greater than one one-i.e. those still recognizable after unlearning-unde
different perturbation norms T, evaluated on small CIFAR-5 (with NTK) and CIFAR-10 (with NGD). Remarkably, even under imperceptibly small per- turbations (e.g., T 0.013), about 11% of the forget samples in small CIFAR-5 and more than 8% (approximately 160 samples) in CIFAR- 10 still exhibit residual knowledge above one. These findings highlight that residual knowledge is not only prevalent but also poses a significant privacy risk, emphasizing the need for stronger certification and mitigation techniques in machine unlcarning.
Table C.6: Percentage of forget samples that have residual knowledge A ((x, u)) large than 1.
Dansets
Mebhes
Grasiia Pesttttsion Noers
0.000
0.005
0.000
0.009
0.013
0.006
0.099
0,02
0.025
0.028
0051
Small CIFAR.5
MCK
0.00..00
3,00
5.00
6.00
11.00
14.00
26.00
34.00
30.00
45.00
48.00
0.00gg
1.85g as
5.25 m
7.00 550g m
6.5 s
9.0
890 s
8:10gmm
7.10gr
6.90gr as
42

---

C.5 Disagreement and unlearn accuracy on perturbed forget samples
We provide the adversarial disagreement ($ 4) and the unlearn accuracy on perturbed forget samples for selected baselines on both Small CIFAR-5 and CIFAR- 10 (cf. Table 1) in the following four tables.
In Table C.7, RURK achieves the lowest disagreement among all baselines when the perturbation norm is small (e-g., below 0.0125), consistent with the stable residual knowledge curves shown in Figure 2. As the perturbation magnitude increases, however, RURK reduces residual knowledge more aggressively, leading to higher disagreement- trade-off that arises naturally in multi-class settings, where low disagreement does not necessarily imply residual knowledge close to I as in binary cases. Similarly, in Table C.8, RURK attains unlearn accuracy on perturbed forget samples closest that of the re- re-trained model, particularly for small perturbations, indicating reduced distinguishability between the two models around the forget region. It worth noting that RURK is designed control residual knowledge rather than disagreement directly, as managing disagreement across multiple classes is inherently more complex.
Disagreement of perturbed forget samples on Small CIFAR-5 and CIFAR-I0 datasets over Table C.7:
varying perturbation norms т.
Datasets
Methodks
Gaussian Pertubation Norm y
0.0000
0.0031
0.0063
0,0094
0.0125
0.0157
0.0188
0.0220
0.0251
0,0282
0.0314
Original
0.1300
0.1273
0.1277
0.1332
0.1450
0.1471
0.1450
0.1472
0.1573
0.1665
0.1836
Ra-traim
0.0000
0.0000
0.0000
0.0000
0.0000
0.0000
0.0000
0.0000
0.0000
0.0000
0.0000
Small CIFAR-5
Fisher
0.1400
0.1345
0.1249
0.1188
0.1129
0.1181
0.1316
0.1532
0.1811
0.2054
0.2214
XTK
0.0500
0.0803
0.0894
0,0873
0.0805
0.0807
0.0753
0.0797
0.0303
0.1084
0.1285
0.0880
0.1510
Original
0.1020
0.1018
0.1148
0,1297
0,1462
0.1626
0.1735
0.1968
0.2136
0.2431
0.2593
CIFAR-10
Re- train
0.0000
0.0000
0.0000
0,0000
0.0000
0.0000
0.0000
0.0000
0.0000
0,0000
0.0000
BURK
0.1511
0.15-19
0.1701
0.1996
0.2395
0.2898
0.3335
0.3855
0.4353
0.1802
0.5310
Table C.8: Unlearn accuracy of the perturbed forget samples on the Small CIFAR-5 and CIFAR-IO datasets over varying perturbation norms т.
Datasets
Methods
Perturbution
0.0000
0.0031
0.0063
0.0099
0.0125
0.0157
0.0188
0.0220
0.0251
0.0282
0.0314
Original
0.0000
0.0000
0.0000
0.0004
0.0053
0.0205
0.0442
0.0680
0.0913
0.1161
0.1364
Re- train
0.1300
0.1273
0.1277
0.1336
0.1502
0.1665
0.1855
0.2082
0.2117
0.2746
0.3119
CIFAR-5
Fisher
0.4149
XTK
0,0700
0.0652
0.0681
0.0797
0.0986
0.1141
0.1333
0.1462
0.1700
0.1891
0.2120
0.1021
0.2110
0.2846
0.3258
0.3713
0.4124
Original
0.0351
0.0896
CIFAR-10
Re-traa train
0.1020
0.1048
0.1148
0.1301
0.1478
0. 0.1666
01 0.18
0.2046
0.2245
0.2478
0.2670
KGD
0.1360
0.1396
0.1754
0.3227
0.5014
0.6437
0.7423
0.8070
0.8336
0.8862
0.9088
RURK
0.1110
0.1387
0.1424
0.1845
0.2330
0.2930
0.3634
0.4256
0.4935
0.5581
0.6193
43


Reviews:
[{"review_id": "5WMLqgETRA", "reviewer": "Reviewer_LG56", "summary": "This manuscript investigates a novel privacy risk in machine unlearning, termed residual knowledge, where slightly perturbed forget samples may still be recognized by unlearned models. The authors provide both theoretical analysis and empirical evidence to demonstrate that this phenomenon arises inevitablely in high-dimensional settings. To mitigate this issue, the authors propose RURK, a fine-tuning-based method that penalizes the model’s ability to re-recognize perturbed forget samples. The authors conduct experiments on small-scale datasets to demonstrate the effectiveness of the proposed RURK", "questions": "- Could the authors provide more results on larger-scale datasets\n- Will the threat of residual knowledge also arise in large language models? I would appreciate it if the authors could include some empirical results on unlearning in LLMs.\n- Is the adversarial attack used during unlearning (i.e., Eq. 6) the same as the one used to evaluate residual knowledge threat? If so, could the authors provide more results under different types of adversarial attacks?", "limitations": "Yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "**Strengths**\n- This manuscript is well-written with clear logic and easy to follow.\n- The authors identify an interesting phenomenon (residual knowledge) in machine unlearning: slight perturbations of forget samples may still be correctly recognized by the unlearned model. This reveals a novel privacy risk for machine unlearning.\n- Both empirical evidence and theoretical insights are provided to support the analysis of model disagreement under sample perturbations.\n- The proposed method RURK, which incorporates adversarial perturbations into the unlearning process, is simple but reasonable to solve the identified issue.\n\n**Weaknesses**\n- The experimental evaluation is limited to small-scale datasets (small CIFAR-5 and CIFAR10) and backbones (ResNet18).\n- In experimental tables (e.g., Table 1), it would be helpful to indicate explicitly whether an increasing or decreasing value represents better performance.\n- The proposed RURK method does not appear to have significant improvements over certain existing baselines such as SSD, even in terms of mitigating the residual knowledge threat (e.g., as shown in Figure 2).\n- This manuscript lacks a conclusion section.", "was_revised": true, "final_justification": "My major concerns have been addressed after the rebuttal. I maintain my initial positive score.", "rebuttal": "We thank the reviewer for their time and thoughtful feedback. We’re glad to hear that the reviewer found the paper well-written and shares our interest in the newly identified privacy risk of residual knowledge. Below, we address each of the reviewer’s comments and suggestions in detail.\n\nFor Weakness 1, we understand the reviewer’s concern and have conducted larger-scale experiments on the ImageNet-100 dataset—a subset of ImageNet-1k containing 100 classes and approximately 1,300 images per class (for a total of 130k images)—using ResNet-50. In this setting, we select class 0 as the forget class and perform 50\\% random sample unlearning, evaluating the results as summarized in Table 1 below. Due to time constraints, we include a subset of competitive baselines for comparison. As shown in the results, RURK consistently achieves the smallest average gap compared to GD and NegGrad+, confirming its effectiveness even at scale. Additionally, we provide the tabular version of the residual knowledge corresponding to Figure 2. The results reveal that GD still exhibits residual knowledge $> 1$, since it only fine-tunes on retain samples and lacks explicit control over forget samples or residual knowledge. Similarly, SSD, when applied to a large architecture like ResNet-50, struggles to identify and suppress all neurons associated with the forget information, leading to suboptimal mitigation. While NegGrad+ does not suffer from residual knowledge in the same way, its residual knowledge remains farther from 1 compared to RURK, indicating less alignment with the re-trained model. These findings support that the phenomenon of residual knowledge is not limited to small-scale settings, and that RURK remains effective in controlling it, even on larger and more complex datasets. We will incorporate and refine this new empirical evidence in the revised version.\n\n Metrics   | Retain Acc.      | Unlearn Acc.     | Test Acc.        | Avg. Gap        \n---------- | ---------------- | ---------------- | ---------------- | ----------------\nOriginal   | 80.75±3.06(5.69) | 3.59±0.19(9.03)  | 72.67±1.18(4.23) | 6.32 \nRe-train   | 75.06±1.54(0.00) | 12.62±1.40(0.00) | 68.44±1.51(0.00) | 0.00 \nGD         | 82.29±0.37(7.23) | 6.26±2.38(7.13)  | 74.44±1.39(6.01) | 6.79\nNegGrad+   | 81.86±1.86(6.80) | 40.67±1.85(27.28)| 73.53±1.06(5.10) | 13.06\nSSD        | 80.35±3.00(5.30) | 90.15±13.92(76.77)|  71.81±1.05(3.38)| 28.48 \nRURK       | 83.09±0.26(8.03) | 11.62±0.69(1.00) | 75.19±0.59(6.75) | 5.26 \n\n\n$\\tau$     | 0.0000 | 0.0031 | 0.0063 | 0.0094 | 0.0125 | 0.0157 | 0.0188 | 0.0220 | 0.0251 | 0.0282 | 0.0314\n-------- | ------ | ------ | ------ | ------ | ------ | ------ | ------ | ------ | ------ | ------ | ------\nOriginal | 1.1000 | 1.0996 | 1.1018 | 1.1011 | 1.1077 | 1.1107 | 1.1174 | 1.1288 | 1.1303 | 1.1421 | 1.1553\nGD       | 1.0656 | 1.0624 | 1.0647 | 1.0650 | 1.0606 | 1.0575 | 1.0595 | 1.0610 | 1.0642 | 1.0665 | 1.0654\nNegGrad+ | 0.6915 | 0.6861 | 0.6791 | 0.6723 | 0.6701 | 0.6624 | 0.6588 | 0.6556 | 0.6469 | 0.6351 | 0.6163\nSSD      | 1.1046 | 1.1032 | 1.1067 | 1.1095 | 1.1133 | 1.1161 | 1.1233 | 1.1291 | 1.1367 | 1.1446 | 1.1514\nRURK     | 1.0000 | 0.9977 | 0.9954 | 0.9898 | 0.9846 | 0.9763 | 0.9624 | 0.9339 | 0.9108 | 0.8858 | 0.8633\n\n\n\nFor Weakness 2, we appreciate the reviewer’s insight and will clarify this point in the revised version. In Table 1, because we focus on the random-sample unlearning setting, the increasing or decreasing values of the metrics do not inherently indicate better performance. Rather, the unlearning performance is considered better when the metrics are closer to those of the re-train baseline, which serves as the gold standard. This is why we report the absolute gaps (in blue) between each method and the re-train side-by-side; the smaller the gap, the better the performance. In class unlearning, (results provided in Appendix Table C.2 and referenced in  Line 306) the criteria is the same, as the re-trained model is still the gold baseline to compare with. \n\nFor Weakness 3, we appreciate the reviewer’s feedback in helping us clarify this point. We would like to emphasize that the performance of the methods is best understood by jointly considering Table 1 and Figure 2. This is because we are concerned not only with standard metrics such as retain and unlearn accuracy (reported in Table 1), but also with residual knowledge (illustrated in Figure 2). As discussed in the main text (Line 354), SSD tends to aggressively erase forget-set information, resulting in high unlearn accuracy but at the cost of test accuracy dropping below 90\\%. Furthermore, although SSD and RURK exhibit similar trends in Figure 2, RURK is more resistant to residual knowledge, as indicated by its longer plateau. Additionally, RURK offers a tunable mechanism for controlling residual knowledge via the parameter $\\tau$ of the ball $\\mathcal{B}_p(x, \\tau)$ in Eq. (6). We included an ablation study of the hyper-parameters of RURK in Appendix C.3, Table C.4 and Figure C.11---parameter $\\tau$ effectively reflects on the increase of unlearning accuracy and the plateau of the residual knowledge curves.\n\nFor Weakness 4, due to space constraints, we chose to omit a summary of the work and instead prioritized discussions on limitations, future directions, and broader impact in Section 6 (Final Remark). However, we are happy to include a conventional concluding paragraph in the revised version.\n\nFor Question 1, please refer to our response in Weakness 1. \n\nFor Question 2, we appreciate the reviewer’s suggestion. Indeed, unlearning has became an active area of research in LLMs (Lines 92-93), and it presents significantly greater complexity due to the inherent nature of language. In particular, the unlearning process---including the definitions of forget/retain samples and the evaluation metrics---differs considerably between LLMs and classification settings (e.g., the baseline methods discussed in Line 307). In this paper, our primary goal is to highlight a new privacy risk: residual knowledge. To do so, we focus on classification settings, which allow for clearer and more controlled evaluation. We will make this focus more explicit in the revised version. That said, several existing works [R1–R3] have demonstrated that sensitive information can still be reverse-engineered from LLMs even after unlearning—either through relearning (using different text corpora with similar contexts) or via jailbreaking techniques. While these observations are not directly equivalent, they are conceptually related to the residual knowledge phenomenon we investigate in the classification setting. We are excited to further explore this connection in future work.\n\nFor Question 3, yes---Figure 2 presents results where RURK is trained using a Gaussian noise attack, and the residual knowledge is also evaluated with respect to the same Gaussian noise, ensuring a fair comparison. We also include results for other adversarial attacks, such as FGSM and PGD, where both RURK and the residual knowledge evaluation are based on the same respective attack. These additional results are provided in Appendix Figure C.11 and referenced in Lines 373–374.\n\nThanks again for the comment! We are happy to provide more information or answer any follow-up questions. Please consider raising our score if you think we have addressed your main concerns.\n\n[R1] Hu, S., Fu, Y., Wu, S. and Smith, V., 2024. Jogging the memory of unlearned models through targeted relearning attacks. In ICML 2024 Workshop on Foundation Models in the Wild.\n\n[R2] Shumailov, I., Hayes, J., Triantafillou, E., Ortiz-Jimenez, G., Papernot, N., Jagielski, M., Yona, I., Howard, H. and Bagdasaryan, E., 2024. Ununlearning: Unlearning is not sufficient for content regulation in advanced generative ai. arXiv preprint arXiv:2407.00106.\n\n[R3] Liu, S., Yao, Y., Jia, J., Casper, S., Baracaldo, N., Hase, P., Yao, Y., Liu, C.Y., Xu, X., Li, H. and Varshney, K.R., 2025. Rethinking machine unlearning for large language models. Nature Machine Intelligence, pp.1-14."}, {"review_id": "Oa82XzgW2Z", "reviewer": "Reviewer_sUpr", "summary": "This paper investigates a new privacy risk in machine unlearning, termed residual knowledge, where an unlearned model retains latent traces of forget samples, observable through small adversarial perturbations, despite formal statistical indistinguishability from a re-trained model. The authors provide theoretical arguments, including isoperimetric inequality-based proofs, suggesting that disagreement between unlearned and re-trained models under perturbations is inevitable in high-dimensional settings. They introduce a metric to quantify residual knowledge and propose RURK, a fine-tuning strategy that penalizes correct classification of perturbed forget samples, to mitigate this risk. The paper presents empirical results on CIFAR-5 and CIFAR-10, evaluating RURK against several baseline unlearning methods.", "questions": "- Can the authors provide a more detailed justification for the use of isoperimetric inequalities and high-dimensional geometric reasoning, particularly regarding how these assumptions map to practical deep learning models and real-world data distributions? \n\n- How do the authors envision applying RURK in deployment settings where the forget set is no longer accessible? Can they propose a realistic variant that does not require the forget set during mitigation? \n\n- Can the authors provide concrete runtime and resource overhead measurements for RURK, including comparisons to existing methods, especially for larger models or datasets? \n\n- Would the authors consider extending the experimental evaluation beyond CIFAR to larger-scale or non-vision tasks to validate both the existence and mitigation of residual knowledge in more realistic settings? \n\n- Is there a reliable, model-agnostic way to estimate residual knowledge in practice without requiring access to a re-trained model? The paper's current proxy approach lacks rigorous evaluation in this regard.", "limitations": "Partially. While the authors acknowledge theoretical and computational limitations, they understate critical practical concerns, including reliance on forget set access, scalability of RURK, and the narrow experimental scope.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "# Strengths: \n\n- The paper formalizes residual knowledge as a distinct privacy concern in unlearning, providing a new perspective on the limitations of current guarantees. \n\n- The theoretical treatment leverages established tools from geometric probability to support the inevitability of local disagreement under adversarial perturbations. \n\n- The proposed RURK method is simple in formulation and integrates with existing gradient-based unlearning pipelines. \n\n- Extensive experiments are conducted on two unlearning scenarios (CIFAR-5, CIFAR-10) with 11 competitive baseline methods, covering multiple evaluation dimensions including accuracy, MIA, and re-learn time. \n\n \n\n# Weaknesses: \n\n- The theoretical results rely heavily on the isoperimetric inequality on unit spheres, assuming normalized data in high-dimensional Euclidean space. However, in practical deep learning settings, especially with complex data distributions and learned feature spaces, the applicability of this geometric abstraction is questionable. The authors do not justify how these assumptions transfer to real neural network representations. \n\n- The formal inevitability of disagreement is interesting theoretically but lacks quantitative grounding in realistic settings. The paper provides no evidence on the actual magnitude of this residual knowledge in large, real-world models beyond small CIFAR benchmarks, making the practical severity unclear. \n\n- The empirical evaluation is narrow, focusing only on vision tasks with modest-scale datasets (CIFAR-5 and CIFAR-10) and small-to-mid-size models (ResNet-18, VGG). There is no experimental evidence on modern large-scale architectures or other modalities like NLP or diffusion models, where both unlearning and adversarial perturbations exhibit different behaviors. \n\n- The proposed RURK method requires direct access to the forget set during fine-tuning to generate adversarial perturbations for mitigation. This undermines practical applicability, especially in privacy-sensitive scenarios where the forget set may be permanently erased after an unlearning request. The paper does not address this constraint meaningfully. \n\n- The notion of residual knowledge is formalized based on comparing outputs of unlearned and re-trained models on perturbed forget samples. However, in practice, access to a fully re-trained model is typically unavailable after deployment. The authors mention a practical proxy but fail to rigorously evaluate how well this proxy reflects true residual knowledge in realistic scenarios. \n\n- The computational cost of RURK, especially regarding the inner loop required to generate vulnerable perturbations during fine-tuning, is not quantified. Given that adversarial training is already known to be computationally expensive, the feasibility of RURK for large models or datasets remains unclear. \n\n- The analysis of baseline methods in Table 1 is surface level. While raw performance numbers are reported, the authors miss an opportunity to investigate why specific baselines (e.g., Fisher, NTK, GA) fail in terms of residual knowledge, leaving the reader with limited insights into the underlying mechanisms. \n\n- Some claims about the generality of the findings (e.g., in line 13, that residual knowledge is \"prevalent across existing unlearning methods\") are overstated given the narrow experimental scope and lack of evidence beyond CIFAR-5 and CIFAR-10.", "was_revised": true, "final_justification": "My confidence of the paper is strengthened by the addition emperical results provided by the authors.", "rebuttal": "We thank the reviewer for their time and constructive feedback. We address the weaknesses and questions point-by-point below.\n\nFor Weakness 1 and Question 1, we appreciate the reviewer’s observation and will further clarify in the revision. While assuming samples lie on the unit sphere is strong, the core idea of the isoperimetric inequality—that a small-volume set must have a large boundary—extends to more realistic settings. Specifically, Prop 2.8 in [R1] generalizes this principle to the unit cube, which better reflects image data where pixel values are normalized in $[0, 1]$. Nonetheless, Prop. 2 remains meaningful: it shows disagreement on perturbed samples even under the stricter unit sphere assumption, making the result compelling under broader assumptions like the unit cube. We will extend Propo 2 accordingly in the revision.\n\nFor Weakness 2, Weakness 8, and Question 4, we conducted larger-scale experiments on the ImageNet-100 dataset (100 classes, ~130k images) using ResNet-50. We unlearn 50\\% of class 0 via random sampling and report results in the updated Table 1 below. Due to time constraints, we include a subset of strong baselines. RURK consistently achieves the smallest average gap relative to the re-trained model, outperforming GD and NegGrad+, and confirming its scalability. We also provide a tabular version of the residual knowledge results from Figure 2. GD shows residual knowledge $> 1$, as it only fine-tunes on retain samples without explicitly mitigating forget signals. SSD struggles with large models like ResNet-50, failing to fully suppress neurons encoding forget information. While NegGrad+ avoids the same failure mode, its residual knowledge remains farther from 1 than RURK, indicating weaker alignment with the re-trained model. These findings confirm that residual knowledge persists at scale and that RURK effectively mitigates it. We will incorporate and refine this evidence in the revised manuscript.\n\n Metrics   | Retain Acc.      | Unlearn Acc.     | Test Acc.        | Avg. Gap        \n-- | -- | -- | -- | --\nOriginal   | 80.75±3.06(5.69) | 3.59±0.19(9.03)  | 72.67±1.18(4.23) | 6.32 \nRe-train   | 75.06±1.54(0.00) | 12.62±1.40(0.00) | 68.44±1.51(0.00) | 0.00 \nGD         | 82.29±0.37(7.23) | 6.26±2.38(7.13)  | 74.44±1.39(6.01) | 6.79\nNegGrad+   | 81.86±1.86(6.80) | 40.67±1.85(27.28)| 73.53±1.06(5.10) | 13.06\nSSD        | 80.35±3.00(5.30) | 90.15±13.92(76.77)|  71.81±1.05(3.38)| 28.48 \nRURK       | 83.09±0.26(8.03) | 11.62±0.69(1.00) | 75.19±0.59(6.75) | 5.26 \n\n$\\tau$     | 0.0000 | 0.0031 | 0.0063 | 0.0094 | 0.0125 | 0.0157 | 0.0188 | 0.0220 | 0.0251 | 0.0282 | 0.0314\n-- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | --\nOriginal | 1.1000 | 1.0996 | 1.1018 | 1.1011 | 1.1077 | 1.1107 | 1.1174 | 1.1288 | 1.1303 | 1.1421 | 1.1553\nGD       | 1.0656 | 1.0624 | 1.0647 | 1.0650 | 1.0606 | 1.0575 | 1.0595 | 1.0610 | 1.0642 | 1.0665 | 1.0654\nNegGrad+ | 0.6915 | 0.6861 | 0.6791 | 0.6723 | 0.6701 | 0.6624 | 0.6588 | 0.6556 | 0.6469 | 0.6351 | 0.6163\nSSD      | 1.1046 | 1.1032 | 1.1067 | 1.1095 | 1.1133 | 1.1161 | 1.1233 | 1.1291 | 1.1367 | 1.1446 | 1.1514\nRURK     | 1.0000 | 0.9977 | 0.9954 | 0.9898 | 0.9846 | 0.9763 | 0.9624 | 0.9339 | 0.9108 | 0.8858 | 0.8633\n\n\nFor Weakness 3, please refer to the response on large-scale classification experiments. We also thank the reviewer for drawing attention to generative tasks. In such settings—e.g., image or text generation—unlearning typically targets entire concepts or classes via labels or prompts, making our definition of residual knowledge less directly applicable. However, exceptions exist. In image-to-image (I2I) generation (e.g., Fan et al., 2023; Li et al., 2024), inputs are partial images, and perturbing them can still trigger forgotten content. Language generation is even more complex due to its compositional nature (Lines 92–93) and differing forget/retain semantics (Line 307), complicating the definition and evaluation of residual knowledge. Our goal in this paper is to introduce and study residual knowledge as a privacy risk in classification, where evaluation is clearer and more controlled. We will clarify this scope in Section 2. That said, prior work [R2–R4] shows that LLMs can still leak forgotten content through context relearning or jailbreaking—phenomena conceptually related to residual knowledge, which we aim to investigate in future work.\n\nFor Weakness 4 and Question 2, we appreciate the reviewer for raising this point. In classification unlearning, access to forget samples is standard practice in recent work—including the baselines we use (Lines 307–327, Appendix B.2), such as Fan et al. (2023), Kurmanji et al. (2024), and Graves et al. (2021). This assumption is also practical: if a user requests deletion, they must either provide the sample or ask the provider to retrieve it. Unlike generative unlearning (which targets entire concepts), classification unlearning becomes infeasible if neither forget nor retain samples are accessible. Moreover, both our residual knowledge definition (Eq. (4)) and RURK objective (Eq. (6)) rely on forget samples. Fine-tuning on retain samples alone cannot ensure residual knowledge is removed. Finally, RURK is an integrated unlearning process—not a post hoc correction—explicitly designed to eliminate residual knowledge at the time of unlearning. We will clarify these points in revised Sections 2 and 4.\n\nFor Weakness 5 and Question 5, we appreciate the reviewer’s concern and offer the following clarification. The notions of $(\\epsilon, \\delta)$-unlearning and residual knowledge (Lines 108–116) are theoretical constructs that require the unlearned model to closely match a re-trained model—either in weight distribution or output behavior on perturbed forget samples. RURK, however, avoids dependence on the re-trained model during unlearning. Rather than forcing residual knowledge to be exactly 1, RURK constrains it to be ≤1, since values above 1 signal a stronger privacy risk. This design choice enables practical unlearning without retraining, as explained in Lines 261–266 and supported by Figure 2. We acknowledge this trade-off in Section 6. Computing the proxy term (ii) in Eq. (6) is analogous to using empirical adversarial examples to estimate adversarial loss. We will highlight this rationale and our design decisions more explicitly in the revised Section 4.\n\nFor Weakness 6 and Question 3, we thank the reviewer for pointing out the lack of clarity. In the main text, we set $v=1$ (i.e., one sample from the vulnerable set using Gaussian noise) in Term (ii) of Eq. (6). RURK’s overhead grows linearly with $v$, but since sampling is done in constant time, its overall complexity remains comparable to other gradient-based methods like GD, NGD, and NegGrad+ (Lines 347–349). Appendix C.3 provides an ablation on $v$. For the larger-scale experiment (on an NVIDIA 8xA10g node) shown above, RURK completes in 42.95 minutes, compared to 1920.48 minutes for full retraining. We will add a more detailed runtime comparison in the revision.\n\nFor Weakness 7, we thank the reviewer for prompting a more comprehensive discussion. In addition to Lines 345–374, we offer the following clarifications. Figure 2 uses different y-axis scales across subfigures. NTK exhibits residual knowledge because it linearizes the model via the neural tangent kernel and ignores higher-order output terms. Fisher aggressively adds Gaussian noise to model weights to erase forget information but harms utility (test accuracy). Fine-tuning-only methods like GD and CF-k show high residual knowledge, underscoring the need to access forget samples—as RURK does. NGD, though also fine-tuning only on retain samples, adds controlled weight noise (similar to Fisher), reducing residual knowledge more effectively than GD and CF-k, but still less than RURK. GA fine-tunes on forget samples using reversed gradients, over-erasing forget information and deviating from the re-trained model. EU-k updates only the last layers on forget samples, leaving residuals in earlier representations. SSD selectively dampens weights and behaves similarly to GA due to over-removal. SCRUB performs comparably to RURK but is more expensive due to its reliance on a teacher model. NegGrad+ has a similar objective to RURK but retains more residual knowledge, validating the effectiveness of Term (ii) in Eq. (6). We will incorporate these points in the revision.\n\nThanks again! We are happy to address any additional questions. If you think that we’ve adequately addressed the limitations, we would greatly appreciate your consideration in updating the score accordingly.\n\n[R1] Ledoux, M., 2001. The concentration of measure phenomenon (No. 89).\nAmerican Mathematical Soc.\n\n[R2] Hu, S., Fu, Y., Wu, S. and Smith, V., 2024. Jogging the memory of unlearned models through targeted relearning attacks. In ICML 2024 Workshop on Foundation Models in the Wild.\n\n[R3] Shumailov, I., Hayes, J., Triantafillou, E., Ortiz-Jimenez, G., Papernot, N., Jagielski, M., Yona, I., Howard, H. and Bagdasaryan, E., 2024. Ununlearning: Unlearning is not sufficient for content regulation in advanced generative ai. arXiv preprint arXiv:2407.00106.\n\n[R4] Liu, S., Yao, Y., Jia, J., Casper, S., Baracaldo, N., Hase, P., Yao, Y., Liu, C.Y., Xu, X., Li, H. and Varshney, K.R., 2025. Rethinking machine unlearning for large language models. Nature Machine Intelligence, pp.1-14."}, {"review_id": "buMTxFwIk2", "reviewer": "Reviewer_2RCn", "summary": "The authors examine what they see as an overlooked threat in the field of machine unlearning: though the samples themselves may appear to be unlearned (as gauged by classification accuracy on these samples), perturbations of the data in $\\mathcal{S}_f$ are still classified with high accuracy by the model.  The authors propose a method of unlearning designed to reduce this threat, and they provide a novel notion of residual knowledge.", "questions": "- Why is your unlearn accuracy 0.00 for the original method, and why are the error bars +/- 0?  This seems like a fishy estimate of the error bars given that the unlearn set is randomly selected from the data.\n- The motivation for the definition of unlearning comes from differential privacy, where the randomness comes from the optimization process.  However, in your definition of residual knowledge, the randomness comes instead from sampling inside of $B_r(p)$ where $p\\in \\mathcal{S}_f$.  How do you reconcile these two definitions of randomness?  It's not entirely clear to me that this is the \"right\" definition of residual knowledge.  Can you motivate this a bit better?\n- As a follow-up to my previous question, in your definition of residual knowledge, you write that randomness comes from sampling inside of the ball around the forget point.  However, you also mention that you are adversarially perturbing.  How are you sampling from $B_r(p)$?  If you are adversarially perturbing, then this doesn't seem like a very realistic threat.\n- I find it a little bit concerning that RURK causes larger adversarial perturbations to do worse against the unlearned model.  Can you explain this behavior?\n- Are you able to convert your insight into a membership inference attack against unlearned models?  This seems like a good practical application.\n- What is the implication of this research for generative models?  If you don't already have the training data you can't perturb it, so I'm not entirely sure how it helps you extract data or do something harmful if perturbations reveal membership in the training data.", "limitations": "- See the questions about definitions and adversarial perturbations.\n-The unlearning accuracy seems way too high for RURK-- you only need to match the test data's accuracy.\n- The experiments only look at classifiers on one dataset (counting CIFAR-5 and CIFAR-10 as the same), making them effectively toy examples relative to the scale of modern machine learning.  This is, of course, very difficult to fix during a review period. \n- I think that it would be nice to see an experiment with randomly labeled data or a mislabeled subset as the unlearning dataset.  In the current experiments, it's not clear to me whether perturbations around the training data recover the original labels because of residual knowledge or because you've unlearned too heavily exactly at the points in $\\mathcal{S}_r$, but once you move away from $\\mathcal{S}_r$ the classifier should function as normal.", "rating": 5, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n- In general, the idea that unlearning doesn't handle perturbations of the samples themselves is really interesting and indeed overlooked.  This was a good insight.\n- The authors do a good job defining their terms with mathematical precision, and providing mathematical support for their ideas.  This is well done.\n- The paper is clearly written and proper care is taken.", "was_revised": true, "final_justification": "The reviewers satisfied my concerns in their rebuttal.", "rebuttal": "Thank you for your time and thoughtful review. We’re glad you recognized the overlooked challenge of handling perturbations in unlearning and appreciated the mathematical formulation. We clarify limitations and address your questions in detail.\n\nFor Question 1, the original model, trained on both retain and forget samples, achieves 100\\% accuracy on the forget set—this is the forget accuracy. However, the Unlearn Accuracy in Table 1 is defined as 1 - forget accuracy (Footnote 6 Page 8), following the evaluation in Fan et al. (2023, Line 330). We ensure the original model performs well to allow a fair comparison; otherwise, it would be unclear if unlearning occurred. This setup leads to consistent 100\\% forget accuracy (i.e., 0\\% unlearn accuracy) and thus zero std. Forget samples are randomly selected, and results are averaged over trials with different model initializations.\n\nFor Questions 2 and 3, we appreciate the reviewer for highlighting this distinction. Our framework involves two sources of randomness: (1) from the unlearning algorithm (e.g., model initialization), and (2) from perturbations of forget samples. Section 3 focuses on the first to show that disagreement between unlearned and re-trained models on perturbed forget samples is inevitable (Prop. 2). But showing existence is not enough—we also need to quantify this disagreement in practice. Thus, in Section 4, we fix a realization of models $m$ and $a$ (sampling once from the first randomness) and evaluate disagreement under the second, by perturbing forget samples (Lines 221–222, Footnote 4). This motivates our definition of residual knowledge (Lines 229–235) and the design of RURK to reduce it. While it is possible to define residual knowledge using both randomness, doing so would require characterizing the distribution of $m$ or $a$—an intractable task without strong assumptions (Guo et al., 2020; Chourasia and Shah, 2023; Chien et al., 2024)—making it impractical for methods like RURK. To capture the second randomness, we perturb forget samples using various methods (Lines 361–374), including Gaussian sampling from the $\\mathcal{B}_p(\\tau)$ ball and adversarial methods like PGD, which can be viewed as non-isotropic sampling. We will revise the draft to clarify this distinction.\n\nFor Question 4, assumed referring to Fig 2, which evaluates residual knowledge—how consistently a perturbed forget sample is treated by the unlearned model relative to the re-trained model. Ideally, an unlearned model should produce a flat line at $y = 1$ across different $\\tau$ (x-axis), indicating full alignment with the re-trained model. For instance, in the bottom-right panel (RURK), we observe residual knowledge close to 1 for all $\\tau \\leq 0.01$, showing greater robustness than other baselines (Lines 366–372). Together with Table 1, these results highlight RURK’s overall strength: minimal accuracy gaps, while effectively reducing residual knowledge. We will clarify this in the revised discussion of Figure 2 in Section 5.\n\nFor Question 5, this connection is interesting. In MIA against an unlearned model, an adversary may exploit side information—such as the minimal perturbation needed for the model to re-recognize a forget sample. A small perturbation norm may suggest prior inclusion in the training set, increasing the chance of successful inference. This presents a compelling direction for future MIA research in unlearning. Emerging work has begun leveraging such side information—e.g., model variation near training points [R1, R2] or perturbation dynamics [R3]—to strengthen attacks. We will expand the discussion in the revision to motivate further exploration.\n\nFor Question 6, we thank the reviewer for highlighting the distinction. In classification, unlearning targets individual samples (random or all samples in a class)—enabling direct perturbation to assess residual knowledge. In contrast, generative tasks often focus on concept unlearning, where inputs are labels or prompts, making our definition less directly applicable. However, exceptions exist. In image-to-image generation (e.g., Fan et al. 2023; Li et al. 2024), inputs are partial images, where perturbations may still trigger forgotten content. Language generation adds further complexity due to its compositional nature (Lines 92–93), and forget/retain definitions differ significantly (Line 307), making residual knowledge harder to formalize. Our focus is to introduce and analyze residual knowledge in a controlled classification setting, where evaluation is clearer and more comparable—we will clarify this in Section 2. That said, prior work [R4–R6] shows LLMs can still leak sensitive information post-unlearning, through relearning or jailbreaking—behaviors conceptually related to residual knowledge, which we aim to explore in future work.\n\nFor Limitation 1, thanks for raising this point. We did not tune RURK’s unlearn accuracy to match the re-trained model. Instead, we ran unlearning until the RURK loss in Eq. (6) converged, allowing a fair comparison across methods. Under this setup, RURK achieves the second-smallest unlearn accuracy gap relative to the re-train baseline (highlighted in blue in Table 1), closely following NGD. This indicates comparable performance. Importantly, in random sample unlearning, absolute unlearn accuracy is less meaningful than the gap from the re-trained model—consistent with the evaluation protocol in Fan et al. (2023).\n\nFor Limitation 2, we conducted larger-scale experiments on the ImageNet-100 dataset (100 classes, ~130k images) using ResNet-50. We unlearn 50\\% of class 0 via random sampling and report results in the updated Table 1 below. Due to time constraints, we include a subset of strong baselines. RURK consistently achieves the smallest average gap relative to the re-trained model, outperforming GD and NegGrad+, and confirming its scalability. We also provide a tabular version of the residual knowledge results from Figure 2. GD shows residual knowledge $> 1$, as it only fine-tunes on retain samples without explicitly mitigating forget signals. SSD struggles with large models like ResNet-50, failing to fully suppress neurons encoding forget information. While NegGrad+ avoids the same failure mode, its residual knowledge remains farther from 1 than RURK, indicating weaker alignment with the re-trained model. These findings confirm that residual knowledge persists at scale and that RURK effectively mitigates it. We will incorporate and refine this evidence in the revised manuscript.\n\n Metrics   | Retain Acc.      | Unlearn Acc.     | Test Acc.        | Avg. Gap        \n-- | -- | -- | -- | --\nOriginal   | 80.75±3.06(5.69) | 3.59±0.19(9.03)  | 72.67±1.18(4.23) | 6.32 \nRe-train   | 75.06±1.54(0.00) | 12.62±1.40(0.00) | 68.44±1.51(0.00) | 0.00 \nGD         | 82.29±0.37(7.23) | 6.26±2.38(7.13)  | 74.44±1.39(6.01) | 6.79\nNegGrad+   | 81.86±1.86(6.80) | 40.67±1.85(27.28)| 73.53±1.06(5.10) | 13.06\nSSD        | 80.35±3.00(5.30) | 90.15±13.92(76.77)|  71.81±1.05(3.38)| 28.48 \nRURK       | 83.09±0.26(8.03) | 11.62±0.69(1.00) | 75.19±0.59(6.75) | 5.26 \n\n$\\tau$     | 0.0000 | 0.0031 | 0.0063 | 0.0094 | 0.0125 | 0.0157 | 0.0188 | 0.0220 | 0.0251 | 0.0282 | 0.0314\n-- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | --\nOriginal | 1.1000 | 1.0996 | 1.1018 | 1.1011 | 1.1077 | 1.1107 | 1.1174 | 1.1288 | 1.1303 | 1.1421 | 1.1553\nGD       | 1.0656 | 1.0624 | 1.0647 | 1.0650 | 1.0606 | 1.0575 | 1.0595 | 1.0610 | 1.0642 | 1.0665 | 1.0654\nNegGrad+ | 0.6915 | 0.6861 | 0.6791 | 0.6723 | 0.6701 | 0.6624 | 0.6588 | 0.6556 | 0.6469 | 0.6351 | 0.6163\nSSD      | 1.1046 | 1.1032 | 1.1067 | 1.1095 | 1.1133 | 1.1161 | 1.1233 | 1.1291 | 1.1367 | 1.1446 | 1.1514\nRURK     | 1.0000 | 0.9977 | 0.9954 | 0.9898 | 0.9846 | 0.9763 | 0.9624 | 0.9339 | 0.9108 | 0.8858 | 0.8633\n\nFor Limitation 3, we will clarify this point in the revision. The approach of obfuscating forget sample labels—via randomization or mislabeling—is effectively captured by the GA method in Table 1. GA pushes predictions away from the original labels, as described in Graves et al. (2021, Page 3), and is already included in our evaluation. Table 1 compares standard unlearning metrics (Lines 117–126, 328–344) to ensure unlearning is not overly aggressive. Even with strong unlearning, a well-behaved model should not re-identify small perturbations of forget samples—unlike a re-trained model, which naturally fails to recognize them. A properly unlearned model should produce uncertain outputs, leading to low residual knowledge—precisely what our metric captures.\n\nThank you again! We’re happy to provide any additional information. If you feel that we’ve adequately addressed your main concerns, we would greatly appreciate your consideration in updating the score accordingly.\n\n[R1] Jalalzai, H., Kadoche, E., Leluc, R. and Plassier, V., 2022. Membership inference attacks via adversarial examples. arXiv:2207.13572.\n\n[R2] Del Grosso, G., Jalalzai, H., Pichler, G., Palamidessi, C. and Piantanida, P., 2022. Leveraging adversarial examples to quantify membership information leakage. In Proceedings of CVPR.\n\n[R3] Xue, J., Sun, Z., Ye, H., Luo, L., Chang, X., Tsang, I. and Dai, G., 2025. Privacy Leaks by Adversaries: Adversarial Iterations for Membership Inference Attack. arXiv:2506.02711.\n\n[R4] Hu, S., Fu, Y., Wu, S. and Smith, V., 2024. Jogging the memory of unlearned models through targeted relearning attacks. In ICML 2024 Workshop on Foundation Models in the Wild.\n\n[R5] Shumailov, I., Hayes, J., Triantafillou, E., Ortiz-Jimenez, G., Papernot, N., Jagielski, M., Yona, I., Howard, H. and Bagdasaryan, E., 2024. Ununlearning: Unlearning is not sufficient for content regulation in advanced generative ai. arXiv:2407.00106.\n\n[R6] Liu, S., Yao, Y., Jia, J., Casper, S., Baracaldo, N., Hase, P., Yao, Y., Liu, C.Y., Xu, X., Li, H. and Varshney, K.R., 2025. Rethinking machine unlearning for large language models. Nature Machine Intelligence."}, {"review_id": "zq0dCNVMhl", "reviewer": "Reviewer_eKwF", "summary": "Machine unlearning is an approach to remove specific data without re-training the model. While its effectiveness has been proven, this paper theoretically proves that the unlearned model may suffer from perturbed forget samples, raising a new risk of unlearning. With the DP framework, they introduce model distinguishability and sample disagreement, depending on which they formulate the vulnerability as residual knowledge and propose RURK to constrain the model's ability to recognize perturbed forget samples. Extensive results show the effectiveness of their method.", "questions": "- I may miss something, but I didn't find any illustration on how to get Fig.1. I think it would be better to give a brief introduction.\n- For Proposition 2:\n    - First, the assumption that the sample space is a sphere is too strict and not very practical.\n    - Second, when $\\epsilon=\\delta=0$, why does Eq.3 become zero, as the first term $\\frac{2\\delta}{1-e^{\\epsilon}}$ is $\\frac{0}{0}$?\n- As $r_{\\tau}$ is a relative ratio, is it enough to measure the residual knowledge inside the unlearned model $m$? For example, if $\\Pr[m(x')=y] = N \\Pr[a(x')=y] = N\\xi$ where $\\xi \\to 0^+$ and $N$ is a large constant. Thus, $r_{\\tau}=N$ can be large but $\\Pr[m(x')=y]$ is small. In that case, can we say the unlearned model has a large amount of residual knowledge since both models are unconfident and unlikely to classify the label correctly?", "limitations": "yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strength:\n\n- The paper is well written and easy to follow.\n- The problem is well motivated. While most prior work focuses on the forget performance of unlearned samples, that of perturbed forget samples is not well studied, though interesting and important.\n- The experiments are comprehensive, including 11 baselines, 5 metrics, and 2 datasets to evaluate their effectiveness in multiple dimensions.\n\nWeakness:\n- The size of the models and datasets used in this paper is too small. As the unlearning of generative models is receiving more and more attention and is a critical part in unlearning, adding experiments on diffusion models can enhance this paper's impact, though I don't see a direct or easy application of the proposed method on diffusion models.\n- The assumptions of some theories are not that practical or maybe there are some mistakes. See questions.\n- The relative ratio $r_{\\tau}$ seems insufficient to measure the residual knowledge. See questions\n- The introduction of residual knowledge is confusing to me. As Sec.3 has proven that an unlearned model can still correctly predict the perturbed forget samples, it is enough to elicit the RURK method, which unlearns the perturbed forget samples. Why do we need residual knowledge and $r_{\\tau}$? Does the RURK loss reduce $r_{\\tau}$? Adding more discussion would enhance the understanding of this paper.", "was_revised": true, "final_justification": "After multiple rounds of discussion, I acknowledge the contribution and novelty of this paper and find it quite interesting. As privacy risks become increasingly important, unlearning has emerged as a critical technique to address them. However, the effect of unlearning under perturbation remains underexplored. This paper provides a theoretical analysis showing that, even when unlearned and re-trained models are indistinguishable, they may still disagree on adversarial samples. The authors introduce the concept of residual knowledge as a special case to quantify this disagreement and propose the RURK method to mitigate it. I believe this work will draw the community’s attention to the privacy risks of unlearning in the presence of adversarial examples.\n\nDespite its strengths, the paper’s main limitation lies in its narrow scope. It focuses solely on (small) classification models, overlooking emerging risks in generative models such as diffusion models. Extending the analysis to broader architectures and tasks would significantly enhance the impact and generality of the results.\n\nIn light of our discussions, I am glad to raise my score to 5 with higher confidence.", "rebuttal": "We thank the reviewer for their time and effort. We are pleased that the reviewer found the paper well-written, well-motivated, and supported by comprehensive experiments. Below, we systematically address each of the weaknesses and questions raised in the review.\n\nFor Weakness 1, we understand the reviewer’s concern. Indeed, for a diffusion model (generative tasks), unlearning often focuses on concept unlearning, where the aim is to forget an entire class or concept represented by a label. In these scenarios, the notion of residual knowledge, as defined in our paper, does not directly apply, since the input is not a sample but a label or prompt. However, there are notable exceptions. For instance, in image-to-image (I2I) generation (as shown in Fan et al. (2023, Figure 6) and Li et al. (2024, Figure 1)), the input is a partial image (e.g., an unmasked region) rather than a label. In such cases, residual knowledge may still manifest: perturbations to the input region could lead the model to regenerate forgotten content. However, we have conducted larger-scale experiments on the ImageNet-100 dataset—a subset of ImageNet-1k containing 100 classes and approximately 1,300 images per class (for a total of 130k images)—using ResNet-50. In this setting, we select class 0 as the forget class and perform 50\\% random sample unlearning, evaluating the results as summarized in Table 1 below. Due to time constraints, we include a subset of competitive baselines for comparison. As shown in the results, RURK consistently achieves the smallest average gap compared to GD and NegGrad+, confirming its effectiveness even at scale. Additionally, we provide the tabular version of the residual knowledge corresponding to Figure 2. The results reveal that GD still exhibits residual knowledge $> 1$, since it only fine-tunes on retain samples and lacks explicit control over forget samples or residual knowledge. Similarly, SSD, when applied to a large architecture like ResNet-50, struggles to identify and suppress all neurons associated with the forget information, leading to suboptimal mitigation. While NegGrad+ does not suffer from residual knowledge in the same way, its residual knowledge remains farther from 1 compared to RURK, indicating less alignment with the re-trained model. These findings support that the phenomenon of residual knowledge is not limited to small-scale settings, and that RURK remains effective in controlling it, even on larger and more complex datasets. We will incorporate and refine this new empirical evidence in the revised version. \n\n Metrics   | Retain Acc.      | Unlearn Acc.     | Test Acc.        | Avg. Gap        \n---------- | ---------------- | ---------------- | ---------------- | ----------------\nOriginal   | 80.75±3.06(5.69) | 3.59±0.19(9.03)  | 72.67±1.18(4.23) | 6.32 \nRe-train   | 75.06±1.54(0.00) | 12.62±1.40(0.00) | 68.44±1.51(0.00) | 0.00 \nGD         | 82.29±0.37(7.23) | 6.26±2.38(7.13)  | 74.44±1.39(6.01) | 6.79\nNegGrad+   | 81.86±1.86(6.80) | 40.67±1.85(27.28)| 73.53±1.06(5.10) | 13.06\nSSD        | 80.35±3.00(5.30) | 90.15±13.92(76.77)|  71.81±1.05(3.38)| 28.48 \nRURK       | 83.09±0.26(8.03) | 11.62±0.69(1.00) | 75.19±0.59(6.75) | 5.26 \n\n\n$\\tau$     | 0.0000 | 0.0031 | 0.0063 | 0.0094 | 0.0125 | 0.0157 | 0.0188 | 0.0220 | 0.0251 | 0.0282 | 0.0314\n-------- | ------ | ------ | ------ | ------ | ------ | ------ | ------ | ------ | ------ | ------ | ------\nOriginal | 1.1000 | 1.0996 | 1.1018 | 1.1011 | 1.1077 | 1.1107 | 1.1174 | 1.1288 | 1.1303 | 1.1421 | 1.1553\nGD       | 1.0656 | 1.0624 | 1.0647 | 1.0650 | 1.0606 | 1.0575 | 1.0595 | 1.0610 | 1.0642 | 1.0665 | 1.0654\nNegGrad+ | 0.6915 | 0.6861 | 0.6791 | 0.6723 | 0.6701 | 0.6624 | 0.6588 | 0.6556 | 0.6469 | 0.6351 | 0.6163\nSSD      | 1.1046 | 1.1032 | 1.1067 | 1.1095 | 1.1133 | 1.1161 | 1.1233 | 1.1291 | 1.1367 | 1.1446 | 1.1514\nRURK     | 1.0000 | 0.9977 | 0.9954 | 0.9898 | 0.9846 | 0.9763 | 0.9624 | 0.9339 | 0.9108 | 0.8858 | 0.8633\n\n\nFor Weakness 2, please refer to our response to Question 2 below.\n\nFor Weaknesses 3, 4 and Question 3, we appreciate the thoughtful comments regarding the residual knowledge $r_\\tau$, and will expand on this discussion in the revised manuscript. The reason we introduce residual knowledge $r_\\tau$ is to bring in the re-trained model $a$, as the ideal outcome of unlearning is that the unlearned model $m$ and $a$ behave consistently on both the training samples and their perturbed counterparts (see Lines 183–186). The residual knowledge captures the tendency of the unlearned model $m$ to re-recognize slightly perturbed forget samples compared to the re-trained model $a$, revealing a potential privacy risk (see Lines 33–39). Therefore, $r_\\tau$ cannot be defined using the unlearned model $m$ alone. As explained in Lines 263–266, RURK is designed to reduce residual knowledge by directly minimizing $\\Pr[m(x') = y]$, avoiding reliance on the re-trained model during the unlearning process. The example raised by the reviewer illustrates exactly the phenomenon that $r_\\tau$ is intended to capture: even if $\\Pr[m(x') = y]$ is small, when divided by a smaller $\\Pr[a(x') = y]$, the ratio $r_\\tau > 1$, indicating that the unlearned model is still more prone to re-identify the forget sample than the re-trained model. This is the precise privacy risk we aim to define and mitigate in this work.\n\n\n\nFor Question 1, Figure 1 provides a pictorial illustration of residual knowledge, which is later supported by the quantitative results in Table 1 and Figure 2. The figure is intended to convey that, after successful unlearning, the unlearned and re-trained models should make consistent decisions on all samples. However, due to slight deviations in the model parameters or decision boundaries (as captured by $(\\epsilon, \\delta)$-indistinguishability), the models may disagree on perturbed samples near the decision boundary. This discrepancy illustrates the presence of residual knowledge. We will incorporate this clarification into the revised caption of Figure 1.\n\n\nFor Question 2, we appreciate the reviewer’s careful attention to detail and will incorporate additional explanation in the revision. (1): The assumption underlying the isoperimetric inequality---that samples lie on the surface of a unit sphere---is indeed a strong one. However, the underlying phenomenon of the isoperimetric inequality (if a subset occupies a small volume, its boundary must be relatively large) can be extended beyond the unit sphere. In fact, this principle generalizes to the entire unit cube (not just its surface), as shown in Proposition 2.8 of [R1]. The unit cube is a more realistic assumption, as images are often scaled such that pixel values lie in $[0, 1]$. That said, Proposition 2 is still meaningful: it illustrates that disagreement on perturbed samples can arise even under the stricter assumption of a unit sphere (which is a smaller space of samples, compared with the unit cube), making the result even more compelling when considering looser assumptions like the unit cube. We will extend Proposition 2 with the unit cube assumption and include it in Line 195 and in the Appendix. (2): The reviewer is also correct that the expression in question is indeterminate unless a specific relationship between $\\epsilon$ and $\\delta$ is defined. To address this, we can make the relationship explicit by setting $\\delta = k\\epsilon$ for some constant $k$. This allows us to apply L’Hôpital’s Rule to evaluate the limit: $\\lim_{\\epsilon \\to 0} \\frac{\\frac{d}{d\\epsilon}(2k\\epsilon)}{\\frac{d}{d\\epsilon}(1-\\exp(-\\epsilon))} =  \\lim_{\\epsilon \\to 0} \\frac{2k}{\\exp(-\\epsilon)} = 0$. We will include this clarification in  Line 206.\n\nThank you again for your thoughtful feedback. We're happy to provide any additional information or address further questions. If you feel that we’ve sufficiently addressed your main concerns, we kindly ask you to consider updating your score accordingly.\n\n\n[R1] Ledoux, M., 2001. The concentration of measure phenomenon (No. 89). American Mathematical Soc."}]

Author rebuttal:

Meta-review: This paper points out an important gap in machine unlearning, even when models “forget” samples, small perturbations can still make them recognizable, which the authors call residual knowledge. They back this up with both theory and experiments, and propose RURK to reduce the issue. Reviewers liked the clarity, novelty, and the combination of proofs + experiments, and the rebuttal added useful larger-scale results (ImageNet-100, ResNet-50) and runtime clarifications. The main weaknesses are that experiments are still mostly small-scale classification tasks, and the method assumes access to forget samples, which may not always be realistic. Still, overall, the contribution is solid and raises awareness of a real privacy risk, so the consensus leans toward acceptance.
Decision: Accept (poster)