Title: The Unseen Threat: Residual Knowledge in Machine Unlearning under Perturbed Samples
Venue: NeurIPS 2025
Accepted: yes
Keywords: Machine Unlearning

robustness

adversarial

perturbation

disagreement

residual knowledge.
Abstract: Machine unlearning offers a practical alternative to avoid full model re-training by approximately removing the influence of specific user data. While existing methods certify unlearning via statistical indistinguishability from re-trained models, these guarantees do not naturally extend to model outputs when inputs are adversarially perturbed. In particular, slight perturbations of forget samples may still be correctly recognized by the unlearned model---even when a re-trained model fails to do so---revealing a novel privacy risk: information about the forget samples may persist in their local neighborhood. In this work, we formalize this vulnerability as residual knowledge and show that it is inevitable in high-dimensional settings. To mitigate this risk, we propose a fine-tuning strategy, named RURK, that penalizes the model’s ability to re-recognize perturbed forget samples. Experiments on vision benchmarks with deep neural networks demonstrate that residual knowledge is prevalent across existing unlearning methods and that our approach effectively prevents residual knowledge.

The Unseen Threat: Residual Knowledge in Machine Unlearning under Perturbed Samples

Abstract

Machine unlearning offers a practical alternative to avoid full model re-training by approximately removing the influence of specific user data. While existing methods certify unlearning via statistical indistinguishability from re-trained models, these guarantees do not naturally extend to model outputs when inputs are adversarially perturbed. In particular, slight perturbations of forget samples may still be correctly recognized by the unlearned model-even when a re-trained model fails to do so-revealing a novel privacy risk: information about the forget samples may persist in their local neighborhood. In this work, we formalize this vulnerability as residual knowledge and show that it is inevitable in high-dimensional settings. To mitigate this risk, we propose a fine-tuning strategy, named RURK, that penalizes the model's ability to re-recognize perturbed forget samples. Experiments on vision benchmarks with deep neural networks demonstrate that residual knowledge is prevalent across existing unlearning methods and that our approach effectively prevents residual knowledge.

1 Introduction

The widespread use of user data in training machine learning (ML) models has raised significant privacy concerns, particularly when user data is memorized by models such as deep neural networks, and can later be extracted or reconstructed (Carlini et al., 2023; Li et al., 2024). This memorization violates regulations such as the 'Right to be Forgotten' in EU's GDPR (Voigt and Von dem Bussche, 2017), which mandates that, upon a user's removal request, an ML service provider must not only delete the user data from databases but also ensure its removal from the ML models themselves (Shastri et al., 2019). As a result, simply deleting the data is often insufficient; instead, re-training the model from scratch without the specific user data is necessary. However, this re-training process is computationally expensive and impractical in real-time or large-scale settings (Ginart et al., 2019).

To address this challenge, machine unlearning has been proposed as a more scalable alternative that approximately removes the influence of the specific data (referred to as forget samples) from a pre-trained model, as a substitute for avoiding full re-training (Cao and Yang, 2015; Bourtoule et al., 2021). This approach, known as approximate unlearning , is often certified through statistical indistinguishability between the unlearned model and one re-trained without the forget samples (Guo et al., 2020; Chourasia and Shah, 2023).

Theoretical formulations of approximate unlearning (cf. § 2) suggest that an unlearned model should behave similarly to a re-trained model on forget samples, for example, by producing similar predictions or classification accuracy. However, these guarantees typically apply only to the original samples and do not extend to their proximities, especially under imperceptible adversarial perturbations. In complex hypothesis spaces, even minor input changes can cause the unlearned and re-trained models

Disagreement

Re-trained

Model

Other Samples (Agreed)

ForgetSample 2

Other Samples (Disagreed)

(Inconsistent)

Forget Samples (Agreed)

Forget Samples (Disagreed)

(Consistent)

Perturbed Region

DecisionBoundaries

Original

Unlearn

Model

HypothesisSpace

Unlearned

Model

to disagree in their predictions, creating an additional layer of privacy risk. A particularly concerning case is when a slightly perturbed forget sample is still correctly classified by the unlearned model but not by the re-trained one. This reveals the presence of residual knowledge -latent traces of the forget samples that still persist in the unlearned model (cf. Figure 1). Residual knowledge is a prevalent issue. For instance, on the CIFAR-10 dataset, when subjected to a small perturbation norm ( ≈ 0 . 03 ), over 7% of the forget samples still exhibit residual knowledge (cf. Appendix C.4).

In this paper, we study how adversarially perturbed inputs affect the indistinguishability between unlearned and re-trained models in the classification setting. In § 3, we show that adversarial examples can reliably distinguish between the two models, even under certified approximate unlearning. We formalize this observation by demonstrating that adversarial attacks can induce probabilistically distinguishable outputs. Further, using geometric probability (Talagrand, 1995), we prove that such disagreement is inevitable in high-dimensional input spaces. These findings underscore the need to explicitly incorporate robustness against such local disagreement into unlearning frameworks.

To capture this risk, § 4 introduces the notion of residual knowledge, which quantifies the likelihood that an unlearned model retains predictive traces of forget samples under perturbation. As a more tractable proxy for disagreement, residual knowledge exposes a vulnerability not addressed by current certification methods. To mitigate this, we propose RURK , a fine-tuning strategy for Robust Unlearning that suppresses Residual Knowledge while maintaining accuracy on the rest of the samples. Our approach identifies and penalizes perturbed inputs that the unlearned model still classifies correctly, thus enhancing robustness against residual knowledge. We empirically validate the existence of such local disagreement and residual knowledge across multiple unlearning algorithms in § 5. We further demonstrate that our fine-tuning strategy effectively reduces residual knowledge on standard vision datasets using deep neural networks. To the best of our knowledge, this is the first work to uncover this novel privacy risk and provide a scalable solution to address it. We conclude in § 6 with a discussion of limitations and future research directions.

Omitted proofs, additional explanations and discussions, details on experiment setups and training, and additional experiments are included in Appendices A, B and C, respectively.

2 Background and related work

Let S ≜ { s i = ( x i , y i ) } n i =1 be a training dataset with x i ∈ X ⊆ R d and y i ∈ Y . The hypothesis space H ≜ { h w : X → Y ; w ∈ W} consists of models parameterized by w . We use h and w interchangeably to refer to a model in H . Let ℓ : W × S → R + be the loss function, and define the empirical risk as L ( w , S ) = 1 n ∑ n i =1 ℓ ( w , s i ) . A (randomized) learning algorithm A : S → H , such as Stochastic Gradient Descent (SGD), returns a model minimizing L ( w , S ) and induces a distribution over H . We denote the ℓ p -norm by ∥ z ∥ p , and the ℓ p ball of radius τ centered at x by B p ( x , τ ) ≜ { z ∈ X : ∥ z -x ∥ p ≤ τ } . Also, let ✶ ( · ) be the indicator function, surf ( Z ) the surface area of a set Z in R d , and O ( · ) the big-O notation.

2.1 Machine unlearning

The forget set S f ⊆ S contains training samples to be removed. Machine unlearning is modeled as a randomized mechanism M ( A ( S ) , S , S f ) that removes the influence of S f from a pre-trained model A ( S ) (Xu et al., 2023). Like the learning algorithm itself, the output of the unlearning mechanism can be regarded as a random variable. A simple approach adds Gaussian noise to the model weights: M ( w , S , S f ) = w + σ n , with n ∼ N (0 , I | w | ) (Golatkar et al., 2020a). However, as σ increases, the model becomes increasingly independent of the training data, potentially degrading performance. To preserve utility, unlearning should retain accuracy on the retain set S r ≜ S \ S f . Anaïve yet effective strategy is to re-train the model from scratch on the retain set, i.e., M ( A ( S ) , S , S f ) = A ( S r ) . While this achieves exact unlearning , it is often impractical due to the high computational cost of re-training for each unlearning request. Therefore, the core objective of machine unlearning is to efficiently eliminate the influence of S f while preserving utility on S r , all without full re-training.

This objective has spurred a vast literature on machine unlearning. Here, we outline the broad trends and defer details of specific algorithms to § 5. For comprehensive surveys, see Nguyen et al. (2022); Xu et al. (2023); Wang et al. (2024). Initial efforts focused on image classifiers, aiming to remove the influence of specific training images from a pre-trained model (Ginart et al., 2019; Wu et al., 2020; Neel et al., 2021; Sekhari et al., 2021; Izzo et al., 2021; Fan et al., 2023; Kurmanji et al., 2024; Goel et al., 2022; Zhang et al., 2024; Kodge et al., 2024). This idea was later extended to erasing abstract concepts (Ravfogel et al., 2022; Belrose et al., 2023), as well as adapting unlearning to broader model classes, including image generators (Li et al., 2024; Gandikota et al., 2023) and large language models (Eldan and Russinovich, 2023; Yao et al., 2024; Liu et al., 2025; Jang et al., 2022; Wang et al., 2023). Another line of work develops unlearning methods tailored to specific model families-such as linear classifiers (Guo et al., 2020), kernel methods (Golatkar et al., 2020b; Zhang and Zhang, 2022), and tree-based models (Brophy and Lowd, 2021; Schelter et al., 2021).

2.2 Certification of machine unlearning

A valid unlearning mechanism ensures that the unlearned model M ( A ( S ) , S , S f ) is statistically similar to a re-trained model A ( S r ) . We denote the unlearned and re-trained models as random variables M and A , with corresponding distributions P M and P A , respectively. This requirement is formalized via ( ϵ, δ ) -indistinguishability:

Definition 1 ( ( ϵ, δ ) -indistinguishability) . Let X and Y be two random variables over a domain Ω .

X and Y are said to be ( ϵ, δ ) -indistinguishable, also denoted as X ϵ,δ ≈ Y , if for all T ⊆ Ω

This notion underlies differential privacy (DP) (Dwork and Roth, 2014) when X and Y are outputs on neighboring datasets. It also quantifies reproducibility of empirical findings when applied to models trained on independent samples from the same data distribution (Kalavasis et al., 2023; Impagliazzo et al., 2022; Bun et al., 2023).

Guo et al. (2020) were among the first to introduce certified machine unlearning using ( ϵ, δ ) -indistinguishability. An unlearning algorithm M satisfies ( ϵ, δ ) -unlearning if M ( A ( S ) , S , S f ) ϵ,δ ≈ A ( S r ) ; this reduces to exact unlearning when ϵ = δ = 0 . Indistinguishability can be measured through various probability divergences as well. For instance, Chourasia and Shah (2023) and Chien et al. (2024) proposed ( α, ϵ ) -Rényi unlearning, which holds when the α -Rényi divergence 2 between P M and P A is bounded by ϵ , and can be translated into ( ϵ, δ ) -unlearning. Indeed, ( α, ϵ ) -Rényi unlearning can be converted to ( ϵ +log(1 /δ ) / ( α -1) , δ ) -unlearning, for any 0 < δ < 1 (Mironov, 2017). Both frameworks assume uniqueness or a unique stationary distribution of the empirical minimizer, which may limit their applicability in complex model classes.

Instead of comparing full model distributions P M and P A , a more practical certification framework evaluates unlearned models with readout functions f : H×S → R that an adversary might use to distinguish unlearned from re-trained models. Indistinguishability is then assessed via the output distribution of f , using divergences such as the Kullback-Leibler (KL) divergence 3 . The certification

condition requires D KL (Pr[ f ( M, T )] ∥ Pr[ f ( A, T )] ≤ ϵ for any subset T ⊆ S , such as the forget, retain, or even hold-out test sets (Nguyen et al., 2020; Golatkar et al., 2020a). This formulation flexibly captures different behaviors: f could represent a binary classifier (e.g., for Membership Inference Attack (MIA)) (Fan et al., 2023), utility metrics like accuracy, or re-learning time-the training epochs needed to re-learn S f (Golatkar et al., 2020a). Focusing on readout functions offers a more tractable and empirically grounded certification approach, especially for complex models.

3 Model indistinguishability and disagreement over sample perturbation

Statistical indistinguishability between unlearned and re-trained models can be certified theoreticallyvia ( ϵ, δ ) - or ( α, ϵ ) -Rényi unlearning-or empirically using a readout function (cf. §2). This, indeed, ensures similarity in model weights or outputs on forget/retain samples. However, such guarantees do not readily extend to perturbed inputs, even with imperceptible adversarial perturbations.

Several studies have investigated the vulnerability of unlearning algorithms to adversarial manipulation. Marchant et al. (2022) show that adversarial inputs can significantly increase the computational cost of unlearning 4 , while Pawelczyk et al. (2025) find that poisoning attacks can obstruct complete forgetting (cf. Appendix B.1). Zhao et al. (2024) further demonstrate that even a small number of malicious unlearning requests can weaken the adversarial robustness of the resulting model. More recently, Xuan and Li (2025) propose an attack that manipulates an unlearned model such that its outputs on forget samples resemble those of the original model (see Proposition 3 and Definition 4 in their paper). However, these works primarily study how adversarial perturbations affect the unlearned model itself, whereas our goal is to evaluate the distinguishability between the unlearned and re-trained models.

This paper intends to explore a new dimension of how adversarial examples affect unlearningspecifically, how such examples may behave differently when fed to an unlearned model versus a re-trained one, even when the two satisfy statistical indistinguishability. Remarkably, despite this indistinguishability, it remains possible to craft adversarial inputs that distinguish between the two, revealing a novel privacy risk. This phenomenon, related to the transferability of adversarial examples (Tramèr et al., 2017), remains largely unexplored in the unlearning literature. The proofs of the propositions in this section are included in Appendix A.

3.1 Distinguishability of model output with adversarial examples

We begin by considering adversarial examples generated against either the unlearned or the re-trained model. The process of finding an adversarial example for a given input x ∈ X can be formalized as a read-out function g x : H → X , which may be deterministic or randomized. For instance, the minimum ℓ 2 -norm perturbation for a binary linear classifier, given by g x ( w ) = x -( x ⊤ x ) w / ∥ w ∥ 2 2 , and the Fast Gradient Sign Method (FGSM) (Goodfellow et al., 2016) are deterministic. In contrast, methods such as Projected Gradient Descent (PGD) (Madry et al., 2018) or random perturbations within B p ( x , τ ) are randomized. Since generating adversarial examples can be seen as a postprocessing of the model, the indistinguishability of adversarial examples can, in principle, be derived from the indistinguishability of the models themselves. Indeed, ( ϵ, δ ) -indistinguishability is preserved under arbitrary post-processing: if two random variables X and Y are ( ϵ, δ ) -indistinguishable, then so are f ( X ) and f ( Y ) for any (deterministic or randomized) function f . However, when considering adversarial examples of specific model realizations drawn from either the unlearning or re-training processes (cf. Lemma A.1), the level of indistinguishability can degrade, as formalized in the following proposition.

Proposition 1 (Adversarial example on a model is less indistinguishable) . Suppose the unlearned M ( A ( S ) , S , S f ) and re-trained A ( S r ) models are ( ϵ, δ ) -indistinguishable, and let x be a fixed sample. Then with probability 2 δ/ (1 -e -ϵ ) , the adversarial example g x ( · ) found against the models m ∼ M or a ∼ A satisfies, for all X ′ ⊆ X

Proposition 1 shows that even if the unlearned and re-trained models satisfy approximate unlearning certified by ( ϵ, δ ) -indistinguishability, the adversarial examples g x ( h ) can become less indistinguishable. Specifically, given a model h , with probability 2 δ/ (1 -e -ϵ ) , the distinguishability of the

adversarial examples increases by a factor of two. Moreover, the indistinguishability assumption in Proposition 1 can be readily generalized to ( α, ϵ ) -Rényi unlearning.

3.2 Disagreement on adversarial examples is inevitable

In the previous section, Proposition 1 showed that even when the unlearned and re-trained models satisfy ( ϵ, δ ) -indistinguishability, there remains a nonzero probability that their likelihood ratio can still be exploited to distinguish them via adversarial examples. However, evaluating the bound in Eq. (2) requires computing probabilities over the entire hypothesis space H , which is often computationally intractable when H is large or complex.

̸

To address this challenge, we instead focus on model outputs at individual samples x , which may belong to the forget or retain sets. We introduce a more tractable binary metric called disagreement , defined as k ( x ) = ✶ ( M ( x ) = A ( x )) , where k ( x ) = 1 indicates that the unlearned and re-trained models produce different predictions at x , and k ( x ) = 0 otherwise. Disagreement has been widely used in the machine learning literature to study model behavior (Krishna et al., 2025; Uma et al., 2021), as well as prediction stability and bias (Kulynych et al., 2023). Notably, the ( ϵ, δ ) -indistinguishability guarantee over distributions can be translated into an upper bound on empirical disagreement across samples. In particular, when ϵ = δ = 0 , perfect indistinguishability (i.e., exact unlearning) implies k ( x ) = 0 for all x ∈ S . Ideally, we seek agreement not only on the training set but across the entire sample space X , including unseen or perturbed data. Yet even small nonzero values of ϵ and δ can result in disagreement on certain inputs, particularly under adversarial perturbations, as we demonstrate in the following analysis.

To formalize this, we consider the sample space X = S d -1 = { x ∈ R d | ∥ x ∥ 2 = 1 } , corresponding to the unit sphere in R d , where all data points are normalized to unit norm. In this setting, the disagreement function k ( x ) can be viewed as a binary classifier over S d -1 . We aim to bound the probability that disagreement occurs not only at a sample x , but also within its local neighborhood under bounded perturbations, i.e., for x ′ ∈ B p ( x , τ ) such that ∥ x -x ′ ∥ p ≤ τ . This motivates the need to understand how disagreement can propagate beyond the observed dataset to its local neighborhood in the broader sample space X . A key mathematical tool for this purpose is the isoperimetric inequality (cf. Lemma A.2 and Talagrand (1995)). In probability theory and geometry, the isoperimetric inequality provides a lower bound on how the measure of a set expands when it is slightly 'extended.' In high-dimensional spaces, particularly under uniform or Gaussian distributions, it formalizes the intuition that if a subset occupies a small volume, its boundary must be relatively large. In our context, this implies that even if two models disagree on only a small region, small perturbations can still cause disagreement to spread over a much larger region in the sample space. Combining this geometric insight with the definition of ( ϵ, δ ) -unlearning (cf. Definition 1), we establish the following result:

Proposition 2 (Inevitable disagreement) . Consider a sample x ∈ S d -1 . Let M and A denote the unlearned and re-trained models, respectively, satisfying ( ϵ, δ ) -unlearning. Suppose 5 the agreement region satisfies surf ( { x ∈ S d -1 | k ( x ) = 0 } ) / surf ( S d -1 ) ≤ 1 / 2 . Then with probability at least

either k ( x ) = 1 , or there exist x ′ ∈ B p ( x , τ ) such that k ( x ′ ) = 1 , where p = 2 or p = ∞ .

When ϵ = δ = 0 , the probability in Eq. (3) is zero 6 , since the unlearned and re-trained models are identical and cannot disagree. Conversely, as ϵ →∞ for a fixed δ -reflecting maximal distinguishability between M and A -the probability of disagreement approaches its upper bound of 2 δ . More generally, for any fixed ( ϵ, δ ) , the probability in Eq. (3) depends solely on the perturbation norm τ : as τ increases, the probability of disagreement rises, as adversarial examples explore a broader neighborhood around the input. Moreover, Proposition 2 holds for both p = 2 and p = ∞ , aligning with the common practice of measuring perturbation size using either the ℓ 2 (Euclidean) norm or the ℓ ∞ (maximum) norm in prior works, e.g., Goodfellow et al. (2014); Madry et al. (2018).

While the assumption that samples lie on a unit sphere is admittedly strong, the core principle of the isoperimetric inequality-that small-volume sets necessarily have large boundaries-extends to

more realistic domains. In particular, Ledoux (2001, Proposition 2.8) generalizes this property to the unit cube, a more suitable setting for image data where pixel values are normalized. Nonetheless, Proposition 2 remains insightful, as it demonstrates that disagreement on perturbed samples can arise even under the stricter unit sphere assumption, thereby strengthening its relevance under looser conditions like the unit cube.

4 Removing residual knowledge in unlearned models

Proposition 2 demonstrates that disagreement between the unlearned and re-trained models can persist even when the unlearning algorithm satisfies the ( ϵ, δ ) -unlearning constraint. Although the two models may agree on the original input ( x , y ) , it is often possible to craft adversarial examples with imperceptible perturbations (small τ ) that cause them to diverge, revealing a potential privacy risk. This result arises from two sources of randomness: (1) that of the unlearning algorithm itself (e.g., model initialization) and (2) that of perturbations applied to forget samples. While the proposition establishes that such disagreement is inevitable, demonstrating existence alone is insufficient-quantifying its extent in practice is equally essential.

̸

To that end, we fix specific realizations of the unlearned and re-trained models and define as adversarial disagreement 7 as k τ ( x ′ ) = ✶ ( m ( x ′ ) = a ( x ′ )) for x ′ ∼ B p ( x , τ ) . The expected value of this indicator gives the probability of disagreement under perturbation: E [ k τ ( x ′ )] = Pr [ m ( x ′ ) = a ( x ′ )] . Adversarial disagreement is challenging to control, especially in multi-class settings, as it involves evaluating probabilities over all possible output combinations of m ( x ′ ) and a ( x ′ ) . To address this complexity, we next introduce a special case of adversarial disagreement-called residual knowledge -which offers a more tractable measure with operational meanings.

4.1 Connecting disagreement with residual knowledge

We focus on a particularly concerning form of adversarial disagreement, i.e., ✶ ( m ( x ′ ) = y, a ( x ′ ) = y ) for a forget sample 8 ( x , y ) ∈ S f . It implies that a forget sample, intended to be fully erased from the model, can be slightly perturbed such that the unlearned model still correctly classifies it, while the re-trained model does not. This discrepancy suggests that traces of the forget samples may remain embedded in the unlearned model's decision boundary, even when formal indistinguishability guarantees are satisfied. This scenario motivates our definition of residual knowledge -the persistence of information about the forget set in the predictive behavior of the unlearned model. The presence of residual knowledge reveals a subtle yet significant vulnerability: it demonstrates how adversarial examples can compromise the effectiveness of unlearning, and underscores the need for stronger robustness guarantees that go beyond conventional ( ϵ, δ ) -unlearning.

̸

Consider a forget sample ( x , y ) ∈ S f , and let m ∼ M ( A ( S ) , S , S f ) and a ∼ A ( S r ) be independently sampled unlearned and re-trained models, respectively. We formally define the residual knowledge around x as the following non-negative ratio:

This definition can be naturally extended to the entire forget set S f by averaging over all forget samples, r τ ( S f ) ≜ 1 |S f | ∑ ( x ,y ) ∈S f r τ (( x , y )) . Among the possible cases, r τ ( S f ) > 1 is especially concerning, as it indicates that the unlearned model m is more likely than the re-trained model a to correctly classify perturbed variants of forget samples. This suggests the presence of residual knowledge, which is the main privacy risk we aim to address in this paper. In this sense, the case where r τ ( S f ) < 1 is more tolerable, as it implies the unlearned model has lost the ability to recognize the forget samples. The case where r τ ( S f ) = 1 can only occur when m = a , meaning the unlearned model m achieves exact unlearning. In practice, r τ (( x , y )) can be estimated via Monte

̸

Carlo sampling. Specifically, by drawing c i.i.d. samples { x ′ i } c i =1 from B p ( x , τ ) , we approximate the ratio as ˆ r τ (( x , y )) = ∑ c i =1 ✶ ( m ( x ′ i )= y ) ∑ c i =1 ( a ( x ′ i )= y ) .

✶ The notion of residual knowledge is closely tied to adversarial disagreement. In particular, residual knowledge offers both upper and lower bounds on the expected adversarial disagreement between the unlearned and re-trained models (cf. Lemma A.4):

As r τ (( x , y )) → 1 , the expected adversarial disagreement E [ k τ ( x ′ )] approaches zero. In this sense, residual knowledge serves as a practical proxy for estimating the distinguishability between the unlearned and re-trained models, and provides a more tractable means to quantify and control adversarial disagreement.

4.2 RURK : Robust unlearning against residual knowledge

To address residual knowledge, we propose a fine-tuning objective that simultaneously enforces unlearning and regulates residual knowledge. Ideally, residual knowledge should be close to 1; however, accurately computing r τ ( S f ) requires access to a re-trained model a ∼ A ( S r ) , which is typically unavailable during unlearning. Thus, our objective is to attenuate residual knowledge and ensure that r τ ( S f ) ≤ 1 . Recall that the numerator of the residual knowledge in Eq. (4) is Pr[ m ( x ′ ) = y ] ; thus, directly minimizing this probability alone can effectively suppress residual knowledge, regardless of the denominator Pr[ a ( x ′ ) = y ] . Based on this insight, we define the set of vulnerable perturbations as V (( x , y ) , τ ) ≜ { x ′ ∈ B p ( x , τ ) | m ( x ′ ) = y } , which captures perturbed samples that continue to be associated with the true label by the unlearned model-indicating residual knowledge. In practice, we construct v such samples by adapting adversarial attack methods, such as FGSM or PGD, to solve the constrained minimization problem min z ∈B p ( x ,τ ) ℓ ( w , ( z , y )) .

Given the vulnerable set, we formulate the following fine-tuning objective for RURK as:

where κ ( w , ( x , y )) = 1 v ∑ { x ′ } v i =1 ∈V (( x ,y ) ,τ ) ℓ ( w , ( x ′ , y )) , and λ ≥ 0 is the regularization strength.

Term (i) preserves performance on the retain set, following prior work such as Neel et al. (2021); Kurmanji et al. (2024); Chien et al. (2024). Term (ii) serves as a regularization term that penalizes residual knowledge by discouraging the unlearned model from re-identifying vulnerable perturbations, thereby improving robustness with respect to residual knowledge. Explicitly searching for vulnerable perturbations in V (( x , y ) , τ ) during each gradient step can be computationally expensive. To mitigate this overhead, we may adopt a more efficient approximation by setting V (( x , y ) , τ ) = B p ( x , τ ) , i.e., removing all label information in the neighborhood of each forget sample. We outline the RURK algorithm in Appendix B.2.

Note that as τ → 0 , the objective in Eq. (6) reduces to that used in prior works such as Neel et al. (2021); Chien et al. (2024). When optimized via (Projected) Noisy Gradient Descent (NGD) (Chourasia et al., 2021), the resulting unlearned model satisfies ( α, ϵ ) -Rényi unlearning, where ϵ depends on the smoothness and Lipschitz continuity of the loss function L ( w , S ) . Specifically, less smooth or less Lipschitz losses lead to larger ϵ , making the unlearned model more distinguishable from the re-trained one (Chien et al., 2024, Theorem 3.2). This reveals a fundamental trade-off between model indistinguishability and robustness against residual knowledge: increasing τ in term (ii) of Eq. (6) enlarges the adversarial perturbation space, which makes the adversarial loss κ ( w , ( x , y )) less smooth and with a larger Lipschitz constant-ultimately increasing ϵ .

5 Empirical Studies

We now evaluate residual knowledge of state-of-the-art unlearning algorithms and its mitigation via RURK on three vision benchmarks. We denote Original as the model A ( S ) trained on the full dataset S , and Re-train as the ideal (but not viable in real world) model A ( S r ) re-trained from scratch without the forget set S f , used here as a reference. For experimental details, including dataset settings and hyper-parameter choices, refer to Appendix B.4; additional results are provided in Appendix C.

Data and unlearning settings. We evaluate unlearning methods on image classification under three random sample unlearning scenarios. The first scenario, small CIFAR-5, follows Golatkar et al. (2020a,b) and uses a reduced CIFAR-10 subset (Krizhevsky et al., 2009) containing 200 training and 200 test images per class from the first five classes, while the second scenario uses the full CIFAR-10 dataset; both employ ResNet-18 (He et al., 2016). The third scenario is based on a larger-scale ImageNet-100, a 100-class subset of ImageNet-1k (Deng et al., 2009) with 1,300 images per class, trained with ResNet-50. In all cases, only 50% of class 0 is unlearned, in contrast to the class unlearning setting (Kodge et al., 2024), which aims to remove all samples from a class. To avoid any external knowledge, both Original and Re-train models are trained from scratch without using any pre-trained weights. See Appendix C.2 for class-unlearning results and Appendix C.3 for ablations on RURK hyper-parameters and architectures (e.g., VGG (Simonyan and Zisserman, 2014)).

Baseline algorithms. We evaluate three machine unlearning algorithms suited for small CIFAR-5 settings { CR , Fisher , NTK }, and eight methods for the full CIFAR-10 { GD , NGD , GA , NegGrad+ , EU-k , CF-k , SCRUB , SSD }. Certified Removal ( CR ) uses influence functions and a one-step Newton update to estimate and remove the effect of forget samples (Guo et al., 2020). This approach was extended by Golatkar et al. (2020a), who incorporate the Fisher information matrix ( Fisher ) computed on the retain set, and by Golatkar et al. (2020b), who apply Neural Tangent Kernel ( NTK ) linearization of neural networks. These pioneering methods, while foundational, are not scalable to large models due to the computational cost of Hessian-based operations. Gradient Descent ( GD ) fine-tunes Original on the retain set S r using standard SGD (Neel et al., 2021). Noisy Gradient Descent ( NGD ) simply modifies GD by adding Gaussian noise in the gradient update steps for better privacy guarantees (Chourasia and Shah, 2023; Chien et al., 2024). On the other hand, Gradient Ascent ( GA ) removes the influence of S f by reversing gradient updates 9 on the forget set (Graves et al., 2021; Jang et al., 2022). NegGrad+ combines both strategies by applying GD to S r and GA to S f simultaneously (Kurmanji et al., 2024). To improve parameter efficiency, Goel et al. (2022) propose two layer-wise methods: Exact Unlearning the last k layers ( EU-k ), which re-trains them from scratch, and Catastrophically Forgetting the last k layers ( CF-k ), which only fine-tunes them on S r . SCRUB casts unlearning as a teacher-student distillation process, where the student selectively learns retain-set knowledge from the teacher (Kurmanji et al., 2024). The final method, SSD , selectively dampens model weights by uses Fisher information to estimate the influence of S f (Foster et al., 2024), a scalable version of Fisher . Further details on these baselines are in Appendix B.3.

Evaluation metrics. We adopt five evaluation metrics, as described in § 2.2, to comprehensively assess both the unlearning efficacy and the utility of the resulting models. To capture different dimensions of unlearning effectiveness, we first report Unlearning Accuracy 10 , following Fan et al. (2023). Second, we conduct a MIA using a support vector classifier (SVC) trained to distinguish training samples from test samples based on the model's output likelihoods; we report the SVC's attack failure rate (MIA Accuracy) as a measure of privacy protection. Third, Re-learn Time quantifies how easily a model can re-acquire the forget-set information: it is defined as the number of fine-tuning epochs needed for the model m to satisfy L ( m, S f ) ≤ (1 + η ) L ( Original , S f ) , with η = 0 . 05 . To assess model utility, we report classification accuracy on the retain set and the hold-out test set, referred to as Retain Accuracy and Test Accuracy, capturing both performance preservation and generalization. As discussed in § 2.1, a desirable unlearning method should minimize the deviation of performance from Re-train in both sample and class unlearning settings. To this end, we compute the Average Gap (Avg. Gap)-the mean absolute gap from Re-train across Retain, Unlearn, Test, and MIA accuracies-where a smaller Avg. Gap indicates better unlearning. Notably, methods such as CR , Fisher , NTK , and NGD already offer certified unlearning guarantees under frameworks like ( ϵ, δ ) -unlearning, Rényi unlearning, or bounded KL divergence; these guarantees do not necessarily imply a small performance gap on accuracy-based metrics.

The performance of all unlearning methods, including RURK , is best understood by jointly examining Table 1 and Figure 2. Table 1 presents standard unlearning metrics to ensure each method maintains reasonable test accuracy without excessive forgetting, while Figure 2 evaluates residual knowledge, measuring a model's ability to resist re-identifying perturbed forget samples. Ideally, a properly unlearned model should match a re-trained one-preserving performance on standard metrics while failing to recognize small perturbations of forgotten data.

Performance comparison. Table 1 compares the performance of Original , the ideal reference Re-train , our proposed RURK , and 11 baseline unlearning methods across two unlearning scenarios. RURK achieves the smallest average performance gap to Re-train in both settings. We apply SGD with τ = 0 . 03 and v = 1 in Term (ii) of Eq. (6), making RURK 's time complexity comparable to other gradient-based approaches such as GD , NGD , and NegGrad+ . In the small CIFAR-5 scenario, NTK shows a competitive Avg. Gap but suffers from the highest MIA accuracy, indicating vulnerability to privacy attacks. Fisher over-forgets the target samples, significantly degrading utility, as seen in its low retain and test accuracies. For CIFAR-10, NegGrad+ performs similarly to RURK in Avg. Gap but shows a much shorter re-learn time than Re-train , implying an implicit retention of forget-set knowledge. GA and SSD aggressively erase forget-set information, achieving high unlearn accuracy but at the cost of test accuracy dropping below 90%. Both EU-k and CF-k (with k = 5 ) re-train or fine-tune only the linear and last two residual blocks, but the remaining layers of ResNet-18 still retain substantial information about the forget set. SCRUB has the closest overall performance to RURK when including re-learn time, but unlike RURK , it requires extra memory to store a student model, whereas RURK supports in-place updates. We defer the performance of other unlearning baselines on small CIFAR-5, and an ablation study on the hyper-parameters (e.g., τ , v and λ ) in RURK to Appendix C.

Residual knowledge. Figure 2 presents 11 estimates of residual knowledge ˆ r τ (( x , y )) under varying perturbation radii τ , computed using Gaussian noise ( p = 2 ) with c = 100 samples from B p ( x , τ ) (cf. Eq. (4)). As expected, Original consistently exhibits residual knowledge greater than 1. In small CIFAR-5, although NTK performs comparably to RURK in Avg. Gap and re-learn time (Table 1), its residual knowledge remains above 1 due to linearization under the neural tangent kernel, which ignores higher-order terms. This highlights residual knowledge as a necessary complement to unlearn accuracy, MIA, and re-learn time. In contrast, RURK maintains values near 1 for τ < 0 . 01 and effectively suppresses them at larger τ . For CIFAR-10, GD and CF-k retain high residual knowledge similar to Original , underscoring the need for access to forget samples, as done by RURK . Conversely, GA and SSD yield residual knowledge below 1 even at τ = 0 , indicating over-unlearning. EU-k updates only the final layers, leaving residuals in earlier representations. Methods such as NGD , NegGrad+ , EU-k , and SCRUB follow trends similar to RURK , though RURK achieves more stable control near τ = 0 . 01 and stronger suppression beyond that. While NGD reduces residual knowledge more effectively than GD or CF-k through controlled weight noise, it remains less effective than RURK . NegGrad+ , with a similar objective, also retains more residual knowledge, validating the role of Term (ii) in Eq. (6). For residual knowledge under other attacks, see Appendix C.1; for adversarial disagreement and unlearn accuracy of the perturbed forget samples, see Appendix C.5.

SmallCIFAR-5

CIFAR-10

Original

Fisher

Original

GA

NGD

NegGrad+

1.3

S

1.175

0.4

1.0

1.15

Residual

K

60

now

1.2

1.2

1.1

0.00

0.00

NTK

0.02

0.02

0.8

0.9

0.8

0.6

0.00

0.00

RURK

0.02

0.02

K

R

d

1.125

1.150

1.0

0.8

0.00

0.00

EU-k

0.02

0.02

1.15

1.10

0.2

0.00

0.00

CF-k

0.02

0.02

1.0

0.5

0.5

0.00

0.00

SCRUB

0.02

0.02

1.10

1.05

0.3

0.2

0.1

0.00

0.00

SSD

0.02

0.02

0.50

0.75

1.0

0.5

0.00

0.00

RURK

0.02

0.02

Perturb Norm t

Perturb Norm t

EvaluationMetrics

NegGrad+

Datasets

Methods

Retain Acc.(%)

Unlearn Acc.(%)

Test Acc. (%)

Avg.Gap

L

e

1.065

1.060

0.675

0.650

0.625

g

Original

80.75±3.06(5.69)

3.59±0.19(9.03)

72.67±1.18(4.23)

6.32

Knowlec

0.00

0.02

0.00

0.02

ImageNet-100

Re-train

75.06±1.54(0.00)

12.62±1.40(0.00)

68.44±1.51(0.00)

0.00

SSD

RURK

1.150

1.0

NegGrad+

82.29±0.37(7.23)

81.86±1.86(6.80)

40.67±1.85(27.28)

6.26±2.38(7.13)

74.44±1.39 (6.01)

73.53±1.06(5.10)

13.06

6.79

Residual

1.125

0.9

SSD

80.35±3.00(5.30)

90.15±13.92(76.77)

71.81±1.05(3.38)

28.48

0.00

0.02

0.00

0.02

RURK

83.09±0.26(8.03)

11.62±0.69(1.00)

75.19±0.59(6.75)

5.26

Perturb Norm t

Large-Scale ImageNet-100. In Figure 3, RURK consistently achieves the smallest average performance gap compared to GD and NegGrad+ , demonstrating strong scalability. Residual knowledge analysis shows that GD still retains forgotten information due to its lack of explicit control over the forget set, while SSD fails to fully suppress forget-related neurons in large architectures like ResNet-50. Although NegGrad+ mitigates residual knowledge more effectively, its alignment with the re-trained model remains weaker than RURK . Overall, residual knowledge persists beyond small-scale settings, and RURK remains the most effective method for mitigating it in larger, more complex models.

6 Final remark

Our study reveals a key limitation of existing unlearning methods: although they erase direct memorization of forget samples, they often fail to remove implicit generalization around them, causing unlearned models to recognize perturbed variants more often than re-trained ones. RURK mitigates this by incorporating perturbed forget samples, disrupting such generalization and reducing residual knowledge while preserving retain-set accuracy.

Limitations. Ideally, unlearned and re-trained models should be statistically indistinguishable not only on original inputs but also on all perturbations within B p ( x , τ ) . However, controlling adversarial disagreement across such perturbations is computationally infeasible-it is as hard as achieving perfect adversarial robustness. Moreover, since the re-trained model is typically unavailable during unlearning, we can only bound one side of the residual knowledge (i.e., r τ ( S f ) ≤ 1 ; see § 4.2).

Future directions. First, our probabilistic analysis in § 3 could be extended to account for hypothesis class complexity and linked to the transferability of adversarial examples (Tramèr et al., 2017). Second, the indistinguishability-robustness trade-off introduced in § 4.2 opens up new directions in both unlearning and adversarial robustness, warranting deeper investigation. Third, our mitigation objective (cf. Eq. (6)) resembles a reverse of adversarial training, suggesting an open question of whether adversarial training or distributionally robust optimization could in fact impede unlearning or amplify residual knowledge. Moreover, in the context of MIA against unlearned models, adversaries could exploit side information-such as the minimal perturbation required for a model to re-recognize a forgotten sample. A smaller perturbation norm may indicate prior inclusion in training, increasing inference success. This perspective opens a promising research direction for future MIA studies in unlearning, as recent work has begun leveraging model variation near training points or perturbation dynamics to strengthen attacks (Jalalzai et al., 2022; Del Grosso et al., 2022; Xue et al., 2025). Finally, extending residual knowledge beyond classification to generative tasks-such as image or text generation-remains an open challenge. In these settings, unlearning often targets concepts rather than samples, yet tasks like image-to-image generation (Fan et al., 2023; Li et al., 2024) show that perturbing partial inputs can still regenerate forgotten content, suggesting analogous residual behaviors. In large language models, the compositional nature of text and differing notions of 'forget' and 'retain' complicate formalization, though phenomena like relearning and jailbreaking indicate conceptual parallels (Hu et al., 2024; Shumailov et al., 2024; Liu et al., 2025).

Broader impact. Residual knowledge introduces new privacy risks. For instance, if a user opts out of a biometric-based payment system (e.g., palm or facial recognition), residual traces may still allow adversaries to craft perturbed inputs that the system accepts-potentially enabling unauthorized access. Such vulnerabilities undermine trust in ML systems and challenge current interpretations of the 'Right to be Forgotten.'

Disclaimer. This paper was prepared for informational purposes by the Global Technology Applied Research center and Artificial Intelligence Research group of JPMorgan Chase & Co. This paper is not a product of the Research Department of JPMorgan Chase & Co. or its affiliates. Neither JPMorgan Chase & Co. nor any of its affiliates makes any explicit or implied representation or warranty and none of them accept any liability in connection with this paper, including, without limitation, with respect to the completeness, accuracy, or reliability of the information contained herein and the potential legal, compliance, tax, or accounting effects thereof. This document is not intended as investment research or investment advice, or as a recommendation, offer, or solicitation for the purchase or sale of any security, financial instrument, financial product or service, or to be used in any way for evaluating the merits of participating in any transaction.

References

Abadi, M., Chu, A., Goodfellow, I., McMahan, H. B., Mironov, I., Talwar, K., and Zhang, L. (2016). Deep learning with differential privacy. In Proceedings of the ACM Conference on Computer and Communications Security (CCS) .

Allouah, Y., Kazdan, J., Guerraoui, R., and Koyejo, S. (2025). The utility and complexity of inand out-of-distribution machine unlearning. In Proceedings of the International Conference on Learning Representations (ICLR) .

Belrose, N., Schneider-Joseph, D., Ravfogel, S., Cotterell, R., Raff, E., and Biderman, S. (2023). Leace: Perfect linear concept erasure in closed form. In Proceedings of Advances in Neural Information Processing Systems (NeurIPS) .

Bourtoule, L., Chandrasekaran, V., Choquette-Choo, C. A., Jia, H., Travers, A., Zhang, B., Lie, D., and Papernot, N. (2021). Machine unlearning. In Proceedings of the IEEE Symposium on Security and Privacy (SSP) .

Brophy, J. and Lowd, D. (2021). Machine unlearning for random forests. In Proceedings of the International Conference on Machine Learning (ICML) .

Bun, M., Gaboardi, M., Hopkins, M., Impagliazzo, R., Lei, R., Pitassi, T., Sivakumar, S., and Sorrell, J. (2023). Stability is stable: Connections between replicability, privacy, and adaptive generalization. In Proceedings of the 55th Annual ACM Symposium on Theory of Computing (STOC) .

Cao, Y. and Yang, J. (2015). Towards making systems forget with machine unlearning. In Proceedings of the IEEE Symposium on Security and Privacy (SSP) .

Carlini, N., Hayes, J., Nasr, M., Jagielski, M., Sehwag, V., Tramer, F., Balle, B., Ippolito, D., and Wallace, E. (2023). Extracting training data from diffusion models. In Proceedings of the 32nd USENIX Security Symposium .

Chien, E., Wang, H., Chen, Z., and Li, P. (2024). Langevin unlearning: A new perspective of noisy gradient descent for machine unlearning. In Proceedings of Advances in Neural Information Processing Systems (NeurIPS) .

Chourasia, R. and Shah, N. (2023). Forget unlearning: Towards true data-deletion in machine learning. In Proceedings of the International Conference on Machine Learning (ICML) .

Chourasia, R., Ye, J., and Shokri, R. (2021). Differential privacy dynamics of langevin diffusion and noisy gradient descent. In Proceedings of Advances in Neural Information Processing Systems (NeurIPS) .

Del Grosso, G., Jalalzai, H., Pichler, G., Palamidessi, C., and Piantanida, P. (2022). Leveraging adversarial examples to quantify membership information leakage. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) .

Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., and Fei-Fei, L. (2009). Imagenet: A large-scale hierarchical image database. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) .

Dwork, C. and Roth, A. (2014). The algorithmic foundations of differential privacy. Foundations and Trends® in Theoretical Computer Science , 9(3-4):211-407.

Eldan, R. and Russinovich, M. (2023). Who's harry potter? approximate unlearning in LLMs. arXiv preprint arXiv:2310.02238 .

Fan, C., Liu, J., Zhang, Y., Wong, E., Wei, D., and Liu, S. (2023). Salun: Empowering machine unlearning via gradient-based weight saliency in both image classification and generation. In Proceedings of the International Conference on Learning Representations (ICLR) .

Foster, J., Schoepf, S., and Brintrup, A. (2024). Fast machine unlearning without retraining through selective synaptic dampening. In Proceedings of the AAAI Conference on Artificial Intelligence .

French, R. M. (1999). Catastrophic forgetting in connectionist networks. Trends in cognitive sciences , 3(4):128-135.

Gandikota, R., Materzynska, J., Fiotto-Kaufman, J., and Bau, D. (2023). Erasing concepts from diffusion models. In Proceedings of the IEEE International Conference on Computer Vision (ICCV) , pages 2426-2436.

Ginart, A., Guan, M., Valiant, G., and Zou, J. Y. (2019). Making AI forget you: Data deletion in machine learning. In Proceedings of Advances in Neural Information Processing Systems (NeurIPS) .

Goel, S., Prabhu, A., Sanyal, A., Lim, S.-N., Torr, P., and Kumaraguru, P. (2022). Towards adversarial evaluations for inexact machine unlearning. arXiv preprint arXiv:2201.06640 .

Golatkar, A., Achille, A., and Soatto, S. (2020a). Eternal sunshine of the spotless net: Selective forgetting in deep networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) .

Golatkar, A., Achille, A., and Soatto, S. (2020b). Forgetting outside the box: Scrubbing deep networks of information accessible from input-output observations. In Proceedings of the European Conference on Computer Vision (ECCV) .

Goodfellow, I., Bengio, Y., Courville, A., and Bengio, Y. (2016). Deep learning . MIT press Cambridge.

Goodfellow, I. J., Shlens, J., and Szegedy, C. (2014). Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572 .

Graves, L., Nagisetty, V., and Ganesh, V. (2021). Amnesiac machine learning. In Proceedings of the AAAI Conference on Artificial Intelligence .

Gross, L. (1975). Logarithmic sobolev inequalities. American Journal of Mathematics , 97(4):10611083.

Guo, C., Goldstein, T., Hannun, A., and Van Der Maaten, L. (2020). Certified data removal from machine learning models. In Proceedings of the International Conference on Machine Learning (ICML) .

He, K., Zhang, X., Ren, S., and Sun, J. (2016). Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) .

Hinton, G., Vinyals, O., and Dean, J. (2014). Distilling the knowledge in a neural network. In Proceedings of Advances in Neural Information Processing Systems (NeurIPS), Deep Learning Workshop .

Hu, S., Fu, Y., Wu, S., and Smith, V. (2024). Jogging the memory of unlearned models through targeted relearning attacks. In Proceedings of the International Conference on Machine Learning (ICML), Workshop on Foundation Models in the Wild .

Impagliazzo, R., Lei, R., Pitassi, T., and Sorrell, J. (2022). Reproducibility in learning. In Proceedings of the 54th annual ACM symposium on theory of computing (STOC) .

Izzo, Z., Smart, M. A., Chaudhuri, K., and Zou, J. (2021). Approximate data deletion from machine learning models. In International Conference on Artificial Intelligence and Statistics (AISTATS) .

Jalalzai, H., Kadoche, E., Leluc, R., and Plassier, V. (2022). Membership inference attacks via adversarial examples. In Proceedings of Advances in Neural Information Processing Systems (NeurIPS), TSRML Workshop .

Jang, J., Yoon, D., Yang, S., Cha, S., Lee, M., Logeswaran, L., and Seo, M. (2022). Knowledge unlearning for mitigating privacy risks in language models. arXiv preprint arXiv:2210.01504 .

Kalavasis, A., Karbasi, A., Moran, S., and Velegkas, G. (2023). Statistical indistinguishability of learning algorithms. In Proceedings of the International Conference on Machine Learning (ICML) .

Kim, H. (2020). Torchattacks: A pytorch repository for adversarial attacks. arXiv preprint arXiv:2010.01950 .

Kodge, S., Saha, G., and Roy, K. (2024). Deep unlearning: Fast and efficient gradient-free class forgetting. Transactions on Machine Learning Research (TMLR) .

Krishna, S., Han, T., Gu, A., Wu, S., Jabbari, S., and Lakkaraju, H. (2025). The disagreement problem in explainable machine learning: A practitioner's perspective. Transactions on Machine Learning Research (TMLR) .

Krizhevsky, A., Hinton, G., et al. (2009). Learning multiple layers of features from tiny images. Technical Report .

Kullback, S. and Leibler, R. A. (1951). On information and sufficiency. The annals of mathematical statistics , 22(1):79-86.

Kulynych, B., Hsu, H., Troncoso, C., and Calmon, F. P. (2023). Arbitrary decisions are a hidden cost of differentially private training. In Proceedings of the ACM Conference on Fairness, Accountability, and Transparency (FAccT) .

Kurmanji, M., Triantafillou, P., Hayes, J., and Triantafillou, E. (2024). Towards unbounded machine unlearning. In Proceedings of Advances in Neural Information Processing Systems (NeurIPS) .

Ledoux, M. (2001). The concentration of measure phenomenon . American Mathematical Society.

Li, G., Hsu, H., Marculescu, R., et al. (2024). Machine unlearning for image-to-image generative models. In Proceedings of the International Conference on Learning Representations (ICLR) .

Liu, S., Yao, Y., Jia, J., Casper, S., Baracaldo, N., Hase, P., Yao, Y ., Liu, C. Y ., Xu, X., Li, H., et al. (2025). Rethinking machine unlearning for large language models. Nature Machine Intelligence , pages 1-14.

Madry, A., Makelov, A., Schmidt, L., Tsipras, D., and Vladu, A. (2018). Towards deep learning models resistant to adversarial attacks. In Proceedings of the International Conference on Learning Representations (ICLR) .

Marcel, S. and Rodriguez, Y. (2010). Torchvision the machine-vision package of torch. In Proceedings of the 18th ACM International Conference on Multimedia (MM) .

Marchant, N. G., Rubinstein, B. I., and Alfeld, S. (2022). Hard to forget: Poisoning attacks on certified machine unlearning. In Proceedings of the AAAI Conference on Artificial Intelligence .

Martens, J. (2020). New insights and perspectives on the natural gradient method. Journal of Machine Learning Research (JMLR) , 21(146):1-76.

Milman, V. D. and Schechtman, G. (1986). Asymptotic theory of finite dimensional normed spaces: Isoperimetric inequalities in riemannian manifolds , volume 1200. Springer Science & Business Media.

Mironov, I. (2017). Rényi differential privacy. In Proceedings of the IEEE Computer Security Foundations Symposium (CSF) .

Neel, S., Roth, A., and Sharifi-Malvajerdi, S. (2021). Descent-to-delete: Gradient-based methods for machine unlearning. In Proceedings of Algorithmic Learning Theory (ALT) .

Nguyen, Q. P., Low, B. K. H., and Jaillet, P. (2020). Variational bayesian unlearning. In Proceedings of Advances in Neural Information Processing Systems (NeurIPS) .

Nguyen, T. T., Huynh, T. T., Ren, Z., Nguyen, P. L., Liew, A. W.-C., Yin, H., and Nguyen, Q. V. H. (2022). A survey of machine unlearning. arXiv preprint arXiv:2209.02299 .

Paszke, A. (2019). Pytorch: An imperative style, high-performance deep learning library. arXiv preprint arXiv:1912.01703 .

Pawelczyk, M., Di, J. Z., Lu, Y., Kamath, G., Sekhari, A., and Neel, S. (2025). Machine unlearning fails to remove data poisoning attacks. In Proceedings of the International Conference on Learning Representations (ICLR) .

Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., et al. (2011). Scikit-learn: Machine learning in python. the Journal of machine Learning research (JMLR) , 12:2825-2830.

Ravfogel, S., Twiton, M., Goldberg, Y., and Cotterell, R. D. (2022). Linear adversarial concept erasure. In Proceedings of the International Conference on Machine Learning (ICML) .

Rényi, A. (1961). On measures of entropy and information. In Proceedings of the 4th Berkeley Symposium on Mathematical Statistics and Probability, Volume 1: Contributions to the Theory of Statistics .

Schelter, S., Grafberger, S., and Dunning, T. (2021). Hedgecut: Maintaining randomised trees for low-latency machine unlearning. In Proceedings of the International Conference on Management of Data .

Sekhari, A., Acharya, J., Kamath, G., and Suresh, A. T. (2021). Remember what you want to forget: Algorithms for machine unlearning. In Proceedings of Advances in Neural Information Processing Systems (NeurIPS) .

Shastri, S., Wasserman, M., and Chidambaram, V. (2019). The seven sins of personal-data processing systems under GDPR. In Proceedings of the 11th USENIX Workshop on Hot Topics in Cloud Computing .

Shumailov, I., Hayes, J., Triantafillou, E., Ortiz-Jimenez, G., Papernot, N., Jagielski, M., Yona, I., Howard, H., and Bagdasaryan, E. (2024). Ununlearning: Unlearning is not sufficient for content regulation in advanced generative AI. arXiv preprint arXiv:2407.00106 .

Simonyan, K. and Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 .

Talagrand, M. (1995). Concentration of measure and isoperimetric inequalities in product spaces. Publications Mathématiques de l'Institut des Hautes Etudes Scientifiques , 81:73-205.

Tramèr, F., Papernot, N., Goodfellow, I., Boneh, D., and McDaniel, P. (2017). The space of transferable adversarial examples. arXiv preprint arXiv:1704.03453 .

Uma, A. N., Fornaciari, T., Hovy, D., Paun, S., Plank, B., and Poesio, M. (2021). Learning from disagreement: A survey. Journal of Artificial Intelligence Research (JAIR) , 72:1385-1470.

Voigt, P. and Von dem Bussche, A. (2017). The EU general data protection regulation (GDPR). A Practical Guide, 1st Ed., Cham: Springer International Publishing , 10(3152676):10-5555.

Wang, L., Chen, T., Yuan, W., Zeng, X., Wong, K.-F., and Yin, H. (2023). KGA: A general machine unlearning framework based on knowledge gap alignment. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (ACL) .

Wang, W., Tian, Z., Zhang, C., and Yu, S. (2024). Machine unlearning: A comprehensive survey. arXiv preprint arXiv:2405.07406 .

Wu, Y., Dobriban, E., and Davidson, S. (2020). Deltagrad: Rapid retraining of machine learning models. In Proceedings of the International Conference on Machine Learning (ICML) .

Xu, H., Zhu, T., Zhang, L., Zhou, W., and Yu, P. S. (2023). Machine unlearning: A survey. ACM Computing Surveys , 56(1).

Xuan, H. and Li, X. (2025). Verifying robust unlearning: Probing residual knowledge in unlearned models. arXiv preprint arXiv:2504.14798 .

Xue, J., Sun, Z., Ye, H., Luo, L., Chang, X., Tsang, I., and Dai, G. (2025). Privacy leaks by adversaries: Adversarial iterations for membership inference attack. arXiv preprint arXiv:2506.02711 .

Yao, J., Chien, E., Du, M., Niu, X., Wang, T., Cheng, Z., and Yue, X. (2024). Machine unlearning of pre-trained large language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (ACL) .

Zhang, B., Dong, Y., Wang, T., and Li, J. (2024). Towards certified unlearning for deep neural networks. In Proceedings of the International Conference on Machine Learning (ICML) .

Zhang, R. and Zhang, S. (2022). Rethinking influence functions of neural networks in the overparameterized regime. In Proceedings of the AAAI Conference on Artificial Intelligence .

Zhao, C., Qian, W., Li, Y., Li, W., and Huai, M. (2024). Rethinking adversarial robustness in the context of the right to be forgotten. In Proceedings of the International Conference on Machine Learning (ICML) .

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: We include a summary of contributions at the end of the introduction, where each contribution and the claims made therein are specifically referred to a section in this paper.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors? Answer: [Yes]

Justification: We include a discussion of limitations regarding theoretical extensions and computational overhead in the last section.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: We clearly state all the assumptions and lemmas that we have used with citations in our theoretical results. We also provide the sketch of proof/intuition in the main text.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: We provide details descriptions of the proposed methodology in the main text and in the appendix. Our methodology is tested on over 2 different vision benchmarks, shows consistency and explainable results over independent trials, and outperforms 11 existing unlearning baseline algorithms.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in

some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [No]

Justification: Due to intellectual property protection and anonymity requirements, we choose to release our codes upon decision. We provide details scripts on how to access the datasets, implement our methodology, and reproduce the empirical results in the main text and appendix.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: We provide all the training and setting details in the main text and appendix, including hyper-parameter settings, optimizer/learning rate scheduler, the GitHub links to all unlearning baselines we have evaluated, the reference to the evaluation metrics, etc.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: All the results reported in this paper include error bars that is computed over 3 independent trials. Moreover, we implement 5 different evaluation metrics (operational meaning and proper reference included) to compare our method against existing baselines.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: We provide the specific type of machine we used for the experiments in the appendix. We also include a discussion of the computational complexity of our method in § 5 and in the appendix.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: We have reviewed and complied the NeurIPS Code of Ethics.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [Yes]

Justification: We cover the societal impacts of the residual knowledge and machine unlearning in the introduction, related work and final remark.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: [NA]

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: We provide proper citations for all datasets including licenses and how to access, and programming packages (Python, Pytorch, Scikit Learn, torchattack) used in this paper.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [NA]

Justification: [NA]

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: [NA]

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: [NA]

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

The appendix is divided into the following parts. Appendix A: Omitted proofs and theoretical results; Appendix B: Details on the experimental setup; and Appendix C: Additional empirical results and ablation studies.

A Omitted proofs and theoretical results

We first introduce (and prove) the following useful lemmas to facilitate the proofs of the propositions. The first lemma is a variant of the ( ϵ, δ ) -indistinguishability in Definition 1.

Lemma A.1 (Probabilistic indistinguishability) . If X and Y are ( ϵ, δ ) -indistinguishable, then with probability at least 1 -2 δ/ (1 -e -ϵ ) over z drawn from support ( X ) ∪ support ( Y ) , we have

Proof. We first consider the complement event of the right inequality in Eq. (A.1). Define Z = { z | Pr[ X = z ] > e 2 ϵ Pr[ Y = z ] } , we directly have Pr[ X ∈ Z ] > e 2 ϵ Pr[ Y ∈ Z ] . By ( ϵ, δ ) -indistinguishable, we have

Now consider the complement event of the left inequality in Eq. (A.1) and define Z ′ = { z | Pr[ X = z ] < e -2 ϵ Pr[ Y = z ] } , by similar algebra, we have

Since δ 1 -e -ϵ is always larger than δ e 2 ϵ (1 -e -ϵ ) (by a factor of e 2 ϵ ), by combining Eq. (A.2) and Eq. (A.3), we have

With the same analysis on Pr[ X ∈ Z ] and Pr[ X ∈ Z ′ ] , we have Pr[ X ∈ Z ∪ Z ′ ] ≤ δ 1 -e -ϵ . Finally, putting together the the probability bounds on Pr[ Y ∈ Z ∪ Z ′ ] and Pr[ X ∈ Z ∪ Z ′ ] , we have

as desired.

The second lemma relates to the isoperimetric inequality (Talagrand, 1995). Before stating it, we define the notion of a τ -expansion of a set. Given a set C ⊂ R d , its τ -expansion with respect to the ℓ p -norm is defined as

The isoperimetric inequality is used to characterize the normalized surface area of the τ -expansion of a half unit sphere, showing how it grows with the expansion radius τ and the dimension d . This property is leveraged in the condition of Proposition 2, and is formally stated below.

Lemma A.2 (Milman and Schechtman (1986)) . Let C be the half unit sphere in R d , i.e., C ∈ S d -1 and surf ( C ) / surf ( S d -1 ) ≥ 1 / 2 , then with p = 2 or p = ∞ , the τ -expansion C has the surface area that satisfies

The third lemma utilizes Lemma A.1 to lower bound the probability of disagreement, i.e., the expected value of the disagreement indicator function k ( x ) .

Lemma A.3 (Lower bound on disagreement probability) . If the unlearned M ( A ( S ) , S , S f ) and retrained A ( S r ) models satisfies ( ϵ, δ ) -unlearning, then with probability 2 δ/ (1 -e -ϵ ) , the probability of disagreement among the two models on a sample x ∈ S is lower bounded by

̸

̸

E [ k ( x )] = E [ ✶ ( M ( x ) = A ( x ))] = Pr[ M ( x ) = A ( x )] > 1 -O ( e -2 ϵ ) . (A.8) Proof. We prove the lower bound by directly decompose Pr[ M ( x ) = A ( x )] , i.e.,

̸

̸

̸

̸

̸

̸

̸

The second integral in Eq. (A.9) is zero since Pr[ m ( x ) = a ( x ) | M = m = h, A = a = h ] = 0 . Moreover, by Lemma A.1, with probability 1 -2 δ/ (1 -e -ϵ ) , we have

̸

̸

In other words, with 2 δ/ (1 -e -ϵ ) , we have

̸

̸

The combination of Eq. (A.9) and Eq. (A.11) yield

̸

̸

̸

̸

the desired result, as ∫ h ∈H Pr[ A = h ] dh = 1 and ∫ h ∈H Pr[ A = h ] 2 dh is a constant.

̸

Lemma A.4 (Bounding adversarial disagreement with the residual knowledge.) . The expected value of k τ ( x ′ ) over the perturbation distribution is upper and lower bounded by

The fourth lemma provides the relation between the adversarial disagreement k τ ( x ′ ) = ✶ ( M ( x ) = A ( x )) and the residual knowledge r τ (( x , y )) .

Proof. We prove the lemma by directly following the definition of adversarial disagreement. First, we have

̸

̸

where the final inequality follows by isolating the contribution of the true label y . By substituting the definition of residual knowledge from Eq. (4), we obtain

′ ′ ′

A.1 Proof of Proposition 1

Suppose the unlearned M ( A ( S ) , S , S f ) and re-trained A ( S r ) models are ( ϵ, δ ) -indistinguishable, and let x be a fixed sample. From Lemma A.1, we have that with probability 2 δ/ (1 -e ϵ ) , h drawn from support ( M ) ∪ support ( A ) , we have

Therefore, for all X ′ ⊆ X , by using the right inequality in Eq. (A.17),

Similarly, for the other inequality, we have

Eq. (A.18) and Eq. (A.19) together give the desired result.

A.2 Proof of Proposition 2

Let R = { x ∈ S d -1 | k ( x ) = 0 } to be the region of the sphere that has agreement between the unlearned and re-trained models. By assumption, we have surf ( R ) / surf ( S d -1 ) ≤ 1 / 2 . Accordingly, we define ¯ R = { x ∈ S d -1 | k ( x ) = 1 } to be the complement of R , denoting the region of the sphere that has disagreement between the unlearned and re-trained models. Since surf ( ¯ R ) / surf ( S d -1 ) ≥ 1 / 2 , its τ -expansion E ( ¯ R , p, τ ) , by Lemma A.2, is at least as large as the epsilon expansion of a half sphere; that is

Note that the τ -expansion E ( ¯ R , p, τ ) represents the set of samples that either has k ( x ) = 1 or admits a x ′ ∈ B p ( x , τ ) such that k ( x ′ ) = 1 . We can therefore define a set E c that is the complement of E ( ¯ R , p, τ ) with surface area satisfies

The probability to draw a sample in E c is then upper bounded by

Using Lemma A.3, we know that with probability 2 δ/ (1 -e -ϵ ) ,

̸

Putting Eq. (A.22) and Eq. (A.23) together, we know that the probability to draw a sample in E ( ¯ R , p, τ ) is at least

B Details on the experimental setup

We provide implementation details of RURK in § 4.2, and introduce the unlearning baselines in § 5, including their mathematical formulations, operational interpretations, implementation specifics, and GitHub links. Finally, we summarize the dataset descriptions, training setups, and evaluation metrics.

B.1 Comparison with Pawelczyk et al. (2025)

Pawelczyk et al. (2025) investigate the failure of machine unlearning under data poisoning attacks. In their setup, the training set S train consists of two disjoint subsets: S clean and S poison, where S poison contains maliciously corrupted samples generated via targeted or backdoor poisoning. The original model is trained on both subsets, and the goal of unlearning is to remove the influence of S poison so that the resulting model resembles one retrained solely on S clean. Ideally, the unlearned model should be 'clean' and exhibit no poisoning effects. However, Pawelczyk et al. (2025) find that even after applying state-of-the-art unlearning algorithms, the unlearned model continues to display backdoor behavior-revealed through persistent correlations with the poison pattern-unlike the clean retrained model. They conclude that unlearning cannot fully sanitize a poisoned model, as residual effects of the attack remain embedded in its representations and decision boundary.

Our work, in contrast, examines clean unlearning under adversarial perturbations at test time rather than under poisoned data at training time. Both the retain and forget sets in our framework are clean; the original model M o is trained on S retain ∪ S forget, and the unlearned model M u aims to forget S forget so as to be indistinguishable from a retrained model M r trained only on S retain. We show that even when M u and M r are distributionally indistinguishable, they can still diverge in behavior on perturbed forget samples S perturbed-samples not seen during training but lying in the local neighborhood of S forget. Specifically, M u tends to re-recognize these perturbed samples more often than M r , revealing a form of residual knowledge that persists despite formal unlearning guarantees.

The key distinction between the two settings lies in the source and nature of the residual information. In Pawelczyk et al. (2025), the unlearned model retains explicit knowledge of malicious patterns learned from S poison (e.g., a backdoor trigger). In our work, the residual information is implicit , arising from the generalization structure around the clean forget samples rather than from any poisoned signal. Thus, while Pawelczyk et al. (2025) highlight the limits of unlearning under training-time data corruption, our work reveals that even with entirely clean data, residual generalization can cause unlearned and retrained models to remain distinguishable-posing a distinct privacy risk not attributable to poisoning.

B.2 Details of RURK in § 4.2

We first provide the pseudo-code of RURK in the following algorithm box Algorithm 1.

We implement RURK using PyTorch ( torch ) (Paszke, 2019), and ensure reproducibility by fixing three random seeds: [131 , 42 , 7] . During optimization, we use a batch size of 128 , the standard cross-entropy loss ( torch.nn.CrossEntropyLoss() ), and the SGD optimizer ( torch.optim.SGD ) with a learning rate of 0 . 01 , momentum of 0 . 90 , and weight decay of 5 × 10 -4 . To stabilize training, we apply a cosine annealing learning rate scheduler ( torch.optim.lr_scheduler.CosineAnnealingLR ) and cap the total number of iterations at 200 . Additionally, we clip the gradient norm to 1 . 0 using torch.nn.utils.clip_grad_norm_ .

For both unlearning scenarios (small CIFAR-5 and CIFAR-10), we set the perturbation budget τ = 0 . 03 and use the TorchAttacks library (Kim, 2020) to identify the vulnerable set V (( x , y ) , τ ) . To improve efficiency, we define V (( x , y ) , τ ) as the entire perturbation ball B p ( x , τ ) and set v = 1 , meaning that only a single perturbed sample is drawn from a multivariate Gaussian distribution centered at x with standard deviation τ .

We configure the hyper-parameters as follows: for small CIFAR-5, we use N = 2 and λ f = λ a = 0 . 03 ; for CIFAR-10, we set N = 2 , λ f = 0 . 03 , and λ a = 0 . 00045 . Since v = 1 and the perturbation is generated via Gaussian noise, the inner loop in line 12 of Algorithm 1 executes only once, making the operations in lines 11-15 constant-time. As a result, the computational complexity of RURK is comparable to fine-tuning-based methods such as GA , GD , NegGrad+ , and NGD .

Algorithm 1: RURK Implementation

However, if stronger adversaries like FGSM or PGD are used to identify the vulnerable set, the computational complexity increases from O ( N ) to O ( N × v ) . Note that unlike NGD , we do not inject additional noise into the gradient update steps.

More discussion on RURK . The regularization over the forget set in the loss L ( w , A ( S )) highlights two key features of our proposed unlearning method, RURK . First, under mild conditions, the unlearning procedure satisfies certified unlearning via Rényi Differential Privacy (RDP), providing formal guarantees-a certificate of unlearning. Second, enhancing robustness against vulnerable perturbations V (( x , y ) , τ ) may increase the distinguishability of the unlearned model from the ideal re-trained model.

To support the first point, consider the case where the forget set contains a single sample (i.e., |S f | = 1 ), so that the original dataset S and the retain set S r are adjacent. Assume the original model m was trained to satisfy ( α, ϵ 0 ) -RDP. Further, suppose the unlearning procedure minimizes the loss in Eq. (6) using Projected Noisy Gradient Descent (PNGD). As shown by Chien et al. (2024), the Markov chain defined by PNGD updates of the form

where W t ∼ N (0 , I d ) and Π R denotes projection onto the ball of radius R , converges to a stationary distribution if the loss L ( w , A ( S )) from Eq. (6) is continuous. The first term of Eq. (6) is continuous by standard assumptions, as the loss ℓ ( · , ( x , y )) is typically continuous in the weights. To show continuity of the second term, consider g ( w ) = min z ∈B p ( x ,τ ) ℓ ( w , ( z , y )) . Let z ∗ be a minimizer; then g ( w ) = ℓ ( w , ( z ∗ , y )) , which is continuous in w since ℓ is. Therefore, the overall loss is continuous, and the PNGD process converges.

Furthermore, when the loss is L -smooth and M -Lipschitz, the stationary distribution obtained via PNGD satisfies ( α, ϵ ) -RDP for some ϵ depending on L , M , and convexity properties of the loss (Chien et al., 2024). For forget sets with multiple elements, one can sequentially apply this procedure, preserving Rényi unlearning at each step.

To justify the second point, observe that Term (ii) in Eq. (6) impacts the loss landscape's smoothness. Specifically, the term κ ( w , ( x , y )) tends to increase with the perturbation radius τ , as the adversarial loss over a larger ball is generally higher. This leads to a gradient norm that increases with τ ,

implying that the smoothness and Lipschitz constants of the overall loss also grow with τ . According to Theorem 3.2 in Chien et al. (2024), the Rényi distinguishability parameter ϵ is inversely related to these constants, implying a trade-off: achieving higher robustness (larger τ ) increases model distinguishability (larger ϵ ).

The assumptions underlying these results are mild and standard in the literature. We assume the loss is L -smooth and M -Lipschitz, and that the learning dynamics for the original dataset satisfy the Log-Sobolev Inequality (LSI) with constant C LSI (Gross, 1975). Moreover, the Original A ( S ) can be trained to satisfy ( α, ϵ 0 ) -RDP under LSI using the same PNGD framework, as discussed by Chien et al. (2024).

B.3 Existing unlearning algorithms

Here, we provide detailed descriptions of the 11 unlearning baseline methods used in § 5, along with links to their corresponding GitHub repositories.

Certified Removal ( CR ). Guo et al. (2020) propose a single-step Newton-Raphson update to remove the influence of forget samples. Assuming that the empirical risk L ( w , S ) is twice differentiable with continuous second derivatives, and letting w ∗ = arg min w ∈W L ( w , S r ) denote the empirical risk minimizer over the retain set S r , the Taylor expansion of the gradient ∇ L around w ∗ yields:

where w o = A ( S ) denotes the Original model parameters, and H w o = ∇ 2 L ( w o , S r ) is the Hessian of the empirical risk over S r evaluated at w o . Re-arranging the terms gives:

since ∇ L ( w ∗ , S r ) = 0 by optimality. The certified removal method ( CR ) then defines the unlearned model as

where λ is a step-size parameter. Under the assumption of convexity (ensuring the uniqueness of minimizers) and bounded approximation error, Guo et al. (2020, Theorem 1) show that this procedure guarantees ( ϵ, δ ) -certified removal, i.e., M ( w o , S , S f ) ϵ,δ ≈ w ∗ .

In our implementation, we follow the official codebase at https://github.com/ facebookresearch/certified-removal . Since CR is designed for binary linear models, we use a pre-trained ResNet-18 as a non-private feature extractor (excluding its final linear layer), and apply the Newton update only to the final layer. Specifically, we train |Y| one-vs-rest logistic regression classifiers on the extracted features and set λ = 0 . 1 .

Fisher Unlearning ( Fisher ). Fisher (Golatkar et al., 2020a) is one of the simplest unlearning mechanisms, which removes the influence of forget samples by directly perturbing the model weights with Gaussian noise (cf.§2.1). Unlike standard Gaussian perturbation, the variance of the noise is scaled according to the inverse of the Fisher information matrix. Specifically, the unlearned model is defined as:

where B denotes the Hessian matrix ∇ 2 L ( w , S r ) . However, computing the exact Hessian is computationally intractable even for moderately sized neural networks, and the matrix may not be positive definite. To address this, Golatkar et al. (2020a) approximate the Hessian using the Levenberg-Marquardt algorithm, yielding a semi-positive-definite matrix closely related to the Fisher information matrix (Martens, 2020), which motivates the name of the method. In our implementation, we follow the official codebase and adopt the same hyper-parameters as in https://github.com/AdityaGolatkar/SelectiveForgetting .

Neural Tangent Kernel Unlearning ( NTK ). NTK (Golatkar et al., 2020b) extends the Newton update idea from CR by applying the Neural Tangent Kernel (NTK) linearization of neural networks. The NTK matrix between two datasets S 1 and S 2 is defined as:

where h w denotes the network output and h w may optionally denote a scalar output function (e.g., pre-activation logits). Let w ∗ = arg min w ∈W L ( w , S ) and w r = arg min w ∈W L ( w , S r ) be the minimizers of the empirical risk over the full and retain sets, respectively. By linearizing the network output around w ∗ using the NTK approximation, NTK enables a closed-form update that shifts the model weights from w ∗ to an approximation of w r via a one-shot adjustment:

where the projection matrix P = I - ∇ w ∗ h w ∗ ( S r ) ⊤ K ( S r , S r ) -1 ∇ w ∗ h w ∗ ( S r ) ensures that the gradient contributions from the forget set are orthogonalized against those from the retain set. The matrix M is defined as

and the re-weighting matrix V is given by

where y f and y r denote the ground truth labels for the forget and retain sets, respectively. We follow the official implementation and use the same settings and hyper-parameters as in https: //github.com/AdityaGolatkar/SelectiveForgetting .

Gradient Descent ( GD ). GD (Neel et al., 2021) is one of the simplest unlearning algorithms. It continues training the original model Original on the retain set S r using standard gradient descent. Specifically, the unlearned model M is obtained by iteratively applying the update:

where η is the step size and g t ( w t ) is a (mini-batch) stochastic gradient of the empirical loss over S r , i.e., 1 |S r | ∑ s ∈S r ℓ ( w , s ) . The key intuition behind GD is that when the forget set is small (i.e., |S f | ≪|S| ), the minimizers of the loss functions over S and S r are expected to be close. Therefore, a few steps of gradient descent starting from the original model w 1 = A ( S ) can efficiently move the parameters toward a minimizer of the updated training objective. Building on this intuition, Neel et al. (2021) also provide theoretical guarantees for the effectiveness of GD in both convex and certain non-convex settings.

Our implementation follows https://github.com/ChrisWaites/descent-to-delete , using the hyper-parameters :

SGD optimizer with a lr = 1e-2, momentum = 0.9, and weight decay = 1e-4.

Random seed for the data loader is 7; random seeds for repeated experiments are 131, 42, 7.

Batch size = 128

Number of epochs = 10

Noisy Gradient Descent ( NGD ). NGD (Chourasia and Shah, 2023) is a simple extension of GD in which Gaussian noise is added to each gradient update. The unlearned model is obtained by iteratively applying the update:

where η is the step size, g t ( w t ) denotes a (mini-batch) stochastic gradient of the empirical loss over the retain set S r , i.e., 1 |S r | ∑ s ∈S r ℓ ( w , s ) , and ξ t ∼ N (0 , σ 2 ) is an independent Gaussian noise term added at each iteration. The key distinction between NGD and GD lies in the injection of noise during optimization. This stochasticity not only increases the robustness of the unlearning process but also enables formal privacy guarantees, as demonstrated in the context of certified unlearning via Rényi differential privacy (Chien et al., 2024). Notably, a similar noise-injection mechanism is employed in the DP-SGD algorithm for training models with differential privacy guarantees (Abadi et al., 2016).

Our implementation follows https://github.com/Graph-COM/Langevin_unlearning and the same hyper-parameters as GD with

( σ, η ) = (0 . 03 , 0 . 1) , trained for 10 epochs for the small CIFAR-5 settings with sample unlearning.

( σ, η ) = (0 . 03 , 0 . 1) , trained for 2 epochs for the CIFAR-10 settings with sample unlearning.

Gradient Descent ( GA ). GA (Graves et al., 2021) aims to remove the influence of the forget set S f from a trained model by reversing the learning process through gradient ascent. It stores all gradient updates involving S f during the initial training phase, and unlearning is then performed by applying the exact reverse updates-i.e., gradient ascent-using the stored gradients. However, this

approach is highly memory-intensive and becomes impractical for large-scale models. To address this limitation, Jang et al. (2022) proposed a more scalable variant, in which unlearning is achieved by performing mini-batch gradient ascent on the forget set. Specifically, the model is updated by minimizing the negated loss -L ( w , S f ) , effectively implementing ascent steps that counteract the original training influence of S f .

Our implementation follows https://github.com/joeljang/knowledge-unlearning . using the similar hyper-parameters as GD :

SGD optimizer with a lr = 1e-5, momentum = 0.9, and weight decay = 1e-4, clipping gradient norm = 1.

Random seed for the data loader is 7; random seeds for repeated experiments are 131, 42, 7.

Batch size = 128

Number of epochs = 1 for Small-CIFAR-5 (both sample and class unlearning) and CIFAR-10 with sample unlearning; while 3 for CIFAR-10 with class unlearning

Negative Gradient Plus ( NegGrad+ ). NegGrad+ (Kurmanji et al., 2024) fine-tunes the model by simultaneously minimizing the loss on the retain set S r and maximizing the loss on the forget set S f , effectively negating the gradient contributions from the latter. Specifically, the unlearned model is obtained by optimizing the following objective:

where β is a hyper-parameter that controls the strength of unlearning by weighting the loss contribution from S f . NegGrad+ shares conceptual similarity with the Gradient Ascent ( GA ) method, as both perform loss maximization on the forget set to induce forgetting. However, NegGrad+ is empirically more stable and yields better performance, as it simultaneously enforces loss minimization on the retain set, ensuring that useful knowledge is preserved during unlearning.

Our implementation follows https://github.com/meghdadk/SCRUB , using similar hyperparameters as GD and GA :

SGD optimizer with a lr = 1e-2, momentum = 0.9, and weight decay = 1e-4, clipping gradient norm = 1.

β = 0.001

Random seed for the data loader is 7; random seeds for repeated experiments are 131, 42, 7.

Batch size = 128

Number of epochs = 1 for Small-CIFAR-5 (both sample and class unlearning) and CIFAR-10 with sample unlearning; while 3 for CIFAR-10 with class unlearning

Exact Unlearning the last k layers ( EU-k ). EU-k (Goel et al., 2022) is a simple, parameter-efficient unlearning method designed for deep learning models, requiring access only to the retain set S r . Given a parameter k , EU-k retrains from scratch the last k layers of the neural network-those closest to the output layer-while keeping the earlier layers fixed. By adjusting k , EU-k provides a tunable trade-off between unlearning effectiveness and computational efficiency.

Our implementation is based on the official repository at https://github.com/shash42/ Evaluating-Inexact-Unlearning , using the following hyper-parameters:

SGD optimizer with a lr = 1e-2, momentum = 0.9, and weight decay = 1e-4.

Last fully-connected layer, and last two residual blocks are reinitialized, and only fine-tune those parameters.

Random seed for the data loader is 7; random seeds for repeated experiments are 131, 42, 7.

Batch size = 128

Number of epochs = 10

Catastrophically Forgetting the last k layers ( CF-k ). CF-k (Goel et al., 2022) builds on the observation that neural networks tend to forget information about data samples encountered early in training-a phenomenon known as catastrophic forgetting (French, 1999). Similar to EU-k , CF-k focuses on the last k layers of the network; however, instead of re-training these layers from scratch, it fine-tunes them starting from the original model parameters w = A ( S ) , using only the retain set S r , while keeping all earlier layers frozen.

As with EU-k , our implementation follows the official repository at https://github.com/ shash42/Evaluating-Inexact-Unlearning , using the following hyper-parameters:

SGD optimizer with a lr = 1e-2, momentum = 0.9, and weight decay = 1e-4.

Last fully-connected layer, and last two residual blocks are fine-tuned.

Random seed for the data loader is 7; random seeds for repeated experiments are 131, 42, 7.

Batch size = 128

Number of epochs = 10

SCalable Remembering and Unlearning unBound ( SCRUB ). SCRUB (Kurmanji et al., 2024) is one of the state-of-the-art unlearning methods for deep learning. It formulates the unlearning task within a student-teacher framework: given a trained teacher model w T , the goal is to train a student model w that selectively imitates the teacher. Specifically, the student should retain the teacher's behavior on the retain set S r while diverging significantly from it on the forget set S f , as measured by the KL divergence. To achieve this, SCRUB optimizes a modified knowledge distillation objective (Hinton et al., 2014):

where α and γ are hyper-parameters that balance knowledge retention and task performance. The first expectation encourages the student to match the teacher on the retain set while also performing well on the classification task, and the second term enforces divergence from the teacher on the forget set.

Our implementation follows https://github.com/meghdadk/SCRUB , with the hyper-parameters:

SGD optimizer with a lr = 5e-4, momentum = 0.9, and weight decay = 5e-4.

Random seed for the data loader is 7; random seeds for repeated experiments are 131, 42, 7.

Batch size = 128

α = 0.001

γ = 1 for small CIFAR-5 settings with both sample and class unlearning

γ = 0.6 for CIFAR-10 setting with sample unlearning and γ = 75 for CIFAR-10 with class unlearning

Selective Synaptic Dampening ( SSD ). SSD was introduced by Foster et al. (2024) as a method to unlearn a specific forget set from a neural network without retraining it from scratch. Building on ideas similar to Fisher , but with a more refined approach, SSD selectively dampens weights that exhibit disproportionately high influence-measured via the Fisher information-on the forget set relative to the retain set. Given a model with parameters w , let F r and F f denote the Fisher information matrices computed over the retain set S r and the forget set S f , respectively. Unlearning is performed by scaling each parameter w i according to its relative Fisher sensitivity:

where F f,i and F r,i denote the i -th diagonal entries of the Fisher matrices for S f and S r , respectively. The hyper-parameter α controls the threshold for selecting influential weights, and the dampening factor β is defined as β = min i λ F r,i / F f,i , 1 for a tunable parameter λ .

Our implementation is based on the official repository at https://github.com/if-loops/ selective-synaptic-dampening , using the following hyper-parameters:

λ = 1.0

α = 10.0 for CIFAR-10 and 100.0 for Small-CIFAR-5.

B.4 Training and evaluation details

Here, we provide details on dataset preparation, unlearning settings, and evaluation procedures.

Dataset and unlearning settings. We evaluate unlearning methods in the context of image classification using CIFAR-10 (Krizhevsky et al., 2009) and ResNet-18 (He et al., 2016), focusing on the random sample unlearning setting across two scenarios. The CIFAR-10 dataset contains 60,000 color images of size 32 × 32 pixels, evenly distributed across 10 classes: airplane, automobile, bird, cat, deer, dog, frog, horse, ship, and truck. Each class includes 5,000 training and 1,000 test samples. CIFAR-10 is widely used in machine unlearning research-for example, by Kurmanji et al. (2024); Chien et al. (2024); Golatkar et al. (2020a,b); Foster et al. (2024). In the first scenario-small CIFAR5-we follow the setup of Golatkar et al. (2020a,b) by creating a reduced version of CIFAR-10 with 200 training and 200 test samples from each of the first five classes. From class 0, we randomly select 100 samples (50%) as the forget set S f . The second scenario uses the full CIFAR-10 dataset. Here, we designate 2,000 samples (50% 12 ) from class 0 as the forget set S f . In addition to sample unlearning, we also consider class unlearning. In this setting, the forget set contains 200 samples from class 0 in the small CIFAR-5 scenario, and 4,000 samples from class 0 in the full CIFAR-10 scenario. All experiments are conducted on an AWS EC2 g5.24xlarge instance.

Training of Original and Re-train . To avoid any external knowledge (e.g., the default ImageNet pre-trained weights from torchvision (Marcel and Rodriguez, 2010)), both the Original and Re-train models are trained entirely from scratch, without loading any pre-trained weights. We use a batch size of 128 and cross-entropy loss ( torch.nn.CrossEntropyLoss() ). For the small CIFAR-5 setting, we first pre-train a ResNet18 model on the full CIFAR-10 dataset using the SGD optimizer ( torch.optim.SGD(lr=0.4, momentum=0.9, weight_decay=2e-4) ) along with a cosine annealing learning rate scheduler ( torch.optim.lr_scheduler.CosineAnnealingLR(T_max=200) ). We then fine-tune this model on small CIFAR-5 for 25 epochs with a learning rate of 0.01 to obtain the Original model reported in Table 1. The Re-train model in Table 1 is obtained using the same setup, except the fine-tuning is performed on the retain set S r for 100 epochs. For the CIFAR-10 setting, both the Original and Re-train models are trained from scratch on the full dataset and the retain set, respectively, for 137 epochs using a learning rate of 0.01. For the VGG-11 ablation study, we follow the same training protocol, including the optimizer, scheduler, and learning rate.

Evaluation details. The retain, forget, and test accuracies reported in Table 1 are computed by directly evaluating each model on the corresponding retain, forget, and test sets. The unlearning accuracy is defined as 1 -forget accuracy. The re-learn time quantifies how easily a model can reacquire knowledge of the forget set. It is defined as the number of fine-tuning epochs required for the model m to satisfy L ( m, S f ) ≤ (1 + η ) L ( Original , S f ) , where we set η = 0 . 05 . This value can be adjusted depending on the desired tolerance.

To evaluate vulnerability to membership inference attacks (MIA), we use a support vector classifier (SVC) implemented via the scikit-learn package, with parameters SVC(C=3, gamma='auto', kernel='rbf') (Pedregosa et al., 2011). The SVC is trained to distinguish between samples seen during training and those that were not, based on the model's output likelihood for the correct class label. In the sample unlearning setting, retain samples are labeled as 1 (seen) and test samples as 0 (unseen) during SVC training; forget samples are then evaluated by the SVC, and an ideal unlearned model should cause them to be classified as 0. In the class unlearning setting, retain samples are labeled as 1 and forget samples as 0 for SVC training; evaluation is then conducted on a held-out subset of the forget class. The reported MIA Accuracy in Table 1 corresponds to the SVC's attack failure rate, i.e., the proportion of forget samples classified as unseen.

Both the construction of the vulnerable set V (( x , y ) , τ ) and the evaluation of residual knowledge r τ ( S f ) involve generating perturbed inputs within the norm ball B p ( x , τ ) . We implement this using the torchattacks package (Kim, 2020). For Gaussian noise (i.e., p = 2 ), we use torchattacks.GN(model, std= τ ) ; for FGSM (i.e., p = ∞ ), we use torchattacks.FGSM(model, eps= τ ) ; and for PGD, we use torchattacks.PGD(model,

eps= τ , alpha=2/255., steps=pgd_epoch, random_start=True) . We apply targeted attacks for both FGSM and PGD.

̸

To estimate residual knowledge, we apply these attacks to the unlearned model m to generate perturbed inputs x ′ , and compute both Pr[ m ( x ′ ) = y ] and Pr[ a ( x ′ ) = y ] over c = 100 independent runs, using the same perturbation x ′ for both models. Constructing the vulnerable set V (( x , y ) , τ ) involves solving the minimization problem min z ∈B p ( x ,τ ) ℓ ( w , ( z , y )) . However, since the torchattacks package is designed to solve the adversarial objective max z ∈B p ( x ,τ ) ℓ ( w , ( z , y )) , we instead define a random target label y ′ = y and solve max z ∈B p ( x ,τ ) ℓ ( w , ( z , y ′ )) . To adapt this into our residual knowledge framework, we flip the sign of the regularization term in Eq. (6), effectively encouraging the model to associate perturbed variants of forget samples with an incorrect label y ′ . This discourages the model from retaining knowledge of the true label and facilitates unlearning. We repeat this process v times for each forget sample to construct its corresponding vulnerable set.

B.5 Details on Figure 1

Figure 1 is based on the UCI Iris dataset (3 classes, 50 samples each, 4 features). For both the original and re-trained models, we use a simple feedforward neural network with one hidden layer of 100 neurons. We select 7 forget samples in total from three class for illustration. While this selection differs from the random sample or class unlearning setup (Lines 297-303), the goal is purely illustrative-to convey the intuition behind the existence of disagreement and residual knowledge. The unlearned model is trained via GD.

C Additional results and experiments

We include additional experimental results and comparisons, including: (i) comprehensive results on sample unlearning for both the small CIFAR-5 and CIFAR-10 scenarios, covering accuracy metrics and residual knowledge estimates under various adversarial attacks; (ii) results on class unlearning for both small CIFAR-5 and CIFAR-10; (iii) ablation studies on the hyper-parameters of RURK ; and (iv) an analysis of the prevalence of residual knowledge across different settings.

C.1 Complete results on sample unlearning for small CIFAR-5

We present the full version of Table 1 under the small CIFAR-5 setting in Table C.2. Combined with Table 1, the results demonstrate that RURK consistently achieves superior performance-surpassing certified removal methods such as CR , Fisher , and NTK , as well as deep-learning-compatible approaches like SCRUB , NegGrad+ , and NGD -by achieving the smallest average gap from the re-trained model Re-train . Notably, the re-learn time of RURK is also comparable to that of Re-train .

We further include the complete residual knowledge curves estimated under different adversarial attacks: Gaussian noise in Figure C.4, FGSM in Figure C.5, and PGD in Figure C.6. For both Gaussian noise and FGSM, most methods exhibit residual knowledge greater than 1-i.e., inheriting information from the Original model-whereas RURK maintains residual knowledge close to 1 for small perturbation norms τ , and reduces it below 1 as τ increases.

With PGD (10 steps), a stronger attack, we observe that methods such as NTK , NGD , and SCRUB still suffer from residual knowledge. In contrast, methods like GD , GA , NegGrad+ , EU-k , CF-k , and SSD maintain residual knowledge near 1 across varying values of τ . In particular, SSD strictly preserves residual knowledge at 1, though this comes at the cost of a larger average gap from Re-train (cf. Table C.2).

Overall, RURK effectively suppresses residual knowledge to values below 1 for small τ , and keeps it close to 1 as τ increases-striking a favorable balance between preserving accuracy and mitigating residual knowledge.

Small CIFAR-5 (Sample Unlearning, Gaussian Noise)

Original

Fisher

NTK

NGD

GA

1Knowledgei

S

1.2

1.3

0.9

1.2

1.1

I

1.05

HIIII

IIII

1.15

1.10

1.025

1.000

0.8

1.00

0.00

0.02

0.00

0.02

0.00

0.02

0.00

0.02

0.00

0.02

0.00

0.02

NegGrad+

EU-k

CF-k

SCRUB

SSD

RURK

Residual H

1.000

1.05

EIII

1.05

1.10

1.05

0.8

0.975

1.00

1.00

1.05

1.00

0.6

0.00

0.02

0.00

0.02

0.00

0.02

0.00

0.02

0.00

0.02

0.00

0.02

PerturbNormt

Small CIFAR-5 (Sample Unlearning,FGSM)

Original

Fisher

NTK

NGD

GA

S

1.2

1.00

1.10

1.02

1.2

1.000

60

1.1

0.95

1.05

1.00

1.1

0.975

e

0.00

0.02

0.00

0.02

0.00

0.02

0.00

0.02

0.00

0.02

0.00

0.02

Knc

NegGrad+

EU-k

CF-k

SCRUB

SSD

RURK

dual

1.000

1.05

1.025

1.10

1.00

0.9

S

0.975

1.00

1.000

1.05

0.98

0.8

R

0.00

0.02

0.00

0.02

0.00

0.02

0.00

0.02

0.00

0.02

0.00

0.02

PerturbNormt

Small CIFAR-5 (SampleUnlearning,PGD)

Original

Fisher

NTK

NGD

GA

S

1.1

1.00

王

1.05

1.0025

1.1

1.002

1.0

0.00

0.02

0.95

0.00

0.02

1.00

0.00

0.02

1.0000

0.00

0.02

1.0

0.00

0.02

1.000

0.998

0.00

0.02

NegGrad+

EU-k

CF-k

SCRUB

SSD

RURK

K

1.0025

1.005

1.005

1.1

1.05

1.00

P

三

S

1.0000

0.9975

1.000

王I王王王室

1.000

1.0

1.00

0.95

0.95

R

0.00

0.02

0.00

0.02

0.00

0.02

0.00

0.02

0.00

0.02

0.00

0.02

PerturbNormt

Weprovide the residual knowledge estimates under FGSM in Figure C.7 and under PGD in Figure C.8, corresponding to the results presented in Table 1 and Figure 2 of the main text. RURK demonstrates consistent behavior across all adversarial attack types-Gaussian noise, FGSM, and PGD-by maintaining residual knowledge close to 1 for small perturbation radii τ , and effectively suppressing it below 1 as τ increases.

CIFAR-10

Original

GA

NGD

NegGrad+

S

1.1

0.4

1.00

1.1

1.0

1.0

0.2

0.75

1.0

0.5

owl

e

0.00

0.02

0.00

0.02

0.00

0.02

0.00

0.02

0.00

0.02

Knc

EU-k

CF-k

SCRUB

SSD

RURK

1.10

1.0

0.4

1.0

1.00

1.05

0.2

0.5

S

0.75

0.5

R

e

0.00

0.02

0.00

0.02

0.00

0.02

0.00

0.02

0.00

0.02

Perturb Norm T

CIFAR-10

Original

GA

NGD

NegGrad+

S

1.10

0.7

1.025

1.100

1.025

0.6

1.000

1.05

0.5

1.000

1.075

0.975

e

0.00

0.02

0.00

0.02

0.00

0.02

0.00

0.02

0.00

0.02

Knc

EU-k

CF-k

SCRUB

SSD

RURK

1.10

1.025

0.6

1.10

0.9

idua

1.05

1.000

0.8

e

S

1.05

0.975

0.4

R

0.00

0.02

0.00

0.02

0.00

0.02

0.00

0.02

0.00

0.02

Perturb Norm t

C.2 Class unlearning: Forgetting all samples in a class

In the main text, we focus on the sample unlearning setting, where 50% of the samples from class 0 are unlearned. Here, we provide analogous results for the class unlearning setting, where 100% of the samples from class 0 are removed, on both small CIFAR-5 and CIFAR-10. Note that in class unlearning, the Re-train model is never exposed to any samples from class 0. As a result, the unlearning accuracy is 100%-that is, Re-train never correctly classifies any forget sample into class 0. In Table C.3, we report the same evaluation metrics as in Table 1, including the average absolute gaps in retain, unlearn, test, and MIA accuracy compared to Re-train .

In both small CIFAR-5 and CIFAR-10, EU-k achieves the smallest average gap relative to Re-train . This result is expected, as class unlearning in this case resembles a transfer learning scenario: from a source domain (small CIFAR-5/CIFAR-10) to a target domain (small CIFAR-4/CIFAR-9), where the forget class is entirely absent. This creates a clear domain shift, allowing EU-k to perform well by fine-tuning only the top layers. In contrast, in sample unlearning, both the source and target domains still contain samples from class 0, making transfer learning less effective and leading to poorer performance for EU-k .

We also present residual knowledge estimates for the small CIFAR-5 class unlearning scenario under Gaussian noise (Figure C.9), FGSM (Figure C.10), and PGD (Figure C.11). While RURK is the second-best performer in terms of average accuracy gap in the small CIFAR-5 setting, it outperforms EU-k in suppressing residual knowledge. In the CIFAR-10 class unlearning scenario, RURK is the best among popular methods such as GA , SCRUB and SSD .

Small CIFAR-5 (Class Unlearning, Gaussian Noise)

Original

Fisher

NTK

NGD

GA

S

1.2

0.9

瓜王亚王王王

0.05

1.05

0.05

1.00

1.1

0.8

0.00

1.00

0.00

nowl

0.00

0.02

0.00

0.02

-0.05

0.00

0.02

0.00

0.02

0.05

0.00

0.02

0.95

0.00

0.02

NegGrad+

EU-k

CF-k

SCRUB

SSD

RURK

idual

1.025

1.000

1.025

1.000

1.025

1.000

0.8

1.025

0.15

0.10

0.975

0.975

0.975

0.6

1.000

0.05

R

0.00

0.02

0.00

0.02

0.00

0.02

0.00

0.02

0.00

0.02

0.00

0.02

PerturbNormT

Small CIFAR-5 (Class Unlearning,FGSM)

Original

Fisher

NTK

NGD

GA

S

1.15

1.05

1.10

0.00

0.02

0.90

0.95

0.00

0.02

0.05

0.05

0.00

0.00

0.02

1.1

1.0

0.00

0.02

0.05

0.05

0.00

0.00

0.02

1.0

1.1

0.00

0.02

Knc

NegGrad+

EU-k

CF-k

SCRUB

SSD

RURK

idual

1.1

1.00

0.95

1.05

0.9

0.8

1.1

0.2

0.1

lesi

1.0

1.00

0.7

1.0

R

0.00

0.02

0.00

0.02

0.00

0.02

0.00

0.02

0.00

0.02

0.00

0.02

PerturbNormt

Small CIFAR-5 (Class Unlearning, PGD)

Original

Fisher

NTK

NGD

GA

S

1.05

1.0

0.05

0.00

1.025

0.05

0.00

1.025

1.00

0.00

0.02

0.9

0.00

0.02

0.05

0.00

0.02

1.000

0.00

0.02

0.05

0.00

0.02

1.000

0.00

0.02

1.025

NegGrad+

王王王王王王

1.05

EU-k

正王王王王王

1.025

CF-k

王王王王王

0.95

SCRUB

1.025

SSD

0.50

RURK

三

0.90

0.25

R

S

e

1.000

0.00

0.02

1.00

0.00

0.02

1.000

0.00

0.02

0.85

0.00

0.02

1.000

0.00

0.02

0.00

0.02

PerturbNormt

C.3 Ablation studies on RURK

In this section, we conduct ablation studies on the key hyper-parameters of RURK , including the perturbation radius τ , regularization strength λ , sample size v , types of adversarial attacks, and alternative model architectures. We focus on the sample unlearning setting in the small CIFAR-5 scenario, with all results summarized in Table C.4, Table C.5, Figure C.12, and Figure C.13.

Different perturbation norms τ . The perturbation norm τ determines the radius of the perturbation ball used to evaluate residual knowledge. We set τ = 0 . 03 ≈ 8 / 255 , which aligns with common practice for the maximum adversarial perturbation. To assess the sensitivity of RURK to this parameter, we conduct an ablation study over τ ∈ { 0 . 00 , 0 . 01 , 0 . 02 , 0 . 03 , 0 . 04 , 0 . 10 } . As shown in Table C.4, when τ = 0 . 00 or 0 . 01 , RURK fails to unlearn the forget samples from Original . However, as τ increases, the unlearning accuracy improves monotonically, while the test accuracy declines-as expected due to the increased regularization. Notably, RURK achieves the best average gap when τ = 0 . 03 . Since τ is the key hyper-parameter controlling the trade-off between unlearning efficacy and utility, we further visualize the estimated residual knowledge under varying τ in Figure C.12. The figure shows that increasing τ improves robustness against residual knowledge under stronger attacks. For instance, under Gaussian noise and FGSM, the residual knowledge remains near 1 for small τ . In contrast, PGD-a stronger adversary-can still uncover residual knowledge at low τ values, but its effectiveness diminishes significantly when τ = 0 . 1 , indicating improved robustness.

Different regularization strengths λ . The regularization strength λ controls the weighting of samples in the vulnerable set within the loss function defined in Eq. (6). We sweep over λ ∈ { 0 . 00 , 0 . 01 , 0 . 02 , 0 . 03 , 0 . 04 , 0 . 10 } and observe that the trend in the average gap closely mirrors that of τ . This similarity is expected, as both τ and λ jointly influence the extent to which residual knowledge is removed during unlearning.

Other methods to search for the vulnerable set. In the main text, we search for samples in the vulnerable set V (( x , y ) , τ ) using Gaussian noise. Here, we extend the evaluation to include stronger adversarial methods, such as FGSM and PGD, with varying numbers of attack steps (indicated by the number following PGD in Table C.4). We observe that FGSM-a targeted attack-achieves MIA accuracy comparable to that of Re-train . Likewise, PGD with 5 steps also yields MIA accuracy close to Re-train , indicating effective removal of forget-set information. These results suggest that targeted attacks (FGSM, PGD) are more effective than untargeted ones like Gaussian noise in eliminating residual knowledge, albeit at the cost of increased unlearning time.

Different samples size v . The sample size v in Eq.(6) determines how extensively the vulnerable set V (( x , y ) , τ ) is explored through sampling. In high-dimensional settings, V (( x , y ) , τ ) may contain a large number of perturbed variants. However, due to computational constraints, it is impractical to include all possible samples during training. Instead, we randomly draw v samples from V (( x , y ) , τ ) for each forget sample. We experiment with v ∈ { 1 , 2 , 3 , 4 } and observe that the average gap achieved by RURK remains relatively stable across these values. This robustness is expected, as the loss term κ ( w , ( x , y )) in Eq.(6) is computed as an average over the v sampled perturbations.

Other learning structures. Thus far, our evaluation of RURK has focused on models trained with ResNet-18. To assess its generality across architectures, we present results on small CIFAR-5 using VGG-11 (Simonyan and Zisserman, 2014), as shown in Table C.5. Compared to NGD , RURK continues to achieve the smallest average gap. Figure C.13 further illustrates the residual knowledge comparison between RURK and NGD , demonstrating that RURK maintains lower and more stable residual knowledge across perturbations. These results confirm that RURK effectively removes forget-set information and controls residual knowledge across different network architectures, including both plain convolutional networks (VGG-11) and those with residual connections (ResNet-18).

Small CIFAR-5(Sample Unlearning)

1.15

TTIIE

T=0.00

1.15

EEEETTT

↑ =0.01

1.1

PEIIEEEETT

T =0.02

1.0

T=0.03

1.00

=0.04

0.75

=0.10

(Js)2

1.10

1.10

1.05

1.0

0.8

0.75

0.50

0.25

Knowledge

0.00

0.02

0.00

0.02

0.00

0.02

0.00

0.02

0.00

0.02

0.00

0.02

GSM

1.15

1.15

1.1

1.0

1.0

0.75

Residual

1.10

1.05

1.10

1.0

0.8

0.8

0.50

0.25

0.00

0.02

0.00

0.02

0.00

0.02

0.00

0.02

0.00

0.02

0.00

0.02

R

D

1.1

1.1

1.1

1.05

1.025

1.000

中1

0.925

P

1.0

1.0

1.0

1.00

0.975

0.900

0.00

0.02

0.00

0.02

0.00

0.02

0.00

0.02

0.00

0.02

0.00

0.02

PerturbNormt

Small CIFAR-5 (Sample Unlearning, VGG-11)

Original

NGD

RURK

lan

1.25

1.2

1.1

1.0

1.20

1.1

0.9

0.00

0.02

0.00

0.02

0.00

0.02

SM

1.15

1.1

FG

1.175

1.10

1.0

Residual

1.150

1.05

0.9

0.00

0.02

0.00

0.02

0.00

0.02

1.15

1.15

1.1

1.10

1.10

1.05

1.05

1.0

0.00

0.02

0.00

0.02

0.00

0.02

PerturbNormt

C.4 The prevalence of residual knowledge

In Figure 2, we report the estimated residual knowledge over the entire forget set ˆ r τ ( S f ) . We also provide per-sample estimates ˆ r τ (( x , y )) for each ( x , y ) ∈ S f , as defined in Eq. (4). Table C.6 summarizes the proportion of forget samples exhibiting residual knowledge greater than one-i.e., those still recognizable after unlearning-under different perturbation norms τ , evaluated on small CIFAR-5 (with NTK ) and CIFAR-10 (with NGD ). Remarkably, even under imperceptibly small perturbations (e.g., τ = 0 . 013 ), about 11% of the forget samples in small CIFAR-5 and more than 8% (approximately 160 samples) in CIFAR-10 still exhibit residual knowledge above one. These findings highlight that residual knowledge is not only prevalent but also poses a significant privacy risk, emphasizing the need for stronger certification and mitigation techniques in machine unlearning.

C.5 Disagreement and unlearn accuracy on perturbed forget samples

We provide the adversarial disagreement (§ 4) and the unlearn accuracy on perturbed forget samples for selected baselines on both Small CIFAR-5 and CIFAR-10 (cf. Table 1) in the following four tables.

In Table C.7, RURK achieves the lowest disagreement among all baselines when the perturbation norm is small (e.g., below 0.0125), consistent with the stable residual knowledge curves shown in Figure 2. As the perturbation magnitude increases, however, RURK reduces residual knowledge more aggressively, leading to higher disagreement-a trade-off that arises naturally in multi-class settings, where low disagreement does not necessarily imply residual knowledge close to 1 as in binary cases. Similarly, in Table C.8, RURK attains unlearn accuracy on perturbed forget samples closest to that of the re-trained model, particularly for small perturbations, indicating reduced distinguishability between the two models around the forget region. It is worth noting that RURK is designed to control residual knowledge rather than disagreement directly, as managing disagreement across multiple classes is inherently more complex.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nMachine unlearning is an approach to remove specific data without re-training the model. While its effectiveness has been proven, this paper theoretically proves that the unlearned model may suffer from perturbed forget samples, raising a new risk of unlearning. With the DP framework, they introduce model distinguishability and sample disagreement, depending on which they formulate the vulnerability as residual knowledge and propose RURK to constrain the model's ability to recognize perturbed forget samples. Extensive results show the effectiveness of their method.\n\nSTRENGTHS AND WEAKNESSES:\nStrength:\n\n- The paper is well written and easy to follow.\n- The problem is well motivated. While most prior work focuses on the forget performance of unlearned samples, that of perturbed forget samples is not well studied, though interesting and important.\n- The experiments are comprehensive, including 11 baselines, 5 metrics, and 2 datasets to evaluate their effectiveness in multiple dimensions.\n\nWeakness:\n- The size of the models and datasets used in this paper is too small. As the unlearning of generative models is receiving more and more attention and is a critical part in unlearning, adding experiments on diffusion models can enhance this paper's impact, though I don't see a direct or easy application of the proposed method on diffusion models.\n- The assumptions of some theories are not that practical or maybe there are some mistakes. See questions.\n- The relative ratio $r_{\\tau}$ seems insufficient to measure the residual knowledge. See questions\n- The introduction of residual knowledge is confusing to me. As Sec.3 has proven that an unlearned model can still correctly predict the perturbed forget samples, it is enough to elicit the RURK method, which unlearns the perturbed forget samples. Why do we need residual knowledge and $r_{\\tau}$? Does the RURK loss reduce $r_{\\tau}$? Adding more discussion would enhance the understanding of this paper.\n\nQUESTIONS:\n- I may miss something, but I didn't find any illustration on how to get Fig.1. I think it would be better to give a brief introduction.\n- For Proposition 2:\n    - First, the assumption that the sample space is a sphere is too strict and not very practical.\n    - Second, when $\\epsilon=\\delta=0$, why does Eq.3 become zero, as the first term $\\frac{2\\delta}{1-e^{\\epsilon}}$ is $\\frac{0}{0}$?\n- As $r_{\\tau}$ is a relative ratio, is it enough to measure the residual knowledge inside the unlearned model $m$? For example, if $\\Pr[m(x')=y] = N \\Pr[a(x')=y] = N\\xi$ where $\\xi \\to 0^+$ and $N$ is a large constant. Thus, $r_{\\tau}=N$ can be large but $\\Pr[m(x')=y]$ is small. In that case, can we say the unlearned model has a large amount of residual knowledge since both models are unconfident and unlikely to classify the label correctly?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-23"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors examine what they see as an overlooked threat in the field of machine unlearning: though the samples themselves may appear to be unlearned (as gauged by classification accuracy on these samples), perturbations of the data in $\\mathcal{S}_f$ are still classified with high accuracy by the model.  The authors propose a method of unlearning designed to reduce this threat, and they provide a novel notion of residual knowledge.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n- In general, the idea that unlearning doesn't handle perturbations of the samples themselves is really interesting and indeed overlooked.  This was a good insight.\n- The authors do a good job defining their terms with mathematical precision, and providing mathematical support for their ideas.  This is well done.\n- The paper is clearly written and proper care is taken.\n\nQUESTIONS:\n- Why is your unlearn accuracy 0.00 for the original method, and why are the error bars +/- 0?  This seems like a fishy estimate of the error bars given that the unlearn set is randomly selected from the data.\n- The motivation for the definition of unlearning comes from differential privacy, where the randomness comes from the optimization process.  However, in your definition of residual knowledge, the randomness comes instead from sampling inside of $B_r(p)$ where $p\\in \\mathcal{S}_f$.  How do you reconcile these two definitions of randomness?  It's not entirely clear to me that this is the \"right\" definition of residual knowledge.  Can you motivate this a bit better?\n- As a follow-up to my previous question, in your definition of residual knowledge, you write that randomness comes from sampling inside of the ball around the forget point.  However, you also mention that you are adversarially perturbing.  How are you sampling from $B_r(p)$?  If you are adversarially perturbing, then this doesn't seem like a very realistic threat.\n- I find it a little bit concerning that RURK causes larger adversarial perturbations to do worse against the unlearned model.  Can you explain this behavior?\n- Are you able to convert your insight into a membership inference attack against unlearned models?  This seems like a good practical application.\n- What is the implication of this research for generative models?  If you don't already have the training data you can't perturb it, so I'm not entirely sure how it helps you extract data or do something harmful if perturbations reveal membership in the training data.\n\nLIMITATIONS:\n- See the questions about definitions and adversarial perturbations.\n-The unlearning accuracy seems way too high for RURK-- you only need to match the test data's accuracy.\n- The experiments only look at classifiers on one dataset (counting CIFAR-5 and CIFAR-10 as the same), making them effectively toy examples relative to the scale of modern machine learning.  This is, of course, very difficult to fix during a review period. \n- I think that it would be nice to see an experiment with randomly labeled data or a mislabeled subset as the unlearning dataset.  In the current experiments, it's not clear to me whether perturbations around the training data recover the original labels because of residual knowledge or because you've unlearned too heavily exactly at the points in $\\mathcal{S}_r$, but once you move away from $\\mathcal{S}_r$ the classifier should function as normal.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 2, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-25"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper investigates a new privacy risk in machine unlearning, termed residual knowledge, where an unlearned model retains latent traces of forget samples, observable through small adversarial perturbations, despite formal statistical indistinguishability from a re-trained model. The authors provide theoretical arguments, including isoperimetric inequality-based proofs, suggesting that disagreement between unlearned and re-trained models under perturbations is inevitable in high-dimensional settings. They introduce a metric to quantify residual knowledge and propose RURK, a fine-tuning strategy that penalizes correct classification of perturbed forget samples, to mitigate this risk. The paper presents empirical results on CIFAR-5 and CIFAR-10, evaluating RURK against several baseline unlearning methods.\n\nSTRENGTHS AND WEAKNESSES:\n# Strengths: \n\n- The paper formalizes residual knowledge as a distinct privacy concern in unlearning, providing a new perspective on the limitations of current guarantees. \n\n- The theoretical treatment leverages established tools from geometric probability to support the inevitability of local disagreement under adversarial perturbations. \n\n- The proposed RURK method is simple in formulation and integrates with existing gradient-based unlearning pipelines. \n\n- Extensive experiments are conducted on two unlearning scenarios (CIFAR-5, CIFAR-10) with 11 competitive baseline methods, covering multiple evaluation dimensions including accuracy, MIA, and re-learn time. \n\n \n\n# Weaknesses: \n\n- The theoretical results rely heavily on the isoperimetric inequality on unit spheres, assuming normalized data in high-dimensional Euclidean space. However, in practical deep learning settings, especially with complex data distributions and learned feature spaces, the applicability of this geometric abstraction is questionable. The authors do not justify how these assumptions transfer to real neural network representations. \n\n- The formal inevitability of disagreement is interesting theoretically but lacks quantitative grounding in realistic settings. The paper provides no evidence on the actual magnitude of this residual knowledge in large, real-world models beyond small CIFAR benchmarks, making the practical severity unclear. \n\n- The empirical evaluation is narrow, focusing only on vision tasks with modest-scale datasets (CIFAR-5 and CIFAR-10) and small-to-mid-size models (ResNet-18, VGG). There is no experimental evidence on modern large-scale architectures or other modalities like NLP or diffusion models, where both unlearning and adversarial perturbations exhibit different behaviors. \n\n- The proposed RURK method requires direct access to the forget set during fine-tuning to generate adversarial perturbations for mitigation. This undermines practical applicability, especially in privacy-sensitive scenarios where the forget set may be permanently erased after an unlearning request. The paper does not address this constraint meaningfully. \n\n- The notion of residual knowledge is formalized based on comparing outputs of unlearned and re-trained models on perturbed forget samples. However, in practice, access to a fully re-trained model is typically unavailable after deployment. The authors mention a practical proxy but fail to rigorously evaluate how well this proxy reflects true residual knowledge in realistic scenarios. \n\n- The computational cost of RURK, especially regarding the inner loop required to generate vulnerable perturbations during fine-tuning, is not quantified. Given that adversarial training is already known to be computationally expensive, the feasibility of RURK for large models or datasets remains unclear. \n\n- The analysis of baseline methods in Table 1 is surface level. While raw performance numbers are reported, the authors miss an opportunity to investigate why specific baselines (e.g., Fisher, NTK, GA) fail in terms of residual knowledge, leaving the reader with limited insights into the underlying mechanisms. \n\n- Some claims about the generality of the findings (e.g., in line 13, that residual knowledge is \"prevalent across existing unlearning methods\") are overstated given the narrow experimental scope and lack of evidence beyond CIFAR-5 and CIFAR-10.\n\nQUESTIONS:\n- Can the authors provide a more detailed justification for the use of isoperimetric inequalities and high-dimensional geometric reasoning, particularly regarding how these assumptions map to practical deep learning models and real-world data distributions? \n\n- How do the authors envision applying RURK in deployment settings where the forget set is no longer accessible? Can they propose a realistic variant that does not require the forget set during mitigation? \n\n- Can the authors provide concrete runtime and resource overhead measurements for RURK, including comparisons to existing methods, especially for larger models or datasets? \n\n- Would the authors consider extending the experimental evaluation beyond CIFAR to larger-scale or non-vision tasks to validate both the existence and mitigation of residual knowledge in more realistic settings? \n\n- Is there a reliable, model-agnostic way to estimate residual knowledge in practice without requiring access to a re-trained model? The paper's current proxy approach lacks rigorous evaluation in this regard.\n\nLIMITATIONS:\nPartially. While the authors acknowledge theoretical and computational limitations, they understate critical practical concerns, including reliance on forget set access, scalability of RURK, and the narrow experimental scope.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis manuscript investigates a novel privacy risk in machine unlearning, termed residual knowledge, where slightly perturbed forget samples may still be recognized by unlearned models. The authors provide both theoretical analysis and empirical evidence to demonstrate that this phenomenon arises inevitablely in high-dimensional settings. To mitigate this issue, the authors propose RURK, a fine-tuning-based method that penalizes the model’s ability to re-recognize perturbed forget samples. The authors conduct experiments on small-scale datasets to demonstrate the effectiveness of the proposed RURK\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n- This manuscript is well-written with clear logic and easy to follow.\n- The authors identify an interesting phenomenon (residual knowledge) in machine unlearning: slight perturbations of forget samples may still be correctly recognized by the unlearned model. This reveals a novel privacy risk for machine unlearning.\n- Both empirical evidence and theoretical insights are provided to support the analysis of model disagreement under sample perturbations.\n- The proposed method RURK, which incorporates adversarial perturbations into the unlearning process, is simple but reasonable to solve the identified issue.\n\n**Weaknesses**\n- The experimental evaluation is limited to small-scale datasets (small CIFAR-5 and CIFAR10) and backbones (ResNet18).\n- In experimental tables (e.g., Table 1), it would be helpful to indicate explicitly whether an increasing or decreasing value represents better performance.\n- The proposed RURK method does not appear to have significant improvements over certain existing baselines such as SSD, even in terms of mitigating the residual knowledge threat (e.g., as shown in Figure 2).\n- This manuscript lacks a conclusion section.\n\nQUESTIONS:\n- Could the authors provide more results on larger-scale datasets\n- Will the threat of residual knowledge also arise in large language models? I would appreciate it if the authors could include some empirical results on unlearning in LLMs.\n- Is the adversarial attack used during unlearning (i.e., Eq. 6) the same as the one used to evaluate residual knowledge threat? If so, could the authors provide more results under different types of adversarial attacks?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-03"}